Data parallelism
AIData parallelism: A training method where each device holds a full copy of the model and processes a different slice of the data.
Data parallelism: A training method where each device holds a full copy of the model and processes a different slice of the data.