Gradient accumulation

Gradient accumulation: A method that sums gradients over several small batches before updating weights, simulating a large batch when memory is limited.

Browse the full glossary →