Activation checkpointing
AIActivation checkpointing: Saving memory during training by discarding some intermediate results and recomputing them later, trading extra compute for lower memory use.
Activation checkpointing: Saving memory during training by discarding some intermediate results and recomputing them later, trading extra compute for lower memory use.