Cosine annealing

Cosine annealing: A learning-rate schedule that lowers the rate following a cosine curve, decaying smoothly from its peak towards zero across training.

Browse the full glossary →