Sliding-window attention

Sliding-window attention: Attention in which each token attends only to a fixed window of nearby tokens, cutting cost for very long sequences.

Browse the full glossary →