Sparse attention
AISparse attention: Attention that connects each token to only a subset of others following a fixed or learned pattern, reducing quadratic cost.
Sparse attention: Attention that connects each token to only a subset of others following a fixed or learned pattern, reducing quadratic cost.