Grouped-query attention (GQA)
AIGrouped-query attention (GQA): A variant where groups of query heads share key and value projections, balancing the quality of multi-head attention with lower memory use.
Grouped-query attention (GQA): A variant where groups of query heads share key and value projections, balancing the quality of multi-head attention with lower memory use.