Attention head
AIAttention head: One parallel attention computation within a layer, with its own query, key and value projections, capturing a particular pattern of relationships.
Attention head: One parallel attention computation within a layer, with its own query, key and value projections, capturing a particular pattern of relationships.