Interactive exploration of multi-head attention mechanisms
The attention mechanism allows the model to focus on different parts of the input sequence when processing each token. Brighter colors indicate stronger attention weights. Hover over the attention matrix to see exact values and relationships.