In plain English
Attention lets a model score every other token and focus on the ones that matter for each word. Many heads watch for different patterns, and many layers build meaning. Long inputs cost much more, because each token may look at all the others.
- Attention
- Weighing which tokens matter for each word.
- Head
- One of several attention views in a layer.
- Layer
- One step in the stack that builds meaning.
Picture it
Attention for one word
1
Take the word it
The model needs its meaning
2
Score every earlier word
Trophy, suitcase, big, fit
3