Attention is just a soft dictionary lookup

Every explanation of attention starts with a wall of matrices. Let’s not. Let’s start with a dictionary.

Hard lookup, soft lookup#

A Python dictionary is a hard lookup: you ask for a key, you get exactly one value, and if the key doesn’t exist you get an error.

prices = {"apple": 1.20, "banana": 0.50, "cherry": 3.00}
prices["apple"]   # 1.20, exact match or nothing

Attention asks: what if the lookup were soft? Instead of demanding an exact key match, you ask “how similar is my query to each key?” and get back a weighted blend of all the values, where more similar keys contribute more.

def soft_lookup(query, keys, values):
    scores = [similarity(query, k) for k in keys]
    weights = softmax(scores)          # similarities → proportions
    return sum(w * v for w, v in zip(weights, values))

That’s it. That’s attention. Everything else is bookkeeping.

So what are Q, K, and V really?#

In a transformer, every token in the sentence plays all three roles at once:

• Query: “here’s what I’m looking for.”
• Key: “here’s what I can be found by.”
• Value: “here’s what I’ll hand over if you pick me.”

When the model processes the word it in “The cat sat because it was tired,” the query for it is effectively asking “who am I referring to?” The key for cat happens to match that question well, so cat’s value flows heavily into the blend, and the representation of it becomes cat-flavored.

Attention lets every word ask a question of every other word, and receive a blended answer weighted by relevance.

Why softmax and not just raw scores?#

Two reasons, both practical:

1. Proportions, not magnitudes. Softmax turns arbitrary similarity scores into weights that sum to 1, so the output stays at a sensible scale no matter how many tokens there are.
2. Sharpness control. Dividing scores by $\sqrt{d_k}$ before the softmax keeps the distribution from collapsing onto a single token early in training, when everything is noise anyway.

Put together, the famous formula:

$$\mathrm{Attention}(Q, K, V) = \mathrm{softmax}\!\left(\frac{QK^\top}{\sqrt{d_k}}\right)V$$

What I still want to understand#

Multi-head attention supposedly lets the model ask different kinds of questions in parallel: one head tracking syntax, another coreference, and so on. I’ve read the claim many times; I haven’t yet seen a demonstration that convinced me the heads specialize that cleanly. That’s a future note.

---

The one-sentence version: attention replaces “fetch the exact match” with “blend everything, weighted by relevance”. It turns out that soft blending is enough to carry meaning across a sentence.