How LLMs Actually Work: From a Single Neuron to the Frontier › Module

Query, Key, Value

One vector cannot be the question, the label and the answer at once. Split the roles, let every word attend to every other, and watch a pronoun resolve itself

Free preview · Part of How LLMs Actually Work: From a Single Neuron to the Frontier

Open module