New: Intelligent Flashcards spaced-repetition decks for every topic, built to make things actually stick.
In a self-attention layer, how does the model build the new representation for a given token?
It averages the embeddings of the neighboring tokens inside a fixed window.
It copies the single token whose key best matches the current query.
It takes a weighted blend of the value vectors from every position, using softmax attention weights.