Google

External memory systems enhance the attention mechanism by first retrieving relevant stored context representations that match a given query. These retrieved representations are then incorporated to augment the attention calculation for that specific query, providing it with more relevant information.

Using Retrieved Context to Improve Attention

One straightforward method for integrating retrieved $$k$$-NN memory is to concatenate it with the local memory. In this approach, the local memory ($$\mathrm{Mem}$$) and the $$k$$-NN memory ($$\mathrm{Mem}_{k\mathrm{nn}}$$) are combined to form a single, larger Key-Value cache, represented as $$[\mathrm{Mem}, \mathrm{Mem}_{k\mathrm{nn}}]$$. The model then performs a standard query-key-value attention operation on this unified cache for a given query $$\mathbf{q}_i$$.

Combined KV Cache for k-NN and Local Memory

A language model is equipped with an external memory system to enhance its responses. Arrange the following steps in the correct chronological order to show how the model uses this system to process a user's query.

Based on the scenario provided, propose a two-stage process to modify the chatbot's architecture that would help it locate and use the precise information needed from the documentation to answer these specific questions. Describe how your proposed process would work.

Improving a Customer Support Chatbot

A language model is designed to answer complex questions about a large, internal database of scientific papers. When it receives a user's question, it first performs a search to find the most relevant paragraphs from the papers. These selected paragraphs are then used to help the model formulate its final answer. Which statement best analyzes the primary role of these retrieved paragraphs in the model's internal processing?

Learn Before

Related