Google

In the context of Large Language Model (LLM) inference, a hypothesis is formally defined as a tuple consisting of an input sequence $$\mathbf{x}$$ and an output sequence $$\mathbf{y}$$. Because the input sequence $$\mathbf{x}$$ remains fixed during the inference process, a hypothesis can be simplified to represent only the generated output sequence.

Hypothesis in LLM Inference

The search space in Large Language Model (LLM) inference, denoted by $$\mathcal{Y}$$, is the complete set of all possible hypotheses that the model is capable of generating. Because a hypothesis in this context is simplified to mean an output sequence, the search space $$\mathcal{Y}$$ effectively consists of every possible output sequence.

Search Space in LLM Inference

During the search process for generating a response from a large language model for a single, fixed input, a 'hypothesis' is often defined simply as a potential output sequence, rather than a pair containing both the input and the output sequence. Justify this simplification, explaining why it is a valid and practical approach in this specific context.

Justification for Hypothesis Simplification

During language model inference, a 'hypothesis' refers exclusively to a potential output sequence, and the input sequence is not considered part of its formal definition.

An LLM is tasked with completing the input sequence `x = 'The best way to learn a new skill is'`. During the inference process, a search algorithm considers several potential continuations, such as `y1 = 'to practice consistently'`, `y2 = 'by reading a book'`, and `y3 = 'through immersive experience'`. How are these potential continuations best described within the context of the search problem?

Learn Before

Related