Google

In autoregressive text generation, after computing the conditional probability distribution for the next token, $\overline{\text{Pr}}(y_i|\mathbf{x}, \mathbf{y}_{<i})$, the next step is to draw a sample from it. This sampling process, which selects a specific token $\bar{y}_i$, is formally expressed as drawing from the distribution: $$\bar{y}_i \sim \overline{\text{Pr}}(y_i|\mathbf{x}, \mathbf{y}_{<i})$$

Token Sampling from a Conditional Probability Distribution

In autoregressive models, the selection of the next token, $$\bar{y}_i$$, is formally represented as drawing a sample from the model's conditional probability distribution. This is expressed by the formula: $$\bar{y}_i \sim \overline{\text{Pr}}(y_i|\mathbf{x}, \mathbf{y}_{<i})$$ This notation signifies that the token $$\bar{y}_i$$ is sampled from the probability distribution over all possible tokens $$y_i$$, conditioned on the input context $$\mathbf{x}$$ and the sequence of previously generated tokens $$\mathbf{y}_{<i}$$.

Formula for Token Sampling in Autoregressive Models

Based on the provided probability distribution, describe the essential next step the model must take to continue generating the sentence. Explain what this step accomplishes, identify the token most likely to be chosen, and clarify why its selection is not guaranteed.

Applying Token Sampling in Text Generation

In autoregressive text generation, after the model computes the conditional probability distribution for the next token, the sampling process always selects the token with the highest probability score.

An autoregressive language model has processed the input sequence 'The cat sat on the' and has calculated the following conditional probability distribution for the next token: P('mat'|context) = 0.6, P('rug'|context) = 0.3, P('floor'|context) = 0.08, P('sky'|context) = 0.02. If the model then *samples* a token from this distribution, which of the following statements is most accurate?

Learn Before

Related