Google

In speculative sampling, a generated token $\hat{y}_{i+t}$ is considered for rejection if its probability under the draft model, $q(\hat{y}_{i+t})$, is greater than its probability under the target model, $p(\hat{y}_{i+t})$. When this condition is met, the token is not rejected outright, but rather with a specific probability calculated as $1 - \frac{p(\hat{y}_{i+t})}{q(\hat{y}_{i+t})}$.

Rejection Criterion in Speculative Sampling

In a speculative sampling process, a draft model proposes four potential next tokens. A more powerful target model then evaluates the probability of each token. Given the probabilities below, which token has the highest probability of being rejected?

An engineer is analyzing a text generation system that uses a small 'draft' model and a large 'target' model. The system's efficiency relies on some tokens being accepted immediately while others undergo a probabilistic check. Given the data below for four consecutively proposed tokens, identify which token is guaranteed to be accepted without a probabilistic rejection check and explain the underlying principle for its acceptance.

Analyzing Speculative Sampling Acceptance

In a text generation process, a small 'draft' model proposes the next token, 'evolution'. The probability assigned to this token by the draft model is 0.8. A larger, more accurate 'target' model then evaluates the same token and assigns it a probability of 0.6. Based on this information, what is the probability that the token 'evolution' will be rejected?

Learn Before

Related