Google

After computing the interpolated final probability distribution, $$\mathrm{Pr}(\cdot|\mathbf{h}_i)$$, a $$k$$-nearest neighbors ($$k$$-NN) language model selects the next token, $$y$$. This selection is achieved by finding the specific token that maximizes the final probability, $$\mathrm{Pr}(y|\mathbf{h}_i)$$, thereby choosing the most probable output based on the combined retrieval and model predictions.

Next Token Selection in k-NN Language Models

Based on the provided information, determine which word the model will select for two different values of the blending coefficient: a) λ = 0.2 and b) λ = 0.8. Justify your answer for each case by showing the final probability calculation for both words.

Analyzing Conflicting Model Predictions

A language model combines a base distribution and a retrieval-based distribution to predict the next token. The model ultimately selects the token 'cat' with a final combined probability of 0.55. If the base model assigned 'cat' a probability of 0.40, and the retrieval-based model assigned it a probability of 0.80, what was the interpolation coefficient (λ) used to balance the two distributions? Provide the final value.

Calculating the Interpolation Coefficient

A language model combines a base probability distribution, P_lm, with a retrieval-based distribution, P_knn, to predict the next token. The final probability is calculated by blending these two distributions using an interpolation coefficient, λ = 0.6. Given the distributions below for a small vocabulary, which token will the model select as its final output?

- P_lm: {'wordA': 0.5, 'wordB': 0.4, 'wordC': 0.1}
- P_knn: {'wordA': 0.2, 'wordB': 0.7, 'wordC': 0.1}

Learn Before

Related