Google

The expression $$s(\bar{\mathbf{y}}_1 \dots \bar{\mathbf{y}}_{k-1})$$ represents a function, denoted by $$s$$, that takes a sequence of averaged vectors as its input. The sequence consists of terms from the first vector, $$\bar{\mathbf{y}}_1$$, up to the $$(k-1)$$-th vector, $$\bar{\mathbf{y}}_{k-1}$$.

Function of a Sequence of Averaged Vectors Notation ($$s(\bar{\mathbf{y}}_1 \dots \bar{\mathbf{y}}_{k-1})$$)

Given the state of the model described in the case study, what is the specific sequence that serves as the input to the function `s` at the moment it is tasked with predicting the fourth word?

Input for Next-Token Prediction

Consider a process where the outcome at step `k` is determined by a function `s(ȳ₁...ȳₖ₋₁)`. This function takes a sequence of averaged vectors from step 1 to `k-1` as input. Based on this definition, the outcome at step 10 is completely independent of the vector `ȳ₂`.

Consider the expression $s(\bar{\mathbf{y}}_1...\bar{\mathbf{y}}_{k-1})$, where 's' is a function that operates on a sequence of averaged vectors. What does this notation imply about the information used to compute an outcome for a given step 'k'?

Learn Before

Related