Learn Before
Start of Sequence (SOS) Token
The (Start of Sequence) token, or , is a special symbol used to mark the beginning of an input sequence in a Large Language Model. It is often denoted as the first token, , in the sequence.
0
1
Tags
Ch.5 Inference - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Ch.2 Generative Models - Foundations of Large Language Models
Related
Formal Definition of LLM Inference
A user provides the input 'Summarize this article', which a language model processes into three distinct tokens ('Summarize', 'this', 'article'). Based on the formal structure where an input sequence is represented by its tokens plus a special start symbol, what is the total number of tokens in the complete sequence given to the model?
A language model receives an input prompt that is tokenized into 10 tokens. According to the formal representation of an input sequence, , which of the following correctly describes the structure of the complete sequence processed by the model?
A language model is given the complete input token sequence: . By analyzing the components of this sequence, identify which token's primary role is to signal the beginning of the input context for the model.
Learn After
A specific language model requires that any input sequence must begin with a special symbol, denoted as
[START], to signal the beginning of the text. Given the sentence 'The model predicts.', which of the following token sequences is correctly formatted for this model?Debugging Model Input
Evaluating Model Design Choices
Learn After
A specific language model requires that any input sequence must begin with a special symbol, denoted as
[START], to signal the beginning of the text. Given the sentence 'The model predicts.', which of the following token sequences is correctly formatted for this model?Debugging Model Input
Evaluating Model Design Choices
Related
Formal Definition of LLM Inference
A user provides the input 'Summarize this article', which a language model processes into three distinct tokens ('Summarize', 'this', 'article'). Based on the formal structure where an input sequence is represented by its tokens plus a special start symbol, what is the total number of tokens in the complete sequence given to the model?
A language model receives an input prompt that is tokenized into 10 tokens. According to the formal representation of an input sequence, , which of the following correctly describes the structure of the complete sequence processed by the model?
A language model is given the complete input token sequence: . By analyzing the components of this sequence, identify which token's primary role is to signal the beginning of the input context for the model.