Google

To prepare the input for a span prediction model, the query and the context are combined into a single sequence. This is often done by concatenating the query tokens (e.g., `xm`), a special separator token `[SEP]`, the context tokens (e.g., `y1 y2 yn`), and another final `[SEP]` token. This packed sequence, `xm [SEP] y1 y2 yn [SEP]`, is then fed into the model.

Input Sequence Formatting for Span Prediction

A language model designed for span-based question answering needs the query and the context document to be combined into a single input string. Arrange the following components into the correct structural order.

When preparing input for a model that identifies an answer span within a context document based on a query, the query and context are typically combined into a single sequence using a special separator token (e.g., `[SEP]`). Explain the primary reason why this specific structure (`query [SEP] context [SEP]`) is used instead of feeding the query and context to the model as two separate, independent inputs.

Rationale for Combined Input Sequence

A language model is designed to find an answer by identifying a specific segment of text (a 'span') within a larger context, based on a given query. If the query is 'What is the capital of France?' and the context is 'Paris is the capital and most populous city of France.', which of the following shows the correctly formatted single input sequence that should be fed into the model?

Learn Before

Related