Google

In the application phase, a pre-trained encoder is adapted for a specific downstream task. The process begins by converting an input sequence of tokens, \{x_0, ..., x_m\}, into their corresponding embeddings, \{e_0, ..., e_m\}. This embedding sequence is then processed by the pre-trained encoder to produce a sequence of rich vector representations. These representations serve as input features for a separate, task-specific prediction network, which in turn generates the final output required for the application.

Applying a Pre-trained Encoder to Downstream Tasks

You are using a pre-trained model component to build a system that classifies the topic of a given sentence. Arrange the following steps in the correct chronological order to show how an input sentence is processed to generate a final topic classification.

Based on the provided scenario, what additional component must the data scientist design and train to complete this system? Describe the specific function of this new component.

System Design for Named Entity Recognition

A developer is tasked with creating a sentiment analysis model to classify movie reviews as 'positive' or 'negative'. They have access to a powerful, pre-existing model component that excels at converting any given text sequence into a sequence of rich numerical vector representations. The developer's goal is to use this component as a fixed feature extractor without altering its internal parameters. Which of the following describes the most appropriate system architecture for this task?

Learn Before

Related