Google

The training objective of instruction fine-tuning differs from that of standard language model pre-training. Instead of maximizing the probability of an entire sequence, instruction fine-tuning aims to maximize the conditional probability of generating the correct output (the remainder of the sequence) given a specific input prefix or instruction.

Comparison of Training Objectives: Instruction Fine-Tuning vs. Pre-training

A language model is first trained on a vast corpus of internet text to learn general language patterns. It is then further trained on a curated dataset of prompts and their desired responses to become a helpful assistant. Describe the primary difference in the mathematical objective being optimized during these two distinct training phases.

Differentiating Training Objectives in Language Models

During instruction fine-tuning, the model's training objective is to maximize the probability of the entire input-output sequence, treating the user's instruction and the desired response as a single, continuous piece of text.

A language model undergoes two distinct training stages. In the first stage, it is trained on a massive, unlabeled dataset of books and websites with the goal of learning to predict the next word in any given sentence. In the second stage, it is trained on a smaller, curated dataset of user prompts paired with ideal answers, with the goal of learning to generate helpful responses to the prompts. Which statement best analyzes the fundamental shift in the model's training objective between these t

Learn Before

Related