Google

The process of generating text with a Large Language Model is formally framed as a conditional probability task. Given an input prompt, denoted as $x$, the model's objective is to produce an output text, $y$, that maximizes the conditional probability $Pr(y|x)$. The prompt, therefore, functions as the condition upon which the prediction is based.

Prompting as a Conditional Probability Task

A user provides a language model with the one-word input: `bank`. The model responds with: `A bank is a financial institution that accepts deposits from the public...`

Using the framework where text generation aims to find an output `y` that maximizes the probability `Pr(y|x)`, explain why the model likely produced this specific definition instead of a response about a river bank.

Explaining Model Behavior with Conditional Probability

A user is trying to learn about the role of eagles in ancient Rome. Analyze the two interactions below. Using the formal framework of text generation as a conditional probability task ($Pr(y|x)$), explain why the second interaction was more successful than the first.

Analyzing Prompt Specificity through Conditional Probability

A user wants a language model to generate a specific two-sentence summary about the sun. Consider two possible inputs:

Input A: `Summarize the sun.`
Input B: `In two sentences, describe the sun's composition and its role as the center of our solar system.`

From the perspective of text generation as a task of maximizing the probability of an output given an input, which statement best analyzes the relationship between these inputs and the desired output?

Learn Before

Related