Google

When a Large Language Model has difficulty providing a correct answer directly, a straightforward method to improve its accuracy is to include demonstrations of similar problems in the prompt. This approach enables the model to learn from the provided examples.

Using Demonstrations to Improve LLM Accuracy

An example of a direct prompt designed to instruct a Large Language Model to perform a specific mathematical task is the request: 'Please calculate the average of the numbers 1, 3, 5, and 7.'

Example of a Prompt for Calculating the Average of 1, 3, 5, and 7

Large Language Models can struggle to solve reasoning tasks correctly, even when provided with a demonstration in the form of a similar question-answer pair. This is because such examples do not explicitly convey the underlying reasoning process, which is often necessary for the model to arrive at the correct solution for a new problem.

Insufficiency of Simple Demonstrations for LLM Reasoning Tasks

Based on the scenario provided, describe how the manager could modify the prompt to achieve more consistent and correctly formatted classifications. Explain your reasoning and provide a specific example of what the new prompt structure would look like.

Improving LLM Classification Consistency

A user is trying to get a language model to create a username from a full name. The desired format is the first initial of the first name followed by the full last name, all in lowercase. The user's initial prompt is: `Create a username for the name: Ada Lovelace.` The model incorrectly responds with `AdaL`. Rewrite the prompt to include one clear demonstration that will guide the model to produce the correct output, `alovelace`.

Crafting an Effective Prompt with Demonstrations

A concrete example of a few-shot prompt for polarity classification involves providing a large language model with an instruction and several demonstrations before asking it to classify a new input. Such a prompt begins with a task description, like 'Assume that the polarity of a text is a label chosen from {positive, negative, neutral}. Identify the polarity of the input.' This is followed by demonstrations pairing inputs with their correct labels, such as 'Input: The traffic is terrible during rush hours, making it difficult to reach the airport on time. Polarity: Negative' and 'Input: The weather here is wonderful. Polarity: Positive'. Finally, a new input is provided for the model to classify, for instance, 'Input: I love the food here. It's amazing! Polarity:'.

Example of a Few-Shot Prompt for Polarity Classification

A user wants a language model to extract the main sentiment from a sentence and express it as a single emoji. The initial prompt, 'What is the sentiment of this sentence, as an emoji: The movie was a spectacular triumph!', results in the model responding with text like 'The sentiment is positive.' Which of the following revised prompts is structured to be the most effective at teaching the model the desired task?

Learn Before

Related