Google

Few-shot Chain-of-Thought (CoT) prompting is an extension of the one-shot approach that involves including multiple demonstrations, each with its detailed reasoning steps, within the prompt. This provides the model with a richer set of examples to learn from, further improving its ability to generate a reasoned response.

Few-Shot Chain-of-Thought (CoT) Prompting

An analyst is using a language model to perform a multi-step calculation: finding the percentage change in quarterly sales. They have drafted two prompts. 

**Prompt A:**
"Q: Last quarter's sales were $80 and this quarter's sales are $100. What is the percentage change?
A: First, find the difference: $100 - $80 = $20. Then, divide the difference by the original amount: $20 / $80 = 0.25. Finally, multiply by 100 to get the percentage: 0.25 * 100 = 25%. The percentage change is 25%.

Q: Last quarter's sales were $500 and this quarter's sales are $450. What is the percentage change?"

**Prompt B:**
"Q: Last quarter's sales were $80 and this quarter's sales are $100. What is the percentage change?
A: First, find the difference: $100 - $80 = $20. Then, divide the difference by the original amount: $20 / $80 = 0.25. Finally, multiply by 100 to get the percentage: 0.25 * 100 = 25%. The percentage change is 25%.

Q: Last quarter's sales were $200 and this quarter's sales are $250. What is the percentage change?
A: First, find the difference: $250 - $200 = $50. Then, divide the difference by the original amount: $50 / $200 = 0.25. Finally, multiply by 100 to get the percentage: 0.25 * 100 = 25%. The percentage change is 25%.

Q: Last quarter's sales were $500 and this quarter's sales are $450. What is the percentage change?"

Analyze the two prompts. Explain why Prompt B is likely to produce more reliable and accurate results for the final question compared to Prompt A.

Analyzing Prompt Effectiveness for Multi-Step Calculations

A significant practical challenge in using few-shot Chain-of-Thought (CoT) prompting is the difficulty of creating the necessary demonstrations. Sourcing or manually crafting detailed, high-quality, multi-step reasoning examples can be a labor-intensive and complex task, whether attempted automatically or by hand.

Difficulty of Creating Few-Shot CoT Demonstrations

Based on the case study, what specific prompting technique should the data scientist use to improve the model's reliability? Explain why this technique would be more effective than the current approach.

Improving Model Reasoning for a New Task

To improve a Large Language Model's performance on algebraic calculations, a few-shot Chain-of-Thought prompt can incorporate detailed problem-solving steps from a similar problem. For instance, a prompt can demonstrate the step-by-step calculation for the mean square of $${}1$$, $${}3$$, $${}5$$, and $${}7$$, which involves squaring each number ($${}1^2 = 1$$, $${}3^2=9$$, $${}5^2=25$$, $${}7^2=49$$), summing the squares ($${}1+9+25+49=84$$), and dividing by the count ($${}84 / 4 = 21$$). By learning from this reasoning process, the model can successfully determine the average of $${}2$$, $${}4$$, and $${}9$$ by computing the sum ($${}2 + 4 + 9 = 15$$), dividing by the count ($${}15 / 3 = 5$$), and arriving at the correct final answer of $${}5$$.

Example of a Few-Shot CoT Prompt with Mean Square Demonstration

A developer is using a large language model to solve complex logic puzzles that require several steps of reasoning. The model consistently provides incorrect final answers without explaining its process. To improve the model's performance and elicit a step-by-step thought process, which of the following prompt structures would be most effective?

Learn Before

Related