Google

When evaluating grammaticality, a prompt can be organized by stating strict constraints on the answer format and meanings before presenting the target sentence. For instance, the prompt might begin with the rules: 'Yes or No? Yes = the following sentence is grammatically correct. No = it contains grammatical errors.' Right after these explicit instructions, the sentence to be judged is provided: 'LLMs are powerful models but are expensive to build.', followed by the desired response 'Yes'.

Example of a Constraint-First Prompt for Grammaticality Judgment

A data scientist is using a large language model to check sentences for grammatical errors but is getting inconsistent and overly verbose responses. Their current prompt is provided below. Rewrite the prompt to improve its clarity and guide the model more effectively by first stating the core task as a direct question, then specifying the required answer format, and finally providing the sentence to be analyzed.

Refining a Prompt for Grammaticality Judgment

You are designing a prompt to ask a language model to judge the grammatical correctness of a sentence. Arrange the following components into the correct 'question-first' structure.

A specific method for asking a language model to evaluate a sentence's grammar involves a three-part structure: 1) The direct question is asked. 2) Instructions for the desired output format are given. 3) The sentence to be judged is provided. Which of the following prompts correctly applies this specific three-part structure?

Learn Before

Related