Learn Before
What the 2023 Process-Supervision Math Study Established
OpenAI’s 2023-05-31 research report compared learned solution scorers, called reward models, on MATH, a collection of mathematics problems. The process-supervised scorer was trained with feedback on individual written reasoning steps; the outcome-supervised scorer with feedback on final results. For each test problem, each trained scorer ranked generated candidate solutions; evaluation checked whether its highest-ranked solution gave a correct final answer. The process-supervised scorer outperformed the compared outcome-supervised scorer in this setup. Whether this worked beyond mathematics was explicitly unknown. Scoring written steps does not directly measure a reader’s comprehension or the model’s internal computation.
0
1
Tags
Scientific Oversight of AI-Assisted Research
1Cademy