Using a Validation Curve to Judge the Value of More Data
Question: In a short analytical response, explain how a target error level and a plotted dev-loss trend can be used to judge whether collecting additional training examples is worthwhile.
Sample answer: Mark the desired error level on the graph, then extend the dev-loss trend beyond the observed points to estimate how much extra data might be needed to approach that level. This gives a rough planning estimate, not a promise of what will happen. In the example, the trend suggested that a larger training set, roughly about twice as large, could be enough to get near the goal.
Key points:
- Mark the goal level on the plot.
- Extend the dev-loss trend beyond the current data.
- Use the projection to estimate the possible gain from more examples.
- Treat the result as an approximation, not a certainty.
- Note that the example pointed to roughly doubling the training set.
Rubric: A strong response states how to place the target on the curve, describes extending the dev-loss trend to estimate the effect of added data, and explains that the conclusion is only an estimate rather than a guarantee.
0
1
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
What does sketching an extension of the dev-error curve help you estimate?
A trend line from past data can give an exact guarantee of the model’s future error after more training data is added.
Add the target _____ to a dev-error curve before estimating the value of more data.
Match each learning-curve element with its role when estimating the benefit of more labeled data.
Order the reasoning steps for judging whether more data is likely to reach a target performance level.
If the validation curve reaches the target after the data set doubles, what should be concluded?
Why Add the Target Line to a Learning Curve?
Which conclusion best matches the example learning curve?
A rough learning-curve estimate can support a tentative prediction about the value of adding more training data, but it does not guarantee the outcome.