It is a good practice to keep checking the test set after every major training change so you can choose the best model.
0
1
Tags
Machine Learning
Deep Learning
Machine Learning Strategy
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Yearning @ DeepLearning.AI
Related
Keep the Test Set Out of Routine Model Decisions
What makes a model gradually adapt itself to the dev set during development?
A large gap with dev performance much better than test performance can indicate overfitting to the dev set.
If repeated evaluation has made the old development set misleading, get a _____ development set.
Match each dev-set overfitting concept to its description.
Put the dev-set overfitting process in the correct order.
After several rounds of tuning, your validation score is much better than your test score. What should you do next?
Repeatedly choosing models based on dev-set results can cause the system to gradually _____ to the dev set.
Match each development-set practice or signal with what it means in model selection.
Order the steps for investigating a validation-versus-test performance gap.
How Repeated Tuning Can Distort the Development Set
Evaluating a Model That Fits the Development Set Too Closely
Sign That the Development Set Has Been Overused