Google

A learned scoring rule can cause inference errors if it gives the correct answer a lower score than the model’s proposed answer. In that situation, the right fix is usually to improve the score-estimating model, not the search procedure.

Scoring Rule Problem

The cross-entropy loss function works very well for models that predict binary classes (aka the output is between 0 and 1). It is defined as -[y*log(y-hat) +(1-y)*log(1-(y-hat))]. If y=0 the left side of the function is dropped and the right side, -log(1-(y-hat)), is used. Otherwise if y=1 the right side of the function is dropped and it uses -log(y-hat). In both instances this loss function encourages probabilities that are close to the true probability. 

Cross-entropy loss

An engineer is training a model to predict housing prices. After running the training process for several hours, they plot the value of the model's error measurement over time. They observe that the error value remains very high and does not decrease, staying almost flat throughout the entire process. Based on this observation, analyze the effectiveness of the training process and explain what this trend indicates about the model's ability to achieve its primary goal.

Diagnosing Model Training Issues

A machine learning model is designed to predict the price of a product. For a small sample of three products, the model predicts prices of [$55, $90, $125], while the actual prices are [$50, $100, $120]. The model's performance is measured by an objective function defined as the average of the squared differences between the predicted and actual values. First, calculate the value of this objective function for the given sample. Second, explain what a lower value of this function would signify about the model's future predictions.

Calculating and Interpreting a Model's Objective Function

When a primary objective function—such as the error rate in classification—is difficult to optimize directly due to non-differentiability or other mathematical complications, machine learning models instead optimize a surrogate objective. This proxy function is chosen because it is easier to compute gradients for while still aligning with the ultimate goal.

Surrogate Objective

A loss function is a specific type of objective function where the convention is that lower values indicate better model performance. Optimization algorithms actively seek to minimize the loss function to improve the model; any objective where higher is better can be converted into a loss function by simply flipping its sign.

Loss Function

A fundamental requirement for training modern machine learning and deep learning models is the use of differentiable objectives. Because the optimization process typically relies on gradient-based methods, such as minibatch stochastic gradient descent, the objective function (or loss function) must be mathematically differentiable with respect to the model's parameters. This differentiability allows the optimization algorithm to compute gradients, which provide the direction and magnitude of the parameter updates needed to minimize the error and improve the model's predictive performance.

Differentiable Objectives

First-order optimization algorithms rely solely on the value and gradient of the objective function. In contrast, second-order optimization algorithms also utilize information about the function's curvature, often represented by the Hessian matrix. By accounting for curvature, these methods can automatically adjust the optimization step, providing a way to circumvent the difficulties of manually tuning a learning rate.

Second-Order Optimization Algorithm

In the context of optimization, the curvature of an objective function refers to the rate at which its gradient changes, conceptually captured by its second-order derivatives. Geometrically, curvature indicates how rapidly the surface of the objective function bends. Understanding this property provides useful intuition for adjusting optimization step sizes: in regions of high curvature where the gradient changes quickly, smaller step sizes help avoid overshooting the optimal solution or diverging; in regions of low curvature, larger step sizes can safely accelerate progress. While computing curvature directly is often too computationally expensive for deep learning, it forms the theoretical foundation for designing advanced adaptive optimization algorithms that automatically adjust their learning rates.

Objective Function Curvature

A convex quadratic objective function is defined by the general mathematical form $$h(\mathbf{x}) = \frac{1}{2} \mathbf{x}^	op \mathbf{Q} \mathbf{x} + \mathbf{x}^	op \mathbf{c} + b$$, where the matrix $$\mathbf{Q}$$ is positive definite ($$\mathbf{Q} \succ 0$$). Because $$\mathbf{Q}$$ possesses strictly positive eigenvalues, this function has a unique global minimizer located at $$\mathbf{x}^* = -\mathbf{Q}^{-1} \mathbf{c}$$. The function can be rewritten by centering it around this minimizer, yielding $$h(\mathbf{x}) = \frac{1}{2} (\mathbf{x} - \mathbf{Q}^{-1} \mathbf{c})^	op \mathbf{Q} (\mathbf{x} - \mathbf{Q}^{-1} \mathbf{c}) + b - \frac{1}{2} \mathbf{c}^	op \mathbf{Q}^{-1} \mathbf{c}$$. Furthermore, its gradient is given by $$\partial_{\mathbf{x}} h(\mathbf{x}) = \mathbf{Q} (\mathbf{x} - \mathbf{Q}^{-1} \mathbf{c})$$, which geometrically represents the distance from the point $$\mathbf{x}$$ to the minimizer scaled by the curvature matrix $$\mathbf{Q}$$.

Convex Quadratic Objective Function

Recognizing a Scoring-Function Issue

If a candidate answer gets a lower score than an incorrect answer, the search procedure is usually the part that should be fixed first.

A scoring rule can mislead learning if it gives the correct output a _____ score than the system's prediction.

When to Treat a Scoring Problem as an Objective-Function Issue

When the Scoring Rule Is the Source of the Error

Question:
Explain why a scoring-rule failure happens and what should be changed to fix it, using a simple verification check that compares the score for the ideal output with the score for an inferior system output.

Sample answer:
A scoring-rule failure happens when the evaluation function gives the ideal output the same score as, or a lower score than, a worse output. In that case, the score does not correctly distinguish good results from bad ones, so the system is guided by the wrong signal. The right fix is to improve the learned scoring or prediction rule, not to focus on making the search procedure more sophisticated.

Key points:
- Failure occurs when the ideal output is not scored above the alternative.
- This shows the score does not separate better outputs from worse ones.
- Improve the scoring or learning component, not the search component.

Rubric:
Evaluates understanding of when a scoring-rule failure occurs and which part of the system should be improved.

Diagnosing a Scoring-Rule Failure

Case context:
A hospital is developing an automatic dictation system. For one audio clip, the reference transcript S* is "The nurse checked the patient's pulse." The system's output Sout is "The nurse checked the patient's purse." A verification run gives ScoreA(S*) = 0.67 and ScoreA(Sout) = 0.74.

Question:
What does this result suggest about the main cause of the error, and which part of the system should you improve first?

Sample answer:
The problem is in the scoring objective, because the reference transcript receives a lower score than the incorrect output. The first thing to improve is the component that learns the score for candidate transcripts, rather than the search procedure that selects among them.

Key points:
- Treat this as a scoring/objective issue.
- The reference score is below the wrong-output score.
- Focus on improving the score model.

Rubric:
The learner must identify the scoring objective problem and recommend improving the learning component that produces the scores.

Automatic Dictation Score Check

Question:
A scheduling system scores candidate plans, but an optimization check shows that the plan selected by the search procedure does not score better than the best plan found by exhaustive checking. What part of the system should be improved?

Sample answer:
Improve the component that learns or approximates the scoring function, because the score model is failing to rank the truly best plan above the selected plan.

Key points:
- The scoring model or its learning procedure
- The approximation of the score used to rank candidate plans

Rubric:
Correctly identifies the scoring model, the learning procedure, or the approximation used to estimate the score.

Improving the Score Estimator

What an Objective Function Must Do

A Metric Can Still Be Wrong

A machine learning model is being trained for a prediction task. A key metric, the objective function, is tracked over time. The value of this function represents the magnitude of the model's error. A graph of this process shows the objective function's value consistently decreasing as the number of training iterations increases. What is the most accurate interpretation of this trend?

Learn Before

Related