https://data.princeton.edu/eco572/smoothing2

University of Michigan - Ann Arbor

Regression splines model a nonlinear relationship by splitting the range of a predictor variable X into K intervals and fitting a separate polynomial on each interval. The pieces are estimated so that the final curve joins smoothly at the knot locations. For example, if X runs from 0 to 100 and knots are placed at 25, 50, and 75, one polynomial is fit on 0-25, another on 25-50, and so on, with the pieces connected at each knot.

How Regression Splines Work

While it may seem intuitive to place more knots where the data has more variance, in practice it is best to uniformly distribute the knots in the data as to not overfit in certain areas. Commonly the number and placement of knots is determined by selecting your desired degrees of freedom and letting data software distribute them automatically. 

Regression Splines: Deciding how many knots to use and where to place them

Regression splines combine the benefits of polynomial curve fitting and step functions to provide a more flexible estimated function. By dividing the range of the independent variable $$X$$ into enough regions, we can theoretically fit any function. However, this flexibility comes at the expense of potential overfitting and reduced interpretability of the coefficients. Therefore, a trade-off must be considered when deciding on the number of regions.

Regression Splines vs. Polynomial Curve Fitting and Step Functions

Regression splines often give superior results compared to polynomial regression. Unlike polynomials, which must use a high degree (the exponent in the highest monomial term, e.g., $$X^{15}$$) to produce flexible fits, splines introduce flexibility by increasing the number of knots but keeping the degree fixed. As shown in the accompanying figure, polynomials can exhibit unexpected and wild behavior, especially near the tails. This is not preferred for larger range predictions.

Learn Before

Related