Your Residual Plot: What a $5 Error Says About Fit

Table of Contents

A residual plot shows a regression model’s errors, exposing patterns that its predictions miss. Suppose a lemonade stand’s predictions look close overall, yet repeatedly fall short on its busiest days: plotting those errors can reveal where the model fails. Use the stand’s examples to calculate residuals, recognize common patterns, and choose the next diagnostic or a plausible model change.

Calculate Errors in Outcome Units

The stand records $50 in revenue when its model predicts $45. Its residual is +$5:

Residual = observed value − predicted value

The positive sign means underprediction; a negative sign means overprediction. Keeping errors in dollars makes their size interpretable: for this observation, revenue was $5 higher than predicted. That difference alone does not show how much extra stock was needed; inventory planning also requires information about prices and quantities sold.

Ordinary least squares (OLS) chooses coefficients that minimize the sum of squared residuals. That fitting rule does not tell the owner which days need more stock.

Plot the Errors Against Predictions

Constructing a residual plot begins with two axes: fitted values horizontally and residuals vertically. Add a zero reference line; each observation supplies one point.

These illustrative values produce the left-hand panel:

Fitted Value	Observed Value	Residual
10	11	+1
20	18	−2
30	32	+2
40	39	−1
50	49	−1
60	62	+2
70	68	−2
80	81	+1

The other panels use different residuals at the same fitted values, advancing by 10.

Scatter-like           Curvature              Funnel-shaped
 3 |                    3 |●             ●     3 |            ●
 2 |    ●     ●         2 |                    2 |        ●
 1 |●             ●     1 |                    1 |    ●
 0 |---------------     0 |  ●         ●       0 |●-●------------
-1 |      ● ●          -1 |                   -1 |      ●
-2 |  ●         ●      -2 |    ●     ●         -2 |          ●
-3 |                   -3 |      ● ●           -3 |              ●
   +---------------       +---------------       +---------------
    10           80        10           80        10           80
      Fitted value           Fitted value           Fitted value

Start with the visible pattern, then choose a follow-up:

Visible pattern	Possible meaning	Next check
Scatter around zero without an obvious curve	No obvious remaining mean pattern in this view	Check spread and plot residuals against time or observation order
Curve	A relationship the model has missed	Compare plausible nonlinear terms, such as a quadratic term or spline
Funnel	Changing error spread	Check a scale–location plot; address mean misfit before modeling variance

Match the Shape to a Model Change

Read a residual plot pattern by separating its center from its spread. Consider a stand whose sales accelerate as temperature rises: a straight line could overpredict in the middle and underpredict at both ends. A quadratic term or spline could capture that curvature.

For a funnel, suppose quiet days have similar sales, while busy days depend heavily on the size of arriving groups. Dollar errors could widen as predicted revenue rises. This changing variance is called heteroscedasticity; a response transformation or variance model may help.

Use this reference to choose the next diagnostic:

Diagnostic option	Residual pattern to check	Best for	Diagnostic strength	Limitation / follow-up
Residuals vs fitted + LOESS	Curved smooth instead of flat near zero	Mean function	Shows systematic prediction errors	Compare plausible nonlinear terms
Scale–location plot	√\|standardized residual\| increases with fitted values	Nonconstant variance	Clarifies changing spread	Address mean misfit first
Residuals vs individual predictor	Curve or step	Missing predictor structure	Connects errors to an input	Assess correlated predictors jointly
Residuals vs time or observation order	Runs, cycles, adjacent similarity	Dependence and drift	Preserves order	Inspect autocorrelation for lag structure
Residuals by group	Different centers or spreads	Group offsets and variance	Exposes subgroup misfit	Sparse groups are unstable; consider group or multilevel terms
Normal Q–Q plot of residuals	Tails leave reference line	Normal-error inference	Separates tail and central fit	Does not assess nonlinearity or independence; departures alone do not invalidate OLS coefficients
Residuals vs leverage + Cook’s distance	Large residual with high leverage	Influential observations	Separates outcome outliers from unusual predictor values	Investigate and compare fits; do not automatically delete

Heteroscedasticity-robust standard errors address inference under suitable conditions, not a wrong mean or prediction intervals. In a linear residual plot, bands can also come from rounded or discrete outcomes: measurement rules matter.

Keep Observation Order in View

Measurement order matters too. A fitted-value display hides chronology; plotting residuals against observation number or time reveals consecutive positive errors followed by negative ones. An autocorrelation plot examines lagged relationships, while a repeating weekly cycle points toward a time-related explanation.

Suppose the stand reaches 100% of capacity on weekends. A temperature-only model misses that constraint. A weekend indicator could capture a level shift, but a capacity-aware model represents why sales stop rising: the stand cannot serve more customers.

Put Large Errors in Context

That sales limit also changes how an unusual day should be interpreted. Suppose festival revenue reaches $160, compared with the usual $20–$60. If the prediction was similarly high, the residual could still be small.

Leverage concerns predictors, not revenue alone. If the festival also brings a customer count far outside the training range, that observation has an unusual predictor position and may pull the fitted relationship toward itself. Cook’s distance helps assess that influence.

Standardized residuals adjust raw errors for estimated variability and leverage. Values beyond about ±2 deserve attention; beyond ±3 are stronger screening flags. Investigate entry mistakes and festival conditions before deciding whether a point belongs in the analysis.

Check More Than One Assumption

Keeping the festival observation creates another question: does a more flexible model predict better? Suppose a high-degree polynomial bends through every training observation, leaving a spotless residuals plot, but predicts negative revenue for a held-out hot day. It has improved the training picture while failing the actual task.

For the stand’s forecasts, compare errors on a later-period holdout or use rolling-origin cross-validation: fit on earlier days, predict subsequent days, and move the cutoff forward. Do not randomly mix earlier and later days between training and validation. Separately, a normal Q–Q plot assesses distributional shape; tail departures can matter for small-sample inference. Residual plots against fitted values cannot settle normality or independence, and normal errors are not required to calculate OLS coefficients.

Key Takeaways

Calculate errors in the outcome’s units to judge practical costs.
Match the diagnostic to the suspected mechanism.
Compare influential observations with their collection context.
Judge revisions on held-out performance as well as training diagnostics.

Frequently Asked Questions

What Is a Residual Plot?

A residual plot displays observed-minus-predicted errors against fitted values, a predictor, or observation order. Its zero line marks where observed and predicted values are equal.

What Does a Residual Plot Tell You?

A residual plot reveals patterns, changing spread, and unusually large prediction errors. Curves can suggest a missed relationship, while widening spread can suggest nonconstant error variance.

What’s the Difference Between a Scatter Plot and a Residual Plot?

A scatter plot shows the relationship between two variables; a residual plot is a specific type that places model residuals on the vertical axis. It shows observed-minus-predicted differences rather than the original outcome values.

What Does It Mean If the Residual Plot Has No Pattern?

No visible pattern means the plot reveals no obvious remaining structure along the chosen horizontal axis. It does not prove the model is correct or replace other diagnostics and held-out validation.

Should an Observation With a Large Residual Always Be Removed?

No; a large residual alone does not justify removing an observation. Investigate recording errors and the observation’s context before deciding whether to correct it, exclude it, or retain it.

Let the Errors Guide Your Revision

Keep a revision log with four entries: the residual pattern, its suspected cause, the model change tested, and the validation result. Record the dates used for training and validation so each comparison is reproducible. If a change removes curvature but leaves a funnel, record that remaining problem rather than marking the model as fixed.