MaizePrepSTATS 250
Exam 3 · Lec 18

Multiple regression and model selection

Interpret adjusted slopes, compare model fit, and recognize collinearity.

25 minDraft

Key ideas

Adjusted slope

The estimated change in mean response per unit of a predictor while holding the other included predictors fixed.

Collinearity

Strong association among predictors can inflate coefficient standard errors and make individual estimates unstable, even when overall prediction is useful.

Adjusted R-squared

A fit measure that penalizes additional coefficients. Unlike ordinary R-squared, it may fall when a predictor is added.

Predictor selection

Consider subject-matter relevance, adjusted R-squared, residual error, and overlap among predictors. The smallest individual p-value is not a complete model-selection strategy.

Formulas

Multiple-regression model
y^=β^0+∑j=1pβ^jxj\hat y=\hat\beta_0+\sum_{j=1}^{p}\hat\beta_jx_j
Here p counts fitted slope coefficients, including indicators and interactions.
Degrees of freedom and adjusted fit
dfE=n−p−1,Radj2=1−(1−R2)n−1n−p−1df_E=n-p-1,\qquad R^2_{adj}=1-(1-R^2)\frac{n-1}{n-p-1}
Compare fits using the same response and observations.

Which procedure?

Quantitative response and multiple relevant explanatory variables
Multiple linear regression.
Predictors strongly overlap
Inspect correlations and coefficient uncertainty before interpreting individual contributions.
Worked example

Parcel weight and route length

For 40 parcels, model delivery cost in dollars from weight in kg and route length in km. Interpret weight and compare fit. This is an original example with composed R output.

coef(lm(cost ~ kg + km, data = parcels))
(Intercept)          kg          km
       4.00        1.50        0.25
# Same 40 observations, same response; selected summaries
# model       R-squared  Adj R-squared  RSE   residual df
# kg only        0.5000         0.4868  4.59       38
# kg + km        0.6300         0.6100  4.00       37
  1. Holding route length fixed, each extra kilogram is associated with an estimated $1.50 increase in mean cost.

  2. For 2 kg and 8 km, fitted cost = 4 + 1.5(2) + 0.25(8) = $9.

  3. Both adjusted R-squared and RSE improve after route length is included.

  4. This supports the expanded model on these criteria; it does not establish that the model controls every confounder.

Answer

The two-predictor model accounts for 63% of sample cost variability and has a typical residual scale of about $4. The weight coefficient is conditional on route length.

Three-question check

1
Lec 18

There are 50 observations and 3 fitted slope coefficients. Residual df is:

2
Lec 18

Two highly correlated predictors have large SEs when included together. This suggests:

3
Lec 18

Which comparison is most useful when an extra predictor is added?

MaizePrep is an independent study resource. It is not affiliated with, sponsored by, or endorsed by the University of Michigan. Course names and numbers are used only to describe which class the material covers. Every practice question is original, written in the style of the course.