A researcher wants to know whether mean daily screen time differs among first-years, sophomores, juniors and seniors, using a random sample from each class year. Which approach fits, and why?
One-way ANOVA
Compare a quantitative response across three or more groups: the F statistic as a ratio of two kinds of variation, its right-tail p-value, and Tukey intervals to find which groups differ.
Key ideas
- When to use it
Quantitative response, categorical explanatory variable with three or more groups (e.g. GPA across three colleges). With exactly two groups a two-sample t-test is enough. Running a t-test for every pair instead adds a false-positive chance with each test; ANOVA tests all the means at once.
- The two things ANOVA looks at
In side-by-side boxplots: (1) how far apart the group means are, and (2) how variable the responses are within each group. Means that look far apart are weak evidence if each group is very spread out.
- Hypotheses
(no association: the population means are all equal). : at least one population mean differs. The alternative is always that sentence; it never says which group, or that all differ.
- F statistic
A ratio: variation between the group means divided by variation within the groups. Define it in words: "the variation among the sample means was F times as large as the variation within groups." Unlike z or t, it does not count standard errors from a null value.
Example F = 10: the numerator is 10 times as large as the denominator.
- F distribution and p-value
F is a ratio of variances, so it is never negative. Its distribution starts at 0 and is skewed right, and the p-value is always the area to the right of the observed F (no two-sided version). Larger F, smaller p, stronger evidence.
- Tukey's procedure
After a small ANOVA p-value, Tukey builds a confidence interval for each pair's difference in population means. Read lwr and upr: an interval containing 0 gives no evidence of a difference for that pair; an interval entirely above or below 0 gives evidence of a difference, in the direction of the subtraction order.
What to know
Which procedure?
Study breaks and quiz scores: the F test
A tutoring center randomly assigns 60 volunteer students (20 per group) to one of three 5-minute study breaks before a 10-point memory quiz: no break (none), a walk (walk), or phone scrolling (scroll). Sample mean scores: none 6.1, scroll 5.9, walk 7.5. Is break type associated with mean quiz score? This is an original example with composed R output.
summary(aov(score ~ breaktype, data = breaks))
Df Sum Sq Mean Sq F value Pr(>F)
breaktype 2 30.4 15.2 7.6 0.001186 **
Residuals 57 114.0 2.0Hypotheses. versus : at least one population mean differs. is the true mean quiz score for all students like these who take a 5-minute walk before the quiz.
Read two numbers only. F = 7.6 and its p-value 0.001186. The Df, Sum Sq and Mean Sq columns are intermediate calculations you can ignore in STATS 250.
Define F. The variation among the three sample mean quiz scores was 7.6 times as large as the variation in individual scores within the break groups.
p-value. Assuming all three population means are equal, the probability of an F statistic of 7.6 or larger is about 0.0012: the right tail of the F curve.
Conclude, and stop there. p ≤ 0.01, so very strong evidence that at least one population mean quiz score differs among the break types. The F test cannot say which one; the plot suggests the walk, but that needs Tukey.
Very strong evidence (p ≈ 0.0012) that at least one break type has a different population mean quiz score. Random assignment supports a causal reading for students like these volunteers; which break differs needs a follow-up.
Study breaks: which pairs differ? (Tukey)
Use the Tukey output for the same study to say which break types differ in population mean quiz score, and in which direction.
TukeyHSD(aov(score ~ breaktype, data = breaks))
Tukey multiple comparisons of means
95% family-wise confidence level
$breaktype
diff lwr upr p adj
scroll-none -0.2 -1.2761829 0.8761829 0.8958
walk-none 1.4 0.3238171 2.4761829 0.0076
walk-scroll 1.6 0.5238171 2.6761829 0.0020Read only lwr and upr. Each row is a 95% interval for the first-named group's population mean minus the second's.
scroll − none: (-1.28, 0.88) contains 0: no evidence that scrolling and no break differ.
walk − none: (0.32, 2.48) is entirely positive: evidence the walk's population mean score is higher than no break's.
walk − scroll: (0.52, 2.68) is entirely positive: evidence the walk's mean is higher than scrolling's.
Put it together. The walk group differs from both others; scrolling and no break do not appear to differ. That pinpoints what the ANOVA conclusion could only call "at least one".
The population mean quiz score appears higher after a walk than after no break (by about 0.32 to 2.48 points) or after scrolling (0.52 to 2.68 points). There is no evidence of a difference between scrolling and no break, which does not mean they are exactly equal.
Check yourself
Mean battery life is compared across three phone brands. Which pair of hypotheses is written correctly?
Select every true statement about the F statistic and its p-value.
An ANOVA comparing mean delivery time across three couriers gives F = 2.8 with p = 0.07. Which conclusion fits the course scale?
After an ANOVA on plant height for fertilizers A, B and C, Tukey gives these 95% intervals: B − A (1.2, 4.8) cm, C − A (−0.9, 2.7) cm, C − B (−3.9, −0.3) cm. Select every conclusion the intervals support.
A study compares mean reaction time across three caffeine doses. Which change, with everything else the same, would make the F statistic larger?