12-03: Exercises — One-Way ANOVA¶
Notes reference: 12-03: One-Way ANOVA
Q1: Why not repeated t-tests?¶
You have 5 groups. If you compare every pair with a t-test at α = 0.05:
- How many tests?
- What is the probability of at least one false positive if all group means are actually equal?
- What does ANOVA do instead?
Solution
1. 5C2 = 10 pairwise comparisons
2. P(at least one false positive) = 1 − (0.95)^10 = 1 − 0.5987 = 0.4013
→ a 40% chance of a spurious "significant" result
With 10 groups: 45 tests, 1 − 0.95^45 = 0.90 — near certainty.
3. ANOVA runs ONE test at α = 0.05, comparing all 5 means at once.
Only if it rejects do you proceed to post-hoc comparisons, which
themselves control the family-wise error rate (Tukey, Bonferroni).
Q2: Build the ANOVA table by hand¶
Four training methods, 5 trainees each. Scores:
| Method A | Method B | Method C | Method D |
|---|---|---|---|
| 78 | 85 | 72 | 90 |
| 82 | 88 | 75 | 92 |
| 80 | 84 | 70 | 88 |
| 76 | 90 | 74 | 94 |
| 84 | 83 | 79 | 86 |
Test at α = 0.05.
Solution
GROUP SUMMARIES (n = 5 each, N = 20, k = 4)
A: Σ = 400, x̄₁ = 80, Σ(x−x̄₁)² = 40 → s₁² = 10.0
B: Σ = 430, x̄₂ = 86, Σ(x−x̄₂)² = 34 → s₂² = 8.5
C: Σ = 370, x̄₃ = 74, Σ(x−x̄₃)² = 46 → s₃² = 11.5
D: Σ = 450, x̄₄ = 90, Σ(x−x̄₄)² = 40 → s₄² = 10.0
Grand total = 1650, grand mean x̄ = 1650/20 = 82.5
VARIANCE CHECK: 11.5/8.5 = 1.35 < 4 ✓ homogeneity is reasonable
SSB = Σ nᵢ(x̄ᵢ − x̄)²
= 5(80−82.5)² + 5(86−82.5)² + 5(74−82.5)² + 5(90−82.5)²
= 5(6.25) + 5(12.25) + 5(72.25) + 5(56.25)
= 31.25 + 61.25 + 361.25 + 281.25
= 735.00 df_B = 4 − 1 = 3
SSW = 40 + 34 + 46 + 40 = 160.00 df_W = 20 − 4 = 16
SST = 735 + 160 = 895.00 df_T = 19
MSB = 735/3 = 245.0000
MSW = 160/16 = 10.0000
F = 245.0000/10.0000 = 24.5000 df = (3, 16)
ANOVA table
| Source | SS | df | MS | F | p |
|---|---|---|---|---|---|
| Between groups | 735.00 | 3 | 245.000 | 24.500 | 3.0 × 10⁻⁶ |
| Within groups | 160.00 | 16 | 10.000 | ||
| Total | 895.00 | 19 |
critical F(0.05, 3, 16) = 3.2389
24.5000 > 3.2389 → REJECT H₀
p-value = P(F₃,₁₆ > 24.5000) = 0.0000030 → REJECT H₀
CONCLUSION
There is very strong evidence at the 5% level that mean scores differ
among at least two of the four training methods.
EFFECT SIZE
η² = SSB/SST = 735/895 = 0.8212
→ 82% of the variation in scores is explained by training method.
' Groups in A2:D6
' Data ▸ Data Analysis ▸ Anova: Single Factor
' Input Range = A1:D6, Grouped By: Columns, tick Labels in first row
' By hand:
=DEVSQ(A2:A6)+DEVSQ(B2:B6)+DEVSQ(C2:C6)+DEVSQ(D2:D6) ' SSW -> 160
=DEVSQ(A2:D6) ' SST -> 895
=F2-F1 ' SSB -> 735
=F3/3 ' MSB -> 245
=F1/16 ' MSW -> 10.00
=F4/F5 ' F -> 24.5000
=F.INV.RT(0.05, 3, 16) ' 3.23887
=F.DIST.RT(24.5, 3, 16) ' 0.0000030
scores <- c(78,82,80,76,84, 85,88,84,90,83, 72,75,70,74,79, 90,92,88,94,86)
method <- factor(rep(c("A","B","C","D"), each = 5))
m <- aov(scores ~ method)
summary(m)
# Df Sum Sq Mean Sq F value Pr(>F)
# method 3 735 245.00 24.5 2.99e-06 ***
# Residuals 16 160 10.00
A = [78,82,80,76,84]; B = [85,88,84,90,83]
C = [72,75,70,74,79]; D = [90,92,88,94,86]
stats.f_oneway(A, B, C, D) # F=24.5000, p=2.99e-06
Q3: Post-hoc — which groups differ?¶
Using the Q2 result, run Tukey's HSD by hand and identify every significant pair.
Solution
______
/ MSW ______
HSD = q(α, k, N−k) · √ ───── = q · √ 10.00/5 = q × 1.41421
n
q(0.05, 4, 16) = 4.046 (from a studentized-range table)
HSD = 4.046 × 1.41421 = 5.7219
Any two means differing by MORE than 5.72 are significantly different.
PAIRWISE DIFFERENCES (means: A=80, B=86, C=74, D=90)
|A − B| = |80 − 86| = 6 > 5.72 → SIGNIFICANT
|A − C| = |80 − 74| = 6 > 5.72 → SIGNIFICANT
|A − D| = |80 − 90| = 10 > 5.72 → SIGNIFICANT
|B − C| = |86 − 74| = 12 > 5.72 → SIGNIFICANT
|B − D| = |86 − 90| = 4 < 5.72 → not significant
|C − D| = |74 − 90| = 16 > 5.72 → SIGNIFICANT
SUMMARY (underline joins groups that are NOT significantly different):
C < A < B D
74 80 86 90
─────────
Methods B and D are the best and are indistinguishable from each other.
Method A is significantly worse than both.
Method C is significantly worse than everything.
RECOMMENDATION: adopt B or D; choose between them on cost, not on score.
TukeyHSD(m)
# diff lwr upr p adj
# B-A 6 0.278100 11.721900 0.0426
# C-A -6 -11.721900 -0.278100 0.0426
# D-A 10 4.278100 15.721900 0.0007
# C-B -12 -17.721900 -6.278100 0.0001
# D-B 4 -1.721900 9.721900 0.2427
# D-C 16 10.278100 21.721900 0.0000
plot(TukeyHSD(m))
pairwise.t.test(scores, method, p.adjust.method = "bonferroni")
from statsmodels.stats.multicomp import pairwise_tukeyhsd
df = pd.DataFrame({"score": A+B+C+D, "method": np.repeat(list("ABCD"), 5)})
print(pairwise_tukeyhsd(df["score"], df["method"], alpha=0.05))
' Excel has no Tukey HSD. Compute it from the ToolPak output:
=4.046*SQRT(10/5) ' HSD -> 5.7219
=ABS(AVERAGE(A2:A6)-AVERAGE(B2:B6))>5.7219 ' TRUE = significant pair
Q4: An ANOVA that does not reject¶
Three suppliers deliver components. Lengths (mm):
| Supplier X | Supplier Y | Supplier Z |
|---|---|---|
| 25.1 | 25.3 | 24.9 |
| 24.8 | 25.0 | 25.2 |
| 25.3 | 24.9 | 25.1 |
| 24.9 | 25.2 | 24.8 |
| 25.4 | 25.1 | 25.5 |
Test at α = 0.05.
Solution
GROUP MEANS
X: Σ = 125.5, x̄₁ = 25.10
Y: Σ = 125.5, x̄₂ = 25.10
Z: Σ = 125.5, x̄₃ = 25.10
Grand mean = 25.10 N = 15, k = 3
SSB = 5(25.10−25.10)² × 3 = 0.0000 df_B = 2
SSW:
X: deviations 0.0, −0.3, 0.2, −0.2, 0.3 → Σ² = 0.26
Y: deviations 0.2, −0.1, −0.2, 0.1, 0.0 → Σ² = 0.10
Z: deviations −0.2, 0.1, 0.0, −0.3, 0.4 → Σ² = 0.30
SSW = 0.26 + 0.10 + 0.30 = 0.66 df_W = 12
MSB = 0.00/2 = 0.0000
MSW = 0.66/12 = 0.0550
F = 0.0000/0.0550 = 0.0000
critical F(0.05, 2, 12) = 3.8853
0.0000 < 3.8853 → FAIL TO REJECT H₀
p-value = 1.0000
CONCLUSION
No evidence that the suppliers differ in mean length. The three group
means are IDENTICAL (25.10), so SSB is exactly zero and F is exactly
zero — the cleanest possible "no effect".
η² = 0/0.66 = 0.000 — group membership explains none of the variation.
DO NOT RUN A POST-HOC TEST. F was not significant; there is nothing to
follow up.
Q5: Reading a ToolPak output¶
Anova: Single Factor
SUMMARY
Groups Count Sum Average Variance
Group 1 12 714 59.500 24.818
Group 2 12 780 65.000 31.091
Group 3 12 822 68.500 28.455
ANOVA
Source of Variation SS df MS F P-value F crit
Between Groups 488.667 2 244.333 8.618 0.000841 3.2849
Within Groups 935.833 33 28.359
Total 1424.500 35
- State the hypotheses and the decision at
α = 0.05. - How many observations in total?
- Verify
F. - Compute
η². - What comes next?
Solution
1. H₀: μ₁ = μ₂ = μ₃ H₁: at least one mean differs
F = 8.618 > F crit = 3.2849, and p = 0.000841 ≤ 0.05
→ REJECT H₀. The three group means are not all equal.
2. N = 12 × 3 = 36 (also visible as df_Total = N − 1 = 35)
3. F = MSB/MSW = 244.333/28.359 = 8.6156 ✓ matches 8.618 to rounding
Check the df: df_B = k − 1 = 2 ✓, df_W = N − k = 33 ✓
Check the SS: 488.667 + 935.833 = 1424.500 ✓
4. η² = SSB/SST = 488.667/1424.500 = 0.3430
→ 34% of the variation is explained by group membership — a LARGE
effect by Cohen's benchmarks (0.01 / 0.06 / 0.14).
5. NEXT STEPS
a) Check the assumptions you skipped: Levene's test for equal
variances (the sample variances 24.8, 31.1, 28.5 look fine —
ratio 1.25), and a Q-Q plot of the residuals.
b) Run a POST-HOC test (Tukey HSD) to find WHICH pairs differ.
From the means 59.5, 65.0, 68.5, the 1-vs-3 gap of 9.0 is the
most likely significant pair.
c) Report F(2, 33) = 8.62, p < .001, η² = 0.34, plus the group
means with their confidence intervals.
Q6: Check the assumptions¶
List the three ANOVA assumptions and how to check each.
Solution
1. INDEPENDENCE
Comes from the DESIGN: random sampling, random assignment, no subject
in two groups, no repeated measures. There is no software test.
Violation is the most serious and the least detectable.
2. NORMALITY (of the residuals, not the raw data)
• Q-Q plot of residuals — the primary check
• Shapiro-Wilk on residuals
ANOVA is fairly ROBUST to moderate non-normality when group sizes
are equal and n per group ≥ 15.
3. HOMOGENEITY OF VARIANCE
• Rule of thumb: largest sᵢ ≤ 2 × smallest sᵢ (or largest variance
≤ 4 × smallest)
• Levene's test (robust — the practical choice)
• Bartlett's test (more powerful, but assumes normality)
• Side-by-side boxplots — look for equal box heights
Violation matters most when the group sizes are UNEQUAL.
m <- aov(scores ~ method)
par(mfrow = c(2, 2)); plot(m); par(mfrow = c(1, 1))
# 1 Residuals vs Fitted → equal spread across groups?
# 2 Normal Q-Q → residuals normal?
# 3 Scale-Location → equal variance?
# 4 Residuals vs Leverage → influential points?
shapiro.test(residuals(m)) # normality of residuals
library(car); leveneTest(scores ~ method) # equal variance — robust
bartlett.test(scores ~ method) # equal variance — assumes normality
boxplot(scores ~ method, col = c("#5B2A86","#8A5FBF","#0FA3A3","#0B7A7A"))
# If variances are unequal:
oneway.test(scores ~ method, var.equal = FALSE) # Welch's ANOVA
# If normality fails badly:
kruskal.test(scores ~ method) # Kruskal-Wallis
stats.levene(A, B, C, D)
stats.bartlett(A, B, C, D)
stats.shapiro(model.resid)
stats.kruskal(A, B, C, D)
' Excel has no Levene's test. Use the variance ratio from the ToolPak
' SUMMARY block:
=MAX(variances)/MIN(variances) ' want < 4
' And Data ▸ Data Analysis ▸ F-Test Two-Sample for Variances
' pairwise on the largest and smallest.
Q7: Unequal group sizes¶
Three groups with n₁ = 8, n₂ = 12, n₃ = 10. Group means 45.2, 51.8, 48.5. Grand mean 48.83. SSW = 620.4.
Complete the ANOVA table and test at α = 0.05.
Solution
N = 8 + 12 + 10 = 30, k = 3
SSB = 8(45.2 − 48.83)² + 12(51.8 − 48.83)² + 10(48.5 − 48.83)²
= 8(13.1769) + 12(8.8209) + 10(0.1089)
= 105.4152 + 105.8508 + 1.0890
= 212.3550 df_B = 2
SSW = 620.4000 df_W = 30 − 3 = 27
SST = 832.7550 df_T = 29
MSB = 212.3550/2 = 106.1775
MSW = 620.4000/27 = 22.9778
F = 106.1775/22.9778 = 4.6209 df = (2, 27)
critical F(0.05, 2, 27) = 3.3541
4.6209 > 3.3541 → REJECT H₀
p-value = P(F₂,₂₇ > 4.6209) = 0.0189
CONCLUSION
There is sufficient evidence at the 5% level that the three group
means are not all equal.
η² = 212.355/832.755 = 0.2550 → 25% of variation explained; large.
NOTE ON UNEQUAL n
• Each group's deviation is weighted by ITS OWN nᵢ in SSB — that is
why the 12-member group contributes as much as the 8-member group
despite a smaller deviation.
• Unequal sizes make ANOVA MORE sensitive to unequal variances, so
run Levene's test before trusting this result.
• For post-hoc comparisons with unequal n, use the Tukey-Kramer
adjustment (R's TukeyHSD does this automatically) or Games-Howell
if the variances also differ.
Q8: ANOVA with two groups = the t-test¶
Two groups: n₁ = n₂ = 10, x̄₁ = 52, x̄₂ = 58, s₁² = 20, s₂² = 24.
Run both a pooled t-test and a one-way ANOVA. Show that F = t².
Solution
POOLED t-TEST
s²_p = [9(20) + 9(24)]/18 = (180 + 216)/18 = 396/18 = 22.0
SE = √(22.0 × (1/10 + 1/10)) = √(22.0 × 0.2) = √4.4 = 2.09762
t = (52 − 58)/2.09762 = −6/2.09762 = −2.86032 df = 18
p (two-tailed) = 0.010374
ONE-WAY ANOVA
Grand mean = (52 + 58)/2 = 55 (equal n, so a simple average)
SSB = 10(52−55)² + 10(58−55)² = 10(9) + 10(9) = 180 df_B = 1
SSW = 9(20) + 9(24) = 396 df_W = 18
MSB = 180/1 = 180.0
MSW = 396/18 = 22.0
F = 180.0/22.0 = 8.18182 df = (1, 18)
p = P(F₁,₁₈ > 8.18182) = 0.010374
VERIFY
t² = (−2.86032)² = 8.18143 ≈ F = 8.18182 ✓ (rounding)
p-values are IDENTICAL: 0.010374
THE RELATIONSHIP
For exactly two groups, F = t² and F(1, df) ≡ t(df) squared.
ANOVA is the generalization of the pooled two-sample t-test to k > 2
groups. That is also why ANOVA cannot be one-tailed — squaring
destroys the sign, so the F-test is inherently two-sided.
⬅️ Previous: 12-02: Exercises — Chi-Square Test of Independence ➡️ Next: 13-01: Exercises — Correlation