Skip to content

12-03: One-Way ANOVA

The two-sample t-test of 11-02 compares two means. ANOVA (Analysis of Variance) compares three or more in a single test — without inflating the Type I error rate the way repeated t-tests would.

Why Not Just Run Many t-Tests?

With k = 4 groups there are 4C2 = 6 pairwise comparisons. If each runs at α = 0.05:

P(at least one false positive) = 1 − (0.95)⁶ = 0.265

A 26.5% chance of a spurious "significant" result. ANOVA tests all groups at once with a single, honest α.

The Idea: Partitioning Variation

ANOVA compares two independent estimates of the same variance.

Total variation  =  variation BETWEEN groups  +  variation WITHIN groups

     SST         =           SSB              +          SSW
  (total)              (between/treatment)          (within/error)

If H₀ is true (all means equal):
    between-group variation ≈ within-group variation   →  F ≈ 1

If the means differ:
    between-group variation  >  within-group variation →  F  >  1

The Formulas

                                                    N = total observations
SSB = Σ nᵢ (x̄ᵢ − x̄_grand)²          df_B = k − 1     k = number of groups
SSW = Σ (nᵢ − 1) sᵢ²                 df_W = N − k     nᵢ = size of group i
SST = SSB + SSW                       df_T = N − 1

MSB = SSB / (k − 1)         mean square between (the "treatment" variance)
MSW = SSW / (N − k)         mean square within  (the pooled error variance)

              MSB
F  =  ─────────────           df = (k − 1, N − k)      ALWAYS right-tailed
              MSW

The ANOVA table

Source          SS       df      MS = SS/df        F        p
──────────────────────────────────────────────────────────────
Between       SSB      k−1      MSB           MSB/MSW    P(F>f)
Within        SSW      N−k      MSW
──────────────────────────────────────────────────────────────
Total         SST      N−1

Hypotheses

H₀:  μ₁ = μ₂ = μ₃ = … = μₖ            all population means are equal

H₁:  At least ONE mean differs         (NOT "all means differ")

Conditions

1.  Independent random samples from each group
2.  Each population approximately normal (or each nᵢ ≥ 30)
3.  Equal population variances — HOMOGENEITY OF VARIANCE
      rule of thumb: largest sᵢ ≤ 2 × smallest sᵢ

If variances are clearly unequal, use Welch's ANOVA (oneway.test in R). If normality fails badly, use the Kruskal-Wallis test.

Worked Example

Three fertilizers are compared on plant growth (cm).

Fertilizer A Fertilizer B Fertilizer C
18, 22, 20, 24, 21 25, 28, 26, 30, 26 20, 19, 23, 21, 22
n₁ = 5, x̄₁ = 21.0, s₁ = 2.236 n₂ = 5, x̄₂ = 27.0, s₂ = 2.000 n₃ = 5, x̄₃ = 21.0, s₃ = 1.581 
STEP 1   H₀: μ_A = μ_B = μ_C
         H₁: at least one mean differs
         α = 0.05

STEP 2   Independent samples ✓   roughly normal ✓
         Variance check: 2.236 / 1.581 = 1.41 ≤ 2   ✓
         N = 15,  k = 3

STEP 3   Grand mean  x̄ = (5(21.0) + 5(27.0) + 5(21.0)) / 15 = 345/15 = 23.0

         SSB = 5(21.0−23)² + 5(27.0−23)² + 5(21.0−23)²
             = 5(4) + 5(16.00) + 5(4)
             = 20 + 80.0 + 20
             = 120.0                          df_B = 3 − 1 = 2

         SSW = 4(2.236²) + 4(2.000²) + 4(1.581²)
             = 4(5.0) + 4(4.00) + 4(2.50)
             = 20.0 + 16.0 + 10.0
             = 46.0                           df_W = 15 − 3 = 12

         MSB = 120.0 / 2  = 60.000
         MSW =  46.0 / 12 =  3.8333

                60.000
         F  =  ─────────  =  15.652         df = (2, 12)
                3.8333

STEP 4   Critical value:  F(0.05, 2, 12) = 3.885
         15.652 > 3.885                                →  REJECT H₀

         p-value:  P(F₂,₁₂ > 15.652) = 0.00045        →  REJECT H₀

STEP 5   There is strong evidence at the 5% level that mean plant growth
         differs among at least two of the three fertilizers.

The ANOVA table

Source SS df MS F p
Between groups 120.00 2 60.000 15.652 0.00045
Within groups 46.00 12 3.833
Total 166.00 14

Post-Hoc Tests: Which Groups Differ?

ANOVA says "at least one differs" — it does not say which. Only after a significant F do you run a post-hoc test, which controls the family-wise error rate.

Test When
Tukey's HSD All pairwise comparisons, equal group sizes — the standard choice
Bonferroni Few planned comparisons; conservative (divides α by the number of tests)
Scheffé Any contrast, including complex ones; very conservative
Games-Howell Unequal variances
Dunnett Comparing every group to one control

For our example, Tukey's HSD would show B differs from both A and C, while A and C do not differ — exactly what the group means suggest.

Effect Size

                SSB                                  SSB − (k−1)·MSW
Eta squared  η² = ────           Omega squared  ω² = ──────────────────
                SST                                    SST + MSW

η² = the proportion of total variation explained by group membership. Here η² = 120.0/166.0 = 0.723 — 72% of the variation in growth is attributable to fertilizer. Benchmarks: 0.01 small, 0.06 medium, 0.14 large.

Excel

' ══ ONE-CLICK: ANALYSIS TOOLPAK ═════════════════════════════════════
' Data ▸ Data Analysis ▸ Anova: Single Factor
'   Input Range = A1:C6  (groups in columns), Grouped By: Columns,
'   tick Labels in first row, Alpha = 0.05
'   → SUMMARY (count, sum, average, variance per group)
'   → ANOVA table (SS, df, MS, F, P-value, F crit)

' ══ BUILT BY HAND ═══════════════════════════════════════════════════
' Groups in A2:A6, B2:B6, C2:C6
=AVERAGE(A2:C6)                     ' grand mean x̄   -> 23
=COUNT(A2:C6)                       ' N              -> 15
=3                                  ' k              -> 3

' SSB — between-group sum of squares
=COUNT(A2:A6)*(AVERAGE(A2:A6)-$F$1)^2 + COUNT(B2:B6)*(AVERAGE(B2:B6)-$F$1)^2 +
 COUNT(C2:C6)*(AVERAGE(C2:C6)-$F$1)^2                      ' -> 120.0

' SSW — within-group sum of squares  (DEVSQ = Σ(x − x̄)² within a range)
=DEVSQ(A2:A6)+DEVSQ(B2:B6)+DEVSQ(C2:C6)                    ' -> 46.0

' SST — total
=DEVSQ(A2:C6)                                              ' -> 166.0
'   check: SSB + SSW = SST

' Mean squares and F
=F3/(F5-1)                          ' MSB = SSB/(k−1)  -> 60.000
=F4/(F2-F5)                         ' MSW = SSW/(N−k)  -> 3.8333
=F6/F7                              ' F                -> 15.6522

' ══ CRITICAL VALUE AND p-VALUE ══════════════════════════════════════
=F.INV.RT(0.05, 2, 12)              ' F critical       -> 3.8853
=F.DIST.RT(15.6522, 2, 12)          ' p-value          -> 0.000453
=IF(F10<=0.05, "Reject H0", "Fail to reject H0")

' ══ EFFECT SIZE ═════════════════════════════════════════════════════
=F3/F8                              ' η² = SSB/SST     -> 0.7229
=(F3-(3-1)*F7)/(F8+F7)              ' ω²               -> 0.6614

' ══ VARIANCE CHECK ══════════════════════════════════════════════════
=MAX(VAR.S(A2:A6),VAR.S(B2:B6),VAR.S(C2:C6)) /
 MIN(VAR.S(A2:A6),VAR.S(B2:B6),VAR.S(C2:C6))               ' ratio -> 2.0

' ══ TUKEY HSD (Excel has no built-in — compute by hand) ═════════════
' HSD = q(α, k, N−k) × SQRT(MSW / n)
' q(0.05, 3, 12) = 3.77 from a studentized-range table
=3.77*SQRT(F7/5)                    ' HSD -> 3.301
' Any pair of means differing by more than 3.258 is significantly different:
=ABS(AVERAGE(A2:A6)-AVERAGE(B2:B6))>3.301     ' A vs B -> TRUE  (diff = 6.0)
=ABS(AVERAGE(A2:A6)-AVERAGE(C2:C6))>3.301     ' A vs C -> FALSE (diff = 0.0)

R

A <- c(18, 22, 20, 24, 21)
B <- c(25, 28, 26, 30, 26)
C <- c(20, 19, 23, 21, 22)

# ANOVA needs LONG format
growth <- data.frame(
  cm  = c(A, B, C),
  fert = factor(rep(c("A", "B", "C"), each = 5))
)

# ══ THE TEST ════════════════════════════════════════════════════════
model <- aov(cm ~ fert, data = growth)
summary(model)
#             Df Sum Sq Mean Sq F value   Pr(>F)
# fert         2  120.0   60.00   15.65 0.000453 ***
# Residuals   12   46.0    3.83

anova(lm(cm ~ fert, data = growth))       # identical, regression view

qf(0.95, 2, 12)                            # critical F -> 3.8853
pf(15.6522, 2, 12, lower.tail = FALSE)     # p-value    -> 0.000453

# Group summaries
aggregate(cm ~ fert, growth, function(x) c(n = length(x), mean = mean(x), sd = sd(x)))

# ══ CHECKING ASSUMPTIONS ════════════════════════════════════════════
boxplot(cm ~ fert, data = growth, col = c("#5B2A86", "#0FA3A3", "#8A5FBF"),
        main = "Growth by fertilizer", ylab = "cm")

library(car)
leveneTest(cm ~ fert, data = growth)       # homogeneity of variance — robust
bartlett.test(cm ~ fert, data = growth)    # assumes normality

shapiro.test(residuals(model))             # normality of the RESIDUALS
par(mfrow = c(2, 2)); plot(model); par(mfrow = c(1, 1))

# ══ POST-HOC ════════════════════════════════════════════════════════
TukeyHSD(model)
#            diff        lwr       upr     p adj
# B-A         6.0   2.69850  9.301496 0.0010
# C-A         0.0  -3.30150  3.301496 1.0000
# C-B        -6.0  -9.30150 -2.698504 0.0010
plot(TukeyHSD(model))

pairwise.t.test(growth$cm, growth$fert, p.adjust.method = "bonferroni")

# ══ EFFECT SIZE ═════════════════════════════════════════════════════
ss <- summary(model)[[1]][["Sum Sq"]]
eta2 <- ss[1] / sum(ss); eta2              # 0.7229
# library(effectsize);  eta_squared(model);  omega_squared(model)

# ══ WHEN VARIANCES ARE UNEQUAL ══════════════════════════════════════
oneway.test(cm ~ fert, data = growth, var.equal = FALSE)    # Welch's ANOVA

# ══ NONPARAMETRIC ALTERNATIVE ═══════════════════════════════════════
kruskal.test(cm ~ fert, data = growth)

Python

import numpy as np
import pandas as pd
from scipy import stats
import statsmodels.api as sm
from statsmodels.formula.api import ols
from statsmodels.stats.multicomp import pairwise_tukeyhsd

A = np.array([18, 22, 20, 24, 21])
B = np.array([25, 28, 26, 30, 26])
C = np.array([20, 19, 23, 21, 22])

# ══ QUICK TEST ══════════════════════════════════════════════════════
stats.f_oneway(A, B, C)
#  F_onewayResult(statistic=15.6522, pvalue=0.000453)

stats.f.ppf(0.95, 2, 12)                 # critical F -> 3.8853
stats.f.sf(15.6522, 2, 12)               # p-value    -> 0.000453

# ══ FULL ANOVA TABLE (statsmodels — long format) ════════════════════
growth = pd.DataFrame({
    "cm":   np.concatenate([A, B, C]),
    "fert": np.repeat(["A", "B", "C"], 5),
})

model = ols("cm ~ C(fert)", data=growth).fit()
sm.stats.anova_lm(model, typ=2)
#              sum_sq    df          F    PR(>F)
# C(fert)      120.00   2.0  15.652174  0.000453
# Residual      46.00  12.0

growth.groupby("fert")["cm"].agg(["count", "mean", "std", "var"])

# ══ CHECKING ASSUMPTIONS ════════════════════════════════════════════
import matplotlib.pyplot as plt
fig, ax = plt.subplots()
ax.boxplot([A, B, C], tick_labels=["A", "B", "C"])
ax.set(ylabel="cm", title="Growth by fertilizer")

stats.levene(A, B, C)                    # homogeneity — robust
stats.bartlett(A, B, C)                  # assumes normality
stats.shapiro(model.resid)               # normality of residuals

sm.qqplot(model.resid, line="s"); plt.show()

# ══ POST-HOC ════════════════════════════════════════════════════════
tukey = pairwise_tukeyhsd(endog=growth["cm"], groups=growth["fert"], alpha=0.05)
print(tukey)
#  group1 group2 meandiff p-adj   lower   upper  reject
#     A      B      6.0   0.0010  2.6985  9.3015   True
#     A      C      0.0   1.0000 -3.3015  3.3015  False
#     B      C     -6.0   0.0010 -9.3015 -2.6985   True
tukey.plot_simultaneous(); plt.show()

# ══ EFFECT SIZE ═════════════════════════════════════════════════════
table = sm.stats.anova_lm(model, typ=2)
eta2 = table["sum_sq"][0] / table["sum_sq"].sum()
eta2                                      # 0.7229

# ══ WHEN VARIANCES ARE UNEQUAL ══════════════════════════════════════
# pingouin.welch_anova(data=growth, dv="cm", between="fert")

# ══ NONPARAMETRIC ALTERNATIVE ═══════════════════════════════════════
stats.kruskal(A, B, C)

Quick Reference

Task Excel R Python
One-way ANOVA ToolPak ▸ Anova: Single Factor aov(y ~ g) / summary() stats.f_oneway(*groups)
Full ANOVA table ToolPak output summary(aov(...)) sm.stats.anova_lm(model)
SSW by hand DEVSQ per group, summed sum((n-1)*s^2) sum((n-1)*var)
SST DEVSQ(all data) sum((y-mean(y))^2) ((y-y.mean())**2).sum()
Critical F F.INV.RT(α, df1, df2) qf(1-α, df1, df2) f.ppf(1-α, df1, df2)
p-value F.DIST.RT(F, df1, df2) pf(F, df1, df2, lower.tail=FALSE) f.sf(F, df1, df2)
Equal-variance test ToolPak ▸ F-Test (2 groups) car::leveneTest stats.levene
Tukey HSD manual with a q table TukeyHSD(model) pairwise_tukeyhsd(...)
Bonferroni α/m per test pairwise.t.test(..., "bonferroni") multipletests(..., "bonferroni")
Effect size η² SSB/SST effectsize::eta_squared sum_sq[0]/sum_sq.sum()
Unequal variances — oneway.test(..., var.equal=FALSE) pingouin.welch_anova
Nonparametric — kruskal.test(y ~ g) stats.kruskal(*groups)

Common Mistakes

  • Running k(k−1)/2 separate t-tests instead of one ANOVA, inflating the Type I error rate.
  • Concluding "all means differ". H₁ says at least one differs.
  • Skipping the post-hoc test and guessing which groups differ from the group means alone.
  • Running a post-hoc test when F was not significant.
  • Ignoring the equal-variance assumption. Check Levene's test; switch to Welch's ANOVA if it fails.
  • df_W = N − 1 instead of N − k.
  • Testing the normality of the raw data rather than of the residuals.
  • Feeding wide-format columns to R or Python. aov and ols need long format.

Exercises: 12-03: Exercises — One-Way ANOVA

⬅️ Previous: 12-02: Chi-Square Test of Independence ➡️ Next: 13-01: Correlation