Skip to content

12-01: Chi-Square Goodness-of-Fit Test

Everything so far tested means and proportions. The chi-square tests work on counts in categories — they compare what you observed against what a hypothesis says you should have expected.

The Chi-Square Statistic

              (O − E)²
χ²  =  Σ    ─────────────           summed over ALL categories
                  E

O = observed frequency (a COUNT, never a percentage)
E = expected frequency under H₀

Reading it: each term measures how far one category's count strays from expectation, scaled by how big that expectation was. χ² = 0 means a perfect match; large χ² means a poor fit.

The chi-square distribution

  • Right-skewed and takes only non-negative values.
  • Shape depends on the degrees of freedom; it becomes more symmetric as df grows.
  • Every chi-square test is right-tailed — only large values are evidence against H₀, because both over- and under-counts contribute positive terms.

The Goodness-of-Fit Test

Tests whether one categorical variable follows a specified distribution.

H₀:  The observed frequencies FIT the claimed distribution
     (e.g. p₁ = 0.25, p₂ = 0.25, p₃ = 0.25, p₄ = 0.25)

H₁:  The distribution differs from the claim (at least one pᵢ is different)

Expected count      E = n · pᵢ

Degrees of freedom  df = k − 1        k = number of categories

If any parameter is estimated from the data (fitting a Poisson, say), subtract one more df per estimated parameter.

Conditions

1.  Random sample
2.  Observations are INDEPENDENT and fall into exactly one category
3.  Every EXPECTED count ≥ 5
      (some texts: all E ≥ 1 and no more than 20% of cells below 5)
4.  Work with COUNTS, not percentages

If an expected count is too small, combine adjacent categories and recompute df.

Worked Example — A Uniform Claim

A die is rolled 120 times. Is it fair? Test at α = 0.05.

Face 1 2 3 4 5 6 Total
Observed O 15 25 18 22 16 24 120
Expected E 20 20 20 20 20 20 120 
STEP 1   H₀: the die is fair — every face has probability 1/6
         H₁: the die is not fair — at least one face differs
         α = 0.05

STEP 2   All E = 120 × (1/6) = 20 ≥ 5   ✓     df = 6 − 1 = 5

STEP 3   (15−20)²/20 = 25/20 = 1.25
         (25−20)²/20 = 25/20 = 1.25
         (18−20)²/20 =  4/20 = 0.20
         (22−20)²/20 =  4/20 = 0.20
         (16−20)²/20 = 16/20 = 0.80
         (24−20)²/20 = 16/20 = 0.80
                                ───────
                        χ²  =    4.50

STEP 4   Critical value:  χ²(0.05, 5) = 11.070
         4.50 < 11.070                                →  FAIL TO REJECT H₀

         p-value:  P(χ²₅ > 4.50) = 0.4799  >  0.05    →  FAIL TO REJECT H₀

STEP 5   There is not sufficient evidence at the 5% level to conclude that
         the die is unfair.

Worked Example — A Non-Uniform Claim

A store claims its sales split 40% online, 35% in-store, 25% phone. A sample of 200 orders gives 68 online, 82 in-store, 50 phone. Test at α = 0.05.

Channel O claimed p E = n·p (O−E)²/E
Online 68 0.40 80 1.800
In-store 82 0.35 70 2.057
Phone 50 0.25 50 0.000
Total 200 1.00 200 χ² = 3.857 
df = 3 − 1 = 2        χ²(0.05, 2) = 5.991        p-value = 0.1454

3.857 < 5.991  →  FAIL TO REJECT H₀.
There is not sufficient evidence that the actual split differs from the claim.

Tip

A cell contribution table — the (O−E)²/E column — is the most useful diagnostic in the whole chapter. When you do reject, the largest contributions tell you which categories drove it.

Chi-Square Test for a Variance

The same distribution, a completely different use — testing a claim about σ².

              (n − 1) s²
χ²  =  ────────────────────           df = n − 1
                  σ₀²

H₀: σ² = σ₀²          Requires the population to be NORMAL.

Unlike the goodness-of-fit test, this one can be left-, right-, or two-tailed.

Excel

' ══ SETUP: categories in A2:A7, observed in B2:B7 ═══════════════════
=SUM(B2:B7)                         ' n                    -> 120
=SUM($B$2:$B$7)/6                   ' C2: expected (equal) -> 20
=SUM($B$2:$B$7)*D2                  ' or: n × claimed p    (D = claimed p)
=(B2-C2)^2/C2                       ' E2: cell contribution
=SUM(E2:E7)                         ' χ² statistic         -> 4.50
=COUNTA(A2:A7)-1                    ' df                   -> 5

' ══ CRITICAL VALUE AND p-VALUE ══════════════════════════════════════
=CHISQ.INV.RT(0.05, 5)              ' critical value       -> 11.0705
=CHISQ.DIST.RT(4.5, 5)              ' p-value (right tail) -> 0.4799
=CHISQ.TEST(B2:B7, C2:C7)           ' p-value in ONE step  -> 0.4799
=IF(CHISQ.TEST(B2:B7,C2:C7)<=0.05, "Reject H0", "Fail to reject H0")

' ══ CONDITION CHECK ═════════════════════════════════════════════════
=MIN(C2:C7)>=5                      ' all expected ≥ 5?    -> TRUE
=COUNTIF(C2:C7,"<5")/COUNTA(C2:C7)  ' proportion of small cells

' ══ ONE-LINE χ² WITHOUT HELPER COLUMNS ══════════════════════════════
=SUMPRODUCT((B2:B7-C2:C7)^2/C2:C7)  ' -> 4.50

' ══ CHI-SQUARE TEST FOR A VARIANCE ══════════════════════════════════
' G1 = n, G2 = s, G3 = σ₀
=(G1-1)*G2^2/G3^2                   ' χ² statistic
=CHISQ.INV.RT(0.05, G1-1)           ' right-tailed critical value
=CHISQ.INV(0.05, G1-1)              ' left-tailed critical value
=CHISQ.DIST.RT(G5, G1-1)            ' right-tailed p-value

' ══ BUILDING OBSERVED COUNTS FROM RAW DATA ══════════════════════════
=COUNTIF($H$2:$H$121, A2)           ' observed count for each category

Warning

CHISQ.TEST(actual_range, expected_range) returns only the p-value, not χ². It also infers df from the shape of the ranges: for a single column or row it uses k − 1; for a two-dimensional block it uses (r−1)(c−1). Build the statistic yourself when you need to report it.

R

observed <- c(15, 25, 18, 22, 16, 24)
names(observed) <- 1:6

# ══ EQUAL-PROPORTIONS (UNIFORM) TEST ════════════════════════════════
chisq.test(observed)
#  X-squared = 4.5, df = 5, p-value = 0.4799

test <- chisq.test(observed)
test$statistic     # 4.5
test$parameter     # df = 5
test$p.value       # 0.4799
test$expected      # 20 20 20 20 20 20
test$residuals     # (O − E)/√E   — signed, per cell
test$residuals^2   # the (O − E)²/E contributions

# ══ SPECIFIED (NON-UNIFORM) PROPORTIONS ═════════════════════════════
orders <- c(online = 68, instore = 82, phone = 50)
claimed <- c(0.40, 0.35, 0.25)
chisq.test(orders, p = claimed)
#  X-squared = 3.8571, df = 2, p-value = 0.1454

# ══ BY HAND — worth doing once ══════════════════════════════════════
gof <- function(O, p = rep(1/length(O), length(O))) {
  E   <- sum(O) * p
  chi <- sum((O - E)^2 / E)
  df  <- length(O) - 1
  data.frame(O, E, contrib = (O - E)^2 / E,
             chi2 = chi, df = df,
             p.value = pchisq(chi, df, lower.tail = FALSE),
             critical = qchisq(0.95, df))
}
gof(observed)
gof(orders, claimed)

qchisq(0.95, df = 5)                       # critical value -> 11.0705
pchisq(4.5, 5, lower.tail = FALSE)         # p-value        -> 0.4799

# ══ FROM RAW DATA ═══════════════════════════════════════════════════
set.seed(1)
rolls <- sample(1:6, 120, replace = TRUE)
chisq.test(table(rolls))

# ══ VISUALISING OBSERVED vs EXPECTED ════════════════════════════════
barplot(rbind(observed, rep(20, 6)), beside = TRUE,
        col = c("#5B2A86", "#0FA3A3"), names.arg = 1:6,
        legend.text = c("Observed", "Expected"),
        main = "Die rolls: observed vs expected", xlab = "Face")

# ══ THE CHI-SQUARE DISTRIBUTION ITSELF ══════════════════════════════
curve(dchisq(x, 5), from = 0, to = 25, col = "#5B2A86", lwd = 2,
      ylab = "density", main = "Chi-square, df = 5")
xs <- seq(qchisq(0.95, 5), 25, length.out = 200)
polygon(c(qchisq(0.95,5), xs, 25), c(0, dchisq(xs, 5), 0),
        col = "#0FA3A380", border = NA)

# ══ TEST FOR A VARIANCE ═════════════════════════════════════════════
var_test <- function(s, n, sigma0, alternative = "two.sided") {
  chi <- (n - 1) * s^2 / sigma0^2
  df  <- n - 1
  p <- switch(alternative,
              greater   = pchisq(chi, df, lower.tail = FALSE),
              less      = pchisq(chi, df),
              two.sided = 2 * min(pchisq(chi, df),
                                  pchisq(chi, df, lower.tail = FALSE)))
  c(chi2 = chi, df = df, p.value = p)
}
var_test(s = 6.2, n = 15, sigma0 = 5)

Python

import numpy as np
import pandas as pd
from scipy import stats

observed = np.array([15, 25, 18, 22, 16, 24])

# ══ EQUAL-PROPORTIONS (UNIFORM) TEST ════════════════════════════════
stats.chisquare(observed)
#  Power_divergenceResult(statistic=4.5, pvalue=0.4799)

res = stats.chisquare(observed)
res.statistic, res.pvalue

# ══ SPECIFIED (NON-UNIFORM) PROPORTIONS ═════════════════════════════
orders  = np.array([68, 82, 50])
claimed = np.array([0.40, 0.35, 0.25])
expected = orders.sum() * claimed                 # must SUM to the same total
stats.chisquare(orders, f_exp=expected)
#  statistic=3.8571, pvalue=0.1454

# ══ BY HAND ═════════════════════════════════════════════════════════
def gof(O, p=None):
    O = np.asarray(O, dtype=float)
    p = np.full(O.size, 1 / O.size) if p is None else np.asarray(p)
    E = O.sum() * p
    contrib = (O - E) ** 2 / E
    chi = contrib.sum()
    df = O.size - 1
    return pd.DataFrame({"O": O, "E": E, "contrib": contrib}).assign(
        chi2=chi, df=df,
        p_value=stats.chi2.sf(chi, df),
        critical=stats.chi2.ppf(0.95, df))

gof(observed)
gof(orders, claimed)

stats.chi2.ppf(0.95, df=5)          # critical value -> 11.0705
stats.chi2.sf(4.5, 5)               # p-value        -> 0.4799

# ══ FROM RAW DATA ═══════════════════════════════════════════════════
rng = np.random.default_rng(1)
rolls = pd.Series(rng.integers(1, 7, 120))
stats.chisquare(rolls.value_counts().sort_index())

# ══ VISUALISING OBSERVED vs EXPECTED ════════════════════════════════
import matplotlib.pyplot as plt
x = np.arange(1, 7); w = 0.4
fig, ax = plt.subplots()
ax.bar(x - w/2, observed, w, label="Observed", color="#5B2A86")
ax.bar(x + w/2, np.full(6, 20), w, label="Expected", color="#0FA3A3")
ax.set(xlabel="Face", ylabel="Count", title="Die rolls"); ax.legend()

# ══ TEST FOR A VARIANCE ═════════════════════════════════════════════
def var_test(s, n, sigma0, alternative="two-sided"):
    chi = (n - 1) * s**2 / sigma0**2
    df = n - 1
    if alternative == "greater":
        p = stats.chi2.sf(chi, df)
    elif alternative == "less":
        p = stats.chi2.cdf(chi, df)
    else:
        p = 2 * min(stats.chi2.cdf(chi, df), stats.chi2.sf(chi, df))
    return {"chi2": chi, "df": df, "p_value": p}

var_test(6.2, 15, 5)

Quick Reference

Task Excel R Python
Expected counts n * p sum(O) * p O.sum() * p
χ² statistic SUMPRODUCT((O-E)^2/E) sum((O-E)^2/E) ((O-E)**2/E).sum()
Critical value CHISQ.INV.RT(α, df) qchisq(1-α, df) chi2.ppf(1-α, df)
p-value CHISQ.DIST.RT(χ², df) pchisq(χ², df, lower.tail=FALSE) chi2.sf(χ², df)
Full test (uniform) CHISQ.TEST(O, E) chisq.test(O) stats.chisquare(O)
Full test (specified p) CHISQ.TEST(O, n*p) chisq.test(O, p = p) chisquare(O, f_exp=n*p)
Cell contributions (O-E)^2/E column test$residuals^2 (O-E)**2/E
Test for a variance (n-1)*s^2/σ₀^2 user function user function

Common Mistakes

  • Running the test on percentages instead of counts. The statistic depends on n; percentages destroy it.
  • Forgetting that the expected counts must sum to the same total as the observed counts.
  • Using a two-tailed critical value. The goodness-of-fit test is always right-tailed.
  • Ignoring the "all E ≥ 5" condition. Combine sparse categories instead.
  • df = k instead of k − 1.
  • Concluding which category is off from the overall test. It only says "something differs" — look at the cell contributions to see where.
  • Assuming CHISQ.TEST reports the statistic. It reports the p-value only.

Exercises: 12-01: Exercises — Chi-Square Goodness-of-Fit Test

⬅️ Previous: 11-03: Paired t-Test ➡️ Next: 12-02: Chi-Square Test of Independence