12-02: Chi-Square Test of Independence

The same χ² statistic as 12-01, applied to a two-way contingency table. It answers the question raised informally in 05-02: are these two categorical variables related, or is the pattern just sampling noise?

---

The Hypotheses

H₀:  The two variables are INDEPENDENT (not associated)

H₁:  The two variables are DEPENDENT (associated)

Note there is no direction and no "greater than" — the alternative is simply "some association exists".

---

Expected Counts Under Independence

If two events are independent, P(A and B) = P(A)·P(B) (05-02). Multiply that joint probability by n and the n's cancel into the row-by-column rule:

                (row total) × (column total)
E_ij   =   ─────────────────────────────────────
                      grand total

              (O − E)²
χ²  =  Σ    ─────────────           df  =  (r − 1)(c − 1)
                  E

r = number of rows, c = number of columns. Always right-tailed.

Conditions

1.  Random sample; each observation falls in exactly ONE cell
2.  Observations independent of one another
3.  All EXPECTED counts ≥ 5
      (a common relaxation: all E ≥ 1, and no more than 20% of cells below 5)
4.  COUNTS, not percentages

For a 2×2 table with small counts, use Fisher's exact test instead, or apply Yates' continuity correction (R's chisq.test applies it by default on 2×2 tables).

---

Worked Example

Is exercise habit associated with self-reported health? A survey of 200 people:

Observed

	Excellent	Good	Poor	Row total
Exercises	45	55	20	120
Does not	15	30	35	80
Column total	60	85	55	200

STEP 1   H₀: exercise habit and health rating are INDEPENDENT
         H₁: they are ASSOCIATED
         α = 0.05

STEP 2 — expected counts

E(Exercises, Excellent) = 120 × 60 / 200 = 36.0
E(Exercises, Good)      = 120 × 85 / 200 = 51.0
E(Exercises, Poor)      = 120 × 55 / 200 = 33.0
E(Does not, Excellent)  =  80 × 60 / 200 = 24.0
E(Does not, Good)       =  80 × 85 / 200 = 34.0
E(Does not, Poor)       =  80 × 55 / 200 = 22.0

Expected	Excellent	Good	Poor
Exercises	36.0	51.0	33.0
Does not	24.0	34.0	22.0

All expected counts ≥ 5 ✓ · df = (2−1)(3−1) = 2

STEP 3 — the statistic

Cell	O	E	(O−E)²/E
Ex, Excellent	45	36.0	2.250
Ex, Good	55	51.0	0.314
Ex, Poor	20	33.0	5.121
No, Excellent	15	24.0	3.375
No, Good	30	34.0	0.471
No, Poor	35	22.0	7.682
			χ² = 19.213

STEP 4 — decide

Critical value:  χ²(0.05, 2) = 5.991
19.213 > 5.991                                  →  REJECT H₀

p-value:  P(χ²₂ > 19.213) = 0.0000672           →  REJECT H₀

STEP 5 — conclude

There is very strong evidence at the 5% level of an association between exercise habit and self-reported health rating.

Where the association lives. The two biggest contributions are "Does not exercise / Poor health" (7.68, observed 35 vs. expected 22) and "Exercises / Poor health" (5.12, observed 20 vs. expected 33). Non-exercisers report poor health far more often than independence would predict.

Warning

A significant chi-square shows association, not causation. This was an observational study (08-01) — healthy people may simply find it easier to exercise, and a confounder such as age could drive both.

---

Measuring the Strength of the Association

χ² grows with n, so it is not an effect size on its own. Report one of these:

                    ______                                 ______
                   /  χ²                                  /  χ²
Phi (2×2 only) φ = √ ────        Cramér's V  =        √ ──────────
                      n                                  n · min(r−1, c−1)


                    ______
                   /   χ²
Contingency C  =  √ ────────
                    χ² + n

Interpreting Cramér's V (for df* = min(r−1, c−1) = 1):

V	Strength
0.10	Small
0.30	Medium
0.50	Large

For our example: V = √(19.213 / (200 × 1)) = √0.09607 = 0.310 — a medium association.

---

Test of Homogeneity — Same Arithmetic, Different Design

	Test of independence	Test of homogeneity
Sampling	One sample, two variables recorded	Several samples, one from each population
Row totals	Random	Fixed by design
Question	"Are these two variables related?"	"Do these populations have the same distribution?"
Example	Survey 200 people, record exercise and health	Sample 100 from each of 3 hospitals, record outcome
Computation	Identical	Identical

The χ² statistic, expected counts, and df are exactly the same. Only the wording of the hypotheses and conclusion changes.

---

Excel

' ══ OBSERVED TABLE in B2:D3, with totals ════════════════════════════
=SUM(B2:D2)                         ' E2: row total        -> 120
=SUM(B2:B3)                         ' B4: column total     -> 60
=SUM(B2:D3)                         ' E4: grand total      -> 200

' ══ EXPECTED TABLE in B7:D8 ═════════════════════════════════════════
=$E2*B$4/$E$4                       ' B7: row × col / grand -> 36
'   copy across B7:D8 — the mixed references make it fill correctly

' ══ CHI-SQUARE STATISTIC ════════════════════════════════════════════
=(B2-B7)^2/B7                       ' B11: one cell's contribution
=SUM(B11:D12)                       ' χ² -> 19.2130
=SUMPRODUCT((B2:D3-B7:D8)^2/B7:D8)  ' the same, in one formula

' ══ DEGREES OF FREEDOM, CRITICAL VALUE, p-VALUE ═════════════════════
=(ROWS(B2:D3)-1)*(COLUMNS(B2:D3)-1) ' df -> 2
=CHISQ.INV.RT(0.05, 2)              ' critical value -> 5.9915
=CHISQ.DIST.RT(19.213, 2)           ' p-value        -> 0.0000672
=CHISQ.TEST(B2:D3, B7:D8)           ' p-value, one step (df inferred)
=IF(CHISQ.TEST(B2:D3,B7:D8)<=0.05, "Reject H0 — associated", "Fail to reject")

' ══ CONDITION CHECK ═════════════════════════════════════════════════
=MIN(B7:D8)>=5                      ' all expected ≥ 5?  -> TRUE
=COUNTIF(B7:D8,"<5")                ' how many small cells

' ══ EFFECT SIZE ═════════════════════════════════════════════════════
=SQRT(F11/E4/MIN(ROWS(B2:D3)-1, COLUMNS(B2:D3)-1))   ' Cramér's V -> 0.3100
=SQRT(F11/E4)                                         ' phi (2×2 only)
=SQRT(F11/(F11+E4))                                   ' contingency coefficient

' ══ BUILDING THE TABLE FROM RAW DATA ════════════════════════════════
' Insert ▸ PivotTable → Rows = exercise, Columns = health, Values = Count
=COUNTIFS($H:$H, $A2, $I:$I, B$1)   ' or COUNTIFS directly

---

R

tab <- matrix(c(45, 55, 20,
                15, 30, 35), nrow = 2, byrow = TRUE,
              dimnames = list(exercise = c("Yes", "No"),
                              health   = c("Excellent", "Good", "Poor")))
addmargins(tab)

# ══ THE TEST ════════════════════════════════════════════════════════
test <- chisq.test(tab)
test
#  Pearson's Chi-squared test
#  X-squared = 19.213, df = 2, p-value = 6.723e-05

test$expected      # the E matrix: 36 51 33 / 24 34 22
test$observed
test$residuals     # (O − E)/√E — signed standardized residuals
test$residuals^2   # the cell contributions
test$stdres        # adjusted residuals; |value| > 2 flags an influential cell

# ══ BY HAND ═════════════════════════════════════════════════════════
E   <- outer(rowSums(tab), colSums(tab)) / sum(tab)
chi <- sum((tab - E)^2 / E);  chi                      # 19.2130
df  <- (nrow(tab) - 1) * (ncol(tab) - 1); df           # 2
qchisq(0.95, df)                                       # 5.9915
pchisq(chi, df, lower.tail = FALSE)                    # 6.723e-05

# ══ EFFECT SIZE ═════════════════════════════════════════════════════
cramers_v <- function(tab) {
  chi <- suppressWarnings(chisq.test(tab)$statistic)
  as.numeric(sqrt(chi / (sum(tab) * min(dim(tab) - 1))))
}
cramers_v(tab)                                         # 0.3100
# library(effectsize);  cramers_v(tab)

# ══ FROM RAW DATA ═══════════════════════════════════════════════════
# tab <- table(df$exercise, df$health)
# chisq.test(tab)

# ══ 2×2 TABLES ══════════════════════════════════════════════════════
small <- matrix(c(8, 2, 3, 7), nrow = 2)
chisq.test(small)                          # Yates correction applied by DEFAULT
chisq.test(small, correct = FALSE)         # uncorrected
fisher.test(small)                         # EXACT — preferred for small counts

# ══ VISUALISING ═════════════════════════════════════════════════════
barplot(prop.table(tab, 1), beside = TRUE,
        col = c("#5B2A86", "#0FA3A3"), ylim = c(0, 0.6),
        legend.text = rownames(tab), args.legend = list(title = "Exercises"),
        main = "Health rating by exercise habit", ylab = "Proportion within row")

mosaicplot(tab, shade = TRUE, main = "Mosaic plot with residual shading")
#   blue  = observed far ABOVE expected
#   red   = observed far BELOW expected

# ══ TEST OF HOMOGENEITY — identical call, different design ══════════
# hospitals <- matrix(c(60,25,15,  55,30,15,  40,35,25), nrow = 3, byrow = TRUE)
# chisq.test(hospitals)

---

Python

import numpy as np
import pandas as pd
from scipy import stats

tab = pd.DataFrame([[45, 55, 20],
                    [15, 30, 35]],
                   index=["Exercises", "Does not"],
                   columns=["Excellent", "Good", "Poor"])

# ══ THE TEST ════════════════════════════════════════════════════════
chi2, p, dof, expected = stats.chi2_contingency(tab)
chi2, dof, p
#  19.2130, 2, 6.7231e-05

pd.DataFrame(expected, index=tab.index, columns=tab.columns)
#            Excellent  Good  Poor
# Exercises       36.0  51.0  33.0
# Does not        24.0  34.0  22.0

# Cell contributions and residuals
contrib = (tab - expected) ** 2 / expected
contrib.round(3)
residuals = (tab - expected) / np.sqrt(expected)     # signed

# ══ BY HAND ═════════════════════════════════════════════════════════
n = tab.values.sum()
E = np.outer(tab.sum(axis=1), tab.sum(axis=0)) / n
chi = ((tab.values - E) ** 2 / E).sum()              # 19.2130
df = (tab.shape[0] - 1) * (tab.shape[1] - 1)         # 2
stats.chi2.ppf(0.95, df)                             # 5.9915
stats.chi2.sf(chi, df)                               # 6.7231e-05

# ══ EFFECT SIZE ═════════════════════════════════════════════════════
def cramers_v(table):
    chi2 = stats.chi2_contingency(table)[0]
    n = np.asarray(table).sum()
    return np.sqrt(chi2 / (n * (min(np.asarray(table).shape) - 1)))

cramers_v(tab)                                       # 0.3100

# ══ FROM RAW DATA ═══════════════════════════════════════════════════
# tab = pd.crosstab(df["exercise"], df["health"])
# stats.chi2_contingency(tab)

# ══ 2×2 TABLES ══════════════════════════════════════════════════════
small = np.array([[8, 2], [3, 7]])
stats.chi2_contingency(small)                        # Yates ON by default
stats.chi2_contingency(small, correction=False)
stats.fisher_exact(small)                            # EXACT

# ══ VISUALISING ═════════════════════════════════════════════════════
import matplotlib.pyplot as plt
row_pct = tab.div(tab.sum(axis=1), axis=0)
row_pct.T.plot(kind="bar", color=["#5B2A86", "#0FA3A3"], rot=0)
plt.ylabel("Proportion within row")
plt.title("Health rating by exercise habit")
plt.show()

# Heatmap of the cell contributions — shows WHERE the association is
fig, ax = plt.subplots()
im = ax.imshow(contrib, cmap="Purples")
ax.set_xticks(range(3), tab.columns); ax.set_yticks(range(2), tab.index)
for i in range(2):
    for j in range(3):
        ax.text(j, i, f"{contrib.iloc[i, j]:.2f}", ha="center", va="center")
fig.colorbar(im); plt.show()

---

Quick Reference

Task	Excel	R	Python
Build the table	PivotTable / COUNTIFS	table(x, y)	pd.crosstab(x, y)
Expected counts	row*col/grand	chisq.test(tab)$expected	chi2_contingency(tab)[3]
χ² statistic	SUMPRODUCT((O-E)^2/E)	chisq.test(tab)$statistic	chi2_contingency(tab)[0]
df	(r-1)*(c-1)	$parameter	chi2_contingency(tab)[2]
Critical value	CHISQ.INV.RT(α, df)	qchisq(1-α, df)	chi2.ppf(1-α, df)
p-value	CHISQ.TEST(O, E)	chisq.test(tab)$p.value	chi2_contingency(tab)[1]
Cell contributions	(O-E)^2/E	$residuals^2	(O-E)**2/E
Cramér's V	SQRT(χ²/(n*min(r-1,c-1)))	effectsize::cramers_v	user function
Small 2×2	—	fisher.test(tab)	stats.fisher_exact(tab)
Mosaic plot	—	mosaicplot(tab, shade=TRUE)	manual heatmap

---

Common Mistakes

• Computing expected counts from the wrong margins. It is row total × column total ÷ grand total — never n/k.
• df = rc − 1. It is (r−1)(c−1).
• Including the totals row/column in the ranges you feed to CHISQ.TEST. Only the interior cells.
• Reporting χ² as an effect size. Use Cramér's V.
• Concluding causation from an association.
• Running the test on percentages, or on a table where one person appears in two cells.
• Using the ordinary chi-square on a 2×2 table with expected counts below 5 — use Fisher's exact test.

---

Exercises: 12-02: Exercises — Chi-Square Test of Independence

---

⬅️ Previous: 12-01: Chi-Square Goodness-of-Fit Test ➡️ Next: 12-03: One-Way ANOVA