11-03: Exercises — Paired t-Test¶
Notes reference: 11-03: Paired t-Test
Q1: Full paired test by hand¶
Eight employees take a typing test before and after a training course (words per minute).
| Employee | Before | After |
|---|---|---|
| 1 | 52 | 58 |
| 2 | 47 | 52 |
| 3 | 61 | 64 |
| 4 | 44 | 51 |
| 5 | 55 | 55 |
| 6 | 49 | 56 |
| 7 | 58 | 61 |
| 8 | 51 | 58 |
Test at α = 0.05 whether the course improves typing speed.
Solution
STEP 1 d = After − Before (so an improvement gives POSITIVE d)
H₀: μ_d ≤ 0 H₁: μ_d > 0 RIGHT-tailed, α = 0.05
STEP 2 Paired design ✓ n = 8 pairs df = 7
Differences look symmetric with no outliers ✓
STEP 3 d values: 6, 5, 3, 7, 0, 7, 3, 7
Σd = 38 d̄ = 38/8 = 4.75
d − d̄: 1.25, 0.25, −1.75, 2.25, −4.75, 2.25, −1.75, 2.25
squares: 1.5625, 0.0625, 3.0625, 5.0625, 22.5625, 5.0625, 3.0625, 5.0625
Σ(d − d̄)² = 45.50
s_d² = 45.50/7 = 6.5000 s_d = 2.54951
SE = 2.54951/√8 = 2.54951/2.82843 = 0.90139
t = 4.75/0.90139 = 5.26963 df = 7
STEP 4 critical t(0.05, 7) = 1.89458
5.26963 > 1.89458 → REJECT H₀
p-value = P(T₇ > 5.26963) = 0.000582 → REJECT H₀
STEP 5 There is very strong evidence at the 5% level that the training
course improves typing speed — by about 4.75 wpm on average.
95% CI for μ_d:
4.75 ± 2.36462(0.90139) = 4.75 ± 2.132 = (2.62, 6.88) wpm
EFFECT SIZE
d = d̄/s_d = 4.75/2.54951 = 1.863 — very large.
' Before in B2:B9, After in C2:C9
=C2-B2 ' D2: differences, fill down
=AVERAGE(D2:D9) ' 4.75
=STDEV.S(D2:D9) ' 2.549510
=AVERAGE(D2:D9)/(STDEV.S(D2:D9)/SQRT(8)) ' t -> 5.269633
=T.DIST.RT(5.269633, 7) ' 0.000582
=T.TEST(C2:C9, B2:B9, 1, 1) ' one tail, PAIRED -> 0.000582
=T.TEST(C2:C9, B2:B9, 2, 1) ' two tails -> 0.001164
' Data ▸ Data Analysis ▸ t-Test: Paired Two Sample for Means
before <- c(52,47,61,44,55,49,58,51)
after <- c(58,52,64,51,55,56,61,58)
t.test(after, before, paired = TRUE, alternative = "greater")
# t = 5.2696, df = 7, p-value = 0.0005818
t.test(after, before, paired = TRUE)$conf.int # 2.6182 6.8818
d <- after - before
c(mean = mean(d), sd = sd(d), se = sd(d)/sqrt(8))
before = np.array([52,47,61,44,55,49,58,51])
after = np.array([58,52,64,51,55,56,61,58])
stats.ttest_rel(after, before, alternative="greater")
res = stats.ttest_rel(after, before)
res.confidence_interval(0.95)
Q2: The cost of ignoring the pairing¶
Run an independent two-sample test on the Q1 data and compare.
Solution
INDEPENDENT (WRONG for this design)
Before: x̄₁ = 52.125, s₁ = 5.6679, n₁ = 8
After: x̄₂ = 56.875, s₂ = 4.3568, n₂ = 8
s²_p = [7(32.1250) + 7(18.9821)]/14 = (224.875 + 132.875)/14 = 25.5536
SE = √(25.5536 × 0.25) = √6.38839 = 2.52753
t = (56.875 − 52.125)/2.52753 = 4.75/2.52753 = 1.87932 df = 14
critical t(0.05, 14) = 1.76131
p (one-tailed) = 0.040733 → barely rejects at α = 0.05
PAIRED (CORRECT)
t = 5.26963, df = 7, p = 0.000582 → rejects overwhelmingly
WHY THE HUGE DIFFERENCE
The independent test's standard error is 2.528; the paired test's is
0.901 — nearly THREE TIMES smaller.
Between-employee variation is large (s ≈ 5 wpm: some people simply
type faster than others). The paired test CANCELS that variation by
working with each person's own change. The independent test leaves
it in the noise, where it buries a very consistent effect.
Note that EVERY employee improved or stayed level — the pairing makes
that pattern visible; the independent test throws it away.
THE COST OF PAIRING: df drops from 14 to 7. When the pairing is real,
that loss is trivial compared with the drop in s.
t.test(after, before) # independent — the wrong test here
cor(before, after) # 0.90 — strong pairing
Q3: Direction of subtraction¶
Redo Q1 with d = Before − After. What changes?
Solution
d values: −6, −5, −3, −7, 0, −7, −3, −7
d̄ = −4.75 (sign flipped)
s_d = 2.54951 (UNCHANGED — spread does not care about sign)
t = −5.26963 (sign flipped)
THE HYPOTHESES MUST FLIP TOO:
With d = After − Before, "improvement" is H₁: μ_d > 0
With d = Before − After, "improvement" is H₁: μ_d < 0
Same p-value (0.000582), same conclusion — PROVIDED you flip the
alternative to match.
THE TRAP: subtracting in one direction and then testing "greater than"
out of habit gives p = 0.99942 and the exact opposite conclusion.
Write down the direction FIRST, in words, before touching the data.
For a TWO-TAILED test the direction does not matter at all — only the
sign of t changes, and |t| is what gets compared.
Q4: A paired test with no effect¶
Ten subjects' cholesterol is measured before and after a diet.
| Subject | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 |
|---|---|---|---|---|---|---|---|---|---|---|
| Before | 210 | 195 | 224 | 188 | 232 | 201 | 215 | 199 | 228 | 205 |
| After | 205 | 198 | 219 | 191 | 226 | 204 | 211 | 196 | 224 | 208 |
Test at α = 0.05 (two-tailed).
Solution
d = Before − After: 5, −3, 5, −3, 6, −3, 4, 3, 4, −3
Σd = 15 d̄ = 1.5
d − d̄: 3.5, −4.5, 3.5, −4.5, 4.5, −4.5, 2.5, 1.5, 2.5, −4.5
squares: 12.25, 20.25, 12.25, 20.25, 20.25, 20.25, 6.25, 2.25, 6.25, 20.25
Σ(d − d̄)² = 140.50
s_d² = 140.50/9 = 15.6111 s_d = 3.95109
SE = 3.95109/√10 = 3.95109/3.16228 = 1.24944
t = 1.5/1.24944 = 1.20054 df = 9
critical t(0.025, 9) = ±2.26216
|1.20054| < 2.26216 → FAIL TO REJECT H₀
p = 2 × P(T₉ > 1.20054) = 2(0.130342) = 0.260684
CONCLUSION
There is not sufficient evidence at the 5% level that the diet
changes mean cholesterol.
95% CI for μ_d: 1.5 ± 2.26216(1.24944) = 1.5 ± 2.826 = (−1.33, 4.33)
Contains 0 — consistent with the test.
WHY THE PAIRING DIDN'T RESCUE THIS ONE
Look at the differences: 5, −3, 5, −3, 6, −3, 4, 3, 4, −3.
They are INCONSISTENT — four subjects got worse. Pairing removes
between-subject variation, but it cannot manufacture an effect that
is not there. That is exactly what a good test should do.
Q5: Checking the assumption¶
For a paired test, whose normality do you check — the before values, the after values, or the differences?
Solution
THE DIFFERENCES. Only the differences.
The original measurements can be any shape at all. The paired t-test is
a ONE-SAMPLE t-test applied to d, so the one-sample assumption applies
to d and nothing else.
d <- after - before
par(mfrow = c(1, 3))
hist(d, breaks = 6, col = "#8A5FBF", border = "white", main = "Differences")
boxplot(d, col = "#0FA3A3", main = "Boxplot of d"); abline(h = 0, lty = 2)
qqnorm(d, pch = 19); qqline(d, col = "#0FA3A3", lwd = 2)
par(mfrow = c(1, 1))
shapiro.test(d) # H0: d is normal
d = after - before
stats.shapiro(d)
fig, axes = plt.subplots(1, 3, figsize=(12, 3))
axes[0].hist(d, bins=6, color="#8A5FBF", edgecolor="white")
axes[1].boxplot(d); axes[1].axhline(0, ls="--", color="grey")
stats.probplot(d, dist="norm", plot=axes[2])
=SKEW(D2:D9) ' skewness of the DIFFERENCES
' Q-Q plot: sort d into column E, then F2: =NORM.S.INV((ROW()-1.5)/n)
' scatter E against F
If the differences are badly non-normal: use the Wilcoxon signed-rank test (wilcox.test(x, y, paired = TRUE) / stats.wilcoxon(x, y)), which tests the median difference and needs only symmetry.
Q6: Choose the design¶
For each research question, say whether a paired design is possible and whether it would be better.
- Compare two teaching methods across a school district.
- Compare two brands of tyre for wear.
- Compare salaries of two departments.
- Compare a drug's effect on blood pressure.
- Compare two soil treatments across a field with a fertility gradient.
Solution
1. PAIRED possible via MATCHED PAIRS: pair students by prior GPA and
randomly assign one of each pair to each method. Better if prior GPA
strongly predicts outcome — which it does.
2. PAIRED, and clearly better: put ONE tyre of each brand on the SAME
car (and rotate positions). This removes driver, route, and vehicle
effects, which are enormous. Independent samples would need hundreds
of cars to see the same effect.
3. NOT PAIRABLE. Employees belong to one department only; there is no
natural pairing. Use an independent two-sample test.
4. PAIRED, and standard: measure each patient before and after. Removes
the huge between-patient variation in baseline blood pressure.
(A placebo control group is still needed to separate the drug from
regression to the mean.)
5. PAIRED via BLOCKING: split each plot in half and apply one treatment
to each half. The fertility gradient affects both halves equally,
so it cancels. This is a classic agricultural split-plot design.
THE GENERAL RULE: pair whenever the units differ a lot from one another
on something that also affects the outcome. The bigger the between-unit
variation, the bigger the payoff from pairing.
Q7: Interpret software output¶
Paired t-test
data: post and pre
t = 3.412, df = 23, p-value = 0.002374
alternative hypothesis: true mean difference is not equal to 0
95 percent confidence interval:
1.8531 7.4802
sample estimates:
mean difference
4.66665
- How many pairs?
- Decide at
α = 0.05and atα = 0.01. - What is
s_d? - Write the one-sentence conclusion.
Solution
1. df = n − 1 = 23 → n = 24 PAIRS (48 measurements)
2. p = 0.002374 ≤ 0.05 → REJECT
p = 0.002374 ≤ 0.01 → REJECT at the 1% level too
3. SE = d̄ / t = 4.66665 / 3.412 = 1.36771
s_d = SE × √n = 1.36771 × √24 = 1.36771 × 4.89898 = 6.7003
Cross-check with the CI:
E = (7.4802 − 1.8531)/2 = 2.81355
t(0.025, 23) = 2.06866
SE = 2.81355/2.06866 = 1.36008 ✓ agrees to rounding
4. "The post-treatment mean exceeded the pre-treatment mean by 4.67
units on average, a statistically significant increase,
t(23) = 3.41, p = .002, 95% CI [1.85, 7.48]."
Note that the interval EXCLUDES 0 and is entirely positive, so both
the significance and the direction are established. The plausible
range 1.85 to 7.48 is what tells the reader whether the effect is
big enough to matter.
Q8: Paired proportions¶
Sixty voters are asked their opinion before and after a debate. 12 switched from oppose to support; 5 switched from support to oppose; 43 did not change. Did the debate shift opinion?
Solution
This is PAIRED CATEGORICAL data — the paired t-test does not apply.
The correct tool is McNEMAR'S TEST, the paired analogue of the
chi-square test.
AFTER
Support Oppose
Support a=20 b=5
BEFORE Oppose c=12 d=23
Only the DISCORDANT pairs (b and c) carry information about change;
the 43 who did not switch tell you nothing about the direction.
(|b − c| − 1)² (|5 − 12| − 1)² 36
McNemar χ² = ───────────────── = ──────────────── = ────── = 2.1176
b + c 5 + 12 17
(with continuity correction)
df = 1, critical χ²(0.05, 1) = 3.8415
2.1176 < 3.8415 → FAIL TO REJECT H₀
p-value = 0.1456
CONCLUSION: there is not sufficient evidence at the 5% level that the
debate shifted opinion. The 12-vs-5 split leans toward support, but
with only 17 switchers it is within chance variation.
WHY NOT AN ORDINARY CHI-SQUARE? Because the same 60 people appear in
both the "before" and "after" margins. Treating them as two independent
samples of 60 would double-count and inflate the evidence.
tab <- matrix(c(20, 5, 12, 23), nrow = 2, byrow = TRUE,
dimnames = list(before = c("Support","Oppose"),
after = c("Support","Oppose")))
mcnemar.test(tab) # with continuity correction
mcnemar.test(tab, correct = FALSE) # without
binom.test(5, 17, p = 0.5) # exact version, on the discordant pairs
from statsmodels.stats.contingency_tables import mcnemar
mcnemar([[20, 5], [12, 23]], exact=False, correction=True)
mcnemar([[20, 5], [12, 23]], exact=True) # exact binomial version
=(ABS(5-12)-1)^2/(5+12) ' 2.117647
=CHISQ.DIST.RT(2.117647, 1) ' 0.145610
=CHISQ.INV.RT(0.05, 1) ' 3.841459
⬅️ Previous: 11-02: Exercises — Two-Sample t-Test ➡️ Next: 12-01: Exercises — Chi-Square Goodness-of-Fit Test