11-02: Exercises — Two-Sample t-Test¶
Notes reference: 11-02: Two-Sample t-Test
Q1: Independent or paired?¶
Classify each design.
- Test scores of 30 students taught by method A vs. 30 different students taught by method B.
- Blood pressure of 25 patients before and after a drug.
- Left-hand vs. right-hand grip strength of 40 people.
- Salaries of 50 men vs. 50 women at a company.
- Yield of 20 plots given fertilizer X vs. 20 other plots given fertilizer Y.
- Reaction time of 15 people measured with and without caffeine.
Solution
1. INDEPENDENT — two different sets of students
2. PAIRED — the same 25 patients measured twice
3. PAIRED — two measurements on the SAME person
4. INDEPENDENT — two different sets of people
5. INDEPENDENT — different plots
6. PAIRED — the same 15 people under two conditions
THE TEST: can you write each row of the data as ONE subject with TWO
numbers? If yes, it is paired (11-03). If the two groups have different
members — and possibly different sizes — it is independent.
Q2: Pooled vs. Welch — from summary statistics¶
Group 1: n₁ = 15, x̄₁ = 84.2, s₁ = 7.1. Group 2: n₂ = 18, x̄₂ = 78.6, s₂ = 9.8. Test H₀: μ₁ = μ₂ at α = 0.05, two-tailed.
Do it both ways.
Solution
VARIANCE CHECK: s₂²/s₁² = 96.04/50.41 = 1.905 < 4 → either test is defensible
── POOLED ─────────────────────────────────────────────────────────────
s²_p = [14(50.41) + 17(96.04)] / (15 + 18 − 2)
= [705.74 + 1632.68] / 31
= 2338.42 / 31
= 75.4329
SE = √(75.4329 × (1/15 + 1/18)) = √(75.4329 × 0.122222) = √9.21958 = 3.03638
t = (84.2 − 78.6)/3.03638 = 5.6/3.03638 = 1.84427 df = 31
critical t(0.025, 31) = 2.03951
|1.84427| < 2.03951 → FAIL TO REJECT H₀
p = 2 × P(T₃₁ > 1.84427) = 2(0.037416) = 0.074832
── WELCH ──────────────────────────────────────────────────────────────
SE = √(50.41/15 + 96.04/18) = √(3.36067 + 5.33556) = √8.69622 = 2.94893
t = 5.6/2.94893 = 1.89898
df = (8.69622)² / [ (3.36067)²/14 + (5.33556)²/17 ]
= 75.62425 / [ 0.806664 + 1.674642 ]
= 75.62425 / 2.481306
= 30.4776
critical t(0.025, 30.48) ≈ 2.04165
|1.89898| < 2.04165 → FAIL TO REJECT H₀
p = 2 × P(T₃₀.₄₈ > 1.89898) ≈ 0.067
── CONCLUSION ─────────────────────────────────────────────────────────
Both tests agree: there is NOT sufficient evidence at the 5% level of a
difference in means. The 5.6-point gap could plausibly be chance.
95% CI (Welch): 5.6 ± 2.04165(2.94893) = 5.6 ± 6.021 = (−0.42, 11.62)
The interval contains 0 — consistent with the test, and it shows the
true difference could be anywhere from slightly negative to 11.6 points.
' n1,x̄1,s1 in D1:D3 ; n2,x̄2,s2 in E1:E3
=((D1-1)*D3^2+(E1-1)*E3^2)/(D1+E1-2) ' s²_p -> 75.4329
=SQRT(G1*(1/D1+1/E1)) ' SE -> 3.03638
=(D2-E2)/G2 ' t -> 1.84427
=T.DIST.2T(ABS(G3), D1+E1-2) ' p -> 0.074832
=SQRT(D3^2/D1+E3^2/E1) ' Welch SE-> 2.94893
=(D3^2/D1+E3^2/E1)^2/((D3^2/D1)^2/(D1-1)+(E3^2/E1)^2/(E1-1)) ' df -> 30.478
stats.ttest_ind_from_stats(84.2, 7.1, 15, 78.6, 9.8, 18, equal_var=True)
# t = 1.8443, p = 0.0748
stats.ttest_ind_from_stats(84.2, 7.1, 15, 78.6, 9.8, 18, equal_var=False)
# t = 1.8990, p = 0.0672
Q3: From raw data¶
Two production lines are compared on cycle time (seconds).
Test at α = 0.05 whether the mean cycle times differ.
Solution
GROUP SUMMARIES
A: Σx = 432, x̄₁ = 43.20, Σ(x−x̄)² = 63.60, s₁² = 7.0667, s₁ = 2.6583
B: Σx = 591, x̄₂ = 49.25, Σ(x−x̄)² = 70.25, s₂² = 6.3864, s₂ = 2.5272
VARIANCE CHECK: 7.0667/6.3864 = 1.11 → variances are nearly identical
→ the POOLED test is clearly appropriate
POOLED
s²_p = [9(7.0667) + 11(6.3864)] / 20 = [63.60 + 70.25]/20 = 133.85/20 = 6.6925
SE = √(6.6925 × (1/10 + 1/12)) = √(6.6925 × 0.183333) = √1.22696 = 1.10768
t = (43.2 − 49.25)/1.10768 = −6.05/1.10768 = −5.46187 df = 20
critical t(0.025, 20) = 2.08596
|−5.46187| > 2.08596 → REJECT H₀
p = 2 × P(T₂₀ < −5.46187) = 0.0000234
CONCLUSION
There is very strong evidence at the 5% level that the two lines
differ in mean cycle time. Line A is faster by about 6.05 seconds.
95% CI for μ₁ − μ₂:
−6.05 ± 2.08596(1.10768) = −6.05 ± 2.311 = (−8.361, −3.739)
Entirely below 0, and Line A's advantage is between 3.7 and 8.4 seconds.
EFFECT SIZE
d = (43.2 − 49.25)/√6.6925 = −6.05/2.58699 = −2.339 — enormous.
' Line A in A2:A11, Line B in B2:B13
' Data ▸ Data Analysis ▸ t-Test: Two-Sample Assuming Equal Variances
' Variable 1 Range = A1:A11, Variable 2 Range = B1:B13, tick Labels
=T.TEST(A2:A11, B2:B13, 2, 2) ' two tails, pooled -> 0.0000234
=T.TEST(A2:A11, B2:B13, 2, 3) ' two tails, Welch -> 0.0000262
=F.TEST(A2:A11, B2:B13) ' equal-variance check -> 0.86 (no evidence
' the variances differ)
A <- c(42,45,39,47,44,41,46,43,40,45)
B <- c(48,52,46,50,53,47,49,51,45,50,48,52)
t.test(A, B, var.equal = TRUE)
# t = -5.4619, df = 20, p-value = 2.344e-05
# 95 percent confidence interval: -8.3606 -3.7394
t.test(A, B) # Welch — same conclusion
var.test(A, B) # F-test for equal variances
boxplot(list(A = A, B = B), col = c("#5B2A86", "#0FA3A3"))
A = np.array([42,45,39,47,44,41,46,43,40,45])
B = np.array([48,52,46,50,53,47,49,51,45,50,48,52])
stats.ttest_ind(A, B, equal_var=True)
stats.levene(A, B)
Q4: One-tailed two-sample test¶
A new fertilizer is claimed to increase yield. Treated plots: n₁ = 12, x̄₁ = 58.4, s₁ = 6.2. Control plots: n₂ = 12, x̄₂ = 53.1, s₂ = 5.8. Test at α = 0.05.
Solution
STEP 1 H₀: μ₁ ≤ μ₂ H₁: μ₁ > μ₂ RIGHT-tailed, α = 0.05
STEP 2 Variance ratio 38.44/33.64 = 1.14 < 4 → pooled test is fine
Equal sample sizes make the pooled and Welch results nearly identical.
STEP 3 s²_p = [11(38.44) + 11(33.64)] / 22 = [422.84 + 370.04]/22
= 792.88/22 = 36.04
SE = √(36.04 × (1/12 + 1/12)) = √(36.04 × 0.166667) = √6.00667 = 2.45085
t = (58.4 − 53.1)/2.45085 = 5.3/2.45085 = 2.16249 df = 22
STEP 4 critical t(0.05, 22) = 1.71714
2.16249 > 1.71714 → REJECT H₀
p = P(T₂₂ > 2.16249) = 0.020852 → REJECT H₀
STEP 5 There is sufficient evidence at the 5% level that the new
fertilizer increases mean yield.
EFFECT SIZE d = 5.3/√36.04 = 5.3/6.00333 = 0.883 — large.
NOTE: had this been TWO-tailed, p = 0.0417 — still significant, but the
one-tailed choice must have been made in advance from agronomic theory,
not after seeing that yields went up.
Q5: Testing equal variances¶
Sample 1: n₁ = 16, s₁ = 12.4. Sample 2: n₂ = 21, s₂ = 6.1. Test H₀: σ₁² = σ₂² at α = 0.05.
Solution
F = larger s² / smaller s² = 12.4²/6.1² = 153.76/37.21 = 4.13223
df₁ = 16 − 1 = 15 (numerator, the LARGER variance)
df₂ = 21 − 1 = 20 (denominator)
Two-tailed at α = 0.05 → compare with F(0.025, 15, 20) = 2.5731
4.13223 > 2.5731 → REJECT H₀
p-value = 2 × P(F₁₅,₂₀ > 4.13223) = 2(0.001723) = 0.003446
CONCLUSION: the variances differ significantly.
→ USE WELCH'S t-TEST, not the pooled version.
Also note the raw ratio: 153.76/37.21 = 4.13, which fails the informal
"largest variance < 4× smallest" rule as well.
=12.4^2/6.1^2 ' 4.13223
=F.INV.RT(0.025, 15, 20) ' 2.57306
=2*F.DIST.RT(4.13223, 15, 20) ' 0.003446
=F.TEST(range1, range2) ' two-tailed p-value, straight from raw data
var.test(x, y) # F test for equal variances
library(car); leveneTest(value ~ group, data = df) # robust alternative
bartlett.test(value ~ group, data = df)
Warning
The F-test is very sensitive to non-normality — a heavy-tailed but equal-variance pair can fail it. Levene's test is the robust choice in practice, and the simplest answer of all is to just use Welch's t-test by default and skip the variance test entirely.
Q6: Read a ToolPak output¶
t-Test: Two-Sample Assuming Unequal Variances
Group A Group B
Mean 72.4 66.8
Variance 84.3 142.7
Observations 25 22
Hypothesized Mean Diff 0
df 38
t Stat 1.8143
P(T<=t) one-tail 0.03874
t Critical one-tail 1.68595
P(T<=t) two-tail 0.07748
t Critical two-tail 2.02439
- Which test was run and why?
- Decide at
α = 0.05, two-tailed. - Decide at
α = 0.05, one-tailed (right). - Why is
df = 38rather than 45? - Compute the 95% CI for the difference.
Solution
1. WELCH'S t-test ("Assuming Unequal Variances"). Appropriate here:
142.7/84.3 = 1.69 is not extreme, but Welch costs nothing and the
sample sizes are unequal.
2. Two-tailed: p = 0.07748 > 0.05 → FAIL TO REJECT H₀.
No significant difference in means.
3. One-tailed right: p = 0.03874 ≤ 0.05 → REJECT H₀.
BUT this is only legitimate if the direction was specified in advance.
4. df = 38 is the WELCH-SATTERTHWAITE approximation, not n₁ + n₂ − 2 = 45.
Unequal variances "cost" degrees of freedom; Welch's df is always
between min(n₁−1, n₂−1) = 21 and n₁+n₂−2 = 45.
5. From t Stat and the mean difference:
difference = 72.4 − 66.8 = 5.6
SE = 5.6 / 1.8143 = 3.08653
95% CI = 5.6 ± 2.02439(3.08653) = 5.6 ± 6.249 = (−0.65, 11.85)
The interval CONTAINS 0 — consistent with the two-tailed decision,
and it makes the ambiguity explicit: the true difference could be
anywhere from −0.6 to +11.9.
Q7: When the assumptions fail¶
Two groups, n₁ = 12 and n₂ = 14, both strongly right-skewed with a couple of outliers. What do you do?
Solution
OPTION 1 — TRANSFORM
log(x) often symmetrizes right-skewed positive data.
t.test(log(A), log(B))
Interpretation shifts to a RATIO of geometric means, not a difference.
OPTION 2 — MANN-WHITNEY U (Wilcoxon rank-sum)
Nonparametric, based on ranks. Tests whether one group tends to
produce larger values. Assumes only that the two distributions have
similar SHAPE (for a location interpretation).
wilcox.test(A, B)
stats.mannwhitneyu(A, B)
OPTION 3 — PERMUTATION TEST
Assumes only exchangeability under H₀. Pool the data, reshuffle the
group labels many times, and see how often the reshuffled difference
exceeds the observed one.
OPTION 4 — BOOTSTRAP the difference in means
Resample each group with replacement, build a percentile CI.
WHAT TO REPORT
Whichever you choose, show the boxplot and say WHY you departed from
the t-test. Running the t-test anyway on 12 skewed observations with
outliers is the one option that is not defensible.
set.seed(2)
# Permutation test
obs <- mean(A) - mean(B)
pool <- c(A, B); nA <- length(A)
perm <- replicate(10000, {
idx <- sample(length(pool), nA)
mean(pool[idx]) - mean(pool[-idx])
})
mean(abs(perm) >= abs(obs)) # two-tailed permutation p-value
wilcox.test(A, B) # Mann-Whitney U
Q8: Power and sample size¶
You expect a difference of 4 units with σ ≈ 8. How many per group do you need for 80% power at α = 0.05, two-tailed?
Solution
Effect size: d = 4/8 = 0.5 (a medium effect)
APPROXIMATE FORMULA
2 (z_{α/2} + z_β)² 2 (1.96 + 0.8416)²
n per group ≈ ────────────────── = ────────────────────
d² 0.25
= 2 (7.849) / 0.25 = 62.8 → 63 per group
EXACT (accounting for the t distribution)
power.t.test(delta = 4, sd = 8, sig.level = 0.05, power = 0.80)
→ n = 63.77 → 64 per group, 128 total
WHAT IF YOU ONLY HAVE 30 PER GROUP?
power.t.test(n = 30, delta = 4, sd = 8, sig.level = 0.05)$power
→ power = 0.478
Under 50%. You would miss a REAL 4-unit difference more often than
you would find it — a coin flip dressed up as a study.
THE LESSON: run the power calculation BEFORE collecting data.
Computing power after a non-significant result ("observed power") adds
nothing — it is just a re-expression of the p-value.
power.t.test(delta = 4, sd = 8, sig.level = 0.05, power = 0.80) # n = 63.77
power.t.test(n = 30, delta = 4, sd = 8, sig.level = 0.05)$power # 0.478
from statsmodels.stats.power import TTestIndPower
TTestIndPower().solve_power(effect_size=0.5, power=0.80, alpha=0.05) # ≈ 63.8
TTestIndPower().power(effect_size=0.5, nobs1=30, alpha=0.05) # ≈ 0.478
⬅️ Previous: 11-01: Exercises — One-Sample t-Test ➡️ Next: 11-03: Exercises — Paired t-Test