04-09: Hypothesis Testing Toolkit¶

Home | All Projects | Notes | Exercises | Quiz Hub

One project, four tests: a one-sample test against a target, an independent two-sample comparison, a paired before/after study, and a proportion test. Every one reported the same way — statistic, df, p-value, effect size, confidence interval, and a plain-English conclusion.

Chapters applied: 10-01 · 10-02 · 11-01 · 11-02 · 11-03

Difficulty: ⭐⭐⭐ Intermediate–Advanced

The Data¶

File	Design	Test
`data/fill_volume.csv`	30 bottles, target 250 ml	One-sample t
`data/two_methods.csv`	40 participants on method A, 38 on method B	Two-sample t (independent)
`data/before_after.csv`	24 subjects, reaction time before and after	Paired t
`data/poll_responses.csv` (from 04-08)	600 Yes/No	z-test for a proportion

What You Produce¶

For each of the four tests:

H₀ and H₁ written in symbols, with the tail justified before looking at the data
The conditions checked — with the boxplot or Q-Q plot that justifies the claim
The test statistic, df, critical value, and p-value
The decision and the conclusion in the context of the problem
An effect size (Cohen's d) and a confidence interval
The nonparametric alternative, run as a sensitivity check

Excel Route¶

Test 1 — One-sample t (fill volume vs. 250 ml)¶

' H0: mu = 250     H1: mu != 250     two-tailed, alpha = 0.05
=COUNT(A2:A31)                                       ' n  = 30
=AVERAGE(A2:A31)                                     ' x-bar
=STDEV.S(A2:A31)                                     ' s
=STDEV.S(A2:A31)/SQRT(COUNT(A2:A31))                 ' SE
=(AVERAGE(A2:A31)-250)/$E$4                          ' t statistic
=COUNT(A2:A31)-1                                     ' df = 29
=T.INV.2T(0.05, $E$6)                                ' critical t = 2.045
=T.DIST.2T(ABS($E$5), $E$6)                          ' p-value
=IF($E$8<=0.05,"Reject H0","Fail to reject H0")

' effect size and interval
=(AVERAGE(A2:A31)-250)/STDEV.S(A2:A31)               ' Cohen's d
=AVERAGE(A2:A31)-T.INV.2T(0.05,$E$6)*$E$4            ' CI lower
=AVERAGE(A2:A31)+T.INV.2T(0.05,$E$6)*$E$4            ' CI upper

Excel has no one-sample t dialog. The standard workaround: put a column of 250s beside the data and run =T.TEST(A2:A31, C2:C31, 2, 1) — a paired test against a constant is exactly the one-sample test.

Test 2 — Two-sample t (method A vs. method B)¶

Split the scores into two columns first (FILTER or sort by method).

' Data ▸ Data Analysis ▸ t-Test: Two-Sample Assuming Unequal Variances (Welch)
'   Variable 1 Range = A1:A41, Variable 2 Range = B1:B39, tick Labels
'   -> means, variances, df, t Stat, one- and two-tail p and critical values

' First check whether the variances are close enough for the pooled test:
=F.TEST(A2:A41, B2:B39)                              ' two-tailed p for equal variances
=MAX(VAR.S(A2:A41),VAR.S(B2:B39))/MIN(VAR.S(A2:A41),VAR.S(B2:B39))   ' want < 4

=T.TEST(A2:A41, B2:B39, 2, 3)                        ' Welch  (type 3)
=T.TEST(A2:A41, B2:B39, 2, 2)                        ' pooled (type 2)

' Cohen's d, pooled
=((COUNT(A2:A41)-1)*VAR.S(A2:A41)+(COUNT(B2:B39)-1)*VAR.S(B2:B39))/(COUNT(A2:A41)+COUNT(B2:B39)-2)
=(AVERAGE(A2:A41)-AVERAGE(B2:B39))/SQRT($H$1)

Test 3 — Paired t (before vs. after)¶

=B2-C2                                               ' D2: differences, fill down
=AVERAGE(D2:D25)                                     ' d-bar
=STDEV.S(D2:D25)                                     ' s_d
=AVERAGE(D2:D25)/(STDEV.S(D2:D25)/SQRT(24))          ' t
=T.DIST.RT($G$4, 23)                                 ' one-tailed p (improvement)
=T.TEST(B2:B25, C2:C25, 1, 1)                        ' same, in one call (type 1 = paired)

' Data ▸ Data Analysis ▸ t-Test: Paired Two Sample for Means
' Then run the WRONG test to see the cost:
=T.TEST(B2:B25, C2:C25, 1, 3)                        ' independent — ignores the pairing

Compare the two p-values. The paired test is dramatically stronger, because it removes the huge between-subject variation in baseline reaction time.

Test 4 — z-test for a proportion¶

' H0: p = 0.50   H1: p != 0.50
=COUNTIF(B2:B601,"Yes")                              ' x
=COUNTA(B2:B601)                                     ' n
=$J$1/$J$2                                           ' p-hat
=SQRT(0.5*0.5/$J$2)                                  ' SE uses p0 = 0.5, NOT p-hat
=($J$3-0.5)/$J$4                                     ' z
=2*NORM.S.DIST(-ABS($J$5), TRUE)                     ' two-tailed p
=AND($J$2*0.5>=5, $J$2*0.5>=5)                       ' condition check

R Route¶

Rscript r/analysis.R

Runs all four tests with assumption checks, effect sizes, intervals, and nonparametric alternatives, and writes testing_plots.png. See r/analysis.R.

Python Route¶

python python/analysis.py

Same via scipy.stats. See python/analysis.py.

Checkpoints¶

Every test states H₀/H₁ before the computation, with the tail justified
Assumptions are checked with a plot, not asserted
The proportion test uses p₀ in the standard error; the interval uses p̂
Every result reports statistic, df, p, effect size, and a confidence interval
The paired data is analysed as paired, and the cost of ignoring that is shown
Each conclusion is written in the language of the problem, not just "reject H₀"

Extend It¶

Add a power analysis for each test: what effect could this sample size actually detect at 80% power?
Add a two-proportion test comparing support between two subgroups
Run all four tests at α = 0.01 and note which conclusions change
Add a bootstrap confidence interval for the mean difference and compare with the t-interval