08-02: Sampling Distributions and the Central Limit Theorem¶
This is the pivot of the entire course. Everything before it describes data; everything after it makes claims about populations. The bridge is one idea: a sample statistic is itself a random variable, and it has its own distribution.
What a Sampling Distribution Is¶
Take ALL possible samples of size n from a population.
Compute the same statistic (say x̄) for each one.
The distribution of those values is the SAMPLING DISTRIBUTION of x̄.
Three distributions get confused constantly — keep them separate:
| Distribution | What varies | Spread |
|---|---|---|
| Population | Individual values in the population | σ |
| Sample | Individual values in one sample | s |
Sampling distribution of x̄ |
The mean, across repeated samples | σ/√n — the standard error |
The Sampling Distribution of the Sample Mean¶
Two facts worth stating plainly:
x̄is an unbiased estimator: across repeated samples it centres exactly onμ.- The standard error shrinks as
√ngrows. Quadruple the sample, halve the standard error.
Finite population correction¶
When sampling without replacement from a small population (n > 0.05N):
Usually ignored, because in practice n is a tiny fraction of N.
The Central Limit Theorem¶
For a sufficiently large n, the sampling distribution of x̄ is approximately
NORMAL, with mean μ and standard error σ/√n,
REGARDLESS of the shape of the original population.
| Population shape | Sample size needed |
|---|---|
| Already normal | any n — exactly normal |
| Roughly symmetric | n ≥ 15 is usually plenty |
| Skewed or unknown | n ≥ 30 is the standard rule of thumb |
| Extremely skewed / heavy-tailed | may need n > 50 |
Note
The CLT is why the normal distribution is everywhere. You almost never assume the data is normal; you rely on the CLT to make the sample mean normal. That is what makes z-tests, t-tests, and confidence intervals work on messy real data.
The z-score for a sample mean¶
Forgetting the √n is the most common single error in Chapters 08–10. The sample mean varies less than an individual value, so its z-score is larger for the same distance from μ.
Worked Example¶
A population of package weights has μ = 12 oz and σ = 0.8 oz (shape unknown).
(a) P(one package weighs more than 12.2 oz) — needs a normality assumption about the population:
(b) P(the mean of 36 packages exceeds 12.2 oz) — the CLT applies since n = 36 ≥ 30:
σ_x̄ = 0.8 / √36 = 0.8 / 6 = 0.1333
z = (12.2 − 12) / 0.1333 = 1.50
P(Z > 1.50) = 1 − 0.9332 = 0.0668 → 6.7%
Why the huge difference? One package easily runs 0.2 oz heavy. The average of 36 running 0.2 oz heavy is a much stronger signal — averaging cancels the individual noise. This is exactly the logic of every hypothesis test that follows.
(c) The middle 95% of sample means for n = 36:
The Sampling Distribution of a Sample Proportion¶
The same story for categorical data. With p̂ = x/n:
Mean μ_p̂ = p
______________
Standard error σ_p̂ = √ p(1 − p) / n
Approximately normal when n·p ≥ 5 AND n(1 − p) ≥ 5
(some texts use 10)
Example. 40% of voters support a measure; sample 200. P(p̂ > 0.45)?
np = 80 ≥ 5, n(1−p) = 120 ≥ 5 ✓
σ_p̂ = √(0.40 × 0.60 / 200) = √0.0012 = 0.03464
z = (0.45 − 0.40) / 0.03464 = 1.44
P(Z > 1.44) = 1 − 0.9251 = 0.0749 → 7.5%
Other Sampling Distributions Used Later¶
| Statistic | Sampling distribution | Where |
|---|---|---|
x̄ with σ known |
Normal z |
10-02 |
x̄ with σ unknown |
t with n − 1 df |
11-01 |
p̂ |
Normal z |
09-02 |
s² |
Chi-square with n − 1 df |
12-01 |
s₁²/s₂² |
F with n₁−1, n₂−1 df |
12-03 |
Excel¶
' ── Standard error ──────────────────────────────────────────────────
=B2/SQRT(B3) ' σ/√n (B2 = σ, B3 = n) -> 0.13333
=STDEV.S(A2:A37)/SQRT(COUNT(A2:A37)) ' estimated SE from a sample
' ── Probability for a SAMPLE MEAN ───────────────────────────────────
=1-NORM.DIST(12.2, 12, 0.8/SQRT(36), TRUE) ' P(x̄ > 12.2) -> 0.06681
=NORM.DIST(12.2, 12, 0.8, TRUE) ' compare: one package
=(12.2-12)/(0.8/SQRT(36)) ' the z-score -> 1.5
=1-NORM.S.DIST(1.5, TRUE) ' -> 0.06681
' ── Middle 95% of sample means ──────────────────────────────────────
=NORM.INV(0.025, 12, 0.8/SQRT(36)) ' -> 11.7387
=NORM.INV(0.975, 12, 0.8/SQRT(36)) ' -> 12.2613
' ── Sample proportion ───────────────────────────────────────────────
=SQRT(0.4*0.6/200) ' SE of p̂ -> 0.034641
=1-NORM.DIST(0.45, 0.4, SQRT(0.4*0.6/200), TRUE) ' P(p̂ > 0.45) -> 0.07477
' ── Simulating the CLT (a genuinely convincing demo) ────────────────
' 1. Column A: 1000 values from a SKEWED population, e.g.
=-LN(RAND())*10 ' exponential with mean 10
' 2. Columns C..AF: 30 more such columns (each row = one sample of 30)
' 3. Column AH: =AVERAGE(C2:AF2) the mean of each sample
' 4. Histogram column A -> strongly right-skewed
' Histogram column AH -> visibly bell-shaped, and much narrower
=STDEV.S(AH2:AH1001) ' ≈ 10/√30 = 1.83
R¶
set.seed(42)
# ── Direct calculations ────────────────────────────────────────────
mu <- 12; sigma <- 0.8; n <- 36
se <- sigma / sqrt(n); se # 0.1333
pnorm(12.2, mu, se, lower.tail = FALSE) # P(x̄ > 12.2) -> 0.0668
pnorm(12.2, mu, sigma, lower.tail = FALSE) # one package -> 0.4013
qnorm(c(0.025, 0.975), mu, se) # 11.7387 12.2613
# Sample proportion
p <- 0.4; n <- 200
se_p <- sqrt(p * (1 - p) / n); se_p # 0.034641
pnorm(0.45, p, se_p, lower.tail = FALSE) # 0.07477
# ── Simulating the CLT from a skewed population ────────────────────
population <- rexp(100000, rate = 1/10) # mean 10, strongly right-skewed
c(mean = mean(population), sd = sd(population))
draw_means <- function(n, reps = 5000)
replicate(reps, mean(sample(population, n)))
m2 <- draw_means(2)
m10 <- draw_means(10)
m30 <- draw_means(30)
par(mfrow = c(2, 2))
hist(population, breaks = 60, col = "#8A5FBF", border = NA,
main = "Population (exponential)", xlab = "x")
hist(m2, breaks = 40, col = "#5B2A86", border = NA, main = "n = 2", xlab = "x̄")
hist(m10, breaks = 40, col = "#0B7A7A", border = NA, main = "n = 10", xlab = "x̄")
hist(m30, breaks = 40, col = "#0FA3A3", border = NA, main = "n = 30", xlab = "x̄")
par(mfrow = c(1, 1))
# Theory vs. simulation — the numbers match
data.frame(
n = c(2, 10, 30),
sim_mean = c(mean(m2), mean(m10), mean(m30)), # all ≈ 10
sim_se = c(sd(m2), sd(m10), sd(m30)),
theory_se = 10 / sqrt(c(2, 10, 30)) # σ/√n
)
# Normality of the sampling distribution improves with n
qqnorm(m2, main = "n = 2"); qqline(m2)
qqnorm(m30, main = "n = 30"); qqline(m30)
Python¶
import numpy as np
import pandas as pd
from scipy import stats
import matplotlib.pyplot as plt
rng = np.random.default_rng(42)
# ── Direct calculations ────────────────────────────────────────────
mu, sigma, n = 12, 0.8, 36
se = sigma / np.sqrt(n) # 0.1333
stats.norm(mu, se).sf(12.2) # P(x̄ > 12.2) -> 0.0668
stats.norm(mu, sigma).sf(12.2) # one package -> 0.4013
stats.norm(mu, se).ppf([0.025, 0.975]) # 11.7387, 12.2613
# Sample proportion
p, n = 0.4, 200
se_p = np.sqrt(p * (1 - p) / n) # 0.034641
stats.norm(p, se_p).sf(0.45) # 0.07477
# ── Simulating the CLT ─────────────────────────────────────────────
population = rng.exponential(scale=10, size=100_000)
population.mean(), population.std(ddof=1) # ≈ 10, ≈ 10
def draw_means(n, reps=5000):
return rng.choice(population, size=(reps, n)).mean(axis=1)
m2, m10, m30 = draw_means(2), draw_means(10), draw_means(30)
fig, axes = plt.subplots(2, 2, figsize=(10, 6))
axes[0, 0].hist(population, bins=60, color="#8A5FBF"); axes[0, 0].set_title("Population")
axes[0, 1].hist(m2, bins=40, color="#5B2A86"); axes[0, 1].set_title("n = 2")
axes[1, 0].hist(m10, bins=40, color="#0B7A7A"); axes[1, 0].set_title("n = 10")
axes[1, 1].hist(m30, bins=40, color="#0FA3A3"); axes[1, 1].set_title("n = 30")
plt.tight_layout(); plt.show()
# Theory vs. simulation
pd.DataFrame({
"n": [2, 10, 30],
"sim_mean": [m2.mean(), m10.mean(), m30.mean()],
"sim_se": [m2.std(ddof=1), m10.std(ddof=1), m30.std(ddof=1)],
"theory_se": 10 / np.sqrt([2, 10, 30]),
})
Quick Reference¶
| Quantity | Formula | Excel | R | Python |
|---|---|---|---|---|
| SE of the mean | σ/√n |
sigma/SQRT(n) |
sigma/sqrt(n) |
sigma/np.sqrt(n) |
| SE of a proportion | √(pq/n) |
SQRT(p*(1-p)/n) |
sqrt(p*(1-p)/n) |
np.sqrt(p*(1-p)/n) |
| z for a sample mean | (x̄−μ)/(σ/√n) |
(xbar-mu)/(sigma/SQRT(n)) |
same | same |
P(x̄ > c) |
— | 1-NORM.DIST(c,μ,SE,TRUE) |
pnorm(c,μ,SE,lower.tail=FALSE) |
norm(μ,SE).sf(c) |
Percentile of x̄ |
— | NORM.INV(p,μ,SE) |
qnorm(p,μ,SE) |
norm(μ,SE).ppf(p) |
| Simulate the CLT | helper columns | replicate(k, mean(sample(pop,n))) |
rng.choice(pop,(k,n)).mean(1) |
Common Mistakes¶
- Using
σwhereσ/√nbelongs. This is the defining error of the chapter — always ask "is this about one value or about a mean?" - Believing the CLT says the data becomes normal. It says the sampling distribution of the statistic becomes normal.
- Applying the CLT to a sample of 8 from a heavily skewed population.
- Treating
s(the sample standard deviation) asσwithout acknowledging it — that switch is exactly what forces the t-distribution in Chapter 11. - Forgetting the
np ≥ 5andn(1−p) ≥ 5check for proportions. - Thinking a bigger population needs a bigger sample. It does not —
σ/√ncontains noN.
Exercises: 08-02: Exercises — Sampling Distributions and the Central Limit Theorem
⬅️ Previous: 08-01: Sampling Methods and Bias ➡️ Next: 09-01: Confidence Interval for a Mean