Skip to content

09-02: Confidence Interval for a Proportion

Every poll you have ever read is this note. When the variable is categorical — supports / opposes, defective / good, clicked / did not — the parameter of interest is a proportion, and the interval has the same shape as 09-01 with a different standard error.

The Point Estimate

        x        number of successes in the sample
p̂  =  ─────  =  ──────────────────────────────────
        n                 sample size

q̂  =  1 − p̂

p̂ is an unbiased estimator of the population proportion p.

The Confidence Interval

                       ____________
                      /   p̂ · q̂
p̂   ±   z_(α/2) ·  √   ───────────
                          n

└─────────────────────────────────┘
        margin of error, E

Conditions

1.  Random sample
2.  n·p̂ ≥ 5   AND   n·q̂ ≥ 5        (some texts use 10 — state which you used)
3.  n ≤ 0.05 N                      (or apply the finite population correction)

Condition 2 is what lets the normal approximation stand in for the binomial (07-02).

Note

There is no "t-interval for a proportion". The standard error √(p̂q̂/n) is estimated from p̂ alone — there is no separate s to correct for, so the critical value stays z.

Worked Example

In a poll of n = 500 voters, 290 support a ballot measure. Build a 95% confidence interval for the true level of support.

p̂ = 290 / 500 = 0.58        q̂ = 0.42

Check:  n·p̂ = 290 ≥ 5   ✓        n·q̂ = 210 ≥ 5   ✓

SE = √(0.58 × 0.42 / 500) = √0.00048720 = 0.022073

z  = 1.96
E  = 1.96 × 0.022073 = 0.043263      →  ±4.3 percentage points

CI = 0.58 ± 0.0433  =  (0.5367, 0.6233)   →  (53.7%, 62.3%)

Interpretation. We are 95% confident that between 53.7% and 62.3% of all voters support the measure. Because the whole interval lies above 0.50, there is evidence of majority support.

The "margin of error" in news reports is exactly this E, almost always at 95% confidence, and almost always ignoring every source of error except sampling.

Sample Size for a Proportion

Solve E = z√(p̂q̂/n) for n:

              ( z_(α/2) )²
n  =  p̂ · q̂ · ( ──────── )              ALWAYS round UP
              (    E    )

Two cases:

Situation Use
A prior estimate of p exists (pilot study, previous poll) that p̂
No prior information p̂ = 0.5 — the most conservative choice, since p̂q̂ is maximized at 0.25

Example — no prior estimate, 95% confidence, E = 0.03:

n = 0.5 × 0.5 × (1.96 / 0.03)² = 0.25 × 4268.44 = 1067.1  →  n = 1068

That is why national polls report about 1,000 respondents and a ±3% margin of error — and why they use ~1,000 whether the country has 5 million people or 300 million.

With a prior estimate of p̂ = 0.58:

n = 0.58 × 0.42 × (1.96/0.03)² = 0.2436 × 4268.44 = 1039.8  →  n = 1040

Better Intervals for Small Samples

The formula above is the Wald interval. It behaves badly when n is small or p̂ is near 0 or 1 — it can even produce bounds outside [0, 1].

Interval Fix When
Wald the standard formula n large, p̂ near 0.5 — the textbook default
Plus-four (Agresti-Coull) Add 2 successes and 2 failures, then use the Wald formula Simple, much better coverage; easy by hand
Wilson score Inverts the test rather than approximating R and Python's default; best all-round
Clopper-Pearson Exact, based on the binomial Guaranteed coverage, but conservative (wider)

The plus-four adjustment is worth knowing because it is a one-line change:

       x + 2
p̃  =  ───────      then use  p̃  ±  z·√( p̃(1−p̃) / (n+4) )
       n + 4

Confidence Interval for the Difference of Two Proportions

For comparing two groups (also the effect size behind a chi-square test of independence):

                             ______________________
                            /  p̂₁q̂₁     p̂₂q̂₂
(p̂₁ − p̂₂)  ±  z_(α/2) ·  √   ────── + ──────
                               n₁         n₂

If the interval contains 0, the two proportions are not significantly different at that confidence level.

Excel

' ── Confidence interval for one proportion ──────────────────────────
' B1 = x = 290, B2 = n = 500, B3 = confidence = 0.95
=B1/B2                                  ' p̂                  -> 0.58
=1-B1/B2                                ' q̂                  -> 0.42
=SQRT((B1/B2)*(1-B1/B2)/B2)             ' standard error     -> 0.022073
=NORM.S.INV(1-(1-B3)/2)                 ' critical z         -> 1.95996
=NORM.S.INV(0.975)*SQRT((B1/B2)*(1-B1/B2)/B2)   ' margin E   -> 0.043263
=B1/B2 - B7                             ' lower bound        -> 0.536737
=B1/B2 + B7                             ' upper bound        -> 0.623263

' Condition check
=AND(B2*(B1/B2)>=5, B2*(1-B1/B2)>=5)    ' must be TRUE

' ── Sample size ─────────────────────────────────────────────────────
=ROUNDUP(0.5*0.5*(NORM.S.INV(0.975)/0.03)^2, 0)        ' no prior -> 1068
=ROUNDUP(0.58*0.42*(NORM.S.INV(0.975)/0.03)^2, 0)      ' prior p̂  -> 1040

' ── Plus-four (Agresti-Coull) interval ──────────────────────────────
=(B1+2)/(B2+4)                                          ' p̃
=NORM.S.INV(0.975)*SQRT(D1*(1-D1)/(B2+4))               ' margin (D1 = p̃)

' ── From raw yes/no data ────────────────────────────────────────────
=COUNTIF(A2:A501, "Yes")/COUNTA(A2:A501)                ' p̂ straight from a column

' ── Difference of two proportions ───────────────────────────────────
' p̂1 in F1, n1 in F2, p̂2 in G1, n2 in G2
=F1-G1                                                  ' point estimate
=SQRT(F1*(1-F1)/F2 + G1*(1-G1)/G2)                      ' standard error
=(F1-G1) - NORM.S.INV(0.975)*SQRT(F1*(1-F1)/F2+G1*(1-G1)/G2)   ' lower
=(F1-G1) + NORM.S.INV(0.975)*SQRT(F1*(1-F1)/F2+G1*(1-G1)/G2)   ' upper

R

x <- 290; n <- 500

# ── By hand (Wald — matches the textbook formula) ──────────────────
ci_prop <- function(x, n, conf = 0.95) {
  phat <- x / n
  z    <- qnorm(1 - (1 - conf)/2)
  se   <- sqrt(phat * (1 - phat) / n)
  e    <- z * se
  c(phat = phat, se = se, margin = e, lower = phat - e, upper = phat + e)
}
round(ci_prop(290, 500), 5)
#     phat       se   margin    lower    upper
#  0.58000  0.02207  0.04326  0.53674  0.62326

# ── Built-in: prop.test  (Wilson score interval by default) ────────
prop.test(x = 290, n = 500)                        # with continuity correction
prop.test(290, 500, correct = FALSE)$conf.int      # Wilson, no correction
# 0.5366 0.6222

# ── Exact (Clopper-Pearson) ────────────────────────────────────────
binom.test(290, 500)$conf.int                      # 0.5359 0.6234

# ── Plus-four (Agresti-Coull) ──────────────────────────────────────
ci_prop(290 + 2, 500 + 4)

# ── Sample size ────────────────────────────────────────────────────
n_for_prop <- function(E, p = 0.5, conf = 0.95)
  ceiling(p * (1 - p) * (qnorm(1 - (1 - conf)/2) / E)^2)

n_for_prop(0.03)              # no prior estimate -> 1068
n_for_prop(0.03, p = 0.58)    # with a prior      -> 1040
n_for_prop(0.01)              # ±1% -> 9604

# ── From raw yes/no data ───────────────────────────────────────────
support <- c(rep("Yes", 290), rep("No", 210))
tb <- table(support)
prop.test(tb["Yes"], sum(tb))$conf.int

# ── Difference of two proportions ──────────────────────────────────
prop.test(x = c(145, 118), n = c(250, 250))$conf.int   # CI for p1 − p2

# ── Verifying the 95% coverage claim by simulation ─────────────────
set.seed(3)
p_true <- 0.58
covered <- replicate(2000, {
  xi <- rbinom(1, 500, p_true)
  ci <- ci_prop(xi, 500)
  ci["lower"] <= p_true && p_true <= ci["upper"]
})
mean(covered)                 # ≈ 0.95

Python

import numpy as np
from scipy import stats
from statsmodels.stats.proportion import (
    proportion_confint, samplesize_confint_proportion, confint_proportions_2indep
)

x, n = 290, 500

# ── By hand (Wald) ─────────────────────────────────────────────────
def ci_prop(x, n, conf=0.95):
    phat = x / n
    z = stats.norm.ppf(1 - (1 - conf) / 2)
    se = np.sqrt(phat * (1 - phat) / n)
    e = z * se
    return phat, se, e, phat - e, phat + e

ci_prop(290, 500)
# (0.58, 0.022073, 0.043263, 0.536737, 0.623263)

# ── statsmodels: several methods in one call ───────────────────────
proportion_confint(290, 500, alpha=0.05, method="normal")      # Wald
proportion_confint(290, 500, method="wilson")                  # Wilson score
proportion_confint(290, 500, method="agresti_coull")           # plus-four family
proportion_confint(290, 500, method="beta")                    # Clopper-Pearson

# ── Sample size ────────────────────────────────────────────────────
def n_for_prop(E, p=0.5, conf=0.95):
    z = stats.norm.ppf(1 - (1 - conf) / 2)
    return int(np.ceil(p * (1 - p) * (z / E) ** 2))

n_for_prop(0.03)             # 1068
n_for_prop(0.03, p=0.58)     # 1040
n_for_prop(0.01)             # 9604

# statsmodels version (uses the same formula)
int(np.ceil(samplesize_confint_proportion(0.5, half_length=0.03)))

# ── From raw yes/no data ───────────────────────────────────────────
import pandas as pd
support = pd.Series(["Yes"] * 290 + ["No"] * 210)
k = (support == "Yes").sum()
proportion_confint(k, support.size, method="wilson")

# ── Difference of two proportions ──────────────────────────────────
confint_proportions_2indep(145, 250, 118, 250, method="wald")

# ── Verifying 95% coverage ─────────────────────────────────────────
rng = np.random.default_rng(3)
p_true = 0.58
hits = 0
for _ in range(2000):
    xi = rng.binomial(500, p_true)
    _, _, _, lo, hi = ci_prop(xi, 500)
    hits += lo <= p_true <= hi
hits / 2000                  # ≈ 0.95

Quick Reference

Task Excel R Python
p̂ x/n or COUNTIF/COUNTA x/n x/n
SE of p̂ SQRT(p*(1-p)/n) sqrt(p*(1-p)/n) np.sqrt(p*(1-p)/n)
Critical z NORM.S.INV(1-α/2) qnorm(1-α/2) norm.ppf(1-α/2)
Wald CI p ± z*SE user function proportion_confint(..., "normal")
Wilson CI — prop.test(x,n)$conf.int proportion_confint(..., "wilson")
Exact CI — binom.test(x,n)$conf.int proportion_confint(..., "beta")
Sample size ROUNDUP(p*q*(z/E)^2,0) ceiling(p*q*(z/E)^2) ceil(p*q*(z/E)**2)
Two-proportion CI manual formula prop.test(c(x1,x2), c(n1,n2)) confint_proportions_2indep

Common Mistakes

  • Using t instead of z. Proportion intervals always use z.
  • Forgetting the np̂ ≥ 5 and nq̂ ≥ 5 check — with p̂ near 0 or 1 the Wald interval fails badly.
  • Using p = 0.5 in the sample-size formula when a good prior estimate exists (wastes sample), or a shaky prior when none is justified (under-samples).
  • Reporting an interval bound below 0 or above 1 without noticing that it is impossible.
  • Rounding the sample size down.
  • Treating the reported "margin of error" as covering non-sampling error. It covers only sampling variability — nonresponse and question wording are not in the formula (08-01).

Exercises: 09-02: Exercises — Confidence Interval for a Proportion

⬅️ Previous: 09-01: Confidence Interval for a Mean ➡️ Next: 10-01: Hypothesis Testing Fundamentals