Skip to content

10-01: Exercises — Hypothesis Testing Fundamentals

Notes reference: 10-01: Hypothesis Testing Fundamentals

Q1: Write the hypotheses

State H₀ and H₁ in symbols and say whether the test is left-, right-, or two-tailed.

  1. A company claims its cereal boxes contain at least 500 g.
  2. A researcher believes the mean class size differs from 24.
  3. A manufacturer claims fewer than 2% of its units are defective.
  4. A trainer claims the new programme increases mean output above 120 units.
  5. A pollster wants to know whether support differs from 50%.

Solution

1.  H₀: μ ≥ 500      H₁: μ < 500        LEFT-tailed
    (the sceptic tests whether boxes are UNDERFILLED)

2.  H₀: μ = 24       H₁: μ ≠ 24         TWO-tailed
    ("differs" gives no direction)

3.  H₀: p ≥ 0.02     H₁: p < 0.02       LEFT-tailed

4.  H₀: μ ≤ 120      H₁: μ > 120        RIGHT-tailed
    (the research claim goes in H₁)

5.  H₀: p = 0.50     H₁: p ≠ 0.50       TWO-tailed

The rule: the equality always sits in H₀; the research claim always sits in H₁; hypotheses are always about parameters (μ, p), never about statistics (x̄, p̂).

Q2: Spot the bad hypotheses

Explain what is wrong with each pair.

  1. H₀: x̄ = 50, H₁: x̄ ≠ 50
  2. H₀: μ > 100, H₁: μ < 100
  3. H₀: μ ≠ 30, H₁: μ = 30
  4. H₀: μ = 45, H₁: μ > 50

Solution

1. Uses the SAMPLE MEAN x̄. Hypotheses are claims about the POPULATION.
   Correct:  H₀: μ = 50,  H₁: μ ≠ 50

2. Neither contains an equality, and the two do not cover μ = 100.
   Correct:  H₀: μ ≥ 100,  H₁: μ < 100

3. Reversed — the equality must be in H₀.
   Correct:  H₀: μ = 30,  H₁: μ ≠ 30

4. Not exhaustive: values between 45 and 50 belong to neither hypothesis.
   Correct:  H₀: μ ≤ 45,  H₁: μ > 45   (or restate the research claim)

Q3: Type I and Type II errors

A pharmaceutical company tests whether a new drug is more effective than the current one. H₀: the new drug is no better.

  1. Describe a Type I error in context.
  2. Describe a Type II error in context.
  3. Which is worse here, and how would you set α?
  4. Now imagine H₀: the drug is safe. Which error is worse?

Solution

1.  TYPE I (α): concluding the new drug IS better when it is NOT.
    Consequence: an ineffective drug reaches patients; the company
    invests in a failure; patients forgo a working treatment.

2.  TYPE II (β): failing to detect a genuine improvement.
    Consequence: a beneficial drug is abandoned.

3.  For an EFFICACY claim, a Type I error is usually worse — it puts a
    useless drug on the market. Regulators therefore demand small α
    (0.01 or 0.05) and typically require TWO independent trials.
    Reduce β instead by increasing the sample size, not by raising α.

4.  If H₀ is "the drug is SAFE", the errors swap roles:
    TYPE II (failing to reject "safe" when the drug is dangerous) is
    catastrophic. Safety testing therefore uses large samples and
    deliberately high power, accepting more false alarms.

    THE LESSON: which error is "worse" depends entirely on how H₀ is
    framed. Always state H₀ before arguing about α.

Q4: Power

A test has α = 0.05 and β = 0.20.

  1. What is the power?
  2. Interpret it.
  3. Name four ways to increase power, and their costs.

Solution

1.  Power = 1 − β = 1 − 0.20 = 0.80    (80%)

2.  If the alternative is true with the assumed effect size, this test
    detects it 80% of the time. It MISSES a real effect 20% of the time.
    80% is the conventional minimum for a well-designed study.

3.  HOW TO INCREASE POWER
    ┌──────────────────────┬──────────────────────────────────────────┐
    │ Increase n           │ The clean fix. Costs money and time.     │
    │ Increase α           │ Cheap but raises the false-positive rate │
    │                      │ — usually unacceptable.                  │
    │ Reduce σ             │ Better measurement, tighter protocol,    │
    │                      │ blocking/pairing. Often the best value.  │
    │ Larger true effect   │ Not under your control (but a bigger     │
    │                      │ dose or a stronger intervention may be). │
    └──────────────────────┴──────────────────────────────────────────┘

    Using a one-tailed test also raises power — but only if the
    direction was justified BEFORE seeing the data.

# How many per group for 80% power to detect a 0.5 SD difference?
power.t.test(delta = 0.5, sd = 1, sig.level = 0.05, power = 0.80)$n   # ≈ 64
# Power of a study that only recruited 30 per group:
power.t.test(n = 30, delta = 0.5, sd = 1, sig.level = 0.05)$power     # ≈ 0.48

Q5: p-values from a test statistic

For each, find the p-value and decide at α = 0.05.

# Test statistic Tail df
1 z = 2.15 right —
2 z = −1.42 left —
3 z = 2.15 two —
4 t = 2.60 right 18
5 t = −1.80 two 24

Solution

1.  p = P(Z > 2.15)      = 1 − 0.98422 = 0.01578    ≤ 0.05  →  REJECT
2.  p = P(Z < −1.42)     = 0.07780                  > 0.05  →  FAIL TO REJECT
3.  p = 2 × 0.01578      = 0.03156                  ≤ 0.05  →  REJECT
4.  p = P(T₁₈ > 2.60)    = 0.00909                  ≤ 0.05  →  REJECT
5.  p = 2 × P(T₂₄ < −1.80) = 2 × 0.04222 = 0.08444  > 0.05  →  FAIL TO REJECT

=1-NORM.S.DIST(2.15, TRUE)          ' 0.015778
=NORM.S.DIST(-1.42, TRUE)           ' 0.077804
=2*(1-NORM.S.DIST(2.15, TRUE))      ' 0.031557
=T.DIST.RT(2.60, 18)                ' 0.009093
=T.DIST.2T(1.80, 24)                ' 0.084437

pnorm(2.15, lower.tail = FALSE); pnorm(-1.42)
2 * pnorm(2.15, lower.tail = FALSE)
pt(2.60, 18, lower.tail = FALSE)
2 * pt(-1.80, 24)

Note #1 vs. #3: the same statistic gives a p-value twice as large in a two-tailed test. That is why the tail must be chosen before seeing the data.

Q6: What a p-value is not

A study reports p = 0.03. Mark each statement true or false.

  1. There is a 3% probability that H₀ is true.
  2. There is a 3% probability the result is due to chance.
  3. If H₀ were true, results this extreme or more would occur 3% of the time.
  4. The effect is large.
  5. There is a 97% probability that H₁ is true.
  6. The result is statistically significant at α = 0.05 but not at α = 0.01.

Solution

1. FALSE — p is computed ASSUMING H₀ is true. It says nothing about
           P(H₀ | data), which requires a prior (see Bayes, 05-03).

2. FALSE — a common but sloppy phrasing. p is a conditional probability
           of the DATA given H₀, not of "chance" given the data.

3. TRUE  — this is the definition.

4. FALSE — p says nothing about effect size. With n = 100,000 a
           meaningless difference gives p < 0.001.

5. FALSE — same error as #1 in the other direction.

6. TRUE  — 0.03 ≤ 0.05 but 0.03 > 0.01.

Q7: The five steps end to end

A machine is set to fill 250 ml bottles. A sample of 40 bottles gives x̄ = 247.2 ml. The process standard deviation is known to be σ = 8 ml. Test at α = 0.05 whether the machine is off target.

Solution

STEP 1  STATE
        H₀: μ = 250        H₁: μ ≠ 250        TWO-tailed
        α = 0.05

STEP 2  CHECK
        Random sample assumed  ✓
        σ known                ✓   →  z-test
        n = 40 ≥ 30            ✓   →  CLT applies

STEP 3  COMPUTE
        SE = σ/√n = 8/√40 = 8/6.3246 = 1.26491

        z = (247.2 − 250)/1.26491 = −2.8/1.26491 = −2.2136

STEP 4  DECIDE
        Critical values:  ±1.9600
        |−2.2136| > 1.9600                     →  REJECT H₀

        p-value: 2 × P(Z < −2.2136) = 2(0.013427) = 0.026854
        0.0269 ≤ 0.05                          →  REJECT H₀

STEP 5  CONCLUDE
        There is sufficient evidence at the 5% significance level to
        conclude that the mean fill volume differs from 250 ml.
        The machine appears to be UNDERFILLING by about 2.8 ml.

SUPPORTING NUMBERS TO REPORT
        95% CI:  247.2 ± 1.96(1.26491) = (244.72, 249.68)
                 250 is OUTSIDE the interval — the same conclusion.
        Effect size: d = (247.2 − 250)/8 = −0.35   (small-to-medium)
        Practical view: 2.8 ml on a 250 ml bottle is a 1.1% shortfall.
                 Whether that matters is a business decision, not a
                 statistical one.

=8/SQRT(40)                                 ' 1.264911
=(247.2-250)/1.264911                       ' -2.213594
=2*NORM.S.DIST(-ABS(-2.213594), TRUE)       ' 0.026854
=NORM.S.INV(0.975)                          ' 1.959964

Q8: Confidence intervals and tests agree

A 95% CI for μ is (18.4, 23.6). For each null value, state the two-tailed decision at α = 0.05 without computing anything.

  1. H₀: μ = 20
  2. H₀: μ = 24
  3. H₀: μ = 18
  4. H₀: μ = 23.6

Solution

1.  20 is INSIDE (18.4, 23.6)   →  FAIL TO REJECT H₀
2.  24 is OUTSIDE               →  REJECT H₀
3.  18 is OUTSIDE               →  REJECT H₀
4.  23.6 is exactly AT the boundary  →  p = 0.05 exactly; conventionally
    "reject" at α = 0.05 since p ≤ α, but this is a knife-edge case that
    should be reported as such rather than dressed up as a finding.

THE EQUIVALENCE
    A (1 − α) confidence interval contains exactly those null values
    that a two-tailed test at level α would FAIL to reject.

WHY IT MATTERS
    The interval carries strictly more information than the p-value: it
    reports the plausible range of μ, not just a yes/no verdict. Report
    both — and if you can only report one, report the interval.

⬅️ Previous: 09-02: Exercises — Confidence Interval for a Proportion ➡️ Next: 10-02: Exercises — One-Sample Z-Tests