05-03: Exercises — Bayes' Theorem and Counting Rules¶

Notes reference: 05-03: Bayes' Theorem and Counting Rules

Q1: Bayes with a medical test¶

A disease affects 2% of a population. A test is 97% sensitive (P(+|D) = 0.97) and 92% specific (P(−|D') = 0.92).

A person tests positive. Find P(D | +).

Solution

Given:   P(D) = 0.02      P(D') = 0.98
         P(+|D)  = 0.97
         P(+|D') = 1 − 0.92 = 0.08          ← the false-positive rate

STEP 1  Total probability of a positive test
        P(+) = (0.97)(0.02) + (0.08)(0.98)
             = 0.0194 + 0.0784
             = 0.0978

STEP 2  Bayes
                  0.0194
        P(D | +) = ──────── = 0.1983        →  19.8%
                  0.0978

Natural-frequency check (per 10,000 people):

                   Disease (200)      No disease (9,800)     Total
Test positive           194                 784              978
Test negative             6               9,016            9,022

P(D | +) = 194 / 978 = 0.1983   ✓

Fewer than one positive in five is a true case, despite a "97% accurate" test — because the 9,800 healthy people generate far more false positives than the 200 sick people generate true ones.

bayes <- function(prior, sens, fpr) sens*prior / (sens*prior + fpr*(1-prior))
bayes(0.02, 0.97, 0.08)      # 0.19836

Q2: Change the prior¶

Repeat Q1 for a high-risk group where the prevalence is 30% instead of 2%.

Solution

P(+) = (0.97)(0.30) + (0.08)(0.70) = 0.2910 + 0.0560 = 0.3470

P(D | +) = 0.2910 / 0.3470 = 0.8386        →  83.9%

Same test, same accuracy, completely different meaning:

    prevalence 2%   →  P(D|+) = 19.8%
    prevalence 30%  →  P(D|+) = 83.9%

This is why screening tests are targeted at high-risk groups.
The evidence has not changed — the PRIOR has.

Q3: Bayes with three sources¶

Three plants make a component: A 50% (defect rate 1%), B 30% (defect rate 3%), C 20% (defect rate 5%).

A defective component arrives. Which plant most likely made it?

Solution

Plant	`P(plant)`	`P(def \\| plant)`	Joint	Posterior
A	0.50	0.01	0.0050	0.0050/0.0240 = 0.2083
B	0.30	0.03	0.0090	0.0090/0.0240 = 0.3750
C	0.20	0.05	0.0100	0.0100/0.0240 = 0.4167
			0.0240	1.0000

P(defective) = 0.0240   (2.4% overall defect rate)

Most likely source: PLANT C (41.7%), even though it makes the FEWEST
components. Its 5× higher defect rate outweighs its smaller volume.

priors <- c(A = 0.50, B = 0.30, C = 0.20)
likel  <- c(A = 0.01, B = 0.03, C = 0.05)
round(priors * likel / sum(priors * likel), 4)     # 0.2083 0.3750 0.4167

priors = np.array([0.50, 0.30, 0.20]); likel = np.array([0.01, 0.03, 0.05])
(priors * likel / (priors * likel).sum()).round(4)

Q4: Fundamental counting principle¶

A password is 3 letters (A–Z) followed by 2 digits. How many are possible if repeats are allowed?
How many if no character may repeat?
A restaurant offers 5 starters, 8 mains, 4 desserts, 3 drinks. How many complete meals?

Solution

1.  26 × 26 × 26 × 10 × 10 = 17,576 × 100 = 1,757,600

2.  Letters (no repeat):  26 × 25 × 24 = 15,600
    Digits  (no repeat):  10 × 9        = 90
    Total:                15,600 × 90   = 1,404,000

3.  5 × 8 × 4 × 3 = 480 meals

=26^3*10^2                       ' 1757600
=PERMUT(26,3)*PERMUT(10,2)       ' 1404000
=5*8*4*3                         ' 480

Q5: Permutation or combination?¶

Decide which applies, then compute.

Choose 3 of 12 employees for a committee.
Choose a president, vice-president and secretary from 12 employees.
Arrange 5 books on a shelf.
Choose 5 questions to answer out of 8 on an exam.
Award gold, silver and bronze among 10 runners.
Deal a 5-card poker hand from 52.

Solution

1. COMBINATION  — a committee has no ranks
   12C3 = 12!/(3!·9!) = 220

2. PERMUTATION  — the three roles are distinct
   12P3 = 12!/9! = 12 × 11 × 10 = 1,320

3. PERMUTATION  — order on the shelf matters
   5! = 120

4. COMBINATION  — which questions, not in what order
   8C5 = 8C3 = 56

5. PERMUTATION  — the medals are ranked
   10P3 = 10 × 9 × 8 = 720

6. COMBINATION  — a hand is a set; the deal order is irrelevant
   52C5 = 2,598,960

The test: if I swap two of the chosen items, is it a different outcome? Yes → permutation. No → combination.

=COMBIN(12,3)     ' 220
=PERMUT(12,3)     ' 1320
=FACT(5)          ' 120
=COMBIN(8,5)      ' 56
=PERMUT(10,3)     ' 720
=COMBIN(52,5)     ' 2598960

Q6: Combinations inside a probability¶

A box has 9 good bulbs and 3 defective. Four bulbs are drawn without replacement.

P(all 4 good)
P(exactly 1 defective)
P(at least 1 defective)

Solution

Total ways to choose 4 from 12:   12C4 = 495

1.  All good:  choose 4 from the 9 good ones
    9C4 = 126
    P = 126 / 495 = 0.2545

2.  Exactly 1 defective:  choose 1 of 3 defective AND 3 of 9 good
    3C1 × 9C3 = 3 × 84 = 252
    P = 252 / 495 = 0.5091

3.  At least 1 defective = 1 − P(none defective)
                        = 1 − 0.2545 = 0.7455

=COMBIN(9,4)/COMBIN(12,4)                    ' 0.25455
=COMBIN(3,1)*COMBIN(9,3)/COMBIN(12,4)        ' 0.50909
=1-COMBIN(9,4)/COMBIN(12,4)                  ' 0.74545
=HYPGEOM.DIST(1,4,3,12,FALSE)                ' 0.50909 — the same thing

This is the hypergeometric distribution (06-02) written out longhand.

Q7: Arrangements with repeated letters¶

How many distinct arrangements are there of the letters in:

LEVEL
MISSISSIPPI
PROBABILITY

Solution

1.  LEVEL — 5 letters:  L×2, E×2, V×1
        5! / (2!·2!·1!) = 120 / 4 = 30

2.  MISSISSIPPI — 11 letters:  M×1, I×4, S×4, P×2
        11! / (1!·4!·4!·2!) = 39,916,800 / (24 × 24 × 2)
                            = 39,916,800 / 1,152
                            = 34,650

3.  PROBABILITY — 11 letters:  P×1, R×1, O×1, B×2, A×1, I×2, L×1, T×1, Y×1
        11! / (2!·2!) = 39,916,800 / 4 = 9,979,200

=MULTINOMIAL(2,2,1)          ' LEVEL       -> 30
=MULTINOMIAL(1,4,4,2)        ' MISSISSIPPI -> 34650
=FACT(11)/(FACT(2)*FACT(2))  ' PROBABILITY -> 9979200

factorial(11) / prod(factorial(c(1,4,4,2)))     # 34650

Q8: Lottery probabilities¶

A lottery draws 6 numbers from 49.

P(matching all 6)
P(matching exactly 5)
P(matching exactly 4)
About how many tickets would you need to buy to have a 50% chance of a jackpot?

Solution

Total combinations:  49C6 = 13,983,816

1.  P(6 of 6) = 1 / 13,983,816 = 7.151 × 10⁻⁸

2.  Exactly 5:  choose 5 of the 6 winners AND 1 of the 43 non-winners
    6C5 × 43C1 = 6 × 43 = 258
    P = 258 / 13,983,816 = 1.845 × 10⁻⁵      (about 1 in 54,201)

3.  Exactly 4:  6C4 × 43C2 = 15 × 903 = 13,545
    P = 13,545 / 13,983,816 = 9.686 × 10⁻⁴   (about 1 in 1,032)

4.  Buying k distinct tickets gives P(win) = k / 13,983,816.
    Set that to 0.5:  k ≈ 6,991,908 tickets.
    At $2 each that is about $14 million — for a 50% chance.

=1/COMBIN(49,6)                              ' 7.151E-08
=COMBIN(6,5)*COMBIN(43,1)/COMBIN(49,6)       ' 1.845E-05
=COMBIN(6,4)*COMBIN(43,2)/COMBIN(49,6)       ' 9.686E-04

choose(6, 4) * choose(43, 2) / choose(49, 6)     # 0.0009686

Q9: Combine Bayes and counting¶

A bag holds two coins: one fair, one two-headed. You draw a coin at random and flip it 3 times, getting 3 heads.

What is the probability you drew the two-headed coin?

Solution

Let F = fair coin, T = two-headed coin, E = "3 heads in 3 flips"

Priors:       P(F) = 0.5           P(T) = 0.5
Likelihoods:  P(E | F) = (0.5)³ = 0.125
              P(E | T) = 1³      = 1.000

P(E) = (0.125)(0.5) + (1.000)(0.5) = 0.0625 + 0.5000 = 0.5625

           (1.000)(0.5)      0.5000
P(T | E) = ─────────────  =  ────────  =  0.8889        →  88.9%
              0.5625          0.5625

Belief moved from 50% to 88.9% on three flips.
A fourth head would push it to  0.5 / (0.5 + 0.03125) = 0.9412.
This is Bayesian updating: each new observation multiplies into the
likelihood and re-normalises.

posterior <- function(flips) {
  pF <- 0.5 * (0.5)^flips
  pT <- 0.5 * 1
  pT / (pF + pT)
}
sapply(0:5, posterior)     # 0.5 0.667 0.8 0.889 0.941 0.970

⬅️ Previous: 05-02: Exercises — Probability Rules and Conditional Probability ➡️ Next: 06-01: Exercises — Random Variables and Expected Value