05-01: Probability Basics¶
Descriptive statistics summarize what happened. Probability describes what could happen and how often — and it is the bridge to every inferential method from Chapter 08 onward. A p-value, a confidence level, a Type I error rate: all three are probabilities.
Vocabulary¶
| Term | Definition | Example (rolling one die) |
|---|---|---|
| Probability experiment | A process with a well-defined, uncertain outcome | Rolling a die |
| Outcome | One result of the experiment | Rolling a 4 |
Sample space S |
The set of all possible outcomes | S = {1, 2, 3, 4, 5, 6} |
Event E |
Any subset of the sample space | "even number" = {2, 4, 6} |
| Simple event | An event with exactly one outcome | {4} |
| Compound event | An event with two or more outcomes | {2, 4, 6} |
Complement E' or Ē |
Everything in S that is not in E |
{1, 3, 5} |
Sample spaces for common experiments¶
One coin S = {H, T} n(S) = 2
Two coins S = {HH, HT, TH, TT} n(S) = 4
One die S = {1,2,3,4,5,6} n(S) = 6
Two dice S = {(1,1), (1,2), …, (6,6)} n(S) = 36
One card S = 52 cards n(S) = 52
Gender of 3 children S = {BBB,BBG,BGB,BGG,GBB,GBG,GGB,GGG} n(S) = 8
Three Ways to Assign a Probability¶
1. Classical (theoretical)¶
Requires equally likely outcomes.
number of outcomes in E n(E)
P(E) = ───────────────────────────── = ──────
total number of outcomes n(S)
P(even on one die) = 3/6 = 0.5
2. Empirical (relative frequency)¶
Based on observed data — no equal-likelihood assumption needed.
If 38 of 200 customers bought the premium plan, P(premium) = 38/200 = 0.19.
The Law of Large Numbers: as the number of trials grows, the empirical probability converges on the true probability. Ten coin flips can easily give 7 heads; ten million cannot stray far from half.
3. Subjective¶
A personal degree of belief, used when the experiment cannot be repeated. "There is a 70% chance this product launches on time." Legitimate but not verifiable by repetition.
The Rules Every Probability Must Obey¶
1. 0 ≤ P(E) ≤ 1 For any event E
2. P(S) = 1 Something in the sample space must happen
3. P(impossible event) = 0
4. P(certain event) = 1
5. Σ P(all simple outcomes) = 1
6. P(E') = 1 − P(E) Complement rule
Tip
The complement rule is the biggest time-saver in the chapter. "At least one" problems are almost always faster as 1 − P(none). See the example below.
Odds vs. Probability¶
Different scales for the same information — gambling and medicine both use odds.
Odds in favour of E = P(E) : P(E') = P(E) / (1 − P(E))
Odds against E = P(E') : P(E)
Given odds a : b in favour, P(E) = a / (a + b)
Example. P(win) = 0.25 → odds in favour = 0.25 : 0.75 = 1 : 3; odds against = 3 : 1. Conversely, odds of 2 : 5 in favour mean P = 2/7 = 0.286.
Worked Examples¶
A single card¶
Draw one card from a standard 52-card deck.
P(heart) = 13/52 = 1/4 = 0.250
P(king) = 4/52 = 1/13 = 0.077
P(face card) = 12/52 = 3/13 = 0.231 (J, Q, K in 4 suits)
P(not a heart) = 1 − 13/52 = 39/52 = 0.750 (complement rule)
Two dice¶
Roll two fair dice; n(S) = 36.
Sum 2 3 4 5 6 7 8 9 10 11 12
Ways 1 2 3 4 5 6 5 4 3 2 1 (total = 36)
P(sum = 7) = 6/36 = 0.1667
P(sum = 2) = 1/36 = 0.0278
P(sum ≥ 10) = (3 + 2 + 1)/36 = 6/36 = 0.1667
P(doubles) = 6/36 = 0.1667
P(sum ≠ 7) = 1 − 6/36 = 30/36 = 0.8333
The complement rule in action¶
A quality process produces 3% defective items. In a sample of 5 items, what is the probability of at least one defective?
Direct route: P(1) + P(2) + P(3) + P(4) + P(5) — five calculations
Complement: P(at least one) = 1 − P(none)
= 1 − (0.97)^5
= 1 − 0.8587
= 0.1413
One calculation instead of five. (The binomial machinery behind this is 06-02.)
The birthday problem¶
With 23 people, what is the probability that at least two share a birthday?
P(all different) = (365/365)(364/365)(363/365) … (343/365) = 0.4927
P(at least two share) = 1 − 0.4927 = 0.5073
Just over 50% — a famously counter-intuitive result, and again the complement doing the work.
Simulation: Probability by Experiment¶
When a probability is hard to derive, simulate it. Generate many trials, count the successes, divide. This is the empirical definition applied deliberately, and it is how the sampling distributions of Chapter 08 are demonstrated.
Excel¶
' ── Basic probabilities from counts ─────────────────────────────────
=13/52 ' P(heart) -> 0.25
=1-13/52 ' complement -> 0.75
=COUNTIF(A2:A201,"premium")/COUNTA(A2:A201) ' empirical probability
' ── Odds ────────────────────────────────────────────────────────────
=B1/(1-B1) ' odds in favour, as a ratio to 1
=2/(2+5) ' odds 2:5 in favour -> P = 0.2857
' ── At-least-one via the complement ─────────────────────────────────
=1-(1-0.03)^5 ' -> 0.1413
' ── Random simulation ───────────────────────────────────────────────
=RANDBETWEEN(1,6) ' one die roll
=RANDBETWEEN(1,6)+RANDBETWEEN(1,6) ' sum of two dice
=RAND() ' uniform random number in [0,1)
=IF(RAND()<0.03,"defective","ok") ' a 3% defect process
' Fill 1000 rows with a two-dice sum in column A, then:
=COUNTIF(A2:A1001, 7)/1000 ' empirical P(sum = 7) ≈ 0.167
' Press F9 to re-randomize. Turn off recalculation while reading results:
' Formulas ▸ Calculation Options ▸ Manual
' ── Exact two-dice distribution without simulation ──────────────────
' Put sums 2..12 in D2:D12, then:
=SUMPRODUCT((SEQUENCE(6)+TRANSPOSE(SEQUENCE(6))=D2)*1)/36
Note
Excel has a Random Number Generation tool: Data ▸ Data Analysis ▸ Random Number Generation, with Uniform, Normal, Bernoulli, Binomial, Poisson, Discrete, and Patterned options. Unlike RAND(), its output is static — it does not change on every recalculation.
R¶
# ── Sample spaces ──────────────────────────────────────────────────
S_die <- 1:6
S_2die <- expand.grid(d1 = 1:6, d2 = 1:6) # all 36 ordered pairs
nrow(S_2die) # 36
deck <- expand.grid(rank = c(2:10, "J","Q","K","A"),
suit = c("Hearts","Diamonds","Clubs","Spades"))
nrow(deck) # 52
# ── Classical probability ──────────────────────────────────────────
mean(S_die %% 2 == 0) # P(even) -> 0.5
sums <- S_2die$d1 + S_2die$d2
mean(sums == 7) # 0.1667
mean(sums >= 10) # 0.1667
table(sums) / 36 # full distribution
# ── Empirical probability ──────────────────────────────────────────
plan <- c(rep("premium", 38), rep("basic", 162))
prop.table(table(plan)) # premium -> 0.19
# ── Complement / at least one ──────────────────────────────────────
1 - 0.97^5 # 0.1413
# Birthday problem
1 - prod((365 - 0:22) / 365) # 0.5073
# ── Odds ───────────────────────────────────────────────────────────
p <- 0.25
p / (1 - p) # odds in favour -> 0.333 = 1:3
2 / (2 + 5) # odds 2:5 -> P = 0.2857
# ── Simulation ─────────────────────────────────────────────────────
set.seed(42)
rolls <- sample(1:6, 10000, replace = TRUE)
mean(rolls == 6) # ≈ 0.167
two <- sample(1:6, 10000, TRUE) + sample(1:6, 10000, TRUE)
mean(two == 7) # ≈ 0.167
round(prop.table(table(two)), 3)
# Law of large numbers, visualised
running <- cumsum(rolls == 6) / seq_along(rolls)
plot(running, type = "l", ylim = c(0, 0.4), col = "#5B2A86",
xlab = "Number of rolls", ylab = "Running P(six)")
abline(h = 1/6, col = "#0FA3A3", lwd = 2, lty = 2)
Python¶
import numpy as np
import pandas as pd
from itertools import product
rng = np.random.default_rng(42)
# ── Sample spaces ──────────────────────────────────────────────────
S_die = [1, 2, 3, 4, 5, 6]
S_2die = list(product(range(1, 7), repeat=2)) # 36 ordered pairs
len(S_2die)
deck = list(product(list(range(2, 11)) + ["J", "Q", "K", "A"],
["Hearts", "Diamonds", "Clubs", "Spades"]))
len(deck) # 52
# ── Classical probability ──────────────────────────────────────────
np.mean([x % 2 == 0 for x in S_die]) # 0.5
sums = np.array([a + b for a, b in S_2die])
(sums == 7).mean() # 0.1667
(sums >= 10).mean() # 0.1667
pd.Series(sums).value_counts(normalize=True).sort_index()
# ── Empirical probability ──────────────────────────────────────────
plan = pd.Series(["premium"] * 38 + ["basic"] * 162)
plan.value_counts(normalize=True) # premium -> 0.19
# ── Complement / at least one ──────────────────────────────────────
1 - 0.97 ** 5 # 0.1413
# Birthday problem
np.prod([(365 - i) / 365 for i in range(23)]) # P(all different)
1 - np.prod([(365 - i) / 365 for i in range(23)]) # 0.5073
# ── Odds ───────────────────────────────────────────────────────────
p = 0.25
p / (1 - p) # 0.333 = 1:3
2 / (2 + 5) # 0.2857
# ── Simulation ─────────────────────────────────────────────────────
rolls = rng.integers(1, 7, size=10_000)
(rolls == 6).mean() # ≈ 0.167
two = rng.integers(1, 7, 10_000) + rng.integers(1, 7, 10_000)
(two == 7).mean() # ≈ 0.167
# Law of large numbers
running = np.cumsum(rolls == 6) / np.arange(1, rolls.size + 1)
Quick Reference¶
| Task | Excel | R | Python |
|---|---|---|---|
| Empirical probability | COUNTIF/COUNTA |
mean(x == v) |
(x == v).mean() |
| Complement | 1-p |
1 - p |
1 - p |
| Enumerate a sample space | manual / SEQUENCE |
expand.grid() |
itertools.product |
| Random integer | RANDBETWEEN(a,b) |
sample(a:b, n, TRUE) |
rng.integers(a, b+1, n) |
| Random uniform [0,1) | RAND() |
runif(n) |
rng.random(n) |
| Random sample without replacement | Data Analysis ▸ Sampling | sample(x, k) |
rng.choice(x, k, replace=False) |
| Reproducible randomness | (not available) | set.seed(42) |
default_rng(42) |
| Frequency table of a simulation | PivotTable | table(x)/length(x) |
value_counts(normalize=True) |
Common Mistakes¶
- Applying the classical formula when outcomes are not equally likely (e.g. "it either rains or it doesn't, so P = 0.5").
- Reporting a probability outside
[0, 1]— always a sign of an arithmetic slip. - Forgetting that
{HH, HT, TH, TT}has four outcomes, soP(one head)is2/4, not1/3. - Counting
(1,2)and(2,1)as one outcome when rolling two distinguishable dice — the sample space has 36 ordered pairs. - Working "at least one" the long way instead of using the complement.
- The gambler's fallacy: five heads in a row does not make tails more likely on the sixth flip.
Exercises: 05-01: Exercises — Probability Basics
⬅️ Previous: 04-02: Measures of Position and Outliers ➡️ Next: 05-02: Probability Rules and Conditional Probability