Skip to content

05-02: Probability Rules and Conditional Probability

Single events are easy. The rules in this note handle combinations of events — "A or B", "A and B", "A given B" — and they are what the chi-square test of independence in 12-02 ultimately tests.

The Addition Rules — "OR"

Mutually exclusive events

Two events are mutually exclusive (disjoint) if they cannot happen at the same time: P(A and B) = 0.

P(A or B)  =  P(A) + P(B)                      [mutually exclusive only]

The general addition rule

If the events can overlap, the overlap gets counted twice — subtract it once.

P(A or B)  =  P(A) + P(B) − P(A and B)         [always valid]

        ┌───────────────┬───────────────┐
        │       A       │       B       │
        │          ┌────┼────┐          │
        │          │ A∩B│    │          │      ← counted twice by P(A)+P(B)
        │          └────┼────┘          │
        └───────────────┴───────────────┘

Example. Draw one card: P(king or heart).

P(king)  = 4/52       P(heart) = 13/52       P(king of hearts) = 1/52

P(king or heart) = 4/52 + 13/52 − 1/52 = 16/52 = 0.3077

For three events:

P(A or B or C) = P(A)+P(B)+P(C) − P(A∩B) − P(A∩C) − P(B∩C) + P(A∩B∩C)

The Multiplication Rules — "AND"

Independent events

Two events are independent if the occurrence of one does not change the probability of the other.

P(A and B)  =  P(A) × P(B)                     [independent only]

Extends to any number: P(A₁ and A₂ and … and Aₙ) = P(A₁)P(A₂)…P(Aₙ).

The general multiplication rule

P(A and B)  =  P(A) × P(B | A)                 [always valid]
            =  P(B) × P(A | B)

Example — with replacement (independent). Draw a card, replace it, draw again.

P(two hearts) = (13/52)(13/52) = 0.0625

Example — without replacement (dependent). Draw two cards, no replacement.

P(two hearts) = (13/52)(12/51) = 0.0588

The second draw's probability changed because the first card is gone — that is dependence.

Tip

"Or" → add. "And" → multiply. Then ask the follow-up question: for "or", can they overlap? For "and", does the first change the second?

Conditional Probability

The probability of A given that B has already occurred.

                 P(A and B)
P(A | B)  =  ─────────────────        provided P(B) > 0
                    P(B)

Conditioning shrinks the sample space: you are no longer working within all of S, only within B.

Testing independence

Any one of these three, all equivalent:

P(A | B) = P(A)              knowing B tells you nothing about A
P(B | A) = P(B)
P(A and B) = P(A) × P(B)

If any one fails, the events are dependent.

Contingency Tables

A two-way table of counts — the natural home for conditional probability, and the exact input to a chi-square test of independence.

Survey of 200 people: exercise habit by health rating.

Excellent Good Poor Total
Exercises 45 55 20 120
Does not 15 30 35 80
Total 60 85 55 200

Reading probabilities off the table:

Marginal      P(Exercises)            = 120/200 = 0.600
Marginal      P(Excellent)            =  60/200 = 0.300

Joint         P(Exercises and Excellent) = 45/200 = 0.225

Conditional   P(Excellent | Exercises)   = 45/120 = 0.375     ← row total
Conditional   P(Exercises | Excellent)   = 45/60  = 0.750     ← column total

Union         P(Exercises or Excellent)
              = 0.600 + 0.300 − 0.225 = 0.675

Are exercise and health independent?

P(Excellent | Exercises) = 0.375
P(Excellent)             = 0.300

0.375 ≠ 0.300   →   DEPENDENT

Equivalently, if they were independent the expected count in that cell would be 120 × 60 / 200 = 36, not the observed 45. That comparison of observed to expected is the chi-square statistic.

Note

Which total goes in the denominator? The one for the condition. P(A | B) divides by the total for B. Getting this backwards — dividing by the wrong margin — is the most common error in the whole chapter.

Tree Diagrams

A tree makes sequential and conditional probabilities visible. Multiply along branches, add across branch endings.

Two machines produce parts: A makes 60% with a 2% defect rate, B makes 40% with a 5% defect rate.

                  0.02  Defective        P = 0.60 × 0.02 = 0.012
        0.60  A ─┤
       ┌─────────┘ 0.98  Good            P = 0.60 × 0.98 = 0.588
Start ─┤
       └─────────┐ 0.05  Defective       P = 0.40 × 0.05 = 0.020
        0.40  B ─┤
                  0.95  Good             P = 0.40 × 0.95 = 0.380
                                         ─────────────────────────
                                         Total              1.000

P(defective) = 0.012 + 0.020 = 0.032        ← the Law of Total Probability
P(A | defective) = 0.012 / 0.032 = 0.375    ← Bayes' theorem (05-03)

The Law of Total Probability

If B₁, B₂, …, Bₖ partition the sample space (mutually exclusive and exhaustive):

P(A)  =  P(A|B₁)P(B₁) + P(A|B₂)P(B₂) + … + P(A|Bₖ)P(Bₖ)

This is exactly the "add across branch endings" step above, and it is the denominator of Bayes' theorem in 05-03.

Excel

' ── Addition rules ──────────────────────────────────────────────────
=B1+B2                              ' mutually exclusive:  P(A)+P(B)
=B1+B2-B3                           ' general:  P(A)+P(B)−P(A and B)
=4/52+13/52-1/52                    ' P(king or heart) -> 0.3077

' ── Multiplication rules ────────────────────────────────────────────
=B1*B2                              ' independent
=B1*B4                              ' general:  P(A) × P(B|A)
=(13/52)*(13/52)                    ' with replacement    -> 0.0625
=(13/52)*(12/51)                    ' without replacement -> 0.0588

' ── Conditional probability ─────────────────────────────────────────
=B3/B2                              ' P(A|B) = P(A and B) / P(B)

' ── Contingency table (counts in B2:D3, totals in E and row 4) ──────
=SUM(B2:D2)                         ' E2  row total (Exercises)   -> 120
=SUM(B2:B3)                         ' B4  column total (Excellent)-> 60
=B2/$E$4                            ' joint       P(Ex and Exc)   -> 0.225
=E2/$E$4                            ' marginal    P(Exercises)    -> 0.600
=B2/$E2                             ' conditional P(Exc | Ex)     -> 0.375
=B2/B$4                             ' conditional P(Ex | Exc)     -> 0.750
=E2/$E$4 + B4/$E$4 - B2/$E$4        ' union                       -> 0.675

' ── Independence check ──────────────────────────────────────────────
=E2*B4/$E$4                         ' expected count if independent -> 36
=ROUND(B2/$E2,4)=ROUND(B4/$E$4,4)   ' TRUE only if independent

' ── Building the table from raw data ────────────────────────────────
' Insert ▸ PivotTable  →  Rows = exercise, Columns = health, Values = Count
' Show Values As ▸ % of Grand Total  → joint probabilities
' Show Values As ▸ % of Row Total    → P(column | row)
' Show Values As ▸ % of Column Total → P(row | column)
=COUNTIFS(A:A,"Exercises", B:B,"Excellent")     ' one cell, from raw data

R

# ── Contingency table from raw data ────────────────────────────────
tab <- matrix(c(45, 55, 20,
                15, 30, 35), nrow = 2, byrow = TRUE,
              dimnames = list(exercise = c("Yes", "No"),
                              health   = c("Excellent", "Good", "Poor")))
tab
addmargins(tab)                       # with row/column/grand totals

# From a data frame:  tab <- table(df$exercise, df$health)

# ── Probabilities ──────────────────────────────────────────────────
joint <- prop.table(tab)              # every cell / grand total
joint
joint["Yes", "Excellent"]             # 0.225

margin.table(tab, 1) / sum(tab)       # marginal P(exercise)  -> 0.6, 0.4
margin.table(tab, 2) / sum(tab)       # marginal P(health)

prop.table(tab, 1)                    # rows sum to 1 -> P(health | exercise)
prop.table(tab, 2)                    # cols sum to 1 -> P(exercise | health)

prop.table(tab, 1)["Yes", "Excellent"]   # P(Excellent | Exercises) -> 0.375
prop.table(tab, 2)["Yes", "Excellent"]   # P(Exercises | Excellent) -> 0.750

# Union
pA <- sum(tab["Yes", ]) / sum(tab)
pB <- sum(tab[, "Excellent"]) / sum(tab)
pAB <- tab["Yes", "Excellent"] / sum(tab)
pA + pB - pAB                          # 0.675

# ── Independence check ─────────────────────────────────────────────
expected <- outer(rowSums(tab), colSums(tab)) / sum(tab)
expected                               # 36 for the top-left cell
round(joint - outer(rowSums(tab), colSums(tab)) / sum(tab)^2, 4)   # 0 if independent

chisq.test(tab)$expected               # the same expected counts (chapter 12)

# ── Sequential probability (tree) ──────────────────────────────────
pA <- 0.60; pB <- 0.40
pD_A <- 0.02; pD_B <- 0.05
pD <- pA * pD_A + pB * pD_B            # law of total probability -> 0.032
pA * pD_A / pD                         # P(A | defective) -> 0.375

# ── Simulating dependence: draws without replacement ───────────────
set.seed(1)
deck <- rep(c("H", "D", "C", "S"), each = 13)
mean(replicate(20000, {
  draw <- sample(deck, 2)              # no replacement
  all(draw == "H")
}))                                    # ≈ 0.0588

Python

import numpy as np
import pandas as pd

rng = np.random.default_rng(1)

# ── Contingency table ──────────────────────────────────────────────
tab = pd.DataFrame([[45, 55, 20],
                    [15, 30, 35]],
                   index=["Exercises", "Does not"],
                   columns=["Excellent", "Good", "Poor"])
tab
# From raw data:  tab = pd.crosstab(df["exercise"], df["health"])

n = tab.values.sum()                              # 200

# ── Probabilities ──────────────────────────────────────────────────
joint = tab / n                                   # joint probabilities
joint.loc["Exercises", "Excellent"]               # 0.225

tab.sum(axis=1) / n                               # marginal P(exercise)
tab.sum(axis=0) / n                               # marginal P(health)

row_cond = tab.div(tab.sum(axis=1), axis=0)       # P(health | exercise)
col_cond = tab.div(tab.sum(axis=0), axis=1)       # P(exercise | health)

row_cond.loc["Exercises", "Excellent"]            # 0.375
col_cond.loc["Exercises", "Excellent"]            # 0.750

# crosstab does it directly
pd.crosstab(df["exercise"], df["health"], normalize="index")   # rows sum to 1
pd.crosstab(df["exercise"], df["health"], normalize="all")     # joint

# Union
pA  = tab.loc["Exercises"].sum() / n
pB  = tab["Excellent"].sum() / n
pAB = tab.loc["Exercises", "Excellent"] / n
pA + pB - pAB                                     # 0.675

# ── Independence check ─────────────────────────────────────────────
expected = np.outer(tab.sum(axis=1), tab.sum(axis=0)) / n
expected                                          # 36 in the top-left cell

from scipy.stats import chi2_contingency
chi2_contingency(tab)                             # chapter 12

# ── Sequential probability (tree) ──────────────────────────────────
pA, pB = 0.60, 0.40
pD_A, pD_B = 0.02, 0.05
pD = pA * pD_A + pB * pD_B                        # 0.032
pA * pD_A / pD                                    # 0.375

# ── Simulating dependence ──────────────────────────────────────────
deck = np.repeat(["H", "D", "C", "S"], 13)
trials = [np.all(rng.choice(deck, 2, replace=False) == "H") for _ in range(20_000)]
np.mean(trials)                                   # ≈ 0.0588

Quick Reference

Rule Formula When
Complement P(A') = 1 − P(A) Always
Addition (exclusive) P(A)+P(B) P(A∩B) = 0
Addition (general) P(A)+P(B)−P(A∩B) Always
Multiplication (independent) P(A)·P(B) Independence verified
Multiplication (general) P(A)·P(B\|A) Always
Conditional P(A∩B)/P(B) P(B) > 0
Total probability Σ P(A\|Bᵢ)P(Bᵢ) Bᵢ partition S
Task Excel R Python
Build a contingency table PivotTable / COUNTIFS table(x, y) pd.crosstab(x, y)
Joint probabilities count/grand total prop.table(tab) tab / n
Row-conditional count/row total prop.table(tab, 1) normalize="index"
Column-conditional count/col total prop.table(tab, 2) normalize="columns"
Add margins Pivot grand totals addmargins(tab) margins=True
Expected if independent row·col/n outer(rowSums, colSums)/n np.outer(...)/n

Common Mistakes

  • Adding probabilities of events that overlap without subtracting the intersection.
  • Multiplying probabilities of dependent events as if independent — especially "without replacement" problems.
  • Reversing the condition: P(A|B) is generally not P(B|A). P(disease | positive test) and P(positive test | disease) can differ by an order of magnitude (05-03).
  • Confusing mutually exclusive with independent. Two events with non-zero probability that are mutually exclusive are necessarily dependent — if one happens, the other definitely did not.
  • Dividing by the grand total when computing a conditional probability.

Exercises: 05-02: Exercises — Probability Rules and Conditional Probability

⬅️ Previous: 05-01: Probability Basics ➡️ Next: 05-03: Bayes' Theorem and Counting Rules