Project 3 — Blackjack with Monte Carlo 🃏

Home  |  Notes  |  Exercises  |  Quiz Hub  |  All Projects

Concepts: 5.1 MC prediction, 5.2 MC control & exploring starts

What this shows

Why Monte Carlo is special: it learns purely from simulated games with no knowledge of transition probabilities. We run MC control with exploring starts to learn a near-optimal Blackjack policy — and it reproduces the structure of the famous "basic strategy" chart.

Run it

python blackjack_mc.py                     # 200k episodes
python blackjack_mc.py --episodes 500000   # crisper policy

Pure numpy. Prints the learned hit/stick grid and win/draw/loss rates.

What to look for

• The learned policy says stick on high sums, hit on low sums, with the threshold depending on the dealer's card — matching known optimal play.
• The win rate stays below 50% — Blackjack genuinely favors the dealer, so even optimal play loses on net. The point is that MC found the best achievable policy from experience alone.

Experiments to try

1. Switch to MC prediction: fix the policy "stick on 20–21, else hit" and average returns to estimate the value surface (Example 5.1).
2. Replace exploring starts with an ε-soft policy (Chapter 5.3) — no forced random first action — and compare.
3. Add the usable-ace grid printout and compare it to the no-ace grid; the strategies differ noticeably.