Skip to content

10-03: Exercises

Question

You have a tiny labeled dataset with two classes, plotted on a simple (x, y) grid:

Class A (label = normal): (1, 1), (1, 2), (2, 1) Class B (label = attack): (5, 5), (5, 6), (6, 5)

  1. Are these two classes linearly separable?
  2. Roughly where would the maximum-margin separating line go, and why there specifically rather than somewhere else that also separates the classes?
  3. Which points would likely be the support vectors?

Solution

Step 1: Checking linear separability

Plotting these by hand: Class A sits in a tight little cluster near the origin (around x=1–2, y=1–2), and Class B sits in a tight cluster further away (around x=5–6, y=5–6). There's a wide empty gap between the two clusters with no points from either class in it.

👉 (1) Yes — a single straight line can cleanly separate the two classes, since there's no overlap and no point of one class sits on the "wrong side" relative to the other cluster.

Step 2: Where the maximum-margin line goes

Per 10-05: SVMs for Classification, an SVM doesn't just find any separating line — infinitely many lines could separate these two clusters (a line could be drawn almost anywhere in that empty gap and technically separate them). An SVM specifically finds the line that maximizes the margin — the distance from the line to the closest point of either class.

  • The closest Class A point to Class B is roughly (2, 1).
  • The closest Class B point to Class A is roughly (5, 5).
  • The maximum-margin line sits exactly halfway between these closest points, roughly perpendicular to the line segment connecting them — intuitively, right in the "middle of the empty gap," as far as possible from both nearest points simultaneously.

A line drawn too close to one cluster (say, hugging Class A tightly) would technically still separate the training data correctly, but it would generalize worse — a new, slightly-different Class A point could easily fall on the wrong side. Maximizing the margin is what makes the SVM's decision boundary robust to new, unseen data, not just correct on the exact training points it saw.

Step 3: Identifying the support vectors

Support vectors are the training points that lie closest to the separating line — the ones that actually "support"/define where the maximum-margin boundary sits. Points far from the boundary don't affect its position at all; only the nearest points on each side matter.

Based on the layout: (2, 1) from Class A (closest to the gap) and (5, 5) from Class B (closest to the gap) are the clear candidates — these are the points nearest the empty space between the clusters, and they're the ones whose exact position determines exactly where the maximum-margin line is drawn. The other points — (1,1), (1,2) from Class A and (5,6), (6,5) from Class B — sit farther back from the boundary and could move around somewhat without changing where the optimal separating line goes.

Final Answer

  • The classes are linearly separable — a straight line can cleanly divide them with no overlap.
  • The maximum-margin line runs through the middle of the gap, positioned to be as far as possible from the nearest point of each class simultaneously (not hugging either cluster).
  • The support vectors are the points closest to that gap — approximately (2, 1) from Class A and (5, 5) from Class B — since these are the points that actually determine where the maximum-margin boundary sits.