🛡️ 04-05: Mitigations — SYN Cookies & Randomization¶

📌 Overview¶

This module has covered four related attacks — SYN flooding, IP spoofing, TCP session hijacking, and RST injection — all of which trace back to the same two root causes:

TCP servers must hold state for incomplete connections (enabling SYN floods).
TCP has no cryptographic authentication of packets within a connection, relying only on numeric fields like sequence numbers (enabling spoofing, hijacking, and RST injection).

This lesson rounds up the practical defenses against both problems.

🍪 SYN Cookies: Defeating SYN Floods Without Holding State¶

The Problem, Restated¶

A SYN flood works because the server must remember each half-open connection (source IP/port, sequence number, etc.) somewhere, and that memory (the backlog queue) is finite. Fill the queue, and legitimate connections get rejected. See 04-01: TCP SYN Flooding for the full attack mechanics.

The Clever Fix: Don't Remember Anything¶

SYN cookies solve this with an elegant trick: instead of storing connection state in memory when a SYN arrives, the server encodes the state into the SYN-ACK's sequence number itself, then simply forgets everything about the connection until (and unless) the client responds.

How SYN Cookies Work, Step by Step¶

Client sends a SYN. Normally, the server would allocate a queue entry recording this attempt.
Instead, the server computes a special "cookie" — derived from a cryptographic hash of key details: the client's IP, port, the server's IP, port, a coarse timestamp, and a locally-kept secret value. This hash becomes the Initial Sequence Number (ISN) the server puts into its SYN-ACK reply.
The server sends the SYN-ACK and forgets the connection entirely — no backlog entry, no memory consumed, nothing to exhaust.
If the client is legitimate, it replies with an ACK, whose acknowledgment number is (cookie + 1) — since real TCP clients always increment the peer's sequence number by one to acknowledge it.
The server recomputes the cookie from the same inputs (source IP/port, etc., which are visible in the returning ACK packet) and checks whether it matches what's encoded in the ACK. If it matches, the server knows this ACK is a legitimate response to a SYN-ACK it actually sent — and only now does it allocate real connection state and complete the handshake.
If it's an attacker's spoofed SYN, no ACK ever arrives (the spoofed source doesn't know a SYN-ACK was even sent to it) — and since the server never stored anything in the first place, there's nothing to clean up or exhaust.

Normal server:  SYN → [allocate queue slot] → SYN-ACK → wait → ACK → [use stored slot]
                                 ▲
                          exhausted by flooding

SYN cookie:     SYN → [compute cookie, store NOTHING] → SYN-ACK → wait → ACK
                                                                     │
                                                     [recompute cookie, verify, THEN allocate]

💡 The key insight: the server turns the sequence number field into a self-verifying token. It doesn't need to remember anything about who it talked to, because the proof of a legitimate handshake is baked directly into a value the real client will echo back.

Trade-offs of SYN Cookies¶

SYN cookies aren't free — encoding all connection details into 32 bits of sequence number space means some information (like certain TCP options negotiated in the original SYN) can't be perfectly preserved, which is why most systems only enable SYN cookies automatically once they detect the backlog queue filling up, rather than using them unconditionally for every connection.

Checking / Toggling SYN Cookies (Linux example)¶

# Check whether SYN cookies are enabled
sysctl net.ipv4.tcp_syncookies

# Enable them
sudo sysctl -w net.ipv4.tcp_syncookies=1

🎲 Initial Sequence Number (ISN) Randomization¶

The Problem¶

Both blind session hijacking and blind RST injection (see 04-03: TCP Session Hijacking and 04-04: RST Injection) depend on an attacker being able to guess a valid sequence number without ever observing the real traffic. If a server's ISN generation is predictable — for example, a simple counter that increments steadily over time — an off-path attacker can estimate the current sequence number with enough accuracy to forge a working packet.

The Fix¶

Modern operating systems generate ISNs using a cryptographically strong pseudo-random process, seeded with connection-specific details (source/destination IP and port) and a secret value, so that:

Sequence numbers appear statistically random to an outside observer
Knowing the ISN for one connection gives an attacker no useful information about the ISN of a different connection
The search space to guess a valid sequence number becomes computationally impractical (the sequence number field is 32 bits — over 4 billion possibilities — and modern randomization spreads real usage across that entire space unpredictably)

💡 This is the same principle as 06-06: The Kaminsky Attack's discussion of DNS transaction ID randomization — a small, guessable space becomes exploitable, so defenders widen and randomize it.

👉 Importantly, ISN randomization does not help against on-path attackers who can simply sniff the real sequence number — it only raises the bar for blind attacks launched from off-network.

⏱️ TCP Timestamps¶

The TCP timestamp option (RFC 7323) was originally designed for performance reasons (round-trip time measurement and protecting against wrapped sequence numbers on very fast connections), but it has a secondary security-relevant side effect: it adds another field that must be correctly guessed or observed for an injected packet to be accepted by some stricter TCP stack configurations, incrementally raising the difficulty of blind injection attacks. It's a minor, supporting mitigation rather than a primary defense on its own.

🧱 Firewalls & Rate-Limiting SYN Floods¶

Beyond protocol-level tricks like SYN cookies, network-level defenses reduce the volume of attack traffic that ever reaches a vulnerable service:

Rate limiting — capping how many new connection attempts a single source IP (or the server overall) can initiate per second, throttling flood traffic before it exhausts resources.
SYN proxies / firewalls — a firewall or load balancer can complete the three-way handshake on behalf of the real server, only forwarding a connection upstream once it's verified as legitimate — effectively applying the SYN-cookie idea at the network edge instead of the end host.
Blackholing / scrubbing services — for large-scale (often distributed, i.e. DDoS) floods, traffic can be redirected through specialized infrastructure designed to absorb and filter attack volume before it reaches the target.

Module 05 covers firewalls and intrusion prevention in much greater depth, including how rate-limiting and traffic filtering rules are actually configured.

🔒 Encryption (TLS) as a Defense Against Hijacking & Injection¶

It's worth being precise about what encryption does and doesn't fix here:

TLS does not prevent a RST injection attack from tearing down a connection — a forged RST is a transport-layer (TCP) event, and TLS operates above that layer. The connection can still be forcibly reset.
TLS does prevent meaningful session hijacking of the application data. Even if an attacker successfully injects a TCP segment with the correct sequence number, the payload they'd need to inject is encrypted application data they cannot forge without the session's cryptographic keys — so there's no way to smuggle in a malicious command the way there was against plaintext Telnet in 04-03: TCP Session Hijacking.
TLS provides integrity protection, meaning any tampering with in-flight encrypted data is detected and the connection is aborted, rather than silently accepting altered content.

Module 09 covers exactly how TLS achieves this (handshakes, key exchange, message authentication) in full detail.

🧭 Defense Summary Table¶

Attack	Primary Defense	Mechanism
SYN flooding	SYN cookies	Encode connection state into the SYN-ACK sequence number instead of storing it
SYN flooding	Rate limiting / SYN proxies	Reduce or filter attack volume at the network edge
Blind session hijacking / RST injection	ISN randomization	Makes sequence numbers computationally impractical to guess
On-path hijacking / RST injection	Encryption (TLS/SSH)	Attacker can't forge meaningful encrypted payloads even with a correct sequence number
All of the above	TCP timestamps	Adds a secondary field that raises the bar for blind injection

📌 Key Takeaways¶

SYN cookies defeat SYN flooding by encoding connection state into the SYN-ACK's sequence number instead of storing it server-side, so a flood of never-completed SYNs costs the server nothing to "forget."
The server only allocates real connection resources after verifying the returning ACK's cookie matches what it would have sent — proof the client legitimately received the SYN-ACK.
ISN (Initial Sequence Number) randomization makes blind session hijacking and blind RST injection computationally impractical by removing predictability from sequence numbers.
ISN randomization does not help against on-path attackers who can simply sniff real sequence numbers.
TCP timestamps offer a minor secondary layer of difficulty against blind injection.
Firewalls, rate limiting, and SYN proxies reduce attack volume at the network edge — covered further in Module 05.
TLS/SSH encryption can't stop a connection from being reset, but it does prevent an attacker from injecting meaningful, valid application data into a hijacked session — full details in Module 09.