TCP Internals Every Network Engineer Should Know
The three-way handshake is the least interesting part. Flow control, congestion control, retransmission, and windowing are where network problems actually live.
Beyond the handshake
Everyone learns "SYN, SYN-ACK, ACK." That handshake is three packets out of millions; the engineer's job lives in the rest. TCP is two clever ideas stacked: reliability (sequence numbers, acknowledgments, retransmission) and adaptation (flow control for the receiver, congestion control for the network). When transfers crawl, one of these two is mis-set.
Sequence numbers are the protocol's memory
Each TCP segment carries a sequence number (which byte this segment starts at, in a 32-bit space) and usually an acknowledgment number ("I have received every byte before this one"). Because ACKs are cumulative, one ACK can confirm a whole batch — and a missing segment shows up as the receiver repeating the same ACK number: duplicate ACKs are the smoking gun of loss or reordering.
In Wireshark, turn on "relative sequence numbers" and the handshake tells its story cleanly: SYN 0 → SYN/ACK 0/1 → ACK 1. After that, sequence numbers climb by the number of data bytes sent. If you can read this, packet captures stop being noise.
Flow control: the receive window
The receive window in each ACK says "I have room for this many bytes." The sender must never have more unacknowledged data in flight than the window. If the receiver is slow (or its buffer is small), the window shrinks — possibly to zero, freezing the connection — and advertised shrinking windows explain transfers that start fast and die at 99%.
Congestion control reacts to the network
While flow control protects the receiver, congestion control protects the network. The sender maintains a congestion window (cwnd) alongside the receive window and uses the smaller of the two. The phases:
- Slow start — exponential growth (doubling per RTT) until a threshold or the first loss. Lightning-fast ramp, deliberately reckless probing.
- Congestion avoidance — on each RTT add ~1 MSS (per ACK: 1/cwnd), i.e., linear, careful growth.
- Loss response — triple duplicate ACKs trigger fast retransmit (skip the timer) and halve the window; a full timeout collapses it to one segment and restarts slow start.
Modern variants (CUBIC, BBR) differ in how they probe for capacity, but the invariant holds: one loss event collapses throughput for at least one RTT. On long, fat pipes that's the whole game.
Windowing: why fast + far = tricky
The formula that ends careers if ignored:
MSS (max segment size) is negotiated per direction from the path's MTU: typically 1500 − 20 (IP) − 20 (TCP) = 1460 bytes. Where MTU drops mid-path and ICMP is filtered (the dreaded PMTU black hole), connections hang instead of fragmenting — the fix is MSS clamping on the router doing encapsulation, not endless retransmits.
The flag vocabulary
| Flag | What it signals | What it looks like broken |
|---|---|---|
| SYN / ACK / FIN | Open, acknowledge, close | SYN without SYN/ACK: the service is down or filtered, not the network |
| RST | "This connection does not exist" — abrupt abort | Sprays during troubleshooting point at firewalls killing sessions |
| PSH | "Deliver this to the app now" (normal on interactive/app data) | — |
| ECE / CWR | ECN congestion signaling — "I felt congestion" without dropping | Blocked by old firewalls that strip the bits |
Tears in the fabric: TIME_WAIT and keepalives
TIME_WAIT (2×MSL, typically 60 s) after closing guards against delayed duplicates corrupting the next connection — exhaust TIME_WAIT ports and new outbound connections start failing, the signature being "works, then suddenly no new connections." Keepalives solve the opposite problem: detecting a dead peer without ending idle connections, though many applications disable them and reschedule idle-timeout battles with their load balancers instead.
The five-minute TCP health check
- Is the handshake completing? (SYN ⇒ SYN/ACK ⇒ ACK)
- Are duplicate ACKs/retransmissions spiking? (loss, reordering, or buffer storms)
- Is the window shrinking to zero? (slow receiver or application)
- Is window scaling on, and is the window large enough for this RTT? (run the BDP calculator)
- Are there RSTs? (who sent them, and was it a policy decision?)
- Sequence and ACK numbers are the protocol's memory — learn to read them.
- The receive window is flow control (per connection); congestion control reacts to the network.
- Retransmissions and duplicate ACKs are the signature of congestion or loss.
- Window scaling is required to fill any fast or long pipe.