TCP Three-Way Handshake and Four-Way Connection Teardown

Difficulty: Intermediate

Question

Explain the TCP three-way handshake and the four-way termination. Why three steps to open and four to close? What is a SYN flood?

Answer

Before two machines exchange any data over TCP, they need to agree that both are alive, both can send and receive, and what initial sequence numbers they will use. The three-way handshake does exactly that. Think of it as: Hello, can you hear me? Yes I can hear you, can you hear me? Yes I can. Then we talk.

Step one: the client sends a SYN segment with its initial sequence number, say x. It moves to state SYN_SENT. Step two: the server replies with SYN-ACK: SYN flag set with its own initial sequence number y, and ACK flag with acknowledgement number x+1 (the SYN consumes one sequence number). The server is in SYN_RECEIVED. Step three: the client sends ACK with acknowledgement y+1, and both sides are ESTABLISHED. The client may piggyback data on this third segment. Why not two steps? Because the server would not know that the client received its sequence number, and a stale duplicate SYN from an old connection could trick the server into opening a phantom connection. Random initial sequence numbers, not zero, also make it harder for attackers to guess and inject packets.

Closing is four steps because TCP connections are full-duplex, and each direction is closed independently. The side that finishes first (active closer) sends FIN and enters FIN_WAIT_1. The other side ACKs it and enters CLOSE_WAIT; the active closer moves to FIN_WAIT_2. At this point the connection is half-closed: the passive side may still have data to send. When it is done, the application calls close and it sends its own FIN, moving to LAST_ACK. The active closer ACKs that FIN and goes to TIME_WAIT before finally closing; the passive side closes immediately on receiving that last ACK. So it is FIN, ACK, FIN, ACK. Sometimes it looks like three steps when the passive side has nothing left to send and combines its ACK and FIN in one segment.

There is also RST, an abrupt reset. It is sent when a segment arrives for a port with no listener (this is what produces the connection refused error), when a connection is aborted, or when something is badly out of sync. Contrast with FIN, which is a graceful close.

Now the attack angle. In a SYN flood, an attacker sends a huge number of SYNs, often with spoofed source addresses, and never completes the handshake. The server allocates state for each half-open connection in its SYN queue, the queue fills, and legitimate users are denied service. Defences include SYN cookies (the server encodes the connection info in the sequence number of its SYN-ACK and allocates no memory until a valid ACK returns), shrinking timeouts, increasing the backlog, and rate limiting at the firewall or CDN.

A few details interviewers like: the maximum segment size and window scale options are negotiated in the SYN and SYN-ACK; each side's initial sequence number is chosen independently; the connection opening costs one round-trip time before data can flow, which is why TLS over TCP needs additional round trips and why TCP Fast Open and QUIC try to shave latency. In netstat, states like SYN_RECV piling up suggest a flood, while lots of CLOSE_WAIT suggests an application bug where the program is not calling close on its sockets.

Code examples

Handshake and teardown in a timeline

Client                              Server
  |--- SYN seq=100 ----------------->|   SYN_SENT / LISTEN
  |<-- SYN-ACK seq=300 ack=101 -------|   SYN_RECEIVED
  |--- ACK ack=301 ------------------>|   ESTABLISHED both
  ...data...
  |--- FIN seq=500 ------------------>|   FIN_WAIT_1
  |<-- ACK ack=501 --------------------|   CLOSE_WAIT
  |<-- FIN seq=700 --------------------|   LAST_ACK
  |--- ACK ack=701 ------------------->|   client TIME_WAIT

The SYN and FIN flags each consume one sequence number, which is why acks are seq+1.

Watching handshake with tcpdump

$ sudo tcpdump -n -i eth0 'tcp port 80 and (tcp[tcpflags] & (tcp-syn|tcp-fin) != 0)'
IP 10.0.0.5.51000 > 93.184.216.34.80: Flags [S], seq 100
IP 93.184.216.34.80 > 10.0.0.5.51000: Flags [S.], seq 300, ack 101
IP 10.0.0.5.51000 > 93.184.216.34.80: Flags [F.], seq 500

S is SYN, dot means ACK, F is FIN.

Key points

• Handshake: SYN, SYN-ACK, ACK.
• Teardown: FIN, ACK, FIN, ACK, because each direction closes separately.
• SYN cookies defend against SYN floods.
• RST aborts a connection abruptly; FIN closes gracefully.

Concepts covered

TCP Handshake, SYN, FIN, Connection States, SYN Flood