Debugging: Site Unreachable, Connection Refused vs Timed Out vs Slow

Difficulty: Advanced

Question

Your service is not reachable from a client. How do you debug it? What is the difference between connection refused, connection timed out and a slow response, and which tools do you use at each layer?

Answer

This is a scenario question, and what interviewers look for is method, not memorised commands. The senior habit is to debug bottom up or top down through the layers, forming a hypothesis at each step, and to let the error message narrow the problem. Let's start by decoding the three classic symptoms because they point to entirely different causes.

Connection refused means a TCP RST came back quickly. The packet reached the destination machine (so routing, network and firewall path are basically fine), but nothing is listening on that port, or a firewall is configured with REJECT. Typical causes: the service is not running or crashed, it listens on 127.0.0.1 instead of 0.0.0.0, you are using the wrong port, or a container port is not published. The failure is instant.

Connection timed out means no response at all: the SYN vanished. Typical causes: a firewall or security group silently dropping (DROP rather than REJECT), a routing problem or wrong IP, the host is down, the network path is broken, or the server's SYN queue or backlog is full. It fails after a long wait, often 20 to 130 seconds depending on SYN retries. So refused says reachable but no listener, timed out says something between you and the service is dropping packets.

A slow response means connection works but latency is high. Break it into phases: DNS lookup time, TCP connect time, TLS time, time to first byte (server processing), and download time. This is precisely what curl -w timing variables show. High DNS time points to resolver problems; high connect time to distance, packet loss or congestion; high TLS time to a slow handshake or missing session reuse; high TTFB to the application or database; slow download to bandwidth or large payloads.

Now a systematic checklist. Application layer: does curl -v show a response, and what status code? Is the URL correct, and are certificates valid? Name resolution: dig or nslookup the hostname, compare the answer to different resolvers (dig @8.8.8.8), check for stale caches and wrong records. Reachability: ping and traceroute (remembering ICMP may be blocked), mtr for loss per hop. Port and service: nc -zv host port or curl to see whether the port is open; on the server run ss -tlnp to see what is listening and on which address. Firewalls: check host firewall (iptables), cloud security groups, NACLs, and corporate proxies. Capture packets with tcpdump on both ends: do SYNs arrive at the server, and does a SYN-ACK leave? If SYNs arrive and no reply leaves, it is the server or host firewall; if SYNs never arrive, it is in the network path.

Other classic culprits: MTU problems where small packets work but large ones hang (often on VPNs, fixed by lowering the MTU or MSS clamping), asymmetric routing where replies take another path and are dropped by a stateful firewall, exhausted ephemeral ports or file descriptors, a full connection tracking table on NAT devices, DNS returning IPv6 addresses that are unreachable, and clock skew breaking TLS. Always isolate by testing from multiple vantage points: from the same subnet, from another network, and from the server itself. If it works locally on the server via localhost but not from outside, the problem is binding or the firewall. Document what you tried and change one thing at a time.

Code examples

A layered debug session

$ dig +short api.example.com
203.0.113.20
$ nc -zv 203.0.113.20 443
nc: connect to 203.0.113.20 port 443 (tcp) failed: Connection refused

# on the server
$ ss -tlnp | grep 443
LISTEN 0 511 127.0.0.1:443 0.0.0.0:* users:(("nginx",pid=812))

# nginx bound to loopback only; change listen to 0.0.0.0:443
$ nc -zv 203.0.113.20 443
Connection to 203.0.113.20 443 port (tcp) succeeded!

Refused plus a loopback-only listener shows a binding problem, not a network problem.

Key points

Concepts covered

Troubleshooting, Connection Refused, Timeout, curl, Layered Debugging