How to Investigate TCP Retransmissions on Linux

TCP retransmissions on Linux are a normal part of TCP communication, but an excessive number of retransmitted packets often indicates a problem somewhere between the application and the remote endpoint. Packet loss, congestion, overloaded systems, faulty interfaces, bad cabling, MTU mismatches, and unstable network paths can all produce similar symptoms.

When investigating TCP retransmissions on Linux, the challenge is rarely detecting that retransmissions exist. The real task is determining where they originate, whether they are occasional or persistent, and whether they are affecting application performance. Looking at only one command usually provides an incomplete picture.

A useful investigation combines kernel TCP counters, interface statistics, socket information, packet captures, and system performance data. In this article, we will use practical Linux tools such as nstat, ss, ethtool, and tcpdump to identify TCP retransmissions and narrow down their possible causes.

What Is a TCP Retransmission?

TCP is designed to provide reliable delivery between two network endpoints. When a sender transmits a TCP segment, it expects the receiving system to acknowledge the data.

If the expected acknowledgment does not arrive within a certain period, or if TCP detects missing data through duplicate acknowledgments, the sender may transmit the segment again.

That retransmitted segment is called a TCP retransmission.

A small number of retransmissions is not necessarily unusual. Networks occasionally drop packets, wireless links may experience interference, and short periods of congestion can occur even in healthy environments.

The concern begins when retransmissions become frequent enough to affect throughput, latency, or application behavior.

Typical symptoms include:

• Slow file transfers;
• Intermittent application delays;
• SSH sessions that briefly freeze;
• API requests taking longer than expected;
• Database connections showing inconsistent response times;
• Reduced throughput despite available bandwidth;
• Connections repeatedly backing off and recovering.

Note: The important point is that retransmissions are usually a symptom, not the root cause.

Start with Linux TCP Statistics

A good first step is to determine whether the operating system is actually recording TCP retransmissions. One of the simplest commands is:

netstat -s

On systems where the net-tools package is not installed, the equivalent information can usually be obtained with:

ss -s

For more detailed TCP counters, nstat is particularly useful:

nstat -az

To focus on retransmission-related counters:

nstat -az | grep -i retrans

You may see counters such as:

TcpRetransSegs
TcpExtTCPFastRetrans
TcpExtTCPSlowStartRetrans
TcpExtTCPLostRetransmit

The most immediately useful value is normally TcpRetransSegs.

For example:

TcpRetransSegs  18452  0.0

The absolute number by itself is not enough to determine whether a problem exists because these counters normally accumulate since boot. What matters is how quickly the value increases.

Run:

nstat -az | grep TcpRetransSegs

Wait several seconds and run it again. If the counter is increasing rapidly during the period when users are experiencing problems, you now have evidence that retransmissions are actively occurring.

For repeated monitoring, you can use:

watch -n 2 "nstat -az | grep TcpRetransSegs"

This provides a simple way to observe whether retransmissions increase during a test or application workload.

Compare Retransmissions with TCP Traffic

A retransmission count is more meaningful when compared with the total number of TCP segments being transmitted.

For example, 1,000 retransmissions may look serious, but the interpretation is very different if the server transmitted several hundred million TCP segments during the same period.

Use:

nstat -az | egrep 'TcpOutSegs|TcpRetransSegs'

Example:

TcpOutSegs       9853421
TcpRetransSegs      1287

A rough retransmission ratio can be estimated by comparing retransmitted segments with transmitted segments. The goal is not to enforce a universal percentage threshold. Acceptable values depend heavily on the environment.

A backend server inside the same data center would normally be expected to experience fewer packet losses than a system communicating with clients over the public Internet.

Instead of focusing only on a percentage, look for changes. If retransmissions suddenly rise from nearly zero to thousands during a performance incident, that change is much more valuable than an isolated lifetime counter.

Identify Which Connections Are Affected

System-wide counters tell you that retransmissions exist, but they do not tell you which application or destination is involved. The ss command can expose useful TCP information for active connections.

Run:

ss -ti

The -t option selects TCP sockets, while -i displays internal TCP information.

A connection may display information similar to:

cubic wscale:7,7 rto:204 rtt:2.841/0.712
mss:1448 cwnd:10 bytes_sent:84732 bytes_acked:83320
bytes_retrans:1412

The bytes_retrans field is particularly useful because it shows retransmitted data associated with the socket.

Other values worth observing include:

• rtt — estimated round-trip time
• rto — retransmission timeout
• cwnd — congestion window
• bytes_sent — transmitted data
• bytes_acked — acknowledged data
• bytes_retrans — retransmitted data

To inspect a specific destination, you can filter the output. For example:

ss -ti dst 192.168.20.50

Or for connections using a particular port:

ss -ti dport = :443

This can help determine whether retransmissions are concentrated on one server, one service, or one network path.

Check Interface Errors and Dropped Packets

Once retransmissions are confirmed, inspect the network interfaces.

Start with:

ip -s link

The output includes received and transmitted packets together with errors and dropped packets.

Pay attention to values such as:

RX:
bytes packets errors dropped missed mcast

TX:
bytes packets errors dropped carrier collsns

Errors that steadily increase deserve attention. Depending on the environment, they may indicate:

• Physical interface problems;
• Driver problems;
• Bad network cables;
• Faulty switch ports;
• Duplex-related problems;
• Hardware errors;
• Receive buffer exhaustion.

Dropped packets also require investigation, although they do not always indicate a physical network failure. Linux may drop packets because the system cannot process them quickly enough.

To watch interface counters repeatedly – replace eth0 to your interface:

watch -n 2 "ip -s link show eth0"

If interface drops increase at the same time as TCP retransmissions, the correlation provides an important clue.

Inspect NIC-Level Statistics with ethtool

The standard interface counters do not expose every hardware-specific condition.

Use:

ethtool -S eth0

The exact counters depend on the network adapter and driver, but common examples include:

rx_errors
tx_errors
rx_dropped
tx_dropped
rx_crc_errors
rx_missed_errors
rx_no_buffer_count

CRC errors are particularly important because they may indicate a physical-layer problem.

For example, increasing CRC errors can point toward:

• Damaged cabling;
• Optical transceiver problems;
• Bad switch ports;
• Electrical interference;
• Physical link instability.

Also verify the negotiated link parameters:

ethtool eth0

Look for values such as:

Speed: 10000Mb/s
Duplex: Full
Link detected: yes

A server expected to operate at 10 Gb/s but negotiating at 1 Gb/s clearly requires further investigation, even if retransmissions are not directly caused by that condition.

Check Whether the System Is Overloaded

Not every retransmission problem originates in the network. A heavily loaded Linux server may delay packet processing enough to produce behavior that looks like network instability.

Check system load:

uptime

Inspect CPU usage:

top

Pay particular attention to sustained CPU saturation.

Also check soft interrupts:

mpstat -P ALL 2

The %soft column can help identify significant softirq activity.

For deeper visibility:

cat /proc/softirqs

Network packets are processed through kernel interrupt and softirq mechanisms. If CPUs handling network traffic become overloaded, packets may be delayed or dropped before applications ever see them.

Memory pressure should also be considered:

free -m

And:

vmstat 2

A server under severe CPU, memory, or I/O pressure may indirectly contribute to TCP performance problems.

Look for Kernel Network Drops

Linux exposes useful networking statistics under /proc.

One useful command is:

cat /proc/net/softnet_stat

The values are hexadecimal and not especially friendly to read manually, but they provide information about packet processing at the kernel level.

The second column represents packets dropped because the networking backlog could not process them quickly enough.

A simpler way to observe softnet behavior on systems with sar available is:

sar -n SOFT 2 10

Fields may include:

total/s
dropd/s
squeezd/s
rx_rps/s
flw_lim/s

An increasing dropd/s value may indicate that packets are being dropped inside the Linux networking stack rather than on the physical network.

Important: This distinction matters because replacing a network cable will not solve a host-side packet-processing bottleneck.

Capture the Traffic with tcpdump

When statistics indicate retransmissions but the cause remains unclear, packet capture becomes extremely valuable.

Capture traffic for the affected host – in this case, for instance, 192.168.20.50 is the IP address of the affected host, using the eth0 interface:

tcpdump -i eth0 host 192.168.20.50 -w tcp-analysis.pcap

For a specific TCP port:

tcpdump -i eth0 port 443 -w tcp-analysis.pcap

Avoid capturing everything on a heavily loaded production interface for long periods. Filter the capture as narrowly as possible. The resulting file can be analyzed with Wireshark.

Wireshark can identify events such as:

TCP Retransmission
TCP Fast Retransmission
TCP Dup ACK
TCP Out-Of-Order
TCP Previous segment not captured

A useful Wireshark display filter is:

tcp.analysis.retransmission

For fast retransmissions:

tcp.analysis.fast_retransmission

Duplicate acknowledgments can be displayed with:

tcp.analysis.duplicate_ack

💡 Packet captures provide something that Linux counters cannot: the sequence of events leading to the retransmission.

Understand Fast Retransmissions

Not every retransmission waits for a timeout. TCP can detect packet loss when the receiver repeatedly acknowledges data while indicating that an earlier segment is missing.

This may trigger a fast retransmission. Conceptually, the sequence may look like:

Segment 1 sent
Segment 2 sent
Segment 3 lost
Segment 4 sent
Segment 5 sent
Duplicate ACKs received
Segment 3 retransmitted

Frequent fast retransmissions commonly indicate packet loss somewhere along the network path. They can also appear when packets are heavily reordered.

This is why Wireshark should not be interpreted mechanically. A packet marked as retransmitted does not automatically prove that the sender, receiver, or immediate switch dropped it. The loss may have occurred anywhere in between.

Verify MTU and Path MTU Problems

MTU mismatches can create confusing TCP behavior, particularly when jumbo frames are used.

Check the local interface:

ip link show eth0

You may see:

mtu 1500

Or in a jumbo-frame environment:

mtu 9000

To test packet size without fragmentation:

ping -M do -s 1472 192.168.20.50

With IPv4, 1472 bytes of payload plus 28 bytes of IP and ICMP headers produces a 1500-byte packet.

For jumbo frames, you might test:

ping -M do -s 8972 192.168.20.50

Failure at larger packet sizes while smaller packets succeed may indicate an MTU inconsistency somewhere along the path.

These problems are especially common in virtualized environments where the physical switch, virtual switch, VLAN, VMkernel-style interface, or guest operating system may use different MTU values.

Compare Multiple Network Paths

If possible, test communication to multiple destinations. Suppose retransmissions occur only when communicating with:

10.20.30.40

but not with several other systems on the same local network. That suggests the local interface itself may not be the primary problem.

Run:

ping 10.20.30.40

Then compare with another host:

ping 10.20.30.41

For routed paths, use:

tracepath 10.20.30.40

tracepath can also provide useful path MTU information.

If available, mtr can reveal latency and apparent packet loss across a routed path:

mtr 10.20.30.40

However, intermediate router loss must be interpreted carefully. Many routers rate-limit or deprioritize ICMP responses while forwarding normal traffic without problems.

The destination behavior is usually more important than isolated loss reported at one intermediate hop.

Correlate the Evidence

The strongest TCP retransmission investigations are based on correlation rather than a single command.

Imagine the following situation:

TcpRetransSegs: increasing rapidly
ip -s link: no errors
ethtool -S: no CRC errors
sar -n SOFT: increasing packet drops
CPU usage: near 100%

That evidence points more strongly toward host-side packet processing pressure than a damaged Ethernet cable.

Now consider another case:

TcpRetransSegs: increasing
ip -s link: RX errors increasing
ethtool -S: CRC errors increasing
CPU usage: normal
softnet drops: zero

This pattern makes a physical or Layer 2 problem much more likely.

The investigation should follow the evidence rather than beginning with a predetermined assumption that every retransmission is caused by congestion.

A Practical Investigation Sequence

When investigating a real incident, the following sequence works well:

nstat -az | grep -i retrans

Confirm that retransmissions are currently increasing.

Then inspect active connections:

ss -ti

Identify the remote systems and applications showing retransmitted data.

Check the interface:

ip -s link

Inspect hardware-level statistics:

ethtool -S eth0

Check host pressure:

mpstat -P ALL 2
vmstat 2
sar -n SOFT 2

Verify MTU if the environment uses non-default values:

ip link show
ping -M do -s 1472 <destination>

Finally, capture the affected traffic when more detail is required:

tcpdump -i eth0 host <destination> -w tcp-analysis.pcap

This progression moves from inexpensive system-wide checks toward increasingly detailed analysis without immediately introducing unnecessary complexity.

Treat Retransmissions as Evidence, Not the Diagnosis

TCP retransmissions are valuable because they tell you that reliable delivery required TCP to recover from missing or delayed data. They do not, by themselves, identify where the failure occurred.

The underlying problem could be a physical interface, a congested network path, a firewall, an overloaded Linux host, a virtual switch, an MTU mismatch, packet reordering, or even a remote system unable to process incoming traffic quickly enough.

A disciplined Linux investigation therefore combines TCP counters, socket data, interface statistics, NIC diagnostics, kernel drop counters, host performance information, and packet captures. Once these different layers begin telling the same story, the retransmission counter stops being just another number and becomes a useful path toward finding the actual problem.