1. Congestion Control: BBR vs Cubic
TCP is not a fixed protocol; it is an Evolving Algorithm. The math of how a connection grows its window determines how it handles congestion.
The Cube and the Model
Cubic (Loss-Based)
Cubic fills the network buffers until a packet drops. This causes 'Bufferbloat'—increased RTT for everyone sharing the link. Use case: Standard web traffic.
BBR (Rate-Based)
BBR measures the physical bottleneck bandwidth. It paces packets to never exceed the pipe's capacity, keeping buffers empty and RTT at the minimum physical limit. Use case: Youtube / Netflix.
2. QUIC Forensics: Connection Migration
Standard TCP dies when your IP changes. QUIC (HTTP/3) survives by decoupling the connection from the network address using Connection Identifiers (CIDs).
Bit-Level Migration Detection
When an IP switch occurs, the new UDP source IP is seen. Forensics tools must look for the PATH_CHALLENGE frame. If the Destination CID matches the previous flow, it is a Migration event, not a new session. Tracking this allows for seamless session reconstruction across heterogeneous networks.
Payload: PATH_CHALLENGE [Data=0x4a2...]
3. Head-of-Line Blocking: The Invisible Wall
In a TCP stream, packet #1 is the gatekeeper. If it drops, packets #2 through #1,000 are held in the kernel buffer, effectively frozen.
SCTP (Multi-Streaming)
Loss in Stream A has ZERO impact on Stream B. Each stream has its own head of line. Perfect for signaling/voice.
MPTCP (Multipath)
Splits traffic across paths, but forces everyone back into a single sequence. One slow path kills the global speed.
Frequently Asked Questions
Technical Standards & References
Related Engineering Resources
TCP Congestion Control Algorithms: Reno, CUBIC, BBR, and Beyond
The fundamental difference between TCP and UDP is not just reliability versus speed; it is TCP's sophisticated congestion control machinery that governs how fast data can be sent without overwhelming the network. TCP congestion control has evolved dramatically since the original Tahoe implementation in the 1980s, and understanding this evolution is essential for any network engineer who needs to optimize throughput on modern networks. The original TCP Reno uses a slow-start phase where the congestion window (cwnd) doubles every round-trip time (RTT) until a packet loss is detected, at which point it enters congestion avoidance and increases cwnd linearly (by one segment per RTT) until the next loss. This "additive increase, multiplicative decrease" (AIMD) behavior produces the classic TCP sawtooth pattern where throughput oscillates between the loss-triggered ceiling and half that value. On a typical 100 Mbps link with 20 ms RTT, TCP Reno achieves an average throughput of approximately 75 Mbps, wasting 25% of the available bandwidth due to the recovery mechanism from loss events.
CUBIC, which has been the default TCP congestion control algorithm in the Linux kernel since version 2.6.19 (2007), represents a significant advancement over Reno for high-bandwidth, high-latency networks. Instead of using linear growth in congestion avoidance, CUBIC uses a cubic function of time since the last loss event, which allows it to scale to very high bandwidths without requiring tens of thousands of RTTs to recover from a loss event. The cubic function grows very rapidly immediately after a loss event (probing for available bandwidth) but flattens out near the previous loss window size to avoid causing another loss. For a 10 Gbps link with 100 ms RTT (a transcontinental path), TCP Reno would require approximately 2 hours to fully recover utilization after a loss event; CUBIC achieves the same recovery in approximately 30 seconds. The difference is so dramatic that the Internet Engineering Task Force (IETF) standardized CUBIC as the recommended TCP congestion control algorithm in RFC 8312 (2018), and it is now the default on virtually all Linux servers, including the majority of web servers and cloud instances.
Google's BBR (Bottleneck Bandwidth and Round-trip propagation time) congestion control algorithm, introduced in 2016, represents a paradigm shift from loss-based congestion control to model-based congestion control. Instead of waiting for packet loss to signal congestion, BBR continuously estimates the bottleneck bandwidth (BtlBw) and the round-trip propagation time (RTprop) and paces outgoing traffic at a rate that keeps the bottleneck queue small—typically less than one RTT's worth of buffering. The key insight of BBR is that packet loss is a poor congestion signal on modern networks, where buffer sizes can be very large (bufferbloat) or very small (in data center switches). A loss-based algorithm like CUBIC would fill a large buffer completely before detecting loss, causing hundreds of milliseconds of queuing delay (bufferbloat), while on a shallow-buffered switch, CUBIC would detect loss at a very low throughput and never utilize the available bandwidth. BBR avoids both extremes by modeling the path characteristics directly and pacing at the model-predicted optimal rate, achieving 2–10x throughput improvement over CUBIC on paths with significant bufferbloat, while simultaneously reducing queuing delay by 10–100x.
The choice of congestion control algorithm has direct implications for the performance of different applications over TCP. For real-time video streaming, BBR provides a clear advantage because it maintains low and stable queuing delay, which directly translates to lower video startup latency and fewer rebuffering events. For bulk data transfer (such as server backups), CUBIC can actually outperform BBR because CUBIC fills available buffer capacity aggressively, achieving slightly higher average throughput on paths with moderate buffer sizes—at the cost of increased latency for other flows sharing the same bottleneck. The network engineer must therefore consider the application mix when configuring the TCP congestion control algorithm on servers. For a general-purpose web server serving a mix of interactive and bulk traffic, BBR is usually the best choice because its lower latency benefits interactive traffic significantly more than its slightly lower throughput penalizes bulk traffic. For dedicated bulk transfer servers (such as file servers or database replication links), CUBIC or even a custom high-throughput variant may be preferable to maximize throughput at the expense of latency.
UDP, by contrast, has no built-in congestion control, which is both its greatest advantage and its greatest risk. Because UDP applications control their own sending rate, they can achieve much higher throughput than TCP on a given path—but they can also completely saturate the bottleneck link, causing catastrophic packet loss for all flows, including TCP flows that politely back off in response to the loss. This is why the IETF has standardized Datagram Congestion Control Protocol (DCCP, RFC 4340) as a congestion-controlled alternative to UDP for applications that need the low-latency characteristics of UDP but the network-friendly behavior of TCP congestion control. DCCP is used by some streaming media applications and online games, but it has not achieved wide adoption because most operating systems and middleboxes do not support it. The practical consequence is that high-throughput UDP-based applications—such as video streaming with QUIC, real-time communications with WebRTC, and online gaming—implement their own congestion control algorithms at the application layer. QUIC, for example, uses a modified version of CUBIC or BBR implemented in the QUIC transport layer, giving it the network-friendly congestion control behavior of TCP with the low-latency characteristics of UDP. This application-layer congestion control, while effective, creates a fragmented ecosystem where each application implements its own congestion control algorithm with varying degrees of quality and network-friendliness, representing a significant challenge for the long-term stability of the internet's congestion control ecosystem.
Protocol Overhead Analysis: TCP and UDP Header Mechanics at Scale
A detailed byte-level comparison of TCP and UDP headers reveals why the choice between the two protocols has practical implications beyond reliability and ordering. The UDP header is a model of minimalism: just 8 bytes consisting of source port (2 bytes), destination port (2 bytes), length (2 bytes), and checksum (2 bytes). The TCP header is a minimum of 20 bytes (without options) and can extend to 60 bytes with the maximum options field. The 12-byte difference (20–8 bytes per segment) seems trivial for a single packet, but at scale, this overhead compounds dramatically. A server handling 1 million TCP connections per second with a typical packet size of 1,500 bytes would transmit 20 GB of TCP header overhead per second, compared to 8 GB for UDP header overhead—a 12 GB per second difference in total link utilization. For a 40 Gbps (5 GB/s) network interface, the TCP header overhead consumes 4% of the available bandwidth purely for header data that carries no application payload.
The TCP options field is a significant and often overlooked contributor to header overhead. The most commonly used TCP options are: Maximum Segment Size (MSS) in the SYN packet (4 bytes), Window Scale (3 bytes), Selective Acknowledgment (SACK) permitted (2 bytes), and Timestamps (10 bytes). The Timestamps option, enabled by default in most modern TCP stacks, adds 10 bytes to every TCP segment—not just the SYN. For a typical web server sending 1,460-byte TCP segments (the MSS for standard Ethernet), the 10-byte timestamp option adds 0.68% overhead beyond the 20-byte base header. For VoIP traffic over TCP (which is rare but possible), where packets are typically 160 bytes, the timestamp overhead jumps to 6.25% of the total transmission. The TCP Timestamps option serves the important function of protecting against wrapped sequence numbers (PAWS) on high-speed links (where the 2³² sequence number space can wrap in under 30 seconds at 10 Gbps) and enables more accurate RTT estimation, but its overhead cost is real and must be accounted for in capacity planning for high-volume services.
The UDP checksum behavior introduces an interesting asymmetry between the two protocols. TCP mandates a checksum covering both the TCP header and payload, and it is calculated over a pseudo-header that includes the source and destination IP addresses, the protocol number, and the segment length. If the checksum validation fails, the segment is silently discarded—no error is reported to the sender, and the reliable delivery mechanism of TCP (retransmission triggered by a missing ACK) eventually recovers the lost data. UDP checksum handling varies by implementation: IPv4 allows UDP checksum to be set to zero (indicating that the checksum is not used), while IPv6 requires the UDP checksum to be present and valid. In practice, most modern IPv4 implementations also require the UDP checksum to be present, but some devices have been observed to accept zero-checksum UDP packets. Accepting zero-checksum UDP packets creates a security vulnerability where an attacker can inject forged UDP packets into a stream without the receiver being able to detect the corruption. This is particularly dangerous for DNS, which uses UDP as its primary transport, because a forged DNS response can redirect a user to a malicious website without any checksum verification to detect the forgery.
The interaction between protocol headers and the MTU is another dimension of TCP vs. UDP overhead analysis. TCP negotiates the Maximum Segment Size (MSS) during the three-way handshake, typically set to 1,460 bytes for standard Ethernet (1,500-byte MTU minus 20-byte IP header minus 20-byte TCP header). UDP does not have an MSS concept; the application sends whatever datagram size it chooses, up to 65,535 bytes (the maximum value of the 16-bit UDP length field). However, sending a UDP datagram larger than the path MTU causes IP fragmentation, where the IP layer splits the datagram into multiple fragments that are reassembled at the destination. IP fragmentation is undesirable for several reasons: if any fragment is lost, the entire original datagram must be retransmitted (UDP does not retransmit, so the application must handle this), each fragment carries its own IP header (20 bytes per fragment), and some network devices drop all fragments for security reasons. The practical guidance for UDP application developers is to keep datagram sizes below the path MTU (typically 1,500 bytes for Ethernet, but as low as 1,280 bytes for IPv6) to avoid fragmentation. For TCP, the MSS negotiation automatically ensures fragmentation is avoided without any application-level awareness.
At the data center scale, the overhead difference between TCP and UDP has driven the adoption of Remote Direct Memory Access over Converged Ethernet (RoCEv2), which encapsulates RDMA traffic over UDP. RoCEv2 uses a UDP destination port (4791) and bypasses the kernel network stack entirely, allowing data to be transferred directly from one server's memory to another's without CPU involvement. The UDP encapsulation adds only 8 bytes of header overhead compared to TCP's 20 bytes, which is significant for RDMA operations that transfer data in 4 KB pages. More importantly, RoCEv2's use of UDP allows it to leverage the hardware offload capabilities of modern network interface cards (NICs) that can process UDP packets entirely in hardware without any kernel intervention, achieving per-packet processing latencies below 1 microsecond. This is impossible with TCP because the kernel must maintain the connection state machine, process ACKs, and manage the congestion control algorithm for each connection. The growth of RoCEv2 in AI/ML training clusters, where thousands of GPUs must synchronize their state with microsecond precision, has made UDP-based transport the de facto standard for high-performance data center networking, demonstrating that the choice between TCP and UDP is increasingly driven not just by reliability requirements but by the performance characteristics of the underlying hardware and the latency demands of the application.
"You are our partner in accuracy. If you spot a discrepancy in calculations, a technical typo, or have a field insight to share, don't hesitate to reach out. Your expertise helps us maintain the highest standards of reliability."
Contributors are acknowledged in our technical updates.