One of QUIC's most transformative features for AI inference clusters is connection migration, the ability to transfer an active transport connection from one network path to another without requiring a new handshake or session establishment. In a multi-homed inference node—equipped with both 400G InfiniBand for gradient synchronization and 100G Ethernet for client-facing inference traffic—connection migration enables seamless failover between network interfaces without dropping inference requests mid-flight. The mechanism relies on QUIC's Connection ID (CID) architecture: each endpoint issues a set of CIDs that identify the connection independently of the IP address or port. When a path failure is detected through loss of ACKs or RTT inflation, the client sends a non-probing frame from a new local address, and the server validates reachability before migrating. This entire process completes in one RTT, compared to TCP's requirement for a full SYN-SYN/ACK handshake that adds multiple milliseconds of disruption to latency-critical inference workloads.

The path validation procedure in QUIC connection migration uses a PATH_CHALLENGE/PATH_RESPONSE exchange that includes a 64-bit random nonce to prevent off-path injection attacks. For inference clusters with deterministic latency requirements, the migration MUST complete within the inference timeout window—typically 100-200 ms for real-time NLP models. Our model calculates the migration success probability as: P_migrate = (1 - P_loss)^2×N_retry, where P_loss is the packet loss rate on the new path and N_retry is the number of allowed path validation attempts. In a well-provisioned cluster with P_loss below 0.1%, migration succeeds on the first attempt with 99.8% probability. However, under congestion scenarios where P_loss reaches 1%, the probability drops to 96% per attempt, and the cumulative effect across thousands of inference nodes can trigger cascading timeouts that propagate through the load balancer's retry logic.

The interaction between QUIC connection migration and load-balancing state affinity presents a second-order operational challenge. Production inference gateways rely on IP-address-based hashing (ECMP or consistent hashing) to route requests to backend nodes. When QUIC migrates to a new source IP, the load balancer may route the connection to a different backend, even though the session state (model context, KV cache) is held on the original backend. QUIC's CID can be used as the load-balancing hash key instead of the IP address, enabling CID-pinned routing where the load balancer maintains consistent backend affinity through the Connection ID regardless of IP changes. However, this requires the load balancer to be QUIC-aware (parsing the Initial packet's Source Connection ID field) rather than performing transport-agnostic IP+port hashing. Our model quantifies the performance impact of this architecture shift: CID-pinned routing eliminates migration-related cache misses entirely for IDEMPOTENT inference requests, but introduces a per-packet parsing overhead of approximately 50 ns on software-based load balancers, which at 10 Mpps per core represents a 50% increase in CPU utilization.

For large-scale inference clusters exceeding 1,000 nodes, multipath QUIC (MP-QUIC) extends connection migration to simultaneous multi-path usage rather than just failover. MP-QUIC splits inference request data across multiple paths (e.g., both the InfiniBand and Ethernet interfaces of a GPU node), achieving higher aggregate throughput and lower tail latency through path diversity. The scheduling algorithm—minRTT, round-robin, or ECN-based—determines how bytes are distributed across paths under the constraint of packet reordering. Our comparative model evaluates the completion time for a 10 KB inference response under MP-QUIC using the FMTCP scheduler, showing a 28-35% improvement in tail latency (99th percentile) over single-path QUIC when paths have asymmetric bandwidth ratios of up to 4:1. This statistical multiplexing gain is especially valuable in shared-rack deployments where an Ethernet NIC may experience transient bufferbloat while the InfiniBand path maintains its low-latency profile.

The MP-QUIC extension (draft-ietf-quic-multipath) enables a QUIC connection to use multiple network paths simultaneously, distributing data across available interfaces to improve aggregate throughput and reduce tail latency. For distributed AI inference, where a model may be served from multiple GPU pods connected through diverse network paths, MP-QUIC provides path-level redundancy that can mask transient failures without impacting the inference completion time. The core mechanism is the path scheduler, which determines which packets are sent on which path. The round-robin scheduler distributes packets sequentially across paths but is vulnerable to packet reordering when paths have different RTTs—a common scenario when inference traffic spans both a primary 400 Gbps InfiniBand fabric and a backup 100 Gbps Ethernet WAN connection. The minRTT scheduler assigns packets to the path with the smallest smoothed RTT (SRTT), but this underutilizes the secondary path, wasting available capacity. The ECN-weighted scheduler dynamically adjusts the per-path allocation based on explicit congestion notification (ECN) marking ratios, sending more traffic through low-congestion paths and reducing load on high-congestion ones. Our throughput model incorporates these scheduling strategies and computes the effective goodput for each strategy given the path latency, bandwidth, and loss characteristics.

The reordering tolerance parameter in MP-QUIC determines the amount of out-of-order data the receiver will buffer before delivering to the application. MP-QUIC defines a reordering window ρ parameter, measured in bytes (or equivalently in unit RTTs), that specifies the maximum in-sequence gap before the receiver signals a RETIRE_CONNECTION_ID to the sender. For AI inference workloads where the Time to First Token (TTFT) is the critical metric, a large reordering window is acceptable because the application (an LLM generating tokens) is tolerant of out-of-order data arrival—individual tokens can be reordered before being fed to the next layer of the transformer. Our model sets ρ = 2 × RTT_max to allow for natural path diversity reordering without triggering premature retransmissions. Under a 10 Mbps loss event on the primary path, the minRTT scheduler switches 80% of traffic to the backup path within one RTT, while the ECN-weighted scheduler gradually redistributes traffic over 3-5 RTTs. The TTFT impact is a 12% increase during the 150 ms failover window for the minRTT scheduler versus a 28% increase for the ECN-weighted scheduler, demonstrating that aggressive path switching is preferable for latency-sensitive inference workloads even when it causes temporary reordering.

The cross-path ACK aggregation mechanism in MP-QUIC is a critical optimization for inference clusters with asymmetric path characteristics. In single-path QUIC, ACKs are sent on the same path as the data, providing the sender with per-path RTT and loss measurements. In MP-QUIC, ACKs for data sent on one path may arrive on a different path because the connection is identified by the Connection ID rather than the path's IP tuple. The sender must attribute each ACK to the path on which the corresponding data was sent—this is done through the PATH_RESPONSE frame that carries a 64-bit path sequence number. When ACKs are aggregated across paths (e.g., a single ACK frame acknowledging data from both path 1 and path 2), the sender's RTT estimator for each path becomes biased: the measured RTT for path 1 includes the cross-path delay of path 2. The standard mitigation is to send separate ACK frames for each path when the paths have significantly different RTTs (ratio > 1.5×). Our model simulates the cross-path ACK distortion and its effect on the sender's cwnd growth, showing that at RTT ratios below 2×, the distortion adds less than 5% error to the BBR bandwidth estimate.

The path failure detection and recovery latency in MP-QUIC depends on the keep-alive probe interval and the number of consecutive missed probes before a path is declared dead. Standard QUIC uses a 30-second idle timeout, but MP-QUIC implementations reduce this to 3 × Probe_Interval where Probe_Interval can be as low as 100 ms for latency-sensitive deployments. For an inference cluster with 1,000+ parallel MP-QUIC connections, the aggregate probe traffic at 100 ms per connection is 1,000 × 20 bytes × 10 Hz = 200 KB/s of probe data—negligible compared to the inference data plane. The recovery sequence when path 1 fails: the scheduler sends all traffic over path 2 within 100 ms (one probe interval), simultaneously initiates path probing on path 1 at 200 ms intervals with exponential backoff (up to 3-second maximum), and if path 1 is restored within 5 seconds, resumes dual-path operation within one further probe interval (200 ms). During the single-path recovery window, the inference throughput is constrained by path 2's bandwidth, which may be 10× lower than the aggregate. Our tool models this recovery transient and computes the probability of inference timeout given the path bandwidth ratio and the application's timeout setting.