Direct
Storage.

When multiple training jobs share a single GPU node via MIG (Multi-Instance GPU) or vGPU partitioning, the memory bandwidth contention at the HBM (High Bandwidth Memory) controller becomes a first-order performance limiter that the direct ROI model must incorporate. Each HBM2e stack on an NVIDIA A100 provides approximately 900 GB/s of bandwidth shared across up to seven MIG instances (for the A100-80GB SKU). When two MIG instances issue concurrent memory transactions — one performing the all-reduce gradient sync and the other computing the forward pass — the HBM controller interleaves the requests at the row-buffer level, causing bank conflicts and row-activation penalties that reduce effective bandwidth by 15-30% compared to the single-instance benchmark. The GDS (GPU Direct Storage) path exacerbates this contention because the DMA engine issuing the PCIe reads must also arbitrate for HBM bandwidth against the compute kernels, and GDS is typically configured with a dedicated DMA channel that bypasses the GPU's L2 cache hierarchy. The resulting memory bandwidth competition between GDS transfers and compute kernels can increase kernel execution time by 12-18%, an effect that is invisible to the simple bandwidth-based ROI calculation but that directly impacts the wall-clock training time that the financial model depends on.

The all-reduce algorithm choice further modulates the memory contention penalty. Ring all-reduce (NCCL) partitions the gradient tensor across GPUs and performs N−1 scatter-reduce and N−1 all-gather steps, each requiring one send and one receive per step. Each NCCL kernel launch triggers a CUDA kernel that reads the gradient buffer from HBM, performs the reduction (addition), and writes the result back — a read-modify-write cycle that consumes approximately 2× the gradient tensor size in HBM traffic per step. For a 175B-parameter model with mixed-precision (FP16) gradients (350 GB total gradient storage), a single all-reduce step generates 700 GB of HBM traffic across the node. When this traffic overlaps with the forward-pass computation (which reads approximately 1.5× the parameter size in activations per layer), the HBM bandwidth becomes the binding constraint. The effective throughput under contention follows a saturation model: B_eff = B_peak / (1 + α × N_contending), where α is the contention coefficient (typically 0.08-0.12 for HBM2e) and N_contending is the number of concurrent memory-intensive kernels. For α = 0.1 and N_contending = 3 (forward, backward, and GDS DMA), B_eff = B_peak / 1.3 = 692 GB/s, a 23% reduction from the nominal 900 GB/s peak.

The GDS advantage also depends critically on the NUMA (Non-Uniform Memory Access) topology of the CPU-to-GPU interconnect. On a dual-socket AMD EPYC or Intel Xeon platform with four A100 GPUs per socket, the GDS path that terminates on a GPU attached to socket 0 must traverse the socket-to-socket Infinity Fabric (xGMI) or UPI link if the NVMe drive is connected to socket 1. The cross-socket bandwidth on a single EPYC 7763 xGMI link is approximately 50 GB/s in each direction, shared with all other inter-socket traffic (MPI communication, file system metadata, etc.). If the GDS data crosses this link, the effective transfer rate drops from the PCIe Gen4 x16 limit of 31.5 GB/s to the cross-socket bottleneck of 12-15 GB/s after accounting for protocol overhead and competing traffic. The ROI simulator should model this topology by accepting the CPU socket topology as an input parameter and adjusting the GDS bandwidth down by the cross-socket penalty factor (typically 0.4-0.5) when the GPU and NVMe target reside on different sockets. Training jobs that are NUMA-aware — pinning the training process to the same socket as the GPU and storage — avoid this penalty entirely and see the full GDS benefit, which is why the financial model should include the NUMA pinning configuration as a binary discriminator that toggles between the full-GDS and penalized-GDS performance curves.