Memory
Physics.

The Through-Silicon Via (TSV) is the vertical conductor that connects each HBM3e die in the 12-Hi stack to the base logic die. Each TSV is a copper-filled cylinder approximately 10 micrometers in diameter and 50 micrometers tall, passing through the silicon substrate of each intermediate die. The TSV introduces parasitic capacitance and inductance that limit the maximum signaling rate between dies. In HBM3e, each TSV operates at 6.4 Gbps per pin (the HBM3e base data rate) with the 1,024 data TSVs organized into 16 channels of 64 bits each.

The parasitic parameters of a single TSV are approximately C_TSV = 50 fF and L_TSV = 10 pH at 6.4 GHz. For a 12-Hi stack, the signal must traverse 11 intermediate TSVs (one per die above the base), totaling 550 fF of capacitance and 110 pH of inductance per signal path from the topmost die to the base logic die. The RC time constant of this path is 550 fF x 50 mOhm (TSV resistance) = 27.5 femtoseconds — negligible. However, the inductive impedance at 6.4 GHz is 2 x pi x 6.4 GHz x 110 pH = 4.4 Ohms, which combined with the 50 fF per-TSV capacitance creates a distributed LC transmission line with a characteristic impedance of 47 Ohms — close enough to the 50 Ohm system impedance to avoid significant reflections.

The real signal integrity challenge is **TSV-Induced Skew** between data TSVs and clock TSVs. The clock signal used for read strobe (DQS) is distributed via dedicated TSVs that are matched in length to the data TSVs within 1 micrometer of physical tolerance. At 6.4 Gbps, the period is 156.25 picoseconds. A 1 micrometer TSV length mismatch corresponds to a time skew of 1 micrometer / (1.5 x 10^8 m/s) = 6.7 femtoseconds — far below the 10% timing margin of 15.6 picoseconds. The dominant skew source is not the TSV length but the **Thermal Gradient** across the 12-die stack. The topmost die (die 12) operates at 75°C while the base logic die (die 0) operates at 85°C, a 10°C gradient that causes a differential thermal expansion of 0.2 micrometers across the 600-micrometer stack height, translating to a timing skew of 1.3 picoseconds.

To compensate for this thermally-induced skew, HBM3e incorporates a **Deskew Calibration Sequence** during the READ training phase. The memory controller sends a known data pattern to the topmost die, measures the round-trip delay, and adjusts the per-pin delay elements (DLL) in 1.95 picosecond steps to align all 1,024 data signals within a 5 picosecond window. The deskew calibration is repeated every 1 millisecond during idle periods, ensuring that the thermal skew never exceeds the 10% timing margin even under the most extreme thermal gradients encountered in liquid-cooled GPU clusters, where the HBM junction temperature can swing by 15°C during a training step.

HBM3e's 8 TB/s of aggregate bandwidth is only deliverable if the memory subsystem is fed with clean, stable power. Each 12-Hi stack operates from a 1.1 V logic rail and a 1.2 V I/O rail, drawing roughly 10-12 W under sustained load. A B200-class GPU carrying eight stacks therefore budgets 80-100 W just for HBM — before a single ALU instruction executes — and the current draw per stack at 1.1 V approaches 10 A. That current must cross the CoWoS interposer through power TSVs and land on the GPU die with a voltage ripple well under the specification's tolerance, or the memory controller's timing margins collapse exactly at the speeds HBM exists to provide.

The power-delivery constraint is captured by the target impedance of the power distribution network: Z_target = V_ripple / delta_I. With a permitted ripple of 1% of 1.1 V (11 mV) and a transient current swing of 10 A per stack, the interposer-level PDN must present no more than 1.1 milliohm of impedance up to the switching frequency of the DRAM I/O (multi-gigahertz). Achieving that requires a dense network of power TSVs, wide interposer metal, and high-density on-package decoupling capacitors positioned within millimeters of the stacks. Every decoupling capacitor the design omits shows up as a switching transient that the voltage regulator module (VRM) — physically on the host board, tens of millimeters away — is too far and too slow to catch.

The VRM bandwidth mismatch is the practical failure mode. A board-level VRM closes its feedback loop at roughly 100 kHz to 1 MHz, but HBM I/O bursts switch at gigahertz rates; the VRM handles the average load, and the on-package decoupling handles the transients. When the power delivery is undersized — as it is on early 8-stack boards that reused 4-stack power designs — the memory subsystem suffers what amounts to a brownout during the read-write turnaround peaks, forcing the controller to insert wait states that shave 5-10% off the measured bandwidth. This is why the 8 TB/s number is only achieved when power delivery for AI accelerators is designed together with the memory stack, not bolted on afterward.

The thermal interaction completes the picture. Power drawn is power dissipated, and the 10-12 W per stack concentrates in a die footprint of a few square millimeters per bank group, producing the hot spots the refresh-window analysis above already penalizes. Every watt saved in the memory subsystem by dropping I/O voltage or by bursting reads instead of sustaining them pays twice: once in the electrical budget and once in the thermal budget, because the stack's junction temperature directly halves or doubles the refresh penalty. A modeler that accounts for both effects — electrical efficiency and thermal refresh — reproduces the 6.8-to-8 TB/s spread seen on real 8-stack boards, which is exactly the kind of boundary GPU performance modeling should capture before the system is fabricated.

HBM3e ships with on-die ECC in every pseudo-channel, adding 8 check bits to each 128-bit data word to implement single-error-correct, double-error-detect (SECDED). The check bits are computed inside the DRAM die itself, which means the memory controller sees a corrected 128-bit word and, crucially, the GPU's caches are protected from the soft errors that plague small-capacitor DRAM. The raw bit error rate for HBM3e at nominal voltage and temperature is on the order of 1e-15 to 1e-16 per bit per hour — but that rate degrades by orders of magnitude with junction temperature, voltage droop, and radiation environment, which is precisely why a memory system with 1.2 trillion bits per GPU (192 GB of 8 stacks) cannot be trusted without correction.

Error scrubbing converts the theoretical correction capability into practical availability. A scrubbing engine reads every cache line on a fixed cadence, and for each single-bit error it detects, it rewrites the corrected value back to DRAM, preventing the accumulation of two-bit errors that would be uncorrectable. At a scrub rate of one line per microsecond, a 192 GB device (1.5 billion cache lines) completes a full pass in about 25 minutes. The scrubbing traffic itself consumes bandwidth — roughly 0.5-1% of the 8 TB/s aggregate — and the memory controller must interleave it with the bursty training access pattern without starving either. Modern GPUs run scrubbing only on idle or low-priority memory regions and stop it entirely during latency-critical inference windows, then resume from a checkpoint offset.

The RAS story extends beyond the die. Multi-bit errors within one 128-bit word are caught but uncorrectable, and the controller must map the failing region to a spare: HBM3e stacks include spare columns and rows, and the device remaps a failing bank via its built-in redundancy at manufacturing test time. In the field, a rising rate of corrected single-bit errors in one stack is the strongest telemetry for predicting an imminent hard failure — it is the memory analog of the rising pre-FEC BER on an optical transceiver, and it feeds the same GPU performance and reliability benchmarks that fleets use to schedule replacements.

Finally, ECC interacts with the bandwidth wall in a way most operators underestimate. Because the on-die ECC hides corrections from software, a fabric can run for weeks with a degraded stack that is burning 2-3% of effective bandwidth on corrective re-reads, all while reporting zero visible errors. Profiling corrected-error counters per stack — rather than waiting for an uncorrectable event — is the only way to see the degradation, and it is why the memory-internal telemetry discussed in the predictive-maintenance context is a required input to any PCIe-era accelerator fleet's health model. The corrected-error rate is the canary; the effective-bandwidth drop is the coal mine.