DPU/IPU: The Third Pillar of Compute
Reclaiming the CPU cores.
In a multi-tenant cloud or a high-performance training cluster, 'Infrastructure services' (traffic policing, encryption, local storage virtualization) can consume up to **30% of your x86 CPU cores**.
This is 'Stranded Capacity'. You are paying for a high-end EPYC/Xeon processor just to move bits around. The **Data Processing Unit (DPU)** or **Infrastructure Processing Unit (IPU)** takes these functions and moves them to its own dedicated Arm cores and specialized hardware accelerators.
Security Isolation
A DPU runs its own OS (usually Linux). If the host host is compromised, the network security policies on the DPU remain untouched.
Storage Offload
The DPU presents remote network storage as local NVMe drives to the host, handling all the encryption and protocol work internally.
Architectural Core Components.
Arm/MIPS Cores
General purpose cores for control plane logic.
P4 Match-Action
Hardware accelerated packet filtering.
Hardware Root of Trust
Encrypted boot and policy persistence.
DDR Memory
Dedicated memory for cache and metadata.
The Programming Model.
Programming a DPU is historically difficult. To solve this, NVIDIA created **DOCA** (Data-Center Infrastructure on-a-Chip Architecture), a set of APIs that allow developers to write applications that sit directly on the DPU hardware, similar to how CUDA exposes GPU acceleration.