The Problem: VLAN Exhaustion and STP

In a classic data center, Layer 2 networks are horizontal. If you want a Virtual Machine (VM) to move from Rack A to Rack B without changing its IP address, that VLAN must exist in both racks. This leads to "STP sprawl," where large loops are formed, and links are blocked to prevent broadcast storms, wasting 50% of available bandwidth.

The Solution: Layer 2 over Layer 3

VXLAN uses a MAC-in-UDP encapsulation. It takes the original Ethernet frame and wraps it in a UDP packet, an IP header, and a new Ethernet header. This allows the underlay (the physical switches) to route the traffic using OSPF, IS-IS, or BGP, utilizing all physical links via ECMP (Equal-Cost Multi-Path).

Key Components: VTEPs and VNIs

• VTEP (VXLAN Tunnel Endpoint): The device (usually a switch or server) that performs the encapsulation and de-encapsulation.
• VNI (VXLAN Network Identifier): The 24-bit ID that designates which virtual network the traffic belongs to.
• The Underlay: The physical L3 network that moves the UDP packets.
• The Overlay: The virtual L2 network seen by the servers.

Encapsulation Overhead

Adding these headers increases the packet size by 50 bytes. Because the standard MTU is 1,500 bytes, using VXLAN without adjustment will cause fragmentation and massive performance drops.

Conclusion

VXLAN is the standard that made the modern cloud possible. It decouples the virtual network from the physical hardware, allowing developers to build complex topologies that can span across an entire data center or even multiple geographical regions.