ARP Mechanics: The Glue Between Layer 2 and Layer 3
Deconstructing the Address Resolution Protocol (RFC 826). Analyzing Broadcast/Unicast Cycles, ARP Cache Persistence, and Security Vulnerabilities.
1. The Identity Problem: Layer 2 vs. Layer 3
In the OSI model, IP addresses (Layer 3) are used for logical routing across networks, but hardware interfaces (Layer 2) only understand MAC addresses. Every time a packet is ready to leave an Ethernet port, the operating system faces a binary crisis: "I know the destination IP, but I have no destination MAC to put in the Ethernet frame header."
The Address Resolution Protocol (ARP), defined in RFC 826, is the "Glue" that resolves these logical addresses into physical ones. It is a stateless protocol that operates directly over Layer 2 (Ethernet Type 0x0806), meaning it does not use IP headers for its own transport.
2. ARP Packet Header Forensics
To understand ARP, one must look at the 28-byte payload that sits inside the Ethernet frame. Unlike IP, ARP was designed to be hardware-agnostic, though it is almost exclusively used for Ethernet/IPv4 today.
Header Breakdown (28 Bytes)
- Hardware Type (2B): Ethernet is 1.
- Protocol Type (2B): IPv4 is 0x0800.
- Hardware Size (1B): 6 bytes for MAC.
- Protocol Size (1B): 4 bytes for IP.
- Opcode (2B): 1 for Request, 2 for Reply.
- Addresses (20B): Sender MAC/IP and Target MAC/IP.
1. The ARP Lifecycle: Request and Reply
ARP operates on a simple transactional cycle.
The Broadcast (Request)
When a host needs a MAC address, it sends an ARP Request. This packet is encapsulated in an Ethernet frame with a destination MAC of **FF:FF:FF:FF:FF:FF** (The Broadcast address). Every device in the local broadcast domain (usually the same VLAN) receives this frame and pulls it up to the CPU.
The Unicast (Reply)
While most devices will discard the request after seeing the target IP doesn't match theirs, the rightful owner of the IP address will formulate a response. Crucially, the ARP Reply is **Unicast**—it is sent directly back to the original sender's MAC address, providing the missing link.
4. The ARP Cache State Machine
An ARP entry doesn't just exist or not exist. Modern operating systems (especially Linux) use a complex state machine to manage entry validity.
Reachable
The mapping is confirmed and usable. Typically lasts 30 seconds.
Stale
The time has expired, but the entry is kept until a packet needs to be sent. It doesn't trigger a probe immediately.
Delay
A packet was sent to a stale entry. The OS waits for an upper-layer confirmation (like a TCP ACK) before probing.
Probe
No confirmation was received. The OS sends unicast ARP requests to refresh the mapping.
5. Proxy ARP: The Helpful Liar
Proxy ARP is a technique where a router answers an ARP request for an IP address that is not on its own interface.
Why would a router lie? If Host A (192.168.1.5/16) tries to talk to Host B (192.168.2.5/24), and Host A incorrectly thinks Host B is on its own subnet, it will send an ARP request for Host B instead of sending the packet to its gateway. A router with Proxy ARP enabled will see this request, realize it knows how to reach Host B, and reply with its own MAC address. Host A then sends the frame to the router, which forwards it correctly.
6. Gratuitous ARP: Unsolicited Announcements
Usually, ARP is a conversation (Request/Reply). Gratuitous ARP (GARP) is a monologue.
- Duplicate Address Detection (DAD): When an interface comes up, it sends a GARP for its own IP. If someone replies, the OS knows there is an IP conflict.
- High Availability (HA): When a secondary firewall takes over for a primary, it sends a GARP for the Shared Virtual IP. This forces the local switch to update its MAC table, redirecting all traffic to the new hardware instantly.
7. Historical Forensics: RARP and Inverse ARP
Before DHCP, there was RARP (Reverse ARP). It allowed a diskless workstation to broadcast its MAC address and ask: "What is my IP?" It was eventually replaced by BOOTP and then DHCP.
Inverse ARP (InARP) was used in Frame Relay and ATM networks to map a Data Link Connection Identifier (DLCI) to an IP address, essentially performing the opposite of standard ARP for non-broadcast multi-access (NBMA) networks.
8. ARP on Wi-Fi: The MC2U Logic
Wi-Fi handles ARP differently than Ethernet. Because broadcast traffic is sent at the lowest "Basic Rate" (to ensure all devices can hear it), it is extremely inefficient.
Many modern Access Points (APs) perform Multicast-to-Unicast (MC2U) conversion. The AP maintains its own ARP table and, when it sees an ARP Request broadcast, it intercepts it and sends it as a unicast frame directly to the target device at high speed, significantly reducing airtime congestion.
9. Security Hardening: DAI and DHCP Snooping
Because ARP is unauthenticated, enterprise networks must implement Layer 2 security features to prevent spoofing.
- DHCP Snooping: The switch monitors DHCP traffic and builds a "Binding Table" of trusted MAC-to-IP mappings.
- Dynamic ARP Inspection (DAI): The switch intercepts every ARP packet and compares the Sender MAC/IP against the DHCP Snooping table. If they don't match, the packet is dropped, and the port is often shut down (err-disabled).
- IP Source Guard: Prevents a device from sending any IP traffic if its source IP doesn't match the DHCP Snooping binding, stopping spoofing before it even hits Layer 3.
10. Case Study: The ARP Flux Storm
In a Linux server environment with multiple bonded NICs (e.g., eth0 and eth1), we once saw a strange "jitter" in performance.
The Forensic Root Cause
By default, Linux may respond to an ARP request for any of its local IPs on any interface. If a request for eth0's IP arrived on eth1, the server would reply via eth1. This caused the upstream switch to constantly flip its MAC address table between ports (MAC flapping), resulting in massive frame loss.
Fix: Setting arp_ignore=1 and arp_announce=2 in sysctl forces the server to only reply on the specific interface the IP belongs to.
11. Troubleshooting: Decoding ARP with TCPDump
When "arp -a" shows <incomplete>, it means the request was sent but no reply was received.
tcpdump -i eth0 -n arp
# Sample Output:
10:21:45.123 ARP, Request who-has 192.168.1.1 tell 192.168.1.5, length 28
10:21:45.124 ARP, Reply 192.168.1.1 is-at 00:0c:29:ab:cd:ef, length 28
13. Data Center Forensics: ARP in VXLAN and EVPN
In modern software-defined data centers, Layer 2 segments are often stretched across Layer 3 boundaries using VXLAN (Virtual Extensible LAN).
Standard ARP broadcasts do not scale in these environments. Instead, EVPN (Ethernet VPN) uses a control plane (BGP) to distribute MAC-to-IP mappings between leaf switches. This allows for ARP Suppression: when a host sends an ARP request, the local leaf switch already knows the answer from its BGP table and replies locally, preventing the broadcast from ever flooding the core network.
14. Multi-Homed Server Forensics: The ARP Flux Control
When a server has multiple interfaces on the same physical network, ARP behavior must be strictly tuned to prevent "Asymmetric Routing" at Layer 2.
Linux Sysctl Tuning
- arp_ignore=1: Only reply to ARP requests if the target IP address is configured on the incoming interface. This prevents the server from answering for
eth1's IP on theeth0wire. - arp_announce=2: Always use the best local address for the target. It forces the server to use the IP address of the outgoing interface in the "Sender IP" field of the ARP request, ensuring the reply comes back to the right port.
15. The Physics of the "Stale" State
Why do ARP entries last so long?
The Stale state is an optimization for high-traffic servers. Instead of constantly probing every 30 seconds, the OS keeps the entry but marks it as "unconfirmed." It only moves to the Delay and then Probe states if a packet is actually queued for that destination. This prevents "Background Noise" ARP traffic from thousands of idle connections on a database server.
16. ARP Forensics Summary Checklist
- Verification: Run
arp -a. Is the MAC correct for the IP? - Duplication: Do multiple IPs map to the same MAC? (Potential Spoofing).
- Incompletes: Does the entry say
<incomplete>? (No response from target). - Flapping: Is the MAC address for a gateway constantly changing? (MAC Flap/Flux).
- Hardware: Is the NIC offloading ARP? (Check
ethtool -kon Linux).
18. Virtualization Forensics: ARP in Open vSwitch (OVS)
In cloud environments like OpenStack or Nutanix, Open vSwitch (OVS) acts as the logical bridge between VMs.
OVS doesn't just flood ARP. Using OpenFlow rules, the controller can intercept ARP requests and respond with "Synthetic" replies from its own internal database of VM locations. This "Logical ARP" prevents broadcast storms in massive multi-tenant clouds where 50,000+ VMs might exist on the same physical fabric.
19. The Case for Static ARP: Total L2 Hardening
For high-security industrial control systems (ICS), Static ARP entries are sometimes used to eliminate the risk of spoofing entirely.
By manually mapping the MAC address of the PLC to the HMI in the ARP table, the devices never send a broadcast request. While this is a management nightmare for 1,000 laptops, it is a bulletproof defense for 10 critical machines on a factory floor. If an attacker tries to spoof the PLC's IP, the HMI will ignore the fake ARP reply because its static entry is immutable.
20. Conclusion: The Foundation of Local Fabric
ARP is the often-overlooked hero of the network stack. It is the bridge that allows the abstract logic of IP to touch the physical reality of copper and fiber. From the simple broadcast/unicast cycle of RFC 826 to the complex BGP-EVPN suppression systems of modern data centers, ARP remains the fundamental language of the local segment. Understanding its forensics—its headers, its states, and its security vulnerabilities—is the mark of a master network engineer.
17. Technical Encyclopedia: ARP Mechanics
A condition where the MAC address associated with an IP address changes rapidly, often due to an IP conflict or spoofing.
A feature in overlay networks (VXLAN/EVPN) that answers ARP requests at the edge switch to prevent core flooding.
Ethernet VPN using BGP as the control plane to synchronize MAC and IP reaches between network nodes.
Inverse ARP. Used in ATM and Frame Relay to map a hardware circuit ID to an IP address.
When a switch sees the same MAC address on two different ports, causing it to constantly update its forwarding table.
Non-Broadcast Multi-Access. Networks like Frame Relay where broadcasts are not supported or are expensive.
Conclusion
ARP is the silent workhorse of the local area network. It is the first step in almost every network communication. Understanding how it requests, replies, and caches mappings is essential for troubleshooting "Connected but not Pinging" scenarios and for understanding how hardware-level delivery truly functions.