1. The Identity Problem: Layer 2 vs. Layer 3

In the OSI model, IP addresses (Layer 3) are used for logical routing across networks, but hardware interfaces (Layer 2) only understand MAC addresses. Every time a packet is ready to leave an Ethernet port, the operating system faces a binary crisis: "I know the destination IP, but I have no destination MAC to put in the Ethernet frame header."

The Address Resolution Protocol (ARP), defined in RFC 826, is the "Glue" that resolves these logical addresses into physical ones. It is a stateless protocol that operates directly over Layer 2 (Ethernet Type 0x0806), meaning it does not use IP headers for its own transport.

2. ARP Packet Header Forensics

To understand ARP, one must look at the 28-byte payload that sits inside the Ethernet frame. Unlike IP, ARP was designed to be hardware-agnostic, though it is almost exclusively used for Ethernet/IPv4 today.

Header Breakdown (28 Bytes)

  • Hardware Type (2B): Ethernet is 1.
  • Protocol Type (2B): IPv4 is 0x0800.
  • Hardware Size (1B): 6 bytes for MAC.
  • Protocol Size (1B): 4 bytes for IP.
  • Opcode (2B): 1 for Request, 2 for Reply.
  • Addresses (20B): Sender MAC/IP and Target MAC/IP.

1. The ARP Lifecycle: Request and Reply

ARP operates on a simple transactional cycle.

The Broadcast (Request)

When a host needs a MAC address, it sends an ARP Request. This packet is encapsulated in an Ethernet frame with a destination MAC of **FF:FF:FF:FF:FF:FF** (The Broadcast address). Every device in the local broadcast domain (usually the same VLAN) receives this frame and pulls it up to the CPU.

The Unicast (Reply)

While most devices will discard the request after seeing the target IP doesn't match theirs, the rightful owner of the IP address will formulate a response. Crucially, the ARP Reply is **Unicast**—it is sent directly back to the original sender's MAC address, providing the missing link.

4. The ARP Cache State Machine

An ARP entry doesn't just exist or not exist. Modern operating systems (especially Linux) use a complex state machine to manage entry validity.

Reachable

The mapping is confirmed and usable. Typically lasts 30 seconds.

Stale

The time has expired, but the entry is kept until a packet needs to be sent. It doesn't trigger a probe immediately.

Delay

A packet was sent to a stale entry. The OS waits for an upper-layer confirmation (like a TCP ACK) before probing.

Probe

No confirmation was received. The OS sends unicast ARP requests to refresh the mapping.

5. Proxy ARP: The Helpful Liar

Proxy ARP is a technique where a router answers an ARP request for an IP address that is not on its own interface.

Why would a router lie? If Host A (192.168.1.5/16) tries to talk to Host B (192.168.2.5/24), and Host A incorrectly thinks Host B is on its own subnet, it will send an ARP request for Host B instead of sending the packet to its gateway. A router with Proxy ARP enabled will see this request, realize it knows how to reach Host B, and reply with its own MAC address. Host A then sends the frame to the router, which forwards it correctly.

6. Gratuitous ARP: Unsolicited Announcements

Usually, ARP is a conversation (Request/Reply). Gratuitous ARP (GARP) is a monologue.

  • Duplicate Address Detection (DAD): When an interface comes up, it sends a GARP for its own IP. If someone replies, the OS knows there is an IP conflict.
  • High Availability (HA): When a secondary firewall takes over for a primary, it sends a GARP for the Shared Virtual IP. This forces the local switch to update its MAC table, redirecting all traffic to the new hardware instantly.

7. Historical Forensics: RARP and Inverse ARP

Before DHCP, there was RARP (Reverse ARP). It allowed a diskless workstation to broadcast its MAC address and ask: "What is my IP?" It was eventually replaced by BOOTP and then DHCP.

Inverse ARP (InARP) was used in Frame Relay and ATM networks to map a Data Link Connection Identifier (DLCI) to an IP address, essentially performing the opposite of standard ARP for non-broadcast multi-access (NBMA) networks.

8. ARP on Wi-Fi: The MC2U Logic

Wi-Fi handles ARP differently than Ethernet. Because broadcast traffic is sent at the lowest "Basic Rate" (to ensure all devices can hear it), it is extremely inefficient.

Many modern Access Points (APs) perform Multicast-to-Unicast (MC2U) conversion. The AP maintains its own ARP table and, when it sees an ARP Request broadcast, it intercepts it and sends it as a unicast frame directly to the target device at high speed, significantly reducing airtime congestion.

9. Security Hardening: DAI and DHCP Snooping

Because ARP is unauthenticated, enterprise networks must implement Layer 2 security features to prevent spoofing.

  • DHCP Snooping: The switch monitors DHCP traffic and builds a "Binding Table" of trusted MAC-to-IP mappings.
  • Dynamic ARP Inspection (DAI): The switch intercepts every ARP packet and compares the Sender MAC/IP against the DHCP Snooping table. If they don't match, the packet is dropped, and the port is often shut down (err-disabled).
  • IP Source Guard: Prevents a device from sending any IP traffic if its source IP doesn't match the DHCP Snooping binding, stopping spoofing before it even hits Layer 3.

10. Case Study: The ARP Flux Storm

In a Linux server environment with multiple bonded NICs (e.g., eth0 and eth1), we once saw a strange "jitter" in performance.

The Forensic Root Cause

By default, Linux may respond to an ARP request for any of its local IPs on any interface. If a request for eth0's IP arrived on eth1, the server would reply via eth1. This caused the upstream switch to constantly flip its MAC address table between ports (MAC flapping), resulting in massive frame loss.

Fix: Setting arp_ignore=1 and arp_announce=2 in sysctl forces the server to only reply on the specific interface the IP belongs to.

11. Troubleshooting: Decoding ARP with TCPDump

When "arp -a" shows <incomplete>, it means the request was sent but no reply was received.

# Capture ARP traffic only
tcpdump -i eth0 -n arp

# Sample Output:
10:21:45.123 ARP, Request who-has 192.168.1.1 tell 192.168.1.5, length 28
10:21:45.124 ARP, Reply 192.168.1.1 is-at 00:0c:29:ab:cd:ef, length 28

13. Data Center Forensics: ARP in VXLAN and EVPN

In modern software-defined data centers, Layer 2 segments are often stretched across Layer 3 boundaries using VXLAN (Virtual Extensible LAN).

Standard ARP broadcasts do not scale in these environments. Instead, EVPN (Ethernet VPN) uses a control plane (BGP) to distribute MAC-to-IP mappings between leaf switches. This allows for ARP Suppression: when a host sends an ARP request, the local leaf switch already knows the answer from its BGP table and replies locally, preventing the broadcast from ever flooding the core network.

14. Multi-Homed Server Forensics: The ARP Flux Control

When a server has multiple interfaces on the same physical network, ARP behavior must be strictly tuned to prevent "Asymmetric Routing" at Layer 2.

Linux Sysctl Tuning

  • arp_ignore=1: Only reply to ARP requests if the target IP address is configured on the incoming interface. This prevents the server from answering for eth1's IP on the eth0 wire.
  • arp_announce=2: Always use the best local address for the target. It forces the server to use the IP address of the outgoing interface in the "Sender IP" field of the ARP request, ensuring the reply comes back to the right port.

15. The Physics of the "Stale" State

Why do ARP entries last so long?

The Stale state is an optimization for high-traffic servers. Instead of constantly probing every 30 seconds, the OS keeps the entry but marks it as "unconfirmed." It only moves to the Delay and then Probe states if a packet is actually queued for that destination. This prevents "Background Noise" ARP traffic from thousands of idle connections on a database server.

16. ARP Forensics Summary Checklist

  • Verification: Run arp -a. Is the MAC correct for the IP?
  • Duplication: Do multiple IPs map to the same MAC? (Potential Spoofing).
  • Incompletes: Does the entry say <incomplete>? (No response from target).
  • Flapping: Is the MAC address for a gateway constantly changing? (MAC Flap/Flux).
  • Hardware: Is the NIC offloading ARP? (Check ethtool -k on Linux).

18. Virtualization Forensics: ARP in Open vSwitch (OVS)

In cloud environments like OpenStack or Nutanix, Open vSwitch (OVS) acts as the logical bridge between VMs.

OVS doesn't just flood ARP. Using OpenFlow rules, the controller can intercept ARP requests and respond with "Synthetic" replies from its own internal database of VM locations. This "Logical ARP" prevents broadcast storms in massive multi-tenant clouds where 50,000+ VMs might exist on the same physical fabric.

19. The Case for Static ARP: Total L2 Hardening

For high-security industrial control systems (ICS), Static ARP entries are sometimes used to eliminate the risk of spoofing entirely.

By manually mapping the MAC address of the PLC to the HMI in the ARP table, the devices never send a broadcast request. While this is a management nightmare for 1,000 laptops, it is a bulletproof defense for 10 critical machines on a factory floor. If an attacker tries to spoof the PLC's IP, the HMI will ignore the fake ARP reply because its static entry is immutable.

20. Conclusion: The Foundation of Local Fabric

ARP is the often-overlooked hero of the network stack. It is the bridge that allows the abstract logic of IP to touch the physical reality of copper and fiber. From the simple broadcast/unicast cycle of RFC 826 to the complex BGP-EVPN suppression systems of modern data centers, ARP remains the fundamental language of the local segment. Understanding its forensics—its headers, its states, and its security vulnerabilities—is the mark of a master network engineer.

17. Technical Encyclopedia: ARP Mechanics

ARP Flap

A condition where the MAC address associated with an IP address changes rapidly, often due to an IP conflict or spoofing.

ARP Suppression

A feature in overlay networks (VXLAN/EVPN) that answers ARP requests at the edge switch to prevent core flooding.

BGP EVPN

Ethernet VPN using BGP as the control plane to synchronize MAC and IP reaches between network nodes.

InARP

Inverse ARP. Used in ATM and Frame Relay to map a hardware circuit ID to an IP address.

MAC Flapping

When a switch sees the same MAC address on two different ports, causing it to constantly update its forwarding table.

NBMA

Non-Broadcast Multi-Access. Networks like Frame Relay where broadcasts are not supported or are expensive.

Conclusion

ARP is the silent workhorse of the local area network. It is the first step in almost every network communication. Understanding how it requests, replies, and caches mappings is essential for troubleshooting "Connected but not Pinging" scenarios and for understanding how hardware-level delivery truly functions.

Share Article

Technical Standards & References

REF [RFC-826]
IETF
RFC 826: Address Resolution Protocol
VIEW OFFICIAL SOURCE
REF [RFC-5227]
IETF
RFC 5227: IPv4 Address Conflict Detection
VIEW OFFICIAL SOURCE
REF [ARP-SPOOF]
NIST
ARP Spoofing: Detection and Prevention
VIEW OFFICIAL SOURCE
Mathematical models derived from standard engineering protocols. Not for human safety critical systems without redundant validation.