Global Traceroute Client
Enter a domain or IP to perform a real-time traceroute from our edge network nodes.
Visual Traceroute Engine
Map the physical path of packets across the internet's backbone. Identify network bottlenecks, high-latency autonomous systems, and packet-loss hotspots.
The Physics of TTL: Forcing the Internet to Reveal Itself
To understand traceroute, one must first understand the **Time-to-Live (TTL)** field in the Internet Protocol (IP) header. TTL was never intended for diagnostics; it was designed as a defensive "dead-man switch" to prevent packets from looping infinitely through misconfigured routing tables. An IP packet with no expiration would circulate forever, consuming bandwidth and router CPU cycles until the network collapsed under its own weight.
Every router that processes a packet is mandated by **RFC 791** to decrement the TTL value by at least one. This is mathematically expressed as:
If the TTL reaches zero before the packet reaches its final destination, the router is obligated to discard the packet and notify the sender via an **ICMP Type 11, Code 0 (Time Exceeded)** message. This error message is the "reveal" that traceroute relies on.
The TTL field is an 8-bit integer, permitting a maximum value of 255. In the early days of the ARPANET, TTL represented seconds; today, it representing hops. Different operating systems use different initial values: Windows typically starts at **128**, while Linux/Unix systems default to **64**. This discrepancy allows network analysts to perform "OS Fingerprinting" by examining the TTL of incoming packets.
Modern traceroute implementations don't just record the IP; they measure the **Round-Trip Time (RTT)** for each probe. By sending three probes per hop (the industry default), engineers can calculate the minimum, maximum, and average latency at every single point in the chain, providing a high-resolution snapshot of network performance.
Protocol Forensics: ICMP, UDP, and TCP Traces
The original traceroute implementation by Van Jacobson used **UDP** packets targeting a range of "unlikely" high-numbered ports (33434 to 33534). However, as firewalls became more aggressive, network engineers had to diversify their probing strategies to ensure they could actually reach the target.
ICMP Echo (Windows)
The Microsoft Standard.
Sends standard Pings. Simple and efficient, but many edge firewalls (like AWS Security Groups) block ICMP by default, leading to the dreaded * * * on the final hop even if the server is up.
UDP Tracing
Standard on Linux/Unix.
Sends probe packets to high ports. When the destination is reached, the server typically replies with a "Port Unreachable" (Type 3, Code 3) instead of "Time Exceeded."
TCP SYN Tracing
Modern Infrastructure Probing.
Mimics a real connection attempt (usually on Port 443). Since firewalls must allow web traffic, TCP traceroutes bypass most blocks to reach the application layer.
The choice of protocol significantly impacts the visibility of the path. For instance, **Layer 4 Load Balancers** and **Application Delivery Controllers (ADCs)** might respond differently to a UDP probe than they would to a TCP SYN on port 443. Advanced tools like `tcptraceroute` or `paratrace` use various flags in the TCP header (like the ACK or FIN flags) to discover routers that would otherwise be invisible.
Path Anatomy: Deciphering the Hops
Every line in a traceroute report represents a unique node or interface. Understanding the taxonomy of these hops is key to isolating problems:
- Hops 1-3:**Local Network & Edge Gateway.** This is your router, your ISP's local CMTS, or the data center edge. High latency here usually indicates local Wi-Fi interference or local link congestion.
- Hops 4-8:**The Core ISP Network.** These are high-speed regional routers. You will typically see very stable, low latency here as packets stay within the provider's fiber backbone.
- Hops 9-15:**Peering & Transit.** This is "The Wild West." Packets cross between Autonomous Systems (AS) at Internet Exchange Points (IXP). This is where 90% of internet performance issues occur due to BGP routing shifts or saturated peering capacity.
ECMP: Why Some Hops Have Multiple IPs
In high-density carrier networks, traffic is rarely sent down a single wire. Instead, **Equal-Cost Multi-Path (ECMP)** routing is used to balance the load across dozens of parallel links. For a traceroute, this presents a unique challenge.
Because traceroute sends multiple packets per hop, the router at Hop N might send Packet 1 over Link A and Packet 2 over Link B. If Link A and Link B lead to different physical routers at Hop N+1, you will see two or three different IP addresses listed for the same hop number. This is not a bug; it is a sign of a robust, high-capacity network. However, it makes isolating a single "bad link" much harder, as your traffic may only hit the failing interface 33% of the time.
The 7 Deadly Traces: Troubleshooting Guide
1. The Routing Loop
Identified by two IP addresses alternating until the max hop limit is reached. This is a fatal BGP misconfiguration where Router A thinks the path is via Router B, and Router B thinks it's via Router A.
2. The ICMP Black Hole
The trace proceeds normally until Hop X, and then shows nothing but asterisks (* * *) until the end. This usually means a firewall is dropping your specific probe type (ICMP/UDP) but allowing others.
3. Low-Priority CPU Overhead
Hop 5 shows 200ms latency, but Hops 6 through 15 show 15ms. This is "Phantom Latency." The router is busy and delayed responding to your probe, but it is forwarding customer traffic at full speed.
4. Asymmetric Return Path
A massive latency spike appears in the middle of a trace but has no logical physical reason (e.g., both routers are in the same city). This often means the *return* path from that router is congested.
Probability of Packet Delivery in Multi-Hop Paths
Packet loss is often a result of Bernoulli trials at each hop. In a path with n hops, the probability that a packet successfully completes the entire journey (P_success) is the product of the survival probabilities of each individual hop:
Note: If P_loss is non-zero at any point in the core network, TCP retransmissions will cause the perceived application-layer latency to increase significantly beyond the raw RTT.
Maintenance & Diagnostic Monitoring Strategies
Effective network maintenance requires moving beyond "one-off" traceroutes toward continuous path monitoring. Professional NetOps teams utilize **MTR (My Traceroute)**—a tool that combines ping and traceroute into a live-updating stream.
Continuous Delta Analysis
Baseline your path daily. A traceroute is only useful if you know what the path looked like *before* the issue started. Use tools like ThousandEyes or custom MTR scripts to log path deltas.
BGP Peering Audits
If your traceroute shows a jump into a Tier-1 provider (like NTT or GTT) and latency spikes, audit your BGP advertisements. You may be accidentally preferring a trans-continental path for regional traffic.
When maintaining high-availability applications (VoIP, Fintech, Gaming), a traceroute should be the first step in the **"Triangulation Protocol"**:
- Baseline RTT: Measure the minimum RTT to establish the "Physical Floor" (Propagation limit).
- Path Stability: Look for IP flapping (ECMP imbalance). If IPs are changing every second, a router in the core may be failing.
- MTU Discovery: Use the "Do Not Fragment" bit in your trace to detect MTU mismatches (Black Hole Routers).
Security & Reconnaissance: The Defender's Dilemma
Traceroute is a double-edged sword. For administrators, it is a scalpel for diagnostics. For attackers, it is a map for reconnaissance. By analyzing a traceroute, an adversary can:
- Identify the physical location of servers (via IP Geolocation of intermediate hops).
- Discover the internal IP addressing schemes of a corporate network.
- Detect the presence of Web Application Firewalls (WAFs) and IDS/IPS systems.
- Target specific "weakest link" routers for DDoS attacks (the "Zero-TTL" attack).
To mitigate this, many enterprises configure their edge routers to be "Stealthy." They will still forward packets (decrementing TTL) but will refuse to send the ICMP Time Exceeded response back to the requester. This maintains network health while obscuring the infrastructure from potential attackers.
Advanced Diagnostics: Anycast and CDN Pathing
In the modern internet, the destination IP you are tracing might not be a single server. **IP Anycast** allows multiple globally distributed servers to share the same IP address. When you run a traceroute to an Anycast IP (like Cloudflare's 1.1.1.1 or Google's 8.8.8.8), BGP routing protocols direct your packets to the "topologically closest" node.
CDNs use traceroute data to optimize their edge delivery. By measuring the latency from thousands of "Vantage Points" (VPs), CDNs calculate the optimal path for high-bandwidth content delivery. The performance metric used is the **Normalized Path Latency**:
Where H_count is the hop count and L_fiber is the physical length of the fiber link. A high Lambda_norm indicates inefficient peering or "tromboning"—where traffic travels far out of its way only to return to a nearby point.
Frequently Asked Questions
Frequently Asked Questions
Technical Standards & References
Related Engineering Resources
Automated Traceroute Monitoring and Network Telemetry Integration
In large-scale network operations, a traceroute executed manually at the moment of an anomaly provides valuable diagnostic data, but it captures only a single point in time. The true power of path discovery emerges when traceroute is automated, continuous, and integrated into the network telemetry pipeline that feeds dashboards, alerting systems, and AI-driven root cause analysis. Automated traceroute monitoring — sometimes called "path monitoring" or "network probing" — systematically executes traceroutes from multiple vantage points to multiple targets on a scheduled basis, building a historical database of path state that enables operators to detect routing changes, latency degradation, and black holes before they impact user-facing services.
The architecture of an automated traceroute monitoring system consists of three layers: probing agents deployed at strategic locations throughout the network, a collection and storage backend that normalizes and indexes the raw traceroute output, and an analytics and alerting engine that compares current path state against historical baselines. Probing agents should be deployed on dedicated monitoring servers or lightweight containers at every major network point of presence — each data center, each core router connected to an Internet Exchange, and each cloud region's VPC. A typical enterprise deployment uses 5-20 probing agents that cover the most critical infrastructure paths. The probes run every 60-300 seconds depending on the criticality of the monitored path, sending ICMP, UDP, and TCP (port 443) probes to a target list that includes every customer-facing VIP, every cloud gateway, every critical third-party service endpoint, and the loopback addresses of all core and edge routers.
The baseline analytics layer transforms raw traceroute output into actionable metrics. For each monitored path, the system computes the path length (number of hops), the minimum RTT (the physical propagation floor, typically determined by the longest fiber link in the path), the jitter per hop (standard deviation of RTT across consecutive probes), and the path stability score (percentage of probes that traversed the exact same sequence of router IP addresses). A path stability score below 95% over a 24-hour window indicates excessive ECMP rebalancing or routing instability that warrants investigation, even if latency and packet loss metrics remain within acceptable bounds. The system should also detect path inflation — a scenario where the number of hops increases by 2 or more without a corresponding change in the minimum RTT, typically indicating traffic being routed through a suboptimal path due to a BGP policy misconfiguration or a failed peering session.
The integration with network telemetry closes the diagnostic loop. When the automated traceroute system detects a path anomaly — a new hop with elevated latency, a permanent RTT increase of more than 20%, or a path failure (complete packet loss at a specific hop) — it should automatically trigger a deeper diagnostic sequence: first, a traceroute from an alternative probing agent to confirm the anomaly is not agent-specific; second, a reverse traceroute from the target back to the primary probing agent to isolate whether the issue is in the forward or return path; third, a query to the network management system to determine the device identity, interface status, and recent syslog events for the router identified as the problematic hop. This automated diagnostic escalation turns a raw traceroute observation into a structured incident that includes the responsible device, the likely root cause (link congestion, interface error, BGP session flap), and a suggested remediation action. The Pingdo traceroute tool provides the structured data format that enables this automated integration, with JSON output that includes per-hop IP addresses, AS numbers, latency metrics, and geolocation context — ready to be consumed by SIEM, SOAR, and network automation platforms.
The ultimate goal of automated traceroute monitoring is proactive detection of routing degradation before it affects application performance. A classic example is the detection of a failing transceiver: the automated system observes the latency at a specific hop increase from 2ms to 5ms over several hours, with an increasing number of ICMP timeouts at that same hop (partial packet loss). These are the hallmarks of a transceiver with degrading optics — the link's Forward Error Correction (FEC) is correcting an increasing number of bit errors, adding latency while the retransmission mechanism of the link layer causes occasional probe loss. The automated system can generate a notification 6-24 hours before the transceiver fails completely, enabling the NOC to schedule a replacement during the next maintenance window. Without automated traceroute monitoring, this failure mode would only be detected when the link drops completely, triggering a troubleshooting fire drill during peak traffic hours.
Load-Sensitive Hops and ICMP Rate Limiting: How Routers Mask True Latency in Traceroute Probes
Standard traceroute relies on ICMP Time Exceeded messages (Type 11, Code 0) generated by routers when the TTL (IPv4) or Hop Limit (IPv6) of a probe packet expires. The generation of these ICMP messages is processed by the router's control plane CPU—not the ASIC forwarding engine—and control plane processing is subject to rate limiting to prevent CPU overload from malicious or excessive ICMP traffic. Cisco IOS applies a default ICMP rate limit of 100-200 packets per second (p/s) to the route processor, enforced by the ip icmp rate-limit command. When the rate limit is exceeded, the router silently drops the excess ICMP Time Exceeded packets, causing traceroute to show either a timeout ("* * *") at that hop or an artificially inflated RTT if only a subset of probes elicit a response. The rate limit effect is hop-dependent: a core router transiting 10,000 traceroute probes per second from all users will drop 9,800-9,900 probes per second, resulting in a 1-2% success rate for each probe. The measured RTT distribution at a rate-limited hop is not representative of the actual forwarding latency: the ICMP response is delayed by the control plane queueing time, which can reach 10-100 ms (the ICMP rate-limit bucket refill interval). Our traceroute model applies a statistical correction factor based on the probe count: with 3 probes per hop, the probability of receiving at least one ICMP response from a rate-limited router with success rate p is P_success = 1 - (1-p)³. For p = 0.02 (2% success rate at a highly rate-limited router), P_success = 1 - (0.98)³ = 5.9%—meaning 94% of traceroutes will show a timeout at that hop even though the router is correctly forwarding data packets. The model reports the confidence interval for each hop's RTT based on the number of successful probes and the estimated ICMP rate limit at that router (derived from the response delay variance).
The control-plane policing (CoPP) filtering effect on traceroute is more selective than simple rate limiting. Cisco's CoPP (Control Plane Policing) classifies ICMP messages into different policy classes and applies different drop probabilities. On a typical core router, ICMP unreachable messages (Type 3, used for "Destination Unreachable" and "Fragmentation Needed") are classified as "critical" (low drop probability, 1-5%) while ICMP Time Exceeded messages (Type 11, used for standard traceroute TTL expiry) are classified as "normal" (high drop probability, 20-50%). A router implementing CoPP with a "normal" class police rate of 100 p/s and burst size of 50 packets will drop 30-50% of traceroute probe responses during peak traffic periods (e.g., 5,000-10,000 traceroutes per second across all users). The CoPP filtering is not uniformly distributed across all traceroute sources: because the police rate is applied per-second (not per-source), a burst of probes from a single source can monopolize the police bucket, starving probes from other sources. Our traceroute model includes a CoPP-aware probe scheduling algorithm that spreads the probe packets across the measurement interval (jittering the inter-probe gap by 50-200 ms) to avoid saturating the router's CoPP bucket. The model also estimates the effective drop probability per hop from the variance of RTT across multiple traceroute sessions and adjusts the latency reported for each hop by subtracting the estimated control-plane processing delay (typically 1-5 ms for the route processor to generate and queue the ICMP response).
The MPLS label switching impact on traceroute introduces a layer of complexity where the standard TTL-expired ICMP response may never reach the probing source. In an MPLS network, the ingress LSR (Label Switch Router) imposes a label stack, and each LSR in the LSP (Label Switched Path) decrements the MPLS TTL in the outermost label rather than the IP TTL. When the MPLS TTL expires, the LSR generates an ICMP Time Exceeded message, but this message must be routed back to the probing source along the IP path—which may be different from the MPLS LSP. If the MPLS LSP's egress LSR strips the label stack and forwards the ICMP response using the IP header's original source address, the response traverses the IP network, potentially encountering routers that treat it as new traffic (subject to CoPP and rate limiting on the return path) rather than as a response to the original probe. The round-trip asymmetry between the MPLS LSP and the IP return path causes the RTT reported by traceroute to be the sum of the MPLS forward path delay (typically 100-500 μs per hop) and the IP reverse path delay (typically 1-10 ms per hop), completely obscuring the forward path latency contribution. Our model decomposes the asymmetric RTT by comparing traceroute results from multiple probing sources: the forward MPLS delay component is computed as the minimum RTT across all sources (assuming the MPLS LSP is shared), and the reverse IP delay component is computed as the variance in RTT across sources (since the IP return path varies per source).
The Equal Cost Multi-Path (ECMP) hashing on traceroute probes creates per-hop RTT variance that our model uses to characterize the load-balancing behavior of intermediate routers. When a router has multiple ECMP next-hops to the same destination, the hash of the traceroute probe's source IP, destination IP, and UDP source port (for UDP-based traceroute) determines which ECMP path the probe takes. If the probe uses the same source port across all TTL values (the default for many traceroute implementations), all probes in the same traceroute session follow the same ECMP path, concealing the alternate-path latency. If the probe randomizes the source port per TTL value (as implemented by Paris-traceroute and Python's scapy traceroute), each probe may follow a different ECMP path, revealing the latency distribution across all parallel paths. Our model sends 3 probes per TTL with different source ports and computes the per-hop latency distribution (mean, min, max, stddev). A per-hop standard deviation exceeding 20% of the mean RTT indicates that the probes are traversing different ECMP paths (path diversity), while a low standard deviation (below 5%) indicates path stability. The model uses the per-hop variance to construct a topological map showing which routers load-balance across multiple next-hops and which routers forward all traffic through a single interface, enabling the operator to identify potential ECMP polarization points in the fabric.
"You are our partner in accuracy. If you spot a discrepancy in calculations, a technical typo, or have a field insight to share, don't hesitate to reach out. Your expertise helps us maintain the highest standards of reliability."
Contributors are acknowledged in our technical updates.