Direct Connect: The Physical Fiber Foundation
The public internet is a best-effort, unpredictable mesh. For enterprise workloads—high-frequency trading, real-time database replication, or AI model training—jitter is the enemy of efficiency. AWS Direct Connect (DX) provides a private, physical path into the AWS global backbone, reducing hop counts and ensuring consistent latency profiles.
1. Layer 1: The Colocation Physical Hand-off.
Direct Connect begins at a **Direct Connect Location** (a carrier-neutral data center like Equinix, Digital Realty, or CoreSite). AWS maintains a fleet of routers inside these facilities.
The Physical Connection Chain
A. The Cross-Connect
A literal Single-Mode Fiber (SMF) patch cable running from your rack in the DC to the AWS patch panel. Usually LC/UPC termination.
B. The Port Speed
AWS offers 1G, 10G, and 100G native ports. LAG (Link Aggregation) allows bonding up to 4 ports for a 400G aggregate pipe.
C. The Demarcation
The physical point where responsibility shifts from the DC operator (Cross-Connect) to AWS (The DX Device).
The "Last Mile" is often handled by a **Direct Connect Partner**. In this model, the partner owns the physical connection to AWS and carves out **Hosted Connections** for users. While simpler, Hosted Connections lack the ability to use native LAG or MACsec encryption.
2. Layer 2: 802.1q VLAN Segmentation.
Once the physical light is green, the single fiber is partitioned into multiple logical pipes called **Virtual Interfaces (VIFs)**. Each VIF is assigned a unique VLAN ID (IEEE 802.1q).
Private VIF
Direct access to a single VPC via a VGW (Virtual Private Gateway). Limited to local region resources.
Transit VIF
The "Pro" choice. Connects to an AWS Transit Gateway (TGW), enabling a single DX connection to reach thousands of VPCs globally.
Public VIF
Access to public AWS services (S3, DynamoDB, Glacier) without crossing the internet. Requires public AS numbers.
3. Layer 3: BGP Peering & BFD Optimization.
Routing is handled via **eBGP (External Border Gateway Protocol)**. AWS advertises its VPC cidrs, and the enterprise router advertises on-prem prefixes.
Scaling with BGP Communities
AWS uses specific BGP communities to give customers control over traffic origination. This is critical for "Hot Potato" vs "Cold Potato" routing.
7224:9100 / 9200 / 9300Local Region / Home Continent / Global scope. Allows you to limit how far AWS distributes your prefixes.
AS_PATH PrependingThe primary mechanism for active/passive failover. AWS honors up to 10 prepends to determine the primary path.
BFD: Sub-Second Failover
BGP is slow. By default, it takes 90 seconds to detect a link failure. **Bidirectional Forwarding Detection (BFD)** sends ultra-fast heartbeats between the on-prem router and the AWS Direct Connect chassis.
BFD detection time (AWS supports 300ms intervals)
Note: Direct Connect LAG interfaces do not support BFD. Instead, they rely on LACP (802.3ad) timeouts which are notoriously slower (approx 3-30s).
Forensics: The BGP Flap
If your Direct Connect logs show "BGP Notification Received (Cease: Peer Unreachable)", check your BGP Local-AS vs Remote-AS. AWS expects 64512-65534 for private AS numbers. If you use a public AS that you don't own, AWS will drop the session at the border.
BGP state = Established, up for 24:02:11
Hold time is 90, keepalive interval is 30 seconds
4. MACsec (802.1AE): Line-Rate Encryption.
For years, the trade-off for Direct Connect was performance vs security. IPsec VPNs could encrypt traffic but capped at ~1.25 Gbps per tunnel. **MACsec** provides Layer-2 encryption at the hardware level, maintaining 100 Gbps line-rate throughput.
Hardware-Assisted Security
Unlike IPsec, which adds complex headers and requires CPU cycles for ESP encapsulation, MACsec is handled by the PHY/SFP+ chip. It encrypts the payload while keeping the ethernet header intact (mostly).
- Cipher AES-GCM-256
- Integrity ICV (Integrity Check Value)
- Latency Penalty < 50ns
- Throughput Up to 400 Gbps
5. Architecture: The DX Gateway (DXGW).
In the early days, a DX connection was tied to a single VPC. This didn't scale. The **Direct Connect Gateway (DXGW)** acts as a global hub.
By associating a Private VIF with a DXGW, you can reach VPCs in any region (excluding China) over the AWS backbone. This simplifies BGP dramatically, as you only peer once with the DXGW rather than managing hundreds of BGP sessions for each VPC.
Exascale Routing Logic
On-Prem Router -> Fiber -> DX Location -> DXGW -> [VPC-US-East-1, VPC-EU-West-1, VPC-AP-South-1]
// Propagation Strategy
Prefixes from On-Prem are automatically redistributed into the route tables of all associated VPCs.
The Verdict: Professional Connectivity
AWS Direct Connect is the ultimate bridge between the physical and virtual worlds. By mastering its L1/L2/L3 nuances, architects can build hybrid infrastructures that rival the performance of unified data centers. In the age of AI, where data movement is the new bottleneck, DX isn't just a network—it's a competitive advantage.
Frequently Asked Questions
Related Engineering Resources
"You are our partner in accuracy. If you spot a discrepancy in calculations, a technical typo, or have a field insight to share, don't hesitate to reach out. Your expertise helps us maintain the highest standards of reliability."
Contributors are acknowledged in our technical updates.