~/netref / Data Center
##

Data Center Networking

Modern DC fabrics are routed, overlayed and built for server-to-server (east-west) traffic.

// spine-leaf fabric

Spine Spine Leaf Leaf Leaf
Every leaf connects to every spine — any server is exactly two hops from any other (predictable east-west latency).

// topology models

Three-tier
core / aggregation / access
North-south optimized; STP blocks redundant links (half idle)
Spine-leaf (Clos)
two tiers — every leaf ↔ every spine
East-west optimized; ECMP keeps all links active; any host 2 hops away
// fabric roles
LeafAccess / top-of-rack; acts as VTEP; servers & storage attach here
SpineHigh-speed backbone; connects every leaf; often the BGP route-reflector
Super-spineInterconnects multiple pods in a 5-stage Clos (hyperscale)
Border leafGateway out of the fabric to WAN / external networks
// overlay & EVPN
UnderlayRouted IP fabric (OSPF/BGP) with ECMP; switches peer over loopbacks
OverlayVXLAN tunnels carry tenant L2/L3 on top of the underlay
VXLANMAC-in-UDP encapsulation, port 4789, 24-bit VNI ≈ 16M segments
VTEPVXLAN tunnel endpoint — encapsulates/decapsulates, usually the leaf
EVPNBGP control plane: Type-2 = MAC/IP, Type-5 = IP prefix
IRBDistributed anycast gateway for routing between VNIs
// resilience — Uptime Institute tiers
TierAvailabilityMax downtimeTopology
Tier I99.671%~28.8 h/yrSingle path, no redundancy
Tier II99.741%~22 h/yrRedundant components (partial N+1)
Tier III99.982%~1.6 h/yrConcurrently maintainable (N+1)
Tier IV99.995%~26 min/yrFault tolerant (2N / 2N+1)

// essentials

  • Spine-leaf fabric is built for east-west traffic
  • VXLAN provides the L2 overlay; EVPN is its control plane
  • Routed underlay (often BGP) + tunneled overlay
  • ToR (top-of-rack) switches aggregate each rack
  • HCI merges compute, storage and network in software
  • East-west (server↔server) now dominates the data center

// efficiency & operations

  • PUE = total facility power ÷ IT equipment power (ideal → 1.0)
  • Hot-aisle / cold-aisle containment separates intake and exhaust air
  • ECMP load-balances flows across every spine uplink
  • Oversubscription ratio = downlink (server) bw ÷ uplink (fabric) bw
  • HCI & SDN/ACI abstract compute, storage and network into software