EVPN-VXLAN Explained: How Modern Data Centers Stretch Networks Without the Flood

First, the plain-English version

Picture a big apartment building. The physical hallways and stairwells are how you actually move — call that the underlay (the real switches, cables, and IP routing underneath everything). The apartment numbers are how residents think about where they live — call that the overlay (the virtual networks tenants actually use). VXLAN is the trick of writing the apartment number on an envelope, then putting that whole envelope inside a second, outer envelope addressed to a building entrance. The hallways only ever read the outer envelope. EVPN is the building directory: instead of shouting down the hallway to find someone (flooding), every entrance keeps an up-to-date list of who lives where, shared through BGP — the same routing protocol that runs the internet.

Why VLANs ran out of road

A VLAN (virtual LAN) tags a frame with a 12-bit ID. Twelve bits give 4,096 values, and two are reserved, so you get 4,094 usable segments — plenty for one office, nowhere near enough for a cloud with thousands of tenants. Worse, classic Ethernet learns addresses by flooding: when a switch does not know where a MAC address lives, it copies the frame out of every port and hopes. And spanning tree, the loop-prevention protocol, deliberately blocks redundant links, so half your expensive fabric sits idle. Put together: a hard ceiling on segments, traffic sprayed everywhere while learning, and links you paid for doing nothing.

VXLAN fixes the ceiling and the idle links. EVPN fixes the flooding. They are almost always deployed together, which is why people say "EVPN-VXLAN" as one word.

VXLAN: the envelope inside the envelope

VXLAN is defined in RFC 7348 (2014). The sending VTEP takes the host's entire Ethernet frame and wraps it: a VXLAN header carrying the 24-bit VNI, then a UDP header (destination port 4789), then an outer IP header addressed from this VTEP to the receiving VTEP, then an outer Ethernet header for the next physical hop. Twenty-four bits of VNI means 16,777,216 possible segments — the 4,094 ceiling is gone.

Two details do quiet, important work. First, the UDP source port is not a real port — the VTEP fills it with a hash of the inner frame, purely so the underlay's ECMP (equal-cost multi-path) hashing sees many different flows and spreads them across every spine link instead of pinning one tunnel to one link. Second, the tax: 50 extra bytes means a 1,500-byte inner frame no longer fits a 1,500-byte underlay. Real fabrics raise the underlay to 9,216-byte jumbo frames; if you cannot, the overlay MTU drops to 1,450. Our MTU guide and calculator below do this math for any tunnel stack.

EVPN: the directory that replaces shouting

Plain VXLAN still learned by flooding. EVPN (Ethernet VPN, RFC 7432; its use with VXLAN is RFC 8365) gives VXLAN a real control plane: leaf switches use MP-BGP (multi-protocol BGP) to advertise where things live, before any traffic needs to go there. You only need three route types to understand 90% of a fabric:

Type 2 — MAC/IP advertisement. "This MAC address, with this IP address, lives behind my VTEP, in this VNI." This is the workhorse. Because every leaf receives Type 2 routes for the whole fabric, a leaf knows exactly which remote VTEP owns a destination MAC — no flooding for known traffic. Type 2 is also what makes ARP suppression possible: when a host ARPs for a neighbour, its own leaf already knows the answer from BGP and replies locally instead of broadcasting the question across the fabric.

Type 3 — Inclusive multicast Ethernet tag. "I have VTEPs' attention for this VNI — send me its BUM traffic." Each leaf advertises one Type 3 route per VNI it hosts, and the set of Type 3 routes becomes the replication list: when a frame genuinely must be copied to everyone in the segment (a broadcast, for example), the ingress leaf copies it only to the VTEPs on that list. This is called ingress replication, and it is flooding with a guest list instead of an open door.

Type 5 — IP prefix. "This whole subnet is reachable through me." Type 5 carries Layer 3 reachability, which is how routed overlays and external connectivity work. Two more types complete the set: Type 1 (Ethernet auto-discovery) and Type 4 (Ethernet segment) handle multi-homing — a host connected to two leaves at once — including electing which leaf forwards BUM traffic for that segment.

Walking one packet through the fabric

Host A (behind Leaf 1) talks to Host B (behind Leaf 2), same VNI, different leaves. 1. A sends a frame to B's MAC address. 2. Leaf 1 looks up that MAC in its EVPN-learned table — installed earlier from Leaf 2's Type 2 route — and knows the owner is Leaf 2's VTEP. 3. Leaf 1 wraps the frame (VNI in the VXLAN header, destination UDP 4789, outer IP to Leaf 2), and the spines route the outer packet like any IP traffic, ECMP-spread by the UDP source-port hash. 4. Leaf 2 unwraps it and delivers the original frame to B, untouched. If A and B are in different subnets, the first leaf routes instead of bridging: with symmetric IRB (integrated routing and bridging), the packet is routed into a shared Layer 3 VNI tied to the tenant's VRF, carried across, and routed again at the far leaf. Either way, A's gateway never changed — the anycast gateway means Leaf 1 and Leaf 2 both answer ARP for the same gateway IP and MAC, so hosts can move between leaves and keep the exact same default gateway.

Five mistakes that break real fabrics

1. Forgetting the 50 bytes. The underlay MTU stays 1,500, small pings pass, and large transfers stall with no clean error — because the encapsulated packet is silently too big. Set jumbo MTU on every underlay interface, or clamp the overlay to 1,450, and test with a large ping with the do-not-fragment bit set.

2. Blaming the overlay for an underlay fault. EVPN cannot fix a broken underlay. Troubleshoot in order: underlay reachability between VTEP loopbacks first, then BGP EVPN sessions and routes, then VTEP peers and VNI state, then the MAC and ARP tables. Most "EVPN bugs" are a missing underlay route.

3. Inconsistent VLAN-to-VNI mapping. The mapping is locally configured on each leaf. If Leaf 1 maps VLAN 10 to VNI 10010 and Leaf 2 maps it to 10020, the two halves of the segment never meet — and nothing alarms, because each leaf is internally consistent. Keep the mapping mechanical (VLAN 10 → VNI 10010) everywhere.

4. Expecting EVPN to flood less but configuring it like old VXLAN. If BGP EVPN sessions are down, many implementations fall back toward flood-and-learn behaviour for BUM traffic. A fabric that "works but is chatty" is often a fabric whose control plane is quietly broken. Check the route counts, not just the pings.

5. One gateway to rule them all. Sending all inter-subnet traffic up to a single central gateway pair recreates the bottleneck EVPN was meant to remove. The anycast gateway on every leaf exists precisely so routing happens where the host is. Use it.

Where this leaves your design

EVPN-VXLAN is the default answer for a modern multi-tenant data center fabric: a simple IP underlay (often eBGP, link by link) that any switch can route, and a smart overlay where BGP carries the tenant state. The skills stack in that order too — get the Clos topology and its oversubscription right first, then the overlay plan (VNIs, route targets, gateways) on top of it. The two free tools below cover exactly those two steps, and the third checks what the extra headers do to your MTU.