Packet Path Troubleshoot

Troubleshoot › Symptom cases › vPC Consistency Check Failed

Data Center · Nexus vPC

vPC Consistency Check Failed

Engineers search this as: 'vpc consistency check failed' · 'vpc type-1 configuration incompatible' · 'ospf over vpc not working'

Evidence: Cisco documentation only — not yet lab-verified

In plain English: Two Nexus switches are pretending to be one single switch, so a server or another switch can plug into both at once and survive either one failing. Before they forward a single frame, the two partners compare notes on the settings that must agree. If a core setting disagrees, one partner politely sits out rather than risk a loop or a black hole — your device keeps working through one switch only, or not at all. If a lesser setting disagrees, they warn you but keep forwarding — which is sneakier, because everything looks up while some traffic quietly goes wrong.

The symptom

A dual-homed server, storage node, or downstream switch is plugged into both Nexus switches of a vPC pair — and it has lost one leg, or both. On the switches, `show vpc` says the consistency check failed, a member vPC sits `down*` with the reason 'Compatibility check failed', or a Type-2 status reads failed while every link light is green. Sometimes nothing looks down at all: the vPC is up, but an SVI exists on only one partner, so traffic that lands on the wrong partner takes a strange path — or, in the routing variant of this fault, an OSPF adjacency that should form over a vPC VLAN never forms, or forms and later drops.

N9K-1# show vpc

vPC domain id                     : 1
Peer status                       : peer adjacency formed ok
vPC keep-alive status             : peer is alive
Configuration consistency status  : failed
Configuration inconsistency reason: vPC type-1 configuration incompatible - STP Mode inconsistent
Per-vlan consistency status       : success
Type-2 consistency status         : success
vPC role                          : primary
Graceful Consistency Check        : Enabled

Check first — in this order

  1. show vpc — and show vpc briefStart here, on BOTH peers. Read four lines before anything else: peer status, keep-alive status, configuration consistency status (and its reason line), and Type-2 status. Then scan the vPC table for any member sitting down* with 'Compatibility check failed'. This one screen tells you which of the ranked causes you are even dealing with.
  2. show vpc consistency-parameters globalThe global Type-1 list, local value beside peer value. Look for the row where the two columns differ — the worked example in Cisco's own guide is STP Mode: Rapid-PVST on one peer, MST on the other. A Type-1 row that differs here suspends vPCs; Type-2 rows in the same output (SVIs, QoS) only warn.
  3. show vpc consistency-parameters vpc <id> — for each failed vPCThe per-vPC Type-1 list: member speed, duplex, MTU, LACP mode, native VLAN, STP port type. Cisco's worked example is a speed mismatch — 10 Gb/s local, 100 Gb/s peer — and its second is STP Port Type: Edge Trunk Port on one side, Default on the other. Diff this output before you diff configs by eye.
  4. show vpc consistency-parameters vlans — and show vpc consistency-parameters interface port-channel <peer-link>Per-VLAN consistency, plus the peer-link's own parameters. If the peer-link itself is the problem, every vPC downstream of it is a victim, not a cause.
  5. Check the peer-keepalive like a network fault, not a vPC fault: correct VRF? ping the peer-keepalive destination from the source, in that VRF — is ARP resolved?The keepalive is just UDP between two addresses over a management path. Wrong VRF, a dead management switch, or unreachable source/destination addresses all read as 'peer is not reachable through peer-keepalive'. Note the trap from Cisco's own guide: the vPC itself can stay fully up while the keepalive is down — that state is a warning about your safety net, not the outage itself.
  6. show running-config on BOTH peers — diff the SVIs and the one mismatched parameter directlyFor a Type-2 failure ('SVI type-2 configuration incompatible'), list every interface-vlan on each peer: an SVI that exists, is admin-up, or is routing-enabled on one peer only is the mismatch. And if your symptom is a routing adjacency over a vPC VLAN, read the 'Operational Layer3 Peer-router' line in show vpc while you are here: Disabled means the design piece this fault usually turns on was never switched on, or was switched on on one peer only.

What the evidence means

The idea in one line

vPC is a promise two switches make to a third device: 'we are one port-channel.' That promise is only safe if both partners forward identically — so before a vPC comes up, and whenever the configuration changes, the partners exchange and compare two tiers of settings.

Type-1: MUST match — a mismatch suspends the vPC

Spanning-tree mode and port type, member speed, duplex, MTU, LACP mode, native VLAN. With the default graceful check, the primary keeps forwarding and the secondary suspends its side — which is why the classic report is 'it works, but only through one switch'.

Type-2: SHOULD match — a mismatch only warns

SVIs and their routing state, QoS values. A Type-2 mismatch never suspends anything: it logs a warning and forwarding continues, possibly wrongly — which is why Type-2 is the more dangerous failure to ignore.

The two lifelines: peer-link and peer-keepalive

Until the peer-link first comes up, all vPCs stay suspended, and a later peer-link failure suspends the secondary's vPCs. The keepalive is the tiebreaker that stops both partners going active alone if the peer-link dies.

Routing over a vPC VLAN is a design rule, not a mismatch

A Layer-3 device that peers with BOTH vPC partners over a vPC VLAN is only supported when peer-gateway is enabled first and layer3 peer-router is then configured under the vPC domain on BOTH peers. layer3 peer-router stops the TTL being decremented for routing packets that cross the peer-link — OSPF hellos travel with TTL 1, so without it an adjacency can form by hashing luck and later fall (documented failure: stuck in EXSTART) when the hashing changes or a vPC leg drops. Configured on one peer only, the operational state stays Disabled. Platform and version support varies (see the platform card below), and peering between the two vPC partners themselves, over the peer-link, is a different, baseline-supported topology — do not confuse the two.

Platforms and defaults

Cisco Nexus NX-OS (N3K/N5K/N6K/N7K/N9K vPC domains). The graceful consistency check is enabled by default (introduced in NX-OS 5.0(2)N2(1) on the Nexus 5000 lineage): on a Type-1 mismatch the PRIMARY keeps its vPCs up and the SECONDARY suspends its side. Routing adjacencies over a vPC VLAN need layer3 peer-router explicitly configured and are version/platform dependent — Nexus 9000 from NX-OS 7.0(3)I5(1), Nexus 7000 from 7.2(0)D1(1) (unsupported on M1, F1, F2 and M2 linecards), Nexus 5000/6000 from 7.3(0)N1(1) (all per Cisco doc 118997, Table 2). Never present one platform's rule as universal; separate Layer-3 links for routing adjacencies are the always-safe design.

Root causes, ranked

Root causeHow you recognize it
Type-1 global mismatchThe two peers disagree on a domain-wide setting — Cisco's worked example is spanning-tree mode, Rapid-PVST on one peer and MST on the other. Every vPC in the domain is exposed; with graceful check the secondary suspends its side.
Type-1 per-vPC mismatchOne vPC's own parameters differ between the peers: member speed (10G vs 100G in Cisco's example), STP port type (Edge Trunk vs Default in its other example), MTU, LACP mode, native VLAN, or duplex. Only that vPC suspends — its neighbours keep working, which is the fingerprint.
Peer-link downA dead cable or SFP, or a peer-link port-channel built wrong: it must be an L2 trunk carrying only vPC VLANs, with STP port type network and Bridge Assurance on both peers. Before the peer-link has ever come up, all vPCs stay suspended; if it fails later, the secondary suspends its vPCs until it recovers.
Peer-keepalive not reachableWrong VRF for the keepalive, management path down, or source/destination addresses that cannot reach each other (check ARP and ping in that VRF). Remember: the vPC can stay up in this state — what you have lost is the dual-active safety net, and if the peer-link then fails, both partners can go active alone.
Type-2 SVI incompatibility'SVI type-2 configuration incompatible': a VLAN interface exists, is admin-up, or has routing enabled on one peer and not the other. No suspension, only a syslog warning — and uneven forwarding on that VLAN while everything reports up. This is the cause engineers walk past because nothing is red.
vPC ID / member interface mismatchThe member port-channel on one peer carries a different vPC ID, or its member interfaces are not the twin of the other peer's. show vpc names the member as failed; show vpc consistency-parameters vpc <id> shows exactly which value disagrees.
Routing over vPC without layer3 peer-router (design variant)A Layer-3 device peers with both vPC partners over a vPC VLAN, but peer-gateway + layer3 peer-router are not configured on both peers — or the platform/NX-OS version does not support it. It presents as a routing fault (adjacency never forms, sticks in EXSTART, or drops when a leg fails) on top of a vPC that may itself report fully consistent. Check the Operational Layer3 Peer-router line before blaming OSPF.
Before you touch production: Almost every fix in this case touches something the whole vPC domain stands on. Changing the spanning-tree mode or an STP port type, or disturbing the peer-link, can itself suspend or flap every vPC on the pair — dual-homed hosts lose a leg, or both, in the middle of your 'fix'. Snapshot the full running-config of BOTH peers first, work in a maintenance window, change ONE parameter at a time, and re-run show vpc after each single change before touching the next. Type-2 SVI fixes are lower risk, but creating, deleting, or enabling routing on an SVI changes live forwarding for that VLAN the moment you do it.

The fix

  1. Match the Type-1 global parameter on both peers — for the STP-mode example, set the same mode on the peer that differs ('spanning-tree mode mst' or 'spanning-tree mode rapid-pvst', both sides identical). Decide which side holds the intended value from the design document, not from which switch you happen to be logged into.
  2. Match the per-vPC Type-1 parameters: same member speed, duplex, MTU, LACP mode, native VLAN, and STP port type on both peers' member port-channels. Fix them in the interface configuration on the side that differs, one parameter at a time.
  3. Restore the peer-link if it is down: physical layer first (cable, SFPs, member interfaces up), then confirm the port-channel is an L2 trunk allowing only vPC VLANs, with STP port type network and Bridge Assurance enabled on both peers.
  4. Restore the peer-keepalive: put both ends in the correct VRF, confirm the management path between the management ports is up, and prove source-to-destination reachability (ping in that VRF, ARP resolved) before expecting the status to clear.
  5. Match the SVIs for a Type-2 failure: for every VLAN in the domain, the SVI should exist — or not exist — in the same admin and routing state on both peers. Add the missing SVI, or shut/remove the stray one, per the design.
  6. Align the vPC ID and member interfaces: the same vPC ID on both peers' port-channels, with truly mirrored members.
  7. For routing over a vPC VLAN (the OSPF variant): on BOTH peers, in the vPC domain configuration, enable peer-gateway first and then layer3 peer-router, and confirm show vpc reports 'Operational Layer3 Peer-router: Enabled'. If your platform or NX-OS version does not support it (check Cisco doc 118997, Table 2 for your series), do not improvise — move the routing adjacency onto separate Layer-3 links between the peers and the Layer-3 device, which is supported everywhere.

Verify it worked

  1. show vpc — on both peersPeer status 'peer adjacency formed ok', keep-alive 'peer is alive', Configuration consistency status: success, Type-2 consistency status: success, and no member vPC down or down*.
  2. show vpc consistency-parameters global — and vpc <id> for the vPC you fixedEvery Type-1 row shows identical Local and Peer values. No row differs.
  3. Traffic check from a dual-homed hostThe host reaches the network with both legs up — and still reaches it when you shut one leg (a controlled test, in the window): losing one partner must now be a non-event, which was the point of vPC in the first place.
  4. For the OSPF variant: show ip ospf neighbors (NX-OS)Adjacencies to the Layer-3 device read FULL on both peers and stay FULL — including after the controlled one-leg test above.

Did this fix it for you? One tap — anonymous, nothing to type, and nothing you send is ever published.

Going deeper — production notes

Note

The two tiers exist because the two mistakes cost differently. A Type-1 disagreement (spanning-tree mode, member speed, MTU) can loop or black-hole traffic, so the pair refuses to forward together: with the default graceful check the primary keeps its vPCs up and the secondary suspends its side. A Type-2 disagreement (an SVI present on one peer only, QoS values) cannot loop anything, so the pair warns and keeps forwarding. Read which tier failed before you read anything else — it tells you whether you are restoring service or hunting a silent wrongness.

Note

Treat the peer-link as the pair's shared nervous system, not as 'just a trunk'. It carries the consistency comparison itself, plus control traffic and any orphan traffic — and until it has come up at least once, every vPC in the domain stays suspended by design. Its build is specific: an L2 trunk carrying only vPC VLANs, STP port type network, Bridge Assurance on both ends. If every vPC fails at once, suspect the peer-link before any individual vPC's parameters.

Note

The peer-keepalive is a separate, humble UDP conversation over a management path — and it can be down while the vPC is fully up, because the peer-link is what actually carries forwarding. What a dead keepalive removes is the tiebreaker: if the peer-link then fails, neither partner can prove the other is dead, and both can go active alone. Fix a dead keepalive the same day even when 'nothing is broken' — it is the safety net, and you find out it was missing at the worst possible moment.

Note

A Layer-3 device peering with both vPC partners over a vPC VLAN is a supported design only with two switches flipped, in order: peer-gateway first, then layer3 peer-router, on BOTH peers. The reason is one number: OSPF hellos carry TTL 1, and layer3 peer-router stops the TTL decrement for routing packets crossing the peer-link. Miss it, and the adjacency can form by hashing luck, sit stable for weeks, then drop into EXSTART the day a leg fails or the hashing changes. If your platform or NX-OS version is not on Cisco's supported list (doc 118997), no knob fixes it — separate Layer-3 links are the design.

Note

Most consistency failures are not component failures at all — they are drift. One peer got a spanning-tree change during an incident, a port-channel was built from memory instead of the template, an SVI was added for a test and never mirrored. The consistency check is simply the first system honest enough to refuse. The operational fix is process: changes touch both peers or neither, and the pair's running-configs get diffed on a schedule, not after an outage.

Note

If you hand this to Cisco TAC, send: 'show vpc' and 'show vpc brief' from both peers taken in the same minute, the full consistency-parameters outputs (global, the failed vPC, the peer-link interface, and vlans), the keepalive state with ping and ARP proof in its VRF, both complete running-configs, and both 'show version' lines. That bundle answers their first five questions and turns a week of callbacks into one reply.

Reproduce this failure: Packet Path's hands-on labs run this class of fault in a simulated network, so you can break it, run the checks above, and fix it there before touching production. Open the labs.

Prevention

Still stuck? Before you open a TAC case

Work this ladder before you open a case. Consistency failures are almost always a real mismatch your eyes scrolled past, a marginal peer-link optic, or drift from the last one-sided change — and if yours survives all five checks, you will hand TAC a case they can answer in one reply.

The last checks — the ones that close most "unsolvable" cases

  1. Diff the outputs, not the screens: save 'show vpc consistency-parameters global', 'show vpc consistency-parameters vpc <id>' and 'show vpc consistency-parameters vlans' from both peers to files and diff them mechanically. The mismatching row is in there — scrolling two CLI sessions side by side is exactly how it hides.
  2. Interrogate the last change: pull the change record and the config archives for both peers. If consistency broke inside the window of a change that touched one peer only, roll that single change back and watch 'show vpc' — a rollback that restores consistency is both the fix and the confession.
  3. Prove the peer-link physically: swap its cables and optics for known-good ones and read the transceiver DOM values. A marginal optic on the peer-link makes comparisons and keepalives flap in ways no configuration change will ever fix.
  4. Bypass the keepalive path, inside a window: temporarily source the peer-keepalive from a known-good path or VRF and watch whether the status clears. If it does, your management network was the fault — not vPC.
  5. Check the software before the hardware: search your exact NX-OS version plus 'vPC consistency' in Cisco's bug search and field notices, and confirm both peers run the same version. Known defects report false mismatches, and mixed versions mid-upgrade are supposed to disagree transiently.

It really is a TAC case when

If you do open one, hand them this bundle

Sources

Related symptoms

Symptom case

OSPF Adjacency Stuck in EXSTART

Two routers that should exchange routes sit forever in the state EXSTART (or bounce between EXSTART and DOWN).

Symptom case

BGP Neighbor Stuck in Active

A BGP session never comes up.

Symptom case

Port Keeps Going Err-Disabled

A switch port shuts itself off.

Symptom case

Bridging Loop / Broadcast Storm

A whole VLAN — sometimes a whole switch or site — melts down at once.