InfiniBand vs RoCE Explained: Two Ways to Move Memory Between GPUs

The short answer

What is the difference between InfiniBand and RoCE? Both move data the same way — RDMA, one network card writing straight into another machine's memory. InfiniBand is a whole separate network built only for that job, and it never drops packets by design. RoCE runs the same trick over ordinary Ethernet, which is cheaper and far more open, but only stays drop-free if engineers carefully configure it to be. If you remember one sentence, make it that one: this is not a contest between a fast technology and a slow one. It is a contest between lossless by design and lossless by configuration.

Think of two ways to run a courier service that must never, ever lose a parcel. InfiniBand is a private railway built for the couriers: its own tracks, its own trains, its own signalling, and a rule built into the track itself — a train may only leave the station when the next station has confirmed a free platform. Nothing is ever dropped because nothing is ever sent into a full station. RoCE is the same courier service run on the public roads: the vans are ordinary, the roads are shared with everyone, and the service stays lossless only because the operators add rules — traffic lights that can freeze one lane (that is PFC), warning signs that tell drivers to slow down early (ECN), and a control centre adjusting each driver's speed (DCQCN). Done well, the road service is excellent. But the losslessness is something the operators do, every day, not something the road is.

The shared job: why RDMA exists

Training a large AI model means thousands of GPUs constantly swapping partial results — an operation called a collective, where every GPU's numbers must reach every other GPU before the next step of training can begin. Supercomputers (the HPC world, High-Performance Computing) have done the same thing for decades with physics and weather models. In both worlds the network is not plumbing in the background; the whole cluster moves at the speed of its slowest exchange, so the exchange has to be as close to "the other machine's memory is just… over there" as physics allows.

That is what RDMA delivers. A normal network transfer hands data to the operating system, copies it through buffers, and interrupts the receiving CPU to deal with it. An RDMA transfer skips all of that: the sending adapter places the data directly in the receiving machine's memory, at an address agreed in advance, and the receiving CPU learns it arrived by checking a small completion note. No copies, no system calls in the data path, almost no CPU spent. The price of that elegance is fragility: the machinery that makes ordinary networks forgiving of dropped packets — software noticing and re-sending — is exactly what RDMA removed. RDMA transports expect the fabric underneath to deliver essentially everything, in order, without drops. Both InfiniBand and RoCE exist to provide a fabric that keeps that promise. They just keep it in opposite ways.

InfiniBand: the private railway

InfiniBand is a complete networking architecture defined by the IBTA (InfiniBand Trade Association). It is not Ethernet with extras — it has its own link layer, its own transport, its own switches, and its own adapters (called HCAs, Host Channel Adapters). A cluster that chooses InfiniBand builds a second, separate network that carries nothing but this traffic, managed as one fabric.

Two design decisions define it. The first is credit-based flow control. Before a sender may transmit, it must hold credits, and credits represent real, advertised buffer space at the receiver on the other end of that link. As the receiver frees buffer, it issues more credits. A sender literally cannot overrun its neighbour, because permission to send is the free space. Packet loss from congestion is not unlikely on InfiniBand; it is structurally impossible, link by link. Nobody configures this, tunes it, or monitors thresholds for it — it is how the link layer works.

The second is the subnet manager: a central service (usually running on a switch or a management node, with a standby ready to take over) that discovers every adapter and switch in the fabric, assigns each port its address (a LID, Local Identifier), computes the routes between them, and programs the switches. Joining the fabric, addressing, and routing are one coordinated act, which is a large part of why InfiniBand fabrics behave so predictably — and why running one is a specialism of its own, with its own tooling (the fabric-management world around tools like UFM) that an Ethernet team does not already know.

Speed generations have names instead of numbers: EDR runs at 100 Gb/s per port, HDR at 200, NDR at 400, and XDR at 800. In practice the InfiniBand market is also a single-vendor stack: NVIDIA (through its Mellanox line) supplies essentially all the adapters and switches, which buys tight integration — including in-network computing features such as SHARP, where the switches themselves help with the collective maths — at the cost of having exactly one supplier to negotiate with.

RoCE: the same couriers, on public roads

RoCE (RDMA over Converged Ethernet) takes the InfiniBand transport — the same RDMA verbs, the same programming model, largely the same adapter silicon — and carries it inside Ethernet. Version 1 wrapped it directly in an Ethernet frame (EtherType 0x8915), which confined it to a single Layer 2 broadcast domain. RoCEv2, the version every modern deployment means, wraps it in UDP inside IP — destination UDP port 4791 — and that one change makes RDMA routable: it crosses subnets, rides a normal leaf-spine IP fabric, gets load-balanced by ECMP like any other flow (the UDP source port doubles as the flow identifier the switches hash on), and is run by the same teams, tools and multi-vendor switches as the rest of the data center. There is no subnet manager; addressing and routing are just IP.

What Ethernet does not give RoCE for free is the promise. Ordinary Ethernet drops packets when a queue overflows — that is normal, expected, and exactly what RDMA cannot tolerate. So a RoCE fabric is deliberately engineered to behave losslessly for the RDMA traffic class, with three mechanisms working together: PFC (Priority Flow Control, IEEE 802.1Qbb) lets a filling switch tell its direct neighbour to pause one traffic priority, hop by hop, before anything is dropped; ECN (Explicit Congestion Notification) lets the switch mark packets as queues start to build, warning receivers early; and DCQCN, the congestion-control algorithm on the adapters, turns those marks into sender rate reductions, so PFC rarely has to fire. When the three are tuned together — thresholds, headroom buffers, priorities all consistent on every switch and adapter — a RoCE fabric delivers RDMA performance close to InfiniBand's for most workloads. When they are not, the failure modes are Ethernet-shaped: PFC storms that pause innocent traffic, deadlocks between paused switches, and silent drops in the one class that must never drop. Our guides on RoCEv2 lossless Ethernet and on PFC and ECN walk through that machinery in full.

Side by side

So which one is faster?

Mostly, this is the wrong first question, and the honest answer disappoints people who want a winner. At the same port speed — and the two reach the same speeds, generation for generation — a well-built RoCE fabric and an InfiniBand fabric deliver broadly comparable bandwidth to real collective workloads, which is why both are credible answers for AI training. InfiniBand keeps an edge in the places its design predicts: latency at the very bottom end and, more importantly, predictability under load, because nothing about its losslessness depends on thresholds a human set. A congested InfiniBand link slows at exactly the rate the credits dictate. A congested RoCE fabric behaves exactly as well as its PFC/ECN tuning — excellent in a well-run shop, and the source of the famous 2 a.m. incidents in shops that copied a config from a different topology. If a vendor shows you a single latency number for either side, treat it the way you would treat a car's top speed as a reason to buy it for a commute: real, measured, and mostly not the thing that decides your day.

The deciding factors are usually the unglamorous ones. People: a hyperscaler with a deep Ethernet team and its own switch software gets compounding value from RoCE — one fabric discipline, one set of tools, several vendors bidding. A research lab with six people and a grant gets compounding value from InfiniBand — the fabric arrives as one vendor's tested stack and the losslessness is not their problem to create. Money: Ethernet's multi-vendor market competes switch and adapter prices down in a way a single-vendor fabric does not, while InfiniBand's integration can be cheaper in engineering hours than it looks on the invoice — both statements are true, and which dominates depends on cluster size and staff. Roadmap: Ethernet's ecosystem is vastly larger and its improvements arrive from many directions at once; InfiniBand's roadmap is one vendor's, executed tightly. And note what does not change: the cluster's front-end network — management, storage access, talking to the outside world — is Ethernet in both worlds, which is why AI clusters run two networks either way, as our front-end vs back-end fabrics guide explains. The InfiniBand-versus-RoCE choice is only ever about the back-end.

Choosing, honestly

Choose InfiniBand when the cluster's whole purpose is tightly coupled collective work (large-scale AI training, classic HPC), the team is small or HPC-raised rather than Ethernet-raised, predictable behaviour under full load matters more than vendor choice, and buying one integrated stack from one vendor is acceptable — even restful. Choose RoCEv2 when the organisation already runs serious Ethernet, wants competing vendors and a multi-source roadmap, plans to grow the fabric with its own designs and software, and — the condition that decides everything — has, or will build, the operational discipline to tune and continuously verify the lossless configuration. A RoCE fabric chosen to save money and then run casually is the worst of both worlds: it has Ethernet's failure modes with RDMA's intolerance of them.

Four mistakes buyers actually make

1. Comparing port speeds and calling it a comparison. 400 is 400 on both sides. The differences that decide a deployment are losslessness mechanics, operations and ecosystem — none of which appear in the bandwidth column.

2. Pricing RoCE switches and adapters, but not RoCE operations. The tuning, the verification tooling, and the engineer who understands why the headroom number is what it is are part of the RoCE bill. They are also, fairly, part of why large Ethernet shops find RoCE cheap: they already employ that person.

3. Assuming InfiniBand needs no network engineering. Lossless-by-design removes one class of work, not the discipline: topology design, subnet-manager redundancy, cable plant at 400 Gb/s signalling rates, and collective-aware placement are all still engineering, just InfiniBand-shaped engineering.

4. Mixing the two in one back-end and expecting a blend. Adapters and fabrics do not translate between InfiniBand and RoCE in the data path; a cluster's back-end is one or the other (gateways exist for storage and edge cases, not as a way to merge two training fabrics). Pick per fabric, not per rack.

Try it yourself

If the RoCE side of this comparison is the one your shop would run, the free RoCEv2 lossless Ethernet planner turns this article's argument into numbers: give it your link speeds, buffer sizes and incast pattern and it works out ECN thresholds, PFC headroom and DCQCN starting points, with the deadlock and pause-storm risk flags that decide whether lossless-by-configuration will hold. Designing the fabric it runs on? The data center fabric designer sizes the leaf-spine underneath it. And because incast bursts are where both fabrics earn their keep, the microburst buffer planner shows what a synchronised burst actually does to a shared switch buffer.