Packet Path/

Kernel Bypass Basics for HFT Networks

Interrupts, copies, and syscalls make the stock kernel path cost 10–30 µs a packet — the whole tick-to-trade budget, spent before your code runs. DPDK, Solarflare Onload, and XDP explained with real numbers, real config, and the gotchas that eat your first month.

Low Latency · Intermediate · 10 min · October 2, 2026

Illustration of packets taking a fast direct path around a tall layered kernel stack
In plain EnglishEvery packet your server receives normally travels through the operating system's kernel — the core of Linux — which inspects it, queues it, and copies it before your program ever sees it. That journey costs 10–30 microseconds: an eternity in high-frequency trading. Kernel bypass lets your program talk to the network card directly, skipping the kernel entirely and cutting that delay to about 1–2 microseconds.

The VIP-lane analogy

Picture airport security. Regular passengers queue up, show ID, unpack laptops, and wait while each bag is screened — that is the kernel path: safe, general-purpose, and slow. Now picture the VIP lane: pre-cleared travelers walk straight to the gate with no line and no re-screening — that is kernel bypass: your application gets a private, pre-approved channel straight to the network card.

The tradeoff is the same in both worlds. The VIP lane is blazing fast, but it only goes to one gate (your NIC can no longer do normal networking on that port), and someone has to staff it around the clock (a CPU core spinning at 100% forever, watching for packets).

A few terms you will see everywhere below:

Why the kernel path costs too much

The Linux kernel network stack is excellent engineering — for the job it was designed for: serving everything from laptops to file servers through one general-purpose code path. High-frequency trading is not that job. Walk an inbound packet through it and count the tolls, like toll booths on a highway:

StepWhat happensTypical cost
NIC interruptThe frame lands and the NIC raises an IRQ — a hardware tap on the CPU's shoulder. Default interrupt coalescing may deliberately wait, batching interrupts to save CPU.~1–3 µs, plus 10s of µs of coalescing delay if enabled
Softirq + protocol processingThe driver builds an sk_buff (the kernel's internal envelope for a packet) and pushes it up through Ethernet, IP, and UDP/TCP: route lookup, socket lookup, checksums, filters.~2–5 µs
Queueing + wakeupThe packet waits in the socket receive queue while the kernel wakes your thread and the scheduler finds it a core — like waiting for a clerk to finish with the previous customer.~1–5 µs — unbounded under load
Copy to user spacecopy_to_user moves the payload from the kernel buffer into your application's buffer — a full memory copy of every byte.~0.5–1 µs per small packet, plus cache eviction
Syscall overheadrecvmsg() entry and exit — the "please hand me the packet" request itself.~0.5–1 µs per call

Add it up: a realistic kernel path costs 10–30 µs one-way on a tuned, idle box — and the tail under load is far worse. Watch a single packet pay every one of those tolls:

One packet, two host paths — watch it travel (animated, loops)
KERNEL PATH · ~10–30 µs NIC~1–3 µs IRQ + softirqwake + batching kernel stack~2–5 µs socket queuewaits here · ~1–5 µs copy + syscall~1–2 µs strategy appfinally ✓ arrived: 10–30 µs KERNEL BYPASS · ~1–2 µs NICDMA write DMA ring in your memoryno copy busy-poll loopno wakeup strategy app~1–2 µs ✓ arrived: ~1–2 µs Both packets start together. The bypass packet is done before the kernel packet leaves the socket queue.

The bypass model: three ideas

Kernel bypass removes the kernel from the data path entirely. Three ideas do all the work:

  1. User-space drivers. Your application links a poll-mode driver that owns the NIC's queues directly; the kernel driver is unbound from the device. The NIC DMA-writes frames into ring buffers living in your process's own memory.
  2. Busy-polling. No interrupts, no wakeups, no scheduler. A dedicated core spins in a tight loop checking the ring — packet or no packet. It sounds wasteful; it is what determinism costs. (Like hiring someone to stare at the door all day instead of installing a doorbell — the doorbell has a delay; the stare doesn't.)
  3. Zero-copy. The frame never moves. Your code reads it in place in the DMA buffer, decides, and releases the slot. No sk_buff, no copy_to_user, no evicting your strategy's data from cache.

The result is wire-to-application latency around 1–2 µs, repeatably. The price: one core pegged at 100% forever, a NIC the kernel can no longer see (no ping, no tcpdump on that port), and a server that has become a single-purpose appliance.

Why polling beats interrupts: watch it happen

The deepest change is philosophical: instead of the CPU sleeping until a packet arrives (and paying a wake-up toll every time), a polling core never sleeps at all. The two timelines below run at the same speed — watch when each packet gets served:

Interrupt-driven vs. poll-mode packet pickup (animated, loops)
INTERRUPT-DRIVEN · the core sleeps to save power core: 💤 sleeping NIC packet arrives… …then the packet waits: 1–3 µs wake-up + however long the scheduler takes POLL-MODE (DPDK) · the core never sleeps polling 100% NIC packets stream in nothing sleeps, nothing wakes — the loop checks the ring thousands of times per µs The interrupt path pays a wake-up toll on every packet. The poll path pays a whole CPU core, once, forever.

The interrupt path pays a wake-up toll on every packet. The poll path pays a whole CPU core, once, forever.

Where the bypass happens: inside the server

Kernel bypass is not just software — the physical layout of the server decides whether it works. The NIC must DMA into memory on the same NUMA node (CPU socket) as the polling core, or every packet pays a cross-socket toll — like having your desk in one building and your filing cabinet in another. Here is the whole machine:

The HFT server: where the bypass lives (animated)
1U TRADING SERVER · NUMA node 0 NIC feed port → DPDK kernel unbound mgmt port → kernel stack ssh · ping OK here PCIe ×16 CPU socket · NUMA node 0 0ctrl 1 2 3 4RX poll100% 5 6strategy 7 DDR memory DMA rings 1 GiB huge pages (2 MiB pages) NIC writes here directly DMA: NIC → memory, no CPU core 4 polls Golden rule: NIC, poller core, strategy core, and DMA memory all on the same NUMA node.

The three approaches worth knowing

DPDK: you own everything above the wire

The Data Plane Development Kit hands you raw frames and gets out of the way. That freedom is total — including the freedom to reimplement ARP, IP, UDP, and TCP yourself. DPDK fits UDP multicast market data and custom feed handlers, where TCP was never invited anyway.

Solarflare Onload and TCP Direct: keep your sockets

Onload accelerates the standard POSIX sockets API in user space on Solarflare (now AMD) NICs: most of your existing TCP/UDP application runs unchanged, just faster. This is the classic fit for order entry: TCP sessions to the exchange where reliability matters and rewriting the app in DPDK would be madness. The trade is vendor lock-in on the NIC.

XDP: the halfway house

XDP runs an eBPF program inside the NIC driver at the earliest possible moment — before the kernel allocates an sk_buff. You keep ip route, tcpdump, and your sanity. Wire-to-app latency is not quite DPDK-class, but for line-rate filtering, DDoS shedding, and a gradual migration path, XDP is the pragmatic middle.

What it actually looks like

Bypass is mostly machine preparation: huge pages for the DMA rings, the NIC handed over to a user-space driver, and cores pinned so the poller never shares. A minimal DPDK host setup — read each step as "prepare the machine, then hand it over":

# 1) reserve huge pages for the DMA rings (512 x 2 MiB = 1 GiB)
#    "set aside 1 GB of memory in big chunks for the packet rings"
echo 512 | sudo tee /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages
sudo mkdir -p /mnt/huge
sudo mount -t hugetlbfs none /mnt/huge

# 2) take the feed NIC away from the kernel driver
#    "unplug this network port from Linux and give it to DPDK instead"
sudo dpdk-devbind.py --bind=vfio-pci 0000:3b:00.0

# 3) run: core 0 = main, core 4 = RX poller, core 6 = strategy
#    (all three on the same NUMA node as the NIC)
#    "start the app: one core supervises, one core polls for packets forever, one core runs the trading logic"
sudo ./build/feedhandler -l 0,4,6 -n 4 -- --rxq 1

The -l 0,4,6 flag is the whole philosophy in one line: core 0 runs control, core 4 does nothing but poll the receive queue, core 6 runs strategy. One queue, one poller, one owner.

Gotchas that eat your first month

Validate like the business depends on it (it does)

The ruleIf you cannot account for where each microsecond went, you do not have a latency number — you have a rumor.
Try it yourselfBudget the path this article optimizes: the tick-to-trade calculator breaks any route into components so you can see exactly where the host stack sits. For the long-haul side, the bandwidth-delay calculator covers window sizing and throughput bounds.
Further reading
Key takeaways

Keep reading