How eBPF and XDP Actually Work: Running Sandboxed Bytecode in the Network Driver

At 100 gigabits per second, a network interface receives a minimum-sized 64-byte Ethernet frame every 6.72 nanoseconds. Within that window, the host operating system must transfer the frame across the peripheral bus, parse packet headers, make a routing or filtering decision, and either forward the data or deliver it to an application socket buffer.

For decades, the Linux kernel handled network frames through a uniform, general-purpose abstraction: the socket buffer, or struct sk_buff. When a frame arrives on a network interface card (NIC), the device driver allocates an sk_buff, initializes over 200 bytes of protocol metadata, executes New API (NAPI) polling loops, and hands the buffer to the core network stack. The packet travels through the Netfilter firewall hooks, traverses connection tracking (conntrack), enters routing decision tables, passes through socket queues, and triggers thread wakeups.

While sk_buff provides the flexibility required for complex IP routing, policy routing, and stateful firewalling, its memory allocation and cache footprint create a computational ceiling. Allocating, initializing, and freeing an sk_buff consumes hundreds of CPU cycles. Under volumetric distributed denial-of-service (DDoS) attacks saturating links at tens of millions of packets per second, the host CPU spends all its cycles allocating and freeing kernel memory structures before the packet filtering rules can even inspect the IP header.

The eXpress Data Path (XDP), coupled with the extended Berkeley Packet Filter (eBPF) virtual machine, solves this throughput bottleneck by executing custom, sandboxed bytecode directly inside the device driver ring buffer. XDP executes before the Linux kernel allocates an sk_buff, enabling line-rate packet drops, programmatic routing, and Layer 4 load balancing at over 20 million packets per second per CPU core.

This article examines the internal architecture of eBPF and XDP. We trace the path of raw Ethernet frames through the NIC receive descriptor ring, dissect the eBPF instruction set and register model, analyze the formal verification algorithms that guarantee kernel safety without runtime crashes, explore Just-In-Time (JIT) machine code generation, and evaluate BPF map memory structures.

+-----------------------------------------------------------------------------------+
|                     LINUX RECEIVE PATH: CLASSICAL VS XDP                          |
+-----------------------------------------------------------------------------------+
|                                                                                   |
|  1. CLASSICAL NETWORK RECEIVE PATH (High Memory & CPU Overhead)                   |
|     Physical Wire -> NIC RX Ring -> Alloc sk_buff -> Netfilter / iptables -> Socket|
|                                     (Expensive!)    (Connection tracking)         |
|                                                                                   |
|  2. EXPRESS DATA PATH (XDP) HOOK (Zero-Allocation Line-Rate Processing)           |
|     Physical Wire -> NIC RX Ring -> [ XDP BPF PROGRAM ]                          |
|                                            |                                      |
|            +-------------------------------+-------------------------------+      |
|            |               |                               |               |      |
|            v               v                               v               v      |
|        XDP_DROP         XDP_TX                        XDP_REDIRECT      XDP_PASS  |
|       (Drop packet   (Echo back out                 (Send to SmartNIC  (Alloc skb |
|        instantly)     same interface)                or AF_XDP socket)  & continue|
|                                                                                   |
+-----------------------------------------------------------------------------------+

The Linux Network Ingress Path and the sk_buff Bottleneck

To understand why XDP delivers a tenfold throughput improvement over traditional kernel networking, one must analyze the physical and memory mechanics of packet arrival.

NIC Direct Memory Access (DMA) and the RX Descriptor Ring

A modern network interface card communicates with the host operating system through circular descriptor queues allocated in host RAM. The receive ring buffer (RX ring) consists of fixed-size hardware descriptors containing:

1. A 64-bit physical memory address pointing to a pre-allocated host memory buffer.
2. Status flags indicating whether the buffer is owned by the NIC or the host driver.
3. Packet length, hardware checksum offload flags, and virtual LAN (VLAN) tags.

When electrical or optical signals arrive on the physical transceiver:

1. The NIC Physical Layer (PHY) deserializes the bitstream and validates the 32-bit Frame Check Sequence (FCS) cyclic redundancy check (CRC).
2. The NIC Media Access Control (MAC) controller reads the next available RX descriptor from host memory across the PCIe bus.
3. The NIC bus-mastering Direct Memory Access (DMA) engine writes the packet payload bytes directly into the host RAM buffer identified by the descriptor address.
4. The NIC updates the descriptor status flags and asserts a PCIe Message Signaled Interrupt (MSI-X) to notify the CPU core.

The NAPI Polling Loop

Servicing an interrupt for every incoming packet would overwhelm the CPU with interrupt handling overhead. Linux uses the New API (NAPI) framework to mitigate this:

1. When the initial packet arrives, the driver Interrupt Service Routine (ISR) disables hardware interrupts for that specific RX queue.
2. The driver schedules a softirq (NET_RX_SOFTIRQ) on the local CPU core.
3. The kernel enters a polling loop (napi_poll()), executing the driver poll method (such as ixgbe_poll or mlx5e_napi_poll) to process packets in batches, typically bounded by a budget of 64 packets per invocation.
4. If the queue is drained completely, the driver re-enables hardware interrupts. If more packets remain, NAPI yields the CPU and reschedules the softirq to preserve fairness.

The Construction of struct sk_buff

In the classical network path, the driver must package every incoming frame into a struct sk_buff. The sk_buff is the central data structure of the Linux network stack, defined in <linux/skbuff.h>.

A fully initialized sk_buff is a complex object:

• The structure header itself occupies over 224 bytes of memory across four distinct 64-byte cache lines.
• It contains dozens of control fields: interface pointers (dev), network namespace (net), protocol identifiers (protocol), transport headers (transport_header), network headers (network_header), MAC headers (mac_header), socket back-pointers (sk), timestamping structures, and packet destination cache pointers (_skb_refdst).
• In addition to the header, memory must be allocated for the packet data buffer, which includes an associated struct skb_shared_info block containing paged fragments and IP checksum state.

+-----------------------------------------------------------------------------------+
|                        STRUCT SK_BUFF MEMORY OVERHEAD                             |
+-----------------------------------------------------------------------------------+
|                                                                                   |
|  +-----------------------------------------------------------------------------+  |
|  | struct sk_buff (224+ bytes header)                                          |  |
|  | - Pointers to dev, net, sk, dst                                             |  |
|  | - Offsets to mac_header, network_header, transport_header                   |  |
|  | - Checksum calculation fields, packet type, priority                        |  |
|  +-----------------------------------------------------------------------------+  |
|         |                                                                         |
|         v points to                                                               |
|  +-----------------------------------------------------------------------------+  |
|  | Data Buffer (Headroom + Packet Payload + struct skb_shared_info)           |  |
|  | [ Headroom ] [ Ethernet Hdr ] [ IP Hdr ] [ TCP Hdr ] [ Payload ]           |  |
|  | [ struct skb_shared_info: page fragments, GSO metadata, refcount ]         |  |
|  +-----------------------------------------------------------------------------+  |
|                                                                                   |
|  TOTAL MEMORY ALLOCATION COST: ~1 KB to 2 KB per packet                           |
|  TOTAL CACHE LINE TOUCHES: 4 to 8 cache lines per packet                          |
|                                                                                   |
+-----------------------------------------------------------------------------------+

When an attacker floods a 10 Gbps link with 14.88 million 64-byte SYN packets per second, allocating and populating 14.88 million sk_buff objects requires allocating gigabytes of dynamic memory per second. The memory subsystem exhausts its slab allocator (kmem_cache), CPU caches thrash continuously on header fields, and the machine stops processing legitimate traffic long before iptables or nftables can evaluate a drop rule.

eBPF Virtual Machine Architecture

eBPF transforms the Linux kernel into an event-driven programmable runtime. It implements a 64-bit Reduced Instruction Set Computer (RISC) virtual machine designed to execute safely inside kernel space at native hardware speeds.

+-----------------------------------------------------------------------------------+
|                        EBPF VIRTUAL MACHINE ARCHITECTURE                          |
+-----------------------------------------------------------------------------------+
|                                                                                   |
|  REGISTERS (64-bit wide)                                                          |
|  +-----+  Function return value / Exit code                                      |
|  | R0  |  (Also stores return value from kernel helper function calls)            |
|  +-----+                                                                          |
|  | R1  |  Argument 1 to helper call / Context pointer on entry (e.g. struct xdp_md)|
|  | R2  |  Argument 2 to helper call                                               |
|  | R3  |  Argument 3 to helper call                                               |
|  | R4  |  Argument 4 to helper call                                               |
|  | R5  |  Argument 5 to helper call                                               |
|  +-----+                                                                          |
|  | R6  |  Callee-saved registers (Preserved across kernel helper function calls)  |
|  | R7  |                                                                          |
|  | R8  |                                                                          |
|  | R9  |                                                                          |
|  +-----+                                                                          |
|  | R10 |  Read-only Frame Pointer (Points to top of 512-byte fixed stack)         |
|  +-----+                                                                          |
|                                                                                   |
|  INSTRUCTION ENCODING (8 Bytes fixed length)                                      |
|  +-----------+-----------+-----------+-----------------------+------------------+ |
|  | opcode: 8 |  dst: 4   |  src: 4   |      offset: 16       |     imm: 32      | |
|  +-----------+-----------+-----------+-----------------------+------------------+ |
|                                                                                   |
+-----------------------------------------------------------------------------------+

The Register Model

The eBPF architecture specifies eleven 64-bit general-purpose registers (R0 through R10):

• R0: Stores the return value of the eBPF program, as well as the return value of in-kernel helper functions. For an XDP program, the value placed in R0 upon exit dictates the packet verdict (e.g. XDP_DROP, XDP_PASS).
• R1 to R5: Function arguments. When an eBPF program begins execution, R1 contains the initial context pointer (for XDP, a pointer to struct xdp_md). When calling kernel helper functions, arguments 1 through 5 are passed in R1 through R5. These registers are caller-saved and their contents are clobbered by helper calls.
• R6 to R9: Callee-saved registers. Their values are preserved across helper function calls. Programs use these registers to store state variables that must survive across table lookups or packet parsing steps.
• R10: Read-only stack frame pointer. It points to the top of the program private 512-byte stack frame. The eBPF program cannot modify R10; it can only read from or write to memory offsets relative to R10 (e.g. *(u32 *)(r10 - 4) = r1).

Instruction Encoding

Every eBPF instruction is encoded as a fixed-size 64-bit (8-byte) structure defined in <uapi/linux/bpf.h>:

struct bpf_insn {
    __u8    opcode;     /* Operation code, instruction class */
    __u8    dst_reg:4;  /* Destination register operand (0-10) */
    __u8    src_reg:4;  /* Source register operand (0-10) */
    __s16   off;        /* Signed memory offset */
    __s32   imm;        /* Signed 32-bit immediate constant */
};

The 8-bit opcode field is divided into three components:

1. Instruction Class (bits 0-2): Specifies the category of operation, such as BPF_LD (load), BPF_ST (store), BPF_ALU (32-bit arithmetic), BPF_ALU64 (64-bit arithmetic), BPF_JMP (jump/branch), or BPF_JMP32 (32-bit branch).
2. Source Modifier (bit 3): Specifies whether the source operand is an immediate value (BPF_K) or another register (BPF_X).
3. Operation Code (bits 4-7): Defines the specific mathematical or logical operation, such as BPF_ADD, BPF_SUB, BPF_AND, BPF_LSH, BPF_JEQ, or BPF_CALL.

For instance, the assembly instruction:

r1 += 14

encodes as:

• opcode: BPF_ALU64 | BPF_K | BPF_ADD = 0x07
• dst_reg: 1 (R1)
• src_reg: 0
• off: 0
• imm: 14 (0x0000000e)

Binary representation: 07 01 00 00 0e 00 00 00.

The In-Kernel Verifier: Formal Safety Verification

The primary design constraint of eBPF is that an unprivileged or privileged user must never be able to crash the host operating system, corrupt kernel memory structures, or trap a CPU core in an infinite execution loop.

Before any eBPF bytecode is permitted to execute, it must pass through the kernel verifier (kernel/bpf/verifier.c). The verifier performs static analysis on the instruction sequence using a Directed Acyclic Graph (DAG) simulation.

+-----------------------------------------------------------------------------------+
|                        EBPF IN-KERNEL VERIFIER WORKFLOW                           |
+-----------------------------------------------------------------------------------+
|                                                                                   |
|  Unverified eBPF Bytecode                                                         |
|         |                                                                         |
|         v                                                                         |
|  +-----------------------------------------------------------------------------+  |
|  | PASS 1: CONTROL FLOW GRAPH (CFG) ANALYSIS                                   |  |
|  | - Detects dead code, unreachable instructions                               |  |
|  | - Validates loop termination (Bounded Loops / BPF Iterators)                |  |
|  | - Enforces maximum program length (1 million verified states)               |  |
|  +-----------------------------------------------------------------------------+  |
|         |                                                                         |
|         v (Valid CFG)                                                             |
|  +-----------------------------------------------------------------------------+  |
|  | PASS 2: ABSTRACT INTERPRETATION & STATE TRACKING                            |  |
|  | - Tracks register types (PTR_TO_PACKET, PTR_TO_STACK, SCALAR_VALUE)          |  |
|  | - Tracks numerical bounds (smin_value, smax_value, umin_value, umax_value)  |  |
|  | - Validates pointer arithmetic and memory dereferences                     |  |
|  | - Enforces packet bounds check: (ptr + offset <= data_end)                   |  |
|  | - Checks helper function argument types and return codes                    |  |
|  +-----------------------------------------------------------------------------+  |
|         |                                                                         |
|         +---> Reject? Return -EACCES with detailed verification log               |
|         |                                                                         |
|         v Passed!                                                                 |
|  JIT Compilation to Native Machine Instructions (x86_64 / ARM64)                  |
|                                                                                   |
+-----------------------------------------------------------------------------------+

Control Flow Graph Construction and Loop Detection

The verifier first constructs a Control Flow Graph of the instruction stream. It inspects all branch instructions (BPF_JEQ, BPF_JGT, etc.) to confirm that:

1. All branch targets jump to valid instruction boundaries within the program body.
2. The program does not contain unconstrained infinite loops.
3. Every path terminates with an exit instruction (BPF_EXIT).

Historically, eBPF banned loops entirely; all repetition had to be unrolled at compile time via #pragma unroll. Linux 5.3 introduced bounded loop support: the verifier simulates loop execution by tracking state convergence. If the verifier can prove mathematically that the loop counter variables decrement toward a termination condition within a bounded number of iterations (bounded by BPF_COMPLEXITY_LIMIT_INSNS, currently 1,000,000 processed states), the program is approved.

Abstract Interpretation and Register State Tracking

The core of the verifier is an abstract interpreter. It walks every possible execution path, maintaining a tracking structure (struct bpf_reg_state) for all eleven registers.

Each register is assigned a conceptual type:

• NOT_INIT: Uninitialized. Reading from this register triggers immediate rejection.
• SCALAR_VALUE: Contains an integer constant, bitmask, or arbitrary numerical value.
• PTR_TO_STACK: Points into the 512-byte eBPF stack frame.
• PTR_TO_PACKET: Points into the raw network packet payload.
• PTR_TO_PACKET_END: Points to the byte immediately following the last valid byte of the packet.
• PTR_TO_MAP_KEY / PTR_TO_MAP_VALUE: Points to keys or values stored inside a BPF map.

For every SCALAR_VALUE, the verifier tracks minimum and maximum possible values across both signed and unsigned 32-bit and 64-bit intervals:

struct bpf_reg_state {
    enum bpf_reg_type type;
    s64 smin_value; /* Minimum possible signed 64-bit value */
    s64 smax_value; /* Maximum possible signed 64-bit value */
    u64 umin_value; /* Minimum possible unsigned 64-bit value */
    u64 umax_value; /* Maximum possible unsigned 64-bit value */
    /* ... bitwise tracking and alignment ... */
};

When the program executes a conditional branch:

r2 = *(u32 *)(r1 + 0); // Read 32-bit integer from packet
if r2 > 100 goto label;

The verifier forks its state simulation into two paths:

1. In the branch taken path, the verifier sets r2->umin_value = 101.
2. In the fall-through path, the verifier sets r2->umax_value = 100.

The Packet Bounds Check Rule

Direct packet inspection in XDP requires reading bytes from memory buffers. Because network packets vary in size, reading beyond the received byte count would trigger an invalid memory dereference, causing an unhandled page fault in kernel space.

To prevent this, the verifier enforces a mandatory packet bounds checking invariant:

// Context structure passed to XDP program (struct xdp_md)
void *data = (void *)(long)ctx->data;
void *data_end = (void *)(long)ctx->data_end;
 
// MANDATORY VERIFICATION GUARD:
// Attempting to read *(u8*)(data + 14) without this check will fail verification!
if (data + 14 > data_end) {
    return XDP_DROP;
}
 
// Memory dereference is now proven safe
struct ethhdr *eth = data;

The verifier internal mechanics are precise:

1. When ctx->data is loaded, the destination register is marked with type PTR_TO_PACKET and off = 0.
2. When ctx->data_end is loaded, its register is marked with type PTR_TO_PACKET_END.
3. When the program adds an offset (e.g. 14 for an Ethernet header) to the packet pointer, the register retains type PTR_TO_PACKET, but its off attribute increases to 14.
4. When the comparison data + 14 > data_end executes, the verifier evaluates the relationship. On the safe path where data + 14 <= data_end, the verifier records that range for this register pointer is at least 14 bytes.
5. Any subsequent load instruction of width $W$ from data at offset $O$ is approved if and only if $O + W \le \text{range}$.

If an engineer attempts to read even a single byte without an explicit comparison against data_end, the verifier rejects the program with an error:

invalid access to packet, off=14 size=1, R1(id=0,off=14,r=0)
R1 offset is outside of the packet

Just-In-Time (JIT) Compilation to Native Assembly

eBPF bytecode is an intermediate representation. While the kernel contains an eBPF interpreter (kernel/bpf/core.c), modern production systems mandate Just-In-Time compilation (/proc/sys/net/core/bpf_jit_enable = 1).

The JIT compiler translates the verified 64-bit eBPF instructions directly into native host machine code (such as x86-64 or ARM64 assembly) during program loading.

Translation Example: x86-64 Machine Code Generation

Consider a basic eBPF packet check instruction:

r1 += 14;              // BPF_ALU64_IMM(BPF_ADD, BPF_REG_1, 14)

On an x86-64 processor, the kernel JIT maps eBPF registers directly to physical CPU registers:

The eBPF instruction r1 += 14 translates directly into a 4-byte native x86-64 instruction:

add $0xe, %rdi

Machine bytes: 48 83 c7 0e.

Because every eBPF register maps one-to-one to an x86-64 architectural register, native execution incurs zero emulation overhead. The generated machine code runs at the exact same speed as native C code compiled directly into the Linux kernel binary.

Once JIT compilation finishes, the memory page containing the native instructions is marked read-only and executable (RX), preventing runtime code tampering.

XDP Execution Modes and the Driver Hook

The eXpress Data Path is an infrastructure layer that provides a standardized hook point for eBPF execution at the lowest possible layer of the network subsystem.

+-----------------------------------------------------------------------------------+
|                        XDP EXECUTION MODES & HOOK POINTS                          |
+-----------------------------------------------------------------------------------+
|                                                                                   |
|  1. OFFLOADED MODE (xdp_offload)                                                  |
|     +-------------------------------------------------------------+               |
|     | SMARTNIC ASIC / FPGA HARDWARE                               |               |
|     | Physical Wire -> [ eBPF JIT Compiled into Firmware / NPU ]  |               |
|     | (Executes on NIC before PCIe bus transfer; ZERO Host CPU)   |               |
|     +-------------------------------------------------------------+               |
|                                                                                   |
|  2. NATIVE / DRIVER MODE (xdp_drv)                                                |
|     +-------------------------------------------------------------+               |
|     | HOST LINUX DEVICE DRIVER (e.g. mlx5, i40e, ixgbe)           |               |
|     | Packet in Host RAM -> [ Native XDP Hook ]                   |               |
|     | (Executes in NAPI loop BEFORE struct sk_buff is allocated)  |               |
|     +-------------------------------------------------------------+               |
|                                                                                   |
|  3. GENERIC MODE (xdp_generic)                                                    |
|     +-------------------------------------------------------------+               |
|     | CORE NETWORK STACK FALLBACK                                 |               |
|     | Allocates sk_buff -> [ Generic XDP Hook ]                   |               |
|     | (Testing fallback for drivers lacking native XDP support)   |               |
|     +-------------------------------------------------------------+               |
|                                                                                   |
+-----------------------------------------------------------------------------------+

XDP functions in three operational modes:

1. Offloaded Mode (XDP_MODE_HW)

The eBPF program is compiled into the instruction set of a SmartNIC (such as Netronome Agilio or AMD Pensando) and loaded into the network processor unit (NPU). Packets are inspected, filtered, or modified directly inside the NIC hardware before traversing the PCIe bus. This consumes zero host CPU cycles.

2. Native / Driver Mode (XDP_MODE_DRV)

The eBPF program executes in the device driver NAPI polling loop on the host CPU. The packet resides in host memory (transferred via DMA), but no struct sk_buff has been allocated. The program inspects raw memory bytes directly from the driver page cache pool. This is the standard production deployment mode, supported by drivers such as mlx5 (Mellanox), i40e / ice (Intel), ixgbe (Intel 10G), and virtio_net (KVM/QEMU).

3. Generic Mode (XDP_MODE_SKB)

A testing fallback mode. The program executes after the driver has already allocated the struct sk_buff, inside the core kernel function netif_receive_skb(). While generic mode provides no performance benefit over standard iptables, it allows developers to test XDP bytecode on arbitrary hardware or virtual interfaces that lack native driver hook support.

The struct xdp_md Context

When native XDP invokes an eBPF program, it passes a pointer to struct xdp_buff (exposed to user space as struct xdp_md):

struct xdp_md {
    __u32 data;             /* User space pointer to packet start */
    __u32 data_end;         /* Pointer to packet end */
    __u32 data_meta;        /* Pointer to packet metadata buffer */
    __u32 ingress_ifindex;  /* Ingress network interface index */
    __u32 rx_queue_index;   /* Hardware RX queue index */
    __u32 egress_ifindex;   /* Target egress interface (for redirect) */
};

• data: Points to the first byte of the raw Ethernet frame (the destination MAC address).
• data_end: Points to the memory address immediately following the payload. The length of the packet in bytes is calculated as: (uintptr_t)data_end - (uintptr_t)data.
• data_meta: Points to a small private memory headroom preceding data. Programs can prepend custom metadata flags (such as parsed protocol hashes or decryption states) that travel with the packet if it is passed upstream to the Linux network stack.

The XDP Action State Machine

The value returned by an XDP program in register R0 dictates what physical action the device driver takes with the packet buffer.

+-----------------------------------------------------------------------------------+
|                        XDP RETURN ACTION STATE MACHINE                            |
+-----------------------------------------------------------------------------------+
|                                                                                   |
|  ACTION           PHYSICAL DRIVER MECHANISM                                       |
|  -------------------------------------------------------------------------------  |
|  XDP_DROP         Immediately recycles the RX descriptor page back to the driver  |
|                   page pool. Zero memory allocation, zero frees. Over 20M pps.    |
|                                                                                   |
|  XDP_TX           Swaps MAC/IP addresses and enqueues the page into the same      |
|                   interface TX descriptor ring for transmission.                  |
|                                                                                   |
|  XDP_REDIRECT     Bypasses local stack: transmits out a different network         |
|                   interface, redirects to a CPU core, or passes to AF_XDP socket. |
|                                                                                   |
|  XDP_PASS         Allocates a struct sk_buff and hands the packet up to the       |
|                   standard Linux networking stack (TCP/IP, Netfilter, sockets).   |
|                                                                                   |
|  XDP_ABORTED      Error trap. Equivalent to XDP_DROP, but records a tracepoint    |
|                   under tracepoint:xdp:xdp_exception for debugging.               |
|                                                                                   |
+-----------------------------------------------------------------------------------+

1. XDP_DROP

The driver discards the packet immediately. Instead of calling memory allocators or unmapping memory pages, the driver resets the descriptor head pointer and reuses the exact same physical memory buffer for the next incoming packet. This recycling mechanism allows a single modern CPU core to discard over 20 to 25 million packets per second.

2. XDP_TX

The driver turns the packet around, queuing the exact same memory buffer into the network card Transmit (TX) ring buffer. This operation enables reflection firewalls, ICMP echo responders, and stateless NAT routers that operate at link line-rate without touching host operating system memory allocators.

3. XDP_REDIRECT

The driver routes the packet to an alternate destination, determined by the bpf_redirect() or bpf_redirect_map() helper functions:

• Cross-interface forwarding: Forwarding a packet from eth0 directly to eth1.
• AF_XDP (XSK) Zero-Copy Sockets: Passing the raw packet memory directly into a user-space memory ring buffer (UMEM), allowing user-space networking engines (such as DPDK or custom packet parsers) to receive packets without copying data through the kernel.
• CPU core distribution (devmap / cpumap): Distributing packet processing across multiple CPU cores before allocating an sk_buff.

4. XDP_PASS

The packet is permitted to continue into the standard Linux networking stack. The driver invokes napi_gro_receive(), allocates the struct sk_buff, sets protocol offsets, and hands the buffer over to the IP layer.

State Storage and Synchronization: BPF Maps

eBPF programs are stateless by default: local stack variables vanish when the program finishes processing a packet. To maintain state (such as rate-limiting counters, connection tracking tables, or IP blocklists), eBPF provides BPF Maps.

A BPF map is a generic, in-kernel key-value data structure allocated in kernel memory and accessible by both kernel eBPF programs and user-space applications via the bpf() system call.

+-----------------------------------------------------------------------------------+
|                        BPF MAP SHARING & ARCHITECTURE                             |
+-----------------------------------------------------------------------------------+
|                                                                                   |
|  USER SPACE APPLICATION                                                           |
|       |                                                                           |
|       |-- bpf(BPF_MAP_UPDATE_ELEM, &key, &value)                                  |
|       |-- bpf(BPF_MAP_LOOKUP_ELEM, &key, &value)                                  |
|       v                                                                           |
|  +-----------------------------------------------------------------------------+  |
|  | IN-KERNEL BPF MAP (Shared Kernel Memory)                                    |  |
|  |                                                                             |  |
|  |  KEY (e.g. __u32 IPv4)              VALUE (e.g. struct stats_counter)       |  |
|  |  +------------------------------+  +-------------------------------------+  |  |
|  |  | 192.168.1.10                 |  | packets: 140293, bytes: 8984920     |  |  |
|  |  | 10.0.0.5                     |  | packets: 12,     bytes: 768         |  |  |
|  |  +------------------------------+  +-------------------------------------+  |  |
|  +-----------------------------------------------------------------------------+  |
|       ^                                                                           |
|       |-- bpf_map_lookup_elem(&map, &key)                                         |
|       |-- bpf_map_update_elem(&map, &key, &value, BPF_ANY)                        |
|  KERNEL SPACE XDP PROGRAM                                                         |
|                                                                                   |
+-----------------------------------------------------------------------------------+

Primary Map Types in High-Throughput Networking

1. BPF_MAP_TYPE_HASH: A standard hash table. Keys and values can be arbitrary binary structures (e.g. 5-tuple connection tracking structures). Lookups use jhash/siphash. Requires locking or RCU synchronization across CPU cores.
2. BPF_MAP_TYPE_PERCPU_HASH: Allocates an independent hash table instance per CPU core. A lookup or update accesses only the local CPU memory bank, avoiding inter-core cache-line bouncing and lock contention.
3. BPF_MAP_TYPE_ARRAY: A continuous memory array indexed by an integer from 0 to max_entries - 1. Guarantees $O(1)$ constant-time lookups with zero hash calculation overhead.
4. BPF_MAP_TYPE_LRU_HASH: A hash table with an internal Least Recently Used eviction policy. When the table fills up, older entries are evicted automatically without requiring user-space garbage collection threads.
5. BPF_MAP_TYPE_RINGBUF: A lock-free, multi-producer single-consumer circular ring buffer used to stream telemetry events from kernel space to user space with minimal latency.

Map Concurrency: Race Conditions and Per-CPU Maps

When multiple CPU cores execute an XDP program simultaneously, updating a shared counter in a standard BPF_MAP_TYPE_HASH generates data races:

// CONCURRENCY HAZARD on standard hash map
struct stats *val = bpf_map_lookup_elem(&stats_map, &key);
if (val) {
    val->packets++; // NON-ATOMIC: Clashing R-M-W cycles across CPU cores!
}

If Core 0 and Core 1 execute val->packets++ at the exact same clock cycle, one increment will be lost due to non-atomic read-modify-write cache line contention.

To resolve this, high-performance networking relies on Per-CPU Array or Per-CPU Hash maps:

struct {
    __uint(type, BPF_MAP_TYPE_PERCPU_ARRAY);
    __type(key, __u32);
    __type(value, struct stats);
    __uint(max_entries, 1);
} drop_stats SEC(".maps");
 
// Lock-free execution: Each CPU writes to its own isolated memory slot
__u32 key = 0;
struct stats *local = bpf_map_lookup_elem(&drop_stats, &key);
if (local) {
    local->packets++; // Completely safe: No other core touches this memory
}

When user space queries the map, it receives an array containing the distinct values for all online CPU cores and sums them together, preserving zero-overhead execution in the fast path.

Concrete XDP Firewall Implementation

The following complete, compilable C program demonstrates an in-kernel XDP packet filter that parses Ethernet and IPv4 headers, blocks traffic originating from an IP blocklist map, tracks drop counts in a per-CPU array, and passes legitimate traffic up to the host stack.

#include <linux/bpf.h>
#include <linux/if_ether.h>
#include <linux/ip.h>
#include <linux/in.h>
#include <bpf/bpf_helpers.h>
#include <bpf/bpf_endian.h>
 
// Structure to track telemetry counters
struct packet_stats {
    __u64 rx_packets;
    __u64 rx_bytes;
    __u64 dropped_packets;
};
 
// 1. Hash map storing blocked IPv4 addresses
struct {
    __uint(type, BPF_MAP_TYPE_HASH);
    __type(key, __u32);   // IPv4 address in network byte order
    __type(value, __u8);  // Flag: 1 = block
    __uint(max_entries, 65536);
} blacklist_map SEC(".maps");
 
// 2. Per-CPU array storing performance statistics
struct {
    __uint(type, BPF_MAP_TYPE_PERCPU_ARRAY);
    __type(key, __u32);
    __type(value, struct packet_stats);
    __uint(max_entries, 1);
} stats_map SEC(".maps");
 
SEC("xdp")
int xdp_firewall_prog(struct xdp_md *ctx) {
    void *data = (void *)(long)ctx->data;
    void *data_end = (void *)(long)ctx->data_end;
    __u64 packet_len = data_end - data;
 
    // Access local per-CPU stats entry
    __u32 stats_key = 0;
    struct packet_stats *stats = bpf_map_lookup_elem(&stats_map, &stats_key);
    if (stats) {
        stats->rx_packets++;
        stats->rx_bytes += packet_len;
    }
 
    // 1. Validate Ethernet Header boundary
    struct ethhdr *eth = data;
    if ((void *)(eth + 1) > data_end) {
        return XDP_DROP; // Truncated Ethernet frame
    }
 
    // Only process IPv4 traffic
    if (eth->h_proto != bpf_htons(ETH_P_IP)) {
        return XDP_PASS;
    }
 
    // 2. Validate IPv4 Header boundary
    struct iphdr *ip = (void *)(eth + 1);
    if ((void *)(ip + 1) > data_end) {
        return XDP_DROP; // Truncated IP header
    }
 
    // Handle IP header with variable options: IHL validation
    __u32 ip_hdr_len = ip->ihl * 4;
    if (ip_hdr_len < sizeof(struct iphdr)) {
        return XDP_DROP; // Invalid IP header length
    }
    if ((void *)ip + ip_hdr_len > data_end) {
        return XDP_DROP; // Options extend beyond packet boundary
    }
 
    // 3. Inspect Source IP against Blacklist Map
    __u32 src_ip = ip->saddr;
    __u8 *blocked = bpf_map_lookup_elem(&blacklist_map, &src_ip);
    if (blocked && *blocked == 1) {
        if (stats) {
            stats->dropped_packets++;
        }
        // Immediate hardware drop: sk_buff is never allocated
        return XDP_DROP;
    }
 
    // Allow legitimate traffic into the standard Linux network stack
    return XDP_PASS;
}
 
char _license[] SEC("license") = "GPL";

Loading and Attaching with bpftool

To compile and attach this program to network interface eth0:

# 1. Compile C source to eBPF ELF binary via Clang/LLVM
clang -O2 -g -target bpf -c xdp_firewall.c -o xdp_firewall.o
 
# 2. Attach program to native driver hook of interface eth0
ip link set dev eth0 xdpgeneric off
ip link set dev eth0 xdp obj xdp_firewall.o sec xdp
 
# 3. Add an offending IP address (198.51.100.44) to the blacklist map
# Convert IP 198.51.100.44 to hex representation in network byte order: 0x2c6433c6
bpftool map update name blacklist_map \
    key hex c6 33 64 2c \
    value hex 01
 
# 4. Dump live per-CPU packet telemetry
bpftool map dump name stats_map

The moment the IP is inserted into blacklist_map, incoming packets from that host are dropped at line rate inside the driver polling loop.

Production Performance and Scalability Limits

XDP bypasses the memory allocation, locking, and protocol traversal bottlenecks that restrict traditional kernel networking.

Summary

The combination of eBPF and XDP represents an architectural shift in Linux operating system networking:

1. Pre-Allocation Interception: XDP operates directly on raw memory buffers in the device driver receive ring before the kernel allocates an sk_buff, eliminating memory slab allocation costs.
2. Deterministic Kernel Safety: The in-kernel verifier mathematically proves that bytecode contains no unbounded loops, invalid pointer dereferences, or uninitialized memory access before execution is permitted.
3. Native Execution Speed: JIT compilation translates RISC bytecode directly into native host assembly, mapping eBPF registers one-to-one with physical CPU hardware registers.
4. Action Flexibility: The five XDP return codes (XDP_DROP, XDP_TX, XDP_REDIRECT, XDP_PASS, XDP_ABORTED) enable building line-rate packet filters, stateful load balancers, and zero-copy user-space datapath pipelines.
5. Shared State via BPF Maps: High-performance data structures (Per-CPU Hashes and Arrays) permit concurrent state updates across multiple CPU cores without lock contention or cache bouncing.

By shifting packet processing to the earliest boundary in the driver receive path, eBPF and XDP provide the performance foundation for modern cloud infrastructure, edge routing, and cloud-native network virtualization.