Linux TUN/TAP: Operate User-Space Network Interfaces Safely

Linux TUN/TAP devices connect the kernel network stack to a userspace program. They are the boundary used by many VPN clients, virtual networking tools, and packet-processing appliances: the kernel routes a packet to a virtual interface, the program reads it from a file descriptor, and the program can inject a received packet back through that descriptor. A device can look correctly configured while no userspace process is actually servicing its queue, so diagnosing only ip address output is not enough.

The distinction is fundamental. A TUN device carries network-layer IP packets without Ethernet framing. A TAP device carries Ethernet frames, including the layer-2 header, and can participate in bridging designs. The userspace program must parse and produce the exact frame format negotiated for the device; a mismatch can create silent drops, malformed packets, or apparent one-way traffic.

Understand the packet boundary

An application creates or attaches a device by opening /dev/net/tun and issuing TUNSETIFF. The ioctl selects IFF_TUN or IFF_TAP, may request IFF_NO_PI to omit the optional packet-information header, and can request other negotiated features. The resulting interface is visible to the kernel as a normal network device. Packets the kernel transmits through it are delivered to userspace; writes to the associated file descriptor inject packets into the kernel receive path.

With IFF_NO_PI clear, each userspace frame begins with a 4-byte tun_pi header containing flags and a protocol value before the network or Ethernet frame. With IFF_NO_PI set, that prefix is absent. Optional virtual-network headers and offload features add further framing and semantics. Read the interface flags negotiated by the actual program before decoding bytes as an IP header; hard-coding offsets without tracking those flags is a common source of packet-parser bugs.

The control plane and data plane have different owners. ip address, ip route, and ip link configure the kernel-facing interface. The VPN or appliance owns the file descriptor and implements the remote transport, encryption, bridge, or packet policy. A correct local route only proves the kernel can select the TUN/TAP device; it does not prove that the userspace service can read, forward, authenticate, or return the packet.

Create a narrowly scoped test interface

For an isolated lab, create a named TUN device owned by a dedicated unprivileged test account. Use an address and route reserved for the lab, and do not add a default route while exploring:

TEST_USER=tunlab
sudo ip tuntap add dev tun-lab mode tun user "$TEST_USER"
sudo ip link set dev tun-lab up
sudo ip address add 192.0.2.10/24 dev tun-lab
sudo ip route add 198.51.100.0/24 dev tun-lab

ip -details link show dev tun-lab
ip address show dev tun-lab
ip route get 198.51.100.10
ip tuntap show

The documentation prefixes are examples only. The route is deliberately limited to a documentation network so it does not capture ordinary production traffic. In a real VPN, use the route plan supplied by the VPN owner and confirm policy rules, DNS behavior, and any split-tunnel design before activation. The ip tuntap add form creates a persistent interface; it does not start a VPN or provide a packet-processing process. The userspace owner must still open and service the queue.

Use the narrowest permissions supported by the deployment model. Do not make /dev/net/tun world-writable as a shortcut. The kernel driver checks device ownership and network-administration privileges for operations that create or attach to interfaces; grant only the required device access and capabilities. A system service should run under a dedicated identity, keep its file descriptors private, and expose a deliberate control plane for routes rather than granting broad network administration to unrelated code.

Remove only the lab device after its owner process has stopped and the route is no longer needed:

sudo ip route del 198.51.100.0/24 dev tun-lab
sudo ip tuntap del dev tun-lab mode tun
ip tuntap show

Before deleting any real interface, check whether a service manager or VPN agent owns it and whether active traffic depends on it. Device deletion and route changes are stateful operations, not read-only diagnostics.

Keep and manage the file descriptor correctly

In the ioctl API, the process opens /dev/net/tun, fills a zero-initialized struct ifreq, sets its requested interface name and flags, then calls ioctl(fd, TUNSETIFF, &ifr). A successful call returns the actual interface name in ifr.ifr_name; retain the returned descriptor in the packet-processing loop. Handle every failure path and preserve errno while closing the descriptor.

#include <errno.h>
#include <fcntl.h>
#include <linux/if.h>
#include <linux/if_tun.h>
#include <stdio.h>
#include <string.h>
#include <sys/ioctl.h>
#include <unistd.h>

int open_tun(const char *requested_name)
{
    struct ifreq ifr = {0};
    int fd = open("/dev/net/tun", O_RDWR | O_CLOEXEC);
    if (fd == -1)
        return -1;

    if (requested_name != NULL && requested_name[0] != '\0') {
        size_t length = strlen(requested_name);
        if (length >= sizeof(ifr.ifr_name)) {
            close(fd);
            errno = ENAMETOOLONG;
            return -1;
        }
        memcpy(ifr.ifr_name, requested_name, length + 1);
    }
    ifr.ifr_flags = IFF_TUN | IFF_NO_PI;

    if (ioctl(fd, TUNSETIFF, &ifr) == -1) {
        int saved_errno = errno;
        close(fd);
        errno = saved_errno;
        return -1;
    }

    fprintf(stderr, "attached TUN device %s\n", ifr.ifr_name);
    return fd; /* Keep this descriptor open while servicing packets. */
}

This is an allocation helper, not a complete VPN. The caller must retain the descriptor, set up the interface through an authorized control path, and run a bounded, backpressure-aware packet loop. On a non-persistent device, closing the last owning descriptor removes the device and its routes. Persistent devices outlive the creating command, but still need an attached userspace queue to move packets. That difference explains why ip tuntap show can list an interface even when its data plane is dead.

Production loops should handle EINTR, short reads/writes, descriptor shutdown, queue saturation, and process cancellation. Use nonblocking I/O only with a correct epoll or equivalent readiness loop; spinning on EAGAIN wastes CPU and can starve other queues. Bound buffer sizes, account for the negotiated packet framing, and treat userspace parse errors as observable drops rather than silently forwarding unvalidated bytes.

Use multiqueue only with an explicit concurrency model

Linux multiqueue TUN/TAP allows multiple file descriptors to attach to one named interface with IFF_MULTI_QUEUE, providing separate queues that userspace can service concurrently. It can help parallel packet processing, but it also changes the application’s ordering, scheduling, and shutdown problem. A queue count is not automatically equivalent to the number of CPU cores or NIC receive queues.

Choose queue count from measured workload and affinity constraints. Test flow distribution, per-queue backlogs, packet ordering where the protocol requires it, CPU saturation, descriptor lifecycle, and behavior when one worker exits. All queues share device-level properties such as the selected TUN/TAP mode; creating later queues with inconsistent flags is not a way to convert an existing interface.

Some kernels support TUN/TAP qdisc backpressure flags that affect whether packets are dropped in the internal ring before the attached qdisc can schedule them. The kernel documentation describes IFF_BACKPRESSURE as a device property and notes performance tradeoffs. Do not enable it based on the flag name alone: verify support in the target kernel, understand whether a qdisc is attached, and load-test both single- and multi-queue behavior with representative traffic.

Diagnose a device that is up but carries no useful traffic

Start with read-only observations and identify the exact interface and namespace:

ip -details link show dev tun-lab
ip address show dev tun-lab
ip route show table all
ip rule show
sudo ss -xlpn

ss -xlpn is useful for Unix-domain control sockets used by VPN software; it does not prove who owns the TUN file descriptor. For the service process, inspect its open descriptors and logs using the platform’s approved observability tools. Avoid dumping unrelated process command lines or secrets into incident tickets. Confirm the process is in the expected network namespace, has opened the intended device, and is reading from every expected queue.

Trace one packet across the boundary. First ask the kernel which route it selects for the destination and source context; then capture on the TUN/TAP interface and correlate timestamps with the userspace service’s read, decrypt/encapsulate, remote send, receive, and reinjection events. For TUN, inspect IP headers; for TAP, include Ethernet headers and any VLAN tags. A packet observed leaving the local TUN device but not reappearing on the expected return path points beyond route selection: check the userspace queue, transport socket, remote peer, and policy in sequence.

Common symptom patterns include:

Interface exists but has no carrier: this can be expected for a point-to-point TUN design. Check the actual route and userspace descriptor rather than treating carrier state alone as proof of failure.
Route selects the device, but the application never reads packets: confirm its namespace, queue attachment, event loop, file-descriptor ownership, and service lifecycle.
Userspace reads packets but no remote response returns: inspect encapsulation, outer transport routing, peer health, and remote policy; the local TUN interface does not provide the tunnel transport itself.
One direction works: compare route selection and policy rules in both directions, then check reverse-path filtering, firewall state, and whether the userspace program reinjects the correct frame format.
Packets fail after a process restart: verify whether the interface is persistent, whether the previous descriptor was released, and whether the new process attached with matching mode, flags, and queue count.

Do not repair routing by replacing the host’s default route unless that is the reviewed design. VPN clients that implement a kill switch or split tunnel often install policy rules and firewall state in addition to interface routes; changing one visible route can bypass those controls or disconnect management access.

Production acceptance tests

Confirm TUN versus TAP, negotiated packet-information and virtual-network-header settings, and whether the device is persistent.
Document the process, network namespace, descriptor ownership, user/group permissions, capabilities, queue count, and restart behavior.
Verify the intended prefix route and source selection without redirecting unrelated host traffic.
Prove a packet crosses kernel-to-userspace and userspace-to-kernel in both directions, with captures and application counters.
Exercise queue saturation, worker exit, reconnect, route withdrawal, process restart, and orderly shutdown.
Validate throughput and latency on the target kernel and host; do not infer production performance from a functional ping.
Remove only the test routes and interface during cleanup, and verify the host’s ordinary routes are unchanged.

TUN/TAP is a narrow interface with wide consequences: it joins kernel routing to code that may encrypt, bridge, forward, or discard packets. Production readiness depends on treating device configuration, descriptor ownership, packet framing, userspace health, and routing as one observable lifecycle.

Related:

Sources: