Skip to content
FreeBSDDeep Dive Published Updated 7 min readViews unavailable

FreeBSD KTLS: Verify TLS Session Offload and Its Limits

Check FreeBSD KTLS patch status, socket-mode support, library handoff, and per-mode counters before treating TLS offload as a performance win.

FreeBSD’s kernel TLS facility, KTLS, moves TLS record processing for a TCP socket into the kernel. It does not move the TLS handshake: the application negotiates the session in userland, then passes direction-specific session material to the kernel. That division can reduce copies or use NIC support on a busy data path, but it also means that a kernel sysctl, an advertised interface capability, or a faster sendfile(2) benchmark alone does not prove that a particular connection is using hardware offload.

There is a security gate before performance testing. FreeBSD’s June 2026 advisory, FreeBSD-SA-26:26.ktls, describes an arbitrary-file-overwrite vulnerability in software KTLS receive processing. The documented attack requires a local, unprivileged user who can read a file and can send it over a loopback connection using KTLS receive; it is not a claim of unauthenticated remote code execution. The advisory says all supported versions were affected before correction. Do not enable or test KTLS receive on an unpatched system. First identify the installed kernel and userland, apply the appropriate supported-branch correction, and reboot as the advisory instructs. The advisory lists corrected release points including 15.1-RC3-p1, 15.0-RELEASE-p10, 14.4-RELEASE-p6, and 14.3-RELEASE-p15, as well as stable branches. Match the correction to the branch you actually run; a newer-looking userland package does not prove that the running kernel has been updated.

Understand the boundary KTLS moves

After the userland TLS library has negotiated a session, an application can request transmit and receive KTLS independently with TCP_TXTLS_ENABLE and TCP_RXTLS_ENABLE. Each direction receives the cipher and session keys in a struct tls_enable. The 15.1 ktls(4) manual notes that session keys may be supplied only once per direction, so applications using KTLS must disable TLS rekeying. A socket can use KTLS for one direction and not the other, and its transmit and receive modes can differ.

FreeBSD documents three modes:

  • TCP_TLS_MODE_SW performs TLS record encryption or decryption in the kernel socket layer using crypto(9). It is kernel processing, but not necessarily NIC offload.
  • TCP_TLS_MODE_IFNET asks a supported network interface to encrypt transmitted records or decrypt received records. The manual identifies cxgbe(4) and mlx5en(4) as drivers with IFNET support.
  • TCP_TLS_MODE_TOE combines TLS with a TCP offload engine. The documented cxgbe(4) driver supports this mode.

Transmit and receive do not have identical controls. The TCP_TXTLS_MODE socket option reports the transmit mode and can switch an active transmit session between software and IFNET modes. The receive mode is queried with TCP_RXTLS_MODE; the manual says it cannot currently be changed. Receive also has a record-oriented interface: applications must read records with recvmsg(2) and inspect the accompanying TLS_GET_RECORD control message. Errors such as EINVAL, EMSGSIZE, and EBADMSG indicate different record-version, length/truncation, and authentication-tag problems, not generic throughput failures.

Gate the test on the running kernel

After patching and rebooting, collect read-only baseline evidence before changing a setting or starting a test. The exact sysctl tree depends on the kernel, so treat missing nodes as version/build evidence rather than inventing a value. The controls described here follow the cited FreeBSD 15.1 manual; newer development branches may document additional per-direction controls, so check the manual matching the running kernel:

freebsd-version -kru
sysctl kern.ipc.tls.enable
sysctl -a | grep '^kern.ipc.tls'
ifconfig -v

The kern.ipc.tls.enable node controls whether new KTLS sessions may be created; it is not a per-socket report and does not identify the mode of an established connection. The sw, ifnet, toe, and general stats nodes expose counters. Record their values before and after one controlled test, using the same sysctl names supported by the installed release. Aggregate counters can be affected by other applications, so isolate the workload where possible and use counter deltas rather than attributing a host-wide total to one connection.

ifconfig -v is a hardware-capability check, not a session check. A driver may advertise TXTLS4, TXTLS6, or TXTLS_RTLMT for IPv4, IPv6, or TLS packet pacing. Those flags say something about interface capability; they do not prove that a library selected IFNET mode, that the tested flow used that interface, or even that KTLS was enabled for the relevant direction. The ktls(4) manual also notes that TLS transmit requires unmapped mbufs, controlled by kern.ipc.mb_use_ext_pgs; inspect the release’s documented prerequisite rather than changing it blindly.

The manual lists kern.ipc.tls.sw, .ifnet, .toe, and .stats as counter trees. It does not define one universal counter key to use as a pass/fail test across every driver and release. If a node is absent or the values do not move, preserve the exact output and check the local ktls(4), driver, and sysctl documentation for that kernel before drawing a conclusion.

Verify the application-to-socket handoff

KTLS support in the kernel is only one layer. FreeBSD documents OpenSSL 3.0 and later as KTLS-capable; base-system OpenSSL needs to have been built with WITH_OPENSSL_KTLS, while ports may need their KTLS option enabled. Applications using supported library interfaces such as SSL_read() and SSL_write() can generally use KTLS without replacing their ordinary I/O calls, but that does not mean every TLS connection automatically offloads. The library, negotiated cipher/session, application configuration, kernel, driver, and direction must all line up.

For the actual socket, ask the application or its diagnostic interface for the results of getsockopt() with TCP_TXTLS_MODE and TCP_RXTLS_MODE. A returned software mode is evidence of kernel KTLS, not interface offload; an IFNET mode is stronger per-socket evidence of NIC record processing. If you own the application, log the mode after the TLS session is established and tie the report to a connection identifier without logging the session keys. Do not use ifconfig flags as a substitute for querying the socket.

The kernel manual says the TLS handshake stays in userland. Keep certificate validation, peer identity, protocol policy, and the library’s ordinary error handling in the application’s TLS layer. KTLS is not a replacement for TLS configuration or a guarantee that all cryptographic work has disappeared from the host. It changes where record protection occurs, not the trust model of the connection.

Benchmark one direction and one claim at a time

Start with the patched system and a repeatable, non-production workload. Record the FreeBSD branch and kernel build, NIC and driver, interface capabilities, TLS library/build options, cipher, TLS version, workload size, concurrency, CPU utilization, and direction. Compare the same application and traffic with KTLS unavailable and with the intended KTLS mode, and repeat enough times to account for warm-up and background load. Test file-send optimization separately from ordinary socket writes rather than assuming two different application paths exercise the same kernel behavior.

Use three pieces of evidence together: the application’s per-socket transmit/receive mode, the relevant before/after KTLS counter deltas, and workload measurements such as CPU time and throughput. If only the software counter moves, you have not demonstrated NIC offload. If throughput rises but CPU time does not change, explain which resource improved; if counters rise with no throughput gain, the test may be limited elsewhere. Keep the conclusion scoped to the tested cipher, driver, direction, and workload instead of generalizing it into “KTLS is faster.”

Do not toggle KTLS globally during an uncontrolled production test. kern.ipc.tls.enable affects whether new sessions can be created, and driver capability changes can affect traffic beyond one benchmark. If a security advisory requires mitigation, follow the advisory’s update-and-reboot path. It also documents disabling KTLS as a workaround, but that is not a substitute for installing the corrected kernel and rebooting.

The practical checklist is simple: verify the running kernel is corrected; confirm the kernel and TLS library support the intended path; inspect driver capabilities without treating them as proof; query each tested socket’s transmit and receive modes; compare counter deltas; and benchmark a controlled workload. That distinguishes ordinary userland TLS, kernel software KTLS, and actual interface offload without confusing them with one another.

Related:

Sources:

Comments