Skip to content

experiment(worker): evaluate ring-native bulk I/O primitives against sendfile #576

Description

@somfornot

Problem / motivation

Talon uses io_uring/Monoio for accept, framing, and scheduling, but cached bulk GET uses libc sendfile in a per-ring blocking pool (uring_conn.rs, sendfile.rs). This is a stable baseline, but every bulk response still crosses a scheduler boundary and a finite helper queue.

DESIGN.md lists fd registration, double-buffered splice, IORING_OP_SPLICE, and send-zero-copy as benchmark-driven future work (DESIGN.md). We need controlled evidence before adopting more complex kernel paths.

Proposed experiments

Build capability-probed alternatives without changing the default:

  • ring-native file-to-pipe-to-socket IORING_OP_SPLICE versus libc sendfile;
  • multishot accept/recv and provided-buffer rings for framed requests;
  • registered files/buffers after the relevant operation runs on the ring;
  • SEND_ZC for unavoidable in-memory responses;
  • optional SQPOLL where privileges and a reserved polling CPU are acceptable.

Test partial progress, EAGAIN, cancellation, slow/disconnected clients, short files, eviction/unlink, kernel fallback, and shutdown. Retain the current sendfile implementation as compatibility fallback.

Acceptance criteria

  • Every experiment has a runtime/kernel probe and transparent protocol-compatible fallback.
  • Identical payloads and admission limits are compared on representative kernels, CPUs, NICs, and NVMe.
  • Results report throughput, p50/p99/p999/max, user/system CPU, context switches, ring submissions/completions, helper queueing, and SQPOLL CPU/power cost.
  • Slow-reader/cancellation tests prove buffers, pipes, registered resources, cache pins, and FDs cannot leak.
  • Nothing becomes default without improving a named workload without unacceptable tail, CPU, portability, or maintenance cost.
  • Negative results are documented.

Related: #9, #10, #11, #273, #285, #291, #570, #572, #574, #575

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions