Skip to content

libc: implement splice(2), vmsplice(2), tee(2) - #1425

Open
gburd wants to merge 1 commit into
cloudius-systems:masterfrom
gburd:pr/splice
Open

libc: implement splice(2), vmsplice(2), tee(2)#1425
gburd wants to merge 1 commit into
cloudius-systems:masterfrom
gburd:pr/splice

Conversation

@gburd

@gburd gburd commented Jul 12, 2026

Copy link
Copy Markdown
Contributor

What

splice(2), vmsplice(2), and tee(2) were missing entirely, so programs that
move data between an fd and a pipe via splice (a common high-throughput idiom)
could not run.

How

OSv has no user/kernel boundary and copies are cheap, so this is a correct
(not zero-copy) implementation built on the existing read/write and pread/pwrite
paths:

  • splice(): move up to len bytes from fd_in to fd_out through a
    bounce buffer, honoring the optional off_in/off_out offset pointers
    (pread/pwrite when set, read/write otherwise), looping over short writes. A
    pipe-page-sharing fast path can be added later if it ever shows up hot;
    correctness does not need it.
  • vmsplice(): implement the common memory -> fd direction as a write of the
    iovec segments into the fd.
  • tee(): duplicating pipe data without consuming it needs a non-consuming
    pipe peek, which OSv's pipe does not support, so it returns ENOSYS rather
    than silently consuming the input.

Unknown flags return EINVAL. Wired as SYS_splice/SYS_vmsplice/SYS_tee
with tracepoints, symbols exported from libc.so.6 and ld-musl.so.1 (the
declarations and SPLICE_F_* flags are already in the musl headers).

Testing

tests/tst-splice.cc: file -> pipe -> file with byte-integrity verification and
offset-pointer handling, vmsplice memory -> pipe round-trip, tee ENOSYS, and
the EINVAL path. Passes on OSv under KVM.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds missing Linux splice-family syscalls to OSv libc (copy-based implementation) to unblock applications that use pipe-based high-throughput fd transfers, and wires them into syscall dispatch, tracing, symbol exports, and the test suite.

Changes:

  • Implement splice(2), vmsplice(2), and tee(2) in libc/ (with tee() returning ENOSYS).
  • Register the new syscalls and tracepoints, and export the symbols from OSv’s libc and musl loader.
  • Add tst-splice coverage for splice/vmsplice behavior and expected error paths.

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
tests/tst-splice.cc New test covering splice/vmsplice success paths and error returns.
syscalls/syscalls.cc.in Adds syscall table entries for splice, vmsplice, tee.
syscalls/syscall_tracepoints.cc.in Adds tracepoints for the new syscalls.
modules/tests/Makefile Builds and includes tst-splice.so in the tests module.
Makefile Adds splice.o into libc build.
libc/splice.cc Implements splice, vmsplice, tee.
exported_symbols/osv_libc.so.6.symbols Exports splice, tee, vmsplice from libc.
exported_symbols/osv_ld-musl.so.1.symbols Exports splice, tee, vmsplice from the musl loader.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread libc/splice.cc
Comment thread libc/splice.cc
Comment thread libc/splice.cc Outdated
Comment thread tests/tst-splice.cc
Comment thread tests/tst-splice.cc
Comment thread libc/splice.cc Outdated
Comment thread libc/splice.cc
Comment thread libc/splice.cc
Comment thread libc/splice.cc Outdated
@gburd

gburd commented Jul 15, 2026

Copy link
Copy Markdown
Contributor Author

Rebased onto current master and squashed into one coherent commit (the branch had drifted and its old base spuriously touched linux.cc/signal.cc/pthread.cc; that's gone - it now touches only the splice files). Addressed the review:

  • "ponytail" word: reworded to a proper forward-looking TODO explaining the zero-copy fast path we'd add if splice ever gets hot.
  • fd types / non-null offset (your comment): splice now rejects a non-null off_in/off_out on a non-seekable (pipe/socket) fd with ESPIPE, matching Linux (verified OSv's lseek returns ESPIPE for such fds). I did NOT implement the full "exactly one end must be a pipe" rule because OSv's anonymous pipe isn't reliably distinguishable via fstat here - documented that limitation honestly in the source rather than half-checking it.
  • short input / "what if fewer than len bytes": the loop ends on a short read / EOF and returns what was moved.
  • infinite loop (Copilot): a legal zero-byte (no-progress) write now stops the loop and returns bytes-moved instead of spinning.
  • overflow (Copilot): splice rejects len > SSIZE_MAX (EINVAL); vmsplice bounds nr_segs at UIO_MAXIOV (like readv/writev) and guards the running total against ssize_t overflow.
  • vmsplice wrong direction (your comment): it implements memory -> fd; the unsupported pipe -> memory direction is rejected naturally because write() returns EBADF on a read-only fd. The comment now names that enforcement rather than just saying "not supported".
  • tee: ENOSYS (honest - OSv's pipe can't peek non-consuming).
  • test flakiness (Copilot): file write / pipe read now loop until complete; added asserts for the new ESPIPE-on-pipe-offset and SSIZE_MAX-overflow guards. Builds/runs on Linux too.

The change was independently reviewed by two reviewers (systems + API/semantics); the vmsplice nr_segs bound and the two new guard tests came out of that review.

@gburd

gburd commented Jul 15, 2026

Copy link
Copy Markdown
Contributor Author

The CHANGES_REQUESTED here predates my fixes - all of these are already addressed in the current head (rebased onto master):

  • "ponytail" -> TODO (your comment): done, worded as a forward-looking TODO.
  • fd types / non-null offset (your comment): splice rejects a non-null off_in/off_out on a non-seekable (pipe/socket) fd with ESPIPE, matching Linux. The full "exactly one end must be a pipe" rule is documented as a known limitation (OSv's anon pipe isn't reliably distinguishable via fstat).
  • short input: loop ends on short read / EOF.
  • vmsplice wrong direction (your comment): implements memory->fd; the unsupported pipe->memory direction is rejected by write() returning EBADF on a read-only fd - the comment now names that enforcement.
  • infinite loop on w==0 (Copilot): a zero-byte no-progress write stops the loop.
  • overflow (Copilot): splice rejects len > SSIZE_MAX; vmsplice bounds nr_segs at UIO_MAXIOV and guards the total against ssize_t overflow.
  • test flakiness (Copilot): file write / pipe read loop until complete; added ESPIPE-on-pipe-offset and SSIZE_MAX-overflow assertions.

Single squashed commit; builds/runs on Linux too.

@gburd

gburd commented Jul 21, 2026

Copy link
Copy Markdown
Contributor Author

Rebased onto current master. The review items from the last round are addressed: the lingering 'ponytail' word is removed, non-seekable fds with an offset return ESPIPE, len>SSIZE_MAX returns EINVAL, a w==0 no-progress guard avoids a spin, vmsplice bounds nr_segs>UIO_MAXIOV and rejects the wrong direction (EBADF), and tee() returns ENOSYS. Squashed to a single commit. Ready for another look.

@gburd

gburd commented Jul 24, 2026

Copy link
Copy Markdown
Contributor Author

Thanks for the careful read. Point-by-point:

  1. "ponytail" comment (line 18) -- removed; the current tip words it as a TODO: (the bounce-buffer vs zero-copy note) and a Limitation: paragraph explaining today's behavior, exactly as you asked.

  2. fd types and non-null offset (line 76) -- fd_in/fd_out can be any fd the underlying read/write path accepts (filesystem files, pipes, sockets). Linux forbids a non-null offset on the pipe end (and any non-seekable fd). I enforce the observable part of that rule: a non-null off_in/off_out on a non-seekable fd is rejected with ESPIPE (probed via lseek(...,SEEK_CUR)). What I deliberately do not enforce is Linux's "exactly one end must be a pipe" rule, because OSv's anonymous pipe isn't reliably distinguishable via fstat here -- I'll expand the Limitation: comment to say this explicitly rather than leave it implicit.

  3. Input shorter than len (line 100) -- handled: read_at returning 0 (EOF) breaks the loop, and a short read (r < want) also breaks after writing what was read; the call returns the byte count actually moved (never blocks forever waiting for more). That's the Linux behavior for a source with fewer bytes than requested.

  4. vmsplice SPLICE_F_GIFT / direction (line 105) -- fair. I only implement the memory->fd direction (a plain write of the iovec). Rather than rely on the implicit "EBADF from write() on a read-only fd", I'll make the unsupported direction and the SPLICE_F_GIFT semantics explicit in the comment (GIFT is a no-op hint for us since there are no shared pipe pages to gift), and keep rejecting truly invalid flag combinations with EINVAL as now.

I'll push the comment tightening for (2) and (4); (1) and (3) are already in the current tip. None of these change behavior, only make the documented contract match what the code does.

Linux implements these as zero-copy transfers through a pipe's page buffers.
OSv has no user/kernel boundary and the copies are cheap, so this is a correct
(not zero-copy) implementation over the existing read/write and pread/pwrite
paths, moving bytes through a bounce buffer.  This unblocks the many programs
that use splice()/sendfile-style loops without caring how the bytes move.

- splice(): loops read_at -> write_at in <=64 KiB chunks; a non-null offset
  uses pread/pwrite and advances *off (leaving the fd position untouched),
  a null offset uses the fd position.  Short reads / EOF end the transfer;
  a legal zero-byte (no-progress) write stops the loop rather than spinning.
  Rejects len > SSIZE_MAX (EINVAL) and a non-null offset on a non-seekable
  (pipe/socket) fd (ESPIPE), matching Linux.  It does not enforce Linux's
  'exactly one end must be a pipe' rule because OSv's anonymous pipe is not
  reliably distinguishable via fstat here; documented in the source.
- vmsplice(): implements the memory -> pipe/fd direction (a write of the
  iovec into the fd); the pipe -> memory direction is rejected naturally by
  write() returning EBADF on a read-only fd.  Bounds nr_segs at UIO_MAXIOV and
  guards the running byte total against ssize_t overflow.
- tee(): returns ENOSYS - a true non-consuming tee needs to peek pipe data,
  which OSv's pipe does not support; better an honest ENOSYS than silently
  consuming the input.

Wired into the syscall table, exported from libc and the musl loader, and
covered by tst-splice (file->pipe->file round-trip, offset semantics, the
EINVAL/ESPIPE/overflow guards, vmsplice, and tee's ENOSYS); it builds and runs
on Linux too.

TODO: a pipe-page-sharing zero-copy fast path could be added if splice ever
shows up as a hot path; correctness does not need it.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants