fs: implement preadv2/pwritev2 and renameat2 RENAME_NOREPLACE - #1429
fs: implement preadv2/pwritev2 and renameat2 RENAME_NOREPLACE#1429gburd wants to merge 1 commit into
Conversation
There was a problem hiding this comment.
Pull request overview
Adds support for three newer Linux filesystem syscalls in OSv’s libc/VFS layer (preadv2, pwritev2, renameat2), including syscall wiring, tracepoints, symbol exports, and a dedicated test to validate expected flag/error behavior.
Changes:
- Implement
preadv2()/pwritev2()wrappers over existingpreadv()/pwritev()with support for selectedRWF_*flags. - Implement
renameat2()withRENAME_NOREPLACEand explicit rejection of unsupportedRENAME_*flags. - Wire up syscall numbers/tracepoints/symbol exports and add a new test module (
tst-fs-syscalls).
Reviewed changes
Copilot reviewed 9 out of 9 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/tst-fs-syscalls.cc | New test covering preadv2/pwritev2 and renameat2(RENAME_NOREPLACE) behavior. |
| fs/vfs/main.cc | Implements preadv2/pwritev2 and renameat2 behavior in the VFS/libc layer. |
| syscalls/syscalls.cc.in | Adds syscall stubs for renameat2, preadv2, pwritev2. |
| syscalls/syscall_tracepoints.cc.in | Adds tracepoints for the new syscalls. |
| include/api/x64/bits/syscall.h | Adds x86-64 syscall numbers for renameat2, preadv2, pwritev2. |
| linux.cc | Declares the new syscall entry points for the Linux syscall layer. |
| exported_symbols/osv_libc.so.6.symbols | Exports preadv2, pwritev2, renameat2 from osv_libc. |
| exported_symbols/osv_ld-musl.so.1.symbols | Exports preadv2, pwritev2, renameat2 from the musl loader binary. |
| modules/tests/Makefile | Adds tst-fs-syscalls.so to the tests image build. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
nyh
left a comment
There was a problem hiding this comment.
Looks mostly good but I (and copilot) had some concerns and suggestions.
Review feedback on cloudius-systems#1429: - preadv2: reject write-only RWF flags (DSYNC/SYNC/APPEND) on reads, and reject RWF_NOWAIT with EOPNOTSUPP rather than faking EAGAIN. Returning EAGAIN for a request we can never satisfy non-blocking would make a retrying caller spin forever (nyh); rejecting is the honest answer. Reads now accept only RWF_HIPRI. - pwritev2: propagate fsync()/fdatasync() failure. A caller that asked for RWF_SYNC/RWF_DSYNC must not be told the write is durable when the flush failed. Also propagate fstat() failure on RWF_APPEND. - renameat2: collapse flag validation to a single '& ~RENAME_NOREPLACE' check; strengthen the FIXME documenting that RENAME_NOREPLACE and RWF_APPEND are best-effort NON-ATOMIC (TOCTOU) implementations that need real VFS-level atomic support before they can be relied on for concurrency. - Move the RWF_*/RENAME_* defines and the three prototypes into a shared header include/api/osv/fs_flags.h used by both fs/vfs/main.cc and the test, so they cannot drift out of sync (nyh).
|
Addressed the review (pushed):
|
nyh
left a comment
There was a problem hiding this comment.
In general I approve, but there a few small things you might want to look again (like the copyright line, and a few comments from copilot), and please squash the PR into one patch with one coherent commit message without outdated information. Thanks.
|
Thanks for the approve. Squashed into one coherent commit (966580c) with an up-to-date message, and addressed the remaining points:
Rebuilt and the test passes. |
nyh
left a comment
There was a problem hiding this comment.
I think the changes in one file are a mistake, perhaps some sort of merge accident?
Or maybe I'm missing something?
|
Correction: my previous push of this branch had accidentally carried a spurious revert of the (since-merged) tgkill/rt_sigtimedwait (#1428) and pthread-timedlock (#1446) changes, because the branch base predated those merges. I've rebased onto current master and confirmed the diff now touches only the preadv2/pwritev2/renameat2 files (fs/vfs/main.cc, the shared fs_flags.h header, linux.cc syscall entries, exports, test) with no changes to linux.cc's rt_sigtimedwait, libc/signal.cc, or libc/pthread.cc. Rebuilt, tst-fs-syscalls passes. |
b397971 to
4053f0b
Compare
|
Rebased onto current master. The earlier concerns are addressed: the one-file change that looked like a merge accident was exactly that (an accidental revert of already-merged work) and has been corrected; RWF_APPEND returns EOPNOTSUPP rather than a racy fstat+pwritev; copyright/style nits fixed. Squashed to a single commit, MERGEABLE + CLEAN. Ready for re-review. |
|
Thanks -- good catches, and I think the current tip already resolves them (a couple were fixed in a push after your review; the rest below):
On the api/osv vs api/uio.h placement: I kept a dedicated |
preadv2(2)/pwritev2(2): the RWF_* variants of preadv/pwritev. - Shared RWF_*/RENAME_* definitions live in a new header include/api/osv/fs_flags.h so the implementation and tests agree. - Reads reject any flag outside RWF_HIPRI (a schedulable hint we ignore). RWF_DSYNC/RWF_SYNC/RWF_APPEND are write-only, and RWF_NOWAIT cannot be honored on OSv's blocking read path, so it is rejected with EOPNOTSUPP rather than faked with EAGAIN (which would make a retrying caller spin). - Writes honor RWF_DSYNC/RWF_SYNC by flushing after the write and PROPAGATE a fsync/fdatasync failure (a caller that asked for durability must not be told the write succeeded durably when it did not). - RWF_APPEND is rejected with EOPNOTSUPP. A correct one-shot append must resolve the offset atomically under the file lock at write time (the mechanism O_APPEND already uses via IO_APPEND in vfs_file::write); doing it with fstat()+pwritev() is racy (two concurrent one-shot appenders both write at the same previous EOF and clobber each other), which is worse than an honest error. Callers that need append open the fd with O_APPEND, which OSv supports atomically. renameat2(2): support RENAME_NOREPLACE; reject any other flag (RENAME_EXCHANGE/RENAME_WHITEOUT need filesystem-level support OSv lacks) in a single 'flags & ~RENAME_NOREPLACE -> EINVAL' check. RENAME_NOREPLACE is a best-effort fstatat()+renameat() with a documented FIXME that it is not atomic against a concurrent create (a correct version needs VFS-level support). Wired into the syscall table, exported from libc/musl, and covered by tst-fs-syscalls (which tolerates both OSv's EOPNOTSUPP and Linux's success for RWF_APPEND so it stays meaningful on both).
What
Three modern file syscalls that were missing or flag-rejecting.
preadv2/pwritev2implemented over the existingpreadv/pwritev.Supported
RWF_*flags:RWF_HIPRIis a scheduling hint and is ignored (the transfer is unchanged);RWF_DSYNC/RWF_SYNC(pwritev2)fdatasync/fsyncthe fd after the write;RWF_NOWAITcannot be honored (OSv's file path can block) so it is rejectedwith
EOPNOTSUPP. FakingEAGAIN(no data is actually pending) would make aretrying caller spin forever, so an honest "unsupported" is safer;
RWF_APPEND(pwritev2) is rejected withEOPNOTSUPP: a correct one-shotappend must resolve the offset atomically under the file lock at write time,
which needs a per-write append ioflag plumbed through the fops write path;
a racy
fstat()+pwritev()would let two concurrent appenders clobber eachother. Callers needing append can open the fd with
O_APPEND;EOPNOTSUPP.renameat2previously returnedEINVALfor any flag. Now honorsRENAME_NOREPLACEvia a pre-check (EEXISTif the destination exists - a benignTOCTOU that plain rename also has, covering the common atomic-create-via-rename
pattern).
RENAME_EXCHANGEandRENAME_WHITEOUTneed filesystem-level supportOSv's VFS does not provide and are rejected with
EINVAL;flags == 0behaveslike
renameat.Wired
SYS_preadv2/SYS_pwritev2/SYS_renameat2(adding the x86-64 numbers327/328/316; aarch64 already had them) with tracepoints, symbols exported. The
RWF_*/RENAME_*values are defined locally since musl 1.2.1 lacks them.Testing
tests/tst-fs-syscalls.cc: pwritev2/preadv2 round-trip,RWF_APPEND,RWF_DSYNC,RWF_NOWAIT(EOPNOTSUPP), unsupported RWF flag (EOPNOTSUPP), andrenameat2RENAME_NOREPLACE(success when dest absent,EEXISTwhen present),flags==0replace,RENAME_EXCHANGE(EINVAL). Passes on OSv under KVM;tst-mmap-file(30/30) unaffected.Note
openat2(needsstruct open_how/RESOLVE_*) is left for a follow-up.