mm: implement the membarrier(2) syscall

[gburd]

What

Implements the membarrier(2) syscall, which was missing entirely (not in the
syscall table, no libc entry, no prototype). Runtimes that use it for cheap
asymmetric synchronization (Go, .NET, some JVMs, RCU-style user libraries) could
not use it on OSv.

How

OSv is a single process whose threads all share one address space, so a "global"
and a "private" membarrier are equivalent here. The expedited barrier forces
every other CPU through an IPI: mmu::flush_tlb_all() already broadcasts an IPI
and waits for each CPU to run its handler (a full memory barrier on that CPU),
and we add a local seq_cst fence. This is a correct superset of what
membarrier promises (it also flushes TLBs, which is harmless), and reuses a
proven, arch-portable primitive rather than adding a dedicated per-arch IPI (x64
inter_processor_interrupt vs aarch64 sgi_interrupt). A lighter dedicated IPI
can replace it if membarrier ever shows up hot in a profile.

• MEMBARRIER_CMD_QUERY returns the supported command mask.
• REGISTER_{GLOBAL,PRIVATE}_EXPEDITED are accepted as no-ops (any thread may
  issue the barrier).
• {GLOBAL, GLOBAL_EXPEDITED, PRIVATE_EXPEDITED} run the barrier.
• SYNC_CORE / RSEQ / CPU-flag variants and unknown commands return EINVAL,
  as does a nonzero flags argument.

Wired as syscall SYS_membarrier (adding __NR_membarrier / SYS_membarrier
for x86-64) with a tracepoint, plus the sys/membarrier.h header with the
command enum and prototype, and the symbol exported from libc.so.6 and
ld-musl.so.1.

Testing

tests/tst-membarrier.cc covers QUERY, register, the three barrier commands,
and the EINVAL paths. Passes on OSv under KVM with 1, 2 and 4 vCPUs (exercising
the cross-CPU IPI broadcast).

(Recreated from #1414, which GitHub auto-closed when its branch was rebased onto current master. Same change, rebased and verified on master 3aba46c.)

[Copilot]

Pull request overview

Adds support for the Linux membarrier(2) syscall to OSv (including libc exposure and syscall plumbing) so runtimes/libraries that rely on it (e.g., Go/.NET/JVM/RCU-style libraries) can run correctly on OSv.

Changes:

• Implements membarrier() in runtime.cc and wires it into the syscall table and syscall tracepoints.
• Adds the public sys/membarrier.h API header and exports the symbol from libc.so.6 and ld-musl.so.1.
• Introduces a dedicated test (tst-membarrier) and adds it to the tests module build.

Reviewed changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
tests/tst-membarrier.cc	Adds a new test covering basic membarrier command behavior and error paths.
syscalls/syscalls.cc.in	Registers membarrier in the syscall table generation.
syscalls/syscall_tracepoints.cc.in	Adds a syscall tracepoint for membarrier.
runtime.cc	Implements the membarrier() libc/syscall behavior using flush_tlb_all() + fence.
modules/tests/Makefile	Includes the new membarrier test in the tests build.
linux.cc	Declares the membarrier entry point for the Linux syscall layer.
include/api/x64/bits/syscall.h	Defines __NR_membarrier / SYS_membarrier for x86-64.
include/api/sys/membarrier.h	Introduces the public API header with command/flag definitions and prototype.
exported_symbols/osv_libc.so.6.symbols	Exports the membarrier symbol from OSv libc.
exported_symbols/osv_ld-musl.so.1.symbols	Exports the membarrier symbol from the musl loader.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[nyh]

Looks good. I got used to the word "ponytail" ;-) and I'll see if github lets me squash the two patches together instead of asking you to do it.

[nyh]

Oh, I see copilot had some comments, and also there is now a conflict, so please rebase, squash the two patches together and address (or dispute) copilot's commnets. Thanks.

[gburd]

Thanks for the approve. Addressed Copilot's points and squashed into one coherent commit (I did the squash so you don't have to fight GitHub's UI):

• cpu_id validation: a nonzero cpu_id is only meaningful with MEMBARRIER_CMD_FLAG_CPU (which we don't support), so it's now rejected with EINVAL alongside nonzero flags. Added a test for it.
• "ponytail" tag: reworded to "Note:".
• aarch64 barrier property (Copilot): clarified the comment - flush_tlb_all() broadcasts an IPI and waits for each CPU to run its handler, and returning from an interrupt is a context-synchronizing full-barrier event on both x86-64 and aarch64, so the cross-CPU barrier holds on both arches (it's a superset - it also flushes TLBs, harmless).

Rebuilt, tst-membarrier passes (QUERY, all barrier commands, and the EINVAL paths for unknown command / nonzero flags / nonzero cpu_id).

[gburd]

Rebased onto current master (now conflict-free) and confirmed it's already a single squashed commit. The copilot points from the earlier round are addressed in it: nonzero cpu_id returns EINVAL (with a comment explaining cpu_id is only meaningful with the MEMBARRIER_CMD_FLAG_CPU flag we reject), the leftover marker comment is reworded to a plain Note, the aarch64-vs-x64 barrier rationale is spelled out, and the test now covers the cpu_id validation path. Ready for another look.