Skip to content

Route net and HTTP through native Socket payloads - #12267

Merged
proggeramlug merged 18 commits into
mainfrom
neta2-lead-port
Oct 9, 2026
Merged

proggeramlug merged 18 commits into
mainfrom
neta2-lead-port

Conversation

@proggeramlug

@proggeramlug proggeramlug commented Oct 9, 2026 •

Copy link
Copy Markdown
Contributor

Move net.Socket and net.Server ownership into native payloads whose completions route through retained owner links. HTTP/1 client/server parsers now sit above the actual Socket; upgrade changes its route and delivers the same Socket plus the head bytes. The inherited P0 prerequisite was previously posted as draft #12064.

The requested flat-micro/RSS target remains unmet. Against the measured base, HTTP 10k instructions increase 4.877% and net echo 1.097%. With THP disabled, their median RSS increases persist: HTTP +2,604 KiB, net +964 KiB. This PR is ready for review with those residual costs retained, not performance acceptance.

Round 2 selects constructor families from immutable class captures instead of mutable function names/prototype properties. Completion paths share validated payload windows and emitter/provider roots; all native borrows end before JS, close or reopen. Callback-free record reads/overwrites avoid allocating keys and retain inline/overflow write barriers. No new cache, latch or identity table is added. Socket open state has one SocketFields.opened field, with live connect/accept/adopted transitions and a mutation-tested gate.

Validation retained from Linux, pinned Node 26.5.1, measured base 678ab79517aa3c3ef939d5c9610af2596bd54be2, production head 0f981f054dd25e80a01308d4c0ee41934cb0dc3a:

  • Net: 22 tests including the 100k sequential/concurrent lifecycle witness; final affected subset 21. HTTP: 208. Runtime payload/transport: 51/10; constructor shapes 11, borrowed root handles 6, moving record overwrite 1, stream 74, global-constructor 1 and upgrade HIR 1.
  • Independent N2/N4/N8 sabotages turn the live clean witnesses red; N3 backlog sabotage is red. N8 aborts at the intended assertion, rather than a segmentation fault.
  • Gap subset: 105 unique fixtures; baseline 87 pass/15 mismatch/1 timeout/2 lane-only, candidate 90 pass/15 shared mismatch/0 timeout. Zero regressions. Final shared-super change rechecks affected constructor/heritage fixtures and four stream controls.
  • Script gates: baseline 102/125, candidate 103/125, no additional failing IDs. Lean/pruned builds pass. Strict Clippy retains the baseline large_const_arrays failure; changed-crate comparison has zero added warning/file pairs. Fresh read-only socket-open gate, mutation self-tests, Node pin consistency and diff whitespace checks pass.

Performance evidence uses n=5 interleaved on qb6, CPUs 0–55, setarch -R, separate coherent compiler/runtime/stdlib targets and output/exit checks against Node for every retained trial. Instructions and RSS were measured; no cycles claim. Full-GC companion counts are retained. Upgrade/churn are correctness witnesses only because the baseline times out or differs from Node. Main advanced afterwards; these results do not compare against its later tip.

Residual follow-ups:

  • net-capability-carry: carry proven Socket/response capabilities across callback-free Agent/request/response helpers.
  • http-app-record-lifetime: compact/reclaim ClientRequest/IncomingMessage/response records with traced JS lifetimes while preserving req.socket and delayed-generation guards. ClientRequest and response records each grow 40 bytes; existing app registries retain them, explaining proportional native-memory growth.
  • net-cold-constructor-dispatch and net-code-residency: isolate the remaining Hello startup instruction offset and reduce duplicated inlined proof/dispatch bodies. THP-off and smaps evidence shows persistent resident code-page costs; sampled percentages do not exactly apportion the instruction deltas.
  • ws-unused-client-row-cleanup: remove seven unused ws Client codegen/manifest rows. The baseline buffer ledger path and raw-socket ws limitations also remain documented; no green full-workspace claim.

The verified source bundle, raw profiles/results, normalized gap outputs, GC companions, THP/smaps controls and build fingerprints are retained offhost. The evidence archive contains fingerprints, not executable binaries; all seven owned targets were deleted and the namespace is inactive. Production source is frozen; the publication changelog is the only added metadata.

Ralph Küpper added 17 commits October 9, 2026 08:44
Bring net transport P0 (#12064, #11919) onto 8476086. Keep current
PayloadVTable, PayloadBuffer support, stream state record, native-this
alias and ServerResponse payload ABI; extend that ABI rather than add
P0's older duplicate implementation.

TransportCore is inline at offset zero of TransportPayload<E>. Tokens
carry route/op/cell, terminal driver completions retain the cell, close
hands the handle to the driver, and generational handles and resolver ops
protect reopen. Link teardown discards completions before heap teardown.
Keep id routes for subsystems that have not migrated; add no id/cell alias.

Use MaybeUninit words for opaque FFI storage and check its ABI before
initialization. Expose binding owner links, lifecycle and typed reopen
through the shared cell/vtable. A checked route store preserves the live
multishot read. Runtime witnesses cover N1, N5, N6, N7, N9, N10, N11,
and the runtime half of N8, with nine sabotages that turn them red.

Phase A's ext-net and HTTP connection conversion remains separate work.
…ted transports

perry-ext-net's sockets and servers are TransportPayload native payloads:
completions route by link token through the payload cell, so the id-keyed
socket, server and event tables are gone (ledger: ext-net 19 tables and
10 producers, now 0 and 0). A closed, dropped Socket is collected and its
WeakRefs clear. Class ids live in perry-abi; codegen drops the net id rows.
…route store

ext-http's server and client HTTP/1 connections are net.Socket payloads
(decision 77), so an HTTP upgrade is one store to the connection's route:
the listener receives the same Socket, with head bytes and every later byte
as its own data. The HIR no longer tags an upgrade listener's socket
parameter as a ws Client (the static tag sent its data listener to a ws id
table that never fired); ws client handles keep their own dynamic dispatch.
The HTTP/2 client gets its own id-routed slot 16; the driver keeps 32 sink
slots, and link routes use the 16 that fit the link token's route field.
N2-N4 binding witnesses with sabotages (late_close, churn_backlog,
early_unref), N8 server and client upgrade tests (route/head sabotages),
the typed upgrade gap fixture, and the HIR test that the upgrade socket
parameter lowers to ordinary method calls.
…ield

Registry census drops 249 to 225 (shrinking maps 167 to 146) because phase A deletes 24 net/HTTP/TLS tables and adds none. Adjust parser liveness floors 250/170 to 220/140 for the deleted population.
Remove the unused direct bytes dependency and the stale main VERDICT_CLASSES registry exemption exposed by the corrected census floor. Simplify the no-op promise pointer cast and the two HTTP Clippy findings without changing transport behavior.
Canonical native exports capture their class identity in a distinct constructor layout. Both explicit receiver calls and dynamic super read that fact, with no constructor-name comparisons or new side table. Ordinary bound methods skip native receiver validation. The fixture mutates function names and prototype constructor properties before call, Reflect.apply and aliased super.
Use callback-free own-data record reads and barriered overwrites instead of allocating field keys. Project each data completion once and batch parser and native write/deadline effects within that proof. Share the emitter/provider dispatch scope, keep short argument spans inline, and avoid JS buffers when no listener can receive them or provider work for empty write acknowledgements. End every native borrow before JS, close or reopen. Add moving inline/overflow-edge and borrowed-handle-scope witnesses; keep the issuer-owned HTTP pending-event scanner custody accurate after the main rebase.
Resolve a bound parent to either its immutable native class fact or the existing export metadata in one shared closure validation. Ordinary dynamic super calls no longer run two receiver probes and two fallback registry lookups. Constructor family selection still compares only static info identity and the numeric class capture.
@coderabbitai

coderabbitai Bot commented Oct 9, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

Note

Currently processing new changes in this PR. This may take a few minutes, please wait...

⚙️ Run configuration
  • Configuration used: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 70bbb399-7c53-44eb-90c2-6e69d58a2594

📥 Commits

Reviewing files that changed from the base of the PR and between c0a83c5 and 491ecaf.


⛔ Files ignored due to path filters (5)
  • Cargo.lock is excluded by !**/*.lock
  • crates/perry-codegen/src/gc_effects/linux-x86_64.tsv is excluded by !**/*.tsv
  • crates/perry-codegen/src/gc_effects/macos-aarch64.tsv is excluded by !**/*.tsv
  • crates/perry-codegen/src/gc_effects/windows-x86_64.tsv is excluded by !**/*.tsv
  • crates/perry-codegen/src/wasm32/runtime_abi.tsv is excluded by !**/*.tsv

📒 Files selected for processing (199)
  • changelog.d/12064-net-transport-p0.md
  • changelog.d/12064-net-transport-phase-a.md
  • changelog.d/12267-native-socket-transport.md
  • crates/perry-abi/src/native_class_ids.rs
  • crates/perry-codegen/src/expr/property_get_names.rs
  • crates/perry-codegen/src/expr/this_super_call.rs
  • crates/perry-codegen/src/ext_registry.rs
  • crates/perry-codegen/src/lower_call/args_sweep_tests.rs
  • crates/perry-codegen/src/lower_call/conversion_window_tests.rs
  • crates/perry-codegen/src/lower_call/native_module_dispatch.rs
  • crates/perry-codegen/src/lower_call/native_table/bun.rs
  • crates/perry-codegen/src/lower_call/native_table/net_classes_state.rs
  • crates/perry-codegen/src/lower_call/native_table/net_events.rs
  • crates/perry-codegen/src/runtime_decls/stdlib_ffi/language_core.rs
  • crates/perry-db-turnloop/src/lib.rs
  • crates/perry-ext-http/Cargo.toml
  • crates/perry-ext-http/src/agent.rs
  • crates/perry-ext-http/src/agent/tls_compat.rs
  • crates/perry-ext-http/src/client_abort.rs
  • crates/perry-ext-http/src/client_connect_override.rs
  • crates/perry-ext-http/src/client_events.rs
  • crates/perry-ext-http/src/client_request_surface.rs
  • crates/perry-ext-http/src/client_turnloop/conn.rs
  • crates/perry-ext-http/src/client_turnloop/mod.rs
  • crates/perry-ext-http/src/client_turnloop/native_client.rs
  • crates/perry-ext-http/src/client_turnloop/native_client_tests.rs
  • crates/perry-ext-http/src/client_turnloop/pool.rs
  • crates/perry-ext-http/src/client_turnloop/protocol.rs
  • crates/perry-ext-http/src/client_turnloop/protocol_tests.rs
  • crates/perry-ext-http/src/client_turnloop/raw_socket.rs
  • crates/perry-ext-http/src/client_turnloop/tests.rs
  • crates/perry-ext-http/src/client_turnloop/tls.rs
  • crates/perry-ext-http/src/lib.rs
  • crates/perry-ext-http/src/pending_dispatch.rs
  • crates/perry-ext-http/src/plain_client.rs
  • crates/perry-ext-http/src/response_headers.rs
  • crates/perry-ext-http/src/root_scanner.rs
  • crates/perry-ext-http/src/server/handle_dispatch.rs
  • crates/perry-ext-http/src/server/http2_server.rs
  • crates/perry-ext-http/src/server/http2_server/controls.rs
  • crates/perry-ext-http/src/server/http2_server/dispatch.rs
  • crates/perry-ext-http/src/server/http2_server/pump.rs
  • crates/perry-ext-http/src/server/http2_server/session.rs
  • crates/perry-ext-http/src/server/http2_server/turnloop_glue.rs
  • crates/perry-ext-http/src/server/http2_server/turnloop_listen.rs
  • crates/perry-ext-http/src/server/mod.rs
  • crates/perry-ext-http/src/server/request.rs
  • crates/perry-ext-http/src/server/response.rs
  • crates/perry-ext-http/src/server/response_turnloop.rs
  • crates/perry-ext-http/src/server/server.rs
  • crates/perry-ext-http/src/server/server/turnloop_listen.rs
  • crates/perry-ext-http/src/server/server/upgrade_tests.rs
  • crates/perry-ext-http/src/server/turnloop_h2/backpressure_tests.rs
  • crates/perry-ext-http/src/server/turnloop_h2/conn.rs
  • crates/perry-ext-http/src/server/turnloop_h2/control.rs
  • crates/perry-ext-http/src/server/turnloop_h2/mod.rs
  • crates/perry-ext-http/src/server/turnloop_h2/native_server.rs
  • crates/perry-ext-http/src/server/turnloop_h2/stream.rs
  • crates/perry-ext-http/src/server/turnloop_h2/target.rs
  • crates/perry-ext-http/src/server/turnloop_route.rs
  • crates/perry-ext-http/src/server/turnloop_serve/conn.rs
  • crates/perry-ext-http/src/server/turnloop_serve/mod.rs
  • crates/perry-ext-http/src/server/turnloop_serve/native_tests.rs
  • crates/perry-ext-http/src/server/types.rs
  • crates/perry-ext-http/src/server/upgrade.rs
  • crates/perry-ext-http/src/tests.rs
  • crates/perry-ext-http/src/tls_client.rs
  • crates/perry-ext-http/tests/turnloop_client_exchange.rs
  • crates/perry-ext-net/Cargo.toml
  • crates/perry-ext-net/src/adopt.rs
  • crates/perry-ext-net/src/buffer_pool.rs
  • crates/perry-ext-net/src/bun_tcp.rs
  • crates/perry-ext-net/src/classes.rs
  • crates/perry-ext-net/src/classes_prototype.rs
  • crates/perry-ext-net/src/cork.rs
  • crates/perry-ext-net/src/dispatch.rs
  • crates/perry-ext-net/src/dispatch_custody.rs
  • crates/perry-ext-net/src/gc_roots.rs
  • crates/perry-ext-net/src/handle_exports.rs
  • crates/perry-ext-net/src/handle_ids.rs
  • crates/perry-ext-net/src/ipc.rs
  • crates/perry-ext-net/src/jsvalue.rs
  • crates/perry-ext-net/src/lib.rs
  • crates/perry-ext-net/src/lifecycle.rs
  • crates/perry-ext-net/src/native_dispatch.rs
  • crates/perry-ext-net/src/native_transport.rs
  • crates/perry-ext-net/src/option_setters.rs
  • crates/perry-ext-net/src/payload_abi.rs
  • crates/perry-ext-net/src/payload_closed.rs
  • crates/perry-ext-net/src/payload_events.rs
  • crates/perry-ext-net/src/payload_io.rs
  • crates/perry-ext-net/src/payload_pipe.rs
  • crates/perry-ext-net/src/payload_prototype.rs
  • crates/perry-ext-net/src/payload_provider.rs
  • crates/perry-ext-net/src/payload_server.rs
  • crates/perry-ext-net/src/payload_socket.rs
  • crates/perry-ext-net/src/payload_tests.rs
  • crates/perry-ext-net/src/payload_tls.rs
  • crates/perry-ext-net/src/payload_transport.rs
  • crates/perry-ext-net/src/pipe.rs
  • crates/perry-ext-net/src/provider_lifecycle.rs
  • crates/perry-ext-net/src/raw_bridge.rs
  • crates/perry-ext-net/src/server_state.rs
  • crates/perry-ext-net/src/socket_emit.rs
  • crates/perry-ext-net/src/socket_events.rs
  • crates/perry-ext-net/src/socket_facade.rs
  • crates/perry-ext-net/src/tests.rs
  • crates/perry-ext-net/src/tls.rs
  • crates/perry-ext-net/src/turnloop_io.rs
  • crates/perry-ext-net/src/turnloop_tls.rs
  • crates/perry-ext-net/src/turnloop_tls_io.rs
  • crates/perry-ext-net/src/unconnected_write_tests.rs
  • crates/perry-ext-net/tests/pump_symbol_identity.rs
  • crates/perry-ext-ws/Cargo.toml
  • crates/perry-ext-ws/src/lib.rs
  • crates/perry-ext-ws/src/native_socket.rs
  • crates/perry-ffi/src/agent_post.rs
  • crates/perry-ffi/src/jsvalue.rs
  • crates/perry-ffi/src/lib.rs
  • crates/perry-ffi/src/native_payload.rs
  • crates/perry-ffi/src/native_payload_links.rs
  • crates/perry-ffi/src/transient_roots.rs
  • crates/perry-ffi/src/turnloop_net.rs
  • crates/perry-hir/src/lower/context.rs
  • crates/perry-hir/src/lower/context_new.rs
  • crates/perry-hir/src/lower/expr_call/mod.rs
  • crates/perry-hir/src/lower/expr_call/prescans.rs
  • crates/perry-hir/src/lower/expr_call/static_and_instance.rs
  • crates/perry-hir/src/lower/expr_member.rs
  • crates/perry-hir/src/lower/lower_module_fn.rs
  • crates/perry-hir/src/lower/lowering_context.rs
  • crates/perry-hir/src/lower/pre_scan.rs
  • crates/perry-hir/src/lower_decl/fn_decl.rs
  • crates/perry-hir/src/lower_patterns.rs
  • crates/perry-hir/tests/upgrade_socket_param_plain_dispatch.rs
  • crates/perry-hir/tests/ws_upgrade_param_native_dispatch.rs
  • crates/perry-runtime/Cargo.toml
  • crates/perry-runtime/src/agent.rs
  • crates/perry-runtime/src/async_hooks.rs
  • crates/perry-runtime/src/async_hooks/provider_ffi.rs
  • crates/perry-runtime/src/async_hooks/test_support.rs
  • crates/perry-runtime/src/closure/dispatch/bound.rs
  • crates/perry-runtime/src/closure/mod.rs
  • crates/perry-runtime/src/closure/registry.rs
  • crates/perry-runtime/src/event_pump/agent_loop.rs
  • crates/perry-runtime/src/gc/mod.rs
  • crates/perry-runtime/src/gc/roots/runtime_handles.rs
  • crates/perry-runtime/src/gc/roots/runtime_handles/tests.rs
  • crates/perry-runtime/src/gc/tests/mod.rs
  • crates/perry-runtime/src/gc/tests/native_payload.rs
  • crates/perry-runtime/src/gc/tests/native_payload_callbacks.rs
  • crates/perry-runtime/src/gc/tests/native_payload_callbacks/lifecycle.rs
  • crates/perry-runtime/src/gc/tests/net_transport.rs
  • crates/perry-runtime/src/gc/tests/residual_prototype_relocation.rs
  • crates/perry-runtime/src/gc/tests/rooted_container_values.rs
  • crates/perry-runtime/src/lib.rs
  • crates/perry-runtime/src/native_class_ids.rs
  • crates/perry-runtime/src/native_handle.rs
  • crates/perry-runtime/src/native_payload.rs
  • crates/perry-runtime/src/native_payload_abi.rs
  • crates/perry-runtime/src/native_payload_attach.rs
  • crates/perry-runtime/src/native_payload_census.rs
  • crates/perry-runtime/src/native_payload_event_abi.rs
  • crates/perry-runtime/src/native_payload_links.rs
  • crates/perry-runtime/src/native_payload_transport_abi.rs
  • crates/perry-runtime/src/node_stream.rs
  • crates/perry-runtime/src/node_stream_event_emitter.rs
  • crates/perry-runtime/src/object/class_constructors.rs
  • crates/perry-runtime/src/object/field_get_set.rs
  • crates/perry-runtime/src/object/field_get_set/accessors.rs
  • crates/perry-runtime/src/object/global_this/fetch_globals.rs
  • crates/perry-runtime/src/object/mod.rs
  • crates/perry-runtime/src/object/native_module.rs
  • crates/perry-runtime/src/object/native_module/callable_exports.rs
  • crates/perry-runtime/src/object/native_module_dispatch.rs
  • crates/perry-runtime/src/object/native_module_dispatch/dispatch_m_p.rs
  • crates/perry-runtime/src/tls.rs
  • crates/perry-runtime/src/turnloop_net/abi.rs
  • crates/perry-runtime/src/turnloop_net/errors.rs
  • crates/perry-runtime/src/turnloop_net/mod.rs
  • crates/perry-runtime/src/turnloop_net/sink.rs
  • crates/perry-runtime/src/turnloop_net/tests.rs
  • crates/perry-runtime/src/turnloop_net/transport.rs
  • crates/perry-runtime/src/turnloop_net/windows_pipe.rs
  • crates/perry-runtime/src/turnloop_net/write_queue.rs
  • crates/perry-stdlib/src/runtime_thread_exit_tests/listeners_tests.rs
  • crates/perry/tests/ws_client_handle_cross_function_dispatch.rs
  • docs/native-payload-pattern.md
  • scripts/check_ext_net_socket_open_invariant.py
  • scripts/ext_net_socket_open_exceptions.json
  • scripts/gc_runtime_root_holders.json
  • scripts/native_handle_ledger.py
  • scripts/native_handle_ledger.txt
  • scripts/registry_lifetime_allowlist.json
  • scripts/registry_lifetime_check.py
  • scripts/thread_exit_address_globals.json
  • scripts/unrooted_local_shape_baseline.json
  • test-files/test_gap_http_upgrade_socket_data.ts
  • test-files/test_gap_net_constructor_class_identity.ts

 _______________________________
< Grow a pair... of test cases. >
 -------------------------------
  \
   \   (\__/)
       (•ㅅ•)
       /   づ
✨ Finishing Touches
📝 Generate docstrings
  • Commit to this branch
  • Create a new PR

🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR


  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant