Route net and HTTP through native Socket payloads

changelog.d/12064-net-transport-p0.md

@@ -0,0 +1,16 @@
+Add an opaque TransportCore at offset zero of a native transport payload and
+link-token dispatch through the payload cell. Each terminal completion owed by
+the driver retains the cell; close moves its handle into the driver, and stale
+generational completions cannot mutate a reopened transport. Accepted resources
+are installed directly into child payloads. Teardown discards link completions.
+
+Rebase P0 onto the current payload ABI: preserve its typed PayloadVTable,
+PayloadBuffer support, stream state record and native-this alias. Binding owner
+links and reopen use the same cell and vtable, with no second native_state word.
+Id routes remain available to subsystems that have not yet migrated; this
+prerequisite does not convert ext-net or HTTP connections.
+
+The binding's opaque storage uses MaybeUninit words, so Rust padding and
+reserved tail bytes may remain uninitialized. Its constructor checks the ABI
+layout before the runtime writes the block. A route switch changes only the
+core route, preserving the outstanding multishot read's operation and token.

changelog.d/12064-net-transport-phase-a.md

@@ -0,0 +1,86 @@
+`net.Socket` and `net.Server` are native payloads with a TransportCore, and
+their completions route by link token through the payload cell. perry-ext-net
+no longer keeps id-keyed socket or server tables (19 tables and 10 producers
+in the native handle ledger, now 0 and 0), so a Socket that closes and is
+dropped is collected and its WeakRefs clear, as in Node.
+
+HTTP/1 server and client connections are `net.Socket` payloads. An HTTP
+`upgrade` is a single store to the connection's route: the listener gets the
+same Socket, with the bytes that arrived with the head as `head` and every
+later byte as its own `data`. The upgrade listener's socket parameter is an
+ordinary value; the compiler no longer tags it as a `ws` Client, which used to
+register its `data` listener in a ws id table that never fired. A server with
+an attached `WebSocketServer` still hands the listener a ws client handle,
+which dispatches through ws's own handle methods.
+
+The socket-open audit now requires one `SocketFields.opened` declaration and
+live connect/accept transitions, and rejects duplicate Rust/JS flags and
+id-keyed maps. Bun's duplicate `bunOpened` latch is removed: its existing
+pending connect promise selects `connectError`. The workflow step is retained.
+
+Registry lifetime census: current main has 249 maps (167 with removal paths),
+phase A has 225 (146 with removal paths). All 24 disappearing identities are
+net/HTTP/TLS tables; none are added. Parser-liveness floors are adjusted from
+250/170 to 220/140 solely because those tables were deleted.
+
+Review notes: main already violates the native handle ledger because
+`buffer/header.rs` moved to `buffer/store/layout.rs`. The seven ws Client
+codegen rows and their manifest entries are unused after removal of the HTTP
+upgrade parameter hint; delete them in the immediate follow-up. Linux ext-ws
+lib tests fail to compile on main (`js_shadow_frame_push` is absent with native
+stack maps). Main also fails issue_9619's manual/callback-only path and
+`nested_object_literal_ws_inbound`: ws handleUpgrade has no raw-socket path.
+
+Round 2 selects net constructor families from immutable class captures in a
+canonical constructor layout. Explicit receiver calls and dynamic super use
+that fact even after function names and prototype constructor properties are
+changed. Dynamic super shares one validated parent resolution with the existing
+native-export dispatch; ordinary parents do not repeat the header or registry
+lookup.
+
+Native completions use callback-free payload windows, ending every borrow
+before JS, close or reopen. HTTP batches parser and native I/O steps within a
+proof. Opaque record reads and overwrites avoid allocating property-key strings;
+inline and overflow stores preserve the runtime write barriers and moving-root
+edges. Emitter arguments and async provider hooks share one dispatch scope and
+short argument spans stay inline. Empty write acknowledgements skip callback
+work, and plaintext allocates a JS Buffer only for a listener that can receive it.
+Deferred events keep their required traced captures. No cache, table or latch is
+added.
+
+Round 2 validation: ext-net 22 (including 100k lifecycle), ext-http 208, moving
+record stores and borrowed roots pass. Constructor mutation/heritage fixtures
+pass after the shared-super fix. N2/N3/N4/N8 sabotage witnesses reject their
+intended faults. The full gap subset has zero regressions: main 87 pass, 15
+mismatch, one timeout, two lane-only; lane 90 pass, 15 mismatch, no timeout.
+The final heritage change rechecks its affected fixtures. Script lint is
+102/125 on main and 103/125 in lane, with no new failing gates. Changed-crate
+Clippy adds no warning/file pairs; strict workspace Clippy inherits main's
+perry-dispatch large_const_arrays failure. Lean/pruned dependency gates pass.
+
+A/B is pinned to the required initial rebase base 678ab79517, with separate
+Linux targets, CPUs 0–55, setarch -R, n=5 interleaved instructions:u/RSS. Main
+advanced while this work ran. TSC +0.011%, Zod -0.006%, fastify +0.392%,
+buffer +0.007% and worker +0.069% instructions are within the observed
+within-arm spans. Hello has a reproducible +709-instruction startup offset
+(+0.052%); exact symbol attribution is still unisolated.
+
+Net echo is improved to +1.10% instructions and HTTP 10k to +4.88%, but the
+requested flat-micro/RSS target is unmet. Perf profiles still show payload
+projection, opaque-record access and rooting outside the batched completions.
+THP-off RSS remains +964 KiB net and +2,604 KiB HTTP. Net's anonymous heap
+is smaller; its remaining footprint is fixed resident code pages. HTTP's
+request/response records each grow by 40 bytes, amplifying existing app
+registry retention proportionally with request count. Follow-up: carry proven
+capabilities through Agent/request/response operations, compact or reclaim
+app records according to traced JS lifetimes without breaking req.socket or
+delayed-generation checks, and isolate the cold native-constructor branch
+from common closure dispatch. No new side table, cache or latch is proposed.
+
+Real RSS controls: TSC's delta changes sign and worker ranges overlap with
+THP off; fastify retains +2,324 KiB, chiefly resident executable mappings
+(+2,404 KiB text at exit) plus +368 KiB anonymous memory. Buffer's anonymous
+heap differs by only +4 KiB. Reduce duplicated inlined proof/dispatch bodies
+in the follow-up and remeasure code residency; exit probes do not exactly
+decompose peak RSS. Full-collection counts are hello/Zod/net/HTTP 0/0,
+TSC/fastify 1/1, buffer 36/36, worker 42/41 (main/lane).

changelog.d/12267-native-socket-transport.md

@@ -0,0 +1,3 @@
+Move net.Socket/net.Server and HTTP/1 connection ownership to native payloads with owner-link completion routing. HTTP upgrade keeps the same Socket and routes head/later data through it. Select native constructor families from immutable class identity, share payload proofs and event/provider roots, and retain exact write barriers, delayed-generation guards and one authoritative opened field.
+
+Linux lifecycle, upgrade, moving-root and independent sabotage witnesses pass; the 105-fixture Node subset has zero regressions against the measured base. The flat-micro/RSS target remains unmet: HTTP instructions +4.877%, net +1.097%, with persistent THP-off RSS increases. Follow-ups cover capability propagation, traced HTTP app-record lifetimes, cold constructor/code footprint and unused ws Client dispatch rows. Full measurements and known baseline failures remain in the PR description.

crates/perry-abi/src/native_class_ids.rs

@@ -12,3 +12,11 @@ pub const ZSTD_COMPRESS: u32 = 0xFFFF_2426;
 pub const ZSTD_DECOMPRESS: u32 = 0xFFFF_2427;
 pub const ZLIB_BASE: u32 = 0xFFFF_2428;
 pub const HTTP_SERVER_RESPONSE: u32 = 0xFFFF_2429;
+pub const NET_BLOCK_LIST: u32 = 0xFFFF_242A;
+pub const NET_SOCKET_ADDRESS: u32 = 0xFFFF_242B;
+pub const NET_SOCKET: u32 = 0xFFFF_242C;
+pub const NET_SERVER: u32 = 0xFFFF_242D;
+pub const HTTP_PARSER: u32 = 0xFFFF_242E;
+pub const WS_CODEC: u32 = 0xFFFF_242F;
+pub const HTTP2_CODEC: u32 = 0xFFFF_2430;
+pub const HTTP_CLIENT_PARSER: u32 = 0xFFFF_2431;

crates/perry-codegen/src/expr/property_get_names.rs

@@ -106,18 +106,10 @@ fn is_net_server_method_name(name: &str) -> bool {
     )
 }
 
-fn is_net_block_list_method_name(name: &str) -> bool {
-    matches!(
-        name,
-        "addAddress" | "addRange" | "addSubnet" | "check" | "toJSON" | "fromJSON"
-    )
-}
-
 pub(super) fn is_net_native_method_value(class_name: &str, name: &str) -> bool {
     match class_name {
         "Socket" => is_net_socket_method_name(name),
         "Server" => is_net_server_method_name(name),
-        "BlockList" => is_net_block_list_method_name(name),
         _ => false,
     }
 }

crates/perry-codegen/src/expr/this_super_call.rs

@@ -555,6 +555,41 @@ pub(crate) fn lower(ctx: &mut FnCtx<'_>, expr: &Expr) -> Result<String> {
                     return Ok(double_literal(0.0));
                 }
             };
+            if let Some((module, export)) = current_class.native_extends.as_ref() {
+                if module.trim_start_matches("node:") == "net"
+                    && matches!(export.as_str(), "Socket" | "Stream" | "Server")
+                {
+                    let undef = double_literal(f64::from_bits(crate::nanbox::TAG_UNDEFINED));
+                    let (args, roots) = crate::lower_call::lower_call_args_rooted(ctx, super_args)?;
+                    let options = args.first().cloned().unwrap_or_else(|| undef.clone());
+                    let callback = args.get(1).cloned().unwrap_or_else(|| undef.clone());
+                    let this = ctx
+                        .this_stack
+                        .last()
+                        .cloned()
+                        .map(|slot| ctx.block().load(DOUBLE, &slot))
+                        .unwrap_or_else(|| undef.clone());
+                    let kind = u32::from(export == "Server").to_string();
+                    ctx.block().call(
+                        DOUBLE,
+                        "js_net_subclass_init",
+                        &[
+                            (DOUBLE, &this),
+                            (DOUBLE, &options),
+                            (DOUBLE, &callback),
+                            (I32, &kind),
+                        ],
+                    );
+                    bind_derived_this_after_super(ctx);
+                    crate::lower_call::apply_field_initializers_recursive(
+                        ctx,
+                        &current_class_name,
+                        crate::lower_call::FieldInitMode::SelfOnly,
+                    )?;
+                    roots.release(ctx);
+                    return Ok(undef);
+                }
+            }
             // #5437 (Next.js p-queue `PQueue`): when HIR captured a dynamic
             // `extends_expr` for this class, the parent is a LEXICAL runtime
             // value (an in-scope local / require result) — NOT the same-named

crates/perry-codegen/src/ext_registry.rs

@@ -476,6 +476,7 @@ const FFI_REGISTRY: &[(&str, OwnerKind)] = &[
     // stdlib-only link resolved them without the wrapper. That copy is
     // deleted; perry-ext-net is the only definition, so an emitted call must
     // put its archive on the link line like every other net symbol.
+    ("js_ext_net_socket_alloc",                    OwnerKind::WellKnown("net")),
     ("js_net_socket_alloc",                         OwnerKind::WellKnown("net")),
     ("js_net_socket_connect",                       OwnerKind::WellKnown("net")),
     ("js_net_socket_method_connect",                OwnerKind::WellKnown("net")),

crates/perry-codegen/src/gc_effects/linux-x86_64.tsv

@@ -3239,10 +3239,10 @@ js_timer_validate_callback	Reenters
 js_tls_check_server_identity	Reenters
 js_tls_client_check_identity	Reenters
 js_tls_client_check_identity_from_metadata	Reenters
-js_tls_client_is_connected	Leaf
+js_tls_client_is_connected	Reenters
 js_tls_client_preflight	Reenters
-js_tls_client_record_closed	Leaf
-js_tls_client_record_connected	Leaf
+js_tls_client_record_closed	Reenters
+js_tls_client_record_connected	Reenters
 js_tls_client_record_start	Reenters
 js_tls_convert_alpn_protocols	Reenters
 js_tls_create_secure_context	Reenters
@@ -3283,7 +3283,7 @@ js_tls_socket_get_protocol	Reenters
 js_tls_socket_get_session	Reenters
 js_tls_socket_get_shared_sigalgs	Reenters
 js_tls_socket_get_x509_certificate	Reenters
-js_tls_socket_is_session_reused	Leaf
+js_tls_socket_is_session_reused	Reenters
 js_tls_socket_set_key_cert	Reenters
 js_tls_socket_set_max_send_fragment	Reenters
 js_tls_tlssocket_constructor	Reenters

crates/perry-codegen/src/gc_effects/macos-aarch64.tsv

@@ -3246,10 +3246,10 @@ js_timer_validate_callback	Reenters
 js_tls_check_server_identity	Reenters
 js_tls_client_check_identity	Reenters
 js_tls_client_check_identity_from_metadata	Reenters
-js_tls_client_is_connected	Leaf
+js_tls_client_is_connected	Reenters
 js_tls_client_preflight	Reenters
-js_tls_client_record_closed	Leaf
-js_tls_client_record_connected	Leaf
+js_tls_client_record_closed	Reenters
+js_tls_client_record_connected	Reenters
 js_tls_client_record_start	Reenters
 js_tls_convert_alpn_protocols	Reenters
 js_tls_create_secure_context	Reenters
@@ -3290,7 +3290,7 @@ js_tls_socket_get_protocol	AllocOnly
 js_tls_socket_get_session	Reenters
 js_tls_socket_get_shared_sigalgs	Reenters
 js_tls_socket_get_x509_certificate	Reenters
-js_tls_socket_is_session_reused	Leaf
+js_tls_socket_is_session_reused	Reenters
 js_tls_socket_set_key_cert	Reenters
 js_tls_socket_set_max_send_fragment	ThrowOnly
 js_tls_tlssocket_constructor	Reenters

crates/perry-codegen/src/gc_effects/windows-x86_64.tsv

@@ -3246,10 +3246,10 @@ js_timer_validate_callback	Reenters
 js_tls_check_server_identity	Reenters
 js_tls_client_check_identity	Reenters
 js_tls_client_check_identity_from_metadata	Reenters
-js_tls_client_is_connected	Leaf
+js_tls_client_is_connected	Reenters
 js_tls_client_preflight	Reenters
-js_tls_client_record_closed	Leaf
-js_tls_client_record_connected	Leaf
+js_tls_client_record_closed	Reenters
+js_tls_client_record_connected	Reenters
 js_tls_client_record_start	Reenters
 js_tls_convert_alpn_protocols	Reenters
 js_tls_create_secure_context	Reenters
@@ -3290,7 +3290,7 @@ js_tls_socket_get_protocol	Reenters
 js_tls_socket_get_session	Reenters
 js_tls_socket_get_shared_sigalgs	Reenters
 js_tls_socket_get_x509_certificate	Reenters
-js_tls_socket_is_session_reused	Leaf
+js_tls_socket_is_session_reused	Reenters
 js_tls_socket_set_key_cert	Reenters
 js_tls_socket_set_max_send_fragment	Reenters
 js_tls_tlssocket_constructor	Reenters

crates/perry-codegen/src/lower_call/args_sweep_tests.rs

@@ -188,8 +188,7 @@ fn console_instance_call_roots_receiver_across_arguments() {
 
 #[test]
 fn native_instance_method_roots_receiver_across_arguments() {
-    let sig = super::native_module_lookup("net", true, "write", Some("Socket"))
-        .expect("net.Socket.write resolves");
+    assert!(super::native_module_lookup("net", true, "write", Some("Socket")).is_none());
     assert_first_operand_rooted(
         native(
             "net",
@@ -198,8 +197,8 @@ fn native_instance_method_roots_receiver_across_arguments() {
             "write",
             vec![collecting("chunk")],
         ),
-        sig.runtime,
-        "native-table instance receiver",
+        "js_native_call_method_nullsafe",
+        "ordinary Socket receiver",
     );
 }
 

crates/perry-codegen/src/lower_call/conversion_window_tests.rs

@@ -115,15 +115,15 @@ fn ui_table_call_roots_first_string_pointer_across_second_conversion() {
 fn native_table_call_roots_first_string_pointer_across_second_conversion() {
     // U2: `lower_native_module_dispatch` took `js_value_to_str_ptr_for_ffi` of
     // each `StrPtr` argument in order.
-    let sig = super::native_module_lookup("net", true, "check", Some("BlockList"))
-        .expect("net.BlockList.check resolves");
+    let sig = super::native_module_lookup("http", true, "appendHeader", Some("ServerResponse"))
+        .expect("http.ServerResponse.appendHeader resolves");
     let ir = compile_stmt(
         native(
-            "net",
-            Some("BlockList"),
+            "http",
+            Some("ServerResponse"),
             Some(object("recv")),
-            "check",
-            vec![object("address"), object("family")],
+            "appendHeader",
+            vec![object("name"), object("value")],
         ),
         CompileOptions::default(),
     );

crates/perry-codegen/src/lower_call/native_module_dispatch.rs

@@ -498,14 +498,26 @@ mod ffi_return_type_tests {
     }
 
     #[test]
-    fn net_socket_write_and_end_match_their_float_ffi_abi() {
-        for method in ["write", "end"] {
-            let sig = super::native_module_lookup("net", true, method, Some("Socket"))
-                .unwrap_or_else(|| panic!("net.Socket.{method} must resolve"));
+    fn net_socket_methods_keep_their_external_float_abi_without_id_lowering() {
+        let abi = include_str!("../wasm32/runtime_abi.tsv");
+        for (method, symbol) in [
+            ("write", "js_ext_net_socket_write3"),
+            ("end", "js_ext_net_socket_end3"),
+        ] {
+            assert!(
+                super::native_module_lookup("net", true, method, Some("Socket")).is_none(),
+                "net.Socket.{method} must use its ordinary prototype method"
+            );
+            let signature = abi
+                .lines()
+                .find_map(|line| {
+                    let columns: Vec<_> = line.split('\t').collect();
+                    (columns.first() == Some(&symbol)).then_some(columns)
+                })
+                .unwrap_or_else(|| panic!("{symbol} must remain available to linked archives"));
             assert_eq!(
-                sig.args,
-                [super::NativeArgKind::F64; 3],
-                "net.Socket.{method} takes three NaN-boxed f64 arguments"
+                signature[2], "i64,f64,f64,f64",
+                "{symbol} takes an ordinary owner address and three NaN-boxed arguments"
             );
         }
     }

crates/perry-codegen/src/lower_call/native_table/bun.rs

@@ -61,7 +61,7 @@ pub(crate) const BUN_ROWS: &[NativeModSig] = &[
         class_filter: None,
         runtime: "js_bun_tcp_listen",
         args: &[NA_F64],
-        ret: NR_HANDLE_ID,
+        ret: NR_GCPTR,
     },
     NativeModSig {
         module: "bun",