Skip to content

node:sqlite on native payloads; cheaper native callback path; statements own their sqlite3_stmt #12109

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
proggeramlug merged 11 commits into from
Oct 6, 2026
Merged

node:sqlite on native payloads; cheaper native callback path; statements own their sqlite3_stmt #12109

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
9ca6e80
refactor(sqlite): node:sqlite objects own native payloads; SQLite cal…
Oct 6, 2026
57ef5c2
perf(runtime): read native callbacks through a traced cell slot
Oct 6, 2026
47ec986
perf(runtime): reuse catch savepoints within native callback calls
Oct 6, 2026
2f15ad0
test(runtime): preserve realm caches in callback tracing witness
Oct 6, 2026
5f2317d
perf(runtime): one owner check and a direct body call per native call…
Oct 6, 2026
e9391fd
build(sqlite): bundled SQLite with SQLITE_DEFAULT_MEMSTATUS=0, as nod…
Oct 6, 2026
319dab4
test(sqlite): owned-statement gap test; catch refresh witness cannot …
Oct 6, 2026
d782ea3
Merge branch 'sqland-src' into sqland
Oct 6, 2026
c13003d
sqlite: build blob results through the Buffer byte API (B1), ratchet …
Oct 6, 2026
74d75d7
Merge origin/main into sqlite-native-payload
Oct 6, 2026
f2775f4
sqlite: import JSValue through perry_runtime in the changeset byte read
Oct 6, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions .cargo/config.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,3 +48,10 @@ rustflags = ["-C", "force-unwind-tables=yes", "-Z", "use-sync-unwind=yes"]
# rather than silent — but it is still a mistake).
[target.'cfg(target_arch = "x86_64")']
rustflags = ["-C", "force-unwind-tables=yes", "-Z", "use-sync-unwind=yes", "-C", "force-frame-pointers=yes"]

# The bundled SQLite is built as node's is in this one respect: node compiles
# it with SQLITE_DEFAULT_MEMSTATUS=0 (`PRAGMA compile_options` on node 26),
# so every sqlite3_malloc skips the global memory-statistics mutex and
# counters. Nothing in perry reads SQLite's memory statistics.
[env]
LIBSQLITE3_FLAGS = "-DSQLITE_DEFAULT_MEMSTATUS=0"
1 change: 1 addition & 0 deletions changelog.d/PENDING-11919-cb-sqlite.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
node:sqlite `DatabaseSync`, `StatementSync`, `StatementSyncIterator`, `SQLTagStore`, `Session` and `db.limits` are now ordinary JavaScript objects that own GC-finalized native payloads, with node's own accessors, prototypes, `instanceof`, `Object.keys` and `JSON.stringify`. SQLite now calls user-defined functions, aggregates and the authorizer back through the database's traced owner edge instead of id registries and root scanners: a throw inside a callback is rethrown as the exact value after SQLite returns (never unwound through SQLite frames), the first throw wins, a callback may re-enter the same database (nested queries, `exec` from a UDF) without deadlocking, and `close()` inside a running callback is deferred until the outer call returns, which then reports the closed database. `close()` / `open()` keep the same object; statements of an earlier open report "statement has been finalized". A database whose UDF captures it is collected without `close()`. Fixed alongside: `%IteratorPrototype%` carries `Symbol.toStringTag` "Iterator", and a native payload's JS state may hold more fields than its inline slots.
5 changes: 5 additions & 0 deletions changelog.d/PENDING-11919-native-callback-path.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
Reduce native payload callback overhead by reading the callback array through a traced slot on the stable cell, reusing an owner link across statement steps, and avoiding owner rooting on a normal native-call completion. Keep callback exception capture, the first pending throw, thread checks, deferred close, and child OpenSerial checks. Callback-array replacement and clearing synchronize the cell edge through the existing state setters. New runtime witnesses cover slot relocation, marking, growth, replacement, clearing, and its exact malloc-parent write barrier, including child-process sabotage.

Callback families alone carry a traced array slot and a native catch-token pointer after the unchanged legacy header. NativeCallGuard::call lazily captures one savepoint for a native call, refreshes root depths per trampoline, and re-arms a trampoline-local jump target for every callback. Existing enter/finish callers retain ordinary per-callback capture. Prepare exposes its validated owner link without another receiver lookup. New sabotage witnesses cover direct data-only array reads, forwarding, cached root refresh, new.target relocation, nested token restoration and catch-frame cleanup.

A callback refreshes only its runtime handle depth (trampolines add only handle scopes), calls a compiler-emitted body directly, checks its owner once (`call_callback`: open, this thread, nothing pending) and reads the slot on an inlined hot path; the scalar trampoline converts allocation-free arguments in one pass. Payload method dispatch drops two redundant descriptor scans (an ordinary object's keys carry its attributes). Payload instances are born in their call site's recorded final shape (own keys, JS-state key, own accessors) holding the family's shared accessor pairs, and their JS state in its recorded shape. node:sqlite statements own the `sqlite3_stmt` their `prepare()` compiled, as node's do: the authorizer runs once per `prepare()` (it ran again on every run), `expandedSQL` and `columns()` read the compiled statement, and the connection's close or the statement's drop finalizes it.
56 changes: 8 additions & 48 deletions crates/perry-codegen/src/gc_effects/linux-x86_64.tsv
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -505,12 +505,20 @@ js_bun_plugin Reenters
js_bun_spawn Reenters
js_bun_sql_new Reenters
js_bun_sqlite_database_call Reenters
js_bun_sqlite_database_close Reenters
js_bun_sqlite_database_filename Reenters
js_bun_sqlite_database_load_extension Reenters
js_bun_sqlite_database_new Reenters
js_bun_sqlite_database_query Reenters
js_bun_sqlite_database_run Reenters
js_bun_sqlite_database_serialize Reenters
js_bun_sqlite_database_transaction Reenters
js_bun_sqlite_is_database_handle Reenters
js_bun_sqlite_is_statement_handle Reenters
js_bun_sqlite_statement_all Reenters
js_bun_sqlite_statement_finalize Reenters
js_bun_sqlite_statement_get Reenters
js_bun_sqlite_statement_run Reenters
js_bun_sqlite_statement_safe_integers Reenters
js_bun_sqlite_statement_values Reenters
js_bun_stderr Reenters
Expand Down Expand Up @@ -1857,61 +1865,13 @@ js_node_inspector_url Reenters
js_node_inspector_wait_for_debugger Reenters
js_node_setheaders_entries_json Reenters
js_node_sqlite_backup Reenters
js_node_sqlite_database_sync_aggregate Reenters
js_node_sqlite_database_sync_apply_changeset Reenters
js_node_sqlite_database_sync_call Reenters
js_node_sqlite_database_sync_close Reenters
js_node_sqlite_database_sync_create_session Reenters
js_node_sqlite_database_sync_create_tag_store Reenters
js_node_sqlite_database_sync_deserialize Reenters
js_node_sqlite_database_sync_dispose Reenters
js_node_sqlite_database_sync_enable_defensive Reenters
js_node_sqlite_database_sync_enable_load_extension Reenters
js_node_sqlite_database_sync_exec Reenters
js_node_sqlite_database_sync_function Reenters
js_node_sqlite_database_sync_is_open Reenters
js_node_sqlite_database_sync_is_transaction Reenters
js_node_sqlite_database_sync_limits Reenters
js_node_sqlite_database_sync_load_extension Reenters
js_node_sqlite_database_sync_location Reenters
js_node_sqlite_database_sync_new Reenters
js_node_sqlite_database_sync_open Reenters
js_node_sqlite_database_sync_prepare Reenters
js_node_sqlite_database_sync_serialize Reenters
js_node_sqlite_database_sync_set_authorizer Reenters
js_node_sqlite_is_database_sync_handle Reenters
js_node_sqlite_is_limits_handle Reenters
js_node_sqlite_is_session_handle Reenters
js_node_sqlite_is_statement_sync_handle Reenters
js_node_sqlite_is_tag_store_handle Reenters
js_node_sqlite_native_dispatch Reenters
js_node_sqlite_session_call Reenters
js_node_sqlite_session_changeset Reenters
js_node_sqlite_session_close Reenters
js_node_sqlite_session_dispose Reenters
js_node_sqlite_session_new Reenters
js_node_sqlite_session_patchset Reenters
js_node_sqlite_sql_tag_store_all Reenters
js_node_sqlite_sql_tag_store_capacity Reenters
js_node_sqlite_sql_tag_store_clear Reenters
js_node_sqlite_sql_tag_store_db Reenters
js_node_sqlite_sql_tag_store_get Reenters
js_node_sqlite_sql_tag_store_iterate Reenters
js_node_sqlite_sql_tag_store_run Reenters
js_node_sqlite_sql_tag_store_size Reenters
js_node_sqlite_statement_sync_all Reenters
js_node_sqlite_statement_sync_call Reenters
js_node_sqlite_statement_sync_columns Reenters
js_node_sqlite_statement_sync_expanded_sql Reenters
js_node_sqlite_statement_sync_get Reenters
js_node_sqlite_statement_sync_iterate Reenters
js_node_sqlite_statement_sync_new Reenters
js_node_sqlite_statement_sync_run Reenters
js_node_sqlite_statement_sync_set_allow_bare_named_parameters Reenters
js_node_sqlite_statement_sync_set_allow_unknown_named_parameters Reenters
js_node_sqlite_statement_sync_set_read_bigints Reenters
js_node_sqlite_statement_sync_set_return_arrays Reenters
js_node_sqlite_statement_sync_source_sql Reenters
js_node_stream_add_abort_signal Reenters
js_node_stream_compose Reenters
js_node_stream_duplex_from_options Reenters
Expand Down
56 changes: 8 additions & 48 deletions crates/perry-codegen/src/gc_effects/macos-aarch64.tsv
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -505,12 +505,20 @@ js_bun_plugin Reenters
js_bun_spawn Reenters
js_bun_sql_new Reenters
js_bun_sqlite_database_call ThrowOnly
js_bun_sqlite_database_close Reenters
js_bun_sqlite_database_filename Reenters
js_bun_sqlite_database_load_extension Reenters
js_bun_sqlite_database_new Reenters
js_bun_sqlite_database_query Reenters
js_bun_sqlite_database_run Reenters
js_bun_sqlite_database_serialize Reenters
js_bun_sqlite_database_transaction ThrowOnly
js_bun_sqlite_is_database_handle Reenters
js_bun_sqlite_is_statement_handle Reenters
js_bun_sqlite_statement_all Reenters
js_bun_sqlite_statement_finalize Reenters
js_bun_sqlite_statement_get Reenters
js_bun_sqlite_statement_run Reenters
js_bun_sqlite_statement_safe_integers Reenters
js_bun_sqlite_statement_values Reenters
js_bun_stderr Reenters
Expand Down Expand Up @@ -1857,61 +1865,13 @@ js_node_inspector_url AllocOnly
js_node_inspector_wait_for_debugger ThrowOnly
js_node_setheaders_entries_json Reenters
js_node_sqlite_backup Reenters
js_node_sqlite_database_sync_aggregate Reenters
js_node_sqlite_database_sync_apply_changeset Reenters
js_node_sqlite_database_sync_call ThrowOnly
js_node_sqlite_database_sync_close Reenters
js_node_sqlite_database_sync_create_session Reenters
js_node_sqlite_database_sync_create_tag_store Reenters
js_node_sqlite_database_sync_deserialize Reenters
js_node_sqlite_database_sync_dispose Reenters
js_node_sqlite_database_sync_enable_defensive Reenters
js_node_sqlite_database_sync_enable_load_extension Reenters
js_node_sqlite_database_sync_exec Reenters
js_node_sqlite_database_sync_function Reenters
js_node_sqlite_database_sync_is_open Reenters
js_node_sqlite_database_sync_is_transaction Reenters
js_node_sqlite_database_sync_limits Reenters
js_node_sqlite_database_sync_load_extension Reenters
js_node_sqlite_database_sync_location Reenters
js_node_sqlite_database_sync_new Reenters
js_node_sqlite_database_sync_open Reenters
js_node_sqlite_database_sync_prepare Reenters
js_node_sqlite_database_sync_serialize Reenters
js_node_sqlite_database_sync_set_authorizer Reenters
js_node_sqlite_is_database_sync_handle Reenters
js_node_sqlite_is_limits_handle Reenters
js_node_sqlite_is_session_handle Reenters
js_node_sqlite_is_statement_sync_handle Reenters
js_node_sqlite_is_tag_store_handle Reenters
js_node_sqlite_native_dispatch Reenters
js_node_sqlite_session_call ThrowOnly
js_node_sqlite_session_changeset Reenters
js_node_sqlite_session_close Reenters
js_node_sqlite_session_dispose Reenters
js_node_sqlite_session_new ThrowOnly
js_node_sqlite_session_patchset Reenters
js_node_sqlite_sql_tag_store_all Reenters
js_node_sqlite_sql_tag_store_capacity Reenters
js_node_sqlite_sql_tag_store_clear Reenters
js_node_sqlite_sql_tag_store_db Reenters
js_node_sqlite_sql_tag_store_get Reenters
js_node_sqlite_sql_tag_store_iterate Reenters
js_node_sqlite_sql_tag_store_run Reenters
js_node_sqlite_sql_tag_store_size Reenters
js_node_sqlite_statement_sync_all Reenters
js_node_sqlite_statement_sync_call ThrowOnly
js_node_sqlite_statement_sync_columns Reenters
js_node_sqlite_statement_sync_expanded_sql Reenters
js_node_sqlite_statement_sync_get Reenters
js_node_sqlite_statement_sync_iterate Reenters
js_node_sqlite_statement_sync_new ThrowOnly
js_node_sqlite_statement_sync_run Reenters
js_node_sqlite_statement_sync_set_allow_bare_named_parameters Reenters
js_node_sqlite_statement_sync_set_allow_unknown_named_parameters Reenters
js_node_sqlite_statement_sync_set_read_bigints Reenters
js_node_sqlite_statement_sync_set_return_arrays Reenters
js_node_sqlite_statement_sync_source_sql Reenters
js_node_stream_add_abort_signal Reenters
js_node_stream_compose Reenters
js_node_stream_duplex_from_options Reenters
Expand Down
56 changes: 8 additions & 48 deletions crates/perry-codegen/src/gc_effects/windows-x86_64.tsv
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -505,12 +505,20 @@ js_bun_plugin Reenters
js_bun_spawn Reenters
js_bun_sql_new Reenters
js_bun_sqlite_database_call Reenters
js_bun_sqlite_database_close Reenters
js_bun_sqlite_database_filename Reenters
js_bun_sqlite_database_load_extension Reenters
js_bun_sqlite_database_new Reenters
js_bun_sqlite_database_query Reenters
js_bun_sqlite_database_run Reenters
js_bun_sqlite_database_serialize Reenters
js_bun_sqlite_database_transaction Reenters
js_bun_sqlite_is_database_handle Reenters
js_bun_sqlite_is_statement_handle Reenters
js_bun_sqlite_statement_all Reenters
js_bun_sqlite_statement_finalize Reenters
js_bun_sqlite_statement_get Reenters
js_bun_sqlite_statement_run Reenters
js_bun_sqlite_statement_safe_integers Reenters
js_bun_sqlite_statement_values Reenters
js_bun_stderr Reenters
Expand Down Expand Up @@ -1857,61 +1865,13 @@ js_node_inspector_url Reenters
js_node_inspector_wait_for_debugger Reenters
js_node_setheaders_entries_json Reenters
js_node_sqlite_backup Reenters
js_node_sqlite_database_sync_aggregate Reenters
js_node_sqlite_database_sync_apply_changeset Reenters
js_node_sqlite_database_sync_call Reenters
js_node_sqlite_database_sync_close Reenters
js_node_sqlite_database_sync_create_session Reenters
js_node_sqlite_database_sync_create_tag_store Reenters
js_node_sqlite_database_sync_deserialize Reenters
js_node_sqlite_database_sync_dispose Reenters
js_node_sqlite_database_sync_enable_defensive Reenters
js_node_sqlite_database_sync_enable_load_extension Reenters
js_node_sqlite_database_sync_exec Reenters
js_node_sqlite_database_sync_function Reenters
js_node_sqlite_database_sync_is_open Reenters
js_node_sqlite_database_sync_is_transaction Reenters
js_node_sqlite_database_sync_limits Reenters
js_node_sqlite_database_sync_load_extension Reenters
js_node_sqlite_database_sync_location Reenters
js_node_sqlite_database_sync_new Reenters
js_node_sqlite_database_sync_open Reenters
js_node_sqlite_database_sync_prepare Reenters
js_node_sqlite_database_sync_serialize Reenters
js_node_sqlite_database_sync_set_authorizer Reenters
js_node_sqlite_is_database_sync_handle Reenters
js_node_sqlite_is_limits_handle Reenters
js_node_sqlite_is_session_handle Reenters
js_node_sqlite_is_statement_sync_handle Reenters
js_node_sqlite_is_tag_store_handle Reenters
js_node_sqlite_native_dispatch Reenters
js_node_sqlite_session_call Reenters
js_node_sqlite_session_changeset Reenters
js_node_sqlite_session_close Reenters
js_node_sqlite_session_dispose Reenters
js_node_sqlite_session_new Reenters
js_node_sqlite_session_patchset Reenters
js_node_sqlite_sql_tag_store_all Reenters
js_node_sqlite_sql_tag_store_capacity Reenters
js_node_sqlite_sql_tag_store_clear Reenters
js_node_sqlite_sql_tag_store_db Reenters
js_node_sqlite_sql_tag_store_get Reenters
js_node_sqlite_sql_tag_store_iterate Reenters
js_node_sqlite_sql_tag_store_run Reenters
js_node_sqlite_sql_tag_store_size Reenters
js_node_sqlite_statement_sync_all Reenters
js_node_sqlite_statement_sync_call Reenters
js_node_sqlite_statement_sync_columns Reenters
js_node_sqlite_statement_sync_expanded_sql Reenters
js_node_sqlite_statement_sync_get Reenters
js_node_sqlite_statement_sync_iterate Reenters
js_node_sqlite_statement_sync_new Reenters
js_node_sqlite_statement_sync_run Reenters
js_node_sqlite_statement_sync_set_allow_bare_named_parameters Reenters
js_node_sqlite_statement_sync_set_allow_unknown_named_parameters Reenters
js_node_sqlite_statement_sync_set_read_bigints Reenters
js_node_sqlite_statement_sync_set_return_arrays Reenters
js_node_sqlite_statement_sync_source_sql Reenters
js_node_stream_add_abort_signal Reenters
js_node_stream_compose Reenters
js_node_stream_duplex_from_options Reenters
Expand Down
22 changes: 10 additions & 12 deletions crates/perry-codegen/src/lower_call/builtin.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -759,6 +759,7 @@ pub(super) fn lower_builtin_new<'a>(
}
// node:sqlite DatabaseSync — keep full NaN-boxed values for path and
// options so the runtime can preserve Node-shaped validation errors.
// The result is an ordinary object (#11919).
"DatabaseSync" => {
// #6986: `path_value` was held in a bare SSA register across
// `options_value`'s lowering (an arbitrary user expression).
Expand All @@ -774,12 +775,11 @@ pub(super) fn lower_builtin_new<'a>(
None => undef(),
};
let blk = ctx.block();
let handle = blk.call(
I64,
Ok(Some(blk.call(
DOUBLE,
"js_node_sqlite_database_sync_new",
&[(DOUBLE, &path_value), (DOUBLE, &options_value)],
);
Ok(Some(nanbox_pointer_inline(blk, &handle)))
)))
}
"StatementSync" => {
// #6986: `arg0` was held in a bare SSA register across `arg1`'s
Expand All @@ -796,12 +796,11 @@ pub(super) fn lower_builtin_new<'a>(
None => undef(),
};
let blk = ctx.block();
let handle = blk.call(
I64,
Ok(Some(blk.call(
DOUBLE,
"js_node_sqlite_statement_sync_new",
&[(DOUBLE, &arg0), (DOUBLE, &arg1)],
);
Ok(Some(nanbox_pointer_inline(blk, &handle)))
)))
}
"Session" => {
// #6986: `arg0` was held in a bare SSA register across `arg1`'s
Expand All @@ -818,12 +817,11 @@ pub(super) fn lower_builtin_new<'a>(
None => undef(),
};
let blk = ctx.block();
let handle = blk.call(
I64,
Ok(Some(blk.call(
DOUBLE,
"js_node_sqlite_session_new",
&[(DOUBLE, &arg0), (DOUBLE, &arg1)],
);
Ok(Some(nanbox_pointer_inline(blk, &handle)))
)))
}
// async_hooks.AsyncLocalStorage — `new AsyncLocalStorage()` produces a
// real handle so `.run(store, cb)` / `.getStore()` / `.enterWith(store)`
Expand Down
Loading
Loading