Skip to content

spike: hiqlite-in-Encore.ts NAPI feasibility (Shape A PoC + report)#40

Draft
bartekus wants to merge 3 commits into
mainfrom
spike/hiqlite-napi-feasibility
Draft

spike: hiqlite-in-Encore.ts NAPI feasibility (Shape A PoC + report)#40
bartekus wants to merge 3 commits into
mainfrom
spike/hiqlite-napi-feasibility

Conversation

@bartekus

@bartekus bartekus commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

What

Feasibility spike: can hiqlite be embedded directly into Encore.ts tenant
apps as an in-process native capability via napi-rs (cache / dlock / counters /
listen_notify), rather than a network sidecar? Two shapes were evaluated:

  • Shape A: a standalone napi-rs addon linking hiqlite, imported by the app.
  • Shape B: forking Encore's Rust runtime to add hiqlite as a first-class
    Encore primitive.

This is strategic / forward-looking. There is no live gap (rate-limiting
stays Postgres; there is no Redis in the stack). This branch is a throwaway
spike for review and record
, not a feature intended to merge to main as-is.

Verdict

Neither shape as a near-term build.

  • Reject Shape B outright. No plugin seam exists; a new primitive is a
    4-surface fork (encore-runtime-core + encore-js-runtime + the encore.dev SDK
    • the Go CLI's static-analysis codegen) tracking a weekly-releasing upstream.
  • Hold Shape A in reserve. Technically clean and proven buildable (see
    PoC below), but it is a delivery mechanism for a value case that does not exist
    yet for tenant apps: a shared cross-replica cache requires StatefulSet Raft
    clustering that fights Encore's autoscaled-Deployment model. Absent that, an
    in-process map or Postgres wins.

Full analysis (Q1 through Q6, licensing, costs) in
docs/analysis/hiqlite-napi-feasibility.md.

PoC (executed, green)

experiments/hiqlite-native-spike/ (throwaway): a napi-rs addon linking hiqlite
=0.13.2 (single-node, cache-only, no SQLite-C) exposing put/get, called
from one Encore.ts endpoint.

  • Q2 (two tokio runtimes): CONFIRMED. Under encore run, put/get
    round-trips through Encore into napi into hiqlite. hiqlite's tokio (in the
    addon's own dylib) coexists with Encore's Rust runtime in one process, no
    contention or nested-runtime panic.
  • Q4 (encore build docker bundles + loads the native addon): CONFIRMED.
    A cross-built Linux .node bundled into the image and loaded at runtime; the
    same round-trip works in-container.

Real costs (all in native distribution, not runtime coexistence): cross-build the
addon per target yourself; ship a real npm package with per-platform
optionalDependencies (not a file: symlink); bundle_source over an
npm-workspace root bundles the whole repo (needs a .dockerignore); the
self-host gateway binds :8080. Details and reproduce steps in
experiments/hiqlite-native-spike/RESULTS.md.

Not in this branch

Prebuilt .node binaries, napi-generated loaders, Rust target/ output, and
node_modules are gitignored. RESULTS.md has the exact commands to regenerate
and re-run everything.

bartekus added 3 commits July 5, 2026 18:05
…port)

Evaluate embedding hiqlite directly into Encore.ts tenant apps as an
in-process native capability via napi-rs, rather than a network sidecar.

Verdict: neither shape as a near-term build. Shape B (native Encore
primitive) is a 4-surface fork of a weekly-moving upstream; Shape A
(standalone napi addon) is technically clean and proven buildable but
solves a value problem that does not exist yet for tenant apps (a shared
cross-replica cache would force StatefulSet Raft clustering that fights
Encore's autoscaled-Deployment model). Hold Shape A in reserve.

PoC (experiments/hiqlite-native-spike, throwaway): a napi-rs addon
linking hiqlite (single-node, cache-only) exposing put/get, called from
one Encore.ts endpoint. Confirmed under `encore run` (Q2: hiqlite's tokio
coexists with Encore's Rust runtime) and inside an `encore build docker`
image (Q4: the native .node bundles and loads). Prebuilt binaries and
Rust build output are gitignored; RESULTS.md has the reproduce steps.

Report: docs/analysis/hiqlite-napi-feasibility.md
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant