Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

51 changes: 51 additions & 0 deletions docs/content/docs/general/node-worker-threads.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
---
title: "Node.js Worker Threads"
description: "Run Rivet Actors across isolated Node.js event loops in a long-running Node.js process."
---

Node.js normally runs every actor in a process on one JavaScript event loop. Set `actorsPerThread` to distribute actor JavaScript across worker threads when one actor must not block every other actor in the process.

<CodeSnippet file="examples/docs/general-worker-threads/basic.ts" />

`actorsPerThread` is a hard limit that includes actors which are starting, running, or stopping. Use `1` to give every actor generation its own event loop. Larger values reduce memory usage by allowing several actors to share one event loop.

## Placement and scaling

RivetKit creates baseline threads on demand, up to `os.availableParallelism()`, and spreads the first actors across them. After reaching that baseline, RivetKit fills existing threads before creating overflow threads. There is no configurable maximum thread count.

An actor generation stays on its selected thread for its entire lifetime. RivetKit does not move running actors to compact the pool. An overflow thread becomes eligible to exit after its final actor completely stops, with a short idle delay to prevent thread churn during bursts. A sleeping actor is no longer resident and does not occupy a thread slot.

Requests for an existing actor are delivered directly from the native runtime to that actor's worker thread. They do not pass through the main JavaScript event loop. If the main event loop is blocked, existing actors on other threads can continue running, but RivetKit cannot create or retire threads until the main event loop becomes available.

## Requirements and constraints

Worker threads require the native runtime and a persistent Node.js process. They are not supported with serverless handlers, the wasm runtime, Bun, Deno, `node -e`, stdin, or the Node.js REPL.

RivetKit loads the file named by `process.argv[1]` in every new worker. That entrypoint must build the same actor registry and call `registry.start()` or `registry.startAndWait()` while the module is evaluated. Do not guard the registry startup call with `isMainThread`.

Each worker validates its actor names and runtime configuration against the main registry before it can receive an actor. The waiting actor start fails if the entrypoint cannot load, does not start a registry, loads a different actor configuration, or cannot initialize the native runtime. Worker acquisition times out after 30 seconds; a worker that finishes booting later remains available for the next actor start.

The complete entrypoint module executes independently in every worker. Guard unrelated main-only side effects individually, but do not guard the RivetKit registry startup call. Only one worker-enabled registry may be started from an entrypoint in this first version. Development runners that replace `process.argv[1]` with a launcher or wrapper are not supported unless that file reloads the same registry entrypoint.

## Isolation and memory

Each worker has its own V8 isolate, event loop, module cache, globals, and module-level singletons. JavaScript objects are not shared between workers. Actor definitions are evaluated separately in each worker, while durable actor state continues to use RivetKit state, KV, and SQLite normally.

In one local Node.js 24 Linux benchmark, an idle worker with the RivetKit native addon added approximately 9.2 MiB PSS or 11.0 MiB RSS after the first worker. In a separate run, a minimal live actor added approximately 0.53 MiB PSS or 0.54 MiB RSS. At full occupancy, a lower-bound estimate per actor is `actor memory + thread memory / actorsPerThread`:

| `actorsPerThread` | Approximate PSS per actor | Approximate RSS per actor |
|---:|---:|---:|
| 1 | 9.7 MiB | 11.6 MiB |
| 4 | 2.8 MiB | 3.3 MiB |
| 8 | 1.7 MiB | 1.9 MiB |
| 16 | 1.1 MiB | 1.2 MiB |

These are local lower bounds, not product guarantees. Application modules, actor definitions, state, queues, SQLite, and caches add to them. The table also assumes full threads. For a pool-wide estimate, use `live actors × actor memory + live threads × thread memory`.

Node.js and the system allocator may retain high-water RSS after a worker exits. If one actor blocks its worker, other actors assigned to that worker are also delayed.

## Worker failures

If a worker exits unexpectedly, its actors fail through the normal Rivet lifecycle. The control plane decides whether and where to start replacement generations. RivetKit never moves or locally resurrects the failed generation.

Worker-pool failures use the `actor.worker_*` error codes, including `actor.worker_acquire_timed_out`, `actor.worker_spawn_failed`, and `actor.worker_thread_lost`.
3 changes: 3 additions & 0 deletions docs/content/docs/general/registry-configuration.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,8 @@ After configuring your registry, start it:

See [Runtime Modes](/actors/docs/general/runtime-modes) for details on when to use each mode.

Long-running Node.js registries can also set `actorsPerThread` to isolate actor JavaScript across multiple event loops. See [Node.js Worker Threads](/actors/docs/general/node-worker-threads) for scheduling, entrypoint, memory, and runtime constraints.

## Environment Variables

Many configuration options can be set via environment variables. See [Environment Variables](/actors/docs/general/environment-variables) for a complete reference.
Expand All @@ -55,4 +57,5 @@ Many configuration options can be set via environment variables. See [Environmen
## Related

- [Actor Configuration](/actors/docs/general/actor-configuration): Configure individual actors
- [Node.js Worker Threads](/actors/docs/general/node-worker-threads): Run actors across isolated Node.js event loops
- [HTTP Server Setup](/actors/docs/general/http-server): Set up HTTP routing and middleware
2 changes: 2 additions & 0 deletions docs/content/docs/general/runtime-modes.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,8 @@ Runner is the default mode. Your app runs as a long-running process that opens a
- **No public endpoint**: Your app connects out to Rivet, so it does not need to be publicly reachable or registered in the dashboard.
- **Custom scaling**: You control how runner processes are pooled and scaled.

Runner deployments on Node.js can optionally distribute actor JavaScript across worker threads. See [Node.js Worker Threads](/actors/docs/general/node-worker-threads) for the entrypoint and isolation constraints.

### Example

<CodeSnippet file="examples/docs/general-runtime-modes/runner.ts" title="runner.ts" />
Expand Down
4 changes: 4 additions & 0 deletions docs/sidebar.json
Original file line number Diff line number Diff line change
Expand Up @@ -296,6 +296,10 @@
"title": "WASM vs Native SDK",
"href": "/actors/docs/general/wasm-vs-native-sdk"
},
{
"title": "Node.js Worker Threads",
"href": "/actors/docs/general/node-worker-threads"
},
{
"title": "Registry Configuration",
"href": "/actors/docs/general/registry-configuration"
Expand Down
16 changes: 16 additions & 0 deletions examples/docs/general-worker-threads/basic.ts

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion rivetkit-asyncapi/asyncapi.json
Original file line number Diff line number Diff line change
Expand Up @@ -556,4 +556,4 @@
}
}
}
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
{
"code": "worker_acquire_timed_out",
"group": "actor",
"message": "Timed out waiting for a worker thread"
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
{
"code": "worker_pool_actor_not_registered",
"group": "actor",
"message": "Actor is not registered"
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
{
"code": "worker_pool_closed",
"group": "actor",
"message": "Worker pool is closed"
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
{
"code": "worker_pool_duplicate_assignment",
"group": "actor",
"message": "Actor generation is already assigned"
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
{
"code": "worker_pool_invalid_config",
"group": "actor",
"message": "Invalid worker pool configuration"
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
{
"code": "worker_registration_rejected",
"group": "actor",
"message": "Worker thread registration was rejected"
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
{
"code": "worker_spawn_failed",
"group": "actor",
"message": "Worker thread failed to start"
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
{
"code": "worker_thread_lost",
"group": "actor",
"message": "Actor worker thread exited."
}
14 changes: 14 additions & 0 deletions rivetkit-rust/packages/rivetkit-core/src/actor/config.rs
Original file line number Diff line number Diff line change
@@ -1,8 +1,10 @@
use std::fmt;
use std::fmt::Write as _;
use std::sync::Arc;
use std::time::Duration;

use rivet_envoy_client::config::HttpRequest;
use sha2::{Digest, Sha256};

use crate::inspector::InspectorTabEntry;

Expand Down Expand Up @@ -216,6 +218,18 @@ pub struct ActorConfigInput {
}

impl ActorConfig {
/// Stable within one runtime build and process. Used to reject worker
/// environments that evaluated a different actor configuration before their
/// callback factories become schedulable.
pub fn worker_pool_fingerprint(&self) -> String {
let digest = Sha256::digest(format!("{self:#?}").as_bytes());
let mut encoded = String::with_capacity(digest.len() * 2);
for byte in digest {
let _ = write!(encoded, "{byte:02x}");
}
encoded
}

pub fn from_input(config: ActorConfigInput) -> Self {
let mut actor_config = Self {
name: config.name,
Expand Down
47 changes: 46 additions & 1 deletion rivetkit-rust/packages/rivetkit-core/src/actor/task.rs
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,7 @@ use futures::FutureExt;
use parking_lot::Mutex;
use tokio::sync::{broadcast, mpsc, oneshot};
use tokio::task::{JoinError, JoinHandle};
use tokio_util::sync::CancellationToken;
use tracing::{Instrument, instrument::WithSubscriber};

use crate::actor::action::ActionDispatchError;
Expand Down Expand Up @@ -348,6 +349,8 @@ pub struct ActorTask {
pub lifecycle: LifecycleState,
pub factory: Arc<ActorFactory>,
pub ctx: ActorContext,
/// Cancels the foreign-runtime adapter when its owning environment exits.
runtime_lost: Option<CancellationToken>,

// === STARTUP ===
pub start_input: Option<Vec<u8>>,
Expand Down Expand Up @@ -402,6 +405,31 @@ impl ActorTask {
factory: Arc<ActorFactory>,
ctx: ActorContext,
start_input: Option<Vec<u8>>,
) -> Self {
Self::new_with_runtime_loss(
actor_id,
generation,
lifecycle_inbox,
dispatch_inbox,
lifecycle_events,
factory,
ctx,
start_input,
None,
)
}

#[allow(clippy::too_many_arguments)]
pub fn new_with_runtime_loss(
actor_id: String,
generation: u32,
lifecycle_inbox: mpsc::UnboundedReceiver<LifecycleCommand>,
dispatch_inbox: mpsc::UnboundedReceiver<DispatchCommand>,
lifecycle_events: mpsc::UnboundedReceiver<LifecycleEvent>,
factory: Arc<ActorFactory>,
ctx: ActorContext,
start_input: Option<Vec<u8>>,
runtime_lost: Option<CancellationToken>,
) -> Self {
let (actor_event_tx, actor_event_rx) = mpsc::unbounded_channel();
let (inspector_overlay_tx, _) = broadcast::channel(INSPECTOR_OVERLAY_CHANNEL_CAPACITY);
Expand All @@ -428,6 +456,7 @@ impl ActorTask {
lifecycle: LifecycleState::default(),
factory,
ctx,
runtime_lost,
start_input,
actor_event_tx: Some(actor_event_tx),
actor_event_rx: Some(actor_event_rx),
Expand Down Expand Up @@ -1347,10 +1376,26 @@ impl ActorTask {
startup_ready: startup_ready_tx,
};
let factory = self.factory.clone();
let runtime_lost = self.runtime_lost.clone();
let run_dispatch = tracing::dispatcher::get_default(Clone::clone);
self.run_handle = Some(RuntimeSpawner::spawn(
async move {
match AssertUnwindSafe(factory.start(start)).catch_unwind().await {
let outcome = if let Some(runtime_lost) = runtime_lost {
if runtime_lost.is_cancelled() {
return Err(crate::error::ActorRuntime::WorkerThreadLost.build());
}
let run = AssertUnwindSafe(factory.start(start)).catch_unwind();
tokio::select! {
biased;
_ = runtime_lost.cancelled() => {
return Err(crate::error::ActorRuntime::WorkerThreadLost.build());
}
outcome = run => outcome,
}
} else {
AssertUnwindSafe(factory.start(start)).catch_unwind().await
};
match outcome {
Ok(result) => result,
Err(_) => Err(ActorRuntime::Panicked {
operation: "run handler".to_owned(),
Expand Down
7 changes: 7 additions & 0 deletions rivetkit-rust/packages/rivetkit-core/src/error.rs
Original file line number Diff line number Diff line change
Expand Up @@ -234,6 +234,13 @@ pub enum ActorRuntime {
"Actor task panicked while running {operation}."
)]
Panicked { operation: String },

#[error(
"worker_thread_lost",
"Actor worker thread exited.",
"The Node.js worker thread hosting this actor exited."
)]
WorkerThreadLost,
}

#[derive(RivetError, Debug, Clone, Deserialize, Serialize)]
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -17,10 +17,10 @@ impl EnvoyCallbacks for RegistryCallbacks {
let actor_name = config.name.clone();
let key = actor_key_from_protocol(config.key.clone());
let input = config.input.clone();
let factory = dispatcher.factories.get(&actor_name).cloned();
let actor_config = dispatcher.actor_config(&actor_name).cloned();

Box::pin(async move {
let factory = factory.ok_or_else(|| {
let actor_config = actor_config.ok_or_else(|| {
ActorRuntime::NotRegistered {
actor_name: actor_name.clone(),
}
Expand All @@ -32,7 +32,7 @@ impl EnvoyCallbacks for RegistryCallbacks {
generation,
&actor_name,
key,
factory.as_ref(),
&actor_config,
)?;

dispatcher
Expand Down
Loading
Loading