Skip to content

fix(core): subscription leak — repository() registers a new usage stream on every call - #268

Merged
rvowles merged 2 commits into
featurehub-io:mainfrom
o-farooq:fix/usage-adapter-per-repository
Sep 5, 2026
Merged

rvowles merged 2 commits into
featurehub-io:mainfrom
o-farooq:fix/usage-adapter-per-repository

Conversation

@o-farooq

@o-farooq o-farooq commented Sep 5, 2026 •

Copy link
Copy Markdown
Contributor

EdgeFeatureHubConfig.repository() builds a new UsageAdapter on every call, and each adapter registers a usage stream on the repository that nothing ever removes. Because newContext() calls repository() internally, an application that builds a context per request accumulates one usage stream per request for the life of the process.

Every feature evaluation then fans out over the whole accumulated list (recordUsageEvent → _usageStreams.forEach), with UsageAdapter.process scheduling a microtask per plugin per event. Evaluation cost is therefore O(contexts ever created): per-request cost grows linearly with requests served, and cumulative work is quadratic.

Affects 2.0.0, 2.0.1 and 2.0.2. Introduced in 5c4cee5 ("Version 2.0 of SDK", #226), which added usage tracking and placed the adapter construction inside what had been a pure getter.

This is a subscription leak, not a memory leak. Each leaked entry retains about 481 bytes — roughly 6 MB after 14,000 requests, measured — so it never shows up as memory pressure. The damage is that every leaked entry is a live callback sitting on the feature-evaluation hot path.


1. What this did to us in production

A Next.js SSR storefront on Node 24 using featurehub-javascript-node-sdk 2.0.1, resolving ~17 flags per request through a context built per request.

Pods got slower for every hour they stayed alive. Pods serving identical request counts in the same hour, differing only in age:

Pod started Requests that hour CPU SSR p95
~17:00 3,391 1,141 mc 13.3 s
~17:00 3,412 1,006 mc 9.4 s
~18:00 3,389 577 mc 2.4 s
~19:00 3,445 388 mc 1.2 s
~19:00 3,370 321 mc 0.7 s

A three-hour-old pod burned 3.7× the CPU of a one-hour-old one for the same work, with 19× the p95.

Once a pod passed roughly one core of JS work the event loop stopped keeping up and every route stalled together — including a static health endpoint that does no work at all, so Kubernetes liveness probes began failing on processes that were perfectly healthy. CPU throttling was zero throughout; the single JS thread was the limit.

Reproduced locally on the same container image at constant load: CPU per request rose from 121 ms to 214 ms in 20 minutes, with _usageStreams.size tracking the request count exactly (851 requests → 934 streams). CPU profiles attributed 34 % of busy CPU to the SDK on a degraded process versus 1.9 % on a freshly started one, almost all of it inside the _usageStreams.forEach callback.

It was hard to find precisely because it looks like nothing: no error, no memory growth worth noticing, and a fresh process profiles completely clean.

2. Reproducing it in your own example

examples/todo-backend-typescript, unmodified. No FeatureHub server or API key needed — local YAML mode is enough.

pnpm install
pnpm -r --filter=./packages/* run build
pnpm -r --filter=./plugins/** run build
cd examples/todo-backend-typescript && pnpm run build

FEATUREHUB_LOCAL_YAML=./flags.yaml node dist/app.cjs

Seed ~10 todos, then time GET /todo/:user repeatedly. Nothing instrumented, no SDK internals touched — just HTTP requests and a stopwatch:

Requests served Median p95 Slowdown With this PR
250 1.1 ms 1.7 ms 1.0× 0.5 ms
1,000 3.2 ms 4.1 ms 3.0× 0.3 ms
2,000 12.6 ms 14.6 ms 11.9× 0.3 ms
3,000 10.9 ms 19.0 ms 10.2× 0.3 ms

Restart the server and it starts from the beginning again, climbing identically. With this PR applied the same example stays flat.

The example is affected because getTodos calls ctx() per request, which does fhConfig.newContext(...).build(), and processTitle then evaluates several features per todo — one leaked stream per request, ~50 evaluations per request, each fanning out over every stream leaked so far.

Note that newContext(repository, edgeService) does repository = repository || this.repository(), so passing null explicitly — as the example does — does not avoid it.

3. Six lines that show it directly

No server, no API key, no example app — the SDK's own no-op mode:

import { ClientFeatureRepository, EdgeFeatureHubConfig } from "featurehub-javascript-node-sdk";

const config = new EdgeFeatureHubConfig();
config.isClientEvaluated = true;
const repository = new ClientFeatureRepository();
config.repository(repository);

for (let i = 0; i < 1000; i++) config.newContext();

console.log("usage streams after 1000 contexts:", repository._usageStreams.size);
2.0.2 today   usage streams after 1000 contexts: 1001
with this PR  usage streams after 1000 contexts: 1

4. The fix

Create the usage adapter once per repository instead of once per call.

This PR tracks which repository the adapter was built for. If a caller replaces the repository through repository(repo), the previous adapter is closed, which releases its stream — nothing in the repo does that today, but it is public API, and without it the adapter would keep listening to a discarded repository.

-    this._usageAdapter = new UsageAdapter(this._repository);
-    this._usageAdapter.registerPlugin(new PassiveRestUsagePlugin(this));
+    if (this._usageAdapterRepository !== this._repository) {
+      this._usageAdapter?.close();
+      this._usageAdapter = new UsageAdapter(this._repository);
+      this._usageAdapter.registerPlugin(new PassiveRestUsagePlugin(this));
+      this._usageAdapterRepository = this._repository;
+    }

Usage tracking is unaffected

Since the change removes adapters, I checked the feature still behaves identically. The same script was run against patched and unpatched builds, recording every event a registered UsagePlugin receives, across three registration orderings — plugin registered after the repository, before it exists (the ordering your example uses), and two plugins registered either side of a context. Output was byte-for-byte identical: same readyness, feature and named-collection events, same userKey, same records.

Test plan

Three new tests in packages/core/src/__tests__/usage_adapter_reuse.test.ts. All three fail without the source change — expected 101 to be 1 — verified by reverting it and re-running.

  • pnpm run lint — clean
  • pnpm run typecheck — clean
  • core test suite — 91 passed (88 existing + 3 new)
  • examples/todo-backend-typescript — flat over 3,000 requests
  • Docker-based integration suites — not run

Two questions for you

  • The new test reads the private _usageStreams map through a narrow cast rather than widening the public API. Happy to add a usageStreamCount accessor to ClientFeatureRepository instead if you'd prefer that.
  • Closing the previous adapter also calls close() on any plugins registered through addUsagePlugin(). That seemed right for an adapter being torn down, but if you'd rather carry plugins across a repository swap, that's a small change.

Unrelated, noticed while tracing this

Happy to raise these separately if useful:

  • FeatureStateBaseHolder.untrackedValue is internalGetValue(true) — identical to value — so it records usage despite its name. It matters here because it's the only obvious way to read a feature without generating a usage event.
  • repository().simpleFeatures() is documented as the way to "get a list of all feature keys", but it materialises every value through the tracked flag/str/num/rawJson getters, firing a usage event per feature, just to return keys. featureKeys / serverProvidedFeatureKeys do the same job without side effects.

EdgeFeatureHubConfig.repository() built a new UsageAdapter on every call, and
each adapter registers a usage stream on the repository that nothing removes.
Because newContext() calls repository(), an application building a context per
request accumulated one usage stream per request for the life of the process.

Every feature evaluation then fanned out over the whole accumulated list
(recordUsageEvent -> _usageStreams.forEach), with UsageAdapter.process
scheduling a microtask per plugin per event, so evaluation cost grew as
O(contexts ever created).

Track which repository the adapter was built for so it is created once per
repository. If a caller replaces the repository via repository(repo), the
previous adapter is closed, releasing its stream.

repository() was a pure getter until 5c4cee5 ("Version 2.0 of SDK", featurehub-io#226),
which added usage tracking and placed the adapter construction inside it.
@changeset-bot

changeset-bot Bot commented Sep 5, 2026 •

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 1f0d07b

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 3 packages
Name Type
featurehub-javascript-core-sdk Patch
featurehub-javascript-client-sdk Patch
featurehub-javascript-node-sdk Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@o-farooq

o-farooq commented Sep 5, 2026

Copy link
Copy Markdown
Contributor Author

@rvowles , really appreciate it if you could have a look at this at the earliest, as it's causing a significantly degraded user experience for our customers in production.

@rvowles rvowles left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmed, the usage adapter is getting recreated on every new context.

@o-farooq
o-farooq deployed to requires-approval September 5, 2026 05:23 — with GitHub Actions Active
@rvowles
rvowles merged commit e193e87 into featurehub-io:main Sep 5, 2026
3 of 4 checks passed

This branch was successfully deployed

1 active deployment
requires-approval — 1f0d07b9 Deployed Sep 5, 2026 by o-farooq via manual_approval_job #43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants