Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ CI is a **single `docker build .`** — the multi-stage Dockerfile is the entire

## Release & Publish (`release-please.yml`)

Release Please detects version bumps per component independently. Each component has its own conditional publish job.
Release Please detects version bumps per component independently. Published packages have conditional publish jobs; the Go provider is released by tag only, and internal WASM crates are versioned without separate publish jobs.

### Publish dependency chain

Expand All @@ -20,7 +20,7 @@ The Rust provider (`openfeature-provider/rust`) depends on `confidence-resolver`
3. If only the Rust provider is released (resolver not changed), it skips the wait


All publish jobs require the **`deployment` GitHub environment** (environment protection rules apply).
Package publish jobs require the **`deployment` GitHub environment** (environment protection rules apply). The Cloudflare deployer image jobs publish directly to GHCR and do not use that environment.

### Post-release

Expand Down
29 changes: 20 additions & 9 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,14 +2,16 @@

## Repository Overview

Multi-language workspace implementing feature flag resolution in Rust, compiled to WebAssembly with bindings for JS, Java, Go, Ruby, Rust, and Python.
Multi-language workspace implementing feature flag resolution and event tracking in Rust. The resolver and event engine compile to WebAssembly for JS, Java, Go, and Python; the Rust provider uses the resolver natively, while Ruby resolves remotely over HTTP.

**Repository**: `spotify/confidence-resolver`

### Key Components

- **confidence-resolver/** — Core Rust resolver library (flag evaluation, targeting, bucketing)
- **confidence-event-engine/** — Shared event batching engine
- **wasm/rust-guest/** — WASM guest (compiles the resolver to `wasm32-unknown-unknown`)
- **wasm/event-guest/** — WASM guest for OpenFeature event tracking
- **wasm-msg/** — WASM messaging layer (alloc/free, protobuf-based host↔guest calls)
- **confidence-cloudflare-resolver/** — Cloudflare Worker WASM build
- **openfeature-provider/js/** — TypeScript OpenFeature provider (npm: `@spotify-confidence/openfeature-server-provider-local`)
Expand All @@ -18,7 +20,7 @@ Multi-language workspace implementing feature flag resolution in Rust, compiled
- **openfeature-provider/ruby/** — Ruby OpenFeature provider (**online/remote resolver, NOT WASM**)
- **openfeature-provider/rust/** — Rust OpenFeature provider (**native resolver, no WASM**)
- **openfeature-provider/python/** — Python OpenFeature provider
- **openfeature-provider/proto/** — Shared protobuf definitions used by all providers
- **openfeature-provider/proto/** — Shared protobuf definitions used by the local providers
- **mock-support-server/** — Go mock server for integration/benchmark testing

## Cargo Workspace Gotcha
Expand All @@ -28,13 +30,15 @@ Several workspace members are **dummy Cargo crates** that exist solely for Relea
## WASM Architecture

```
Host (JS/Java/Go/Python/Ruby)
Host (JS/Java/Go/Python)
↓ protobuf message via wasm-msg (alloc → write → call → read → free)
WASM Guest (rust-guest, compiled from confidence-resolver)
↓ returns protobuf response
Host
```

Event tracking in those providers follows the same host/guest pattern through `wasm/event-guest`, backed by `confidence-event-engine`. Ruby does not use either WASM guest, and the Rust provider links the resolver crate directly.

- `wasm-msg` provides memory management (`wasm_msg_alloc`/`wasm_msg_free`) and the `wasm_msg_guest!`/`wasm_msg_host!` macros
- Guest exports are prefixed: `wasm_msg_guest_resolve_flags`, `wasm_msg_guest_set_resolver_state`, etc.
- Host imports are prefixed: `wasm_msg_host_log_message`, `wasm_msg_host_current_time`
Expand Down Expand Up @@ -70,13 +74,19 @@ make sync-wasm-go

This builds the WASM in Docker and copies it to `openfeature-provider/go/confidence/internal/local_resolver/assets/`. The updated `.wasm` file must be committed.

The Go provider also embeds the event engine WASM. After changes to `confidence-event-engine/` or `wasm/event-guest/`, run:

```bash
make sync-wasm-event-go
```

## Protobuf Schema Locations

There are 4 separate proto directories — this is the most common source of confusion:

- **`confidence-resolver/protos/`** — Core resolver protos (flags, admin, resolver API, types, events)
- **`openfeature-provider/proto/`** — Shared provider protos (WASM messages, flag types) — used by JS, Java, Go, Ruby, Python providers
- **`wasm/proto/`** — WASM guest message definitions (`messages.proto`, `types.proto`)
- **`openfeature-provider/proto/`** — Shared provider protos (flags, WASM messages, and events) — used by JS, Java, Go, Python, and the published Rust provider
- **`wasm/proto/`** — Resolver WASM guest message definitions (`messages.proto`, `types.proto`, and resolver API imports)
- **`wasm-msg/proto/`** — Low-level messaging layer protos

## Publishing & Security
Expand All @@ -94,7 +104,7 @@ RUN --mount=type=secret,id=my_secret \
```

- **JS** — Build in Docker (`npm pack`), publish via GitHub Actions OIDC (no npm tokens). Requires npm Trusted Publishers config.
- **Java** — Credentials mounted as Docker secrets. Requires GitHub secrets: `MAVEN_SETTINGS`, `GPG_PRIVATE_KEY`, `SIGN_KEY_PASS`. Uses `central-publishing-maven-plugin` (not nexus-staging).
- **Java** — Credentials mounted as Docker secrets. Requires GitHub secrets: `MAVEN_CENTRAL_USERNAME`, `MAVEN_CENTRAL_PASSWORD`, `GPG_PRIVATE_KEY`, `SIGN_KEY_PASS`. Uses `central-publishing-maven-plugin` (not nexus-staging).
- **Rust** — Published via Docker stages to crates.io.

## Post-Release Checklist
Expand All @@ -109,14 +119,15 @@ After releasing a new version of any SDK from this repo, update internal version
make # lint + test + build everything
make test # run all component tests
make lint # run all linters
make build # build WASM + all provider packages
make build # build both WASM artifacts and provider build targets
make wasm/confidence_resolver.wasm # build WASM artifact only
make sync-wasm-go # build WASM in Docker and sync to Go assets (for committing)
make sync-wasm-event-go # build event WASM in Docker and sync to Go assets
make go-bench # Go benchmark via docker-compose
make js-bench # JS benchmark via docker-compose
```

Each component has its own Makefile with `build`, `test`, `lint`, `clean` targets.
Component Makefiles expose the targets relevant to that component; not every component implements every target.

### Docker

Expand All @@ -130,7 +141,7 @@ docker build --target openfeature-provider-js.test . # run JS tests
docker build --target openfeature-provider-java.build . # build Java provider
```

Stage naming pattern: `<component>.{build,test,test_e2e,lint,artifact,publish}`
Common stage naming pattern: `<component>.{build,test,test_e2e,lint,artifact,publish}`. Available actions vary by component.

## Environment Variables

Expand Down
8 changes: 8 additions & 0 deletions confidence-cloudflare-resolver/CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ A Cloudflare Worker that serves the Confidence flag resolver at the edge. Compil
- **Compile-time state** — Resolver state is embedded at build time via `include_bytes!("../../data/resolver_state_current.pb")`. No runtime state fetching. Redeployment required to update.
- **Optional sticky assignments** — Opt-in via `ENABLE_STICKY_ASSIGNMENTS` (KV-backed).
- **Queue-based log shipping** — Flag logs are serialized to JSON and sent to a Cloudflare Queue (`flag_logs_queue`), then consumed by a queue handler that aggregates and ships them.
- **Resolver telemetry** — Resolve latency/rate telemetry is included in flag-log batches; the queue consumer can accumulate Prometheus snapshots in KV.
- **JSON API** — Request/response bodies are JSON (not protobuf), unlike the WASM-based providers.
- **CORS** — All responses include CORS headers with configurable `ALLOWED_ORIGIN`.

Expand All @@ -22,8 +23,11 @@ A Cloudflare Worker that serves the Confidence flag resolver at the edge. Compil
| POST | `/v1/flags:apply` | Apply flags (JSON body) |
| POST | `/v1/events:publish` | Publish events (JSON body, queued and batched before delivery to backend) |
| GET | `/v1/state:etag` | Returns deployment state ETag and resolver version |
| GET | `/metrics` | Returns KV-backed Prometheus metrics (requires client-secret authorization) |
| OPTIONS | `*` | CORS preflight |

`POST /v1/telemetry:upload` is accepted as a no-op for SDK compatibility.

Note: The Worker router uses `*path` matching because `:name` conflicts with Cloudflare router parameter syntax.

## Queue Consumers
Expand Down Expand Up @@ -53,6 +57,9 @@ Both queues use `max_batch_size=100` and `max_batch_timeout=10s`.
| `MATERIALIZATION_TTL_SECONDS` | TTL for KV-backed sticky assignments (optional) |
| `FORCE_APPLY` | Default `true`: force `apply=true` on every resolve. Set `"false"` to honor the SDK-sent `apply` value (deferred apply) |
| `RESOLVE_TOKEN_ENCRYPTION_KEY` | (secret) AES-128 key for resolve token encryption; auto-generated by deployer |
| `ENABLE_APPLY_DEDUP` | Experimental: deduplicate repeated assignment events; defaults to `false` |

The deployer also accepts `ENABLE_METRICS` to create and bind `CONFIDENCE_METRICS_KV`; it is a deployer setting rather than a Worker environment variable.

## Build & Test

Expand Down Expand Up @@ -91,3 +98,4 @@ The `deployer/` directory contains a deployment script and Dockerfile for automa
3. **JSON, not protobuf** — API uses JSON request/response bodies
4. **Edge deployment** — Runs on Cloudflare's edge network, not in application processes
5. **Queue logging** — Uses Cloudflare Queues instead of gRPC for log transport
6. **Optional Prometheus endpoint** — Queue telemetry is accumulated in KV for `/metrics`
16 changes: 9 additions & 7 deletions openfeature-provider/go/CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,16 +9,17 @@ Go OpenFeature provider using the Confidence resolver compiled to WASM, loaded v
## Architecture

- **WASM runtime**: wazero — pure Go, zero dependencies, no CGo
- **WASM embedding**: The WASM binary is embedded at compile time via `//go:embed` in `confidence/internal/local_resolver/wasm.go`
- **Pool architecture**: Multiple WASM instances run in a pool (`pool.go`) sized to `GOMAXPROCS` by default. Each instance is mutex-protected.
- **WASM embedding**: Resolver and event-engine WASM binaries are embedded at compile time via `//go:embed`
- **Pool architecture**: Resolver WASM instances run in a pool (`pool.go`) with a default size of 2, capped at `GOMAXPROCS`. Each instance is mutex-protected.
- **Crash recovery**: `recover.go` wraps resolvers with automatic WASM instance reload on panic/trap, preserving state and buffering logs through crashes.
- **gRPC**: Flag logs are shipped via gRPC to `edge-grpc.spotify.com` using `InternalFlagLoggerServiceClient`.
- **Destination-aware flag logs**: Resolver state selects gRPC delivery to Spotify Edge or HTTP delivery to the Cloudflare ingestor, with ordered fallback.
- **Event tracking**: OpenFeature `Track` calls are batched by `confidence_event_engine.wasm` and published to the events service over gRPC.

## Key API

- **`NewProvider(ctx, ProviderConfig)`** (`provider_builder.go`) — Main factory function. Creates gRPC connection, state fetcher, flag logger, and wires everything together.
- **`NewProviderForTest(ctx, ProviderTestConfig)`** — Factory with injectable `StateProvider` and `FlagLogger` for testing.
- **`ProviderConfig`** — `ClientSecret`, `Logger`, `TransportHooks`, `MaterializationStore`, `UseRemoteMaterializationStore`, `StatePollInterval`, `LogPollInterval`, `ResolverPoolSize`, `UseWasmInterpreter`
- **`ProviderConfig`** — `ClientSecret`, `EncryptionKey`, `Logger`, `TransportHooks`, `MaterializationStore`, `UseRemoteMaterializationStore`, `StatePollInterval`, `LogPollInterval`, `ResolverPoolSize`, `UseWasmInterpreter`, `EnableApplyDedup`, `DisableExposureCollection`
- **`TransportHooks`** interface — Allows customizing both gRPC and HTTP transports (for proxying or testing): `ModifyGRPCDial(target, opts)` and `WrapHTTP(transport)`.

## Build & Test
Expand All @@ -34,10 +35,11 @@ make proto

The provider starts background goroutines on `Init()`:
1. **State polling** — Fetches resolver state from CDN at `StatePollInterval` (default 10s)
2. **Log flushing** — Flushes resolve + assign logs via gRPC at `LogPollInterval` (default 15s)
2. **Log flushing** — Flushes resolve + assign logs at `LogPollInterval` (default 15s)
3. **Event flushing** — Flushes tracked events at `LogPollInterval`

Both are cancelled via context on `Shutdown()`.
All are cancelled via context on `Shutdown()`; pending events are drained before the event tracker closes.

## WASM Build

WASM is automatically built from source and copied to `confidence/internal/local_resolver/assets/` when building locally (skipped in Docker where it's provided by the build stage).
The resolver WASM is automatically built from source and copied to `confidence/internal/local_resolver/assets/` when building locally (skipped in Docker where it is provided by the build stage). The committed resolver and event-engine binaries are synchronized reproducibly with `make sync-wasm-go` and `make sync-wasm-event-go` from the repository root.
12 changes: 9 additions & 3 deletions openfeature-provider/java/CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,14 +4,20 @@

Maven coordinates: `com.spotify.confidence:openfeature-provider-local`

Java OpenFeature provider using the Confidence resolver compiled to WASM, with Chicory AOT compilation for near-native performance.
Java OpenFeature provider using the Confidence resolver compiled to WASM, with Chicory AOT compilation for near-native flag resolution and a separate event-engine WASM for OpenFeature tracking.

## Key Architecture

- **Chicory WASM AOT** — The WASM binary (`src/main/resources/wasm/confidence_resolver.wasm`) is AOT-compiled to Java bytecode at build time via `chicory-compiler-maven-plugin`. This generates `com.spotify.confidence.sdk.ConfidenceResolverModule`.
- **gRPC** — Used for flag log shipping and materialization store communication. Protobuf + gRPC stubs are generated from `../proto/`.
- **Resolver pool and recovery** — A configurable pool (default 2, capped at available processors) wraps recovering resolver instances.
- **Event tracking** — `confidence_event_engine.wasm` is loaded through Chicory at runtime; tracked events are flushed every 15 seconds and published over gRPC.
- **Transport** — Flag logs use destination-aware gRPC/HTTP delivery. gRPC is also used for event publishing and remote materializations. Protobuf + gRPC stubs are generated from `../proto/`.
- **Shaded JAR** — gRPC, protobuf, and guava are relocated to `com.spotify.confidence.sdk.shaded.*` to avoid version conflicts with consumers.

## Configuration

`LocalProviderConfig.builder()` supports custom channel and HTTP client factories, remote materializations, resolver pool size, an optional AES-256 state encryption key, experimental apply deduplication, and disabling exposure collection.

## Main Provider Class

```java
Expand All @@ -28,7 +34,7 @@ public class OpenFeatureLocalResolveProvider implements FeatureProvider {
## Build & Test

```bash
make build # build WASM (if needed) + mvn package -DskipTests
make build # build both WASM resources (if needed) + mvn package -DskipTests
make test # build + mvn test (excludes *E2ETest)
make test-e2e # build + mvn verify (integration tests against shaded JAR)
```
Expand Down
16 changes: 12 additions & 4 deletions openfeature-provider/js/CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,11 +4,11 @@

npm package: `@spotify-confidence/openfeature-server-provider-local`

TypeScript OpenFeature provider using the Confidence resolver compiled to WASM. Supports multiple WASM loading strategies and React Server Components.
TypeScript OpenFeature provider using resolver and event-engine WASM modules. Supports multiple WASM loading strategies, React Server Components, and Next.js Pages Router helpers.

## Entry Points & Exports

The package has 5 build targets configured in `tsdown.config.ts`:
The package has 8 build targets configured in `tsdown.config.ts`:

| Export Path | Entry File | Platform | WASM Loading |
| ------------------ | ---------------------- | -------- | ------------------------------- |
Expand All @@ -17,10 +17,13 @@ The package has 5 build targets configured in `tsdown.config.ts`:
| `"./fetch"` | `src/index.fetch.ts` | neutral | `fetch()` (Deno, Bun, browsers) |
| `"./react-server"` | `src/react/server.tsx` | neutral | React Server Component |
| `"./react-client"` | `src/react/client.tsx` | neutral | React Client Component |
| `"./pages-router/server"` | `src/pages-router/server.ts` | neutral | Pages Router server helpers |
| `"./pages-router/client"` | `src/pages-router/client.tsx` | neutral | Pages Router client provider |
| `"./pages-router/api"` | `src/pages-router/api.ts` | neutral | Deferred-apply API handler |

Each entry point exports a `createConfidenceServerProvider` factory function.
The default, `./node`, and `./fetch` entry points export `createConfidenceServerProvider`; the React and Pages Router entry points expose framework-specific APIs.

The `./node` entry point extends options with `wasmPath?: string` and `./fetch` with `wasmUrl?: URL | string`.
The default entry point inlines both WASM modules. The `./node` and `./fetch` builds copy both modules alongside the bundle; their `wasmPath`/`wasmUrl` overrides apply to the resolver module.

## ProviderOptions

Expand All @@ -29,14 +32,19 @@ Defined in `src/ConfidenceServerProviderLocal.ts`:
```typescript
interface ProviderOptions {
flagClientSecret: string;
encryptionKey?: string; // hex-encoded AES-256 key for CDN state
initializeTimeout?: number;
stateUpdateInterval?: number; // ms between state polls (default: 30000)
flushInterval?: number; // ms between log flushes (default: 15000)
fetch?: typeof fetch;
materializationStore?: MaterializationStore | 'CONFIDENCE_REMOTE_STORE';
enableApplyDedup?: boolean; // experimental, default false
disableExposureCollection?: boolean;
}
```

OpenFeature `track()` calls are buffered in `confidence_event_engine.wasm` and published over HTTP on the regular flush interval.

## Build & Test

```bash
Expand Down
Loading
Loading