Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions .github/actions/deploy-setup/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,12 +2,12 @@ name: deploy setup
description: >-
Everything an alchemy deploy job needs after checkout: the toolchain,
Infisical secrets, AWS OIDC credentials, dependencies, and (optionally)
the prebuilt ingest binary. Shared by deploy-prd, deploy-stg,
deploy-pr-preview and aws-probe so the order is decided once.
the prebuilt ingest binary. Shared by deploy-prd, deploy-pr-preview and
aws-probe so the order is decided once.

inputs:
infisical-env-slug:
description: Infisical environment to export (prod, staging, dev).
description: Infisical environment to export (prod, dev).
required: true
infisical-identity-id:
description: The Infisical machine identity (a repo secret; composites cannot read secrets).
Expand Down
8 changes: 5 additions & 3 deletions .github/workflows/aws-probe.yml
Original file line number Diff line number Diff line change
Expand Up @@ -58,9 +58,11 @@ jobs:
# so a slow build shows up as a slow build rather than masquerading as
# the hang being investigated.
timeout-minutes: 20
# The OIDC role's trust policy only accepts
# `repo:MapleTechLabs/maple:environment:{production,staging}` subs, so the
# job has to declare one of them to get credentials at all.
# The OIDC role's trust policy accepts the
# `repo:MapleTechLabs/maple:environment:production` sub (it was also
# written for a `staging` subject, a GitHub Environment that no longer
# exists), so the job has to declare `production` to get credentials at
# all.
environment: production
env:
PROBE_LEVEL: ${{ inputs.level }}
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/build-ingest-binary.yml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
name: Build ingest binary

# Compiles `apps/ingest` for the deploy image and hands it to the caller as an
# artifact. Called by deploy-prd / deploy-stg / deploy-pr-preview / aws-probe,
# artifact. Called by deploy-prd / deploy-pr-preview / aws-probe,
# which all COPY the result via `apps/ingest/Dockerfile.prebuilt`.
#
# It is a separate job purely because of the RUNNER. The ECS tasks run on
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/deploy-pr-preview.yml
Original file line number Diff line number Diff line change
Expand Up @@ -114,8 +114,8 @@ jobs:

# ⚠️ The deploy role's trust policy must accept this job's OIDC
# subject (`repo:<org>/<repo>:environment:pr-preview`). It was
# written for the production and staging subjects only; previews get
# AccessDenied until it is widened.
# written for the production subject only; previews get AccessDenied
# until it is widened.
- name: Deploy setup
id: setup
uses: ./.github/actions/deploy-setup
Expand Down
103 changes: 0 additions & 103 deletions .github/workflows/deploy-stg.yml

This file was deleted.

6 changes: 1 addition & 5 deletions .github/workflows/tinybird-cd.yml
Original file line number Diff line number Diff line change
Expand Up @@ -30,16 +30,12 @@ jobs:
# One leg per long-lived Tinybird workspace. Each GitHub Environment supplies
# its own TINYBIRD_HOST/TINYBIRD_TOKEN, so the matching workspace admin token
# scopes each deploy to that workspace — no `--branch`/`__tb__` params needed.
# Staging is a separate workspace (Tinybird's documented long-lived pattern;
# branches are ephemeral and share prod compute, so they're unsuitable here).
strategy:
fail-fast: false
matrix:
target:
- label: production
environment: tinybird-cd
- label: staging
environment: tinybird-cd-stg
# The us-east-1 workspace being migrated to. Both production
# legs must receive every schema deploy for the whole
# dual-emit window — a month of drift between the two
Expand All @@ -61,7 +57,7 @@ jobs:
ALLOW_DESTRUCTIVE: ${{ github.event.inputs.allow_destructive }}
run: |
set -euo pipefail
# Graceful no-op until an environment is provisioned: lets the staging
# Graceful no-op until an environment is provisioned: lets a new
# leg stay green before its workspace/secrets exist, without ever
# affecting the production leg (whose token is always set).
if [ -z "${TINYBIRD_TOKEN:-}" ]; then
Expand Down
8 changes: 4 additions & 4 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -148,8 +148,8 @@ isolated database and exercises run/compare/inspect against real Maple builders.
## Application database (PlanetScale Postgres)

Relational state (issues, alert rules, dashboards, org config, keys) is Drizzle/`pgTable` in
`packages/db/src/schema/`, one PS branch per deployed stage (`main`=prd, `stg`), reached from
Workers via the Hyperdrive binding `MAPLE_DB`.
`packages/db/src/schema/`, on the PlanetScale `main` branch (prd — the only stage with a
database), reached from Workers via the Hyperdrive binding `MAPLE_DB`.

- App code keeps epoch-ms numbers and converts at the drizzle boundary — use `msToDate` /
`dateToMs` from `packages/backend/src/platform/time.ts` rather than bare `new Date(ms)` /
Expand Down Expand Up @@ -269,12 +269,12 @@ refuses to run the command at all rather than running it with egress. Every bug
that the unit tests could not see (a `mktemp -d` mode, a git flag this image predates, `runuser`
adding `USER` and `LOGNAME` after `env -i`) was found by running the image.

End to end needs a real deployment: `stageDeploysSandbox` is `prd`/`stg` only, so `bun dev` binds no
End to end needs a real deployment: `stageDeploysSandbox` is `prd` only, so `bun dev` binds no
`SANDBOX` and the four tools report that no sandbox is available.

The container is **not** in `apps/api` — Cloudflare's Sandbox is a Durable Object class the script
must export, and an Effect-native Worker's generated entry exports only its own bridge classes. It
lives in `apps/sandbox`, on `prd`/`stg` only; see `docs/infra.md` § Single-module Workers.
lives in `apps/sandbox`, on `prd` only; see `docs/infra.md` § Single-module Workers.

## Self-observability (trace loop prevention)

Expand Down
17 changes: 7 additions & 10 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -104,9 +104,9 @@ single `Alchemy.Stack("maple", …)` whose program yields one module per app:
- `apps/web/src/worker.ts` / `apps/landing/src/worker.ts` / `apps/local-ui/src/worker.ts`
— static builds via `Command.Build` + asset-serving Workers

Stage grammar is `prd` / `stg` / `pr-<number>` / dev names, resolved via
Stage grammar is `prd` / `pr-<number>` / dev names, resolved via
`@maple/infra/cloudflare` (`parseMapleStage`, `resolveMapleDomains`, `resolveWorkerName`,
`resolveHyperdriveRefId`, `resolveDatabaseMode`). stg/prd bind the
`resolveHyperdriveRefId`, `resolveDatabaseMode`). prd binds the
dashboard-managed Hyperdrive by config ID (`resolveHyperdriveRefId`) — origin credentials
never touch a deploy. `MAPLE_PG_URL` is only needed for dev stages, whose Hyperdrive alchemy
manages itself. PR previews bind **no database at all** (`resolveDatabaseMode` → `"none"`):
Expand All @@ -115,7 +115,6 @@ DB-backed routes 500, everything else in the preview works.
Run locally:

```bash
bun run alchemy:deploy:stg
PR_NUMBER=123 bun run alchemy:deploy:pr
```

Expand All @@ -126,13 +125,11 @@ never run a v1 `alchemy destroy` against a live stage.
Tear down:

```bash
bun run alchemy:destroy:stg
PR_NUMBER=123 bun run alchemy:destroy:pr
```

CI workflows:

- STG (default on push to `main`): `.github/workflows/deploy-stg.yml`
- PRD (manual only via `workflow_dispatch`): `.github/workflows/deploy-prd.yml`
- PR preview lifecycle: `.github/workflows/deploy-pr-preview.yml` (`pull_request` opened/synchronize/reopened/closed)

Expand All @@ -146,8 +143,8 @@ Secrets source model (CI):
slug like `maple` would then blank out the PR-preview deployment URL
`app-pr-<n>.maple.dev`)
- GitHub repo **secret** `INFISICAL_MACHINE_IDENTITY_ID` (the machine identity ID)
- Infisical environments (`prod`, `staging`, `dev` — mapped from the old Doppler
`prd`/`stg`/`pr` configs) must define:
- Infisical environments (`prod`, `dev` — mapped from the old Doppler
`prd`/`pr` configs) must define:
- `CLOUDFLARE_API_TOKEN`
- `CLOUDFLARE_DEFAULT_ACCOUNT_ID` (bridged to alchemy v2's `CLOUDFLARE_ACCOUNT_ID` in the root `alchemy.run.ts`; `ALCHEMY_PASSWORD`/`ALCHEMY_STATE_TOKEN` were v1-only and are no longer read)
- `TINYBIRD_HOST`
Expand All @@ -161,11 +158,11 @@ Secrets source model (CI):
- `CLERK_PUBLISHABLE_KEY`
- `CLERK_JWT_KEY`

Setup note: the machine identity must have a **GitHub OIDC** auth method configured in Infisical (scoped to this repo, ideally to the `production`/`staging`/`pr-preview` GitHub environments) and read access to the project. The workflows select secrets via `project-slug` (`INFISICAL_PROJECT_SLUG`) and per-stage `env-slug` (`prod`/`staging`/`dev`).
Setup note: the machine identity must have a **GitHub OIDC** auth method configured in Infisical (scoped to this repo, ideally to the `production`/`pr-preview` GitHub environments) and read access to the project. The workflows select secrets via `project-slug` (`INFISICAL_PROJECT_SLUG`) and per-stage `env-slug` (`prod`/`dev`).

Runtime API URL behavior:

- Deploy-time web builds resolve `VITE_API_BASE_URL` from the Cloudflare api worker domain (`api.maple.dev` in `prd`, `api-staging.maple.dev` in `stg`, worker.dev URL for `pr-*`).
- Deploy-time web builds resolve `VITE_API_BASE_URL` from the Cloudflare api worker domain (`api.maple.dev` in `prd`, worker.dev URL for `pr-*`).
- Local `bun --filter=@maple/web dev` can still use root `.env` `VITE_API_BASE_URL` for local API routing.

## Environment
Expand Down Expand Up @@ -193,7 +190,7 @@ For ingest + key auth, set these at minimum in your root `.env` when running the

Maple persists application state in PostgreSQL:

- Production and staging use PlanetScale Postgres through Cloudflare Hyperdrive.
- Production uses PlanetScale Postgres through Cloudflare Hyperdrive.
- Wrangler development uses the Docker Postgres started by `bun db:up`.
- Non-Worker local entrypoints use embedded PGlite under `apps/api/.data/pglite`; set
`MAPLE_DB_URL=memory://` for an ephemeral database.
Expand Down
8 changes: 4 additions & 4 deletions alchemy.run.ts
Original file line number Diff line number Diff line change
Expand Up @@ -199,8 +199,8 @@ export default Alchemy.Stack(
)
}

// The Rust OTLP gateway on ECS Fargate (prd/stg/pr — dev stages run it
// through docker-compose instead). On prd/stg `domains.ingest` reaches it
// The Rust OTLP gateway on ECS Fargate (prd/pr — dev stages run it
// through docker-compose instead). On prd `domains.ingest` reaches it
// via a Cloudflare CNAME at the ALB, so the URL below stays a plain string
// and does not depend on the service resource; a PR preview gets no ingest
// domain, so its ALB answers plain HTTP on 80 at `ingest.serviceUrl`.
Expand All @@ -210,7 +210,7 @@ export default Alchemy.Stack(

// The application database. Each Worker binds `MAPLE_DB` from its own init
// (`MapleDb` in `@maple/infra/cloudflare`: the managed Hyperdrive on dev
// stages, a dashboard-managed config by id on stg/prd, nothing on previews).
// stages, a dashboard-managed config by id on prd, nothing on previews).
// The managed declaration is yielded here first so its `MAPLE_PG_URL` read
// happens outside any Worker init, where alchemy would bind it as a secret.
if (resolveDatabaseMode(stage) === "managed") yield* ManagedMapleDb
Expand All @@ -234,7 +234,7 @@ export default Alchemy.Stack(
)
yield* serveWorker("api", api)

// Self-hosted ElectricSQL on ECS Fargate (prd/stg — dev stages use the
// Self-hosted ElectricSQL on ECS Fargate (prd — dev stages use the
// docker `electric` service, and PR previews have no database to replicate
// from). Deliberately NOT wired into the sync worker's env here: the worker
// reads `ELECTRIC_URL` from the secret store, so standing this service up
Expand Down
10 changes: 5 additions & 5 deletions apps/alerting/src/worker.ts
Original file line number Diff line number Diff line change
Expand Up @@ -145,10 +145,10 @@ const props = Effect.gen(function* () {
const ALERTING_CRONS = ["* * * * *", "*/5 * * * *", "*/15 * * * *", "0 * * * *"] as const

/**
* Non-prod stages (stg, PR previews) share live org data — stg's Hyperdrive
* points at the prod database — so their crons would iterate real orgs with
* stage-local Tinybird/Clerk credentials: every tick fails per-org and floods
* the error dashboards (and historically sent duplicate emails, see #237).
* Non-prod stages (PR previews, dev) share live org data, so their crons would
* iterate real orgs with stage-local Tinybird/Clerk credentials: every tick
* fails per-org and floods the error dashboards (and historically sent
* duplicate emails, see #237).
* Same gating philosophy as the prd-only EMAIL binding, with an explicit
* override for deliberately exercising crons on a non-prod stage.
*/
Expand All @@ -166,7 +166,7 @@ export default class Alerting extends Cloudflare.Worker<Alerting>()(
// validation or in the deploy process. A rejected import is retried on
// the next fire rather than pinned (`Effect.cached` keeps the failure).
const scheduled = yield* cachedRecoverable(Effect.promise(() => import("./scheduled")))
// `MAPLE_DB` in the stage's flavor — on stg/prd its own dashboard-managed
// `MAPLE_DB` in the stage's flavor — on prd its own dashboard-managed
// config: `alerting` issues ~97% of the workers' Postgres traffic and was
// starving the api's connection pool when the two shared one. The ticks
// read it off the fire's env.
Expand Down
11 changes: 6 additions & 5 deletions apps/api/scripts/BENCH.md
Original file line number Diff line number Diff line change
Expand Up @@ -75,8 +75,8 @@ only the raw ClickHouse HTTP interface exposes.
```

Skim the table to confirm the queries look real — they should start with
`SELECT`, contain `OrgId =`, and the p95 column should match the staging
trace dashboard.
`SELECT`, contain `OrgId =`, and the p95 column should match the trace
dashboard.

2. **Measure baseline.** Replay them locally, 5 runs each:

Expand All @@ -102,9 +102,10 @@ only the raw ClickHouse HTTP interface exposes.
— add a `PREWHERE`, narrow the projection, switch from `Traces` to an
already-aggregated MV, whatever the EXPLAIN suggested.

5. **Re-run.** Either deploy the change to staging and run `bench:fetch`
again (so the new fingerprint shows up in traces), or hand-edit the SQL in
a copy of the baseline JSON for a faster local loop. Then:
5. **Re-run.** Either deploy the change to **production** and run `bench:fetch`
again (so the new fingerprint shows up in the traces this tool mines — a PR
or dev stage's traces are not queried), or hand-edit the SQL in a copy of the
baseline JSON for a faster local loop. Then:

```
bun bench:run .bench/errorsByType-after-prewhere.json --runs 5 \
Expand Down
2 changes: 1 addition & 1 deletion apps/api/src/resources/replay-blobs.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
* bucket-scoped token is minted in `apps/ingest/alchemy.run.ts`, next to the
* writer — and the api Worker binds the same bucket to hydrate
* `session_replay_events` rows whose `Events` is empty. Stage-isolated, so a
* pr/stg deploy can never serve or overwrite prd recordings. Declared once
* pr deploy can never serve or overwrite prd recordings. Declared once
* here; alchemy registers a resource by id, so the second yield returns the
* first's registration.
*
Expand Down
Loading