Skip to content

fix: activate and automatically index Media Usage - #2538

Open
khoinguyenpham04 wants to merge 31 commits into
chore/media-usage-dev-seed-readinessfrom
feature/media-usage-activation-and-indexing
Open

fix: activate and automatically index Media Usage#2538
khoinguyenpham04 wants to merge 31 commits into
chore/media-usage-dev-seed-readinessfrom
feature/media-usage-activation-and-indexing

Conversation

@khoinguyenpham04

@khoinguyenpham04 khoinguyenpham04 commented Aug 18, 2026

Copy link
Copy Markdown
Collaborator

What does this PR do?

Completes Media Usage setup so administrators can turn it on once and let EmDash index existing content in the background.

This PR:

  • adds an Admin-only Settings → Media Usage page and a Media Library setup notice;
  • uses one Enable Media Usage button and one Turn on confirmation;
  • shows Setting up, Starting indexing, Indexing existing content, Finishing setup, Ready, and Needs attention states;
  • continues setup after the browser closes;
  • processes up to 1,000 existing entries in each background batch;
  • starts Cloudflare work immediately through a Queue and continues it with one consumer at a time;
  • runs the same continuous background process through the Node.js scheduler;
  • removes the dedicated Media Usage Cron and scheduled recovery path;
  • preserves durable work rows, leases, retries, stale-worker fencing, live-write safety, collection-deletion safety, and truthful coverage;
  • updates Cloudflare templates and the Node.js, Cloudflare, Media Library, and REST API documentation.

The activation request prepares at most one collection. It never indexes the full site inside the request. Cloudflare Queue messages only wake the worker; durable database rows remain the source of truth across retries and interruptions.

The Queue consumer or Node.js process must remain running until Media Usage reaches Ready. The general Cron does not restart interrupted Media Usage indexing.

This PR is stacked directly on #2472 and should be reviewed relative to chore/media-usage-dev-seed-readiness.

No linked issue.

Type of change

  • Bug fix
  • Feature (requires maintainer-approved Discussion)
  • Refactor (no behavior change)
  • Translation
  • Documentation
  • Performance improvement
  • Tests
  • Chore (dependencies, CI, tooling)

Checklist

  • I have read CONTRIBUTING.md
  • pnpm typecheck passes
  • pnpm lint passes
  • pnpm test passes (targeted tests for this change)
  • pnpm format has been run
  • I have added/updated tests for my changes (if applicable)
  • User-visible strings in the admin UI are wrapped for translation (if applicable). The focused English catalog fix restores readable progress text; unrelated generated catalogs are unchanged.
  • I have added a changeset (if this PR changes a published package)
  • New features link to an approved Discussion: not applicable; this completes and accelerates the existing Media Usage system

AI-generated code disclosure

  • This PR includes AI-generated code — model/tool: OpenAI Codex (GPT-5.6), with GPT-5.6 Terra adversarial review

Screenshots / test output

The CI preview can be used to review the setup page, Media Library notice, activation states, and indexing progress.

Verified locally across the implementation commits:

  • targeted Node.js, workerd/D1, Cloudflare scheduler, and admin Media Usage tests;
  • package typechecks and builds;
  • the documentation build, type-aware lint with zero diagnostics, formatting, changeset validation, and git diff --check;
  • a complete 10,000-entry local backfill with exact final coverage;
  • iterative adversarial review with no remaining findings.

Try this PR

Open a fresh playground →

A full working EmDash site, deployed from this branch. Each visit gets its own session-scoped sandbox: no login needed and no shared state. Try the admin, edit content, hit the public site.

Tracks feature/media-usage-activation-and-indexing. Updated automatically when the playground redeploys.

@changeset-bot

changeset-bot Bot commented Aug 18, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: c9083f4

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 17 packages
Name Type
emdash Minor
@emdash-cms/cloudflare Minor
@emdash-cms/admin Minor
@emdash-cms/sandbox-workerd Patch
@emdash-cms/plugin-mcp-smoke Major
@emdash-cms/fixture-perf-site Patch
@emdash-cms/perf-demo-site Patch
@emdash-cms/cache-demo-site Patch
@emdash-cms/do-demo-site Patch
@emdash-cms/do-solo-demo-site Patch
@emdash-cms/auth Minor
@emdash-cms/blocks Minor
@emdash-cms/gutenberg-to-portable-text Minor
@emdash-cms/x402 Minor
create-emdash Minor
@emdash-cms/auth-atproto Patch
@emdash-cms/plugin-embeds Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@github-actions

Copy link
Copy Markdown
Contributor

Scope check

This PR changes 3,521 lines across 63 files. Large PRs are harder to review and more likely to be closed without review.
This PR spans 5 different areas (area/core, area/admin, area/docs, area/templates, area/cloudflare). Consider breaking it into smaller, focused PRs.

If this scope is intentional, no action needed. A maintainer will review it. If not, please consider splitting this into smaller PRs.

See CONTRIBUTING.md for contribution guidelines.

@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Aug 18, 2026

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Updated (UTC)
✅ Deployment successful!
View logs
emdash-demo-do c9083f4 Aug 21 2026, 01:50 PM

@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Aug 18, 2026

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Updated (UTC)
✅ Deployment successful!
View logs
emdash-playground 1fcbc86 Aug 18 2026, 10:43 AM

@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Aug 18, 2026

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Updated (UTC)
✅ Deployment successful!
View logs
docs c9083f4 Aug 21 2026, 01:48 PM

@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Aug 18, 2026

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Updated (UTC)
✅ Deployment successful!
View logs
emdash-demo-cache c9083f4 Aug 21 2026, 01:50 PM

@github-actions

github-actions Bot commented Aug 18, 2026

Copy link
Copy Markdown
Contributor

Lunaria Status Overview

🌕 This pull request will trigger status changes.

Learn more

By default, every PR changing files present in the Lunaria configuration's files property will be considered and trigger status changes accordingly.

You can change this by adding one of the keywords present in the ignoreKeywords property in your Lunaria configuration file in the PR's title (ignoring all files) or by including a tracker directive in the merged commit's description.

Tracked Files

File Note
packages/admin/src/locales/en/messages.po Source changed, localizations will be marked as outdated.
Warnings reference
Icon Description
🔄️ The source for this localization has been updated since the creation of this pull request, make sure all changes in the source have been applied.

@khoinguyenpham04
khoinguyenpham04 marked this pull request as ready for review August 20, 2026 14:44
@github-actions github-actions Bot added the review/needs-review No maintainer or bot review yet label Aug 20, 2026
…ctivation-and-indexing

# Conflicts:
#	packages/admin/src/components/MediaLibrary.tsx
#	packages/core/src/emdash-runtime.ts
#	packages/core/tests/integration/runtime/media-usage-scheduled-driver.test.ts
@pkg-pr-new

pkg-pr-new Bot commented Aug 21, 2026

Copy link
Copy Markdown

Open in StackBlitz

@emdash-cms/admin

npm i https://pkg.pr.new/@emdash-cms/admin@2538

@emdash-cms/auth

npm i https://pkg.pr.new/@emdash-cms/auth@2538

@emdash-cms/auth-atproto

npm i https://pkg.pr.new/@emdash-cms/auth-atproto@2538

@emdash-cms/blocks

npm i https://pkg.pr.new/@emdash-cms/blocks@2538

@emdash-cms/cloudflare

npm i https://pkg.pr.new/@emdash-cms/cloudflare@2538

@emdash-cms/contentful-to-portable-text

npm i https://pkg.pr.new/@emdash-cms/contentful-to-portable-text@2538

emdash

npm i https://pkg.pr.new/emdash@2538

create-emdash

npm i https://pkg.pr.new/create-emdash@2538

@emdash-cms/gutenberg-to-portable-text

npm i https://pkg.pr.new/@emdash-cms/gutenberg-to-portable-text@2538

@emdash-cms/plugin-cli

npm i https://pkg.pr.new/@emdash-cms/plugin-cli@2538

@emdash-cms/plugin-types

npm i https://pkg.pr.new/@emdash-cms/plugin-types@2538

@emdash-cms/registry-client

npm i https://pkg.pr.new/@emdash-cms/registry-client@2538

@emdash-cms/registry-lexicons

npm i https://pkg.pr.new/@emdash-cms/registry-lexicons@2538

@emdash-cms/registry-verification

npm i https://pkg.pr.new/@emdash-cms/registry-verification@2538

@emdash-cms/sandbox-workerd

npm i https://pkg.pr.new/@emdash-cms/sandbox-workerd@2538

@emdash-cms/x402

npm i https://pkg.pr.new/@emdash-cms/x402@2538

@emdash-cms/plugin-ai-moderation

npm i https://pkg.pr.new/@emdash-cms/plugin-ai-moderation@2538

@emdash-cms/plugin-atproto

npm i https://pkg.pr.new/@emdash-cms/plugin-atproto@2538

@emdash-cms/plugin-audit-log

npm i https://pkg.pr.new/@emdash-cms/plugin-audit-log@2538

@emdash-cms/plugin-color

npm i https://pkg.pr.new/@emdash-cms/plugin-color@2538

@emdash-cms/plugin-embeds

npm i https://pkg.pr.new/@emdash-cms/plugin-embeds@2538

@emdash-cms/plugin-field-kit

npm i https://pkg.pr.new/@emdash-cms/plugin-field-kit@2538

@emdash-cms/plugin-forms

npm i https://pkg.pr.new/@emdash-cms/plugin-forms@2538

@emdash-cms/plugin-webhook-notifier

npm i https://pkg.pr.new/@emdash-cms/plugin-webhook-notifier@2538

commit: c9083f4

@github-actions github-actions Bot mentioned this pull request Aug 21, 2026
18 tasks
@khoinguyenpham04 khoinguyenpham04 added the bot:review Trigger an emdashbot code review on this PR label Aug 21, 2026
@emdashbot emdashbot Bot removed the bot:review Trigger an emdashbot code review on this PR label Aug 21, 2026
@khoinguyenpham04 khoinguyenpham04 added the bot:review Trigger an emdashbot code review on this PR label Aug 21, 2026
@emdashbot emdashbot Bot removed the bot:review Trigger an emdashbot code review on this PR label Aug 21, 2026
@khoinguyenpham04 khoinguyenpham04 added the bot:review Trigger an emdashbot code review on this PR label Aug 21, 2026
@emdashbot emdashbot Bot removed the bot:review Trigger an emdashbot code review on this PR label Aug 21, 2026
@khoinguyenpham04 khoinguyenpham04 added the bot:review Trigger an emdashbot code review on this PR label Aug 21, 2026
@emdashbot emdashbot Bot removed the bot:review Trigger an emdashbot code review on this PR label Aug 21, 2026
@ascorbic ascorbic added the bot:review Trigger an emdashbot code review on this PR label Aug 21, 2026
@emdashbot emdashbot Bot removed the bot:review Trigger an emdashbot code review on this PR label Aug 21, 2026
@khoinguyenpham04 khoinguyenpham04 added the bot:review Trigger an emdashbot code review on this PR label Aug 21, 2026
@emdashbot emdashbot Bot removed the bot:review Trigger an emdashbot code review on this PR label Aug 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants