Skip to content

feat(cli): compile source trees with relative URIs and a claimed language - #294

Merged
DamianReeves merged 2 commits into
mainfrom
DamianReeves/core-wasm-document-trees
Nov 1, 2026
Merged

DamianReeves merged 2 commits into
mainfrom
DamianReeves/core-wasm-document-trees

Conversation

@DamianReeves

@DamianReeves DamianReeves commented Oct 31, 2026 •

Copy link
Copy Markdown
Member

Summary

This PR implements the source-tree slice of the OpenSpec change run-core-wasm-guests-from-any-language
(tasks 1.3, 2.1, 2.2, 2.4 and 2.5; beads morphir-ukvn.5.64.37.1 and morphir-ukvn.5.64.37.2).

  • --language (SEL-3). When the guest claims one language, the CLI uses it. When it claims several,
    --language must name one: without it the command exits 2 with extension.input_invalid, and the message names
    the claimed languages. An unclaimed language exits 2 with extension.capability_unavailable. The walk reads only
    the file extensions of the selected language. Both refusals start no operation session.
  • URIs from the source base (SRC-1). Each document URI is the file's path relative to --source-base, or,
    without it, to the directory that the user names (option B, decided by Damian Reeves on 2026-11-01). Thus
    compile src sends app/money.gleam from any current directory, as main does. With --source-base /work/shop,
    compile /work/shop/src/app sends src/app/money.gleam. An input outside the source base exits 2 with
    extension.input_invalid, and the message names the base. A single file without --source-base keeps the
    earlier rule, so MoonBit requests do not change. The request has no sources.root.
  • Percent-encoded segments (SRC-3). Unreserved characters stay; every other UTF-8 byte is %XX.
    my module.gleam becomes my%20module.gleam. Names with : or \ are now encoded, not refused.
  • Order and refusals (SRC-2, FIT-1). Documents are sorted by the whole URI in byte order (a-c.gleam before
    a/b.gleam). A repeated URI is refused. A new 1 MiB bound applies to one file. Every limit message names the
    limit and its value; a link or special file is named by its path relative to the base; a file that is not UTF-8
    is named by its URI. All of these keep extension.input_invalid, exit 2. The 4 MiB request check stays last.
  • Determinism (2.4). Two compiles of the same tree send byte-identical compileModel requests; other trees
    with the same package and options keep the same contextDigest.
  • Docs (2.5). apps/morphir/docs/extension.md step 4 and a new section "Source trees and document URIs";
    the ignore-file docs move to task 2.6, which waits for Ignore-file library and moonx tool: Git-compatible, custom ignore files, explain, layer and containment options moonrockz/moonx-tools#16.
  • A one-line fix clears the only MoonBit warning in apps/morphir on wasm and wasm-gc (unused stop_requested).

Without --source-base, a directory compile sends the same URIs as main, so this PR has no breaking change for
directory input. The first version made the current directory the default base (option A); the extension
qualification suite of #298 showed that it broke the published Elm 0.5.0-preview.2 and Python 0.4.2 extensions,
so Damian Reeves chose option B. A Cargo-style source root (from the project manifest and the extension's
workspace discovery) is a separate follow-up.

Out of scope: 2.6 (ignore files), the group 1 conversion tasks (1.6 to 1.13), groups 3 and 4.

Check with the released Gleam guest

Local pin (contract 0.2.0, wasm / morphir-bytes-v1, descriptor) over extension/gleam/v0.4.0
(morphir-gleam-0.4.0.wasm SHA-256 12349dac…205e, release.json SHA-256 be337f8f…93dc), on macOS arm64.
Tree: src/shared/numbers.gleam, src/app/money.gleam (imports shared/numbers.{type Amount}),
src/app/orders.gleam (imports app/money.{type Money}).

Run Result
native extension compile src from the project, no --model-format exit 0, JSON tuple agreed, modules app/money, app/orders, shared/numbers, cross-module reference acme/shop:shared/numbers#amount
native, compile . from the project, and compile <abs>/src from another directory exit 0, model.json byte-identical to the run above (option B, MoonBit 0.10.18)
native, compile <abs>/src --source-base <abs> from another directory exit 0, model.json byte-identical to the run above
native, compile src/shared exit 0, module shared/numbers
native, --source-base /tmp/elsewhere exit 2, extension.input_invalid naming the base
native, --model-format ion exit 2, extension.tuple_unavailable
native, extension generate model/model-envelope.json exit 0, app/money.gleam, app/orders.gleam, shared/numbers.gleam
npm CLI, same compile and generate exit 0; model.json byte-identical to the native one
file src/app/my notes.gleam URI src/app/my%20notes.gleam; the guest decodes it once to my notes and refuses it as a module name (INVALID_MODULE_URI)

The last row is evidence for the open point "Percent-encoding (SRC-3)" of design.html: the guest decodes encoded
segments. Gleam 0.4.0 itself does not resolve imported or qualified values (GLEAM_UNSUPPORTED_VALUE), so the
tree imports types only; that is a guest limit, not a host one.

Test plan

  • moon test in apps/morphir on native, js, wasm and wasm-gc
  • moon check in apps/morphir on all four targets: no warnings in the module
  • mise run lint, mise run lint:moonbit, mise run validate
  • openspec validate run-core-wasm-guests-from-any-language --strict
  • Released morphir-gleam 0.4.0 module through the native and npm CLI (table above)
  • mise run test:wasm-workers -- --target native: 3791 of 3792. The failure is the timing test
    a launch waits while another process briefly holds the copy open for writing
    (pkgs/morphir-host/worker/process/native/unix_wbtest.mbt:124), a package that this PR does not change; it
    passes 3 of 3 when run alone
  • bun run --cwd apps/morphir test native: the process-worker acceptance and the Extism acceptance (Python
    0.4.2 compiles src) pass. Each of two runs stopped on a native SIGSEGV in another
    command (extension describe in run 1, morphir generate in test-generation.ts in run 2), which matches the
    open bead morphir-fr6z (flaky native SIGSEGV); the same describe passes 3 of 3 when run by hand

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 31, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-31T15:45:33.192728Z 398e86a PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

…uage

Implement tasks 1.3, 2.1, 2.2, 2.4 and 2.5 of the OpenSpec change
run-core-wasm-guests-from-any-language.

- `--language` selects one of several claimed languages (SEL-3). Without
  it, several claimed languages give extension.input_invalid; an
  unclaimed language gives extension.capability_unavailable. The walk
  reads only the file extensions of the selected language.
- Each document URI is the path relative to the source base:
  `--source-base`, else the current directory (SRC-1). An input outside
  the base is extension.input_invalid. A single file without
  `--source-base` keeps the earlier rule.
- Each URI segment is percent-encoded as an RFC 3986 path segment, so
  names with `:`, `\` or a space are encoded, not refused (SRC-3).
- Documents are sorted by the whole URI in byte order, a repeated URI is
  refused, and one file is bounded to 1 MiB. Limit messages name the
  limit and its value; link messages name the relative path; UTF-8
  messages name the URI (SRC-2, FIT-1).
- Tests prove byte-identical requests for the same tree and the same
  contextDigest for other trees.
- Document the source base, the encoding, the order and the limits in
  apps/morphir/docs/extension.md.
- Read the stop probe in the WASI entry, which clears an unused-field
  warning on wasm and wasm-gc.

Checked with the released morphir-gleam 0.4.0 module (native and npm CLI).
Without --source-base, the document URIs of a directory compile are
relative to that directory again, as before this change. The current
directory default broke the published Elm 0.5.0-preview.2 and Python
0.4.2 extensions in the extension qualification suite: compile
generated/src sent generated/src/Acme/Rules/Main.elm, and compile src
named the Python module src.acme.rules. The Gleam 0.4.0 guest gives the
same module names for compile src and compile . with this default.

--source-base still overrides the base, and a single file keeps its
earlier rule. Damian Reeves chose this option (B) on 2026-11-01. The
spec, design (SRC-1), tasks, proposal, CLI help and documentation now
describe it.
@DamianReeves
DamianReeves force-pushed the DamianReeves/core-wasm-document-trees branch from 398e86a to 12163ba Compare November 1, 2026 02:35
@DamianReeves

Copy link
Copy Markdown
Member Author

Rebased onto main at 109ee20 (MoonBit 0.10.18+6c83a7f62, moonbitlang/async 0.22.7, #286 and #290). New head: 12163ba.

Rebase. There were no git conflicts. But the extension qualification of #298 showed a behavior conflict with the first default source base (the current directory, option A). compile generated/src sent generated/src/Acme/Rules/Main.elm, so Elm 0.5.0-preview.2 refused the recompile (elm.module_name). compile src named the Python 0.4.2 module src.acme.rules, so the consumer import and the recompile failed. Damian Reeves chose option B on 2026-11-01: without --source-base, a directory input is its own base (the URIs of main), and --source-base overrides it. Commit 12163ba makes this change and updates the spec "Relative document URIs", design.html (SRC-1), tasks.md, proposal.md, the CLI help and docs/extension.md. The qualification driver needs no change.

Checks on 0.10.18 (macOS arm64):

Check Result
moon test in apps/morphir native 3792/3792, js 3883/3883, wasm 3476/3476, wasm-gc 3430/3430
moon check, 4 targets 0 warnings in the module
mise run lint, lint:moonbit, validate; openspec validate --all --strict pass; 5 of 5
mise run test:cli pass: worker host and Extism acceptance on native, js and npm
mise run test:extension-qualification all 5 suites pass: elm 16 + 2 known failures, python 32, moonbit-native 12 + 1 known, moonbit-portable 11 + 1 known, conformance 11
pre-push gates pass
Gleam 0.4.0 module compile src, compile . and compile <abs>/src from another directory give a byte-identical model.json (modules app/money, app/orders, shared/numbers), equal to the 0.10.17 run; generate gives the same 3 files

Under option A (before 12163ba), one native run had a failure in the known timing test unix_wbtest.mbt:124; it passed 39/39 when run again. No native SIGSEGV (morphir-fr6z) occurred. No test asserts on the new 0.10.18 stderr output, so no test changed for the toolchain. Locally, test:cli needs moon build pkgs/morphir-host/worker/thread/entry --target js --release first, as CI does.

@DamianReeves
DamianReeves added this pull request to the merge queue Nov 1, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Nov 1, 2026
@DamianReeves
DamianReeves added this pull request to the merge queue Nov 1, 2026
Merged via the queue into main with commit fb6d5cb Nov 1, 2026
71 of 73 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant