diff --git a/.agents/skills/release/SKILL.md b/.agents/skills/release/SKILL.md index 43bf52ce2..8d985ee8c 100644 --- a/.agents/skills/release/SKILL.md +++ b/.agents/skills/release/SKILL.md @@ -123,7 +123,7 @@ not satisfy these gates. ```sh SUBMILLI_SKIP_HTTP_TESTS=1 SUBMILLI_TEST_NIGHTLY_ONLY=1 \ - cargo test --locked -p interpreter --test determinism -- --nocapture + cargo test --locked -p submilli-engine --test determinism -- --nocapture ``` `SUBMILLI_FULL_TEST` does not enable determinism. Require successful execution @@ -157,7 +157,7 @@ not satisfy these gates. ```sh SUBMILLI_SKIP_HTTP_TESTS=1 SUBMILLI_TEST_NIGHTLY_ONLY=1 \ - cargo test --locked -p interpreter --test type_limits -- --nocapture + cargo test --locked -p submilli-engine --test type_limits -- --nocapture ``` Require all test bodies to execute successfully. Record the candidate and @@ -166,7 +166,7 @@ not satisfy these gates. ```sh SUBMILLI_SKIP_HTTP_TESTS=1 SUBMILLI_TEST_NIGHTLY_ONLY=1 \ - cargo test --locked -p interpreter --test git_memory -- --nocapture + cargo test --locked -p submilli-engine --test git_memory -- --nocapture ``` Require both memory-limit test bodies to execute successfully. The optional @@ -176,7 +176,7 @@ not satisfy these gates. ```sh SUBMILLI_SKIP_HTTP_TESTS=1 SUBMILLI_TEST_NIGHTLY_ONLY=1 \ - cargo test --locked -p interpreter --test host_memory -- --nocapture + cargo test --locked -p submilli-engine --test host_memory -- --nocapture SUBMILLI_SKIP_HTTP_TESTS=1 SUBMILLI_TEST_NIGHTLY_ONLY=1 \ cargo test --locked -p submilli-server --test memory_cap -- --nocapture ``` diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index ceb3a580f..33e424de0 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -38,6 +38,12 @@ jobs: - name: Clippy (gate) run: cargo clippy --workspace --all-targets -- -D warnings + # The engine is published to crates.io. Packaging builds it from the + # packaged files alone, so a source file left out of the package fails + # here rather than at release. + - name: Package for crates.io + run: cargo publish --dry-run --locked -p submilli-engine + # The fixture harness defaults to a smoke subset (196 of 1054) for fast local # iteration. CI has no such constraint — the full sweep is a few seconds — and # the subset keeps one fixture per top-level directory, so whole trees never run. @@ -87,7 +93,7 @@ jobs: - uses: Swatinem/rust-cache@v2 - name: VFS containment suite - run: cargo test -p interpreter --test vfs_containment + run: cargo test -p submilli-engine --test vfs_containment - name: Skill installation across platforms run: cargo test -p submilli --test skill --test upgrade @@ -108,7 +114,7 @@ jobs: # Unicode aliases, and identity read through a handle on Windows. - name: VFS mounts run: >- - cargo test -p interpreter --lib -- + cargo test -p submilli-engine --lib -- stdlib::fs::tests::mount stdlib::git::integration_tests::mount # The size limit keys files by device and inode, which Windows reports only @@ -118,7 +124,7 @@ jobs: # writers hold open. - name: VFS size limit run: >- - cargo test -p interpreter --lib -- + cargo test -p submilli-engine --lib -- runtime::vfs::tests::size_limit runtime::vfs::tests::measuring runtime::vfs::tests::a_tree stdlib::fs::tests::size_limit stdlib::fs::tests::a_ stdlib::fs::tests::copying_ stdlib::fs::tests::rewriting_ stdlib::code::tests::edits_are_counted diff --git a/.github/workflows/conformance.yml b/.github/workflows/conformance.yml index 9bc4d7339..af50d675a 100644 --- a/.github/workflows/conformance.yml +++ b/.github/workflows/conformance.yml @@ -29,16 +29,16 @@ jobs: run: cargo test --locked -p conformance --test typescript -- --nocapture - name: Compiler determinism if: ${{ !cancelled() }} - run: cargo test --locked -p interpreter --test determinism -- --nocapture + run: cargo test --locked -p submilli-engine --test determinism -- --nocapture - name: Compiler type limits if: ${{ !cancelled() }} - run: cargo test --locked -p interpreter --test type_limits -- --nocapture + run: cargo test --locked -p submilli-engine --test type_limits -- --nocapture - name: Git memory limits if: ${{ !cancelled() }} - run: cargo test --locked -p interpreter --test git_memory -- --nocapture + run: cargo test --locked -p submilli-engine --test git_memory -- --nocapture - name: Host memory bounds if: ${{ !cancelled() }} - run: cargo test --locked -p interpreter --test host_memory -- --nocapture + run: cargo test --locked -p submilli-engine --test host_memory -- --nocapture - name: Server memory caps if: ${{ !cancelled() }} run: cargo test --locked -p submilli-server --test memory_cap -- --nocapture diff --git a/AGENTS.md b/AGENTS.md index 96243b8f1..93ff8e3ed 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -300,7 +300,7 @@ Whenever modifying the Git standard library (`crates/interpreter/src/stdlib/git/ run the Git memory-limit tests in addition to other affected checks: ```sh -SUBMILLI_SKIP_HTTP_TESTS=1 SUBMILLI_FULL_TEST=0 SUBMILLI_TEST_NIGHTLY_ONLY=1 cargo test --locked -p interpreter --test git_memory -- --nocapture +SUBMILLI_SKIP_HTTP_TESTS=1 SUBMILLI_FULL_TEST=0 SUBMILLI_TEST_NIGHTLY_ONLY=1 cargo test --locked -p submilli-engine --test git_memory -- --nocapture ``` These tests use an in-process HTTP transport backed by local `git upload-pack`, @@ -371,7 +371,7 @@ modes with synthetic package tests and a focused localhost test; contact real APIs only when their integration behavior is affected. Select the affected crate/test or package rather than all network integrations, -for example `SUBMILLI_SKIP_HTTP_TESTS=0 cargo test -p interpreter --test http_live` +for example `SUBMILLI_SKIP_HTTP_TESTS=0 cargo test -p submilli-engine --test http_live` or `cargo run -p submilli -- build test -p @submilli/jina --env-var JINA_API_KEY`. These tests may require execution outside the sandbox, even for local listeners. Request network access only for selected checks that require it. Record why HTTP diff --git a/Cargo.lock b/Cargo.lock index 718063490..5500ed56a 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -998,7 +998,7 @@ dependencies = [ name = "conformance" version = "0.3.0" dependencies = [ - "interpreter", + "submilli-engine", "submilli-wasm", "tokio", ] @@ -3239,57 +3239,6 @@ dependencies = [ "tempfile", ] -[[package]] -name = "interpreter" -version = "0.3.0" -dependencies = [ - "async-trait", - "base64 0.22.1", - "cap-fs-ext", - "cap-primitives", - "cap-std", - "filetime", - "flate2", - "futures", - "getrandom 0.2.17", - "gimli 0.33.0", - "gix", - "globset", - "hmac 0.12.1", - "httpmock", - "ignore", - "insta", - "ipnet", - "jiff", - "jiff-tzdb", - "libc", - "num-bigint", - "num-traits", - "percent-encoding", - "pollster", - "regex", - "regex-syntax", - "reqwest", - "serde", - "serde_json", - "sha2 0.10.9", - "similar", - "submilli-wasm", - "tempfile", - "tokio", - "tokio-rustls", - "unicode-ident", - "unicode-normalization", - "unicode-width", - "ureq", - "url", - "uuid", - "wasm-encoder 0.248.0", - "wasmparser 0.248.0", - "wast", - "zstd", -] - [[package]] name = "io-extras" version = "0.19.0" @@ -5436,7 +5385,6 @@ dependencies = [ "futures", "getrandom 0.2.17", "hmac 0.12.1", - "interpreter", "ipnet", "libc", "notify-debouncer-mini", @@ -5450,6 +5398,7 @@ dependencies = [ "sha2 0.10.9", "submilli-blueprint", "submilli-build", + "submilli-engine", "submilli-server", "submilli-shared", "submilli-wasm", @@ -5481,15 +5430,66 @@ dependencies = [ name = "submilli-build" version = "0.3.0" dependencies = [ - "interpreter", "serde", "serde_json", "serde_yml", "submilli-blueprint", + "submilli-engine", "tempfile", "toml", ] +[[package]] +name = "submilli-engine" +version = "0.3.0" +dependencies = [ + "async-trait", + "base64 0.22.1", + "cap-fs-ext", + "cap-primitives", + "cap-std", + "filetime", + "flate2", + "futures", + "getrandom 0.2.17", + "gimli 0.33.0", + "gix", + "globset", + "hmac 0.12.1", + "httpmock", + "ignore", + "insta", + "ipnet", + "jiff", + "jiff-tzdb", + "libc", + "num-bigint", + "num-traits", + "percent-encoding", + "pollster", + "regex", + "regex-syntax", + "reqwest", + "serde", + "serde_json", + "sha2 0.10.9", + "similar", + "submilli-wasm", + "tempfile", + "tokio", + "tokio-rustls", + "unicode-ident", + "unicode-normalization", + "unicode-width", + "ureq", + "url", + "uuid", + "wasm-encoder 0.248.0", + "wasmparser 0.248.0", + "wast", + "zstd", +] + [[package]] name = "submilli-server" version = "0.3.0" @@ -5503,7 +5503,6 @@ dependencies = [ "http-body-util", "httpmock", "insta", - "interpreter", "ipnet", "jiff", "libc", @@ -5520,6 +5519,7 @@ dependencies = [ "sqlx", "submilli-blueprint", "submilli-build", + "submilli-engine", "submilli-shared", "submilli-wasm", "subtle", @@ -5550,7 +5550,6 @@ dependencies = [ "futures", "http 1.4.0", "httpmock", - "interpreter", "rcgen", "reqwest", "rmcp", @@ -5561,6 +5560,7 @@ dependencies = [ "sse-stream", "submilli-blueprint", "submilli-build", + "submilli-engine", "tar", "tempfile", "tokio", diff --git a/crates/conformance/Cargo.toml b/crates/conformance/Cargo.toml index 2ce2745f8..fa40f8a12 100644 --- a/crates/conformance/Cargo.toml +++ b/crates/conformance/Cargo.toml @@ -10,6 +10,6 @@ publish.workspace = true workspace = true [dev-dependencies] -interpreter = { path = "../interpreter" } +interpreter = { package = "submilli-engine", path = "../interpreter" } tokio = { workspace = true, features = ["rt"] } wasmtime = { workspace = true } diff --git a/crates/interpreter/Cargo.toml b/crates/interpreter/Cargo.toml index b46965bc0..1112a1be6 100644 --- a/crates/interpreter/Cargo.toml +++ b/crates/interpreter/Cargo.toml @@ -1,10 +1,16 @@ [package] -name = "interpreter" +name = "submilli-engine" +description = "Compiles Submilli programs, a TypeScript subset, to WebAssembly and runs them under capability policy" version.workspace = true edition.workspace = true license.workspace = true repository.workspace = true -publish.workspace = true +readme = "README.md" +keywords = ["submilli", "typescript", "webassembly", "sandbox", "agents"] +categories = ["compilers", "wasm"] +publish = true +# Tests and their snapshots stay in the repository; the package is the library. +exclude = ["tests/**", "**/snapshots/**"] [lints] workspace = true diff --git a/crates/interpreter/README.md b/crates/interpreter/README.md new file mode 100644 index 000000000..6f1ebba18 --- /dev/null +++ b/crates/interpreter/README.md @@ -0,0 +1,12 @@ +# submilli-engine + +The engine behind [Submilli](https://github.com/submilli/submilli-runtime): it +compiles Submilli programs, a subset of TypeScript, to WebAssembly and runs +them in a sandbox where every effect goes through a capability check. + +An embedder chooses the standard library its programs see with `Stdlib`, and +supplies the host services through traits on the store: the policy +(`SecurityCheck`), HTTP, secrets, model calls (`LlmProvider`), sub-agents +(`AgentProvider`) and skills (`SkillProvider`), among others. + +Licensed under Apache-2.0. diff --git a/crates/interpreter/src/capability_derivation.rs b/crates/interpreter/src/capability_derivation.rs index 5d3eb4913..7e413ce55 100644 --- a/crates/interpreter/src/capability_derivation.rs +++ b/crates/interpreter/src/capability_derivation.rs @@ -156,7 +156,19 @@ fn binding_filter( return None; } let normalized = normalized_string(tag, field, value, warn_at, warnings); - format!("\"{}\"", escape(&normalized)) + let Some(literal) = filter_string_literal(&normalized) else { + if let Some(span) = warn_at { + warnings.push(warning( + span, + format!( + "`{field}` value contains `{VAR_PLACEHOLDER}`, which a filter reads as a variable reference; no static filter for `{}`", + tag.capability + ), + )); + } + return None; + }; + literal } StaticValue::Number(spelling) => spelling, StaticValue::Boolean(value) => value.to_string(), @@ -193,7 +205,7 @@ fn normalized_string( } fn field_normalization(capability: &str, field: &str) -> FieldNormalization { - capabilities::find(capability) + capabilities::find_any(capability) .and_then(|entry| entry.filter_fields.iter().find(|f| f.name == field)) .map_or(FieldNormalization::Verbatim, |f| f.normalization) } @@ -433,8 +445,34 @@ fn url_component(url: &str, component: &str) -> Option { Some(value) } +/// What a filter reads inside a quoted string as the start of a variable reference. +const VAR_PLACEHOLDER: &str = "${vars."; + +/// `value` as a quoted filter string, in the canonical form the policy parser +/// reads back as exactly `value`. `None` when no quoted string can spell it: the +/// parser reads `${vars.` inside quotes as a variable reference. +/// +/// Must agree with `submilli-blueprint`'s `quote_literal`, which the engine +/// cannot depend on. +#[doc(hidden)] +pub fn filter_string_literal(value: &str) -> Option { + (!value.contains(VAR_PLACEHOLDER)).then(|| format!("\"{}\"", escape(value))) +} + +/// The escapes the filter tokenizer reads inside a quoted string. fn escape(value: &str) -> String { - value.replace('\\', "\\\\").replace('"', "\\\"") + let mut out = String::with_capacity(value.len()); + for ch in value.chars() { + match ch { + '"' => out.push_str("\\\""), + '\\' => out.push_str("\\\\"), + '\n' => out.push_str("\\n"), + '\t' => out.push_str("\\t"), + '\r' => out.push_str("\\r"), + c => out.push(c), + } + } + out } fn number_literal(value: f64) -> String { @@ -537,6 +575,36 @@ mod tests { assert!(derived.warnings.is_empty(), "{:?}", derived.warnings); } + /// A filter reads `${vars.` inside quotes as a variable, so no literal can spell + /// such a value: the binding is dropped, with a warning, rather than written as + /// a variable reference the program never meant. + #[test] + fn literal_holding_a_variable_reference_contributes_no_filter() { + let (ta, tag, params, args) = first_doc_capability( + "/** @capability secrets.get { name } */\n\ + function callee(name: string): void { }\n\ + function main(): void { callee(\"${vars.who}\"); }\n", + ); + let derived = derive_call_site_capability(&tag, ¶ms, &ta, &args).unwrap(); + assert_eq!(derived.filter, None); + assert!(derived.known_bindings.is_empty()); + assert_eq!(derived.warnings.len(), 1, "{:?}", derived.warnings); + assert!(derived.warnings[0].message.contains("${vars.")); + } + + #[test] + fn string_literal_escapes_what_the_filter_tokenizer_reads() { + assert_eq!( + filter_string_literal("a\"b\\c\nd\te\rf").as_deref(), + Some("\"a\\\"b\\\\c\\nd\\te\\rf\""), + ); + assert_eq!(filter_string_literal("${vars.x}"), None); + assert_eq!( + filter_string_literal("${other}").as_deref(), + Some("\"${other}\"") + ); + } + #[test] fn non_literal_arg_retains_requirement_without_warning() { let (ta, tag, params, args) = first_doc_capability( diff --git a/crates/interpreter/src/codegen/snapshots/interpreter__codegen__tests__assert_false_throws_with_backtrace.snap b/crates/interpreter/src/codegen/snapshots/submilli_engine__codegen__tests__assert_false_throws_with_backtrace.snap similarity index 100% rename from crates/interpreter/src/codegen/snapshots/interpreter__codegen__tests__assert_false_throws_with_backtrace.snap rename to crates/interpreter/src/codegen/snapshots/submilli_engine__codegen__tests__assert_false_throws_with_backtrace.snap diff --git a/crates/interpreter/src/codegen/snapshots/interpreter__codegen__tests__closure_method_accessor_and_constructor_frames_are_named.snap b/crates/interpreter/src/codegen/snapshots/submilli_engine__codegen__tests__closure_method_accessor_and_constructor_frames_are_named.snap similarity index 100% rename from crates/interpreter/src/codegen/snapshots/interpreter__codegen__tests__closure_method_accessor_and_constructor_frames_are_named.snap rename to crates/interpreter/src/codegen/snapshots/submilli_engine__codegen__tests__closure_method_accessor_and_constructor_frames_are_named.snap diff --git a/crates/interpreter/src/codegen/snapshots/interpreter__codegen__tests__code_after_a_cast_keeps_its_backtrace_location.snap b/crates/interpreter/src/codegen/snapshots/submilli_engine__codegen__tests__code_after_a_cast_keeps_its_backtrace_location.snap similarity index 100% rename from crates/interpreter/src/codegen/snapshots/interpreter__codegen__tests__code_after_a_cast_keeps_its_backtrace_location.snap rename to crates/interpreter/src/codegen/snapshots/submilli_engine__codegen__tests__code_after_a_cast_keeps_its_backtrace_location.snap diff --git a/crates/interpreter/src/codegen/snapshots/interpreter__codegen__tests__dwarf_die_snapshot.snap b/crates/interpreter/src/codegen/snapshots/submilli_engine__codegen__tests__dwarf_die_snapshot.snap similarity index 100% rename from crates/interpreter/src/codegen/snapshots/interpreter__codegen__tests__dwarf_die_snapshot.snap rename to crates/interpreter/src/codegen/snapshots/submilli_engine__codegen__tests__dwarf_die_snapshot.snap diff --git a/crates/interpreter/src/codegen/snapshots/interpreter__codegen__tests__dwarf_line_program_covers_helper.snap b/crates/interpreter/src/codegen/snapshots/submilli_engine__codegen__tests__dwarf_line_program_covers_helper.snap similarity index 100% rename from crates/interpreter/src/codegen/snapshots/interpreter__codegen__tests__dwarf_line_program_covers_helper.snap rename to crates/interpreter/src/codegen/snapshots/submilli_engine__codegen__tests__dwarf_line_program_covers_helper.snap diff --git a/crates/interpreter/src/codegen/snapshots/interpreter__codegen__tests__dwarf_line_program_snapshot.snap b/crates/interpreter/src/codegen/snapshots/submilli_engine__codegen__tests__dwarf_line_program_snapshot.snap similarity index 100% rename from crates/interpreter/src/codegen/snapshots/interpreter__codegen__tests__dwarf_line_program_snapshot.snap rename to crates/interpreter/src/codegen/snapshots/submilli_engine__codegen__tests__dwarf_line_program_snapshot.snap diff --git a/crates/interpreter/src/codegen/snapshots/interpreter__codegen__tests__dwarf_subprograms_for_each_user_function.snap b/crates/interpreter/src/codegen/snapshots/submilli_engine__codegen__tests__dwarf_subprograms_for_each_user_function.snap similarity index 100% rename from crates/interpreter/src/codegen/snapshots/interpreter__codegen__tests__dwarf_subprograms_for_each_user_function.snap rename to crates/interpreter/src/codegen/snapshots/submilli_engine__codegen__tests__dwarf_subprograms_for_each_user_function.snap diff --git a/crates/interpreter/src/codegen/snapshots/interpreter__codegen__tests__failed_cast_backtrace_points_at_the_cast.snap b/crates/interpreter/src/codegen/snapshots/submilli_engine__codegen__tests__failed_cast_backtrace_points_at_the_cast.snap similarity index 100% rename from crates/interpreter/src/codegen/snapshots/interpreter__codegen__tests__failed_cast_backtrace_points_at_the_cast.snap rename to crates/interpreter/src/codegen/snapshots/submilli_engine__codegen__tests__failed_cast_backtrace_points_at_the_cast.snap diff --git a/crates/interpreter/src/codegen/snapshots/interpreter__codegen__tests__multi_frame_backtrace.snap b/crates/interpreter/src/codegen/snapshots/submilli_engine__codegen__tests__multi_frame_backtrace.snap similarity index 100% rename from crates/interpreter/src/codegen/snapshots/interpreter__codegen__tests__multi_frame_backtrace.snap rename to crates/interpreter/src/codegen/snapshots/submilli_engine__codegen__tests__multi_frame_backtrace.snap diff --git a/crates/interpreter/src/codegen/snapshots/interpreter__codegen__tests__three_frame_backtrace_labels_caller_in_middle.snap b/crates/interpreter/src/codegen/snapshots/submilli_engine__codegen__tests__three_frame_backtrace_labels_caller_in_middle.snap similarity index 100% rename from crates/interpreter/src/codegen/snapshots/interpreter__codegen__tests__three_frame_backtrace_labels_caller_in_middle.snap rename to crates/interpreter/src/codegen/snapshots/submilli_engine__codegen__tests__three_frame_backtrace_labels_caller_in_middle.snap diff --git a/crates/interpreter/src/compile.rs b/crates/interpreter/src/compile.rs index 7282a0783..ac99c0cdc 100644 --- a/crates/interpreter/src/compile.rs +++ b/crates/interpreter/src/compile.rs @@ -9,6 +9,7 @@ use std::time::{Duration, Instant}; use crate::compile_capabilities; use crate::compiler_error::{CompileError, CompilerFailure, CompilerStage}; use crate::runtime::prelude; +use crate::stdlib::Stdlib; use crate::typechecker::rules::check_script; use crate::typed_ast::TypedAst; use crate::{ @@ -75,7 +76,12 @@ impl ParsedScript { } if let Some(server) = module.strip_prefix("@mcp/") { imports.mcp_servers.insert(server.to_string()); - } else if stdlib_names.contains(module) { + } else if stdlib_names.contains(module) + || crate::stdlib::OptionalPackage::from_module_name(module).is_some() + { + // An optional package counts as stdlib whether or not the embedder + // enables it: a disabled one is reported as unavailable, never + // looked up as a registry package. imports.stdlib.insert(module.clone()); } else { imports.registry_packages.insert(module.clone()); @@ -256,7 +262,29 @@ pub fn typecheck_checked(source: &str, file: FileId) -> Result, fn typecheck_script(source: &str, file: FileId) -> Result, CompileError> { let parsed = parse_script(source, file); let stdlib_defs = runtime::stdlib_package_declarations(); - let (ta, diags, _timings) = front_end(source, &parsed, &stdlib_defs, &[])?; + typecheck_parsed_script(source, &parsed, &stdlib_defs) +} + +/// [`typecheck_checked`] for an already parsed script, against `stdlib_defs` +/// (see [`Stdlib::package_declarations`](crate::stdlib::Stdlib::package_declarations)). +pub fn typecheck_parsed_checked( + source: &str, + parsed: &ParsedScript, + stdlib_defs: &[PackageDeclaration], +) -> Result, CompileError> { + let file = parsed.file; + typecheck_parsed_script(source, parsed, stdlib_defs).map_err(|error| { + error.with_limit_span_cut(|span_file| (span_file == file).then_some(source)) + }) +} + +fn typecheck_parsed_script( + source: &str, + parsed: &ParsedScript, + stdlib_defs: &[PackageDeclaration], +) -> Result, CompileError> { + let file = parsed.file; + let (ta, diags, _timings) = front_end(source, parsed, stdlib_defs, &[])?; if has_errors(&diags) { return Err(diags.into()); } @@ -499,10 +527,32 @@ pub fn compile_package_with_transitive_checked( modules: &[PackageSourceModule<'_>], dependencies: &[&PackageDeclaration], transitive: &[&PackageDeclaration], +) -> Result { + compile_package_with_transitive_checked_for( + Stdlib::core(), + package_name, + root_module, + modules, + dependencies, + transitive, + ) +} + +/// [`compile_package_with_transitive_checked`] for an embedder that offers +/// `stdlib`, so the package may import its optional packages. +pub fn compile_package_with_transitive_checked_for( + stdlib: Stdlib, + package_name: &str, + root_module: ModulePath, + modules: &[PackageSourceModule<'_>], + dependencies: &[&PackageDeclaration], + transitive: &[&PackageDeclaration], ) -> Result { let mut sources = Sources::new(); + let stdlib_defs = stdlib.package_declarations(); compile_package_sources( &mut sources, + &stdlib_defs, package_name, root_module, modules, @@ -516,6 +566,7 @@ pub fn compile_package_with_transitive_checked( fn compile_package_sources( sources: &mut Sources, + stdlib_defs: &[PackageDeclaration], package_name: &str, root_module: ModulePath, modules: &[PackageSourceModule<'_>], @@ -552,7 +603,6 @@ fn compile_package_sources( .iter() .map(|(path, file, ast)| (path.clone(), *file, ast)) .collect(); - let stdlib_defs = runtime::stdlib_package_declarations(); let mut external_packages: BTreeMap = stdlib_defs .iter() .map(|defs| (defs.package_name.clone(), defs.clone())) @@ -593,7 +643,7 @@ fn compile_package_sources( compile_capabilities::derive_package_requirements( &declaration, &ta, - &stdlib_defs, + stdlib_defs, dependencies, transitive, ) @@ -633,7 +683,6 @@ fn compile_package_sources( fatal: Some(fatal), })?; - let stdlib_defs = runtime::stdlib_package_declarations(); let mut codegen_deps: Vec<&PackageDeclaration> = prelude_defs.iter().collect(); codegen_deps.extend(host_defs.iter()); codegen_deps.extend(internal_defs.iter()); diff --git a/crates/interpreter/src/lib.rs b/crates/interpreter/src/lib.rs index 07bf2c6a4..4e4eeb33f 100644 --- a/crates/interpreter/src/lib.rs +++ b/crates/interpreter/src/lib.rs @@ -81,6 +81,7 @@ pub use runtime::{RunResult, RuntimeConfig, dispatch_main_async, instantiate_pro pub use shape::Shape; pub use source::{ModulePath, SourceFile, Sources}; pub use span::{FileId, LineIndex, Span}; +pub use stdlib::{OptionalPackage, Stdlib}; pub use token::{Token, TokenKind}; pub use type_info::{FieldInfo, TypeInfo, TypeInfoId, TypeInfoIndex, TypeInfoKind, TypeInfoTable}; pub use typechecker::{capability_binding_type, capture, check, desugar, infer, infer_package}; diff --git a/crates/interpreter/src/packages.rs b/crates/interpreter/src/packages.rs index 081f36eeb..e633f0644 100644 --- a/crates/interpreter/src/packages.rs +++ b/crates/interpreter/src/packages.rs @@ -12,7 +12,7 @@ use std::collections::{BTreeMap, BTreeSet}; use std::fmt::Write; use crate::runtime::prelude::declaration::prelude_package_declaration; -use crate::stdlib::stdlib_package_declarations; +use crate::stdlib::Stdlib; use crate::types::escape_string_literal; use crate::{ ClassExtends, DocCapabilityBindingKind, DocCapabilityLiteral, DocComment, FileId, @@ -41,28 +41,199 @@ pub struct ModuleSummary { /// handle `@mcp/*` and unknown names). Every module resolves, opt-in ones such /// as `submilli:git` included; what a caller's scope shows is its own decision. pub fn docs(name: &str) -> Option { - user_modules() - .into_iter() - .find(|d| d.package_name == name) - .map(|defs| ModuleDoc { - name: defs.package_name.clone(), - description: module_description(&defs.package_name).to_string(), - declarations: render_declarations(&defs), - }) + Stdlib::core().docs(name) } /// Modules whose name, description, or an exported symbol contains `query` /// (case-insensitive). An empty query lists every module. pub fn search(query: &str) -> Vec { - let q = query.trim().to_lowercase(); - user_modules() - .iter() - .filter(|defs| matches_query(defs, &q)) - .map(|defs| ModuleSummary { - name: defs.package_name.clone(), - description: module_description(&defs.package_name).to_string(), - }) - .collect() + Stdlib::core().search(query) +} + +/// Discovery over the modules of a [`Stdlib`]: what the free functions above do +/// for [`Stdlib::core`], for any set. +impl Stdlib { + /// [`docs`] over the modules of this set. + pub fn docs(self, name: &str) -> Option { + user_modules(self) + .into_iter() + .find(|d| d.package_name == name) + .map(|defs| ModuleDoc { + name: defs.package_name.clone(), + description: module_description(&defs.package_name).to_string(), + declarations: render_declarations(&defs), + }) + } + + /// [`search`] over the modules of this set. + pub fn search(self, query: &str) -> Vec { + let q = query.trim().to_lowercase(); + user_modules(self) + .iter() + .filter(|defs| matches_query(defs, &q)) + .map(|defs| ModuleSummary { + name: defs.package_name.clone(), + description: module_description(&defs.package_name).to_string(), + }) + .collect() + } + + /// [`resolve`] over the modules of this set. + pub fn resolve(self, name: &str) -> Resolution { + if let Some(doc) = self.docs(name) { + return Resolution::Module(doc); + } + match builtin_lookup(name) { + BuiltinLookup::Found(declarations) => Resolution::Builtin { + name: name.to_string(), + declarations, + }, + BuiltinLookup::UnknownMember { + path, + member, + members, + } => Resolution::UnknownMember { + path, + member, + members, + }, + BuiltinLookup::Unknown => Resolution::Unknown, + } + } + + /// [`suggest_filtered`] over the modules of this set. + pub fn suggest_filtered( + self, + name: &str, + extra: &[String], + visible: impl Fn(&str) -> bool, + ) -> Option { + let mut candidates: Vec = self.search("").into_iter().map(|m| m.name).collect(); + let builtins = builtins(); + candidates.extend(builtins.types); + candidates.extend(builtins.namespaces); + candidates.extend(extra.iter().cloned()); + candidates.retain(|candidate| visible(candidate)); + + // Only the head segment is in question: `Temporel.Instant` misses because + // of `Temporel`, and the tail only inflates the distance past threshold. + let query = name.split('.').find(|s| !s.is_empty()).unwrap_or(name); + // `closest_match` floors its threshold at 2 edits, so a one- or two-character + // query sits within reach of an unrelated name. Below that length there is no + // signal to match on. + if query.len() < 3 { + return None; + } + + if let Some((_, replacement)) = OMITTED_GLOBALS + .iter() + .find(|(omitted, _)| omitted.eq_ignore_ascii_case(name)) + { + return Some((*replacement).to_string()); + } + + // Never offer the query back to the caller: a name that resolved nowhere is + // not its own repair, however it was spelled. + let is_self = |c: &String| c.eq_ignore_ascii_case(name); + if let Some(hit) = candidates + .iter() + .find(|c| !is_self(c) && name_tail(c).eq_ignore_ascii_case(query)) + { + return Some(hit.clone()); + } + // Match full names and scheme-stripped tails in one pass, so the globally + // closest key wins: `htp` should reach `submilli:http` (one edit from its + // tail) rather than whichever unrelated full name lands inside threshold. + let mut keys: Vec<(&str, &str)> = Vec::new(); + for c in candidates.iter().filter(|c| !is_self(c)) { + keys.push((c.as_str(), c.as_str())); + let tail = name_tail(c); + if tail != c.as_str() { + keys.push((tail, c.as_str())); + } + } + let hit = crate::did_you_mean::closest_match(query, keys.iter().map(|(k, _)| *k))?; + keys.iter() + .find(|(k, _)| *k == hit) + .map(|(_, owner)| (*owner).to_string()) + } + + /// [`catalog_filtered`] over the modules of this set. + pub fn catalog_filtered( + self, + extra: Vec, + visible: impl Fn(&str) -> bool, + ) -> Catalog { + let mut entries: Vec = self + .search("") + .into_iter() + .map(|m| CatalogEntry { + name: m.name, + source: SOURCE_STDLIB.to_string(), + description: m.description, + }) + .collect(); + entries.extend(extra); + entries.retain(|entry| visible(&entry.name)); + let remaining = entries.len().saturating_sub(CATALOG_LIMIT); + entries.truncate(CATALOG_LIMIT); + Catalog { entries, remaining } + } + + /// [`render_stdlib_d_ts`] for the modules of this set. + pub fn render_stdlib_d_ts(self) -> String { + let mut out = String::new(); + for defs in &editor_stdlib_modules(self) { + render_declare_module(&mut out, defs); + } + out.trim_end().to_string() + } + + /// [`render_packages_d_ts`] against the modules of this set. + pub fn render_packages_d_ts( + self, + packages: &[&PackageDeclaration], + context: &[&PackageDeclaration], + ) -> String { + let stdlib = editor_stdlib_modules(self); + let exporters = TypeExporters::new( + stdlib + .iter() + .chain(packages.iter().copied()) + .chain(context.iter().copied()), + ); + let globals = global_classes(); + let mut out = String::new(); + for defs in packages { + let _ = writeln!(out, "declare module \"{}\" {{", defs.package_name); + let imports = block_imports(defs, &exporters); + for (local, (public, module)) in &imports.types { + if local == public { + let _ = writeln!(out, " import type {{ {local} }} from \"{module}\";"); + } else { + let _ = writeln!( + out, + " import type {{ {public} as {local} }} from \"{module}\";" + ); + } + } + let parent = |extends: &ClassExtends| { + if let Some(local) = imports.parents.get(extends.parent.as_str()) { + return Some(ts_named_type(local, &extends.args)); + } + // A package value of the global's name would hide it; the class + // then renders without its parent rather than extend the value. + let global = globals.get(extends.parent.as_str())?; + if declares(defs, global, true) { + return None; + } + Some(ts_named_type(global, &extends.args)) + }; + render_ts_declarations(&mut out, defs, " ", "export ", &parent); + let _ = writeln!(out, "}}\n"); + } + out.trim_end().to_string() + } } /// The language built-ins always in scope without an `import`: headline prelude @@ -548,25 +719,7 @@ pub enum Resolution { /// Try `name` as a stdlib module, then as a built-in. Every discovery surface /// runs this same ladder so MCP, REST, and the CLI reach the same answer. pub fn resolve(name: &str) -> Resolution { - if let Some(doc) = docs(name) { - return Resolution::Module(doc); - } - match builtin_lookup(name) { - BuiltinLookup::Found(declarations) => Resolution::Builtin { - name: name.to_string(), - declarations, - }, - BuiltinLookup::UnknownMember { - path, - member, - members, - } => Resolution::UnknownMember { - path, - member, - members, - }, - BuiltinLookup::Unknown => Resolution::Unknown, - } + Stdlib::core().resolve(name) } /// A did-you-mean candidate for a name that resolved nowhere, drawn from the @@ -592,54 +745,7 @@ pub fn suggest_filtered( extra: &[String], visible: impl Fn(&str) -> bool, ) -> Option { - let mut candidates: Vec = search("").into_iter().map(|m| m.name).collect(); - let builtins = builtins(); - candidates.extend(builtins.types); - candidates.extend(builtins.namespaces); - candidates.extend(extra.iter().cloned()); - candidates.retain(|candidate| visible(candidate)); - - // Only the head segment is in question: `Temporel.Instant` misses because - // of `Temporel`, and the tail only inflates the distance past threshold. - let query = name.split('.').find(|s| !s.is_empty()).unwrap_or(name); - // `closest_match` floors its threshold at 2 edits, so a one- or two-character - // query sits within reach of an unrelated name. Below that length there is no - // signal to match on. - if query.len() < 3 { - return None; - } - - if let Some((_, replacement)) = OMITTED_GLOBALS - .iter() - .find(|(omitted, _)| omitted.eq_ignore_ascii_case(name)) - { - return Some((*replacement).to_string()); - } - - // Never offer the query back to the caller: a name that resolved nowhere is - // not its own repair, however it was spelled. - let is_self = |c: &String| c.eq_ignore_ascii_case(name); - if let Some(hit) = candidates - .iter() - .find(|c| !is_self(c) && name_tail(c).eq_ignore_ascii_case(query)) - { - return Some(hit.clone()); - } - // Match full names and scheme-stripped tails in one pass, so the globally - // closest key wins: `htp` should reach `submilli:http` (one edit from its - // tail) rather than whichever unrelated full name lands inside threshold. - let mut keys: Vec<(&str, &str)> = Vec::new(); - for c in candidates.iter().filter(|c| !is_self(c)) { - keys.push((c.as_str(), c.as_str())); - let tail = name_tail(c); - if tail != c.as_str() { - keys.push((tail, c.as_str())); - } - } - let hit = crate::did_you_mean::closest_match(query, keys.iter().map(|(k, _)| *k))?; - keys.iter() - .find(|(k, _)| *k == hit) - .map(|(_, owner)| (*owner).to_string()) + Stdlib::core().suggest_filtered(name, extra, visible) } /// The part of a package name after its scheme or scope: `submilli:http` and @@ -709,19 +815,7 @@ pub fn catalog(extra: Vec) -> Catalog { /// Filter before applying the catalog limit so omitted counts reflect visibility. pub fn catalog_filtered(extra: Vec, visible: impl Fn(&str) -> bool) -> Catalog { - let mut entries: Vec = search("") - .into_iter() - .map(|m| CatalogEntry { - name: m.name, - source: SOURCE_STDLIB.to_string(), - description: m.description, - }) - .collect(); - entries.extend(extra); - entries.retain(|entry| visible(&entry.name)); - let remaining = entries.len().saturating_sub(CATALOG_LIMIT); - entries.truncate(CATALOG_LIMIT); - Catalog { entries, remaining } + Stdlib::core().catalog_filtered(extra, visible) } /// Name the head, then list what it actually has, so the repair is one edit. @@ -780,8 +874,9 @@ fn resolve_ignore_case<'a>(keys: impl Iterator, name: &str) - } /// The stdlib modules an agent may import, minus internal plumbing. -fn user_modules() -> Vec { - stdlib_package_declarations() +fn user_modules(stdlib: Stdlib) -> Vec { + stdlib + .package_declarations() .into_iter() .filter(|d| d.package_name != INTERNAL_MODULE) .collect() @@ -791,6 +886,9 @@ fn user_modules() -> Vec { /// `packages.search`/`docs` and (manually mirrored) MVP.md §I. fn module_description(name: &str) -> &'static str { match name { + "submilli:agents" => { + "Harness sub-agents: run(agent, input), and list() to discover them." + } "submilli:crypto" => "Hashing, HMAC, and random bytes.", "submilli:code" => { "Workspace tools: numbered reads, search, glob, tree, anchored edits and unified diffs." @@ -806,6 +904,7 @@ fn module_description(name: &str) -> &'static str { "submilli:llm" => "Gated model calls: call/batch, and models() to discover them.", "submilli:secrets" => "Policy-gated access to Blueprint-declared secrets.", "submilli:session" => "Session-scoped key-value state: get/has/set/remove/list.", + "submilli:skills" => "Harness skills: list(), load(name) and readFile(name, path).", "submilli:url" => "URL parse/build and query-string handling. Pure compute.", "submilli:uuid" => "UUID v4/v7 generation and validation.", _ => "", @@ -851,11 +950,7 @@ pub fn render_declarations(defs: &PackageDeclaration) -> String { /// it — the editor should complete both; `build check` remains the gate /// against importing them elsewhere. pub fn render_stdlib_d_ts() -> String { - let mut out = String::new(); - for defs in &editor_stdlib_modules() { - render_declare_module(&mut out, defs); - } - out.trim_end().to_string() + Stdlib::core().render_stdlib_d_ts() } /// Render editor-facing TypeScript declarations for `packages`, one @@ -871,44 +966,7 @@ pub fn render_packages_d_ts( packages: &[&PackageDeclaration], context: &[&PackageDeclaration], ) -> String { - let stdlib = editor_stdlib_modules(); - let exporters = TypeExporters::new( - stdlib - .iter() - .chain(packages.iter().copied()) - .chain(context.iter().copied()), - ); - let globals = global_classes(); - let mut out = String::new(); - for defs in packages { - let _ = writeln!(out, "declare module \"{}\" {{", defs.package_name); - let imports = block_imports(defs, &exporters); - for (local, (public, module)) in &imports.types { - if local == public { - let _ = writeln!(out, " import type {{ {local} }} from \"{module}\";"); - } else { - let _ = writeln!( - out, - " import type {{ {public} as {local} }} from \"{module}\";" - ); - } - } - let parent = |extends: &ClassExtends| { - if let Some(local) = imports.parents.get(extends.parent.as_str()) { - return Some(ts_named_type(local, &extends.args)); - } - // A package value of the global's name would hide it; the class - // then renders without its parent rather than extend the value. - let global = globals.get(extends.parent.as_str())?; - if declares(defs, global, true) { - return None; - } - Some(ts_named_type(global, &extends.args)) - }; - render_ts_declarations(&mut out, defs, " ", "export ", &parent); - let _ = writeln!(out, "}}\n"); - } - out.trim_end().to_string() + Stdlib::core().render_packages_d_ts(packages, context) } /// The built-in classes a package's class may extend, such as `Error`, by @@ -925,8 +983,8 @@ fn global_classes() -> BTreeMap { .collect() } -fn editor_stdlib_modules() -> Vec { - let mut modules = user_modules(); +fn editor_stdlib_modules(stdlib: Stdlib) -> Vec { + let mut modules = user_modules(stdlib); modules.push(crate::stdlib::test::package_declaration()); modules.push(crate::stdlib::security::package_declaration()); modules.sort_by(|a, b| a.package_name.cmp(&b.package_name)); @@ -2746,9 +2804,17 @@ mod tests { assert!(builtin_no_import_note("Temporal").contains("import")); } + /// Every package any embedder can enable, so the declaration checks below + /// cover the optional packages too. + fn every_package() -> Stdlib { + crate::stdlib::OptionalPackage::ALL + .iter() + .fold(Stdlib::core(), |set, package| set.with(*package)) + } + #[test] fn stdlib_capability_tags_parse_cleanly() { - for module in user_modules() { + for module in user_modules(every_package()) { for symbol in module.values.values() { let Some(doc) = value_doc(&symbol.kind) else { continue; @@ -2768,7 +2834,7 @@ mod tests { #[test] fn every_importable_stdlib_symbol_has_docs() { - for module in user_modules() { + for module in user_modules(every_package()) { for symbol in module.values.values() { assert!( value_doc(&symbol.kind).is_some(), @@ -3348,8 +3414,8 @@ mod tests { #[test] fn stdlib_d_ts_declares_each_user_module() { - let docs = render_stdlib_d_ts(); - for module in user_modules() { + let docs = every_package().render_stdlib_d_ts(); + for module in user_modules(every_package()) { assert!( docs.contains(&format!("declare module \"{}\" {{", module.package_name)), "missing declare module for {}", diff --git a/crates/interpreter/src/runtime/agents.rs b/crates/interpreter/src/runtime/agents.rs new file mode 100644 index 000000000..52fb9c50d --- /dev/null +++ b/crates/interpreter/src/runtime/agents.rs @@ -0,0 +1,121 @@ +//! The embedder-supplied seam for `submilli:agents`: a program hands work to a +//! sub-agent and gets its result back. +//! +//! The engine owns the guest surface, the capability check and the call log; +//! the harness owns what an agent is and how it runs. Everything that crosses +//! [`AgentProvider`] is owned, serializable data, so a harness may implement it +//! in process or forward it over the wire. +//! +//! **The provider bounds the run.** The execution timeout is checked only while +//! Wasm runs, so it does not interrupt a pending host call: a sub-agent run +//! lasts as long as the provider lets it. Time limits, token spend, recursion +//! depth and cancellation of a child when its parent goes away are the +//! provider's to enforce. Dropping the future returned by +//! [`AgentProvider::run`] is how the engine cancels a run. + +use std::future::Future; +use std::pin::Pin; + +use serde::{Deserialize, Serialize}; + +use crate::stdlib::shared::truncated; + +pub const AGENTS_MODULE_NAME: &str = "submilli:agents"; + +/// One sub-agent run, as the program asked for it. +#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] +pub struct AgentRequest { + /// The agent's name, exactly as the program wrote it. Not checked against + /// [`AgentProvider::agents`]: an unknown name is the provider's + /// [`AgentCallError::NotFound`]. The policy decided on it as written, so a + /// provider must match it byte for byte — no case folding, Unicode + /// normalization or path canonicalization — and must not use it as a path. + pub agent: String, + /// The task for the agent. + pub input: String, + /// A JSON Schema the result must satisfy, present when the program wrote a + /// type argument. The engine checks the result against the type itself; the + /// schema tells the agent what shape to answer in. + pub schema: Option, + /// The package whose code made the call: `main` for the program itself. + pub caller: String, +} + +/// What a finished run hands back. +#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] +pub struct AgentOutcome { + /// The agent's final answer. When the request carried a schema, this is the + /// JSON text of a value meant to satisfy it. + pub text: String, + /// What the run cost, as far as the provider knows. Recorded in the call + /// log; the engine charges nothing for it. + pub usage: Option, +} + +/// Tokens a run spent. `None` means unknown, never zero. +#[derive(Debug, Clone, Default, PartialEq, Eq, Serialize, Deserialize)] +pub struct AgentUsage { + pub input_tokens: Option, + pub output_tokens: Option, +} + +/// One agent a program may hand work to. +#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] +pub struct AgentInfo { + pub name: String, + /// Operator-authored text on what the agent is for. The engine reduces it to + /// one bounded line before a program sees it. + pub description: Option, +} + +/// Why a run did not produce a result. Every message is safe to show the +/// program: none carries the input or the agent's output. +#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] +pub enum AgentCallError { + /// No [`AgentProvider`] is wired into the runtime. + NotConfigured, + /// The harness has no agent of this name. + NotFound { agent: String }, + /// The run started and failed. `message` is a fixed classification chosen by + /// the provider, not the agent's output. + Failed { agent: String, message: String }, + /// The run was stopped before it finished. + Cancelled { agent: String }, +} + +impl std::fmt::Display for AgentCallError { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + match self { + Self::NotConfigured => write!( + f, + "no agent provider is configured: this harness does not run sub-agents" + ), + Self::NotFound { agent } => write!( + f, + "no agent named `{}`; call `list()` for the agents you may use", + truncated(agent) + ), + Self::Failed { agent, message } => { + write!(f, "agent `{}` failed: {message}", truncated(agent)) + } + Self::Cancelled { agent } => write!(f, "agent `{}` was cancelled", truncated(agent)), + } + } +} + +impl std::error::Error for AgentCallError {} + +/// The harness side of `submilli:agents`. +pub trait AgentProvider: Send + Sync { + /// Run `request.agent` on `request.input` and return its final answer. + fn run<'a>( + &'a self, + request: AgentRequest, + ) -> Pin> + Send + 'a>>; + + /// Every agent the harness offers. The engine removes the ones the policy + /// denies to the caller before the program sees the list. + fn agents<'a>( + &'a self, + ) -> Pin, AgentCallError>> + Send + 'a>>; +} diff --git a/crates/interpreter/src/runtime/call_log.rs b/crates/interpreter/src/runtime/call_log.rs index f6b8fb461..0440cab48 100644 --- a/crates/interpreter/src/runtime/call_log.rs +++ b/crates/interpreter/src/runtime/call_log.rs @@ -536,6 +536,8 @@ mod tests { "llm.call", "embedding.embed", "mcp.", + "agent.run", + "skill.load", ]; /// The request is the decision's context; the response is kept for whole reads and /// session reads. @@ -563,7 +565,7 @@ mod tests { #[test] fn every_capability_has_a_recording_class() { - for group in crate::stdlib::capabilities::catalog() { + for group in crate::stdlib::capabilities::all_groups() { for capability in group.capabilities { let classes = [REQUEST_AND_RESPONSE, RESPONSE_ONLY, TIMING_ONLY] .iter() @@ -580,7 +582,7 @@ mod tests { #[test] fn every_gating_module_ends_the_calls_it_begins() { - for group in crate::stdlib::capabilities::catalog() { + for group in crate::stdlib::capabilities::all_groups() { // The catalog names `@mcp/` imports; their calls dispatch through the // internal MCP module. let module = match group.module { @@ -614,6 +616,14 @@ mod tests { /// host functions run it. `None` marks the shared helpers: they run inside whichever /// module calls them. const CALLING_FILES: &[(&str, Option<&str>)] = &[ + ( + "src/stdlib/agents/mod.rs", + Some(crate::stdlib::agents::MODULE_NAME), + ), + ( + "src/stdlib/skills/mod.rs", + Some(crate::stdlib::skills::MODULE_NAME), + ), ( "src/stdlib/code/mod.rs", Some(crate::stdlib::code::MODULE_NAME), diff --git a/crates/interpreter/src/runtime/host.rs b/crates/interpreter/src/runtime/host.rs index 7681f9485..eca0537cd 100644 --- a/crates/interpreter/src/runtime/host.rs +++ b/crates/interpreter/src/runtime/host.rs @@ -16,6 +16,7 @@ pub(crate) use crate::runtime::intrinsic_types::{ }; use crate::runtime::number::{parse_float_js, parse_int_with_radix, string_to_number_js}; use crate::runtime::{StoreData, read_submilli_string}; +use crate::stdlib::Stdlib; use crate::{PackageDeclaration, Param, Span, Type, ValueKind, ValueSymbol}; pub const NUMBER_MODULE_NAME: &str = "submilli:number"; @@ -31,13 +32,30 @@ pub async fn install_async( linker: &mut Linker, store: &mut Store, ) -> wasmtime::Result<()> { - install_host_functions(linker)?; + install_async_for(linker, store, Stdlib::core()).await +} + +/// [`install_async`] for an embedder that offers `stdlib`. +pub async fn install_async_for( + linker: &mut Linker, + store: &mut Store, + stdlib: Stdlib, +) -> wasmtime::Result<()> { + install_host_functions_for(linker, stdlib)?; install_store_bound(linker, store) } pub fn install_host_functions(linker: &mut Linker) -> wasmtime::Result<()> { + install_host_functions_for(linker, Stdlib::core()) +} + +/// [`install_host_functions`] for an embedder that offers `stdlib`. +pub fn install_host_functions_for( + linker: &mut Linker, + stdlib: Stdlib, +) -> wasmtime::Result<()> { install_core_host_functions(linker)?; - crate::stdlib::install_host_functions(linker)?; + stdlib.install_host_functions(linker)?; super::mcp::install_mcp_async(linker)?; Ok(()) } @@ -1656,6 +1674,8 @@ pub(crate) const CALL_MODULES: &[&str] = &[ crate::stdlib::git::MODULE_NAME, crate::stdlib::code::MODULE_NAME, crate::stdlib::security::MODULE_NAME, + crate::stdlib::agents::MODULE_NAME, + crate::stdlib::skills::MODULE_NAME, super::mcp::MCP_MODULE_NAME, ]; diff --git a/crates/interpreter/src/runtime/mod.rs b/crates/interpreter/src/runtime/mod.rs index 3b5830d6e..244879119 100644 --- a/crates/interpreter/src/runtime/mod.rs +++ b/crates/interpreter/src/runtime/mod.rs @@ -1,5 +1,6 @@ //! Wasmtime engine configuration for Submilli. +pub mod agents; pub(crate) mod array_storage; pub mod blocking; pub mod call_log; @@ -23,10 +24,15 @@ pub mod prelude; pub mod secrets; pub mod security; pub mod session_kv; +pub mod skills; pub mod token_ledger; pub mod vfs; pub mod watchdog; +pub use agents::{ + AGENTS_MODULE_NAME, AgentCallError, AgentInfo, AgentOutcome, AgentProvider, AgentRequest, + AgentUsage, +}; pub use call_log::{BodyCopy, CallOutcome, CallRecord, ModelUsage, PayloadRecord}; pub use decision::{ CallSite, CallTicket, DecisionAction, DecisionCause, DecisionExplanation, DecisionLog, @@ -44,8 +50,9 @@ pub use embedding::{ pub use exec::{RunResult, dispatch_main_async, instantiate_program_async}; pub use host::{ INTERNAL_MODULE_NAME, NUMBER_MODULE_NAME, host_package_declarations, - install_async as install_runtime_async, + install_async as install_runtime_async, install_async_for as install_runtime_async_for, install_host_functions as install_runtime_host_functions, + install_host_functions_for as install_runtime_host_functions_for, install_store_bound as install_runtime_store_bound, internal_host_package_declarations, stdlib_package_declarations, }; @@ -72,6 +79,7 @@ pub use session_kv::{ InMemorySessionKv, SessionKvEntry, SessionKvError, SessionKvLimitKind, SessionKvLimits, SessionKvPage, SessionKvStore, SharedKvBudget, }; +pub use skills::{SKILLS_MODULE_NAME, Skill, SkillError, SkillInfo, SkillProvider}; pub use vfs::{ Access, CopyDirError, MAX_MEASURED_DEPTH, MountError, MountSpec, Vfs, VfsMode, copy_host_dir, copy_host_subdir, measure_dir, measure_host_dir, measure_host_dir_skipping_vanished, @@ -153,6 +161,14 @@ pub struct StoreData { /// becoming per-execution scratch state that no later `execute` can read — /// the guest surface reports that as a configuration error. pub session_kv: Option>, + /// The harness `submilli:agents` runs sub-agents through, present only when + /// the embedder wires one. `None` makes every call a catchable + /// [`AgentCallError::NotConfigured`]. + pub agent_provider: Option>, + /// The harness `submilli:skills` reads skills through, present only when the + /// embedder wires one. `None` makes every call a catchable + /// [`SkillError::NotConfigured`]. + pub skill_provider: Option>, /// Embedder sink for host-operation metrics (HTTP transport latencies). /// Defaults to [`NoopMetricsSink`]; the server installs a Sentry-backed one. pub metrics: Arc, @@ -269,6 +285,8 @@ impl StoreData { embedding_provider: None, embedding_budget: None, session_kv: None, + agent_provider: None, + skill_provider: None, metrics: Arc::new(metrics::NoopMetricsSink), tenant_limits: TenantLimits::new(max_store_bytes), host_fuel: 0, diff --git a/crates/interpreter/src/runtime/skills.rs b/crates/interpreter/src/runtime/skills.rs new file mode 100644 index 000000000..b93bd0029 --- /dev/null +++ b/crates/interpreter/src/runtime/skills.rs @@ -0,0 +1,133 @@ +//! The embedder-supplied seam for `submilli:skills`: a program reads the skills +//! the harness offers, their instructions and the files bundled with them. +//! +//! Where skills come from is the harness's choice. Everything that crosses +//! [`SkillProvider`] is owned, serializable data, so a harness may implement it +//! in process or forward it over the wire. + +use std::future::Future; +use std::pin::Pin; + +use serde::{Deserialize, Serialize}; + +use crate::stdlib::shared::truncated; + +pub const SKILLS_MODULE_NAME: &str = "submilli:skills"; + +/// One skill, as a listing shows it. +#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] +pub struct SkillInfo { + pub name: String, + /// What the skill is for. The engine reduces it to one bounded line before a + /// program sees it in a listing. + pub description: Option, +} + +/// A loaded skill. +#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] +pub struct Skill { + pub name: String, + pub description: Option, + /// The skill's instructions. + pub content: String, +} + +/// Why a skill or one of its files could not be read. Every message is safe to +/// show the program. +#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] +pub enum SkillError { + /// No [`SkillProvider`] is wired into the runtime. + NotConfigured, + /// The harness has no skill of this name. + NotFound { name: String }, + /// The skill has no file at this path. + FileNotFound { name: String, path: String }, + /// The name is not one skill name. The engine refuses these before asking + /// the provider. + InvalidName { name: String }, + /// The path is not a relative path inside the skill. The engine refuses + /// these before asking the provider; a provider may refuse more. + InvalidPath { path: String }, + /// The provider answered `load(name)` with a skill of another name. The + /// engine refuses it: the policy decided on `name`. + WrongSkill { name: String }, + /// Reading failed for another reason. `message` is a fixed classification + /// chosen by the provider. + Failed { message: String }, +} + +impl std::fmt::Display for SkillError { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + match self { + Self::NotConfigured => write!( + f, + "no skill provider is configured: this harness does not offer skills" + ), + Self::NotFound { name } => write!( + f, + "no skill named `{}`; call `list()` for the skills you may load", + truncated(name) + ), + Self::FileNotFound { name, path } => write!( + f, + "skill `{}` has no file `{}`", + truncated(name), + truncated(path) + ), + Self::InvalidName { name } => write!( + f, + "`{}` is not a skill name: use a name as `list()` returns it — one segment of \ + at most 4096 bytes, not empty, `.` or `..`, and without `/`, `\\` or NUL", + truncated(name) + ), + Self::InvalidPath { path } => write!( + f, + "`{}` is not a path inside a skill: use a relative path of at most 4096 bytes \ + with `/` separators, no empty, `.` or `..` segment, and no `\\`, `:` or NUL", + truncated(path) + ), + Self::WrongSkill { name } => write!( + f, + "the harness answered `{}` with a different skill", + truncated(name) + ), + Self::Failed { message } => write!(f, "{message}"), + } + } +} + +impl std::error::Error for SkillError {} + +/// The harness side of `submilli:skills`. +pub trait SkillProvider: Send + Sync { + /// Every skill the harness offers, each under a name [`Self::load`] accepts. + /// The engine removes the ones the policy denies to the caller, and any whose + /// name it would refuse, before the program sees the list. + fn list<'a>( + &'a self, + ) -> Pin, SkillError>> + Send + 'a>>; + + /// The skill named `name`, whose returned `name` must be exactly `name`. + /// + /// `name` comes from the program and the policy decided on it as written, so + /// match it byte for byte: no case folding, Unicode normalization or path + /// canonicalization, or a policy that denies `secrets` would let `Secrets` + /// through. The engine passes only one segment of at most 4096 bytes that is + /// not empty, `.` or `..` and has no `/`, `\` or NUL; it may contain `:`, so + /// never use it as a path. + fn load<'a>( + &'a self, + name: &'a str, + ) -> Pin> + Send + 'a>>; + + /// The text of the file at `path` inside skill `name`. `name` is checked and + /// must be matched as for [`Self::load`]. `path` is relative, uses `/` + /// separators, has no empty, `.` or `..` segment and no `\`, `:` or NUL, and + /// is at most 4096 bytes. The engine does not decode it: a provider that + /// percent-decodes must check the decoded path again. + fn read_file<'a>( + &'a self, + name: &'a str, + path: &'a str, + ) -> Pin> + Send + 'a>>; +} diff --git a/crates/interpreter/src/snapshots/interpreter__asi__tests__mvp_fixture_snapshot.snap b/crates/interpreter/src/snapshots/submilli_engine__asi__tests__mvp_fixture_snapshot.snap similarity index 100% rename from crates/interpreter/src/snapshots/interpreter__asi__tests__mvp_fixture_snapshot.snap rename to crates/interpreter/src/snapshots/submilli_engine__asi__tests__mvp_fixture_snapshot.snap diff --git a/crates/interpreter/src/snapshots/interpreter__diagnostics__tests__render_clamps_at_eof.snap b/crates/interpreter/src/snapshots/submilli_engine__diagnostics__tests__render_clamps_at_eof.snap similarity index 100% rename from crates/interpreter/src/snapshots/interpreter__diagnostics__tests__render_clamps_at_eof.snap rename to crates/interpreter/src/snapshots/submilli_engine__diagnostics__tests__render_clamps_at_eof.snap diff --git a/crates/interpreter/src/snapshots/interpreter__diagnostics__tests__render_clamps_at_sof.snap b/crates/interpreter/src/snapshots/submilli_engine__diagnostics__tests__render_clamps_at_sof.snap similarity index 100% rename from crates/interpreter/src/snapshots/interpreter__diagnostics__tests__render_clamps_at_sof.snap rename to crates/interpreter/src/snapshots/submilli_engine__diagnostics__tests__render_clamps_at_sof.snap diff --git a/crates/interpreter/src/snapshots/interpreter__diagnostics__tests__render_multi_note_error_with_gutter_alignment.snap b/crates/interpreter/src/snapshots/submilli_engine__diagnostics__tests__render_multi_note_error_with_gutter_alignment.snap similarity index 100% rename from crates/interpreter/src/snapshots/interpreter__diagnostics__tests__render_multi_note_error_with_gutter_alignment.snap rename to crates/interpreter/src/snapshots/submilli_engine__diagnostics__tests__render_multi_note_error_with_gutter_alignment.snap diff --git a/crates/interpreter/src/snapshots/interpreter__diagnostics__tests__render_multiline_span_carets.snap b/crates/interpreter/src/snapshots/submilli_engine__diagnostics__tests__render_multiline_span_carets.snap similarity index 100% rename from crates/interpreter/src/snapshots/interpreter__diagnostics__tests__render_multiline_span_carets.snap rename to crates/interpreter/src/snapshots/submilli_engine__diagnostics__tests__render_multiline_span_carets.snap diff --git a/crates/interpreter/src/snapshots/interpreter__diagnostics__tests__render_multiple_helps.snap b/crates/interpreter/src/snapshots/submilli_engine__diagnostics__tests__render_multiple_helps.snap similarity index 100% rename from crates/interpreter/src/snapshots/interpreter__diagnostics__tests__render_multiple_helps.snap rename to crates/interpreter/src/snapshots/submilli_engine__diagnostics__tests__render_multiple_helps.snap diff --git a/crates/interpreter/src/snapshots/interpreter__diagnostics__tests__render_single_line_error.snap b/crates/interpreter/src/snapshots/submilli_engine__diagnostics__tests__render_single_line_error.snap similarity index 100% rename from crates/interpreter/src/snapshots/interpreter__diagnostics__tests__render_single_line_error.snap rename to crates/interpreter/src/snapshots/submilli_engine__diagnostics__tests__render_single_line_error.snap diff --git a/crates/interpreter/src/snapshots/interpreter__diagnostics__tests__render_warning_severity.snap b/crates/interpreter/src/snapshots/submilli_engine__diagnostics__tests__render_warning_severity.snap similarity index 100% rename from crates/interpreter/src/snapshots/interpreter__diagnostics__tests__render_warning_severity.snap rename to crates/interpreter/src/snapshots/submilli_engine__diagnostics__tests__render_warning_severity.snap diff --git a/crates/interpreter/src/snapshots/interpreter__diagnostics__tests__render_with_help.snap b/crates/interpreter/src/snapshots/submilli_engine__diagnostics__tests__render_with_help.snap similarity index 100% rename from crates/interpreter/src/snapshots/interpreter__diagnostics__tests__render_with_help.snap rename to crates/interpreter/src/snapshots/submilli_engine__diagnostics__tests__render_with_help.snap diff --git a/crates/interpreter/src/snapshots/interpreter__diagnostics__tests__render_with_help_and_notes.snap b/crates/interpreter/src/snapshots/submilli_engine__diagnostics__tests__render_with_help_and_notes.snap similarity index 100% rename from crates/interpreter/src/snapshots/interpreter__diagnostics__tests__render_with_help_and_notes.snap rename to crates/interpreter/src/snapshots/submilli_engine__diagnostics__tests__render_with_help_and_notes.snap diff --git a/crates/interpreter/src/snapshots/interpreter__diagnostics__tests__render_zero_length_span_at_eof.snap b/crates/interpreter/src/snapshots/submilli_engine__diagnostics__tests__render_zero_length_span_at_eof.snap similarity index 100% rename from crates/interpreter/src/snapshots/interpreter__diagnostics__tests__render_zero_length_span_at_eof.snap rename to crates/interpreter/src/snapshots/submilli_engine__diagnostics__tests__render_zero_length_span_at_eof.snap diff --git a/crates/interpreter/src/snapshots/interpreter__lexer__tests__renders_bare_exponent_diagnostic.snap b/crates/interpreter/src/snapshots/submilli_engine__lexer__tests__renders_bare_exponent_diagnostic.snap similarity index 100% rename from crates/interpreter/src/snapshots/interpreter__lexer__tests__renders_bare_exponent_diagnostic.snap rename to crates/interpreter/src/snapshots/submilli_engine__lexer__tests__renders_bare_exponent_diagnostic.snap diff --git a/crates/interpreter/src/snapshots/interpreter__lexer__tests__renders_unterminated_string_diagnostic.snap b/crates/interpreter/src/snapshots/submilli_engine__lexer__tests__renders_unterminated_string_diagnostic.snap similarity index 100% rename from crates/interpreter/src/snapshots/interpreter__lexer__tests__renders_unterminated_string_diagnostic.snap rename to crates/interpreter/src/snapshots/submilli_engine__lexer__tests__renders_unterminated_string_diagnostic.snap diff --git a/crates/interpreter/src/snapshots/interpreter__lexer__tests__renders_unterminated_template_diagnostic.snap b/crates/interpreter/src/snapshots/submilli_engine__lexer__tests__renders_unterminated_template_diagnostic.snap similarity index 100% rename from crates/interpreter/src/snapshots/interpreter__lexer__tests__renders_unterminated_template_diagnostic.snap rename to crates/interpreter/src/snapshots/submilli_engine__lexer__tests__renders_unterminated_template_diagnostic.snap diff --git a/crates/interpreter/src/snapshots/interpreter__lexer__tests__tokenize_mvp_fixture.snap b/crates/interpreter/src/snapshots/submilli_engine__lexer__tests__tokenize_mvp_fixture.snap similarity index 100% rename from crates/interpreter/src/snapshots/interpreter__lexer__tests__tokenize_mvp_fixture.snap rename to crates/interpreter/src/snapshots/submilli_engine__lexer__tests__tokenize_mvp_fixture.snap diff --git a/crates/interpreter/src/snapshots/interpreter__parser__tests__full_precedence_chain_snapshot.snap b/crates/interpreter/src/snapshots/submilli_engine__parser__tests__full_precedence_chain_snapshot.snap similarity index 100% rename from crates/interpreter/src/snapshots/interpreter__parser__tests__full_precedence_chain_snapshot.snap rename to crates/interpreter/src/snapshots/submilli_engine__parser__tests__full_precedence_chain_snapshot.snap diff --git a/crates/interpreter/src/snapshots/interpreter__parser__tests__parse_sequence_of_atoms.snap b/crates/interpreter/src/snapshots/submilli_engine__parser__tests__parse_sequence_of_atoms.snap similarity index 100% rename from crates/interpreter/src/snapshots/interpreter__parser__tests__parse_sequence_of_atoms.snap rename to crates/interpreter/src/snapshots/submilli_engine__parser__tests__parse_sequence_of_atoms.snap diff --git a/crates/interpreter/src/snapshots/interpreter__parser__tests__snapshot_else_if_chain.snap b/crates/interpreter/src/snapshots/submilli_engine__parser__tests__snapshot_else_if_chain.snap similarity index 100% rename from crates/interpreter/src/snapshots/interpreter__parser__tests__snapshot_else_if_chain.snap rename to crates/interpreter/src/snapshots/submilli_engine__parser__tests__snapshot_else_if_chain.snap diff --git a/crates/interpreter/src/snapshots/interpreter__parser__tests__snapshot_function_declaration.snap b/crates/interpreter/src/snapshots/submilli_engine__parser__tests__snapshot_function_declaration.snap similarity index 100% rename from crates/interpreter/src/snapshots/interpreter__parser__tests__snapshot_function_declaration.snap rename to crates/interpreter/src/snapshots/submilli_engine__parser__tests__snapshot_function_declaration.snap diff --git a/crates/interpreter/src/snapshots/interpreter__parser__tests__snapshot_function_with_if_and_return.snap b/crates/interpreter/src/snapshots/submilli_engine__parser__tests__snapshot_function_with_if_and_return.snap similarity index 100% rename from crates/interpreter/src/snapshots/interpreter__parser__tests__snapshot_function_with_if_and_return.snap rename to crates/interpreter/src/snapshots/submilli_engine__parser__tests__snapshot_function_with_if_and_return.snap diff --git a/crates/interpreter/src/snapshots/interpreter__parser__tests__snapshot_function_with_while.snap b/crates/interpreter/src/snapshots/submilli_engine__parser__tests__snapshot_function_with_while.snap similarity index 100% rename from crates/interpreter/src/snapshots/interpreter__parser__tests__snapshot_function_with_while.snap rename to crates/interpreter/src/snapshots/submilli_engine__parser__tests__snapshot_function_with_while.snap diff --git a/crates/interpreter/src/snapshots/interpreter__parser__tests__snapshot_mixed_declarations.snap b/crates/interpreter/src/snapshots/submilli_engine__parser__tests__snapshot_mixed_declarations.snap similarity index 100% rename from crates/interpreter/src/snapshots/interpreter__parser__tests__snapshot_mixed_declarations.snap rename to crates/interpreter/src/snapshots/submilli_engine__parser__tests__snapshot_mixed_declarations.snap diff --git a/crates/interpreter/src/snapshots/interpreter__parser__tests__snapshot_multi_call_program.snap b/crates/interpreter/src/snapshots/submilli_engine__parser__tests__snapshot_multi_call_program.snap similarity index 100% rename from crates/interpreter/src/snapshots/interpreter__parser__tests__snapshot_multi_call_program.snap rename to crates/interpreter/src/snapshots/submilli_engine__parser__tests__snapshot_multi_call_program.snap diff --git a/crates/interpreter/src/snapshots/interpreter__parser__tests__snapshot_multi_error_render.snap b/crates/interpreter/src/snapshots/submilli_engine__parser__tests__snapshot_multi_error_render.snap similarity index 100% rename from crates/interpreter/src/snapshots/interpreter__parser__tests__snapshot_multi_error_render.snap rename to crates/interpreter/src/snapshots/submilli_engine__parser__tests__snapshot_multi_error_render.snap diff --git a/crates/interpreter/src/snapshots/interpreter__parser__tests__snapshot_object_and_array_literals.snap b/crates/interpreter/src/snapshots/submilli_engine__parser__tests__snapshot_object_and_array_literals.snap similarity index 100% rename from crates/interpreter/src/snapshots/interpreter__parser__tests__snapshot_object_and_array_literals.snap rename to crates/interpreter/src/snapshots/submilli_engine__parser__tests__snapshot_object_and_array_literals.snap diff --git a/crates/interpreter/src/snapshots/interpreter__parser__tests__snapshot_union_type_mix.snap b/crates/interpreter/src/snapshots/submilli_engine__parser__tests__snapshot_union_type_mix.snap similarity index 100% rename from crates/interpreter/src/snapshots/interpreter__parser__tests__snapshot_union_type_mix.snap rename to crates/interpreter/src/snapshots/submilli_engine__parser__tests__snapshot_union_type_mix.snap diff --git a/crates/interpreter/src/span.rs b/crates/interpreter/src/span.rs index d3b43eb9c..30501e49b 100644 --- a/crates/interpreter/src/span.rs +++ b/crates/interpreter/src/span.rs @@ -14,7 +14,7 @@ use serde::{Deserialize, Serialize}; pub struct FileId(pub u32); impl FileId { - pub const FIRST_RESERVED: u32 = u32::MAX - 22; + pub const FIRST_RESERVED: u32 = u32::MAX - 24; pub const COMPILER: FileId = FileId(Self::FIRST_RESERVED); pub const PRELUDE: FileId = FileId(u32::MAX); @@ -39,6 +39,8 @@ impl FileId { pub const EMBEDDING: FileId = FileId(u32::MAX - 21); pub const CODE: FileId = FileId(u32::MAX - 20); pub const GIT: FileId = FileId(u32::MAX - 19); + pub const AGENTS: FileId = FileId(u32::MAX - 22); + pub const SKILLS: FileId = FileId(u32::MAX - 23); /// Virtual display path for a reserved (prelude/stdlib) id, e.g. /// `submilli:fs`. Returns `None` for ordinary user/script files, which @@ -67,6 +69,8 @@ impl FileId { FileId::SECRETS => "submilli:secrets", FileId::SESSION => "submilli:session", FileId::LLM => "submilli:llm", + FileId::AGENTS => "submilli:agents", + FileId::SKILLS => "submilli:skills", FileId::TEST => "submilli:test", _ => return None, }) diff --git a/crates/interpreter/src/stdlib/agents/declaration.rs b/crates/interpreter/src/stdlib/agents/declaration.rs new file mode 100644 index 000000000..847a6c4b3 --- /dev/null +++ b/crates/interpreter/src/stdlib/agents/declaration.rs @@ -0,0 +1,159 @@ +//! Type surface of `submilli:agents`. +//! +//! `run` is declared generic and typed the way `llm.call` is: the typechecker +//! matches it by mangled name ([`is_checked_run`]), fills the trailing `schema` +//! parameter with the JSON Schema emitted from `T`, and wraps the result in a +//! structural check against `T`. Without a type argument `T` is `string`, the +//! agent's final answer as text. See `stdlib/llm/declaration.rs` for why the +//! generic declaration is sound only together with that interception. + +use std::collections::BTreeMap; + +use crate::{ + Dispatch, PackageDeclaration, Param, PropertySig, Span, Type, TypeKind, TypeSymbol, ValueKind, + ValueSymbol, +}; + +use super::MODULE_NAME; + +const RUN: &str = "run"; +const RUN_TYPE_PARAM: &str = "T"; + +/// Whether `mangled` is the typed `run`, so the typechecker's interception +/// matches however the symbol was imported. +pub fn is_checked_run(mangled: &crate::MangledName) -> bool { + *mangled == crate::mangle::package_symbol(MODULE_NAME, RUN) +} + +pub fn package_declaration() -> PackageDeclaration { + let mut defs = PackageDeclaration::with_package(MODULE_NAME); + insert_agent_info_interface(&mut defs); + insert_fn( + &mut defs, + RUN, + vec![RUN_TYPE_PARAM.to_string()], + vec![ + Param::new("agent", Type::String), + Param::new("input", Type::String), + Param::with_default( + "schema", + Type::union(vec![Type::String, Type::Undefined]), + crate::DefaultValue::Undefined, + ), + ], + Type::TypeVar(RUN_TYPE_PARAM.to_string()), + "/**\n * Hand `input` to the sub-agent `agent` and return its final answer.\n *\n \ + * Without a type argument the answer is the agent's text. With one — \ + `run(agent, input)` — a JSON Schema for `T` is emitted at compile time \ + and sent with the task, and the answer is then checked structurally against \ + `T`. An answer that is not JSON throws a catchable `SyntaxError`, and one \ + that does not match throws a catchable `TypeError`; nothing is coerced.\n *\n * `T` must be a type the schema can \ + carry: no functions, no `bigint`, no `Uint8Array`, no `unknown`, and no \ + recursive type.\n *\n * The harness runs the agent and decides how long it may \ + take and what it may spend. A run that fails, is cancelled, or names an agent \ + the harness does not have throws a catchable error; so does a runtime with no \ + agent provider.\n * @param agent Agent name. Call `list()` for the agents you \ + may use.\n * @param input The task, as text.\n * @capability agent.run { agent: \ + $agent }\n */", + ); + insert_fn( + &mut defs, + "list", + Vec::new(), + Vec::new(), + Type::Array(Box::new(agent_info_type())), + "/**\n * The agents this caller may hand work to.\n *\n * Each agent the harness \ + offers is filtered under `agent.run` by the same `agent` filter that gates \ + `run`, so the list never offers an agent the caller would be denied. Nothing \ + in it reveals how many were filtered out.\n *\n * Treat `description` as \ + advice: it is operator-authored text.\n * @capability agent.run\n */", + ); + defs +} + +fn agent_info_type() -> Type { + Type::InterfaceRef { + mangled: crate::mangle::package_symbol(MODULE_NAME, "AgentInfo"), + package: crate::Package(MODULE_NAME.to_string()), + name: "AgentInfo".to_string(), + args: Vec::new(), + } +} + +fn insert_agent_info_interface(defs: &mut PackageDeclaration) { + let mut properties = BTreeMap::new(); + insert_property( + &mut properties, + "name", + Type::String, + "/** The agent name, exactly as `run` expects it. */", + ); + insert_property( + &mut properties, + "description", + Type::union(vec![Type::String, Type::Undefined]), + "/** What the agent is for, or `undefined` when the harness gives none. Operator-authored advice, always a single line within a fixed length bound. */", + ); + defs.types.insert( + "AgentInfo".to_string(), + TypeSymbol { + name: "AgentInfo".to_string(), + mangled_name: crate::mangle::package_symbol(MODULE_NAME, "AgentInfo"), + declaration_span: Span::at(crate::FileId::AGENTS), + kind: TypeKind::Interface { + index: None, + generics: Vec::new(), + methods: BTreeMap::new(), + properties, + dispatch: Dispatch::Direct, + doc: crate::doc( + crate::FileId::AGENTS, + "/** One agent this caller may hand work to. Constructed only by `list()`. */", + ), + }, + }, + ); +} + +fn insert_property( + properties: &mut BTreeMap, + name: &str, + ty: Type, + doc: &str, +) { + properties.insert( + name.to_string(), + PropertySig { + ty, + readonly: true, + optional: false, + intrinsic: false, + doc: crate::doc(crate::FileId::AGENTS, doc), + }, + ); +} + +fn insert_fn( + defs: &mut PackageDeclaration, + name: &str, + generics: Vec, + params: Vec, + ret: Type, + doc: &str, +) { + defs.values.insert( + name.to_string(), + ValueSymbol { + name: name.to_string(), + mangled_name: crate::mangle::package_symbol(MODULE_NAME, name), + declaration_span: Span::at(crate::FileId::AGENTS), + kind: ValueKind::Function { + generics, + params, + ret, + type_predicate: None, + doc: crate::doc(crate::FileId::AGENTS, doc), + }, + }, + ); +} diff --git a/crates/interpreter/src/stdlib/agents/mod.rs b/crates/interpreter/src/stdlib/agents/mod.rs new file mode 100644 index 000000000..3baf302f2 --- /dev/null +++ b/crates/interpreter/src/stdlib/agents/mod.rs @@ -0,0 +1,315 @@ +//! `submilli:agents` — hand work to a sub-agent from inside a Submilli program. +//! +//! An optional package: it exists only for an embedder that enables +//! [`OptionalPackage::Agents`](crate::stdlib::OptionalPackage::Agents). The +//! harness runs the agents through [`StoreData::agent_provider`]; a runtime with +//! none refuses every op with a catchable error rather than inventing an answer. +//! +//! **One capability.** `run` and `list()` are both `agent.run`, filtered by +//! `agent`. `list()` gates each candidate with the same filter, so a listing +//! never offers an agent the caller would be denied — the `llm.models()` shape. +//! +//! **No input or output in metadata.** The filter context is the agent name +//! only; the task and the answer reach the call log as payload bodies, never in +//! an error or a policy context. + +pub mod declaration; + +use std::sync::Arc; + +use wasmtime::{FuncType, HeapType, Linker, RefType, StructType, Val, ValType}; + +use crate::runtime::StoreData; +use crate::runtime::agents::{ + AgentCallError, AgentInfo, AgentOutcome, AgentProvider, AgentRequest, +}; +use crate::runtime::call_log::{ModelUsage, Payload, Side, record_payload, record_usage}; +use crate::runtime::decision::CallTicket; +use crate::runtime::fuel; +use crate::runtime::host::{ + abi_arg, abi_result, read_string_arg, register_host_fn_async, write_submilli_string_struct, +}; +use crate::runtime::intrinsic_types::build_intrinsic_types; +use crate::stdlib::abi::{ + self, backing_struct, install_field_getters, nullable_object_field, string_field, +}; +use crate::stdlib::shared::{ + check_security_call, permitted_candidates, preflight_listing, running_package, + sanitize_description, truncated, +}; + +pub use declaration::package_declaration; + +pub const MODULE_NAME: &str = crate::runtime::agents::AGENTS_MODULE_NAME; + +/// The capability gating every op in this package. +pub const CAPABILITY: &str = "agent.run"; + +// `$AgentInfoBacking` field indices (0 is the vtable). +const A_NAME: usize = 1; +const A_DESCRIPTION: usize = 2; + +pub fn install(linker: &mut Linker) -> wasmtime::Result<()> { + let engine = linker.engine().clone(); + let intr = build_intrinsic_types(&engine)?; + let string = ValType::Ref(RefType::new( + false, + HeapType::ConcreteStruct(intr.string.clone()), + )); + let nullable_object = ValType::Ref(RefType::new( + true, + HeapType::ConcreteStruct(intr.object.clone()), + )); + let array = ValType::Ref(RefType::new( + false, + HeapType::ConcreteStruct(intr.array.clone()), + )); + + register_host_fn_async( + linker, + MODULE_NAME, + crate::mangle::package_symbol(MODULE_NAME, "run"), + FuncType::new( + &engine, + [string.clone(), string.clone(), nullable_object.clone()], + // `run` is generic, so its result travels in the erased `T` slot. + [nullable_object.clone()], + ), + /* deterministic = */ false, + |caller, params, results| { + Box::pin(async move { + let agent = + read_string_arg(&mut *caller, abi_arg(params, 0)?, "agents.run (agent)")?; + let input = + read_string_arg(&mut *caller, abi_arg(params, 1)?, "agents.run (input)")?; + let schema = read_optional_string(caller, abi_arg(params, 2)?)?; + let result = abi_result(results, 0)?; + *result = run(caller, agent, input, schema).await?; + Ok(()) + }) + }, + )?; + + register_host_fn_async( + linker, + MODULE_NAME, + crate::mangle::package_symbol(MODULE_NAME, "list"), + FuncType::new(&engine, [], [array]), + /* deterministic = */ false, + |caller, _params, results| { + Box::pin(async move { + *abi_result(results, 0)? = list(caller).await?; + Ok(()) + }) + }, + )?; + + let receiver = ValType::Ref(RefType::new( + false, + HeapType::ConcreteStruct(intr.object.clone()), + )); + install_field_getters( + linker, + MODULE_NAME, + "AgentInfo", + &engine, + &receiver, + &[ + ("name", A_NAME, string), + ("description", A_DESCRIPTION, nullable_object), + ], + ) +} + +/// One run: gate, charge, record what is sent, dispatch, then build the result +/// without refusing for fuel, because the run has already happened. +async fn run( + caller: &mut wasmtime::Caller<'_, StoreData>, + agent: String, + input: String, + schema: Option, +) -> wasmtime::Result { + let ticket = gate(caller, &agent)?; + let provider = provider(caller).map_err(|error| fail(caller, ticket, "run", &error))?; + let principal = running_package(caller).map_err(|error| error.into_denial(CAPABILITY))?; + let sent = (input.len() + schema.as_deref().map_or(0, str::len)) as u64; + fuel::charge(&mut *caller, fuel::IO, sent)?; + record_payload(&*caller, ticket, Side::Request, || { + Payload::meta(serde_json::json!({ "op": "run", "agent": agent, "schema": schema })) + .with_owned_body(input.clone().into_bytes()) + .with_size(sent) + }); + let typed = schema.is_some(); + let request = AgentRequest { + agent: agent.clone(), + input, + schema, + caller: principal, + }; + match provider.run(request).await { + Ok(outcome) => { + record_outcome(caller, ticket, &outcome); + fuel::settle(&mut *caller, fuel::IO, outcome.text.len() as u64)?; + fuel::settle_result(caller, |caller| { + if typed { + crate::runtime::json::parse_json_as_unknown( + caller, + &outcome.text, + &format!( + "agents.run(\"{}\"): the agent's result is not JSON", + truncated(&agent) + ), + ) + } else { + let text = write_submilli_string_struct(caller, &outcome.text)?; + Ok(Val::AnyRef(Some(text.to_anyref()))) + } + }) + } + Err(error) => Err(fail(caller, ticket, "run", &error)), + } +} + +/// Record `error` as the call's response and turn it into the guest's error. +fn fail( + caller: &wasmtime::Caller<'_, StoreData>, + ticket: Option, + op: &str, + error: &AgentCallError, +) -> wasmtime::Error { + record_payload(caller, ticket, Side::Response, || { + Payload::meta(serde_json::json!({ "call_error": call_error_record(error) })) + }); + throw(op, error) +} + +fn record_outcome( + caller: &wasmtime::Caller<'_, StoreData>, + ticket: Option, + outcome: &AgentOutcome, +) { + let usage = outcome.usage.clone().unwrap_or_default(); + record_payload(caller, ticket, Side::Response, || { + Payload::meta(serde_json::json!({ + "usage": { + "input_tokens": usage.input_tokens, + "output_tokens": usage.output_tokens, + } + })) + .with_owned_body(outcome.text.clone().into_bytes()) + .with_size(outcome.text.len() as u64) + }); + record_usage( + caller, + ticket, + ModelUsage { + input_tokens: usage.input_tokens, + output_tokens: usage.output_tokens, + }, + ); +} + +/// The capability check, run before anything reaches the harness. The context +/// is the agent name only: a policy can choose which agents a caller reaches, +/// not read the task. +fn gate( + caller: &mut wasmtime::Caller<'_, StoreData>, + agent: &str, +) -> wasmtime::Result> { + check_security_call(caller, CAPABILITY, serde_json::json!({ "agent": agent })) +} + +/// `list()`: every agent the harness offers that the caller may run. A denied +/// candidate is left out, never reported. Whether a provider is wired is not +/// hidden here: a listing has no candidate to gate before asking it. +async fn list(caller: &mut wasmtime::Caller<'_, StoreData>) -> wasmtime::Result { + preflight_listing(caller, CAPABILITY, &serde_json::json!({ "agent": "" }))?; + let provider = provider(caller).map_err(|error| throw("list", &error))?; + let candidates = provider.agents().await.map_err(|e| throw("list", &e))?; + let visible = permitted_candidates( + caller, + candidates, + |agent| agent.name.as_str(), + |caller, name| gate(caller, name).map(|_| ()), + )?; + let mut built = Vec::with_capacity(visible.len()); + for agent in visible { + built.push(build_agent_info(caller, agent)?); + } + abi::new_array(caller, &built) +} + +/// Clone the provider out of the store before any `await`. `run` looks it up +/// after the capability check, so a denied caller cannot learn whether a harness +/// is wired. +fn provider( + caller: &wasmtime::Caller<'_, StoreData>, +) -> Result, AgentCallError> { + caller + .data() + .agent_provider + .clone() + .ok_or(AgentCallError::NotConfigured) +} + +fn throw(op: &str, error: &AgentCallError) -> wasmtime::Error { + wasmtime::Error::msg(format!("agents.{op}: {error}")) +} + +/// A run's failure as the call log keeps it: a stable kind and the agent. +fn call_error_record(error: &AgentCallError) -> serde_json::Value { + match error { + AgentCallError::NotConfigured => serde_json::json!({ "kind": "not-configured" }), + AgentCallError::NotFound { agent } => { + serde_json::json!({ "kind": "not-found", "agent": agent }) + } + AgentCallError::Failed { agent, message } => { + serde_json::json!({ "kind": "failed", "agent": agent, "message": message }) + } + AgentCallError::Cancelled { agent } => { + serde_json::json!({ "kind": "cancelled", "agent": agent }) + } + } +} + +/// Read the compiler-filled `schema` argument. Undefined means an untyped call. +fn read_optional_string( + caller: &mut wasmtime::Caller<'_, StoreData>, + val: &Val, +) -> wasmtime::Result> { + if crate::runtime::prelude::undefined::is_undefined(caller, val)? { + return Ok(None); + } + read_string_arg(caller, val, "agents.run (schema)").map(Some) +} + +fn agent_info_backing_struct(engine: &wasmtime::Engine) -> wasmtime::Result { + let intr = build_intrinsic_types(engine)?; + backing_struct( + engine, + &intr, + vec![ + string_field(&intr), // name + nullable_object_field(&intr), // description + ], + ) +} + +fn build_agent_info( + caller: &mut wasmtime::Caller<'_, StoreData>, + agent: AgentInfo, +) -> wasmtime::Result { + let name = write_submilli_string_struct(caller, &agent.name)?; + let description = match agent.description.as_deref().and_then(sanitize_description) { + Some(text) => Val::AnyRef(Some( + write_submilli_string_struct(caller, &text)?.to_anyref(), + )), + None => crate::runtime::prelude::undefined::value(caller)?, + }; + let ty = agent_info_backing_struct(caller.engine())?; + abi::new_backing( + caller, + ty, + &[Val::AnyRef(Some(name.to_anyref())), description], + ) +} diff --git a/crates/interpreter/src/stdlib/capabilities.rs b/crates/interpreter/src/stdlib/capabilities.rs index 15ff8ff92..388baab9b 100644 --- a/crates/interpreter/src/stdlib/capabilities.rs +++ b/crates/interpreter/src/stdlib/capabilities.rs @@ -9,6 +9,8 @@ //! update the matching entry here — see AGENTS.md. The `capabilities` test in //! the `submilli` CLI asserts every `example_filter` below parses. +use super::{OptionalPackage, Stdlib}; + /// One context field a capability's `filter:` expression can match on. pub struct FilterField { pub name: &'static str, @@ -356,7 +358,7 @@ const HTTP: &[Capability] = &[ /// Every other method `http.request` takes, gated as `http.` with the /// method lowercased: `http.request("TRACE", …)` checks `http.trace`. Kept -/// out of [`CATALOG`], whose consumers read a templated name as +/// out of [`CORE_CATALOG`], whose consumers read a templated name as /// `mcp.` and concretize it per declared server; see /// [`uncataloged_http_method`]. pub const HTTP_OTHER_METHOD: Capability = Capability { @@ -485,7 +487,8 @@ const SESSION: &[Capability] = &[ }, ]; -const CATALOG: &[CapabilityGroup] = &[ +/// The capabilities of the core packages, which every embedder has. +const CORE_CATALOG: &[CapabilityGroup] = &[ CapabilityGroup { module: "submilli:embedding", capabilities: EMBEDDING, @@ -520,16 +523,98 @@ const CATALOG: &[CapabilityGroup] = &[ }, ]; -/// Every stdlib capability the runtime can gate, grouped by source module. +const AGENT: FilterField = field( + "agent", + "string", + "The sub-agent's name, as `submilli:agents` list() names it", +); + +const AGENTS: CapabilityGroup = CapabilityGroup { + module: "submilli:agents", + capabilities: &[Capability { + name: "agent.run", + main_denial: None, + summary: "Hand work to a harness sub-agent, or list the agents", + filter_fields: &[AGENT], + example_filter: "agent == \"researcher\"", + }], +}; + +const SKILL: FilterField = field( + "name", + "string", + "The skill's name, as `submilli:skills` list() names it", +); + +const SKILLS: CapabilityGroup = CapabilityGroup { + module: "submilli:skills", + capabilities: &[Capability { + name: "skill.load", + main_denial: None, + summary: "Load a harness skill, read its files, or list the skills", + filter_fields: &[SKILL], + example_filter: "name == \"code-review\"", + }], +}; + +/// The capabilities of one optional package. +fn optional_group(package: OptionalPackage) -> &'static CapabilityGroup { + match package { + OptionalPackage::Agents => &AGENTS, + OptionalPackage::Skills => &SKILLS, + } +} + +/// Every capability the core packages gate, grouped by source module. An +/// optional package's capabilities are listed only by [`catalog_for`]. pub fn catalog() -> &'static [CapabilityGroup] { - CATALOG + CORE_CATALOG +} + +/// Every capability the packages of `stdlib` gate, grouped by source module, +/// in [`catalog`]'s order: by module name without its `submilli:` or `@` prefix. +pub fn catalog_for(stdlib: Stdlib) -> Vec<&'static CapabilityGroup> { + let mut groups: Vec<&'static CapabilityGroup> = CORE_CATALOG.iter().collect(); + groups.extend(stdlib.optional_packages().map(optional_group)); + groups.sort_by_key(|group| catalog_order(group.module)); + groups +} + +fn catalog_order(module: &str) -> &str { + module + .strip_prefix("submilli:") + .or_else(|| module.strip_prefix('@')) + .unwrap_or(module) } -/// Look up a capability by its exact name (templates included, by their +/// Every group the runtime can gate, whichever packages an embedder enables. +/// Runtime checks use this: a capability reaches them only from a host +/// function, which is installed only when its package is enabled. +pub(crate) fn all_groups() -> impl Iterator { + CORE_CATALOG.iter().chain( + OptionalPackage::ALL + .iter() + .map(|package| optional_group(*package)), + ) +} + +/// Look up a core capability by its exact name (templates included, by their /// literal `mcp.` spelling). pub fn find(name: &str) -> Option<&'static Capability> { - CATALOG - .iter() + find_for(Stdlib::core(), name) +} + +/// [`find`] among the capabilities of `stdlib`. +pub fn find_for(stdlib: Stdlib, name: &str) -> Option<&'static Capability> { + catalog_for(stdlib) + .into_iter() + .flat_map(|group| group.capabilities) + .find(|cap| cap.name == name) +} + +/// [`find`] among every group the runtime can gate. +pub(crate) fn find_any(name: &str) -> Option<&'static Capability> { + all_groups() .flat_map(|group| group.capabilities) .find(|cap| cap.name == name) } @@ -545,7 +630,12 @@ pub fn uncataloged_http_method(name: &str) -> Option<&str> { /// The entry describing `name`: its catalog entry, or [`HTTP_OTHER_METHOD`] /// for a name that fills it. pub fn find_gating(name: &str) -> Option<&'static Capability> { - find(name).or_else(|| uncataloged_http_method(name).map(|_| &HTTP_OTHER_METHOD)) + find_gating_for(Stdlib::core(), name) +} + +/// [`find_gating`] among the capabilities of `stdlib`. +pub fn find_gating_for(stdlib: Stdlib, name: &str) -> Option<&'static Capability> { + find_for(stdlib, name).or_else(|| uncataloged_http_method(name).map(|_| &HTTP_OTHER_METHOD)) } /// An RFC 9110 method token, the set `http.request` accepts, in lowercase. @@ -565,8 +655,7 @@ mod tests { /// answer for every caller, so growing the set is a deliberate act. #[test] fn only_secrets_get_is_refused_to_main() { - let refused: Vec<&str> = catalog() - .iter() + let refused: Vec<&str> = all_groups() .flat_map(|group| group.capabilities) .filter(|cap| !cap.grantable_to_main()) .map(|cap| cap.name) @@ -574,10 +663,22 @@ mod tests { assert_eq!(refused, ["secrets.get"]); } + /// The core set lists exactly the core catalog, in the same order, so every + /// surface that reads `catalog()` agrees with one that reads `catalog_for`. + #[test] + fn core_set_lists_the_core_catalog() { + let core: Vec<&str> = catalog().iter().map(|group| group.module).collect(); + let for_core: Vec<&str> = catalog_for(Stdlib::core()) + .iter() + .map(|group| group.module) + .collect(); + assert_eq!(for_core, core); + } + #[test] fn catalog_entries_are_well_formed() { let mut seen = HashSet::new(); - for group in catalog() { + for group in all_groups() { for cap in group.capabilities { assert!( cap.name.contains('.') && !cap.name.is_empty(), diff --git a/crates/interpreter/src/stdlib/mod.rs b/crates/interpreter/src/stdlib/mod.rs index 880a06320..6c575642f 100644 --- a/crates/interpreter/src/stdlib/mod.rs +++ b/crates/interpreter/src/stdlib/mod.rs @@ -9,6 +9,7 @@ //! vtables separately for each store. pub(crate) mod abi; +pub mod agents; pub mod capabilities; pub mod code; pub mod crypto; @@ -22,6 +23,7 @@ pub mod secrets; pub mod security; pub mod session; pub mod shared; +pub mod skills; /// Test-authoring package. Deliberately absent from /// [`stdlib_package_declarations`] and [`install_host_functions`]: only /// `submilli build test` makes it importable (by passing its declaration into @@ -33,10 +35,240 @@ pub mod uuid; use wasmtime::Linker; -use crate::PackageDeclaration; use crate::runtime::StoreData; +use crate::{MangledName, PackageDeclaration, Type}; +/// A standard-library package that exists only when the embedder provides it. +/// +/// Each one is backed by a provider trait the harness implements. An embedder +/// that does not enable a package gets none of it: the package cannot be +/// imported, discovery and the capability catalog do not list it, and its host +/// functions are not installed. +#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash, PartialOrd, Ord)] +#[non_exhaustive] +pub enum OptionalPackage { + /// `submilli:agents`, backed by [`AgentProvider`](crate::runtime::AgentProvider). + Agents, + /// `submilli:skills`, backed by [`SkillProvider`](crate::runtime::SkillProvider). + Skills, +} + +impl OptionalPackage { + pub const ALL: &'static [OptionalPackage] = &[Self::Agents, Self::Skills]; + + pub const fn module_name(self) -> &'static str { + match self { + Self::Agents => agents::MODULE_NAME, + Self::Skills => skills::MODULE_NAME, + } + } + + pub fn from_module_name(name: &str) -> Option { + Self::ALL + .iter() + .copied() + .find(|package| package.module_name() == name) + } + + fn package_declaration(self) -> PackageDeclaration { + match self { + Self::Agents => agents::package_declaration(), + Self::Skills => skills::package_declaration(), + } + } + + fn install(self, linker: &mut Linker) -> wasmtime::Result<()> { + match self { + Self::Agents => agents::install(linker), + Self::Skills => skills::install(linker), + } + } + + const fn bit(self) -> u8 { + match self { + Self::Agents => 1, + Self::Skills => 1 << 1, + } + } +} + +/// A host call the typechecker types from its type argument: the compiler emits +/// a JSON Schema for `T` into the call's trailing `schema` parameter and wraps +/// the result in a structural check against `T`. What differs between such +/// calls is what a bare call returns and how diagnostics name it. +pub(crate) struct SchemaCheckedCall { + /// The call as diagnostics name it, e.g. `llm.call`. + pub callee: &'static str, + /// A typed call to show in a diagnostic. + pub example: &'static str, + /// What the structural check is applied to. + pub checks: &'static str, + /// Who the schema is sent to. + pub answerer: &'static str, + /// The parameters a program writes, in prose; the schema parameter follows. + pub written_params: &'static str, + pub written_param_count: usize, + /// Ends the "cannot be verified" diagnostics: what to do instead of a type + /// argument the runtime cannot test. + pub untyped_hint: &'static str, + /// Ends the `` diagnostic. + pub untyped_hint_for_unknown: &'static str, + /// Ends the "has no JSON Schema" diagnostic. + pub untyped_hint_for_no_schema: &'static str, + /// What `T` binds to when the program wrote no type argument. + pub untyped_result: Type, +} + +impl SchemaCheckedCall { + /// The schema parameter's position, as an ordinal word. + pub fn schema_argument_ordinal(&self) -> &'static str { + match self.written_param_count { + 0 => "first", + 1 => "second", + 2 => "third", + 3 => "fourth", + _ => "last", + } + } +} + +/// The schema-checked call `mangled` names, if it is one. +pub(crate) fn schema_checked_call(mangled: &MangledName) -> Option { + if llm::declaration::is_checked_call(mangled) { + return Some(SchemaCheckedCall { + callee: "llm.call", + example: "llm.call(model, prompt)", + checks: "the model's response", + answerer: "the model", + written_params: "the model and the prompt", + written_param_count: 2, + untyped_hint: "read the `Completion` envelope yourself", + untyped_hint_for_unknown: "read the `Completion` envelope's `ok` and `text` yourself", + untyped_hint_for_no_schema: "parse the `Completion` text yourself", + untyped_result: llm::declaration::untyped_result_type(mangled), + }); + } + if agents::declaration::is_checked_run(mangled) { + return Some(SchemaCheckedCall { + callee: "agents.run", + example: "agents.run(agent, input)", + checks: "the agent's result", + answerer: "the agent", + written_params: "the agent and the input", + written_param_count: 2, + untyped_hint: "read the returned text yourself", + untyped_hint_for_unknown: "read the returned text yourself", + untyped_hint_for_no_schema: "parse the returned text yourself", + untyped_result: Type::String, + }); + } + None +} + +/// The standard library an embedder offers its programs: the core packages +/// every embedder has, plus the [`OptionalPackage`]s it enables. +/// +/// Compiling, discovery, the capability catalog and host-function installation +/// all read the same set, so a package is either fully present or fully absent. +/// The discovery methods (`docs`, `search`, `resolve`, …) are in +/// [`crate::packages`]. +#[derive(Clone, Copy, Debug, Default, PartialEq, Eq, Hash)] +pub struct Stdlib { + optional: u8, +} + +impl Stdlib { + /// The core packages only, what `submilli run` and the server offer. + pub const fn core() -> Self { + Stdlib { optional: 0 } + } + + #[must_use] + pub const fn with(self, package: OptionalPackage) -> Self { + Stdlib { + optional: self.optional | package.bit(), + } + } + + pub const fn enables(self, package: OptionalPackage) -> bool { + self.optional & package.bit() != 0 + } + + pub fn optional_packages(self) -> impl Iterator { + OptionalPackage::ALL + .iter() + .copied() + .filter(move |package| self.enables(*package)) + } + + /// Whether `name` is a package of this set. + pub fn contains_module(self, name: &str) -> bool { + OptionalPackage::from_module_name(name).map_or_else( + || { + core_package_declarations() + .iter() + .any(|defs| defs.package_name == name) + }, + |package| self.enables(package), + ) + } + + /// Every package of this set, sorted by name. + pub fn package_declarations(self) -> Vec { + let mut declarations = core_package_declarations(); + declarations.extend( + self.optional_packages() + .map(OptionalPackage::package_declaration), + ); + // Codegen emits imports in this order, so sorting keeps builds byte-for-byte stable. + declarations.sort_by(|a, b| a.package_name.cmp(&b.package_name)); + declarations + } + + pub fn install_host_functions(self, linker: &mut Linker) -> wasmtime::Result<()> { + install_core_host_functions(linker)?; + for package in self.optional_packages() { + package.install(linker)?; + } + Ok(()) + } +} + +/// Why `module` cannot be imported though it names a standard-library package, +/// as a message and its help lines. `None` when `module` is no such package. +/// +/// Reached only for a package missing from the declarations the compile was +/// given, so an optional package here is one the embedder did not enable. +pub(crate) fn unavailable_import(module: &str) -> Option<(String, Vec)> { + if module == test::MODULE_NAME { + return Some(( + format!("package `{module}` not found"), + vec![ + "`submilli:test` is only available to test files run via `submilli build test`; it is not importable from a program run with `submilli run`" + .to_string(), + ], + )); + } + let package = OptionalPackage::from_module_name(module)?; + Some(( + format!("package `{module}` is not available in this harness"), + vec![format!( + "`{}` exists only where the harness running the program provides it", + package.module_name() + )], + )) +} + +/// The core packages: what [`Stdlib::core`] offers. pub fn stdlib_package_declarations() -> Vec { + Stdlib::core().package_declarations() +} + +pub fn install_host_functions(linker: &mut Linker) -> wasmtime::Result<()> { + Stdlib::core().install_host_functions(linker) +} + +fn core_package_declarations() -> Vec { vec![ // Alphabetical by package name; codegen import-emission relies on this order. code::package_declaration(), @@ -54,7 +286,7 @@ pub fn stdlib_package_declarations() -> Vec { ] } -pub fn install_host_functions(linker: &mut Linker) -> wasmtime::Result<()> { +fn install_core_host_functions(linker: &mut Linker) -> wasmtime::Result<()> { code::install(linker)?; crypto::install(linker)?; embedding::install(linker)?; @@ -77,3 +309,71 @@ pub(crate) fn install_store_bound( ) -> wasmtime::Result<()> { git::class::install(linker, store) } + +#[cfg(test)] +mod optional_tests; + +#[cfg(test)] +mod tests { + use super::*; + + /// Every subset of the optional packages. + fn every_set() -> Vec { + let mut sets = vec![Stdlib::core()]; + for package in OptionalPackage::ALL { + let with: Vec = sets.iter().map(|set| set.with(*package)).collect(); + sets.extend(with); + } + sets + } + + #[test] + fn declarations_are_sorted_for_every_set() { + for set in every_set() { + let names: Vec = set + .package_declarations() + .into_iter() + .map(|defs| defs.package_name) + .collect(); + let mut sorted = names.clone(); + sorted.sort(); + assert_eq!(names, sorted, "{set:?}"); + } + } + + #[test] + fn a_set_contains_exactly_its_packages() { + for set in every_set() { + let names: Vec = set + .package_declarations() + .into_iter() + .map(|defs| defs.package_name) + .collect(); + for name in &names { + assert!(set.contains_module(name), "{set:?} misses {name}"); + } + for package in OptionalPackage::ALL { + assert_eq!( + names.iter().any(|name| name == package.module_name()), + set.enables(*package), + "{set:?} and {package:?}" + ); + } + } + } + + #[test] + fn core_is_the_global_list() { + let core: Vec = Stdlib::core() + .package_declarations() + .into_iter() + .map(|defs| defs.package_name) + .collect(); + let global: Vec = stdlib_package_declarations() + .into_iter() + .map(|defs| defs.package_name) + .collect(); + assert_eq!(core, global); + assert!(!Stdlib::core().contains_module(test::MODULE_NAME)); + } +} diff --git a/crates/interpreter/src/stdlib/optional_tests.rs b/crates/interpreter/src/stdlib/optional_tests.rs new file mode 100644 index 000000000..417a8e62b --- /dev/null +++ b/crates/interpreter/src/stdlib/optional_tests.rs @@ -0,0 +1,779 @@ +//! `submilli:agents` and `submilli:skills` end to end: a program compiled and run +//! under a [`Stdlib`] that enables them, against fake providers. + +use std::future::Future; +use std::pin::Pin; +use std::sync::{Arc, Mutex}; + +use crate::runtime::agents::{ + AgentCallError, AgentInfo, AgentOutcome, AgentProvider, AgentRequest, AgentUsage, +}; +use crate::runtime::skills::{Skill, SkillError, SkillInfo, SkillProvider}; +use crate::runtime::{ + CheckOutcome, RuntimeConfig, SecurityCheck, StoreData, Vfs, dispatch_main_async, + install_runtime_async_for, +}; +use crate::stdlib::{OptionalPackage, Stdlib, capabilities}; + +type Boxed<'a, T> = Pin + Send + 'a>>; + +fn harness_stdlib() -> Stdlib { + Stdlib::core() + .with(OptionalPackage::Agents) + .with(OptionalPackage::Skills) +} + +/// Answers every run with `answer`, lists `listing`, and records what it was +/// asked to run. +struct FakeAgents { + answer: Result, + listing: Result, AgentCallError>, + requests: Mutex>, +} + +impl FakeAgents { + fn answering(answer: &str) -> Arc { + Arc::new(Self { + answer: Ok(answer.to_string()), + listing: Ok(two_agents()), + requests: Mutex::new(Vec::new()), + }) + } + + fn failing(error: AgentCallError) -> Arc { + Arc::new(Self { + answer: Err(error), + listing: Ok(two_agents()), + requests: Mutex::new(Vec::new()), + }) + } + + fn listing(listing: Result, AgentCallError>) -> Arc { + Arc::new(Self { + answer: Ok(String::new()), + listing, + requests: Mutex::new(Vec::new()), + }) + } + + fn requests(&self) -> Vec { + self.requests.lock().expect("requests").clone() + } +} + +impl AgentProvider for FakeAgents { + fn run<'a>(&'a self, request: AgentRequest) -> Boxed<'a, Result> { + self.requests.lock().expect("requests").push(request); + let answer = self.answer.clone(); + Box::pin(async move { + answer.map(|text| AgentOutcome { + text, + usage: Some(AgentUsage { + input_tokens: Some(3), + output_tokens: None, + }), + }) + }) + } + + fn agents<'a>(&'a self) -> Boxed<'a, Result, AgentCallError>> { + let listing = self.listing.clone(); + Box::pin(async move { listing }) + } +} + +fn two_agents() -> Vec { + vec![ + AgentInfo { + name: "researcher".to_string(), + description: Some("Finds\nsources".to_string()), + }, + AgentInfo { + name: "deployer".to_string(), + description: None, + }, + ] +} + +/// Two skills; `review` bundles one file. Records every name it is asked to +/// load and every file it is asked to read. +struct FakeSkills { + loads: Mutex>, + reads: Mutex>, +} + +impl FakeSkills { + fn new() -> Arc { + Arc::new(Self { + loads: Mutex::new(Vec::new()), + reads: Mutex::new(Vec::new()), + }) + } +} + +impl SkillProvider for FakeSkills { + fn list<'a>(&'a self) -> Boxed<'a, Result, SkillError>> { + Box::pin(async { + Ok(vec![ + SkillInfo { + name: "review".to_string(), + description: Some("Review a diff".to_string()), + }, + SkillInfo { + name: "internal".to_string(), + description: None, + }, + // Not one name: `load` would refuse it, so `list()` drops it. + SkillInfo { + name: "team/review".to_string(), + description: None, + }, + ]) + }) + } + + fn load<'a>(&'a self, name: &'a str) -> Boxed<'a, Result> { + self.loads.lock().expect("loads").push(name.to_string()); + Box::pin(async move { + match name { + "review" => Ok(Skill { + name: "review".to_string(), + description: Some("Review a diff".to_string()), + content: "Read the diff twice.".to_string(), + }), + _ => Err(SkillError::NotFound { + name: name.to_string(), + }), + } + }) + } + + fn read_file<'a>( + &'a self, + name: &'a str, + path: &'a str, + ) -> Boxed<'a, Result> { + self.reads + .lock() + .expect("reads") + .push((name.to_string(), path.to_string())); + Box::pin(async move { + match (name, path) { + ("review", "templates/report.md") => Ok("# Report".to_string()), + _ => Err(SkillError::FileNotFound { + name: name.to_string(), + path: path.to_string(), + }), + } + }) + } +} + +/// Denies `capability` calls whose context field `field` equals `value`, and +/// records every context it is asked about. +struct DenyOne { + field: &'static str, + value: &'static str, + contexts: Mutex>, +} + +impl DenyOne { + fn new(field: &'static str, value: &'static str) -> Arc { + Arc::new(Self { + field, + value, + contexts: Mutex::new(Vec::new()), + }) + } +} + +impl SecurityCheck for DenyOne { + fn check(&self, _caller: &str, _capability: &str, context: &serde_json::Value) -> CheckOutcome { + self.contexts + .lock() + .expect("contexts") + .push(context.clone()); + if context[self.field] == self.value { + CheckOutcome::Deny { + rule: None, + reason: format!("policy denies {}", self.value), + } + } else { + CheckOutcome::Allow { rule: None } + } + } +} + +#[derive(Default)] +struct Harness { + agents: Option>, + skills: Option>, + policy: Option>, +} + +impl Harness { + fn agents(mut self, agents: Arc) -> Self { + self.agents = Some(agents); + self + } + + fn skills(mut self, skills: Arc) -> Self { + self.skills = Some(skills); + self + } + + fn policy(mut self, policy: Arc) -> Self { + self.policy = Some(policy); + self + } + + /// Compile `source` under the harness set and run it; `main`'s result. + async fn run(self, source: &str) -> wasmtime::Result { + let stdlib = harness_stdlib(); + let parsed = crate::parse_script(source, crate::FileId(0)); + let compiled = crate::compile_parsed_script_timed( + source, + "test.ts", + &parsed, + &stdlib.package_declarations(), + &[], + &[], + ) + .expect("compile clean"); + let cfg = RuntimeConfig::default(); + let engine = cfg.engine().expect("engine"); + let mut data = StoreData::with_vfs(Vfs::tempdir().expect("tempdir")); + data.install_type_info(compiled.type_info.clone()); + data.agent_provider = self.agents; + data.skill_provider = self.skills; + if let Some(policy) = self.policy { + data.security_check = policy; + } + let mut store = cfg.store_async(&engine, data).expect("store"); + let module = wasmtime::Module::new(&engine, &compiled.wasm).expect("module"); + let mut linker = wasmtime::Linker::::new(&engine); + install_runtime_async_for(&mut linker, &mut store, stdlib) + .await + .expect("install"); + let inst = linker + .instantiate_async(&mut store, &module) + .await + .expect("instantiate"); + dispatch_main_async(&mut store, &inst) + .await + .map(Option::unwrap_or_default) + } +} + +fn compile_errors(source: &str, stdlib: Stdlib) -> Vec { + let parsed = crate::parse_script(source, crate::FileId(0)); + match crate::compile_parsed_script_timed( + source, + "test.ts", + &parsed, + &stdlib.package_declarations(), + &[], + &[], + ) { + Ok(_) => Vec::new(), + Err(diagnostics) => diagnostics + .into_iter() + .map(|d| format!("{} {}", d.message, d.help.join(" "))) + .collect(), + } +} + +#[test] +fn a_disabled_package_is_not_available_in_this_harness() { + for module in ["submilli:agents", "submilli:skills"] { + let source = + format!("import {{ list }} from \"{module}\";\nfunction main(): void {{ list(); }}\n"); + let errors = compile_errors(&source, Stdlib::core()); + assert!( + errors + .iter() + .any(|e| e.contains(&format!("`{module}` is not available in this harness"))), + "{errors:?}" + ); + let parsed = crate::parse_script(&source, crate::FileId(0)); + let imports = parsed.external_imports().expect("imports"); + assert!( + imports.stdlib.contains(module), + "{module} is stdlib, not a registry package" + ); + assert!(imports.registry_packages.is_empty()); + } +} + +#[test] +fn only_an_enabling_set_discovers_and_catalogs_the_packages() { + let core_modules: Vec = crate::packages::search("") + .into_iter() + .map(|m| m.name) + .collect(); + let harness_modules: Vec = harness_stdlib() + .search("") + .into_iter() + .map(|m| m.name) + .collect(); + for module in ["submilli:agents", "submilli:skills"] { + assert!( + !core_modules.iter().any(|m| m == module), + "{core_modules:?}" + ); + assert!( + harness_modules.iter().any(|m| m == module), + "{harness_modules:?}" + ); + assert!(crate::packages::docs(module).is_none()); + assert!(harness_stdlib().docs(module).is_some()); + } + for capability in ["agent.run", "skill.load"] { + assert!(capabilities::find(capability).is_none()); + assert!(capabilities::find_for(harness_stdlib(), capability).is_some()); + } + let agents_only = Stdlib::core().with(OptionalPackage::Agents); + assert!(capabilities::find_for(agents_only, "agent.run").is_some()); + assert!(capabilities::find_for(agents_only, "skill.load").is_none()); +} + +#[test] +fn a_written_schema_argument_is_refused() { + let errors = compile_errors( + "import agents from \"submilli:agents\";\n\ + function main(): void { agents.run(\"researcher\", \"go\", \"{}\"); }\n", + harness_stdlib(), + ); + assert!( + errors + .iter() + .any(|e| e.contains("takes the agent and the input")), + "{errors:?}" + ); +} + +#[tokio::test] +async fn an_untyped_run_returns_the_agents_text() { + let agents = FakeAgents::answering("found three sources"); + let out = Harness::default() + .agents(agents.clone()) + .run( + r#"import agents from "submilli:agents"; + function main(): string { return agents.run("researcher", "find sources"); }"#, + ) + .await + .expect("program completes"); + assert_eq!(out, "found three sources"); + assert_eq!( + agents.requests(), + [AgentRequest { + agent: "researcher".to_string(), + input: "find sources".to_string(), + schema: None, + caller: "main".to_string(), + }] + ); +} + +#[tokio::test] +async fn a_typed_run_sends_a_schema_and_checks_the_answer() { + let agents = FakeAgents::answering(r#"{"score": 7, "summary": "fine"}"#); + let out = Harness::default() + .agents(agents.clone()) + .run( + r#"import agents from "submilli:agents"; + interface Review { score: number; summary: string } + function main(): string { + const review = agents.run("researcher", "review it"); + return review.summary + " " + String(review.score); + }"#, + ) + .await + .expect("program completes"); + assert_eq!(out, "fine 7"); + let schema = agents.requests()[0].schema.clone().expect("a schema"); + assert!(schema.contains("\"score\""), "{schema}"); +} + +#[tokio::test] +async fn a_typed_run_throws_when_the_answer_does_not_match() { + let out = Harness::default() + .agents(FakeAgents::answering(r#"{"score": "high"}"#)) + .run( + r#"import agents from "submilli:agents"; + interface Review { score: number } + function main(): string { + try { + agents.run("researcher", "review it"); + return "accepted"; + } catch (e: TypeError) { + return "rejected"; + } + }"#, + ) + .await + .expect("program completes"); + assert_eq!(out, "rejected"); +} + +#[tokio::test] +async fn a_denied_run_never_reaches_the_harness() { + let agents = FakeAgents::answering("deployed"); + let policy = DenyOne::new("agent", "deployer"); + let out = Harness::default() + .agents(agents.clone()) + .policy(policy.clone()) + .run( + r#"import agents from "submilli:agents"; + function main(): string { + try { + agents.run("deployer", "ship it"); + return "ran"; + } catch (e: PermissionDeniedError) { + return "denied"; + } + }"#, + ) + .await + .expect("program completes"); + assert_eq!(out, "denied"); + assert!(agents.requests().is_empty()); + let contexts = policy.contexts.lock().expect("contexts").clone(); + assert_eq!(contexts, [serde_json::json!({ "agent": "deployer" })]); +} + +#[tokio::test] +async fn list_leaves_out_the_agents_the_caller_may_not_run() { + let out = Harness::default() + .agents(FakeAgents::answering("")) + .policy(DenyOne::new("agent", "deployer")) + .run( + r#"import agents from "submilli:agents"; + function main(): string { + return agents.list().map((a) => a.name + ":" + (a.description ?? "-")).join(","); + }"#, + ) + .await + .expect("program completes"); + assert_eq!(out, "researcher:Finds sources"); +} + +#[tokio::test] +async fn harness_failures_and_a_missing_provider_are_catchable() { + let source = r#"import agents from "submilli:agents"; + function main(): string { + try { agents.run("researcher", "go"); return "ran"; } + catch (e) { return e.message; } + }"#; + let failed = Harness::default() + .agents(FakeAgents::failing(AgentCallError::Failed { + agent: "researcher".to_string(), + message: "step limit reached".to_string(), + })) + .run(source) + .await + .expect("program completes"); + assert_eq!( + failed, + "agents.run: agent `researcher` failed: step limit reached" + ); + + let unconfigured = Harness::default() + .run(source) + .await + .expect("program completes"); + assert!( + unconfigured.contains("no agent provider is configured"), + "{unconfigured}" + ); +} + +#[tokio::test] +async fn skills_list_load_and_read_files() { + let skills = FakeSkills::new(); + let out = Harness::default() + .skills(skills.clone()) + .policy(DenyOne::new("name", "internal")) + .run( + r#"import skills from "submilli:skills"; + function main(): string { + const names = skills.list().map((s) => s.name).join(","); + const review = skills.load("review"); + const report = skills.readFile("review", "templates/report.md"); + return names + "|" + review.content + "|" + report; + }"#, + ) + .await + .expect("program completes"); + assert_eq!(out, "review|Read the diff twice.|# Report"); +} + +#[tokio::test] +async fn read_file_is_gated_by_the_skill_and_refuses_paths_outside_it() { + let skills = FakeSkills::new(); + let out = Harness::default() + .skills(skills.clone()) + .policy(DenyOne::new("name", "internal")) + .run( + r#"import skills from "submilli:skills"; + function main(): string { + let out = ""; + try { skills.readFile("internal", "SKILL.md"); out += "read "; } + catch (e: PermissionDeniedError) { out += "denied "; } + try { skills.readFile("review", "../secrets"); out += "escaped"; } + catch (e: RangeError) { out += "range"; } + return out; + }"#, + ) + .await + .expect("program completes"); + assert_eq!(out, "denied range"); + assert!( + skills.reads.lock().expect("reads").is_empty(), + "neither call may reach the harness" + ); +} + +#[test] +fn an_unknown_type_argument_is_refused_in_agents_words() { + let errors = compile_errors( + "import agents from \"submilli:agents\";\n\ + function main(): void { agents.run(\"researcher\", \"go\"); }\n", + harness_stdlib(), + ); + assert!( + errors.iter().any( + |e| e.contains("`agents.run` would not verify anything") + && e.contains("agents.run(agent, input)") + ), + "{errors:?}" + ); +} + +#[test] +fn a_parsed_script_typechecks_against_an_enabling_set() { + let source = "import { list } from \"submilli:skills\";\nfunction main(): void { list(); }\n"; + let parsed = crate::parse_script(source, crate::FileId(0)); + crate::compile::typecheck_parsed_checked( + source, + &parsed, + &harness_stdlib().package_declarations(), + ) + .expect("skills is available"); + assert!( + crate::compile::typecheck_parsed_checked( + source, + &parsed, + &Stdlib::core().package_declarations() + ) + .is_err() + ); +} + +#[test] +fn a_package_compiles_against_an_enabling_set() { + let source = "import { list } from \"submilli:agents\";\n\ + export function names(): string[] { return list().map((a) => a.name); }\n"; + let modules = [crate::PackageSourceModule { + path: crate::ModulePath::from("lib"), + source, + }]; + crate::compile::compile_package_with_transitive_checked_for( + harness_stdlib(), + "@acme/fanout", + crate::ModulePath::from("lib"), + &modules, + &[], + &[], + ) + .expect("agents is importable"); + assert!( + crate::compile::compile_package_with_transitive_checked( + "@acme/fanout", + crate::ModulePath::from("lib"), + &modules, + &[], + &[], + ) + .is_err() + ); +} + +#[tokio::test] +async fn a_typed_run_with_a_prose_answer_throws_a_syntax_error() { + let out = Harness::default() + .agents(FakeAgents::answering("Sure, here is the report")) + .run( + r#"import agents from "submilli:agents"; + interface Review { score: number } + function main(): string { + try { + agents.run("researcher", "review it"); + return "accepted"; + } catch (e: SyntaxError) { + return "not json"; + } + }"#, + ) + .await + .expect("program completes"); + assert_eq!(out, "not json"); +} + +#[tokio::test] +async fn an_empty_or_failed_listing_is_what_the_program_sees() { + let empty = Harness::default() + .agents(FakeAgents::listing(Ok(Vec::new()))) + .run( + r#"import agents from "submilli:agents"; + function main(): string { return String(agents.list().length); }"#, + ) + .await + .expect("program completes"); + assert_eq!(empty, "0"); + + let failed = Harness::default() + .agents(FakeAgents::listing(Err(AgentCallError::Failed { + agent: String::new(), + message: "catalog unavailable".to_string(), + }))) + .run( + r#"import agents from "submilli:agents"; + function main(): string { + try { agents.list(); return "listed"; } catch (e) { return e.message; } + }"#, + ) + .await + .expect("program completes"); + assert!(failed.contains("catalog unavailable"), "{failed}"); +} + +#[tokio::test] +async fn a_skill_name_that_is_not_one_segment_never_reaches_the_harness() { + let skills = FakeSkills::new(); + let out = Harness::default() + .skills(skills.clone()) + .run( + r#"import skills from "submilli:skills"; + function main(): string { + let out = ""; + for (const name of ["../other", "a/b", "", ".."]) { + try { skills.load(name); out += "loaded "; } + catch (e: RangeError) { out += "range "; } + } + try { skills.readFile("../../etc", "passwd"); out += "read"; } + catch (e: RangeError) { out += "range"; } + return out; + }"#, + ) + .await + .expect("program completes"); + assert_eq!(out, "range range range range range"); + assert!(skills.loads.lock().expect("loads").is_empty()); + assert!(skills.reads.lock().expect("reads").is_empty()); +} + +#[tokio::test] +async fn an_unknown_skill_and_a_missing_provider_are_catchable() { + let source = r#"import skills from "submilli:skills"; + function main(): string { + try { skills.load("nope"); return "loaded"; } catch (e) { return e.message; } + }"#; + let unknown = Harness::default() + .skills(FakeSkills::new()) + .run(source) + .await + .expect("program completes"); + assert_eq!( + unknown, + "skills.load: no skill named `nope`; call `list()` for the skills you may load" + ); + + let unconfigured = Harness::default() + .run(source) + .await + .expect("program completes"); + assert!( + unconfigured.contains("no skill provider is configured"), + "{unconfigured}" + ); +} + +#[tokio::test] +async fn a_namespaced_skill_name_reaches_the_harness() { + let skills = FakeSkills::new(); + let out = Harness::default() + .skills(skills.clone()) + .run( + r#"import skills from "submilli:skills"; + function main(): string { + try { skills.load("plugin:review"); return "loaded"; } catch (e) { return e.message; } + }"#, + ) + .await + .expect("program completes"); + assert!(out.contains("no skill named `plugin:review`"), "{out}"); + assert_eq!(*skills.loads.lock().expect("loads"), ["plugin:review"]); +} + +/// Answers every load with the `review` skill, whatever was asked for. +struct CaseFoldingSkills; + +impl SkillProvider for CaseFoldingSkills { + fn list<'a>(&'a self) -> Boxed<'a, Result, SkillError>> { + Box::pin(async { Ok(Vec::new()) }) + } + + fn load<'a>(&'a self, _name: &'a str) -> Boxed<'a, Result> { + Box::pin(async { + Ok(Skill { + name: "review".to_string(), + description: None, + content: "Read the diff twice.".to_string(), + }) + }) + } + + fn read_file<'a>( + &'a self, + _name: &'a str, + _path: &'a str, + ) -> Boxed<'a, Result> { + Box::pin(async { Ok(String::new()) }) + } +} + +/// The policy decided on the name the program wrote, so a skill answered under +/// another name is refused rather than handed over. +#[tokio::test] +async fn a_skill_answered_under_another_name_is_refused() { + let out = Harness::default() + .skills(Arc::new(CaseFoldingSkills)) + .policy(DenyOne::new("name", "review")) + .run( + r#"import skills from "submilli:skills"; + function main(): string { + try { return skills.load("Review").content; } catch (e) { return e.message; } + }"#, + ) + .await + .expect("program completes"); + assert!(out.contains("with a different skill"), "{out}"); +} + +#[tokio::test] +async fn an_oversized_name_is_refused_with_a_short_message() { + let out = Harness::default() + .skills(FakeSkills::new()) + .run( + r#"import skills from "submilli:skills"; + function main(): string { + try { skills.load("x".repeat(5000)); return "loaded"; } + catch (e: RangeError) { return e.message; } + }"#, + ) + .await + .expect("program completes"); + assert!(out.contains("at most 4096 bytes"), "{out}"); + assert!(out.len() < 600, "{}", out.len()); +} diff --git a/crates/interpreter/src/stdlib/shared.rs b/crates/interpreter/src/stdlib/shared.rs index 58d76b934..03bc2652e 100644 --- a/crates/interpreter/src/stdlib/shared.rs +++ b/crates/interpreter/src/stdlib/shared.rs @@ -203,7 +203,7 @@ pub(crate) fn authorize_capability( // existing, and the default check allows everything. if caller == crate::mangle::USER_PACKAGE && let Some(reason) = - crate::stdlib::capabilities::find(capability).and_then(|entry| entry.main_denial) + crate::stdlib::capabilities::find_any(capability).and_then(|entry| entry.main_denial) { audit_denial_at( security_check, @@ -556,6 +556,38 @@ pub(crate) fn filters_candidate(checked: wasmtime::Result<()>) -> wasmtime::Resu } } +/// The candidates of a listing the caller may use. `gate` runs the capability +/// check one candidate's use would; a policy denial leaves the candidate out and +/// marks the recorded decision filtered, and any other refusal propagates, the +/// [`filters_candidate`] rule. +pub(crate) fn permitted_candidates( + caller: &mut wasmtime::Caller<'_, StoreData>, + candidates: Vec, + name_of: impl Fn(&T) -> &str, + mut gate: impl FnMut(&mut wasmtime::Caller<'_, StoreData>, &str) -> wasmtime::Result<()>, +) -> wasmtime::Result> { + let mut permitted = Vec::with_capacity(candidates.len()); + for candidate in candidates { + if filters_candidate(gate(caller, name_of(&candidate)))? { + permitted.push(candidate); + } else { + mark_filtered(&*caller); + } + } + Ok(permitted) +} + +/// At most the first 200 bytes of a program-supplied value, for an error that +/// echoes it: a refused value may be refused for its size. +pub(crate) fn truncated(value: &str) -> std::borrow::Cow<'_, str> { + const MAX_ECHO_BYTES: usize = 200; + if value.len() <= MAX_ECHO_BYTES { + return std::borrow::Cow::Borrowed(value); + } + let end = value.floor_char_boundary(MAX_ECHO_BYTES); + std::borrow::Cow::Owned(format!("{}…", value.get(..end).unwrap_or_default())) +} + /// Reduce an operator-authored `description` to inert single-line data. /// /// This is sanitization, not merely a bound. The text flows verbatim into a @@ -583,6 +615,20 @@ pub(crate) fn preflight_models( caller: &mut wasmtime::Caller<'_, StoreData>, capability: &str, count_field: &str, +) -> wasmtime::Result<()> { + preflight_listing( + caller, + capability, + &serde_json::json!({ "model": "", count_field: 0 }), + ) +} + +/// [`preflight_models`] for any listing filtered per candidate: `context` is what +/// an invariant denial is audited with, the context of an empty candidate. +pub(crate) fn preflight_listing( + caller: &mut wasmtime::Caller<'_, StoreData>, + capability: &str, + context: &serde_json::Value, ) -> wasmtime::Result<()> { fuel::charge_host_fuel(&mut *caller, fuel::GATE)?; running_package(caller).map(|_| ()).map_err(|error| { @@ -591,7 +637,7 @@ pub(crate) fn preflight_models( &*caller, unknown.label, capability, - &serde_json::json!({ "model": "", count_field: 0 }), + context, "invariant", unknown.reason, ); diff --git a/crates/interpreter/src/stdlib/skills/declaration.rs b/crates/interpreter/src/stdlib/skills/declaration.rs new file mode 100644 index 000000000..9f925fdec --- /dev/null +++ b/crates/interpreter/src/stdlib/skills/declaration.rs @@ -0,0 +1,163 @@ +//! Type surface of `submilli:skills`. + +use std::collections::BTreeMap; + +use crate::{ + Dispatch, PackageDeclaration, Param, PropertySig, Span, Type, TypeKind, TypeSymbol, ValueKind, + ValueSymbol, +}; + +use super::MODULE_NAME; + +pub fn package_declaration() -> PackageDeclaration { + let mut defs = PackageDeclaration::with_package(MODULE_NAME); + insert_interface( + &mut defs, + "SkillInfo", + skill_properties(false), + "/** One skill this caller may load. Constructed only by `list()`. */", + ); + insert_interface( + &mut defs, + "Skill", + skill_properties(true), + "/** A loaded skill: its instructions, and the name to read its files under. Constructed only by `load()`. */", + ); + insert_fn( + &mut defs, + "list", + Vec::new(), + Type::Array(Box::new(interface_type("SkillInfo"))), + "/**\n * The skills this caller may load.\n *\n * Each skill the harness offers is \ + filtered under `skill.load` by the same `name` filter that gates `load`, so \ + the list never offers a skill the caller would be denied. Nothing in it \ + reveals how many were filtered out.\n * @capability skill.load\n */", + ); + insert_fn( + &mut defs, + "load", + vec![Param::new("name", Type::String)], + interface_type("Skill"), + "/**\n * Load the skill `name`: its description and its instructions.\n *\n * \ + A name that is not one skill name — empty, `.` or `..`, containing `/`, \ + `\\` or NUL, or over 4096 bytes — throws a `RangeError` without asking the \ + harness. Throws a catchable error when the harness has no skill of that name, \ + or when this runtime has no skill provider.\n * @param name Skill name, as \ + `list()` returns it.\n * @capability skill.load { name: $name }\n */", + ); + insert_fn( + &mut defs, + "readFile", + vec![ + Param::new("name", Type::String), + Param::new("path", Type::String), + ], + Type::String, + "/**\n * Read a file bundled with the skill `name`, as text.\n *\n * `path` is \ + relative to the skill, with `/` separators, as the skill's instructions name \ + it — `\"templates/report.md\"`. An absolute path, one with an empty, `.` \ + or `..` segment, one containing `\\`, `:` or NUL, or one over 4096 bytes, \ + throws a `RangeError` \ + without asking the harness, as does a name that is not one skill name; a file \ + the skill does not have throws a catchable error.\n * @param name Skill name, \ + as `list()` returns it.\n \ + * @param path The file's path inside the skill.\n * @capability skill.load { \ + name: $name }\n */", + ); + defs +} + +fn interface_type(name: &str) -> Type { + Type::InterfaceRef { + mangled: crate::mangle::package_symbol(MODULE_NAME, name), + package: crate::Package(MODULE_NAME.to_string()), + name: name.to_string(), + args: Vec::new(), + } +} + +/// `name` and `description`, plus `content` for a loaded skill. +fn skill_properties(loaded: bool) -> BTreeMap { + let mut properties = BTreeMap::new(); + insert_property( + &mut properties, + "name", + Type::String, + "/** The skill name, exactly as `load` and `readFile` expect it. */", + ); + insert_property( + &mut properties, + "description", + Type::union(vec![Type::String, Type::Undefined]), + "/** What the skill is for, or `undefined` when the harness gives none. */", + ); + if loaded { + insert_property( + &mut properties, + "content", + Type::String, + "/** The skill's instructions. */", + ); + } + properties +} + +fn insert_interface( + defs: &mut PackageDeclaration, + name: &str, + properties: BTreeMap, + doc: &str, +) { + defs.types.insert( + name.to_string(), + TypeSymbol { + name: name.to_string(), + mangled_name: crate::mangle::package_symbol(MODULE_NAME, name), + declaration_span: Span::at(crate::FileId::SKILLS), + kind: TypeKind::Interface { + index: None, + generics: Vec::new(), + methods: BTreeMap::new(), + properties, + dispatch: Dispatch::Direct, + doc: crate::doc(crate::FileId::SKILLS, doc), + }, + }, + ); +} + +fn insert_property( + properties: &mut BTreeMap, + name: &str, + ty: Type, + doc: &str, +) { + properties.insert( + name.to_string(), + PropertySig { + ty, + readonly: true, + optional: false, + intrinsic: false, + doc: crate::doc(crate::FileId::SKILLS, doc), + }, + ); +} + +fn insert_fn(defs: &mut PackageDeclaration, name: &str, params: Vec, ret: Type, doc: &str) { + defs.values.insert( + name.to_string(), + ValueSymbol { + name: name.to_string(), + mangled_name: crate::mangle::package_symbol(MODULE_NAME, name), + declaration_span: Span::at(crate::FileId::SKILLS), + kind: ValueKind::Function { + generics: Vec::new(), + params, + ret, + type_predicate: None, + doc: crate::doc(crate::FileId::SKILLS, doc), + }, + }, + ); +} diff --git a/crates/interpreter/src/stdlib/skills/mod.rs b/crates/interpreter/src/stdlib/skills/mod.rs new file mode 100644 index 000000000..c205f4612 --- /dev/null +++ b/crates/interpreter/src/stdlib/skills/mod.rs @@ -0,0 +1,499 @@ +//! `submilli:skills` — read the skills a harness offers from inside a Submilli +//! program. +//! +//! An optional package: it exists only for an embedder that enables +//! [`OptionalPackage::Skills`](crate::stdlib::OptionalPackage::Skills). The +//! harness serves skills through [`StoreData::skill_provider`]; a runtime with +//! none refuses every op with a catchable error. +//! +//! **One capability.** `list()`, `load` and `readFile` are all `skill.load`, +//! filtered by `name`. `list()` gates each candidate with the same filter, so a +//! listing never offers a skill the caller would be denied. + +pub mod declaration; + +use std::sync::Arc; + +use wasmtime::{FuncType, HeapType, Linker, RefType, StructType, Val, ValType}; + +use crate::runtime::StoreData; +use crate::runtime::call_log::{Payload, Side, record_payload}; +use crate::runtime::decision::CallTicket; +use crate::runtime::fuel; +use crate::runtime::host::{ + abi_arg, abi_result, range_error, read_string_arg, register_host_fn_async, + write_submilli_string_struct, +}; +use crate::runtime::intrinsic_types::build_intrinsic_types; +use crate::runtime::skills::{Skill, SkillError, SkillInfo, SkillProvider}; +use crate::stdlib::abi::{ + self, backing_struct, install_field_getters, nullable_object_field, string_field, +}; +use crate::stdlib::shared::{ + check_security_call, permitted_candidates, preflight_listing, sanitize_description, truncated, +}; + +pub use declaration::package_declaration; + +pub const MODULE_NAME: &str = crate::runtime::skills::SKILLS_MODULE_NAME; + +/// The capability gating every op in this package. +pub const CAPABILITY: &str = "skill.load"; + +// Backing field indices (0 is the vtable). `Skill` extends `SkillInfo`'s layout. +const S_NAME: usize = 1; +const S_DESCRIPTION: usize = 2; +const S_CONTENT: usize = 3; + +pub fn install(linker: &mut Linker) -> wasmtime::Result<()> { + let engine = linker.engine().clone(); + let intr = build_intrinsic_types(&engine)?; + let string = ValType::Ref(RefType::new( + false, + HeapType::ConcreteStruct(intr.string.clone()), + )); + let nullable_object = ValType::Ref(RefType::new( + true, + HeapType::ConcreteStruct(intr.object.clone()), + )); + let object = ValType::Ref(RefType::new( + false, + HeapType::ConcreteStruct(intr.object.clone()), + )); + let array = ValType::Ref(RefType::new( + false, + HeapType::ConcreteStruct(intr.array.clone()), + )); + + register_host_fn_async( + linker, + MODULE_NAME, + crate::mangle::package_symbol(MODULE_NAME, "list"), + FuncType::new(&engine, [], [array]), + /* deterministic = */ false, + |caller, _params, results| { + Box::pin(async move { + *abi_result(results, 0)? = list(caller).await?; + Ok(()) + }) + }, + )?; + + register_host_fn_async( + linker, + MODULE_NAME, + crate::mangle::package_symbol(MODULE_NAME, "load"), + // A `Skill` crosses as the universal `(ref null $Object)` lowering. + FuncType::new(&engine, [string.clone()], [nullable_object.clone()]), + /* deterministic = */ false, + |caller, params, results| { + Box::pin(async move { + let name = + read_string_arg(&mut *caller, abi_arg(params, 0)?, "skills.load (name)")?; + *abi_result(results, 0)? = load(caller, name).await?; + Ok(()) + }) + }, + )?; + + register_host_fn_async( + linker, + MODULE_NAME, + crate::mangle::package_symbol(MODULE_NAME, "readFile"), + FuncType::new(&engine, [string.clone(), string.clone()], [string.clone()]), + /* deterministic = */ false, + |caller, params, results| { + Box::pin(async move { + let name = + read_string_arg(&mut *caller, abi_arg(params, 0)?, "skills.readFile (name)")?; + let path = + read_string_arg(&mut *caller, abi_arg(params, 1)?, "skills.readFile (path)")?; + *abi_result(results, 0)? = read_file(caller, name, path).await?; + Ok(()) + }) + }, + )?; + + let info_rows = [ + ("name", S_NAME, string.clone()), + ("description", S_DESCRIPTION, nullable_object), + ]; + install_field_getters( + linker, + MODULE_NAME, + "SkillInfo", + &engine, + &object, + &info_rows, + )?; + let mut skill_rows = info_rows.to_vec(); + skill_rows.push(("content", S_CONTENT, string)); + install_field_getters(linker, MODULE_NAME, "Skill", &engine, &object, &skill_rows) +} + +/// The capability check, run before anything reaches the harness. +fn gate( + caller: &mut wasmtime::Caller<'_, StoreData>, + name: &str, +) -> wasmtime::Result> { + check_security_call(caller, CAPABILITY, serde_json::json!({ "name": name })) +} + +/// `list()`: every skill the harness offers that the caller may load. A denied +/// candidate is left out, never reported. Whether a provider is wired is not +/// hidden here: a listing has no candidate to gate before asking it. +async fn list(caller: &mut wasmtime::Caller<'_, StoreData>) -> wasmtime::Result { + preflight_listing(caller, CAPABILITY, &serde_json::json!({ "name": "" }))?; + let provider = provider(caller).map_err(|error| throw("list", &error))?; + let candidates = provider.list().await.map_err(|e| throw("list", &e))?; + // A name `load` would refuse is no use to the program, so it is not offered. + let loadable = candidates + .into_iter() + .filter(|skill| is_skill_name(&skill.name)) + .collect(); + let visible = permitted_candidates( + caller, + loadable, + |skill| skill.name.as_str(), + |caller, name| gate(caller, name).map(|_| ()), + )?; + let mut built = Vec::with_capacity(visible.len()); + for skill in visible { + built.push(build_skill_info(caller, skill)?); + } + abi::new_array(caller, &built) +} + +/// `load(name)`: gate, refuse what cannot be sent, record the request, then ask +/// the provider. A refusal before the provider is asked is recorded as the +/// call's error, with no request. +async fn load(caller: &mut wasmtime::Caller<'_, StoreData>, name: String) -> wasmtime::Result { + let ticket = gate(caller, &name)?; + if !is_skill_name(&name) { + return Err(fail( + caller, + ticket, + "load", + &SkillError::InvalidName { name }, + )); + } + let provider = provider(caller).map_err(|error| fail(caller, ticket, "load", &error))?; + record_payload(&*caller, ticket, Side::Request, || { + Payload::meta(serde_json::json!({ "op": "load", "name": name })) + }); + match provider.load(&name).await { + // The policy decided on `name`; a provider that answered with another + // skill would hand the program one the policy never saw. + Ok(skill) if skill.name != name => { + // The read happened, so it is paid for; its content is not kept. + fuel::settle(&mut *caller, fuel::IO, skill.content.len() as u64)?; + Err(fail( + caller, + ticket, + "load", + &SkillError::WrongSkill { name }, + )) + } + Ok(skill) => { + record_text(caller, ticket, &skill.content); + fuel::settle(&mut *caller, fuel::IO, skill.content.len() as u64)?; + fuel::settle_result(caller, |caller| build_skill(caller, skill)) + } + Err(error) => Err(fail(caller, ticket, "load", &error)), + } +} + +/// `readFile(name, path)`: in the order `load` follows, with the path checked +/// beside the name. +async fn read_file( + caller: &mut wasmtime::Caller<'_, StoreData>, + name: String, + path: String, +) -> wasmtime::Result { + let ticket = gate(caller, &name)?; + if !is_skill_name(&name) { + return Err(fail( + caller, + ticket, + "readFile", + &SkillError::InvalidName { name }, + )); + } + if !is_skill_path(&path) { + return Err(fail( + caller, + ticket, + "readFile", + &SkillError::InvalidPath { path }, + )); + } + let provider = provider(caller).map_err(|error| fail(caller, ticket, "readFile", &error))?; + record_payload(&*caller, ticket, Side::Request, || { + Payload::meta(serde_json::json!({ "op": "readFile", "name": name, "path": path })) + }); + match provider.read_file(&name, &path).await { + Ok(text) => { + record_text(caller, ticket, &text); + fuel::settle(&mut *caller, fuel::IO, text.len() as u64)?; + fuel::settle_result(caller, |caller| { + let text = write_submilli_string_struct(caller, &text)?; + Ok(Val::AnyRef(Some(text.to_anyref()))) + }) + } + Err(error) => Err(fail(caller, ticket, "readFile", &error)), + } +} + +/// The longest skill name, and the longest path, the engine passes to a +/// provider. +const MAX_SKILL_ARG_BYTES: usize = 4096; + +/// A skill name: one segment that is not `.` or `..`, with no `/`, `\\` or NUL. +/// `:` is allowed, for namespaced names such as `plugin:skill`; a provider must +/// not read a name as a path. +fn is_skill_name(name: &str) -> bool { + name.len() <= MAX_SKILL_ARG_BYTES + && !name.contains(['/', '\\', '\0']) + && !matches!(name, "" | "." | "..") +} + +/// A path inside a skill: relative, `/`-separated, with no empty, `.` or `..` +/// segment, and none of the characters a provider could read as structure — a +/// backslash or `:` (Windows separators, drives and streams) or NUL. +fn is_skill_path(path: &str) -> bool { + path.len() <= MAX_SKILL_ARG_BYTES + && !path.contains(['\\', ':', '\0']) + && path + .split('/') + .all(|segment| !matches!(segment, "" | "." | "..")) +} + +fn record_text(caller: &wasmtime::Caller<'_, StoreData>, ticket: Option, text: &str) { + record_payload(caller, ticket, Side::Response, || { + Payload::meta(serde_json::Value::Null) + .with_owned_body(text.as_bytes().to_vec()) + .with_size(text.len() as u64) + }); +} + +/// Record `error` as the call's response and turn it into the guest's error. +fn fail( + caller: &wasmtime::Caller<'_, StoreData>, + ticket: Option, + op: &str, + error: &SkillError, +) -> wasmtime::Error { + record_payload(caller, ticket, Side::Response, || { + Payload::meta(serde_json::json!({ "call_error": call_error_record(error) })) + }); + throw(op, error) +} + +/// Clone the provider out of the store before any `await`. `load` and +/// `readFile` look it up after the capability check, so a denied caller cannot +/// learn whether a harness is wired. +fn provider( + caller: &wasmtime::Caller<'_, StoreData>, +) -> Result, SkillError> { + caller + .data() + .skill_provider + .clone() + .ok_or(SkillError::NotConfigured) +} + +/// A name or path outside the skills is an argument the program got wrong; every +/// other failure keeps the base error type. +fn throw(op: &str, error: &SkillError) -> wasmtime::Error { + let message = format!("skills.{op}: {error}"); + if matches!( + error, + SkillError::InvalidName { .. } | SkillError::InvalidPath { .. } + ) { + range_error(message) + } else { + wasmtime::Error::msg(message) + } +} + +/// A failure as the call log keeps it: a stable kind and what it was about. +fn call_error_record(error: &SkillError) -> serde_json::Value { + match error { + SkillError::NotConfigured => serde_json::json!({ "kind": "not-configured" }), + SkillError::NotFound { name } => serde_json::json!({ "kind": "not-found", "name": name }), + SkillError::FileNotFound { name, path } => { + serde_json::json!({ "kind": "file-not-found", "name": name, "path": path }) + } + SkillError::InvalidName { name } => { + serde_json::json!({ "kind": "invalid-name", "name": truncated(name) }) + } + SkillError::WrongSkill { name } => { + serde_json::json!({ "kind": "wrong-skill", "name": name }) + } + SkillError::InvalidPath { path } => { + serde_json::json!({ "kind": "invalid-path", "path": truncated(path) }) + } + SkillError::Failed { message } => { + serde_json::json!({ "kind": "failed", "message": message }) + } + } +} + +fn info_backing_struct(engine: &wasmtime::Engine) -> wasmtime::Result { + let intr = build_intrinsic_types(engine)?; + backing_struct( + engine, + &intr, + vec![string_field(&intr), nullable_object_field(&intr)], + ) +} + +fn skill_backing_struct(engine: &wasmtime::Engine) -> wasmtime::Result { + let intr = build_intrinsic_types(engine)?; + backing_struct( + engine, + &intr, + vec![ + string_field(&intr), + nullable_object_field(&intr), + string_field(&intr), + ], + ) +} + +fn build_skill_info( + caller: &mut wasmtime::Caller<'_, StoreData>, + skill: SkillInfo, +) -> wasmtime::Result { + let name = write_submilli_string_struct(caller, &skill.name)?; + let description = listing_description(caller, skill.description.as_deref())?; + let ty = info_backing_struct(caller.engine())?; + abi::new_backing( + caller, + ty, + &[Val::AnyRef(Some(name.to_anyref())), description], + ) +} + +fn build_skill( + caller: &mut wasmtime::Caller<'_, StoreData>, + skill: Skill, +) -> wasmtime::Result { + let name = write_submilli_string_struct(caller, &skill.name)?; + let description = optional_text(caller, skill.description.as_deref())?; + let content = write_submilli_string_struct(caller, &skill.content)?; + let ty = skill_backing_struct(caller.engine())?; + abi::new_backing( + caller, + ty, + &[ + Val::AnyRef(Some(name.to_anyref())), + description, + Val::AnyRef(Some(content.to_anyref())), + ], + ) +} + +/// A listing shows a description as one bounded line, because it steers which +/// skill a program picks. +fn listing_description( + caller: &mut wasmtime::Caller<'_, StoreData>, + text: Option<&str>, +) -> wasmtime::Result { + optional_text(caller, text.and_then(sanitize_description).as_deref()) +} + +/// `text` as a string, or `undefined` when absent. +fn optional_text( + caller: &mut wasmtime::Caller<'_, StoreData>, + text: Option<&str>, +) -> wasmtime::Result { + match text { + Some(text) => Ok(Val::AnyRef(Some( + write_submilli_string_struct(caller, text)?.to_anyref(), + ))), + None => crate::runtime::prelude::undefined::value(caller), + } +} + +#[cfg(test)] +mod path_tests { + use super::{MAX_SKILL_ARG_BYTES, is_skill_name, is_skill_path}; + use crate::stdlib::shared::truncated; + + #[test] + fn only_relative_paths_inside_the_skill_are_accepted() { + let longest = "a".repeat(MAX_SKILL_ARG_BYTES); + for ok in [ + "SKILL.md", + "templates/report.md", + "a/b/c.txt", + ".hidden", + "notes/résumé ✓.md", + "%2e%2e/x", + longest.as_str(), + ] { + assert!(is_skill_path(ok), "{ok}"); + } + let too_long = "a".repeat(MAX_SKILL_ARG_BYTES + 1); + // 4095 ASCII bytes and a two-byte character: 4097 bytes, 4096 chars. + let straddling = format!("{}é", "a".repeat(MAX_SKILL_ARG_BYTES - 1)); + for bad in [ + "", + "/etc/passwd", + "a//b", + "./a", + "a/.", + "../x", + "a/../b", + "a/", + "a\\b", + "a\0b", + "a:b", + "C:/x", + "C:x", + "SKILL.md:stream", + too_long.as_str(), + straddling.as_str(), + ] { + assert!(!is_skill_path(bad), "{bad:?}"); + } + } + + #[test] + fn a_skill_name_is_one_segment_and_may_be_namespaced() { + let longest = "n".repeat(MAX_SKILL_ARG_BYTES); + for ok in [ + "code-review", + "code.review", + "plugin:skill", + "C:", + longest.as_str(), + ] { + assert!(is_skill_name(ok), "{ok}"); + } + let too_long = "n".repeat(MAX_SKILL_ARG_BYTES + 1); + for bad in [ + "", + ".", + "..", + "../other", + "a/b", + "a\\b", + "a\0b", + too_long.as_str(), + ] { + assert!(!is_skill_name(bad), "{bad:?}"); + } + } + + #[test] + fn an_echoed_value_is_cut_on_a_character_boundary() { + assert_eq!(truncated("short"), "short"); + let exactly = "a".repeat(200); + assert_eq!(truncated(&exactly), exactly.as_str()); + // Three-byte characters: byte 200 falls inside one, so the cut is at 198. + let cut = truncated(&"€".repeat(100)).into_owned(); + assert_eq!(cut, format!("{}…", "€".repeat(66))); + } +} diff --git a/crates/interpreter/src/typechecker/infer/expr.rs b/crates/interpreter/src/typechecker/infer/expr.rs index 2ebd345b0..b3765da8e 100644 --- a/crates/interpreter/src/typechecker/infer/expr.rs +++ b/crates/interpreter/src/typechecker/infer/expr.rs @@ -3510,8 +3510,9 @@ impl Inferer<'_> { }) } - /// Rewrite `llm.call(model, prompt)` into a runtime-checked cast, and - /// emit the JSON Schema for `T` that rides along to the provider. + /// Rewrite a schema-checked call such as `llm.call(model, prompt)` into a + /// runtime-checked cast, and emit the JSON Schema for `T` that rides along + /// to the provider. /// /// Mirrors [`Self::checked_session_get`] — same soundness argument, same /// `checked_cast_around` — with one addition: the schema gate. A typed call @@ -3525,20 +3526,30 @@ impl Inferer<'_> { /// has to replace the trailing `schema` argument before the call node's /// argument list is frozen. `Ok(None)` is the untyped form, which sends no /// schema and gets no check; `Ok(Some(schema))` means both gates passed and - /// the caller must wrap the call in [`Self::checked_llm_cast`]; `Err(())` + /// the caller must wrap the call in [`Self::schema_checked_cast`]; `Err(())` /// means a diagnostic was reported (or deliberately suppressed) and the /// call is an error. /// - /// Call only for a [`crate::stdlib::llm::declaration::is_checked_call`] - /// mangled name. `type_args_written` separates an explicit `call` - /// from a bare `call(...)`, whose result is the `Completion` envelope and - /// needs no check. - pub(super) fn llm_call_schema( + /// `checked` describes the call, from [`crate::stdlib::schema_checked_call`]. + /// `type_args_written` separates an explicit `call` from a bare + /// `call(...)`, whose result is the untyped default and needs no check. + pub(super) fn schema_checked_call_schema( &mut self, + checked: &crate::stdlib::SchemaCheckedCall, result_ty: &Type, type_args_written: bool, span: Span, ) -> Result, ()> { + let crate::stdlib::SchemaCheckedCall { + callee, + example, + checks, + answerer, + untyped_hint, + untyped_hint_for_unknown, + untyped_hint_for_no_schema, + .. + } = checked; // No type argument: the call keeps its pre-generic meaning and returns // the `Completion` envelope, which the declaration already typed. There // is nothing to check and no schema to send. @@ -3552,16 +3563,15 @@ impl Inferer<'_> { if matches!(result_ty.peel(), Type::Unknown) { self.error_with_help( span, - "`llm.call` would not verify anything: `unknown` admits every \ - value, so no runtime check is possible and its JSON Schema could only \ - be `{}`, which constrains the model to nothing" - .to_string(), - vec![ - "name the shape you expect — `llm.call(model, prompt)` — \ - or drop the type argument and read the `Completion` envelope's \ - `ok` and `text` yourself." - .to_string(), - ], + format!( + "`{callee}` would not verify anything: `unknown` admits every \ + value, so no runtime check is possible and its JSON Schema could only \ + be `{{}}`, which constrains {answerer} to nothing" + ), + vec![format!( + "name the shape you expect — `{example}` — or drop the type argument \ + and {untyped_hint_for_unknown}." + )], ); return Err(()); } @@ -3580,14 +3590,13 @@ impl Inferer<'_> { { self.error_with_help( span, - format!("`llm.call<{result_ty}>` cannot be verified at runtime: {reason}"), + format!("`{callee}<{result_ty}>` cannot be verified at runtime: {reason}"), vec![format!( - "`llm.call` checks the model's response against its type argument, so \ + "`{callee}` checks {checks} against its type argument, so \ that argument must be one the runtime can test, and the schema is \ emitted at compile time where `{result_ty}` is not yet known. Call it \ - at a concrete type — `llm.call(model, prompt)` — inside the \ - generic function, or drop the type argument and read the `Completion` \ - envelope yourself." + at a concrete type — `{example}` — inside the \ + generic function, or drop the type argument and {untyped_hint}." )], ); return Err(()); @@ -3597,7 +3606,7 @@ impl Inferer<'_> { // narrower (KTD5) *and* its rejections name the offending field. A // recursive type fails both; the schema reason points at the field that // closes the cycle, which is the one the author has to change. - match self.llm_schema_for(result_ty) { + match self.json_schema_for(result_ty) { Ok(schema) => { // The cast gate still has to pass: it is what `checked_cast_around` // will re-run, and the two surfaces agree everywhere except the @@ -3607,14 +3616,14 @@ impl Inferer<'_> { { self.error_with_help( span, - format!("`llm.call<{result_ty}>` cannot be verified at runtime: {reason}"), + format!("`{callee}<{result_ty}>` cannot be verified at runtime: {reason}"), vec![format!( - "`llm.call` checks the model's response against its type \ + "`{callee}` checks {checks} against its type \ argument, so that argument must be one the runtime can test: \ an object, array, tuple, union, or primitive shape. Call it \ at a concrete type instead of `{result_ty}` — \ - `llm.call(model, prompt)` — or drop the type \ - argument and read the `Completion` envelope yourself." + `{example}` — or drop the type \ + argument and {untyped_hint}." )], ); return Err(()); @@ -3628,18 +3637,16 @@ impl Inferer<'_> { self.error_with_help( span, format!( - "`llm.call<{result_ty}>` has no JSON Schema: {}", + "`{callee}<{result_ty}>` has no JSON Schema: {}", reject.describe() ), - vec![ - "the schema sent to the model is fully inlined and carries no \ + vec![format!( + "the schema sent to {answerer} is fully inlined and carries no \ `$ref`, so every field must be an object, array, tuple, union, \ enum, or primitive shape. Replace that field with one — a \ `string` for bytes or a big number, a named shape in place of \ - `unknown` — or drop the type argument and parse the \ - `Completion` text yourself." - .to_string(), - ], + `unknown` — or drop the type argument and {untyped_hint_for_no_schema}." + )], ); } Err(()) @@ -3647,12 +3654,13 @@ impl Inferer<'_> { } } - /// Wrap a gated `llm.call` in its checked cast. Split from - /// [`Self::llm_call_schema`] because the gates must run before the call - /// node's arguments are frozen and this must run after. `checked_cast_around` - /// cannot fail here: `llm_call_schema` already ran the same cast gate on the - /// same type, and passing it is what produced the schema that got us here. - pub(super) fn checked_llm_cast( + /// Wrap a gated schema-checked call in its checked cast. Split from + /// [`Self::schema_checked_call_schema`] because the gates must run before the + /// call node's arguments are frozen and this must run after. + /// `checked_cast_around` cannot fail here: `schema_checked_call_schema` + /// already ran the same cast gate on the same type, and passing it is what + /// produced the schema that got us here. + pub(super) fn schema_checked_cast( &mut self, call: TypedExprKind, result_ty: &Type, @@ -3663,7 +3671,7 @@ impl Inferer<'_> { Ok(checked) => checked, Err(reason) => { return Err(super::inference_failure(&format!( - "LLM result cast rejected a previously validated schema: {reason}" + "schema-checked result cast rejected a previously validated schema: {reason}" )) .with_span(span)); } @@ -3711,7 +3719,7 @@ impl Inferer<'_> { /// resolves alias back-edges through the type namespace — without that /// expander every `AliasRef` survives the walk and is rejected as /// unresolvable. - fn llm_schema_for( + fn json_schema_for( &mut self, target_ty: &Type, ) -> Result { diff --git a/crates/interpreter/src/typechecker/infer/generic.rs b/crates/interpreter/src/typechecker/infer/generic.rs index 01d2f4809..a3a6b2887 100644 --- a/crates/interpreter/src/typechecker/infer/generic.rs +++ b/crates/interpreter/src/typechecker/infer/generic.rs @@ -2252,11 +2252,12 @@ impl Inferer<'_> { // ordinary generic call; the two sites below are the two halves of that // one decision, so it is read once here. let checked_get = crate::stdlib::session::declaration::is_checked_get(&mangled); - // `llm.call` / `llm.batch` lower the same way and for the same - // reason. They differ from `session.get` only in what a bare call means - // — the `Completion` envelope rather than an unchecked `unknown` — and - // in carrying a compile-time schema into the trailing argument. - let checked_llm = crate::stdlib::llm::declaration::is_checked_call(&mangled); + // Schema-checked calls (`llm.call`, `llm.batch`, `agents.run`) + // lower the same way and for the same reason. They differ from + // `session.get` only in what a bare call means — a typed default such as + // the `Completion` envelope rather than an unchecked `unknown` — and in + // carrying a compile-time schema into the trailing argument. + let schema_checked = crate::stdlib::schema_checked_call(&mangled); if let Some(targs) = &type_args { if targs.len() != generics.len() { @@ -2371,11 +2372,13 @@ impl Inferer<'_> { // is not an unchecked read, it is a different fully-typed result. `T` // appears only in the return type, so it is never inferable from an // argument and would otherwise always trip the unbound-parameter error. - if checked_llm && !type_args_written { + if let Some(checked) = &schema_checked + && !type_args_written + { bind_remaining( &mut sub, &generics, - crate::stdlib::llm::declaration::untyped_result_type(&mangled), + checked.untyped_result.clone(), &self.type_limits, ) .map_err(type_limit_at(span))?; @@ -2431,30 +2434,34 @@ impl Inferer<'_> { // unaffected in behavior — `substitute_schema_argument` overwrites the // slot — but a written argument there is equally meaningless, so both // forms are refused and no working program changes. - if checked_llm && args.len() > 2 { + if let Some(checked) = &schema_checked + && args.len() > checked.written_param_count + { let name = &callee_ident.name; + let written = checked.written_params; + let extra = checked.schema_argument_ordinal(); self.error_with_help( span, format!( - "`{name}` takes the model and the prompt; the schema argument is filled by \ + "`{name}` takes {written}; the schema argument is filled by \ the compiler from the type argument" ), vec![format!( - "drop the third argument — write `{name}(...)` to send a schema for `T`, \ + "drop the {extra} argument — write `{name}(...)` to send a schema for `T`, \ or `{name}(...)` for an untyped call" )], ); return Ok((TypedExprKind::Null, Type::Error)); } - let mut llm_schema = None; - if checked_llm { - match self.llm_call_schema(&result_ty, type_args_written, span) { - Ok(schema) => llm_schema = schema, + let mut emitted_schema = None; + if let Some(checked) = &schema_checked { + match self.schema_checked_call_schema(checked, &result_ty, type_args_written, span) { + Ok(schema) => emitted_schema = schema, Err(()) => return Ok((TypedExprKind::Null, Type::Error)), } } - if let Some(schema) = &llm_schema { + if let Some(schema) = &emitted_schema { self.substitute_schema_argument(¶ms, &mut typed_args, schema, span)?; } self.record_generic_call_arguments(&typed_args, ¶ms, &ret, type_args_written); @@ -2497,7 +2504,7 @@ impl Inferer<'_> { mangled, type_args: runtime_args, args: generic_args, - return_cast: if checked_get || checked_llm { + return_cast: if checked_get || schema_checked.is_some() { None } else { return_cast @@ -2507,10 +2514,10 @@ impl Inferer<'_> { if checked_get { return self.checked_session_get(call, &result_ty, type_args_written, span); } - // An untyped `llm.call` emitted no schema and needs no check: it returns - // the `Completion` envelope the declaration already typed. - if checked_llm && llm_schema.is_some() { - return self.checked_llm_cast(call, &result_ty, span); + // An untyped schema-checked call emitted no schema and needs no check: it + // returns the default the declaration already typed. + if schema_checked.is_some() && emitted_schema.is_some() { + return self.schema_checked_cast(call, &result_ty, span); } Ok((call, result_ty)) } diff --git a/crates/interpreter/src/typechecker/infer/imports.rs b/crates/interpreter/src/typechecker/infer/imports.rs index e5fb3ff65..c040523ae 100644 --- a/crates/interpreter/src/typechecker/infer/imports.rs +++ b/crates/interpreter/src/typechecker/infer/imports.rs @@ -257,15 +257,8 @@ impl<'a> Inferer<'a> { ); continue; } - if module == crate::stdlib::test::MODULE_NAME { - self.error_with_help( - module_span, - format!("package `{module}` not found"), - vec![ - "`submilli:test` is only available to test files run via `submilli build test`; it is not importable from a program run with `submilli run`" - .to_string(), - ], - ); + if let Some((message, help)) = crate::stdlib::unavailable_import(&module) { + self.error_with_help(module_span, message, help); continue; } // In the closure but not declared here: the types resolve, the diff --git a/crates/interpreter/src/typechecker/infer/snapshots/interpreter__typechecker__infer__expr__tests__snapshot_mixed_program.snap b/crates/interpreter/src/typechecker/infer/snapshots/submilli_engine__typechecker__infer__expr__tests__snapshot_mixed_program.snap similarity index 100% rename from crates/interpreter/src/typechecker/infer/snapshots/interpreter__typechecker__infer__expr__tests__snapshot_mixed_program.snap rename to crates/interpreter/src/typechecker/infer/snapshots/submilli_engine__typechecker__infer__expr__tests__snapshot_mixed_program.snap diff --git a/crates/interpreter/src/typechecker/infer/snapshots/interpreter__typechecker__infer__format_definition__tests__array_substitutes_interface_generic.snap b/crates/interpreter/src/typechecker/infer/snapshots/submilli_engine__typechecker__infer__format_definition__tests__array_substitutes_interface_generic.snap similarity index 100% rename from crates/interpreter/src/typechecker/infer/snapshots/interpreter__typechecker__infer__format_definition__tests__array_substitutes_interface_generic.snap rename to crates/interpreter/src/typechecker/infer/snapshots/submilli_engine__typechecker__infer__format_definition__tests__array_substitutes_interface_generic.snap diff --git a/crates/interpreter/src/typechecker/infer/snapshots/interpreter__typechecker__infer__format_definition__tests__function_renders_arrow_form.snap b/crates/interpreter/src/typechecker/infer/snapshots/submilli_engine__typechecker__infer__format_definition__tests__function_renders_arrow_form.snap similarity index 100% rename from crates/interpreter/src/typechecker/infer/snapshots/interpreter__typechecker__infer__format_definition__tests__function_renders_arrow_form.snap rename to crates/interpreter/src/typechecker/infer/snapshots/submilli_engine__typechecker__infer__format_definition__tests__function_renders_arrow_form.snap diff --git a/crates/interpreter/src/typechecker/infer/snapshots/interpreter__typechecker__infer__format_definition__tests__interface_ref_substitutes_multiple_args.snap b/crates/interpreter/src/typechecker/infer/snapshots/submilli_engine__typechecker__infer__format_definition__tests__interface_ref_substitutes_multiple_args.snap similarity index 100% rename from crates/interpreter/src/typechecker/infer/snapshots/interpreter__typechecker__infer__format_definition__tests__interface_ref_substitutes_multiple_args.snap rename to crates/interpreter/src/typechecker/infer/snapshots/submilli_engine__typechecker__infer__format_definition__tests__interface_ref_substitutes_multiple_args.snap diff --git a/crates/interpreter/src/typechecker/infer/snapshots/interpreter__typechecker__infer__format_definition__tests__interface_renders_optional_property.snap b/crates/interpreter/src/typechecker/infer/snapshots/submilli_engine__typechecker__infer__format_definition__tests__interface_renders_optional_property.snap similarity index 100% rename from crates/interpreter/src/typechecker/infer/snapshots/interpreter__typechecker__infer__format_definition__tests__interface_renders_optional_property.snap rename to crates/interpreter/src/typechecker/infer/snapshots/submilli_engine__typechecker__infer__format_definition__tests__interface_renders_optional_property.snap diff --git a/crates/interpreter/src/typechecker/infer/snapshots/interpreter__typechecker__infer__format_definition__tests__interface_renders_properties.snap b/crates/interpreter/src/typechecker/infer/snapshots/submilli_engine__typechecker__infer__format_definition__tests__interface_renders_properties.snap similarity index 100% rename from crates/interpreter/src/typechecker/infer/snapshots/interpreter__typechecker__infer__format_definition__tests__interface_renders_properties.snap rename to crates/interpreter/src/typechecker/infer/snapshots/submilli_engine__typechecker__infer__format_definition__tests__interface_renders_properties.snap diff --git a/crates/interpreter/src/typechecker/infer/snapshots/interpreter__typechecker__infer__format_definition__tests__interface_substitutes_property_generic.snap b/crates/interpreter/src/typechecker/infer/snapshots/submilli_engine__typechecker__infer__format_definition__tests__interface_substitutes_property_generic.snap similarity index 100% rename from crates/interpreter/src/typechecker/infer/snapshots/interpreter__typechecker__infer__format_definition__tests__interface_substitutes_property_generic.snap rename to crates/interpreter/src/typechecker/infer/snapshots/submilli_engine__typechecker__infer__format_definition__tests__interface_substitutes_property_generic.snap diff --git a/crates/interpreter/src/typechecker/infer/snapshots/interpreter__typechecker__infer__format_definition__tests__object_renders_structural_shape.snap b/crates/interpreter/src/typechecker/infer/snapshots/submilli_engine__typechecker__infer__format_definition__tests__object_renders_structural_shape.snap similarity index 100% rename from crates/interpreter/src/typechecker/infer/snapshots/interpreter__typechecker__infer__format_definition__tests__object_renders_structural_shape.snap rename to crates/interpreter/src/typechecker/infer/snapshots/submilli_engine__typechecker__infer__format_definition__tests__object_renders_structural_shape.snap diff --git a/crates/interpreter/src/typechecker/infer/snapshots/interpreter__typechecker__infer__format_definition__tests__primitive_with_declared_methods.snap b/crates/interpreter/src/typechecker/infer/snapshots/submilli_engine__typechecker__infer__format_definition__tests__primitive_with_declared_methods.snap similarity index 100% rename from crates/interpreter/src/typechecker/infer/snapshots/interpreter__typechecker__infer__format_definition__tests__primitive_with_declared_methods.snap rename to crates/interpreter/src/typechecker/infer/snapshots/submilli_engine__typechecker__infer__format_definition__tests__primitive_with_declared_methods.snap diff --git a/crates/interpreter/src/typechecker/infer/snapshots/interpreter__typechecker__infer__type_diff__tests__function_arity_diff.snap b/crates/interpreter/src/typechecker/infer/snapshots/submilli_engine__typechecker__infer__type_diff__tests__function_arity_diff.snap similarity index 100% rename from crates/interpreter/src/typechecker/infer/snapshots/interpreter__typechecker__infer__type_diff__tests__function_arity_diff.snap rename to crates/interpreter/src/typechecker/infer/snapshots/submilli_engine__typechecker__infer__type_diff__tests__function_arity_diff.snap diff --git a/crates/interpreter/src/typechecker/infer/snapshots/interpreter__typechecker__infer__type_diff__tests__function_combined_arity_param_return.snap b/crates/interpreter/src/typechecker/infer/snapshots/submilli_engine__typechecker__infer__type_diff__tests__function_combined_arity_param_return.snap similarity index 100% rename from crates/interpreter/src/typechecker/infer/snapshots/interpreter__typechecker__infer__type_diff__tests__function_combined_arity_param_return.snap rename to crates/interpreter/src/typechecker/infer/snapshots/submilli_engine__typechecker__infer__type_diff__tests__function_combined_arity_param_return.snap diff --git a/crates/interpreter/src/typechecker/infer/snapshots/interpreter__typechecker__infer__type_diff__tests__function_param_type_diff.snap b/crates/interpreter/src/typechecker/infer/snapshots/submilli_engine__typechecker__infer__type_diff__tests__function_param_type_diff.snap similarity index 100% rename from crates/interpreter/src/typechecker/infer/snapshots/interpreter__typechecker__infer__type_diff__tests__function_param_type_diff.snap rename to crates/interpreter/src/typechecker/infer/snapshots/submilli_engine__typechecker__infer__type_diff__tests__function_param_type_diff.snap diff --git a/crates/interpreter/src/typechecker/infer/snapshots/interpreter__typechecker__infer__type_diff__tests__function_return_type_diff.snap b/crates/interpreter/src/typechecker/infer/snapshots/submilli_engine__typechecker__infer__type_diff__tests__function_return_type_diff.snap similarity index 100% rename from crates/interpreter/src/typechecker/infer/snapshots/interpreter__typechecker__infer__type_diff__tests__function_return_type_diff.snap rename to crates/interpreter/src/typechecker/infer/snapshots/submilli_engine__typechecker__infer__type_diff__tests__function_return_type_diff.snap diff --git a/crates/interpreter/src/typechecker/infer/snapshots/interpreter__typechecker__infer__type_diff__tests__object_all_three_dimensions.snap b/crates/interpreter/src/typechecker/infer/snapshots/submilli_engine__typechecker__infer__type_diff__tests__object_all_three_dimensions.snap similarity index 100% rename from crates/interpreter/src/typechecker/infer/snapshots/interpreter__typechecker__infer__type_diff__tests__object_all_three_dimensions.snap rename to crates/interpreter/src/typechecker/infer/snapshots/submilli_engine__typechecker__infer__type_diff__tests__object_all_three_dimensions.snap diff --git a/crates/interpreter/src/typechecker/infer/snapshots/interpreter__typechecker__infer__type_diff__tests__object_extra_field.snap b/crates/interpreter/src/typechecker/infer/snapshots/submilli_engine__typechecker__infer__type_diff__tests__object_extra_field.snap similarity index 100% rename from crates/interpreter/src/typechecker/infer/snapshots/interpreter__typechecker__infer__type_diff__tests__object_extra_field.snap rename to crates/interpreter/src/typechecker/infer/snapshots/submilli_engine__typechecker__infer__type_diff__tests__object_extra_field.snap diff --git a/crates/interpreter/src/typechecker/infer/snapshots/interpreter__typechecker__infer__type_diff__tests__object_field_type_mismatch.snap b/crates/interpreter/src/typechecker/infer/snapshots/submilli_engine__typechecker__infer__type_diff__tests__object_field_type_mismatch.snap similarity index 100% rename from crates/interpreter/src/typechecker/infer/snapshots/interpreter__typechecker__infer__type_diff__tests__object_field_type_mismatch.snap rename to crates/interpreter/src/typechecker/infer/snapshots/submilli_engine__typechecker__infer__type_diff__tests__object_field_type_mismatch.snap diff --git a/crates/interpreter/src/typechecker/infer/snapshots/interpreter__typechecker__infer__type_diff__tests__object_missing_field.snap b/crates/interpreter/src/typechecker/infer/snapshots/submilli_engine__typechecker__infer__type_diff__tests__object_missing_field.snap similarity index 100% rename from crates/interpreter/src/typechecker/infer/snapshots/interpreter__typechecker__infer__type_diff__tests__object_missing_field.snap rename to crates/interpreter/src/typechecker/infer/snapshots/submilli_engine__typechecker__infer__type_diff__tests__object_missing_field.snap diff --git a/crates/interpreter/tests/checked_arena.rs b/crates/interpreter/tests/checked_arena.rs index f5e6f5803..bb8b80c3c 100644 --- a/crates/interpreter/tests/checked_arena.rs +++ b/crates/interpreter/tests/checked_arena.rs @@ -1,4 +1,4 @@ -use interpreter::{ +use submilli_engine::{ Ast, Expr, ExprId, ExprKind, FileId, Span, Stmt, StmtId, StmtKind, Type, TypedAst, TypedExpr, TypedExprKind, TypedStmt, TypedStmtKind, arena::{ArenaError, ArenaKind, ArenaOperation}, diff --git a/crates/interpreter/tests/checked_rendering.rs b/crates/interpreter/tests/checked_rendering.rs index 69f76ca5b..55c8fc711 100644 --- a/crates/interpreter/tests/checked_rendering.rs +++ b/crates/interpreter/tests/checked_rendering.rs @@ -1,6 +1,6 @@ //! Malformed renderer inputs are injected internal state, not guest exploits. -use interpreter::rendering::{RenderError, RenderLimits, TRUNCATED}; -use interpreter::{Diagnostic, FileId, Severity, Sources, Span, Type}; +use submilli_engine::rendering::{RenderError, RenderLimits, TRUNCATED}; +use submilli_engine::{Diagnostic, FileId, Severity, Sources, Span, Type}; fn diagnostic(span: Span, message: String) -> Diagnostic { Diagnostic { @@ -30,16 +30,16 @@ fn invalid_metadata_preserves_original_failure_and_next_render_succeeds() { ] { let diag = diagnostic(span, "original failure".into()); assert!(matches!( - interpreter::diagnostics::render_checked(&diag, &sources), + submilli_engine::diagnostics::render_checked(&diag, &sources), Err(RenderError::Source(_)) )); - let fallback = interpreter::diagnostics::render(&diag, &sources); + let fallback = submilli_engine::diagnostics::render(&diag, &sources); assert!(fallback.contains("original failure")); assert!(fallback.contains("internal reporting failure")); assert!(!fallback.contains("test.ts:")); } let good = diagnostic(Span::new(file, 0, 2).unwrap(), "healthy error".into()); - let rendered = interpreter::diagnostics::render_checked(&good, &sources).unwrap(); + let rendered = submilli_engine::diagnostics::render_checked(&good, &sources).unwrap(); assert!(rendered.text.contains("test.ts:1:1")); assert!(!rendered.truncated); } @@ -52,7 +52,7 @@ fn huge_source_context_and_many_diagnostics_share_limits() { Span::new(file, 0, source.len() as u32).unwrap(), "original failure".into(), ); - let rendered = interpreter::diagnostics::render_checked(&diag, &sources).unwrap(); + let rendered = submilli_engine::diagnostics::render_checked(&diag, &sources).unwrap(); assert!(rendered.truncated); assert!(rendered.text.len() <= RenderLimits::default().bytes); assert!(rendered.text.contains("original failure")); @@ -60,10 +60,11 @@ fn huge_source_context_and_many_diagnostics_share_limits() { diag.span = Span::at(FileId::COMPILER); diag.message = "large error ".repeat(10_000); let diagnostics = vec![diag; 100]; - let collection = interpreter::diagnostics::render_collection(&diagnostics, &sources).unwrap(); + let collection = + submilli_engine::diagnostics::render_collection(&diagnostics, &sources).unwrap(); assert!(collection.truncated); assert!(collection.text.len() <= RenderLimits::collection().bytes); - let list = interpreter::diagnostics::render_list(&diagnostics, &sources).unwrap(); + let list = submilli_engine::diagnostics::render_list(&diagnostics, &sources).unwrap(); assert!(list.iter().map(String::len).sum::() <= RenderLimits::collection().bytes); assert!(list.last().unwrap().contains(TRUNCATED)); } @@ -102,7 +103,7 @@ fn type_depth_boundary_and_wide_types_abbreviate() { .unwrap() .truncated ); - let wide = Type::Tuple(interpreter::TupleType { + let wide = Type::Tuple(submilli_engine::TupleType { elements: vec![Type::Number; 100_000], optional: 0, }); @@ -173,11 +174,11 @@ fn oversized_message_does_not_hide_an_invalid_note() { diag.notes .push((Span::at(FileId(999)), "invalid note".into())); assert!(matches!( - interpreter::diagnostics::render_checked(&diag, &sources), + submilli_engine::diagnostics::render_checked(&diag, &sources), Err(RenderError::Source(_)) )); diag.notes = vec![(Span::at(file), "valid note".into()); 100_000]; - let rendered = interpreter::diagnostics::render_checked(&diag, &sources).unwrap(); + let rendered = submilli_engine::diagnostics::render_checked(&diag, &sources).unwrap(); assert!(rendered.truncated); assert!(rendered.text.contains("primary error")); } diff --git a/crates/interpreter/tests/checked_source.rs b/crates/interpreter/tests/checked_source.rs index ccbd6fb7f..fa9a6211e 100644 --- a/crates/interpreter/tests/checked_source.rs +++ b/crates/interpreter/tests/checked_source.rs @@ -1,4 +1,4 @@ -use interpreter::{ +use submilli_engine::{ Ast, Expr, ExprKind, FileId, LineIndex, Sources, Span, StmtKind, Type, TypeAnnotationKind, TypedAst, TypedExpr, TypedExprKind, compiler_error::{CompileError, CompilerFailure, CompilerStage}, @@ -68,7 +68,7 @@ fn checked_source_errors_preserve_classification_and_have_no_false_location() { let diagnostics = CompileError::from(failure).into_diagnostics(FileId(0)); assert_eq!(diagnostics[0].span, Span::at(FileId::COMPILER)); let (sources, _) = Sources::single("script.ts", "first line").unwrap(); - let rendered = interpreter::diagnostics::render(&diagnostics[0], &sources); + let rendered = submilli_engine::diagnostics::render(&diagnostics[0], &sources); assert!(rendered.contains("internal compiler failure")); assert!(!rendered.contains("first line")); assert!(!rendered.contains("script.ts:1:1")); @@ -95,14 +95,14 @@ fn checked_source_diagnostics_preserve_original_error_on_bad_metadata() { }, Span::at(FileId(999)), ] { - let diagnostic = interpreter::Diagnostic { - severity: interpreter::Severity::Error, + let diagnostic = submilli_engine::Diagnostic { + severity: submilli_engine::Severity::Error, span, message: "original failure".into(), help: vec![], notes: vec![], }; - let rendered = interpreter::diagnostics::render(&diagnostic, &sources); + let rendered = submilli_engine::diagnostics::render(&diagnostic, &sources); assert!(rendered.contains("original failure")); assert!(rendered.contains("source context unavailable")); assert!(!rendered.contains("script.ts:")); @@ -134,20 +134,21 @@ fn checked_source_registration_keeps_text_and_file_identity_together() { fn checked_source_inference_rejects_corrupt_type_metadata_before_resolution() { let source = "function main(x: number): number { return x; }"; for corrupt in 0..3 { - let parsed = interpreter::parse_script(source, FileId(0)); + let parsed = submilli_engine::parse_script(source, FileId(0)); assert!(!parsed.has_errors()); // Parse separately because ParsedScript intentionally encapsulates its AST. - let mut lexer = interpreter::Asi::new(source, FileId(0)); + let mut lexer = submilli_engine::Asi::new(source, FileId(0)); let mut tokens = Vec::new(); loop { let token = lexer.next_token(); - let eof = matches!(token.kind, interpreter::TokenKind::Eof); + let eof = matches!(token.kind, submilli_engine::TokenKind::Eof); tokens.push(token); if eof { break; } } - let (mut ast, _) = interpreter::parser::parse_checked(source, tokens, FileId(0)).unwrap(); + let (mut ast, _) = + submilli_engine::parser::parse_checked(source, tokens, FileId(0)).unwrap(); let root = ast.top_level[0]; let StmtKind::Function { params, .. } = &mut ast.try_stmt_mut(root).unwrap().kind else { panic!("function"); @@ -167,7 +168,7 @@ fn checked_source_inference_rejects_corrupt_type_metadata_before_resolution() { } } } - let error = interpreter::typechecker::infer::infer_with_transitive_checked( + let error = submilli_engine::typechecker::infer::infer_with_transitive_checked( source, "main", &ast, @@ -185,7 +186,7 @@ fn checked_source_inference_rejects_corrupt_type_metadata_before_resolution() { )); assert!(error.to_string().contains("invalid source span")); } - interpreter::compile::typecheck_checked("function main(): number { return 1; }", FileId(0)) + submilli_engine::compile::typecheck_checked("function main(): number { return 1; }", FileId(0)) .unwrap(); } @@ -198,8 +199,9 @@ fn checked_source_codegen_rejects_invalid_spans_before_emission() { span: Span::new(FileId(9), 0, 0).unwrap(), }) .unwrap(); - let error = interpreter::codegen::codegen_with_type_info("", "script.ts", FileId(0), &ast, &[]) - .unwrap_err(); + let error = + submilli_engine::codegen::codegen_with_type_info("", "script.ts", FileId(0), &ast, &[]) + .unwrap_err(); assert!(matches!( error, CompilerFailure::Internal { @@ -229,11 +231,11 @@ fn checked_source_direct_ast_validation_checks_unreachable_nodes_too() { fn checked_source_multiline_unicode_docs_keep_original_positions() { let source = "/**\n * @param {string} text description\n * @capability x/op { label: \"x\",\n * other: \"é😀é😀é😀\" }\n */\nfunction main(): number { return 1; }"; let file = FileId(0); - let raw = interpreter::doc_comment::RawDoc { + let raw = submilli_engine::doc_comment::RawDoc { text: source.split("\nfunction").next().unwrap().into(), span: Span::new(file, 0, source.find("\nfunction").unwrap() as u32).unwrap(), }; - let doc = interpreter::doc_comment::parse_doc_comment(&raw).unwrap(); + let doc = submilli_engine::doc_comment::parse_doc_comment(&raw).unwrap(); assert_eq!(doc.params[0].name_span.text(source, file).unwrap(), "text"); let capability = &doc.capabilities[0]; assert_eq!( @@ -243,12 +245,13 @@ fn checked_source_multiline_unicode_docs_keep_original_positions() { .unwrap(), "other" ); - let interpreter::DocCapabilityBindingKind::Literal { span, .. } = capability.bindings[1].kind + let submilli_engine::DocCapabilityBindingKind::Literal { span, .. } = + capability.bindings[1].kind else { panic!("literal"); }; assert_eq!(span.text(source, file).unwrap(), "\"é😀é😀é😀\""); - let parsed = interpreter::parse_script(source, file); + let parsed = submilli_engine::parse_script(source, file); assert!(!parsed.has_errors(), "{parsed:?}"); } @@ -266,7 +269,7 @@ fn checked_source_doc_unicode_errors_and_empty_first_lines_keep_valid_spans() { #[test] fn checked_source_auxiliary_spans_are_validated() { - use interpreter::ast::{ + use submilli_engine::ast::{ ArrayLiteralElement, ExportedDecl, Ident, ObjectLiteralMember, PatternOrigin, }; let file = FileId(0); @@ -284,7 +287,7 @@ fn checked_source_auxiliary_spans_are_validated() { }) .unwrap(); let statement = ast - .try_push_stmt(interpreter::Stmt { + .try_push_stmt(submilli_engine::Stmt { kind: StmtKind::Block(vec![]), span: Span::at(file), }) @@ -349,7 +352,7 @@ fn checked_source_package_validates_before_import_graph_errors() { start: 5, end: 1, }; - let error = interpreter::typechecker::infer::infer_package_checked( + let error = submilli_engine::typechecker::infer::infer_package_checked( "test", "index".into(), vec![("index".into(), file, &ast)], @@ -365,7 +368,7 @@ fn checked_source_package_validates_before_import_graph_errors() { .. }) )); - let error = interpreter::typechecker::infer::infer_package_checked( + let error = submilli_engine::typechecker::infer::infer_package_checked( "test", "index".into(), vec![("index".into(), file, &ast)], @@ -399,17 +402,17 @@ fn checked_source_validates_predicate_outer_span() { } fn parse_ast(source: &str, file: FileId) -> Ast { - let mut lexer = interpreter::Asi::new(source, file); + let mut lexer = submilli_engine::Asi::new(source, file); let mut tokens = Vec::new(); loop { let token = lexer.next_token(); - let eof = matches!(token.kind, interpreter::TokenKind::Eof); + let eof = matches!(token.kind, submilli_engine::TokenKind::Eof); tokens.push(token); if eof { break; } } - interpreter::parser::parse_checked(source, tokens, file) + submilli_engine::parser::parse_checked(source, tokens, file) .unwrap() .0 } diff --git a/crates/interpreter/tests/checked_typed_phases.rs b/crates/interpreter/tests/checked_typed_phases.rs index 619f3c720..cf519d44b 100644 --- a/crates/interpreter/tests/checked_typed_phases.rs +++ b/crates/interpreter/tests/checked_typed_phases.rs @@ -1,5 +1,5 @@ //! Corrupt IDs below are injected internal state, not guest-input reproducers. -use interpreter::{ +use submilli_engine::{ ExprId, FileId, StmtId, TypedAst, TypedExprKind, TypedStmtKind, compiler_error::{CompilerFailure, CompilerStage}, }; @@ -7,7 +7,7 @@ use interpreter::{ const SOURCE: &str = "function main(): number { return 1; }"; fn typed(source: &str) -> TypedAst { - let (ast, diagnostics) = interpreter::compile::typecheck_to_typed_ast(source, FileId(0)); + let (ast, diagnostics) = submilli_engine::compile::typecheck_to_typed_ast(source, FileId(0)); assert!(diagnostics.is_empty(), "{diagnostics:?}"); ast } @@ -41,11 +41,11 @@ fn rules_and_capture_propagate_invalid_expression_and_statement_ids() { } else { ast.functions[0].body = StmtId(u32::MAX); } - assert_invalid_id(interpreter::check(&ast).unwrap_err().fatal.unwrap()); - assert_invalid_id(interpreter::capture(ast).unwrap_err()); + assert_invalid_id(submilli_engine::check(&ast).unwrap_err().fatal.unwrap()); + assert_invalid_id(submilli_engine::capture(ast).unwrap_err()); } - assert!(interpreter::check(&original).unwrap().is_empty()); - interpreter::capture(original).unwrap(); + assert!(submilli_engine::check(&original).unwrap().is_empty()); + submilli_engine::capture(original).unwrap(); } #[test] @@ -55,7 +55,7 @@ fn rules_keep_diagnostics_collected_before_a_fatal_failure() { let first = ast.functions[0].body; ast.try_stmt_mut(first).unwrap().kind = TypedStmtKind::Block(Vec::new()); ast.functions[1].body = StmtId(u32::MAX); - let error = interpreter::check(&ast).unwrap_err(); + let error = submilli_engine::check(&ast).unwrap_err(); assert_invalid_id(error.fatal.unwrap()); assert!( error @@ -80,8 +80,8 @@ fn desugaring_rejects_an_invalid_loop_body() { } } assert!(replaced); - assert_invalid_id(interpreter::desugar(ast, FileId(0)).unwrap_err()); - interpreter::desugar(original, FileId(0)).unwrap(); + assert_invalid_id(submilli_engine::desugar(ast, FileId(0)).unwrap_err()); + submilli_engine::desugar(original, FileId(0)).unwrap(); } #[test] @@ -89,7 +89,7 @@ fn indirect_arena_reads_validate_ids_even_when_metadata_exists() { let mut ast = typed(SOURCE); let id = ExprId(u32::MAX); ast.runtime_source_types - .insert(id, interpreter::Type::Number); + .insert(id, submilli_engine::Type::Number); assert!(ast.source_type(id).is_err()); assert!(ast.is_effect_free(id).is_err()); let literal = ast @@ -100,16 +100,17 @@ fn indirect_arena_reads_validate_ids_even_when_metadata_exists() { assert!(ast.is_effect_free(literal).unwrap()); assert_eq!( ast.source_type(literal).unwrap(), - &interpreter::Type::Number + &submilli_engine::Type::Number ); } #[test] fn codegen_returns_failure_instead_of_partial_wasm() { let original = - interpreter::desugar(interpreter::capture(typed(SOURCE)).unwrap(), FileId(0)).unwrap(); + submilli_engine::desugar(submilli_engine::capture(typed(SOURCE)).unwrap(), FileId(0)) + .unwrap(); let (prelude, host, internal) = - interpreter::runtime::prelude::cached_runtime_package_declarations(); + submilli_engine::runtime::prelude::cached_runtime_package_declarations(); let dependencies: Vec<_> = prelude.iter().chain(host).chain(internal).collect(); for corrupt_expression in [false, true] { let mut ast = original.clone(); @@ -119,7 +120,7 @@ fn codegen_returns_failure_instead_of_partial_wasm() { ast.functions[0].body = StmtId(u32::MAX); } let failure = - interpreter::codegen::codegen(SOURCE, "typed.ts", FileId(0), &ast, &dependencies) + submilli_engine::codegen::codegen(SOURCE, "typed.ts", FileId(0), &ast, &dependencies) .unwrap_err(); assert!(matches!( failure, @@ -131,7 +132,7 @@ fn codegen_returns_failure_instead_of_partial_wasm() { assert_invalid_id(failure); } assert!( - !interpreter::codegen::codegen(SOURCE, "typed.ts", FileId(0), &original, &dependencies) + !submilli_engine::codegen::codegen(SOURCE, "typed.ts", FileId(0), &original, &dependencies) .unwrap() .is_empty() ); @@ -145,13 +146,13 @@ fn invalid_postfix_kind_is_a_typed_failure() { let mut replaced = false; for id in ast.expr_ids().unwrap() { if let TypedExprKind::PostfixUnary { op, .. } = &mut ast.try_expr_mut(id).unwrap().kind { - *op = interpreter::PostfixOp::NonNullAssert; + *op = submilli_engine::PostfixOp::NonNullAssert; replaced = true; } } assert!(replaced); assert!(matches!( - interpreter::desugar(ast, FileId(0)).unwrap_err(), + submilli_engine::desugar(ast, FileId(0)).unwrap_err(), CompilerFailure::Internal { stage: CompilerStage::Infer, .. @@ -161,7 +162,7 @@ fn invalid_postfix_kind_is_a_typed_failure() { #[test] fn capability_failure_retains_prior_warnings_and_does_not_return_a_filter() { - use interpreter::{ + use submilli_engine::{ DocCapability, DocCapabilityBinding, DocCapabilityBindingKind, Param, Span, Type, }; let span = Span::at(FileId(0)); @@ -184,8 +185,9 @@ fn capability_failure_retains_prior_warnings_and_does_not_return_a_filter() { }; let params = [Param::new("value", Type::Number)]; let ast = typed(SOURCE); - let error = interpreter::derive_call_site_capability(&tag, ¶ms, &ast, &[ExprId(u32::MAX)]) - .unwrap_err(); + let error = + submilli_engine::derive_call_site_capability(&tag, ¶ms, &ast, &[ExprId(u32::MAX)]) + .unwrap_err(); assert_invalid_id(error.fatal.unwrap()); assert_eq!(error.diagnostics.len(), 1); assert!(error.diagnostics[0].message.contains("unknown parameter")); @@ -195,7 +197,7 @@ fn capability_failure_retains_prior_warnings_and_does_not_return_a_filter() { .find(|id| matches!(ast.try_expr(*id).unwrap().kind, TypedExprKind::Number(_))) .unwrap(); assert!( - interpreter::derive_call_site_capability(&tag, ¶ms, &ast, &[value]) + submilli_engine::derive_call_site_capability(&tag, ¶ms, &ast, &[value]) .unwrap() .filter .is_some() @@ -250,15 +252,15 @@ fn desugaring_validates_rewritten_loop_references() { } } assert!(replaced); - assert_invalid_id(interpreter::desugar(ast, FileId(0)).unwrap_err()); + assert_invalid_id(submilli_engine::desugar(ast, FileId(0)).unwrap_err()); } - interpreter::desugar(original, FileId(0)).unwrap(); + submilli_engine::desugar(original, FileId(0)).unwrap(); } } #[test] fn desugaring_validates_postfix_operands() { - use interpreter::PostfixTarget; + use submilli_engine::PostfixTarget; for source in [ "function main(): void { const obj = { value: 1 }; obj.value++; }", "function main(): void { const arr = [1]; arr[0]++; }", @@ -289,15 +291,15 @@ fn desugaring_validates_postfix_operands() { } } assert!(replaced); - assert_invalid_id(interpreter::desugar(ast, FileId(0)).unwrap_err()); + assert_invalid_id(submilli_engine::desugar(ast, FileId(0)).unwrap_err()); } - interpreter::desugar(original, FileId(0)).unwrap(); + submilli_engine::desugar(original, FileId(0)).unwrap(); } } #[test] fn capability_failure_retains_deferred_http_warning() { - use interpreter::{ + use submilli_engine::{ DocCapability, DocCapabilityBinding, DocCapabilityBindingKind, Ident, Param, Span, Type, TypedExpr, }; @@ -340,9 +342,13 @@ fn capability_failure_retains_deferred_http_warning() { Param::new("url", Type::String), Param::new("other", Type::String), ]; - let error = - interpreter::derive_call_site_capability(&tag, ¶ms, &ast, &[dynamic, ExprId(u32::MAX)]) - .unwrap_err(); + let error = submilli_engine::derive_call_site_capability( + &tag, + ¶ms, + &ast, + &[dynamic, ExprId(u32::MAX)], + ) + .unwrap_err(); assert_invalid_id(error.fatal.unwrap()); assert_eq!(error.diagnostics.len(), 1); assert!( @@ -351,6 +357,7 @@ fn capability_failure_retains_deferred_http_warning() { .contains("cannot statically resolve the host") ); let healthy = - interpreter::derive_call_site_capability(&tag, ¶ms, &ast, &[dynamic, dynamic]).unwrap(); + submilli_engine::derive_call_site_capability(&tag, ¶ms, &ast, &[dynamic, dynamic]) + .unwrap(); assert_eq!(healthy.warnings.last(), error.diagnostics.first()); } diff --git a/crates/interpreter/tests/checked_untyped_phases.rs b/crates/interpreter/tests/checked_untyped_phases.rs index e668e6331..bd4ca782a 100644 --- a/crates/interpreter/tests/checked_untyped_phases.rs +++ b/crates/interpreter/tests/checked_untyped_phases.rs @@ -1,23 +1,23 @@ //! Injected arena corruption is an internal failure, not a language diagnostic. use std::collections::BTreeMap; -use interpreter::{ +use submilli_engine::{ Ast, ExprId, FileId, ModulePath, Sources, StmtId, StmtKind, compiler_error::{CompileError, CompilerFailure, CompilerStage}, }; fn parse(source: &str, file: FileId) -> Ast { - let mut lexer = interpreter::Asi::new(source, file); + let mut lexer = submilli_engine::Asi::new(source, file); let mut tokens = Vec::new(); loop { let token = lexer.next_token(); - let eof = matches!(token.kind, interpreter::TokenKind::Eof); + let eof = matches!(token.kind, submilli_engine::TokenKind::Eof); tokens.push(token); if eof { break; } } - interpreter::parser::parse_checked(source, tokens, file) + submilli_engine::parser::parse_checked(source, tokens, file) .unwrap() .0 } @@ -25,10 +25,11 @@ fn parse(source: &str, file: FileId) -> Ast { fn infer( source: &str, ast: &Ast, -) -> Result<(interpreter::TypedAst, Vec), CompileError> { - let (prelude, host, _) = interpreter::runtime::prelude::cached_runtime_package_declarations(); +) -> Result<(submilli_engine::TypedAst, Vec), CompileError> { + let (prelude, host, _) = + submilli_engine::runtime::prelude::cached_runtime_package_declarations(); let packages: Vec<_> = prelude.iter().chain(host).collect(); - interpreter::typechecker::infer::infer_with_transitive_checked( + submilli_engine::typechecker::infer::infer_with_transitive_checked( source, "main", ast, @@ -55,7 +56,7 @@ fn assert_internal(error: &CompileError) { #[test] fn inference_rejects_invalid_statement_and_expression_ids() { let source = "function main(): number { return 1; }"; - let original = interpreter::lower_patterns(parse(source, FileId(0))).unwrap(); + let original = submilli_engine::lower_patterns(parse(source, FileId(0))).unwrap(); for corrupt_expression in [false, true] { let mut ast = original.clone(); if corrupt_expression { @@ -76,7 +77,7 @@ fn inference_rejects_invalid_statement_and_expression_ids() { #[test] fn binding_failure_retains_earlier_language_diagnostics() { let source = "function main(): number { let value = value; return 1; }"; - let mut ast = interpreter::lower_patterns(parse(source, FileId(0))).unwrap(); + let mut ast = submilli_engine::lower_patterns(parse(source, FileId(0))).unwrap(); ast.top_level.push(StmtId(u32::MAX)); let error = infer(source, &ast).unwrap_err(); assert_internal(&error); @@ -94,7 +95,7 @@ fn binding_failure_retains_earlier_language_diagnostics() { fn lowering_rejects_bad_top_level_and_pattern_body_ids() { let mut ast = Ast::new(); ast.top_level.push(StmtId(u32::MAX)); - let error = interpreter::lower_patterns(ast).unwrap_err(); + let error = submilli_engine::lower_patterns(ast).unwrap_err(); assert_internal(&error.into()); let source = "function main([x]: number[]): number { return x; }"; @@ -104,14 +105,14 @@ fn lowering_rejects_bad_top_level_and_pattern_body_ids() { panic!("function") }; *body = StmtId(u32::MAX); - assert_internal(&interpreter::lower_patterns(ast).unwrap_err().into()); - interpreter::lower_patterns(parse(source, FileId(0))).unwrap(); + assert_internal(&submilli_engine::lower_patterns(ast).unwrap_err().into()); + submilli_engine::lower_patterns(parse(source, FileId(0))).unwrap(); } #[test] fn export_metadata_reads_are_checked_after_binding_analysis() { let source = "export function value(): number { return 1; }"; - let mut ast = interpreter::lower_patterns(parse(source, FileId(0))).unwrap(); + let mut ast = submilli_engine::lower_patterns(parse(source, FileId(0))).unwrap(); ast.exported_decls[0].stmt = StmtId(u32::MAX); assert_internal(&infer(source, &ast).unwrap_err()); } @@ -122,9 +123,9 @@ fn package_graph_failure_preserves_earlier_import_diagnostic() { let source = "import { value } from './missing'; export function read(): number { return value(); }"; let file = sources.add("lib.ts", source).unwrap(); - let mut ast = interpreter::lower_patterns(parse(source, file)).unwrap(); + let mut ast = submilli_engine::lower_patterns(parse(source, file)).unwrap(); ast.top_level.push(StmtId(u32::MAX)); - let error = interpreter::typechecker::infer::infer_package_checked( + let error = submilli_engine::typechecker::infer::infer_package_checked( "test", ModulePath::from("lib"), vec![(ModulePath::from("lib"), file, &ast)], @@ -150,21 +151,22 @@ fn later_module_arena_failure_preserves_earlier_module_diagnostics() { let last = "export function read(): number { return 1; }"; let first_file = sources.add("a.ts", first).unwrap(); let last_file = sources.add("lib.ts", last).unwrap(); - let first_ast = interpreter::lower_patterns(parse(first, first_file)).unwrap(); - let mut last_ast = interpreter::lower_patterns(parse(last, last_file)).unwrap(); + let first_ast = submilli_engine::lower_patterns(parse(first, first_file)).unwrap(); + let mut last_ast = submilli_engine::lower_patterns(parse(last, last_file)).unwrap(); let root = last_ast.top_level[0]; let StmtKind::Function { body, .. } = &mut last_ast.try_stmt_mut(root).unwrap().kind else { panic!("function") }; *body = StmtId(u32::MAX); - let (prelude, host, _) = interpreter::runtime::prelude::cached_runtime_package_declarations(); + let (prelude, host, _) = + submilli_engine::runtime::prelude::cached_runtime_package_declarations(); let packages = prelude .iter() .chain(host) .cloned() .map(|p| (p.package_name.clone(), p)) .collect(); - let error = interpreter::typechecker::infer::infer_package_checked( + let error = submilli_engine::typechecker::infer::infer_package_checked( "test", ModulePath::from("lib"), vec![ @@ -186,9 +188,9 @@ fn later_module_arena_failure_preserves_earlier_module_diagnostics() { #[test] fn inference_rejects_wrong_node_kinds_without_losing_prior_diagnostics() { - use interpreter::ExprKind; + use submilli_engine::ExprKind; let source = "function main(): number { let early = early; let x = 0; return (x = 1); }"; - let mut ast = interpreter::lower_patterns(parse(source, FileId(0))).unwrap(); + let mut ast = submilli_engine::lower_patterns(parse(source, FileId(0))).unwrap(); let target = ast .expr_ids() .unwrap() @@ -214,13 +216,13 @@ fn inference_rejects_wrong_node_kinds_without_losing_prior_diagnostics() { #[test] fn inference_rejects_wrong_function_and_loop_body_kinds() { - use interpreter::ExprKind; + use submilli_engine::ExprKind; for source in [ "function main(): number { const f = function(): number { return 1; }; return f(); }", "function main(): number { while (false) { return 1; } return 0; }", "function main(): number { return 1; }", ] { - let mut ast = interpreter::lower_patterns(parse(source, FileId(0))).unwrap(); + let mut ast = submilli_engine::lower_patterns(parse(source, FileId(0))).unwrap(); if source.contains("const f") { let function = ast .expr_ids() @@ -292,7 +294,7 @@ fn inference_fits_production_worker_stack() { &equality_chain, "function main(): number { return 42; }", ] { - let compiled = interpreter::compile::compile_script_checked( + let compiled = submilli_engine::compile::compile_script_checked( source, "stack.ts", FileId(0), diff --git a/crates/interpreter/tests/class_rec_group.rs b/crates/interpreter/tests/class_rec_group.rs index 3feb21f42..3a511f029 100644 --- a/crates/interpreter/tests/class_rec_group.rs +++ b/crates/interpreter/tests/class_rec_group.rs @@ -5,7 +5,7 @@ //! must still validate, and the per-class struct + vtable types must appear in //! the emitted rec group with the expected `$ObjectShape`/`$VTable` nesting. -use interpreter::{FileId, compile_script}; +use submilli_engine::{FileId, compile_script}; use wasmparser::{CompositeInnerType, Parser, Payload, Validator}; fn compile(source: &str) -> Vec { diff --git a/crates/interpreter/tests/closure_arity.rs b/crates/interpreter/tests/closure_arity.rs index 50086da11..53ba9e1aa 100644 --- a/crates/interpreter/tests/closure_arity.rs +++ b/crates/interpreter/tests/closure_arity.rs @@ -1,11 +1,11 @@ //! Supported closure ABI boundaries; rejection cases live in closure_arity_diagnostics. -use interpreter::compiler_limits::MAX_CLOSURE_ARITY; -use interpreter::runtime::{ +use submilli_engine::compiler_limits::MAX_CLOSURE_ARITY; +use submilli_engine::runtime::{ LinkedPackageModule, StoreData, Vfs, install_package_modules_async, install_runtime_async, install_tenant_limits, }; -use interpreter::{ +use submilli_engine::{ CompiledPackage, FileId, ModulePath, PackageSourceModule, RuntimeConfig, compile_package, compile_script, dispatch_main_async, }; diff --git a/crates/interpreter/tests/closure_arity_diagnostics.rs b/crates/interpreter/tests/closure_arity_diagnostics.rs index d6c755be2..f718c53b5 100644 --- a/crates/interpreter/tests/closure_arity_diagnostics.rs +++ b/crates/interpreter/tests/closure_arity_diagnostics.rs @@ -1,6 +1,10 @@ //! Source validation must run during check, before lowering to Wasm. -use interpreter::compile::{compile_package_checked, compile_script_checked, typecheck_checked}; -use interpreter::{Diagnostic, FileId, ModulePath, PackageSourceModule, Severity, Type, ValueKind}; +use submilli_engine::compile::{ + compile_package_checked, compile_script_checked, typecheck_checked, +}; +use submilli_engine::{ + Diagnostic, FileId, ModulePath, PackageSourceModule, Severity, Type, ValueKind, +}; fn parameters(arity: usize) -> String { (0..arity) @@ -96,7 +100,7 @@ fn constructors_keep_their_own_abi() { compile_script_checked(&source, "constructor.ts", FileId(0), &[], &[]).unwrap(); } -fn imported_declaration() -> interpreter::PackageDeclaration { +fn imported_declaration() -> submilli_engine::PackageDeclaration { compile_package_checked("aritydep", ModulePath::from("lib"), &[PackageSourceModule { path: ModulePath::from("lib"), source: "export function bad(a: number): number { return a; } export function healthy(): number { return 42; }", @@ -111,7 +115,7 @@ fn imported_signatures_are_checked_on_use_and_unused_surface_is_allowed() { panic!("function") }; *params = (0..256) - .map(|i| interpreter::Param::new(format!("a{i}"), Type::Number)) + .map(|i| submilli_engine::Param::new(format!("a{i}"), Type::Number)) .collect(); // Keep the physical ABI consistent so this tests payload arity, not corrupt metadata. let runtime_name = declaration.values.get("bad").unwrap().mangled_name.clone(); @@ -231,13 +235,13 @@ fn imported_interface_methods_are_validated_when_dispatched() { path: ModulePath::from("lib"), source: "export interface I { f(): number; } export function get(): I { return { f: (): number => 42 }; }", }], &[]).unwrap().declaration; - let interpreter::TypeKind::Interface { methods, .. } = + let submilli_engine::TypeKind::Interface { methods, .. } = &mut declaration.types.get_mut("I").unwrap().kind else { panic!("interface") }; methods.get_mut("f").unwrap().params = (0..256) - .map(|i| interpreter::Param::new(format!("a{i}"), Type::Number)) + .map(|i| submilli_engine::Param::new(format!("a{i}"), Type::Number)) .collect(); let args = (0..256) .map(|i| i.to_string()) @@ -312,7 +316,7 @@ fn static_only_class_usage_checks_required_callable_payloads() { path: ModulePath::from("lib"), source: "export class C { static f: (a: number) => number = (a: number): number => a; static healthy(): number { return 42; } }", }], &[]).unwrap().declaration; - let interpreter::TypeKind::Class { static_fields, .. } = + let submilli_engine::TypeKind::Class { static_fields, .. } = &mut declaration.types.get_mut("C").unwrap().kind else { panic!("class") @@ -339,13 +343,13 @@ fn static_calls_validate_unmentioned_instance_method_payloads() { path: ModulePath::from("lib"), source: "export class C { f(): number { return 42; } static healthy(): number { return 42; } }", }], &[]).unwrap().declaration; - let interpreter::TypeKind::Class { methods, .. } = + let submilli_engine::TypeKind::Class { methods, .. } = &mut declaration.types.get_mut("C").unwrap().kind else { panic!("class") }; methods.get_mut("f").unwrap().params = (0..256) - .map(|i| interpreter::Param::new(format!("a{i}"), Type::Number)) + .map(|i| submilli_engine::Param::new(format!("a{i}"), Type::Number)) .collect(); let source = "import { C } from \"aritydep\"; function main(): number { return C.healthy(); }"; let error = @@ -363,7 +367,7 @@ fn raw_host_functions_only_need_the_limit_when_adapted_to_closures() { panic!("function") }; *params = (0..256) - .map(|i| interpreter::Param::new(format!("a{i}"), Type::Number)) + .map(|i| submilli_engine::Param::new(format!("a{i}"), Type::Number)) .collect(); let args = (0..256) .map(|i| i.to_string()) @@ -434,7 +438,7 @@ fn inherited_statics_validate_the_declaring_class_only() { // Public exports and hidden runtime class surfaces share the physical signature. fn set_oversized_method( - declaration: &mut interpreter::PackageDeclaration, + declaration: &mut submilli_engine::PackageDeclaration, class: &str, method: &str, ) { @@ -445,14 +449,14 @@ fn set_oversized_method( .chain(declaration.runtime_types.values_mut()) .filter(|symbol| symbol.mangled_name == class_name) { - let interpreter::TypeKind::Class { methods, .. } = &mut symbol.kind else { + let submilli_engine::TypeKind::Class { methods, .. } = &mut symbol.kind else { panic!("class") }; methods.get_mut(method).unwrap().params = (0..256) - .map(|i| interpreter::Param::new(format!("a{i}"), Type::Number)) + .map(|i| submilli_engine::Param::new(format!("a{i}"), Type::Number)) .collect(); } - let runtime_name = interpreter::mangle::extend(&class_name, method); + let runtime_name = submilli_engine::mangle::extend(&class_name, method); declaration .runtime_functions .get_mut(&runtime_name) diff --git a/crates/interpreter/tests/closure_lowering_errors.rs b/crates/interpreter/tests/closure_lowering_errors.rs index dc2f6cb99..c2a88cda6 100644 --- a/crates/interpreter/tests/closure_lowering_errors.rs +++ b/crates/interpreter/tests/closure_lowering_errors.rs @@ -1,11 +1,11 @@ //! Compiler failures below exercise source limits and injected invalid codegen state. -use interpreter::codegen::{ +use submilli_engine::codegen::{ closures::{self, ClosureSig}, symbol_table::SymbolTable, }; -use interpreter::compile::{compile_package_checked, compile_script_checked}; -use interpreter::compiler_error::{CompilerFailure, CompilerStage}; -use interpreter::{FileId, ModulePath, PackageSourceModule, Type}; +use submilli_engine::compile::{compile_package_checked, compile_script_checked}; +use submilli_engine::compiler_error::{CompilerFailure, CompilerStage}; +use submilli_engine::{FileId, ModulePath, PackageSourceModule, Type}; fn signature(arity: usize, ret: Type) -> Type { Type::Function { diff --git a/crates/interpreter/tests/compiler_structure_limits.rs b/crates/interpreter/tests/compiler_structure_limits.rs index 44c1ec40b..89ac93dde 100644 --- a/crates/interpreter/tests/compiler_structure_limits.rs +++ b/crates/interpreter/tests/compiler_structure_limits.rs @@ -4,9 +4,9 @@ use std::process::Command; use std::time::{Duration, Instant}; -use interpreter::compiler_error::CompilerFailure; -use interpreter::compiler_limits::COMPILER_STACK_BYTES; -use interpreter::{ +use submilli_engine::compiler_error::CompilerFailure; +use submilli_engine::compiler_limits::COMPILER_STACK_BYTES; +use submilli_engine::{ FileId, compile::{compile_script_checked, typecheck_checked}, }; diff --git a/crates/interpreter/tests/console_failure.rs b/crates/interpreter/tests/console_failure.rs index f8a716212..1c76c421d 100644 --- a/crates/interpreter/tests/console_failure.rs +++ b/crates/interpreter/tests/console_failure.rs @@ -1,11 +1,11 @@ //! Console infrastructure failures must terminate execution even inside guest catch blocks. use std::io::{self, Write}; -use interpreter::runtime::host::FatalHostError; -use interpreter::runtime::{ +use submilli_engine::runtime::host::FatalHostError; +use submilli_engine::runtime::{ RuntimeConfig, StoreData, Vfs, install_runtime_async, install_tenant_limits, }; -use interpreter::{FileId, compile_script, dispatch_main_async, instantiate_program_async}; +use submilli_engine::{FileId, compile_script, dispatch_main_async, instantiate_program_async}; use wasmtime::{Linker, Module}; struct FailedWriter; @@ -23,7 +23,7 @@ impl Write for FailedWriter { #[test] fn console_writer_failure_bypasses_guest_catch_and_next_run_succeeds() { std::thread::Builder::new() - .stack_size(interpreter::compiler_limits::COMPILER_STACK_BYTES) + .stack_size(submilli_engine::compiler_limits::COMPILER_STACK_BYTES) .spawn(|| pollster::block_on(check_console_failure())) .unwrap() .join() diff --git a/crates/interpreter/tests/cross_module_gc_types.rs b/crates/interpreter/tests/cross_module_gc_types.rs index 60b99e8f9..b38d72e1b 100644 --- a/crates/interpreter/tests/cross_module_gc_types.rs +++ b/crates/interpreter/tests/cross_module_gc_types.rs @@ -2,7 +2,7 @@ use std::borrow::Cow; -use interpreter::RuntimeConfig; +use submilli_engine::RuntimeConfig; use wasm_encoder::{ BlockType, CodeSection, CompositeInnerType, CompositeType, ElementSection, Elements, ExportKind, ExportSection, FieldType, Function, FunctionSection, HeapType, ImportSection, diff --git a/crates/interpreter/tests/decision_records.rs b/crates/interpreter/tests/decision_records.rs index ba3b05def..4a9e92149 100644 --- a/crates/interpreter/tests/decision_records.rs +++ b/crates/interpreter/tests/decision_records.rs @@ -4,8 +4,9 @@ use std::sync::atomic::{AtomicU64, Ordering}; use std::sync::{Arc, Mutex}; -use interpreter::runtime::security::AuditDecision; -use interpreter::runtime::{ +use serde_json::{Value, json}; +use submilli_engine::runtime::security::AuditDecision; +use submilli_engine::runtime::{ Access, BodyCopy, CallOutcome, CallRecord, CheckOutcome, DecisionAction, DecisionCause, DecisionExplanation, DecisionLog, DecisionLogConfig, DecisionLogOutput, DecisionRecord, EmbeddingBatch, EmbeddingError, EmbeddingLimits, EmbeddingModel, EmbeddingProvider, @@ -16,15 +17,14 @@ use interpreter::runtime::{ install_package_modules_async, install_runtime_host_functions, install_runtime_store_bound, install_tenant_limits, limits::ExecutionUsage, }; -use interpreter::stdlib::git::GitConfig; -use interpreter::stdlib::http::transport::{ +use submilli_engine::stdlib::git::GitConfig; +use submilli_engine::stdlib::http::transport::{ DownloadMeta, EgressAt, HttpClient, HttpError, HttpRequest, HttpResponse, RedirectHop, }; -use interpreter::{ +use submilli_engine::{ CompiledPackage, FileId, ModulePath, PackageSourceModule, RuntimeConfig, compile_package_with_transitive, compile_script, dispatch_main_async, }; -use serde_json::{Value, json}; use wasmtime::{Linker, Module}; // ---- fixtures ------------------------------------------------------------------------- @@ -502,7 +502,7 @@ impl EmbeddingProvider for FakeEmbedding { &'a self, alias: &'a str, texts: &'a [String], - _purpose: interpreter::runtime::Purpose, + _purpose: submilli_engine::runtime::Purpose, budget: &'a EmbeddingTokenBudget, ) -> std::pin::Pin< Box> + Send + 'a>, @@ -514,14 +514,14 @@ impl EmbeddingProvider for FakeEmbedding { // or after none reported usage (`dark`). let failed_after = |reported| EmbeddingError::Provider { alias: alias.to_string(), - reason: interpreter::runtime::EmbeddingFailureReason::Transport, + reason: submilli_engine::runtime::EmbeddingFailureReason::Transport, settlements: vec![ - interpreter::runtime::SubBatchSettlement { + submilli_engine::runtime::SubBatchSettlement { estimate: 4, reported, indeterminate: 4 - reported, }, - interpreter::runtime::SubBatchSettlement { + submilli_engine::runtime::SubBatchSettlement { estimate: 3, reported: 0, indeterminate: 3, @@ -542,7 +542,7 @@ impl EmbeddingProvider for FakeEmbedding { ) .expect("shape") .with_input_tokens(Some(7)) - .with_settlements(vec![interpreter::runtime::SubBatchSettlement { + .with_settlements(vec![submilli_engine::runtime::SubBatchSettlement { estimate, reported: 7, indeterminate: 0, @@ -1519,7 +1519,7 @@ function main(): string { get("https://example.test/a"); return "done"; } // ---- throwaway cost measurement --------------------------------------------------------------- -/// Run with `cargo test --release -p interpreter --test decision_records -- --ignored +/// Run with `cargo test --release -p submilli-engine --test decision_records -- --ignored /// --nocapture line_lookup_cost`; prints per-call costs, asserts nothing about time. #[tokio::test] #[ignore = "timing measurement, not a check"] @@ -1731,7 +1731,7 @@ function main(): string { return llm.call("open", "what is two plus two").text ? assert!(body_text(response.body.as_ref()).is_some_and(|t| t.contains("answer"))); assert_eq!( call.usage, - Some(interpreter::runtime::ModelUsage { + Some(submilli_engine::runtime::ModelUsage { input_tokens: Some(10), output_tokens: Some(10), }) @@ -1783,7 +1783,7 @@ function main(): string { assert_eq!(response.bytes, 2 * 4 * 4); assert_eq!( call.usage, - Some(interpreter::runtime::ModelUsage { + Some(submilli_engine::runtime::ModelUsage { input_tokens: Some(7), output_tokens: None, }) @@ -1795,7 +1795,7 @@ async fn a_failed_embed_call_keeps_the_usage_its_sent_batches_reported() { for (alias, usage) in [ ( "open-embed-flaky", - Some(interpreter::runtime::ModelUsage { + Some(submilli_engine::runtime::ModelUsage { input_tokens: Some(4), output_tokens: None, }), @@ -1872,7 +1872,7 @@ function main(): string { // The call's own usage stays the sum, absent where any prompt left a count out. assert_eq!( call.usage, - Some(interpreter::runtime::ModelUsage { + Some(submilli_engine::runtime::ModelUsage { input_tokens: Some(13), output_tokens: None, }) diff --git a/crates/interpreter/tests/determinism.rs b/crates/interpreter/tests/determinism.rs index e413c1c8d..03de9e86e 100644 --- a/crates/interpreter/tests/determinism.rs +++ b/crates/interpreter/tests/determinism.rs @@ -11,7 +11,7 @@ use std::fs; use std::path::{Path, PathBuf}; -use interpreter::{ +use submilli_engine::{ CompiledPackage, Diagnostic, FileId, ModulePath, PackageSourceModule, compile_package, compile_script, }; @@ -165,7 +165,7 @@ fn check_in_parallel(paths: Vec) -> Report { .into_iter() .map(|chunk| { std::thread::Builder::new() - .stack_size(interpreter::compiler_limits::COMPILER_STACK_BYTES) + .stack_size(submilli_engine::compiler_limits::COMPILER_STACK_BYTES) .spawn(move || find_unstable(&chunk)) .expect("spawn compile worker") }) diff --git a/crates/interpreter/tests/diagnostic_snapshots.rs b/crates/interpreter/tests/diagnostic_snapshots.rs index 61992fb74..4d2f9c0b6 100644 --- a/crates/interpreter/tests/diagnostic_snapshots.rs +++ b/crates/interpreter/tests/diagnostic_snapshots.rs @@ -2,7 +2,7 @@ //! Each test drives a small .subm program through the full pipeline and //! renders every emitted diagnostic; this file is the integration guarantee. -use interpreter::{ +use submilli_engine::{ Asi, FileId, Severity, Sources, Token, TokenKind, check, diagnostics, infer, parse, }; @@ -26,7 +26,7 @@ fn render_all(source: &str) -> String { // a partial AST would just produce cascading noise. if !diags.iter().any(|d| d.severity == Severity::Error) { let (prelude_defs, host_defs, _) = - interpreter::runtime::prelude::cached_runtime_package_declarations(); + submilli_engine::runtime::prelude::cached_runtime_package_declarations(); let mut package_refs = Vec::with_capacity(prelude_defs.len() + host_defs.len()); package_refs.extend(prelude_defs.iter()); package_refs.extend(host_defs.iter()); diff --git a/crates/interpreter/tests/field_lookup_size.rs b/crates/interpreter/tests/field_lookup_size.rs index 5ef5e9a7e..be3a99f08 100644 --- a/crates/interpreter/tests/field_lookup_size.rs +++ b/crates/interpreter/tests/field_lookup_size.rs @@ -1,4 +1,4 @@ -use interpreter::{ModulePath, PackageSourceModule, compile_package}; +use submilli_engine::{ModulePath, PackageSourceModule, compile_package}; fn module_size(accesses: usize, write: bool) -> usize { let statement = if write { diff --git a/crates/interpreter/tests/fixtures.rs b/crates/interpreter/tests/fixtures.rs index b15089ba0..4fdbaca70 100644 --- a/crates/interpreter/tests/fixtures.rs +++ b/crates/interpreter/tests/fixtures.rs @@ -32,7 +32,7 @@ use std::io::Write; use std::path::{Path, PathBuf}; use std::sync::{Arc, Mutex}; -use interpreter::runtime::{ +use submilli_engine::runtime::{ EmbeddingBatch, EmbeddingError, EmbeddingFailureReason, EmbeddingLimits, EmbeddingMalformedReason, EmbeddingModel, EmbeddingProvider, EmbeddingTokenBudget, ExecutionTokenBudget, FailureReason, InMemorySessionKv, LinkedPackageModule, LlmCallError, @@ -40,7 +40,7 @@ use interpreter::runtime::{ StoreData, SubBatchSettlement, Vfs, estimate_embedding_tokens, install_package_modules_async, install_runtime_host_functions, install_runtime_store_bound, install_tenant_limits, }; -use interpreter::{ +use submilli_engine::{ Asi, BacktraceMode, CompiledPackage, Diagnostic, ModulePath, PackageDeclaration, PackageSourceModule, RunResult, RuntimeConfig, Sources, Token, TokenKind, compile_package_with_transitive, compile_script, diagnostics, infer_package, lower_patterns, @@ -115,7 +115,7 @@ impl PreparedRuntime { async fn run( &self, - compiled: &interpreter::compile::CompiledScript, + compiled: &submilli_engine::compile::CompiledScript, deny_capabilities: &[String], deny_llm_models: &[String], deny_embedding_models: &[String], @@ -156,7 +156,7 @@ impl PreparedRuntime { install_runtime_store_bound(&mut linker, &mut store)?; let inst = linker.instantiate_async(&mut store, &module).await?; let _watchdog = self.config.arm_timeout(&self.engine)?; - let value = interpreter::dispatch_main_async(&mut store, &inst).await?; + let value = submilli_engine::dispatch_main_async(&mut store, &inst).await?; let captured = buf.lock().unwrap().clone(); let console = String::from_utf8(captured) .map_err(|e| wasmtime::Error::msg(format!("console output not utf-8: {e}")))?; @@ -171,7 +171,7 @@ impl PreparedRuntime { async fn run_packages( &self, deps: &[&CompiledPackage], - root: &interpreter::compile::CompiledScript, + root: &submilli_engine::compile::CompiledScript, deny_capabilities: &[String], fixture: &str, ) -> wasmtime::Result { @@ -206,7 +206,7 @@ impl PreparedRuntime { let root_module = Module::new(&self.engine, &root.wasm)?; let inst = linker.instantiate_async(&mut store, &root_module).await?; let _watchdog = self.config.arm_timeout(&self.engine)?; - let value = interpreter::dispatch_main_async(&mut store, &inst).await?; + let value = submilli_engine::dispatch_main_async(&mut store, &inst).await?; Ok(RunResult { value, console: String::new(), @@ -226,7 +226,7 @@ fn run_fixtures_parallel(paths: Vec, runtime: Arc) -> let runtime = Arc::clone(&runtime); handles.push( std::thread::Builder::new() - .stack_size(interpreter::compiler_limits::COMPILER_STACK_BYTES) + .stack_size(submilli_engine::compiler_limits::COMPILER_STACK_BYTES) .spawn(move || { let tokio = tokio::runtime::Builder::new_current_thread() .enable_all() @@ -408,7 +408,7 @@ fn run_one( let filename = rel(path); let expectations = parse_expectations(&src)?; - let compiled = compile_script(&src, &filename, interpreter::FileId(0), &[], &[]); + let compiled = compile_script(&src, &filename, submilli_engine::FileId(0), &[], &[]); let diags = match &compiled { Ok(compiled) => &compiled.warnings, Err(diags) => diags, @@ -495,13 +495,13 @@ fn run_multi(path: &Path) -> Result<(), String> { owned_asts.push((module, file_id, ast)); } - let stdlib_defs = interpreter::runtime::stdlib_package_declarations(); + let stdlib_defs = submilli_engine::runtime::stdlib_package_declarations(); let mut external_packages: std::collections::BTreeMap = stdlib_defs .into_iter() .map(|defs| (defs.package_name.clone(), defs)) .collect(); let (prelude_defs, host_defs, _) = - interpreter::runtime::prelude::cached_runtime_package_declarations(); + submilli_engine::runtime::prelude::cached_runtime_package_declarations(); for defs in prelude_defs { external_packages.insert(defs.package_name.clone(), defs.clone()); } @@ -526,7 +526,7 @@ fn run_multi(path: &Path) -> Result<(), String> { match ( all_diags .iter() - .any(|d| d.severity == interpreter::Severity::Error), + .any(|d| d.severity == submilli_engine::Severity::Error), error_needles.as_slice(), ) { (false, n) if !n.is_empty() => Err(format!( @@ -743,7 +743,7 @@ fn run_packages_fixture( let root_script = match compile_script( root_src, root_path.as_str(), - interpreter::FileId(0), + submilli_engine::FileId(0), &all_decls, &[], ) { @@ -830,7 +830,10 @@ fn assert_diagnostic_count(expected: Option, diags: &[Diagnostic]) -> Res Ok(()) } -fn parse_source(src: &str, file: interpreter::FileId) -> (interpreter::Ast, Vec) { +fn parse_source( + src: &str, + file: submilli_engine::FileId, +) -> (submilli_engine::Ast, Vec) { let mut asi = Asi::new(src, file); let mut tokens: Vec = Vec::new(); loop { @@ -1389,21 +1392,21 @@ fn install_embedding_fixture_support(data: &mut StoreData, fixture: &str) { /// where filter-conditional denial gets its integration coverage. struct FixtureDenyLlmModel(Vec); -impl interpreter::runtime::SecurityCheck for FixtureDenyLlmModel { +impl submilli_engine::runtime::SecurityCheck for FixtureDenyLlmModel { fn check( &self, _caller: &str, capability: &str, context: &serde_json::Value, - ) -> interpreter::runtime::CheckOutcome { + ) -> submilli_engine::runtime::CheckOutcome { let model = context.get("model").and_then(serde_json::Value::as_str); if capability == "llm.call" && model.is_some_and(|m| self.0.iter().any(|d| d == m)) { - return interpreter::runtime::CheckOutcome::Deny { + return submilli_engine::runtime::CheckOutcome::Deny { rule: None, reason: "denied by fixture model filter".to_string(), }; } - interpreter::runtime::CheckOutcome::Allow { rule: None } + submilli_engine::runtime::CheckOutcome::Allow { rule: None } } } @@ -1411,21 +1414,21 @@ impl interpreter::runtime::SecurityCheck for FixtureDenyLlmModel { /// `embedding.embed` for the named aliases, keyed off the check context. struct FixtureDenyEmbeddingModel(Vec); -impl interpreter::runtime::SecurityCheck for FixtureDenyEmbeddingModel { +impl submilli_engine::runtime::SecurityCheck for FixtureDenyEmbeddingModel { fn check( &self, _caller: &str, capability: &str, context: &serde_json::Value, - ) -> interpreter::runtime::CheckOutcome { + ) -> submilli_engine::runtime::CheckOutcome { let model = context.get("model").and_then(serde_json::Value::as_str); if capability == "embedding.embed" && model.is_some_and(|m| self.0.iter().any(|d| d == m)) { - return interpreter::runtime::CheckOutcome::Deny { + return submilli_engine::runtime::CheckOutcome::Deny { rule: None, reason: "denied by fixture model filter".to_string(), }; } - interpreter::runtime::CheckOutcome::Allow { rule: None } + submilli_engine::runtime::CheckOutcome::Allow { rule: None } } } @@ -1433,20 +1436,20 @@ impl interpreter::runtime::SecurityCheck for FixtureDenyEmbeddingModel { /// with a fixed reason the fixture can assert against. struct FixtureDeny(Vec); -impl interpreter::runtime::SecurityCheck for FixtureDeny { +impl submilli_engine::runtime::SecurityCheck for FixtureDeny { fn check( &self, _caller: &str, capability: &str, _context: &serde_json::Value, - ) -> interpreter::runtime::CheckOutcome { + ) -> submilli_engine::runtime::CheckOutcome { if self.0.iter().any(|needle| capability.contains(needle)) { - interpreter::runtime::CheckOutcome::Deny { + submilli_engine::runtime::CheckOutcome::Deny { rule: None, reason: "denied by fixture policy".to_string(), } } else { - interpreter::runtime::CheckOutcome::Allow { rule: None } + submilli_engine::runtime::CheckOutcome::Allow { rule: None } } } } @@ -1580,8 +1583,8 @@ fn diagnostic_count_directive_rejects_missing_and_duplicate_reports() { ) .unwrap(); let diagnostic = Diagnostic { - severity: interpreter::Severity::Error, - span: interpreter::Span::at(interpreter::FileId(0)), + severity: submilli_engine::Severity::Error, + span: submilli_engine::Span::at(submilli_engine::FileId(0)), message: "mismatch".into(), help: vec![], notes: vec![], diff --git a/crates/interpreter/tests/git_memory.rs b/crates/interpreter/tests/git_memory.rs index 56029b9b3..18a6e525b 100644 --- a/crates/interpreter/tests/git_memory.rs +++ b/crates/interpreter/tests/git_memory.rs @@ -12,11 +12,11 @@ use std::path::Path; use std::sync::atomic::{AtomicUsize, Ordering}; use std::sync::{Arc, Mutex}; -use interpreter::stdlib::git::GitConfig; -use interpreter::stdlib::http::transport::{ +use submilli_engine::stdlib::git::GitConfig; +use submilli_engine::stdlib::http::transport::{ DownloadMeta, HttpClient, HttpError, HttpRequest, HttpResponse, }; -use interpreter::{ +use submilli_engine::{ compile_script, dispatch_main_async, runtime::{RuntimeConfig, StoreData, Vfs, install_runtime_async, install_tenant_limits}, }; @@ -247,7 +247,7 @@ fn measure( seed: impl FnOnce(&Path), configure: impl FnOnce(&mut StoreData), ) -> Result { - let compiled = compile_script(src, "git_memory.ts", interpreter::FileId(0), &[], &[]) + let compiled = compile_script(src, "git_memory.ts", submilli_engine::FileId(0), &[], &[]) .unwrap_or_else(|error| panic!("{error:#?}")); let runtime = tokio::runtime::Builder::new_multi_thread() .worker_threads(2) @@ -470,7 +470,7 @@ fn measure_operation(operation: &Operation, shape: Shape, cap: u64) -> Result (usize, tempfile::TempDir) { - let compiled = - compile_script(src, "host_memory.subm", interpreter::FileId(0), &[], &[]).expect("compile"); + let compiled = compile_script( + src, + "host_memory.subm", + submilli_engine::FileId(0), + &[], + &[], + ) + .expect("compile"); let cfg = RuntimeConfig { memory_reservation: TEST_MEMORY_RESERVATION, ..RuntimeConfig::default() @@ -335,8 +341,8 @@ fn nightly_only_requested() -> bool { #[test] fn rejected_string_output_does_not_allocate_its_buffer() { - use interpreter::runtime::limits::{MemoryCapExceeded, TenantLimits}; - use interpreter::runtime::prelude::string::{Str, concat, pad_end, pad_start, repeat}; + use submilli_engine::runtime::limits::{MemoryCapExceeded, TenantLimits}; + use submilli_engine::runtime::prelude::string::{Str, concat, pad_end, pad_start, repeat}; if !nightly_only_requested() { eprintln!("host memory: skipped; set SUBMILLI_TEST_NIGHTLY_ONLY=1 to run"); diff --git a/crates/interpreter/tests/http_live.rs b/crates/interpreter/tests/http_live.rs index 033ab606b..40bbd3c30 100644 --- a/crates/interpreter/tests/http_live.rs +++ b/crates/interpreter/tests/http_live.rs @@ -5,7 +5,7 @@ use std::path::{Path, PathBuf}; use httpmock::Method::{HEAD, PATCH}; use httpmock::prelude::*; -use interpreter::{ +use submilli_engine::{ BacktraceMode, Diagnostic, RuntimeConfig, Sources, compile_script, diagnostics, render_backtrace, }; @@ -49,7 +49,7 @@ fn run_one(path: &Path, base: &str) -> Result<(), String> { let filename = rel(path); let src = raw.replace("{{BASE}}", base); - let compiled = compile_script(&src, &filename, interpreter::FileId(0), &[], &[]) + let compiled = compile_script(&src, &filename, submilli_engine::FileId(0), &[], &[]) .map_err(|diags| format!("compile failed:\n{}", render_diags(&diags, &filename, &src)))?; // `run` is async (and this fixture hits a real server via reqwest, so a tokio diff --git a/crates/interpreter/tests/lexer_limits.rs b/crates/interpreter/tests/lexer_limits.rs index 4070791cd..d5881457a 100644 --- a/crates/interpreter/tests/lexer_limits.rs +++ b/crates/interpreter/tests/lexer_limits.rs @@ -1,5 +1,5 @@ -use interpreter::FileId; -use interpreter::compile::compile_script_checked; +use submilli_engine::FileId; +use submilli_engine::compile::compile_script_checked; #[test] fn invalid_source_has_a_bounded_diagnostic_set_and_recovery_is_healthy() { diff --git a/crates/interpreter/tests/memory_caps.rs b/crates/interpreter/tests/memory_caps.rs index 720cb5eaa..31f9d49be 100644 --- a/crates/interpreter/tests/memory_caps.rs +++ b/crates/interpreter/tests/memory_caps.rs @@ -1,6 +1,6 @@ use std::sync::Arc; -use interpreter::{ +use submilli_engine::{ compile_script, dispatch_main_async, runtime::{ MemoryExhausted, RuntimeConfig, StoreData, Vfs, install_runtime_async, @@ -21,7 +21,7 @@ fn cap_run_with_host_bytes( src: &str, max_store_bytes: u64, ) -> wasmtime::Result<(wasmtime::Result<()>, u64)> { - let compiled = compile_script(src, "test.subm", interpreter::FileId(0), &[], &[]) + let compiled = compile_script(src, "test.subm", submilli_engine::FileId(0), &[], &[]) .map_err(|d| wasmtime::Error::msg(format!("compile failed: {d:#?}")))?; // One engine may serve tenants with smaller caps than its own defaults. let cfg = RuntimeConfig::default(); @@ -236,7 +236,7 @@ function main(): void { let cfg = RuntimeConfig::default(); let compiled = - compile_script(src, "test.subm", interpreter::FileId(0), &[], &[]).expect("compile"); + compile_script(src, "test.subm", submilli_engine::FileId(0), &[], &[]).expect("compile"); let engine = cfg.engine().expect("engine"); let data = StoreData::with_vfs(Vfs::tempdir().expect("tempdir")); let mut store = cfg.store(&engine, data).expect("store"); @@ -377,8 +377,14 @@ function main(): void { } "#; - let compiled = compile_script(src, "memory_caps.subm", interpreter::FileId(0), &[], &[]) - .expect("compile clean"); + let compiled = compile_script( + src, + "memory_caps.subm", + submilli_engine::FileId(0), + &[], + &[], + ) + .expect("compile clean"); let cfg = RuntimeConfig { max_store_bytes: CAP_BYTES, ..RuntimeConfig::default() @@ -520,7 +526,7 @@ function main(): void { } "#; - let compiled = compile_script(src, "size_limit.subm", interpreter::FileId(0), &[], &[]) + let compiled = compile_script(src, "size_limit.subm", submilli_engine::FileId(0), &[], &[]) .expect("compile clean"); let cfg = RuntimeConfig::default(); let engine = cfg.engine().expect("engine"); @@ -558,7 +564,7 @@ function main(): void { } "#; - let compiled = compile_script(src, "overwrite.subm", interpreter::FileId(0), &[], &[]) + let compiled = compile_script(src, "overwrite.subm", submilli_engine::FileId(0), &[], &[]) .expect("compile clean"); let cfg = RuntimeConfig::default(); let engine = cfg.engine().expect("engine"); @@ -604,7 +610,7 @@ fn string_builders_refuse_native_output_and_refund_inputs() { assert!(err.is::(), "{expression}: {err:#}"); assert_eq!(host_bytes, 0, "{expression}: all native buffers refunded"); assert!( - err.is::(), + err.is::(), "{expression}: native admission refused before GC: {err:#}" ); } diff --git a/crates/interpreter/tests/namespace_limits.rs b/crates/interpreter/tests/namespace_limits.rs index 73cfe58d8..0d8722b3f 100644 --- a/crates/interpreter/tests/namespace_limits.rs +++ b/crates/interpreter/tests/namespace_limits.rs @@ -1,13 +1,13 @@ -use interpreter::compiler_limits::{ +use std::collections::BTreeMap; +use std::process::{Command, Stdio}; +use std::time::{Duration, Instant}; +use submilli_engine::compiler_limits::{ COMPILER_STACK_BYTES, MAX_NAMESPACE_DEPTH, MAX_NAMESPACE_NODES, MAX_NAMESPACE_PATH_BYTES, }; -use interpreter::type_size::TypeTooLarge; -use interpreter::{ +use submilli_engine::type_size::TypeTooLarge; +use submilli_engine::{ FileId, ModulePath, NamespaceSymbol, PackageDeclaration, PackageSourceModule, Span, }; -use std::collections::BTreeMap; -use std::process::{Command, Stdio}; -use std::time::{Duration, Instant}; #[test] fn namespace_limits_and_cleanup() { @@ -46,7 +46,7 @@ fn namespace_limits_and_cleanup() { fn namespace() -> NamespaceSymbol { NamespaceSymbol { name: "N".into(), - mangled_prefix: interpreter::mangle::prelude("N"), + mangled_prefix: submilli_engine::mangle::prelude("N"), declaration_span: Span::at(FileId(0)), values: BTreeMap::new(), types: BTreeMap::new(), @@ -109,7 +109,7 @@ fn check_namespace_limits() { }]; let very_deep = declaration(30_000); assert!( - interpreter::compile::compile_package_with_transitive_checked( + submilli_engine::compile::compile_package_with_transitive_checked( "pkg", ModulePath::from("lib"), &modules, @@ -119,7 +119,7 @@ fn check_namespace_limits() { .is_err() ); assert!( - interpreter::compile::compile_package_with_transitive_checked( + submilli_engine::compile::compile_package_with_transitive_checked( "pkg", ModulePath::from("lib"), &modules, @@ -134,7 +134,7 @@ fn check_namespace_limits() { let source = "export function main(): number { return 1; }"; let deep = declaration(MAX_NAMESPACE_DEPTH + 1); assert!( - interpreter::compile::compile_script_checked( + submilli_engine::compile::compile_script_checked( source, "main.ts", FileId(0), @@ -145,7 +145,7 @@ fn check_namespace_limits() { ); let healthy = declaration(2); assert!( - interpreter::compile::compile_script_checked( + submilli_engine::compile::compile_script_checked( source, "main.ts", FileId(0), diff --git a/crates/interpreter/tests/object_vtable_dispatch.rs b/crates/interpreter/tests/object_vtable_dispatch.rs index 94accf779..bb503fd70 100644 --- a/crates/interpreter/tests/object_vtable_dispatch.rs +++ b/crates/interpreter/tests/object_vtable_dispatch.rs @@ -8,7 +8,7 @@ use wasm_encoder::{ }; use wasmtime::{Linker, Module as WtModule}; -use interpreter::RuntimeConfig; +use submilli_engine::RuntimeConfig; // Outside the rec group: const STRING_IDX: u32 = 0; diff --git a/crates/interpreter/tests/package_diagnostic_snapshots.rs b/crates/interpreter/tests/package_diagnostic_snapshots.rs index 9654425bc..cc4406ad2 100644 --- a/crates/interpreter/tests/package_diagnostic_snapshots.rs +++ b/crates/interpreter/tests/package_diagnostic_snapshots.rs @@ -5,7 +5,7 @@ use std::collections::BTreeMap; -use interpreter::{ +use submilli_engine::{ Asi, FileId, ModulePath, PackageDeclaration, Sources, Token, TokenKind, diagnostics, infer_package, lower_patterns, parse, }; @@ -45,7 +45,7 @@ fn render_package(body: &str) -> String { out } -fn parse_source(source: &str, file: FileId) -> interpreter::Ast { +fn parse_source(source: &str, file: FileId) -> submilli_engine::Ast { let mut asi = Asi::new(source, file); let mut tokens: Vec = Vec::new(); loop { @@ -64,12 +64,13 @@ fn parse_source(source: &str, file: FileId) -> interpreter::Ast { } fn runtime_declarations() -> BTreeMap { - let (prelude, host, _) = interpreter::runtime::prelude::cached_runtime_package_declarations(); + let (prelude, host, _) = + submilli_engine::runtime::prelude::cached_runtime_package_declarations(); prelude .iter() .chain(host) .cloned() - .chain(interpreter::stdlib::stdlib_package_declarations()) + .chain(submilli_engine::stdlib::stdlib_package_declarations()) .map(|declaration| (declaration.package_name.clone(), declaration)) .collect() } diff --git a/crates/interpreter/tests/parser_depth.rs b/crates/interpreter/tests/parser_depth.rs index f033ed0ba..0fe9a75a6 100644 --- a/crates/interpreter/tests/parser_depth.rs +++ b/crates/interpreter/tests/parser_depth.rs @@ -2,7 +2,7 @@ use std::process::Command; use std::time::{Duration, Instant}; -use interpreter::{Asi, FileId, TokenKind, parse, parse_script}; +use submilli_engine::{Asi, FileId, TokenKind, parse, parse_script}; #[test] fn parser_depth_is_bounded() { diff --git a/crates/interpreter/tests/record_limits.rs b/crates/interpreter/tests/record_limits.rs index f4f81ac60..010c7685b 100644 --- a/crates/interpreter/tests/record_limits.rs +++ b/crates/interpreter/tests/record_limits.rs @@ -44,10 +44,10 @@ fn check_alias_limits() { source.push_str(&format!("type Alias{index} = Alias{};\n", index + 1)); } source.push_str(&format!("type Alias{depth} = ZBase;\ninterface ZBase {{ [key: string]: number; }}\nfunction main(): void {{}}")); - let result = interpreter::compile_script( + let result = submilli_engine::compile_script( &source, "record-limit.ts", - interpreter::FileId(0), + submilli_engine::FileId(0), &[], &[], ); @@ -62,10 +62,10 @@ fn check_alias_limits() { "{diagnostics:?}" ); } - interpreter::compile_script( + submilli_engine::compile_script( "function main(): number { return 42; }", "healthy.ts", - interpreter::FileId(0), + submilli_engine::FileId(0), &[], &[], ) diff --git a/crates/interpreter/tests/substitution_limits.rs b/crates/interpreter/tests/substitution_limits.rs index cf8173918..208ff2bc3 100644 --- a/crates/interpreter/tests/substitution_limits.rs +++ b/crates/interpreter/tests/substitution_limits.rs @@ -1,8 +1,8 @@ -use interpreter::Type; -use interpreter::type_size::{TypeLimits, TypeTooLarge}; -use interpreter::typechecker::type_param_substitution::TypeParamSubstitution; use std::process::{Command, Stdio}; use std::time::{Duration, Instant}; +use submilli_engine::Type; +use submilli_engine::type_size::{TypeLimits, TypeTooLarge}; +use submilli_engine::typechecker::type_param_substitution::TypeParamSubstitution; #[test] fn binding_chains_on_small_stack() { diff --git a/crates/interpreter/tests/traps_in_callbacks.rs b/crates/interpreter/tests/traps_in_callbacks.rs index 447cd715c..9b0db57e9 100644 --- a/crates/interpreter/tests/traps_in_callbacks.rs +++ b/crates/interpreter/tests/traps_in_callbacks.rs @@ -4,7 +4,7 @@ use std::time::Duration; -use interpreter::{compile_script, runtime::RuntimeConfig}; +use submilli_engine::{compile_script, runtime::RuntimeConfig}; use wasmtime::{Trap, WasmBacktrace}; /// Every program below returns normally from its handler, so a run that reaches @@ -12,7 +12,7 @@ use wasmtime::{Trap, WasmBacktrace}; fn run(raise: &str, body: &str, cfg: RuntimeConfig) -> wasmtime::Result<()> { let source = format!("{PRELUDE}\nfunction main(): string {{\n{body}\n}}\n").replace("RAISE", raise); - let compiled = compile_script(&source, "test.ts", interpreter::FileId(0), &[], &[]) + let compiled = compile_script(&source, "test.ts", submilli_engine::FileId(0), &[], &[]) .map_err(|diagnostics| wasmtime::Error::msg(format!("compile failed: {diagnostics:#?}")))?; // Re-entry nests interpreter frames on the native stack, so the thread is // sized for the config as the CLI and the server size theirs. @@ -162,7 +162,7 @@ fn a_timeout_under_a_callback_is_not_catchable() { #[test] fn a_timeout_in_top_level_statements_ends_the_run() { let source = "let spins = 0;\nwhile (true) { spins = spins + 1; }\nfunction main(): void { }\n"; - let compiled = compile_script(source, "test.ts", interpreter::FileId(0), &[], &[]) + let compiled = compile_script(source, "test.ts", submilli_engine::FileId(0), &[], &[]) .unwrap_or_else(|diagnostics| panic!("compile failed: {diagnostics:#?}")); let cfg = RuntimeConfig { fuel: u64::MAX, diff --git a/crates/interpreter/tests/type_limits.rs b/crates/interpreter/tests/type_limits.rs index 77756ba70..43a187bf7 100644 --- a/crates/interpreter/tests/type_limits.rs +++ b/crates/interpreter/tests/type_limits.rs @@ -4,10 +4,12 @@ //! //! Set `SUBMILLI_TEST_NIGHTLY_ONLY=1` to run these limit-sized inputs. //! `SUBMILLI_FULL_TEST` does not enable them. -use interpreter::compile::{PackageSourceModule, compile_package_checked, compile_script_checked}; -use interpreter::compiler_error::{CompilerFailure, CompilerStage}; -use interpreter::compiler_limits::{COMPILER_STACK_BYTES, MAX_TYPE_DEPTH}; -use interpreter::{FileId, ModulePath, PackageDeclaration, Span, Type, ValueKind, ValueSymbol}; +use submilli_engine::compile::{ + PackageSourceModule, compile_package_checked, compile_script_checked, +}; +use submilli_engine::compiler_error::{CompilerFailure, CompilerStage}; +use submilli_engine::compiler_limits::{COMPILER_STACK_BYTES, MAX_TYPE_DEPTH}; +use submilli_engine::{FileId, ModulePath, PackageDeclaration, Span, Type, ValueKind, ValueSymbol}; const SCRIPT: &str = "export function main(): number { return 1; }"; @@ -21,7 +23,7 @@ fn declaring(ty: Type) -> PackageDeclaration { "value".into(), ValueSymbol { name: "value".into(), - mangled_name: interpreter::mangle::package_symbol("dep", "value"), + mangled_name: submilli_engine::mangle::package_symbol("dep", "value"), declaration_span: Span::at(FileId(0)), kind: ValueKind::Const { ty, doc: None }, }, @@ -263,7 +265,7 @@ fn comparing_types_draws_on_the_work_limit() { message, format!( "building and comparing types takes more than the compiler limit of {} steps", - interpreter::compiler_limits::MAX_TYPE_WORK + submilli_engine::compiler_limits::MAX_TYPE_WORK ) ); } @@ -319,7 +321,7 @@ fn a_runtime_check_too_large_for_one_function_is_reported_at_the_cast() { message, format!( "the runtime check of `I | null` makes its function larger than the {} bytes a function may have", - interpreter::compiler_limits::MAX_FUNCTION_BODY_BYTES + submilli_engine::compiler_limits::MAX_FUNCTION_BODY_BYTES ) ); assert_eq!(text, "JSON.parse(\"null\") as I | null"); @@ -330,7 +332,7 @@ fn a_function_with_too_many_locals_is_reported_where_it_runs_out() { if !nightly_only_requested() { return; } - let locals = interpreter::compiler_limits::MAX_FUNCTION_LOCALS + 10; + let locals = submilli_engine::compiler_limits::MAX_FUNCTION_LOCALS + 10; let body: String = (0..locals) .map(|i| format!(" const v{i} = {i};\n")) .collect(); @@ -340,14 +342,14 @@ fn a_function_with_too_many_locals_is_reported_where_it_runs_out() { message, format!( "the function this code compiles into needs more than the {} locals a function may have", - interpreter::compiler_limits::MAX_FUNCTION_LOCALS + submilli_engine::compiler_limits::MAX_FUNCTION_LOCALS ) ); // The initializer of the binding whose local crosses the limit, not the // whole body. assert_eq!( text, - interpreter::compiler_limits::MAX_FUNCTION_LOCALS.to_string() + submilli_engine::compiler_limits::MAX_FUNCTION_LOCALS.to_string() ); } @@ -357,7 +359,7 @@ fn a_function_with_too_many_locals_is_reported_where_it_runs_out() { fn function_whose_loop_crosses_the_locals_limit(name: &str) -> String { // Seven locals short of the limit leaves `arr` and `s` within it, so the // loop's own locals cross it. - let consts = interpreter::compiler_limits::MAX_FUNCTION_LOCALS - 7; + let consts = submilli_engine::compiler_limits::MAX_FUNCTION_LOCALS - 7; let body: String = (0..consts) .map(|i| format!(" const v{i} = {i};\n")) .collect(); @@ -384,7 +386,7 @@ fn a_check_after_code_that_crossed_the_locals_limit_does_not_take_the_blame() { if !nightly_only_requested() { return; } - let locals = interpreter::compiler_limits::MAX_FUNCTION_LOCALS + 10; + let locals = submilli_engine::compiler_limits::MAX_FUNCTION_LOCALS + 10; let body: String = (0..locals) .map(|i| format!(" const v{i} = {i};\n")) .collect(); @@ -397,12 +399,12 @@ fn a_check_after_code_that_crossed_the_locals_limit_does_not_take_the_blame() { message, format!( "the function this code compiles into needs more than the {} locals a function may have", - interpreter::compiler_limits::MAX_FUNCTION_LOCALS + submilli_engine::compiler_limits::MAX_FUNCTION_LOCALS ) ); assert_eq!( text, - interpreter::compiler_limits::MAX_FUNCTION_LOCALS.to_string() + submilli_engine::compiler_limits::MAX_FUNCTION_LOCALS.to_string() ); } diff --git a/crates/interpreter/tests/vfs_containment.rs b/crates/interpreter/tests/vfs_containment.rs index c23599c68..1a9abc12b 100644 --- a/crates/interpreter/tests/vfs_containment.rs +++ b/crates/interpreter/tests/vfs_containment.rs @@ -26,10 +26,10 @@ use std::path::{Path, PathBuf}; -use interpreter::runtime::{ +use submilli_engine::runtime::{ RuntimeConfig, StoreData, Vfs, install_runtime_async, install_tenant_limits, }; -use interpreter::{FileId, compile_script, dispatch_main_async}; +use submilli_engine::{FileId, compile_script, dispatch_main_async}; use wasmtime::{Linker, Module}; const ESCAPE_DIAGNOSTIC: &str = "path escapes the VFS root"; diff --git a/crates/submilli-blueprint/src/lib.rs b/crates/submilli-blueprint/src/lib.rs index 52d3eb3d6..8343ef5c7 100644 --- a/crates/submilli-blueprint/src/lib.rs +++ b/crates/submilli-blueprint/src/lib.rs @@ -176,10 +176,6 @@ impl Blueprint { self.default_action.is_some() || !self.permissions.is_empty() } - /// Resolve a capability `check()` for `caller` against this blueprint's - /// permission rules. Pure — no parsing happens here (filter ASTs are parsed - /// at registration), so it is cheap to call per check. The fall-through is - /// `default:` when set, else `deny`. /// Resolve a permission and its zero-based index within the caller's rules. /// `None` identifies the blueprint's fall-through default. pub fn resolve_permission_with_rule( @@ -220,6 +216,10 @@ impl Blueprint { ) } + /// Resolve a capability `check()` for `caller` against this blueprint's + /// permission rules. Pure — no parsing happens here (filter ASTs are parsed + /// at registration), so it is cheap to call per check. The fall-through is + /// `default:` when set, else `deny`. pub fn resolve_permission( &self, caller: &str, diff --git a/crates/submilli-build/Cargo.toml b/crates/submilli-build/Cargo.toml index 199f58b72..1d710a416 100644 --- a/crates/submilli-build/Cargo.toml +++ b/crates/submilli-build/Cargo.toml @@ -10,7 +10,7 @@ publish.workspace = true workspace = true [dependencies] -interpreter = { path = "../interpreter" } +interpreter = { package = "submilli-engine", path = "../interpreter" } submilli-blueprint = { path = "../submilli-blueprint" } serde = { workspace = true, features = ["derive"] } serde_json = { workspace = true } diff --git a/crates/submilli-server/Cargo.toml b/crates/submilli-server/Cargo.toml index 2742ee4e3..fb5c52b12 100644 --- a/crates/submilli-server/Cargo.toml +++ b/crates/submilli-server/Cargo.toml @@ -17,7 +17,7 @@ path = "src/main.rs" path = "src/lib.rs" [dependencies] -interpreter = { path = "../interpreter" } +interpreter = { package = "submilli-engine", path = "../interpreter" } submilli-blueprint = { path = "../submilli-blueprint" } submilli-build = { path = "../submilli-build" } submilli-shared = { path = "../submilli-shared" } diff --git a/crates/submilli-shared/Cargo.toml b/crates/submilli-shared/Cargo.toml index 30fbb19ef..7b4a9cfdd 100644 --- a/crates/submilli-shared/Cargo.toml +++ b/crates/submilli-shared/Cargo.toml @@ -13,7 +13,7 @@ workspace = true path = "src/lib.rs" [dependencies] -interpreter = { path = "../interpreter" } +interpreter = { package = "submilli-engine", path = "../interpreter" } submilli-blueprint = { path = "../submilli-blueprint" } submilli-build = { path = "../submilli-build" } anyhow = { workspace = true } diff --git a/crates/submilli/Cargo.toml b/crates/submilli/Cargo.toml index 5db918b2c..1a0937d16 100644 --- a/crates/submilli/Cargo.toml +++ b/crates/submilli/Cargo.toml @@ -14,7 +14,7 @@ name = "submilli" path = "src/main.rs" [dependencies] -interpreter = { path = "../interpreter" } +interpreter = { package = "submilli-engine", path = "../interpreter" } submilli-blueprint = { path = "../submilli-blueprint" } submilli-build = { path = "../submilli-build" } submilli-shared = { path = "../submilli-shared" } diff --git a/plans/sub-1269-host-fuel-costs.md b/plans/sub-1269-host-fuel-costs.md index de268febc..8916bb819 100644 --- a/plans/sub-1269-host-fuel-costs.md +++ b/plans/sub-1269-host-fuel-costs.md @@ -1816,6 +1816,32 @@ The result holds its vectors as little-endian `f32` bytes in a hidden packed-`i8 | `submilli:embedding#Embeddings#bytes` | Copy one row out of the hidden array into a new `Uint8Array` (`read_row`, `write_submilli_uint8array_struct`) | `CALL + COPY(d x 4) + COPY(d x 4)` | before | One `COPY` for the read out of the result and one for the new array, charged by `write_uint8_array`. | | `submilli:embedding#Embeddings#count`, `#dimensions`, `#identity`, `#model`, `#inputTokens`; `EmbeddingModel` fields | Field read | `CALL` | before | Through `install_field_getters`. | +### submilli:agents + +Optional: installed only for an embedder that enables it. Variables: `I` = input bytes; `S` = schema bytes; `T` = bytes of the agent's answer; `m` = agents offered. + +| Function | What the host does | Formula | Charge point | Notes | +|---|---|---|---|---| +| `submilli:agents#run` | Read the agent, input and schema, gate, one provider run, then build a string or, typed, parse the answer and allocate the value tree (`stdlib/agents/mod.rs` `run`) | `CALL + GATE + IO(I + S) + IO(T)`; untyped adds the answer string through `write_submilli_string_struct`, typed adds `PARSE(T) + ELEM(nodes)` | `IO(I + S)` before `provider.run(...).await`; `IO(T)` and the result after it | The run is the effect, so everything after it uses `settle` or `settle_result` and never refuses. Waiting on the run costs nothing; the provider bounds its time and tokens. | +| `submilli:agents#list` | Gate, list the agents, run the policy check once per agent, sanitize each description, build `AgentInfo` structs and an array (`stdlib/agents/mod.rs` `list`) | `CALL + (1 + m) x GATE + ELEM(m)` plus the strings through `write_submilli_string_struct` | before + output | `m` is known after `provider.agents().await`. | +| `submilli:agents#AgentInfo#description` | Field read | `CALL` | before | | +| `submilli:agents#AgentInfo#name` | Field read | `CALL` | before | | + +### submilli:skills + +Optional: installed only for an embedder that enables it. Variables: `T` = bytes of the skill's content or the file read; `m` = skills offered. + +| Function | What the host does | Formula | Charge point | Notes | +|---|---|---|---|---| +| `submilli:skills#list` | Gate, list the skills, run the policy check once per skill, sanitize each description, build `SkillInfo` structs and an array (`stdlib/skills/mod.rs` `list`) | `CALL + (1 + m) x GATE + ELEM(m)` plus the strings through `write_submilli_string_struct` | before + output | `m` is known after `provider.list().await`. | +| `submilli:skills#load` | Gate, check the name, one provider read, build the `Skill` struct (`stdlib/skills/mod.rs` `load`) | `CALL + GATE + IO(T)` plus the strings through `write_submilli_string_struct` | `IO(T)` and the result after the read | The read is the effect, so the result is built under `settle_result`. A name that is not one skill name is refused before the provider is asked. | +| `submilli:skills#readFile` | Gate, check the name and the path, one provider read, build the string (`stdlib/skills/mod.rs` `read_file`) | `CALL + GATE + IO(T)` plus the string through `write_submilli_string_struct` | `IO(T)` and the result after the read | A name or path outside the skills is refused before the provider is asked. | +| `submilli:skills#Skill#content` | Field read | `CALL` | before | | +| `submilli:skills#Skill#description` | Field read | `CALL` | before | | +| `submilli:skills#Skill#name` | Field read | `CALL` | before | | +| `submilli:skills#SkillInfo#description` | Field read | `CALL` | before | | +| `submilli:skills#SkillInfo#name` | Field read | `CALL` | before | | + ### submilli:mcp | Function | What the host does | Formula | Charge point | Notes |