Parser rework#9
Conversation
Introduces a new op_table module with a scope-based operation lookup system for dynamic operator dispatch in cel-parser. The parser now supports custom operation scopes, efficient built-in operator tables using phf, and exposes op_lookup_mut for extensibility. Includes new tests and documentation, and updates dependencies to add phf and once_cell.
Enhanced literal parsing in LexLexer to use syn::parse2 for better span preservation and added the 'parsing' feature to the syn dependency. Expanded documentation for parser structures and methods in lib.rs, including detailed doc comments for primary expressions, probe results, error formatting, and parsing entry points. Improved test assertions and error reporting in parser tests.
Refactored conditional logic in cel-parser to use && let chains for improved readability and conciseness. Expanded .vscode/tasks.json with comprehensive build, test, documentation, clippy, and sanitizer tasks for multiple platforms to streamline development workflows.
Added and reorganized tasks in .vscode/tasks.json for improved Rust workflow. Expanded sanitizer support for Linux/WSL2 and Windows, clarified platform-specific notes, and added a comprehensive test task for Windows. This update enhances cross-platform development and testing capabilities.
Updated .vscode/tasks.json to streamline task definitions by changing command types to 'cargo' for clippy and adjusting environment variables for better clarity. Removed experimental Windows tasks to focus on macOS and Linux/WSL2 support, enhancing the overall development experience.
Updated the error handling logic in the expression function to use the is_err() method for improved clarity and conciseness. This change enhances the readability of the parser code.
The lexer (`LexLexer`) no longer returns `Result` and is now infallible, as all input is pre-validated by `proc_macro2`. The parser (`CELParser`) is updated to remove error handling for lexer errors, simplifying token peeking and advancement. Documentation is updated to clarify error handling in both lexer and parser. Macro now only reports errors for non-matching expressions, not parse errors.
Move the CEL parser implementation from the separate cel-parser crate into cel-runtime::parser, remove the cel-parser crate and its example docs, and update workspace/Cargo manifests. CELParser API changed: constructor now takes an OpLookup, tokens are set via set_tokens/parse_tokens/parse_str, postfix/call parsing added, and error/reporting helpers adjusted. Updated lex_lexer to use a concrete TokenStreamIter type and relocated files under cel-runtime/src/parser; cel-rs-macros now depends on cel-runtime and uses CELParser/OpLookup accordingly. Also export parser types and implement FromStr for DynSegment to allow parsing from strings; tests updated to the new API.
Add a .cursor rule encouraging no-heap allocations and implement several refactors to reduce temporaries and allocations. Introduce PunctOp (no-heap punctuation token) and replace String-based punct tokens. Extend StackInfo with Cow<'static, str> type_name, padding flag, dropper, and AssociatedType to record nested types; store argument_names in DynSegment. Replace peek_types_vec with peek_stack_infos(&[StackInfo]) returning a slice to avoid allocating Vec<TypeId>. Change ScopeFn/OpLookup APIs to accept &mut DynSegment and a num_operands so scopes can inspect the stack without extra allocations; update builtin lookup and parser call sites and tests accordingly. Improve error messages to include human-readable type names.
Add a dedicated CELError type (with message and proc_macro2::Span) and a rustc-style formatter, and refactor the parser to return CELError instead of anyhow::Error. Removed the parser's compile_error token-output plumbing and helper extraction/formatting methods; replaced report_error with error_at that constructs CELError at the current span. Updated CEL macro to emit compile_error! using CELError spans and literals. Re-export CELError from runtime, adjusted FromStr and parser APIs/Result type, and updated tests to work with the new error type and formatting.
Introduce a Send+Sync SourceSpan and switch CELError to store it (with helper from_proc_macro2) so parse errors can be used across threads. Adjust proc-macro handling to emit compile_error! at call_site and use SourceSpan defaults. Add fallible operation support: DynSegment.op1r/op2r with stack-unwinding droppers and RawSegment::raw2 helper to run fallible binary ops. Update op_table to use op1r/op2r for checked integer arithmetic and shifts (returning errors on overflow/invalid shifts) and add tests for overflow and unwind behavior.
Drop manual unsafe impl Send/Sync for SourceSpan and CELError in parser/error.rs. Reformat cel-runtime/src/parser/op_table.rs: reorder anyhow import, reflow many long closure expressions across multiple lines for readability, compact some signature arrays onto single lines, and tidy minor test formatting. These are mostly non-functional formatting/import changes; only behavioral change is removal of the explicit unsafe Send/Sync impls.
Replace standalone matches! calls with assert!(matches!(...)) in cel-runtime/src/parser/lex_lexer.rs tests. The previous matches! usages were no-ops and did not fail the test on mismatch; wrapping them with assert! ensures the tests actually verify token shapes and will fail when tokens differ from expectations.
Use integer checked_div/checked_rem wrapped with op2r for all integer division and modulo signatures so division-by-zero returns an error instead of panicking; float behavior remains unchanged. Add explanatory comments and unit tests (division_by_zero and modulo_by_zero) that assert the error contains "division by zero". This prevents runtime panics and surfaces a clear error to callers.
Added a workflow_dispatch trigger to the CI configuration for manual workflow execution. Included a new job to check Rust code formatting using `cargo fmt --check`. Additionally, refactored several function signatures in the `cel-runtime` module for improved readability by removing unnecessary line breaks and ensuring consistent formatting across the codebase.
There was a problem hiding this comment.
Pull request overview
This PR consolidates CEL parsing into cel-runtime by removing the standalone cel-parser crate, updating macro validation to use the new runtime parser, and expanding developer tooling/docs to match the new workspace structure.
Changes:
- Remove
cel-parserfrom the workspace and route parsing APIs throughcel-runtime(including new parser modules andDynSegment: FromStr). - Refactor
cel-rs-macrosto validate expressions viacel-runtimeparser/error types. - Add/expand developer workflow assets (VSCode tasks, CI fmt check, and Cursor/CLAUDE guidance).
Reviewed changes
Copilot reviewed 22 out of 23 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
src/lib.rs |
Updates crate-level docs/re-exports to reflect parser living in cel-runtime. |
notes.txt |
Adds a cargo clippy --fix --workspace workflow note. |
CLAUDE.md |
Documents workspace architecture, parser pipeline, and workflow commands. |
cel-runtime/src/segment.rs |
Aligns type-id tracking with updated StackInfo shape. |
cel-runtime/src/raw_segment.rs |
Adds fallible stack-aware binary op support (raw2). |
cel-runtime/src/parser/op_table.rs |
Introduces operator lookup/dispatch table with built-in ops and custom scopes. |
cel-runtime/src/parser/mod.rs |
Adds the recursive-descent parser producing executable segments. |
cel-runtime/src/parser/lex_lexer.rs |
Adds lexer layer that flattens/normalizes token streams for the parser. |
cel-runtime/src/parser/error.rs |
Adds CELError + rustc-style diagnostic formatting. |
cel-runtime/src/lib.rs |
Exposes parser APIs and implements DynSegment: FromStr. |
cel-runtime/src/dyn_segment.rs |
Enhances stack metadata, adds fallible ops (op1r/op2r), improves error reporting. |
cel-runtime/Cargo.toml |
Adds dependencies needed for parser/diagnostics and removes old macro dependency coupling. |
cel-rs-macros/src/lib.rs |
Switches expression! validation to the runtime parser and updates docs. |
cel-rs-macros/Cargo.toml |
Depends on cel-runtime instead of removed cel-parser. |
cel-parser/src/lib.rs |
Removed (parser implementation moved into cel-runtime). |
cel-parser/Cargo.toml |
Removed (crate removed from workspace). |
Cargo.toml |
Removes cel-parser from workspace members/dependencies. |
Cargo.lock |
Updates lockfile for removed/added crates. |
.vscode/tasks.json |
Adds comprehensive build/test/lint/sanitizer tasks (needs update for removed crate). |
.vscode/settings.json |
Updates spellchecker dictionary. |
.github/workflows/ci.yml |
Adds workflow_dispatch and cargo fmt --check. |
.cursor/rules/doc-comments.mdc |
Expands/enforces Rust doc comment conventions. |
.cursor/rules/avoid-heap-allocations.mdc |
Adds allocation-avoidance guidelines. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Updated the push_literal function to return a Result type, allowing for better error handling when parsing integer and float literals with invalid suffixes. Added unit tests to verify correct parsing of valid suffixes and proper error reporting for invalid ones. This change improves the robustness of the parser by ensuring that suffix-related errors are caught and reported accurately.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…rror - Added new dependencies: `annotate-snippets` and `anstyle`. - Removed unused dependencies: `hermit-abi`, `is-terminal`, `is_ci`, `owo-colors`, `supports-color`. - Refactored `line_col_to_byte_offset` to `span_to_byte_range` for improved clarity and functionality in error reporting. - Enhanced test coverage for function calls in the CEL parser, including handling of empty argument lists and error cases.
Introduces ParseError (not Send+Sync) for use as the parser return type, carrying the original proc_macro2::Span for precise IDE diagnostics. From<ParseError> for CELError extracts the SourceSpan for async/runtime use. CELError::with_proc_macro_span is retained for Task 2 migration. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Updated CELParser to return ParseError instead of CELError, allowing for more precise diagnostics with proc_macro2::Span. - Enhanced CELError to include a message method for better error reporting. - Modified the expression macro to include tests for error handling scenarios. - Adjusted VSCode task configuration to improve testing command options. These changes improve error handling and diagnostics in the CEL parser, enhancing the developer experience.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Replaces unwrap_or(self.last_span) with a join_spans() helper that falls back to the start span when proc_macro2::Span::join returns None. On stable Rust inside a real proc-macro, proc_macro2 gates Span::join behind the nightly-only proc_macro_span feature and returns None; the helper gracefully degrades to the start span rather than panicking or silently swallowing the failure with the wrong fallback. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…in() proc_macro::Span::join() is not yet stable. Revised design stores start and end spans separately in ParseError::new_range(), merges them at the SourceSpan layer for runtime, and emits two compile_error!() diagnostics for compile-time.
…asks Replaces the prior plan that used Span::join() with an approach that stores start and end spans separately, requiring no unstable APIs.
…rrors Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
… to improve error handling in expression macro. The changes include new permissions for various scripts and a refined approach to compile_error!() handling in the expression! macro, ensuring both start and end errors are reported correctly.
…dependencies. Add trybuild as a dev-dependency in cel-rs-macros and enhance error handling in CELParser with new tests for identifier scope errors.
…rror message and adding check for undefined identifiers with appropriate error reporting.
…e redundant prefixes, and wrap identifiers in backticks across various modules.
…and spacing for improved readability in parse_str and test functions.
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit ffa26c9. Configure here.
…t tests for validation

This pull request makes several improvements to Rust development workflows and code quality standards for the project. The most significant changes include the removal of the
cel-parsercrate, enhancements to documentation and allocation guidelines, expanded VSCode build/test tasks, and updates to dependencies and macro usage to reflect the new crate structure.Project structure and dependencies:
cel-parsercrate from the workspace and dependencies, consolidating parsing functionality intocel-runtime. All references and dependencies oncel-parserhave been eliminated fromCargo.tomlfiles and the workspace configuration. ([[1]](https://github.com/stlab/cel-rs/pull/9/files#diff-2e9d962a08321605940b5a657135052fbcef87b5e360662bb527c96d9a615542L9-R15),[[2]](https://github.com/stlab/cel-rs/pull/9/files#diff-b9957dc1b6aedbcdeff4fbce5b4221da59ef2dd8884b80c051f4fe6ee3108f8dL1-L12),[[3]](https://github.com/stlab/cel-rs/pull/9/files#diff-3823bfb8141007195f06dc4ffabcc9e2db3fd93d37624332d46e88bcfd20c48bL14-R14))cel-rs-macroscrate to depend oncel-runtimeinstead ofcel-parser, and refactored macro code to use parsing and error types fromcel-runtime. ([[1]](https://github.com/stlab/cel-rs/pull/9/files#diff-3823bfb8141007195f06dc4ffabcc9e2db3fd93d37624332d46e88bcfd20c48bL14-R14),[[2]](https://github.com/stlab/cel-rs/pull/9/files#diff-c187b9b22359c2452046a640b14f4e3748f18cc98c62ce1ed00bfc82ca709c20L33-R36),[[3]](https://github.com/stlab/cel-rs/pull/9/files#diff-c187b9b22359c2452046a640b14f4e3748f18cc98c62ce1ed00bfc82ca709c20L48-L52))Development tooling:
[.vscode/tasks.jsonR4-R352](https://github.com/stlab/cel-rs/pull/9/files#diff-7d76d7533653c23b753fc7ce638cf64bdb5e419927d276af836d3a03fdf1745aR4-R352))[.vscode/settings.jsonL10-R11](https://github.com/stlab/cel-rs/pull/9/files#diff-a5de3e5871ffcc383a2294845bd3df25d3eeff6c29ad46e3a396577c413bf357L10-R11))Code quality and documentation standards:
[.cursor/rules/avoid-heap-allocations.mdcR1-R87](https://github.com/stlab/cel-rs/pull/9/files#diff-d13685b400c5d4957f64a3d3644543afc23c9f80bff059aa3542cfe5adc431aeR1-R87))[.cursor/rules/doc-comments.mdcL2-R154](https://github.com/stlab/cel-rs/pull/9/files#diff-bc71edfe8cc02c0facfc56fc6371c9bf2d90055cb84af398575c9f3dc2c204ddL2-R154))Dependency updates:
cel-runtime(e.g.,proc-macro2,quote,owo-colors,syn,phf,once_cell) to support the consolidated parsing and macro infrastructure. ([cel-runtime/Cargo.tomlL10-R15](https://github.com/stlab/cel-rs/pull/9/files#diff-b401c530f9e812ac243c304a31f8c72e20f1a285e1fdbd95fdb804581cdd1bf2L10-R15))Macro and parsing improvements:
expressionmacro incel-rs-macrosto use the new parser interface fromcel-runtime, improving error reporting and aligning with the new crate structure. Also updated theprint_tokens!macro documentation example. ([[1]](https://github.com/stlab/cel-rs/pull/9/files#diff-c187b9b22359c2452046a640b14f4e3748f18cc98c62ce1ed00bfc82ca709c20L33-R36),[[2]](https://github.com/stlab/cel-rs/pull/9/files#diff-c187b9b22359c2452046a640b14f4e3748f18cc98c62ce1ed00bfc82ca709c20L48-L52),[[3]](https://github.com/stlab/cel-rs/pull/9/files#diff-c187b9b22359c2452046a640b14f4e3748f18cc98c62ce1ed00bfc82ca709c20L61-R74))Detailed list of changes:
Project structure and dependencies
cel-parsercrate from the workspace and all related dependencies, consolidating parsing functionality intocel-runtime. ([[1]](https://github.com/stlab/cel-rs/pull/9/files#diff-2e9d962a08321605940b5a657135052fbcef87b5e360662bb527c96d9a615542L9-R15),[[2]](https://github.com/stlab/cel-rs/pull/9/files#diff-b9957dc1b6aedbcdeff4fbce5b4221da59ef2dd8884b80c051f4fe6ee3108f8dL1-L12))cel-rs-macrosto depend oncel-runtimeinstead ofcel-parser, and refactored macro code accordingly. ([[1]](https://github.com/stlab/cel-rs/pull/9/files#diff-3823bfb8141007195f06dc4ffabcc9e2db3fd93d37624332d46e88bcfd20c48bL14-R14),[[2]](https://github.com/stlab/cel-rs/pull/9/files#diff-c187b9b22359c2452046a640b14f4e3748f18cc98c62ce1ed00bfc82ca709c20L33-R36),[[3]](https://github.com/stlab/cel-rs/pull/9/files#diff-c187b9b22359c2452046a640b14f4e3748f18cc98c62ce1ed00bfc82ca709c20L48-L52))Development tooling
[.vscode/tasks.jsonR4-R352](https://github.com/stlab/cel-rs/pull/9/files#diff-7d76d7533653c23b753fc7ce638cf64bdb5e419927d276af836d3a03fdf1745aR4-R352))[.vscode/settings.jsonL10-R11](https://github.com/stlab/cel-rs/pull/9/files#diff-a5de3e5871ffcc383a2294845bd3df25d3eeff6c29ad46e3a396577c413bf357L10-R11))Code quality and documentation
[.cursor/rules/avoid-heap-allocations.mdcR1-R87](https://github.com/stlab/cel-rs/pull/9/files#diff-d13685b400c5d4957f64a3d3644543afc23c9f80bff059aa3542cfe5adc431aeR1-R87))[.cursor/rules/doc-comments.mdcL2-R154](https://github.com/stlab/cel-rs/pull/9/files#diff-bc71edfe8cc02c0facfc56fc6371c9bf2d90055cb84af398575c9f3dc2c204ddL2-R154))Dependencies
cel-runtimeto support parsing, macro expansion, and colorized output. ([cel-runtime/Cargo.tomlL10-R15](https://github.com/stlab/cel-rs/pull/9/files#diff-b401c530f9e812ac243c304a31f8c72e20f1a285e1fdbd95fdb804581cdd1bf2L10-R15))Macros and parsing
expressionmacro to use the newCELParserinterface fromcel-runtime, improving error handling and reporting. Updated documentation for theprint_tokens!macro. ([[1]](https://github.com/stlab/cel-rs/pull/9/files#diff-c187b9b22359c2452046a640b14f4e3748f18cc98c62ce1ed00bfc82ca709c20L33-R36),[[2]](https://github.com/stlab/cel-rs/pull/9/files#diff-c187b9b22359c2452046a640b14f4e3748f18cc98c62ce1ed00bfc82ca709c20L48-L52),[[3]](https://github.com/stlab/cel-rs/pull/9/files#diff-c187b9b22359c2452046a640b14f4e3748f18cc98c62ce1ed00bfc82ca709c20L61-R74))Note
High Risk
Large behavioral change to the core parser, macro compile-time validation, and a new type-dispatch op table; mistakes affect expression parsing, runtime semantics, and proc-macro diagnostics across the workspace.
Overview
Reworks
cel-parserso parsing produces an executableDynSegmentviacel-runtime, instead of only validating token trees and emittingcompile_error!into an output stream.Parser pipeline and API: Adds
LexLexer(flatten groups, combine compound punct,syn::Litliterals) and rewiresCELParserto takeOpLookup, exposeparse_str/parse_tokens/set_tokens, and returnResult<_, ParseError>while building ops on aDynSegment. Grammar gains postfix calls (parameter_list,"()"lookup) and richer literal suffix handling.Errors: New
error.rswithSourceSpan,CELError,ParseError, andformat_rustc_stylevia annotate-snippets (replacing owo-colors/litrs manual formatting).Operations: New
op_table.rswithphfbuilt-ins (checked signed math, div/mod by zero, wide shift matrix), scope stack, and typed “no operation” / undefined identifier messages with expression range spans.Macros & CI:
expression!validates throughparse_tokensand spawns span-awarecompile_error!(plus optional end-span diagnostic); trybuild UI tests added. CI addscargo fmt --checkandworkflow_dispatch. Workspace addsplaygroundfeature,missing_docslint, and expanded editor/AI guidance (CLAUDE.md, Cursor rules, VS Code tasks).Reviewed by Cursor Bugbot for commit 4064d1d. Bugbot is set up for automated code reviews on this repo. Configure here.