Skip to content

feat(editor): validate XML with native eXist schemas - #877

Open
duncdrum wants to merge 7 commits into
eXist-db:developfrom
duncdrum:feat/native-schemas-842
Open

duncdrum wants to merge 7 commits into
eXist-db:developfrom
duncdrum:feat/native-schemas-842

Conversation

@duncdrum

@duncdrum duncdrum commented Sep 8, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • Resolve known eXist/EXPath grammars for POST /api/editor/validate, preferring $EXIST_HOME/schema/ (exist#6528) and falling back to synced copies under resources/schema/ (eXide#842).
  • Cover namespaced configs (collection.xconf, controller-config, expath-pkg, security-manager) and no-namespace roots (conf.xml, descriptor, mime-types, users, server), plus xsi:schemaLocation / noNamespaceSchemaLocation when they point at a known grammar or an absolute file://db/ URI.
  • Validate with validation:jaxv-report + XSD 1.1 (interim until catalog-based jaxp works against native grammars — exist#6686). Response includes grammar (native|bundled|instance|none) and schema for observability.
  • Keep resources/schema/catalog.xml for tooling / a future catalog path; document deprecation of the bundled copies.

Closes #842.

Test plan

  • xst package install local-files build/eXide-4.0.1.xar --force --config .existdb.json
  • Smoke: collection.xconf + mime-types return grammar: "native" on existdb/existdb:latest
  • npx cypress run --spec cypress/e2e/xml_validate_spec.cy.js (7/7)
  • CI green on existdb/existdb:latest
  • Optional: server without $EXIST_HOME/schema/ still validates via bundled XSDs (grammar: "bundled")

@duncdrum duncdrum added this to v7.0.0 Sep 8, 2026
@duncdrum
duncdrum force-pushed the feat/native-schemas-842 branch from 6296f84 to 990f5a7 Compare September 8, 2026 16:00
@duncdrum duncdrum changed the title feat(editor): prefer native eXist schemas for XML validation feat(editor): validate XML with native eXist schemas Sep 8, 2026
@line-o line-o moved this to In progress in v7.0.0 Sep 18, 2026
@duncdrum
duncdrum force-pushed the feat/native-schemas-842 branch from 990f5a7 to 86de385 Compare September 29, 2026 15:03
@duncdrum
duncdrum marked this pull request as ready for review September 29, 2026 15:21
@duncdrum
duncdrum requested a review from a team October 19, 2026 08:59
@duncdrum duncdrum moved this from In progress to In review in v7.0.0 Oct 19, 2026
@duncdrum duncdrum added this to the v7-beta milestone Oct 19, 2026
@joewiz

joewiz commented Oct 21, 2026

Copy link
Copy Markdown
Member

[This response was prompted by Joe, drafted by Claude Code, and reviewed by Joe.]

@duncdrum I went through this against head 19805337. The hardening in the last commit holds up — I confirmed an instance-supplied file: schema location is now ignored rather than opened as a grammar, and the SCHEMA_BY_ROOT_CHILD corroboration does stop a stray <auth> or <server> from being force-matched. One thing I'd fix before this goes in, plus a couple of smaller notes.

Namespaced grammars still match on namespace alone

The corroboration you added covers SCHEMA_BY_ROOT, but SCHEMA_BY_NS maps http://exist.sourceforge.net/NS/exist — eXist-db's general-purpose namespace, not a controller-specific one — straight to controller-config.xsd with no second signal. Anything else in that namespace gets force-validated against it:

document result
a REST <query> envelope invalid — "Cannot find the declaration of element 'query'"
a controller <dispatch> fragment invalid — three further errors about url and pattern

Both are perfectly good documents someone might have open in the editor. The same trick you used for roots would do it — require a corroborating child (controller-config has a documented shape), or drop that namespace from the map and rely on xsi:schemaLocation for controller configs.

Minor

No-grammar documents come back as "status": "valid" with "grammar": "none", and no src/ file changed in this PR, so src/xml-helper.js's compileError() still branches only on status == "invalid". The editor therefore shows the same all-clear whether a document was schema-validated or merely parsed. One line in the status update — "validated against collection.xconf.xsd" versus "well-formed only" — would make the new grammar/schema fields visible to the person who benefits from them.

Verification

Built the branch and ran its spec against existdb/existdb:latest (eXist-db 7.0.0-SNAPSHOT, build 2026-10-17): xml_validate_spec passes 7/7.

I also ticked the open box in your test plan — the bundled fallback. Masking $EXIST_HOME/schema/ so the native lookup misses, a valid collection.xconf and an expath-pkg.xml both come back "grammar": "bundled", and an invalid collection.xconf is still rejected by the XSD 1.1 assertion. So the fallback path works as intended, not just the native one.

@duncdrum

Copy link
Copy Markdown
Contributor Author

@joewiz thanks, and done. Ci failures are unrelated see #910

duncdrum and others added 4 commits November 3, 2026 08:32
Prefer $EXIST_HOME/schema grammars (exist#6528) via jaxv XSD 1.1, with
synced resources/schema fallbacks, until catalog-based jaxp works (exist#6686).

Co-authored-by: Cursor <cursoragent@cursor.com>
- Re-sync resources/schema/conf.xsd (2.6.0 -> 2.7.0, omit-xml-declaration
  doc/attribute change) and controller-config.xsd (1.1.1 -> 1.1.2, new
  optional attribute) from exist-core's schema/ tree, which drifted since
  this branch was cut.
- Update comments in editor.xqm, resources/schema/catalog.xml and
  resources/schema/README.md: exist#6686 (catalog-based jaxp resolution of
  native XSD 1.1 grammars by namespace) is fixed upstream as of
  exist-core#6687 / eXist 7.0.0-beta4, not still open. eXide keeps
  validation:jaxv-report + an explicit resolved URI rather than switching to
  validation:jaxp-report + catalog, since stable eXist releases still
  predate that fix and the WEB-INF system catalog doesn't cover every
  namespace in SCHEMA_BY_NS.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…ration

editor:resolve-schema had two issues found by code review:

- xsi:schemaLocation / xsi:noNamespaceSchemaLocation values matching
  file:|/db/ were used verbatim as the validation:jaxv-report schema URI.
  A client could point noNamespaceSchemaLocation at an arbitrary local path
  (e.g. file:///etc/passwd) and force the server to open it as a grammar.
  Only /db/ is honoured now, since eXist's own permission checks already
  gate access there; a raw file: hint is never trusted.

- No-namespace documents were matched to an eXist config grammar by root
  local-name alone. Names like <auth> or <server> are generic enough that
  an unrelated document could collide and get force-validated (and marked
  invalid) against users.xsd/server.xsd. Now also requires the root's
  documented required first child (SCHEMA_BY_ROOT_CHILD) to be present
  before matching.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
… in status

The controller-config grammar was matched on the general-purpose eXist
namespace alone, so REST <query> envelopes and <dispatch> fragments were
reported invalid. Require a <configuration> root for that namespace.

Also show "Validated against X" vs "Well-formed only" in the editor status.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@duncdrum
duncdrum force-pushed the feat/native-schemas-842 branch from c1c2459 to 33f3651 Compare November 3, 2026 07:32
@dizzzz

dizzzz commented Nov 3, 2026

Copy link
Copy Markdown
Member

could you post some screen post, when possible?

xml_wellformed_spec still asserted an empty #error-status after a successful
fix, but compileError now writes the informational status Joe asked for.

Co-authored-by: Cursor <cursoragent@cursor.com>
@duncdrum

duncdrum commented Nov 3, 2026

Copy link
Copy Markdown
Contributor Author

@dizzzz its not very exciting, its mainly about the plumbing, using exists schemas instead of its own copies

Screenshot 2026-11-03 at 12 49 06

The UX is somewhat misleading, let me add one more commit

duncdrum and others added 2 commits November 3, 2026 13:00
Move "Well-formed only" / "Validated against …" to a calm #validation-status
segment so successful validation no longer looks like an error.

Co-authored-by: Cursor <cursoragent@cursor.com>
Distinguish success from the red error pill with a check badge, separator,
and dedicated Cypress coverage for schema-ok / well-formed / schema-error.

Co-authored-by: Cursor <cursoragent@cursor.com>
@duncdrum

duncdrum commented Nov 3, 2026

Copy link
Copy Markdown
Contributor Author

@dizzzz here you go
validation-status-schema-error

validation-status-wellformed-only validation-status-schema-ok

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

Status: In review

Development

Successfully merging this pull request may close these issues.

[feature] use native schemas provided by exist

3 participants