Skip to content

Remove the unused stub OVAL definition from the datastream - #172

Merged
stevebeattie merged 1 commit into
chainguard-dev:mainfrom
stevebeattie:remove-oval-stub
Sep 3, 2026
Merged

stevebeattie merged 1 commit into
chainguard-dev:mainfrom
stevebeattie:remove-oval-stub

Conversation

@stevebeattie

Copy link
Copy Markdown
Member

Summary

The datastream carries a "Stub OVAL Definition" component, described as being
"for manual checks that do not have automated OVAL content." Remove it — it is
unreachable, it would give the wrong answer if it were reached, and the mechanism
it duplicates is both correct and already in use.

Three reasons

Unreachable. Of 198 rules, 91 carry a check-content-ref and none names
the stub. Nothing has ever evaluated it.

Wrong answer if it were reached. Its criterion is a textfilecontent54 test
over /dev/null with pattern ^$ and check_existence="all_exist". Evaluated
directly:

$ oscap oval eval stub.xml
Definition oval:org.stub:def:1: false

So a rule wired to it reports fail. For a check whose entire point is that it
needs manual review, asserting non-compliance is worse than saying nothing.

Not fixable into correctness. The verdict a manual check wants is
notchecked, and that comes from a rule having no check at all — not from a
definition's result. No OVAL definition can produce it. The 107 rules here that
are manual or not-applicable already do exactly that: no check-content-ref, a
<rationale> instead, reported as notchecked.

Bonus: one fewer validation warning

oscap xccdf validate goes 235 → 234, and the single line that disappears is
SRC-207-1|oval-def:definition oval:org.stub:def:1 — the stub was the only thing
that warning was about.

Verification

  • Every xlink:href in the datastream still resolves — no dangling references.
  • xmllint clean, oscap ds sds-validate rc=0, make validate_checks passes.
  • The mirror check's "present only in the datastream" list is now empty, so
    that report carries signal rather than a permanent caveat.
  • Pre-existing and untouched: ./AslrCheck.sh is referenced by a
    check-content-ref without a matching cat:uri, on main as well as here.

Upstream

The component-ref is emitted by oscap-playground's has_stub_oval metadata
flag. Filed there separately so a future regeneration does not reintroduce it.

🤖 Generated with Claude Code

The datastream carried a "Stub OVAL Definition" component, described as being
"for manual checks that do not have automated OVAL content". Remove it: it is
unreachable, it would give the wrong answer if it were reached, and the
mechanism it duplicates is both correct and already in use.

Unreachable: of 198 rules, 91 carry a check-content-ref and none of them name
the stub. Nothing has ever evaluated it.

Wrong answer if reached: its criterion is a textfilecontent54 test over
/dev/null with pattern ^$ and check_existence="all_exist", which evaluates to
false — so a rule wired to it reports *fail*. For a check whose whole point is
that it needs manual review, asserting non-compliance is worse than saying
nothing.

And it cannot be fixed into correctness. The verdict a manual check wants is
notchecked, and that comes from a rule having no check at all, not from a
definition's result — no OVAL definition can produce it. The 107 rules here
that are manual or not-applicable already do exactly that: no
check-content-ref, a <rationale> instead, and oscap reports them as
notchecked.

Removing it also drops one pre-existing validation warning. `oscap xccdf
validate` goes from 235 to 234, and the single line that disappears is
SRC-207-1 for oval:org.stub:def:1 — the stub was the only thing that warning
was about.

Verified no reference is left dangling: every xlink:href in the datastream
still resolves, sds-validate passes, xmllint is clean, and validate_checks
passes. The mirror check's "present only in the datastream" list is now empty,
so that report carries signal rather than a permanent caveat.

The upstream source of this component is oscap-playground's has_stub_oval
metadata flag, which emits the component-ref; filed separately there so a
future regeneration does not reintroduce it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@stevebeattie
stevebeattie requested a review from egibs September 1, 2026 18:05

@0xDom-S 0xDom-S left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖: Verdict: APPROVE. This is a 35-line, zero-addition deletion of the unused/unreachable "Stub OVAL Definition" (oval:org.stub:def:1) from gpos/xml/scap/ssg/content/ssg-chainguard-gpos-ds.xml. The synthesis review independently re-verified every claim in the PR body with its own XML parser: none of the 91 check-content-refs ever named the stub, the catalog never mapped a cat:uri to it (so it was unbindable even by name), its criterion could only ever evaluate to fail (never the notchecked a manual check needs), all xlink:href/cat:uri references still resolve post-deletion, and go test -run TestRepositoryMirrorsMatch passes green with the datastream-only mirror list now empty. Critical Issues: none. Zero in-scope findings at any severity.

None of the findings below anchor to a diff-map line (the diff map only covers ssg-chainguard-gpos-ds.xml lines 17-22 and 6252-6257, i.e., the deleted component-ref/component itself); they reference other, unchanged files and are included here as follow-up suggestions and signals rather than inline comments:

Suggestions (non-blocking, natural follow-ups this PR enables):

  1. tests/oscap-offline/internal/mirrors/mirrors_test.go:398 — mirrors.Unmirrored blocks are only t.Logf'd while report.Functional/Descriptive are t.Errorf'd (mirrors_test.go:379-390). The stub was this list's only permanent entry; the review confirmed the list is now empty. Flipping this to t.Errorf would cheaply guard against oscap-playground's has_stub_oval metadata flag silently reintroducing the component on a future regeneration (a risk the PR body itself names).
  2. .github/workflows/create-release.yaml:52 — Full-datastream reference-integrity validation is left commented out here, and mirrors.Check (tests/oscap-offline/internal/mirrors/mirrors.go:328-331) only fails on the standalone-file-without-datastream-block direction. A check that resolves every check-content-ref/@href through cat:catalog and fails on any unresolved href would make this class of change mechanically verifiable in CI, and would immediately surface the pre-existing, unrelated ./AslrCheck.sh mismatch (ssg-chainguard-gpos-ds.xml:6079, :6098 — no matching cat:uri; present on main, disclosed in the PR body, out of scope for this change).

Signals from Dismissed Claims:

  1. Multiple agents cited ephemeral, container-local diff-map filenames (e.g. claude-guard-diffmap-*.json under randomized mount roots) as corroborating evidence for cross-agent claims; since these paths/suffixes are randomized per container, this produced at least one false accusation of a fabricated artifact path between agents. Reviews should cite repo-relative paths only, never container-local artifact paths, when cross-referencing other agents' claims.
  2. tests/oscap-offline/internal/mirrors/mirrors.go:328-331 and mirrors_test.go:379-381 — the mirror gate does catch deletion of any of the 8 embedded OVAL blocks that mirror a standalone file (it would orphan the standalone file, tripping report.Functional → t.Errorf); it only misses rules with no standalone counterpart, as was true of the stub. Separately, tests/e2e/fixtures/*/expected.txt only asserts outcomes for 8 lines across 4 unique rule ids out of 198 rules (baseline-clean/expected.txt:10: "Rules not listed here are not asserted on.") — verdict-assertion coverage is thin; a similar future deletion affecting one of the ~180 unasserted, unmirrored rules would not be caught. This substantiates Suggestion 2 above.
  3. tests/stamps/run.sh:83-94 and :178 — the changed datastream is in fact a source that reaches a shell sink: run.sh parses <pattern> text from textfilecontent54_object elements in the datastream and feeds it into grep -Pq -- "${pattern}" for a trust-store rule. The deleted stub node was exactly this element type. This is not a finding here — selection is id-keyed at run.sh:87-88 and all three requested object ids (oval:org.CABundleHash:obj:4/6/10) survive with one occurrence each — but is noted since an earlier taint-lens pass had incorrectly declared this artifact flow-free. A follow-up self-test asserting oval_pattern errors on an unknown id (tests/stamps/run.sh:87-88) would lock this invariant against future regressions.

@0xDom-S
0xDom-S self-requested a review September 3, 2026 20:58

@0xDom-S 0xDom-S left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved

@stevebeattie
stevebeattie merged commit 1e979cb into chainguard-dev:main Sep 3, 2026
5 checks passed
stevebeattie added a commit that referenced this pull request Sep 24, 2026
The mirror check reported three classes of difference and failed on two of
them. `report.Unmirrored` -- definitions embedded in the datastream with no
standalone file -- was only logged, because the stub OVAL definition was its
one permanent entry and failing while that stub was still embedded would have
meant disabling the check rather than fixing content.

#172 removed the stub, so the list is empty and the gate can close behind it.
Verified both directions rather than just that the suite stays green:

  - passes on current content (8 standalone files compared, list empty), so
    this is a no-op today;
  - moving DetectOpenSslTest.xml out of the standalone directory makes its
    embedded block unmirrored and the test fails naming
    `oval:org.OpenSsl:def:1`, with the remedy in the message.

What this guards is narrow but real: oscap-playground's `has_stub_oval`
metadata flag can reintroduce a datastream-only component on a regeneration,
and nothing else in the suite would notice. The standalone-file-without-a-block
direction was already covered; this is the other one.

Also corrects the function's doc comment, which still claimed descriptive
differences were logged. They have failed since #171 closed that gate, so the
comment described neither the body nor the behaviour. It now names all three
classes and records that "logged" is the state to return to if a future
disagreement is judged not worth fixing.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants