Make the datastream's OVAL reference sources say Custom, and gate it - #171
Merged
Merged
Conversation
Five OVAL definitions embedded in the datastream carried <reference source="CIS"/> while every standalone file said "Custom", and two embedded blocks said "Custom" as well — so the datastream disagreed both with its sources and with itself. "Custom" is the intended value, and this is a missed hand-sync rather than a judgement call. Commit 3f69d3f ("All of the formatting", 2025-07-18) deliberately changed exactly these five references from CIS to Custom in the standalone files. It touched the datastream in the same commit — 3402 lines — but left its reference sources alone. The value has never been anything but Custom in the standalone files since. These are Chainguard-authored checks for a DISA GPOS SRG profile, not CIS benchmark content, and the elements carry no ref_id, so "CIS" asserted a provenance that was not real. This does not change any scan verdict, but it is not invisible either. The value is absent from the HTML report oscap renders, and present in the machine-readable results a scan emits — so it reaches compliance evidence customers retain, as well as the published datastream itself. Convert the five, and close the gate behind them: descriptive differences in the mirror check now fail rather than being logged. They were logged deliberately while these five existed, because failing on a difference that changes no verdict would have meant disabling the check instead of fixing content. The content is fixed, so the exemption goes. Verified: no CIS values remain anywhere; the mirror check reports zero descriptive differences across all 8 files; reintroducing a single CIS now fails the check, naming the file and definition. gofmt/vet clean, validate_checks passes, sds-validate rc=0, and oscap xccdf validate reports the same 235 pre-existing errors as main. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
egibs
approved these changes
Aug 31, 2026
stevebeattie
added a commit
that referenced
this pull request
Sep 24, 2026
The mirror check reported three classes of difference and failed on two of them. `report.Unmirrored` -- definitions embedded in the datastream with no standalone file -- was only logged, because the stub OVAL definition was its one permanent entry and failing while that stub was still embedded would have meant disabling the check rather than fixing content. #172 removed the stub, so the list is empty and the gate can close behind it. Verified both directions rather than just that the suite stays green: - passes on current content (8 standalone files compared, list empty), so this is a no-op today; - moving DetectOpenSslTest.xml out of the standalone directory makes its embedded block unmirrored and the test fails naming `oval:org.OpenSsl:def:1`, with the remedy in the message. What this guards is narrow but real: oscap-playground's `has_stub_oval` metadata flag can reintroduce a datastream-only component on a regeneration, and nothing else in the suite would notice. The standalone-file-without-a-block direction was already covered; this is the other one. Also corrects the function's doc comment, which still claimed descriptive differences were logged. They have failed since #171 closed that gate, so the comment described neither the body nor the behaviour. It now names all three classes and records that "logged" is the state to return to if a future disagreement is judged not worth fixing. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Five OVAL definitions embedded in the datastream carried
<reference source="CIS"/>while every standalone file saidCustom— andtwo embedded blocks said
Customtoo. So the datastream disagreed both with itssources and with itself.
Customis the intended value — this is a missed hand-sync, not a judgement callCommit
3f69d3f("All of the formatting", 2025-07-18) deliberately changedexactly these five references from
CIStoCustomin the standalone files. Ittouched the datastream in the same commit — 3402 lines — and left its
reference sources alone. The standalone value has been
Customever since.Supporting evidence: the elements carry no
ref_id, and these areChainguard-authored checks for a DISA GPOS SRG profile rather than CIS benchmark
content, so
CISasserted a provenance that was not real.This is the second confirmed instance of the same failure class as
openssh-sftp-serverin #162: a change applied to the standalone files and notto the datastream.
Is it user-visible?
Partly, and worth being precise about — I tested rather than assumed:
CISpresent?No scan verdict changes. But it reaches the compliance evidence a customer
retains, and the shipped content.
Change
Convert the five, and close the gate behind them: descriptive differences in
the mirror check now fail rather than being logged.
That exemption existed only because these five instances did — failing on a
difference that changes no verdict would have meant disabling the check instead
of fixing content. With the content fixed, the exemption goes, so this class
cannot drift back in silently.
Verification
CISvalues remain anywhere in the repo; datastream and standalone filesnow both report
Customexclusively.CISnow fails the check, naming the file anddefinition — confirmed by mutation, then reverted.
gofmt/go vetclean,validate_checkspasses,sds-validaterc=0, andoscap xccdf validatereports the same 235 pre-existing errors asmain.🤖 Generated with Claude Code