docs: state which images CertificateAudit can assess

[stevebeattie]

Docs only — no datastream, fixture or tooling change.

Why

CertificateAudit reads each expected digest from a sidecar the image build
writes, so an image with no sidecar has nothing to compare against, and the
rule fails rather than skipping. That is deliberate — tst:4/tst:6/tst:11
exist so a missing sidecar cannot pass vacuously, and
certificate_audit/fail_missing_stamp pins it — but it puts a floor on which
images the rule can meaningfully assess, and nothing said so.

The floor, pinned to a version

The doc already cited the apko commit; a version is more useful to whoever hits
this:

image	outcome
built by apko v1.2.30 or later	assessable
built by apko v1.2.29 or earlier	fails — no sidecars exist
not built by apko at all	fails — no sidecars exist

v1.2.30 is the first release carrying writeCABundleChecksums. Checked each
cached version and pulled v1.2.30 specifically to close the bracket. Every
current image is well past it, so this matters only when scanning something
older — an archived release, a customer's pinned image from before the change,
or an image built by other tooling.

Telling that failure from a real one

The rule verdict is fail either way, but the scan artifact distinguishes
them — so no access to the image is needed, and a results file kept as
evidence can be read after the fact:

	no usable sidecar	trust store actually drifted
tst:4 — sidecar exists and parses	false	true
tst:2 — bundle matches the sidecar digest	error	false

The error is the tell: the variable behind the comparison collected no values
because there was nothing to read one from. A false there means a sidecar was
read and disagreed.

Measured, not reasoned from the criteria — I built one image with the sidecar
removed and the bundle intact, and one with the sidecar intact and the bundle
appended to, and scanned both. Same reading applies to tst:6/tst:7 (Java) and
tst:11/tst:12 (/kaniko).

Two stale things corrected while here

CertificateAuditTest.xml leaves Known gaps. #164 added it and
validate_checks now covers all 8 definitions; the doc still claimed it was
missing.

The /kaniko branch a real image takes has changed.
kaniko/ssl/certs/ca-certificates.crt was not in apko's caBundlePaths at
v1.2.35 but is at v1.2.43, and the published kaniko image now ships its own
sidecar where in August it did not. Confirmed by running the guard against it:
the copy matches its own sidecar, and that sidecar matches obj:10's pattern —
the first time that pattern has been checked against anything but a synthetic
fixture. So production kaniko is on tst:11/tst:12, and the fallback is now
what an older kaniko image would take. Both branches remain fixture-covered,
so nothing needs changing.

The ECS bundle reference also moves v1.2.35 → v1.2.43; it is still unstamped by
this rule and stays in Known gaps.

🤖 Generated with Claude Code

[0xDom-S]

🤖: PR #175 adds a "Which images this rule can assess" section to docs/certificate-audit.md documenting the apko v1.2.30 sidecar floor, a two-test triage table for distinguishing "no usable sidecar" from real trust-store drift, an update to the /kaniko branch discussion, and two stale-content corrections (removed validate_checks Known gap; ECS bundle version bump v1.2.35→v1.2.43).

This is a careful, well-evidenced documentation-only change. I independently re-derived the load-bearing OVAL mechanism (tst:4 only_one_exists over obj:4; tst:2's comparator resolves via var:1←obj:4, so an absent sidecar yields error, not a vacuous pass — tst:4 is AND-ed at CertificateAuditTest.xml:20 and fixtures_test.go:619-622 pins Fail) and it holds. No critical issues; nothing security-blocking. Inline comments below cover the accuracy issues worth fixing before merge (see S1–S6 in review). Two supplementary points could not be anchored to a diff line and are recorded here:

• docs/certificate-audit.md:399-402 (unchanged) — This deferral rationale ("the copy is currently byte-identical to its system bundle... revisit if the /kaniko copy ever starts diverging, or gains a sidecar of its own") is contradicted by the new text at :185-194, which reports the /kaniko copy now does have its own sidecar and real images have moved onto tst:11/tst:12. The PR's own evidence fires the stated revisit trigger, then the new text concludes "nothing needs changing." Recommend updating :399-402 so the file doesn't argue both sides.
• docs/certificate-audit.md:413-416 (unchanged) — The Known gap here should be restated on the new facts: the branch production images now take (obj:10, per the new :185-194 text) is the unguarded one — tests/stamps/run.sh:61 marks obj:10 required=no, and .github/workflows/update-ca-cert.yaml:35 stamps only the jre image daily. Deferral may still be correct; the rationale just needs updating.
• docs/certificate-audit.md:370 (unchanged) — Says the /kaniko copy exists "only" on cgr.dev/chainguard-private/kaniko, in tension with the new :187-189 wording "the published kaniko image." One word (e.g. "published private image") would remove the ambiguity.
• tests/oscap-offline/internal/scan/fixtures_test.go:341-343 (unchanged) — This comment ("cgr.dev/chainguard/kaniko ships none today, so the copy falls back to the stamp at caStampPath") now reads false against the new doc text at :185-194 describing real kaniko images shipping their own sidecar. Comment-only fix, but will mis-scope future fixture work if left stale.

Verdict from the underlying review: COMMENT. Fix the unqualified generalization at :114-115 (see inline comment) and reconcile the /kaniko deferral rationale — both are one-clause fixes. The apko version claims (:86-87, :90, :187, :407) have no in-repo regression guard and rest on the author's stated manual verification; flagged for reviewer awareness, not a defect.

[0xDom-S]

🤖: The generalization "The same reading applies to tst:6/tst:7 for the Java truststore and tst:11/tst:12 for a /kaniko copy" is unqualified and will mis-triage ordinary images.

The reading only transfers when the truststore or /kaniko copy actually exists. On a non-Java image (the common case): tst:5 is none_exist over obj:5 (CertificateAuditTest.xml:64-66) → true, satisfying the Java OR at :39-45. But tst:6 is only_one_exists over obj:6 (:67-69, :117-122) → false, and tst:7's state ste:3 resolves via var:2←obj:6 (:152-155, :168-170) → error. That is byte-for-byte the "no usable sidecar" column of the new triage table (:102-105) appearing on a fully assessable, passing scan. The same holds for non-kaniko images with tst:11/tst:12.

An analyst triaging a fail per :111-115 could wrongly conclude "the Java truststore / /kaniko copy could not be assessed" on an image that has neither.

Suggested fix: qualify the sentence — "…where that truststore or copy is present. On an image with neither, tst:6/tst:11 are false and tst:7/tst:12 error as a matter of course; tst:5 and tst:3 carry those branches and the absence is not a finding."

[stevebeattie]

Fixed in 24981fc — this was the one worth blocking on.

Confirmed the mechanism independently: var:2 ← obj:6 (CertificateAuditTest.xml:171-173) and var:3 ← obj:10 (:174-176), so on an image with no Java, obj:6 collects nothing, tst:6 (only_one_exists) is false, and ste:3's var_ref cannot resolve, giving tst:7 error. The Java OR at :39-45 is satisfied by tst:5 (none_exist) regardless, so — as you say — that signature shows up on an image that is fully assessable and very likely passing. Same shape for tst:11/tst:12 via obj:10 on non-kaniko images.

Taking the practical consequence seriously: non-Java, non-kaniko is the common case, so the old wording would have mis-triaged more images than it triaged correctly. An analyst would report "the Java truststore could not be assessed" against an image with no Java in it.

The sentence is now qualified along the lines you suggested, with the additional instruction to read tst:5/tst:3 first, since those are what actually carry the branches — the absence is not a finding and should not be the thing an analyst reasons from.

[0xDom-S]

🤖: The disjunction "tst:4 false says the rule could not assess this image — it predates the mechanism, or the sidecar is malformed" omits post-build sidecar deletion.

Because tst:2's comparator comes from var:1←obj:4 and not from the bundle, the tst:4 false / tst:2 error signature is content-independent on the bundle side. A sidecar deleted after build — maliciously or via an image-slimming step — produces the identical signature to a pre-v1.2.30 image. The doc's own measurement at :117-118 was taken with the bundle intact, so it doesn't separate the two cases.

Note that in-image corroboration is not available either: sidecars are unowned (apk info -W → "Could not find owner package"), and apk audit always reports benign A for them, so the X ("owned but missing") code can never fire for a deleted sidecar (apk audit --system will NOT surface this).

Suggested fix: add "or the sidecar was removed after the build" to :111-112 and state that tst:4 false is not an exoneration. As a disambiguator, the bundle's actual SHA-256 is collected by obj:2 (filehash58_object, :100-103) and can be compared out-of-band against the signed build.

[stevebeattie]

Fixed in 24981fc.

Confirmed the content-independence: tst:2 uses ste:1, whose hash comes from var:1 ← obj:4, so the comparator is read from the sidecar and never from the bundle. Deleting the sidecar post-build therefore yields the same tst:4 false / tst:2 error pair as a pre-v1.2.30 image, and the doc's own measurement was indeed taken with the bundle intact, so it did not separate them.

Verified your in-image corroboration point rather than restating it — in wolfi-base:

$ apk info -W /etc/ssl/certs/.ca-certificates.crt.sha256
ERROR: ...: Could not find owner package
$ apk info -W /etc/ssl/certs/ca-certificates.crt
... is owned by ca-certificates-bundle-20260611-r0

So the sidecar is unowned while the bundle beside it is owned, which is exactly why apk audit can never report the sidecar as missing. That contrast is now in the doc, since it is what makes the "no in-image corroboration" claim concrete rather than assertive.

Also took the obj:2 disambiguator — it is an ind:filehash58_object over the bundle (:100-103), so the bundle's real SHA-256 is collected whatever tst:2 does, and comparing it out-of-band against a signed build is the thing that actually separates the two cases. Both that and "tst:4 false is not an exoneration" are now stated explicitly.

[0xDom-S]

🤖: Two issues with the --oval-results guidance here.

(a) Invocation: this is the only occurrence of --oval-results/ARF anywhere in the repo; in OpenSCAP, --oval-results is functional only alongside --results. Worth spelling out the full invocation so a reader doesn't get an empty result — the repo's own harness (tests/oscap-offline/internal/scan/command.go:148-150, tests/e2e/run.sh:150-152) passes only --results and won't produce this per-test detail.

(b) Sensitivity: full OVAL results/ARF serialize textfilecontent54_item matched text for every rule in the profile, not just CertificateAudit — including /etc/shadow-matching rules co-selected in the same profile (e.g. UserPasswordConfiguredTest.xml, NoUsersCheck.xml use a regex whose [^:\n]* group captures the crypt field). This is bounded (check_existence="none_exist", instance=1, first match only, image already failing that rule) and not a vulnerability, but since :98-99 explicitly anticipates the file being read by someone with no image access, a clause noting full OVAL output is credential-bearing (or scoping the guidance to oscap oval eval --id oval:org.CABundleHash:def:1) is cheap and correct.

[stevebeattie]

Split verdict on this one: (b) is right and is now in the doc; (a) is not.

(a) Invocation — not reproducible. --oval-results does not need --results. Tested both ways against the in-repo datastream:

# --oval-results alone
$ oscap xccdf eval --profile xccdf_basic_profile_.check \
                   --rule <CertificateAudit rule> --oval-results <ds>
CertificateAuditTest.xml.result.xml   DetectOpenSslTest.xml.result.xml
LibraryPermissionsTest.xml.result.xml NoUsersCheck.xml.result.xml
PackageSignatureTest.xml.result.xml   RemoteAccessServicesTest.xml.result.xml
UserPasswordConfiguredTest.xml.result.xml VarLogPermissionsTest.xml.result.xml

All eight per-test result files are produced; adding --results res.xml only adds res.xml alongside them. So the guidance was already correct and I have not changed it. What was genuinely missing is where the files land — CWD, one per OVAL document, named after it — so that is now stated, since a reader looking for a single named output would otherwise not find it.

Your point about the repo's own harness passing only --results (command.go:150, run.sh:152) is correct, but that is a property of the harness rather than of the flag.

(b) Sensitivity — correct, and demonstrated. Reproduced it rather than reasoning about it: against a synthetic root containing a planted canary,

$ grep -o '<ind-sys:text>[^<]*' oval-res.xml
<ind-sys:text>fakeuser:FAKEHASHCANARY123:

The matched line, password field included, is serialized verbatim.

One correction that strengthens it: this is not a hypothetical co-selection. 31 of the 91 rules xccdf_basic_profile_.check selects resolve to UserPasswordConfiguredTest.xml or NoUsersCheck.xml. The results also carry primary_host_name, os_version and architecture, so there is a host-identification dimension beyond the credential one.

The doc now flags a retained results file as credential-bearing and offers the scoped oscap oval eval --id oval:org.CABundleHash:def:1 alternative you suggested.

[0xDom-S]

🤖: The apko v1.2.30 sidecar-floor claim here (and the related version claims at :90, :187, and :407 for the ECS bundle bump v1.2.35→v1.2.43) are the only apko version references in the tree and nothing pins them in-repo. They rest entirely on the author's stated manual verification, which could not be checked offline in this review. Not a defect — flagged so reviewers know which claims carry no regression guard.

[stevebeattie]

Fair flag, and it prompted the right fix. Rather than leave the claims resting on my say-so, all three are now re-derived from apko's pkg/build/certificates.go:

claim	check
v1.2.30 first carries writeCABundleChecksums	0 occurrences at v1.2.29, 3 at v1.2.30
kaniko/ssl/certs/ca-certificates.crt not in caBundlePaths at v1.2.35, is at v1.2.43	absent at v1.2.35; at v1.2.43 the list gains "kaniko/ssl/certs/ca-certificates.crt", // Kaniko
ECS bundle still in caBundlePaths at v1.2.43	var/lib/ecs/deps/execute-command/certs/tls-ca-bundle.pem present

24981fc records the file and symbols in the doc so a future reader can re-check at any tag instead of trusting this PR. I deliberately did not add a test pinning apko's caBundlePaths: it would couple this repo's CI to a third party's source layout and break on an upstream refactor that changes nothing about the rule. Naming where to look seemed the better trade for a claim about a moving external dependency — open to the other view if you think the drift risk warrants it.

[0xDom-S]

🤖: This new text (asserting the /kaniko copy now ships its own sidecar and real images have moved onto tst:11/tst:12) contradicts the unchanged deferral rationale later in the file at docs/certificate-audit.md:399-402, which still reads: "The copy is currently byte-identical to its system bundle, so the drift being guarded against is remote… Revisit if the /kaniko copy ever starts diverging, or gains a sidecar of its own." This PR's own evidence fires that revisit trigger verbatim, yet :193-194 concludes "nothing needs changing."

Two follow-ups worth doing in this PR (both landing in unchanged lines outside the diff, so not separately anchorable):

• Update :399-402 so the file doesn't argue both sides.
• The Known gap at :413-416 should note that the branch production images now take (obj:10) is the unguarded one — tests/stamps/run.sh:61 marks obj:10 required=no, and .github/workflows/update-ca-cert.yaml:35 stamps only cgr.dev/chainguard/jre:latest daily. Deferral may still be the right call; the rationale just needs restating on the new facts.

[stevebeattie]

Fixed in 24981fc, and you are right that the file was arguing both sides — the deferral names "gains a sidecar of its own" as its own revisit trigger, and the new text reports exactly that happening.

Verified both supporting facts:

• tests/stamps/run.sh:61 — oval:org.CABundleHash:obj:10|kaniko/ssl/certs|...|no
• .github/workflows/update-ca-cert.yaml — STAMP_IMAGE_REF: cgr.dev/chainguard/jre:latest, so the daily run stamps that image only

The deferral paragraph now states the original rationale, that it has been overtaken, and why the conclusion nevertheless stands — the two trust-surface additions are unchanged, and a divergent copy is now caught by tst:11/tst:12 against its own sidecar rather than needing the fallback. What the sidecar's arrival costs is guard coverage, not criteria coverage.

The Known gap is restated on the new facts and split in two, because the second half is the part that was invisible: the gap now falls on the branch production images actually take, since they resolve through obj:10 and obj:10 is the one sidecar the guard marks required=no. I kept required=no — older kaniko images legitimately ship none and the criteria fall back to tst:13/tst:9 for them — but the consequence is now written down: "present but wrong" is guarded there, "absent" is not.

run.sh's own comment had the same problem, arguing required=no from facts that have since changed, so it is corrected too (comment only).

[0xDom-S]

🤖: Minor wording ambiguity: this line says "the published kaniko image", while the unchanged line at docs/certificate-audit.md:370 says the /kaniko copy exists "only" on cgr.dev/chainguard-private/kaniko, and :413-414 says the daily workflow "inspects only public images." Both readings are individually defensible, but one word here (e.g. "published private image", or naming the ref explicitly) would remove the ambiguity for future readers.

[stevebeattie]

Fixed in 24981fc — and it turned out to be more than an ambiguity.

There is no public kaniko image at all:

$ crane digest cgr.dev/chainguard/kaniko:latest
MANIFEST_UNKNOWN: Unknown manifest

So "the published kaniko image" was not merely unclear about which of two refs it meant; it pointed at something that does not exist. :370 and run.sh:55 were the accurate ones. The line now names cgr.dev/chainguard-private/kaniko and says outright that it is the only kaniko image there is, which resolves the ambiguity toward the fact rather than just picking a phrasing.

The same problem is in the comment you flagged at fixtures_test.go:341-343, and worse there: it names cgr.dev/chainguard/kaniko specifically as the image shipping no sidecar. Both halves were wrong — the ref does not resolve, and the image that does exist ships one. Corrected (comment only, no fixture behaviour change).

The underlying claim is now verified too, in 91c7822. This is the one thing I initially had to take on trust, since the private ref needs credentials; re-authed and checked it against cgr.dev/chainguard-private/kaniko@sha256:f57d9900:

kaniko/ssl/certs/.ca-certificates.crt.sha256	shipped, 86 bytes
copy vs its own sidecar	matches
sidecar vs obj:10's pattern	matches, exactly one line
copy vs the system bundle	still byte-identical (b8d83784…)

tests/stamps/run.sh cgr.dev/chainguard/jre:latest cgr.dev/chainguard-private/kaniko:latest exits 0 and reports obj:10's pattern satisfied.

That last row is the one that mattered, and it sharpened the S5 fix: the deferral rested on two grounds — byte-identical copy, no sidecar of its own — and only the second has changed. The revisit trigger has fired, but the divergence the guard exists to catch still has not occurred. The doc now shows those as measurements rather than reasoning about them.

[stevebeattie]

Adjacent finding from verifying the /kaniko claims, since it cost time and the doc's own guidance pointed the wrong way.

:423-427 explains that a plain chainctl auth login leaves chainctl auth status reporting Valid: True while a cgr.dev pull still fails. True, but incomplete: without --audience, that command reports on the console-API token, which is valid whether or not the cgr.dev login has happened. So it misleads in both directions — it reads as success before the re-auth, and after a successful --audience=cgr.dev login it still shows Audience: https://console-api.enforce.dev, which reads as failure.

The check that answers the question is:

chainctl auth status --audience cgr.dev

which reports Audience | cgr.dev. One caveat worth having in writing: it is not a pure read — if that token has expired, asking for its status re-requests one.

Both the corrected diagnostic and the caveat are in 91c7822.