ci(ai-tutors): add prek hook enforcing inject-knowledge-base.py --check - #1080
Conversation
`ai-tutors/lesson-08-eval-driven-development.md` had drifted from its `docs/education/` source. Regenerate it via `ai-tutors/inject-knowledge-base.py` so the tutor prompt matches the knowledge base again. This is the one prompt currently reported stale by `inject-knowledge-base.py --check`, which #1080 is about to enforce as a prek hook.
6b179c2 to
ff0599a
Compare
potiuk
left a comment
There was a problem hiding this comment.
Approving. Nothing was wrong with this PR — the red prek was the new hook
correctly doing its job, and the drift it found was real.
What was actually failing
The only failing hook on the previous head was the one this PR adds:
Check ai-tutors prompts are in sync with docs/education...........Failed
I ran python3 ai-tutors/inject-knowledge-base.py --check against main to
see whether that was a bug in the hook or a genuine finding. It was genuine,
and precisely scoped:
1 AI tutor prompt(s) need knowledge-base refresh.
ai-tutors/lesson-08-eval-driven-development.md
Exactly one prompt, and it was already fixed in a separate open PR (#1050,
regenerating that lesson's knowledge base). So this PR was blocked behind a
regeneration rather than containing any defect — which is the best possible
outcome for a hook whose entire purpose is to catch that class of drift. It
found the drift before it was enforced.
What I did
Merged #1050 so main is back in sync, then rebased this branch onto it
(cherry-picked; your authorship is preserved on ff0599ab). Verified on the
rebased branch:
python3 ai-tutors/inject-knowledge-base.py --checkexits clean with no
stale prompts.prek run ai-tutors-kb-check --all-filespasses. That is the mode that
matters: the hook'sfiles:filter is^(ai-tutors/|docs/education/), and
this PR only touches.pre-commit-config.yaml, so a--from-refrun would
not fire it at all. Running it explicitly over the whole tree confirms it
is wired up and actually executes in the CI path rather than silently
never matching.
On the hook itself
The pass_filenames: false choice is right and the comment explains why —
the script always scans every lesson, so the files: filter gates whether
the check fires, not what it inspects. Pointing the entry at --check rather
than the regenerating invocation is also the right call: a hook that silently
rewrites tracked content on a contributor's machine is a worse experience
than one that fails and tells them which command to run.
Thanks for closing this gap — a generated artefact with no enforcement is
exactly the kind of thing that drifts quietly for months.
This review was drafted by an AI-assisted tool and
confirmed by a Magpie maintainer. The maintainer
approving this PR has read the findings and signed off. If
something feels off, please reply on the PR and a maintainer
will follow up.More on how Magpie handles maintainer review:
CONTRIBUTING.md.
Summary
python3 ai-tutors/inject-knowledge-base.py --check, so adocs/education/source-page edit could leave the generated tutor promptsin
ai-tutors/silently stale — both shipped prompts had already driftedthis way before being noticed (see the issue for the history).
--checkas a local prek hook scoped to^(ai-tutors/|docs/education/), mirroring thevendor-neutrality-scorehook's generated-content pattern (own
repo: localblock,pass_filenames: false, explanatory comment). The script itself isuntouched —
--checkalready prints the drifting prompts and exitsnon-zero.
prek run --all-files, so the one config change gates bothcommit time and CI.
Type of change
prek, workflows, validators)Test plan
main(
ai-tutors/lesson-08-eval-driven-development.md), the hook fails andnames the stale file.
(The regeneration itself is not part of this PR — see the ordering note
below.)
prek run --files .pre-commit-config.yamlpasses, including the"Check if all hooks apply to the repository" gate that validates the
new
files:pattern.RFC-AI-0004 compliance
CI-only; no principles touched.
Linked issues
Closes apache/magpie#1051
Notes for reviewers
Ordering: blocked by apache/magpie#1050.
maincurrently has a live lesson-08 drift, which #1050 fixes. Until thatmerges, this PR's own CI will fail on exactly the new hook — a live
demonstration that the gate works, but it means this PR should merge
after #1050 (and then rebase/re-run to go green). Regenerating lesson-08
here instead would duplicate #1050, so this PR deliberately stays hook-only.