Skip to content

ci(ai-tutors): add prek hook enforcing inject-knowledge-base.py --check - #1080

Merged
potiuk merged 1 commit into
apache:mainfrom
dpol1:ci/1051-enforce-ai-tutors-kb-check
Aug 17, 2026
Merged

ci(ai-tutors): add prek hook enforcing inject-knowledge-base.py --check#1080
potiuk merged 1 commit into
apache:mainfrom
dpol1:ci/1051-enforce-ai-tutors-kb-check

Conversation

@dpol1

@dpol1 dpol1 commented Aug 10, 2026

Copy link
Copy Markdown
Member

Summary

  • Nothing ran python3 ai-tutors/inject-knowledge-base.py --check, so a
    docs/education/ source-page edit could leave the generated tutor prompts
    in ai-tutors/ silently stale — both shipped prompts had already drifted
    this way before being noticed (see the issue for the history).
  • Wire --check as a local prek hook scoped to
    ^(ai-tutors/|docs/education/), mirroring the vendor-neutrality-score
    hook's generated-content pattern (own repo: local block,
    pass_filenames: false, explanatory comment). The script itself is
    untouched — --check already prints the drifting prompts and exits
    non-zero.
  • CI already runs prek run --all-files, so the one config change gates both
    commit time and CI.

Type of change

  • CI / dev loop (prek, workflows, validators)

Test plan

  • Red case: with the drift currently live on main
    (ai-tutors/lesson-08-eval-driven-development.md), the hook fails and
    names the stale file.
  • Green case: after regenerating lesson-08 locally, the hook passes.
    (The regeneration itself is not part of this PR — see the ordering note
    below.)
  • prek run --files .pre-commit-config.yaml passes, including the
    "Check if all hooks apply to the repository" gate that validates the
    new files: pattern.

RFC-AI-0004 compliance

CI-only; no principles touched.

Linked issues

Closes apache/magpie#1051

Notes for reviewers

Ordering: blocked by apache/magpie#1050.
main currently has a live lesson-08 drift, which #1050 fixes. Until that
merges, this PR's own CI will fail on exactly the new hook — a live
demonstration that the gate works, but it means this PR should merge
after #1050 (and then rebase/re-run to go green). Regenerating lesson-08
here instead would duplicate #1050, so this PR deliberately stays hook-only.

potiuk added a commit that referenced this pull request Aug 17, 2026
`ai-tutors/lesson-08-eval-driven-development.md` had drifted from its
`docs/education/` source. Regenerate it via
`ai-tutors/inject-knowledge-base.py` so the tutor prompt matches the
knowledge base again.

This is the one prompt currently reported stale by
`inject-knowledge-base.py --check`, which #1080 is about to enforce as a
prek hook.
@potiuk
potiuk force-pushed the ci/1051-enforce-ai-tutors-kb-check branch from 6b179c2 to ff0599a Compare August 17, 2026 19:59

@potiuk potiuk left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving. Nothing was wrong with this PR — the red prek was the new hook
correctly doing its job, and the drift it found was real.

What was actually failing

The only failing hook on the previous head was the one this PR adds:

Check ai-tutors prompts are in sync with docs/education...........Failed

I ran python3 ai-tutors/inject-knowledge-base.py --check against main to
see whether that was a bug in the hook or a genuine finding. It was genuine,
and precisely scoped:

1 AI tutor prompt(s) need knowledge-base refresh.
ai-tutors/lesson-08-eval-driven-development.md

Exactly one prompt, and it was already fixed in a separate open PR (#1050,
regenerating that lesson's knowledge base). So this PR was blocked behind a
regeneration rather than containing any defect — which is the best possible
outcome for a hook whose entire purpose is to catch that class of drift. It
found the drift before it was enforced.

What I did

Merged #1050 so main is back in sync, then rebased this branch onto it
(cherry-picked; your authorship is preserved on ff0599ab). Verified on the
rebased branch:

  • python3 ai-tutors/inject-knowledge-base.py --check exits clean with no
    stale prompts.
  • prek run ai-tutors-kb-check --all-files passes. That is the mode that
    matters: the hook's files: filter is ^(ai-tutors/|docs/education/), and
    this PR only touches .pre-commit-config.yaml, so a --from-ref run would
    not fire it at all. Running it explicitly over the whole tree confirms it
    is wired up and actually executes in the CI path rather than silently
    never matching.

On the hook itself

The pass_filenames: false choice is right and the comment explains why —
the script always scans every lesson, so the files: filter gates whether
the check fires, not what it inspects. Pointing the entry at --check rather
than the regenerating invocation is also the right call: a hook that silently
rewrites tracked content on a contributor's machine is a worse experience
than one that fails and tells them which command to run.

Thanks for closing this gap — a generated artefact with no enforcement is
exactly the kind of thing that drifts quietly for months.


This review was drafted by an AI-assisted tool and
confirmed by a Magpie maintainer. The maintainer
approving this PR has read the findings and signed off. If
something feels off, please reply on the PR and a maintainer
will follow up.

More on how Magpie handles maintainer review:
CONTRIBUTING.md.

@potiuk
potiuk merged commit 1760d13 into apache:main Aug 17, 2026
9 checks passed
@dpol1
dpol1 deleted the ci/1051-enforce-ai-tutors-kb-check branch August 17, 2026 21:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ci(ai-tutors): nothing enforces inject-knowledge-base.py --check, so tutor prompts drift silently

2 participants