Skip to content

ci(ai-tutors): nothing enforces inject-knowledge-base.py --check, so tutor prompts drift silently #1051

Description

@potiuk

What

Nothing runs python3 ai-tutors/inject-knowledge-base.py --check. No CI job, no prek hook. A source-page edit under docs/education/ can therefore leave the generated tutor prompt in ai-tutors/ silently stale, and nothing surfaces it.

Why it matters

This is not hypothetical — both generated prompts had drifted before anyone noticed:

Neither was caught by review. Both were found only because someone happened to run --check by hand while working on something else. The failure mode is quiet and the content is learner-facing, which is a bad combination: a tutor teaching from an example that no longer exists is worse than one that is merely out of date.

It also produced a near-miss. #928 regenerated lesson-04 before those source-page changes landed, so by the time it was looked at, merging it would have rolled the file backwards. It was closed rather than merged, but only because someone diffed it against main first.

Suggested fix

ai-tutors/ became a uv workspace member in #1011, so it already has a pyproject.toml, a dev dependency group, and a pytest (ai-tutors) CI job. Adding the check is cheap. Either:

  • a prek hook running inject-knowledge-base.py --check, scoped to files: ^(ai-tutors/|docs/education/), so it fires when a source page or a prompt changes; or
  • a pytest case in ai-tutors/ asserting --check reports nothing, which reuses the job that already exists.

The hook is probably the better fit: the failure is "you edited a source page and forgot to regenerate", and a hook can say exactly that at commit time.

The script side needs no work — --check already prints the drifting prompts and exits non-zero, so it is ready to be wired in as-is. The gap is purely that nothing calls it.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions