Skip to content

docs(plugins): reconcile the hook reference with HookType (#1743) - #1772

Open
bernalalexis-try wants to merge 4 commits into
generative-computing:mainfrom
bernalalexis-try:docs/plugins-hook-reference
Open

bernalalexis-try wants to merge 4 commits into
generative-computing:mainfrom
bernalalexis-try:docs/plugins-hook-reference

Conversation

@bernalalexis-try

Copy link
Copy Markdown

Fixes #1743.

The hook reference in docs/docs/concepts/plugins.mdx had drifted from the HookType enum, which defines 27 hooks in 8 groups.

  • Add reference sections and policy table rows for the 9 missing hooks: generation_batch_pre_call, generation_batch_post_call, generation_batch_error, generation_event, adapter_function_invocation_complete, adapter_function_phase_complete, streaming_start, streaming_event, streaming_end.
  • Add the Adapter function lifecycle and Streaming pipeline categories to the overview and the reference.
  • Fix the counts ("eight hook categories", "all 27 hook types").
  • Add test/plugins/test_plugin_docs.py, which fails when a HookType member has no #### section or policy table row, or when the count text goes stale.

Writable fields in the policy table were taken from mellea/plugins/policies.py; the new hooks other than generation_batch_pre_call are observe-only.

Checked that every HookType value has a section and a table row. I haven't run the new pytest file locally.

…-computing#1743)

Document the 9 missing hooks (generation batch and event, adapter function, streaming), add the Adapter and Streaming categories, fix the hook counts, and add a test that fails when a HookType member has no section or policy table row.

Signed-off-by: Alexis <alexisnahuel.bernal@estudiantes.unahur.edu.ar>

@ajbozarth ajbozarth left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am still on the fence about codifying this, but overall this is a solid PR

Some feedback from Claude. Verified the inventory against HookType, the payload dataclasses, and policies.py — counts, sections, table rows, and writable fields all check out, and the guard test passes.

One optional note: the sampling_loop_start "Fires:" line (existing, not touched by this PR) still reads as firing for any strategy, but stream() runs never fire sampling_loop_* — they use streaming_* + RetryEvent. Now that the Streaming pipeline section sits right below it, a one-line caveat there would close the reader-confusion that motivated #1743. Optional, since every explicit ask in the issue is satisfied.

@bernalalexis-try

Copy link
Copy Markdown
Author

Thanks for the review. Pushed 089da12, which merges current main and:

  • Fixes the red code-checks / quality runs: ruff format wanted PLUGINS_DOC in test/plugins/test_plugin_docs.py on a single line. ruff check and ruff format --check are clean now, and the new test file passes locally (55 passed).
  • Adds the caveat you suggested under sampling_loop_start: stream runs do not fire the sampling_loop_* hooks and use the streaming pipeline hooks and RetryEvent instead, linked to the streaming section.

I could not run markdownlint or the full mypy pass locally, so I am relying on CI for those.

@AngeloDanducci AngeloDanducci left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the update, started CI, one small nit about resolve.

Comment thread test/plugins/test_plugin_docs.py Outdated

from mellea.plugins.types import HookType

PLUGINS_DOC = Path(__file__).parents[2] / "docs" / "docs" / "concepts" / "plugins.mdx"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
PLUGINS_DOC = Path(__file__).parents[2] / "docs" / "docs" / "concepts" / "plugins.mdx"
PLUGINS_DOC = Path(__file__).resolve().parents[2] / "docs" / "docs" / "concepts" / "plugins.mdx"

Probably okay without this but the other path walks we have use resolve.

Comment thread docs/docs/concepts/plugins.mdx Outdated
3. **Returns a result** — continue unchanged, continue with a modified payload, or block execution entirely

Plugins are organized into six hook categories:
Plugins are organized into eight hook categories:

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: perhaps we should make this count generic so we don't need to update it in the future (less thrash)

Suggested change
Plugins are organized into eight hook categories:
Plugins are organized into different hook categories:

@ajbozarth

Copy link
Copy Markdown
Member

@bernalalexis-try soft ping in case you missed our above feedback notifications :)

@bernalalexis-try

Copy link
Copy Markdown
Author

Thanks both. Pushed fdd4bb5 with the two nits:

  • test/plugins/test_plugin_docs.py: PLUGINS_DOC now goes through .resolve() before walking up the parents.
  • docs/docs/concepts/plugins.mdx: the intro now says "different hook categories" instead of a number, so it won't need updating when a category is added.

test_plugin_docs.py passes (55) and ruff check / ruff format --check are clean.

@ajbozarth ajbozarth left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This LGTM, pending @AngeloDanducci approval we can merge this. If you want to fix the DCO error you can follow Claude advice pasted below, otherwise I can bypass it since its correct for at least one commit (the actual requirement)


Thanks, both nits and the sampling_loop_start caveat look good. One thing left: DCO is failing because fdd4bb51 is missing its Signed-off-by line (it also looks like it was committed with a different email than the earlier commits). Running git commit --amend --signoff --no-edit followed by git push --force-with-lease should fix it.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

docs(plugins): reconcile the hook reference with HookType (missing 9 hooks + 2 categories)

4 participants