Skip to content

feat(api)!: represent multi-grader graders as named maps - #4253

Closed
markstuart-oai wants to merge 1 commit into
nextfrom
castiron/promotions/pr-407
Closed

markstuart-oai wants to merge 1 commit into
nextfrom
castiron/promotions/pr-407

Conversation

@markstuart-oai

@markstuart-oai markstuart-oai commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

Draft on hold. This preview combines the release-branch update with named graders. With the same inputs and compiler, enabling named graders changes 3 generated files, including the file inventory. We’ll review the branch update separately, then regenerate the grader change against the updated next branch.

Summary

MultiGrader accepts a dictionary of named graders and returns typed grader values for each name.

Changes

  • Use dictionaries for MultiGrader graders and MultiGraderParam graders.
  • Keep calculate_output formulas associated with the supplied grader names.

@markstuart-oai
markstuart-oai requested a review from a team as a code owner November 6, 2026 02:38
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 6, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-06T02:42:50.479667Z dc6614f PR opened
🔒 Security Review ✅ Completed 2026-11-06T02:41:04.272476Z dc6614f PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@markstuart-oai
markstuart-oai marked this pull request as draft November 6, 2026 02:38

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: dc6614f17b

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread CHANGELOG.md
@@ -1,5 +1,139 @@
# Changelog

## [3.46.0](https://github.com/openai/openai-python/compare/v3.45.0...v3.46.0) (2026-11-05)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Synchronize release metadata with the imported releases

This imports the released 3.40.0–3.46.0 history and corresponding SDK code, but pyproject.toml, src/openai/_version.py, .release-please-manifest.json, and the editable package in uv.lock all remain at 3.39.0. Consequently, builds still identify this SDK as 3.39.0, and release-please will calculate the next release from stale 3.39.0 state despite the changelog already declaring 3.46.0; carry over the matching release metadata and lockfile updates.

Useful? React with 👍 / 👎.

@@ -1,12 +0,0 @@
{

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Restore the Pyright configs used by CI

Deleting these configs leaves three retained jobs in .github/workflows/ci.yml pointing at nonexistent files: browser-demo-types uses scripts/pyrightconfig.browser-demo.json, while the browser and computer matrix entries construct the other two deleted paths. Each job therefore exits when scripts/run-pyright tries to load its configured project instead of checking the optional examples; restore the configs or update all three workflow references to replacements.

AGENTS.md reference: AGENTS.md:L17-L20

Useful? React with 👍 / 👎.

Comment thread scripts/run-pyright Outdated
cd "$(dirname "$0")/.."

CONFIG=${PYRIGHT_PROJECT:-pyproject.toml}
CONFIG=${PYRIGHT_PROJECT:-scripts/pyrightconfig.breaking-changes.json}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Keep the full-project Pyright config as the default

scripts/lint invokes this wrapper without setting PYRIGHT_PROJECT, so changing the default to pyrightconfig.breaking-changes.json reduces normal lint coverage to the four test paths listed by that specialized config. The previous pyproject.toml default checked the SDK and the rest of the repository, including modules such as _files.py that are deliberately excluded from mypy because Pyright is expected to cover them; keep pyproject.toml as the wrapper default and pass the compatibility config explicitly from detect-breaking-changes.

Useful? React with 👍 / 👎.

@@ -993,7 +993,6 @@ def test_untrusted_provenance_leaves_no_dependency_install_reachable(tmp_path: P
"browser-demo-types",
"optional-provider-types",
"test",
"test-httpx2",
"examples",

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Keep the HTTPX2 job in the installer expectation

The unchanged test-httpx2 workflow job still uses astral-sh/setup-uv and runs uv sync --locked --all-extras, so dependency_installer_jobs() continues to include it. Removing it from this expected set makes every parameterization of test_untrusted_provenance_leaves_no_dependency_install_reachable fail at this assertion before the provenance scenarios run; restore "test-httpx2" unless the corresponding installer job is actually removed.

Useful? React with 👍 / 👎.

Castiron-Internal-PR: openai/openai-python-internal#407
Castiron-Source-SHA: 402d40c977a6a96c24cfd6c809b95203eb4070ab
Castiron-Public-Base-SHA: 1fae55f
@markstuart-oai
markstuart-oai force-pushed the castiron/promotions/pr-407 branch from dc6614f to a6bac30 Compare November 6, 2026 02:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant