Skip to content

fix: honor LangGraph chat model kwargs#713

Open
Kovbo wants to merge 1 commit into
mainfrom
fix/langgraph-init-chat-model-474
Open

fix: honor LangGraph chat model kwargs#713
Kovbo wants to merge 1 commit into
mainfrom
fix/langgraph-init-chat-model-474

Conversation

@Kovbo

@Kovbo Kovbo commented Jun 4, 2026

Copy link
Copy Markdown
Collaborator

Summary

Closes #474

Fixes ART's LangGraph init_chat_model wrapper so it no longer silently ignores the model argument, provider kwargs, or timeout configuration. Also clarifies that RULER's default judge calls OpenAI unless callers choose a different LiteLLM judge model.

Changes

  • Forward init_chat_model(..., **kwargs) to ChatOpenAI, including provider-level timeout, temperature, and other supported kwargs.
  • Add invoke_timeout for ART's outer asyncio.wait_for timeout, keeping it separate from provider request timeout.
  • Honor an explicit string model name while still defaulting to the active ART rollout context model.
  • Allow callers to pass a prebuilt LangChain chat model instance, such as ChatOllama, for non-OpenAI providers.
  • Reject unsupported string model_provider values instead of silently constructing ChatOpenAI anyway.
  • Preserve configured kwargs and invoke timeout across bind_tools(), with_structured_output(), and with_config().
  • Add unit coverage for argument forwarding, custom model wrapping, timeout behavior, and LangGraph binding/config behavior.
  • Document that RULER's default judge is OpenAI-backed and should be overridden for local/custom judge backends.

Test plan

  • uv run ruff format --check src/art/rewards/ruler.py src/art/langgraph/llm_wrapper.py tests/unit/test_langgraph_llm_wrapper.py
  • uv run ruff check src/art/rewards/ruler.py src/art/langgraph/llm_wrapper.py tests/unit/test_langgraph_llm_wrapper.py
  • uv run ty check src/art/rewards/ruler.py src/art/langgraph/llm_wrapper.py tests/unit/test_langgraph_llm_wrapper.py
  • uv run pytest tests/unit/test_langgraph_llm_wrapper.py tests/unit/test_ruler_metrics.py -q
  • uv run pytest tests/unit/test_langgraph_llm_wrapper.py tests/unit/test_ruler_metrics.py tests/unit/test_model_openai_client_costs.py -q
  • uv run pytest tests/unit -q --ignore=tests/unit/test_pipeline_trainer_local_backend.py --ignore=tests/unit/test_tinker_renderers.py --ignore=tests/unit/test_moe_routing_replay.py (251 passed, 7 skipped)

Full uv run pytest tests/unit -q is blocked in this local environment by missing optional dependencies (megatron, tinker_cookbook); the unignored failures were limited to tests requiring those packages.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

init_chat_model always uses ChatOpenAI, ignores args, and still calls OpenAI / hits 10-minute timeout with Ollama

1 participant