Skip to content

fix: handle tool validation errors in stopWhen and willContinue#241

Open
elpiarthera wants to merge 1 commit intoget-convex:mainfrom
elpiarthera:fix/issue-172-hasSuccessfulToolCall
Open

fix: handle tool validation errors in stopWhen and willContinue#241
elpiarthera wants to merge 1 commit intoget-convex:mainfrom
elpiarthera:fix/issue-172-hasSuccessfulToolCall

Conversation

@elpiarthera
Copy link
Copy Markdown

@elpiarthera elpiarthera commented Apr 6, 2026

Summary

  • hasSuccessfulToolCall(toolName) — new stop condition that only matches tool calls which produced a tool-result (not tool-error). Use instead of the AI SDK's hasToolCall when agents should retry on argument validation failures.
  • willContinue fix — counts tool-error content parts as completed outputs. In AI SDK v6, failed tool calls produce tool-error instead of tool-result, causing willContinue to incorrectly treat them as pending and stop prematurely.

Context

When a tool call fails Zod argument validation in AI SDK v6, the result is a tool-error content part (not tool-result). Two problems occurred:

  1. willContinue checked step.toolCalls.length > step.toolResults.length — since tool-error isn't in toolResults, it looked like a pending result, returning false (stop)
  2. The AI SDK's hasToolCall matches any tool call regardless of success, so stopWhen: [hasToolCall("myTool")] would stop even on validation failures

Test plan

  • npm run build passes
  • npm run typecheck passes
  • npm run lint passes (0 errors)
  • npm test passes (262/262)
  • Impact analysis: willContinue has a single caller (streamText.ts:143), change is backwards-compatible

Fixes #172

Orchestrator: Zeta — VantageOS Team Dev | 2026-04-06

Summary by CodeRabbit

  • New Features

    • Added functionality to check whether a specific tool call completed successfully.
  • Bug Fixes

    • Enhanced tool call completion detection to properly recognize and handle both successful results and error responses in tool interactions.

In AI SDK v6, failed tool calls produce tool-error content parts
instead of tool-result. willContinue was only checking toolResults,
treating validation failures as pending — causing premature stops.

Also adds hasSuccessfulToolCall(toolName) stop condition that only
matches tool calls which produced a tool-result, so agents retry
on argument validation failures instead of stopping.

Fixes get-convex#172
@coderabbitai
Copy link
Copy Markdown

coderabbitai bot commented Apr 6, 2026

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 00b72f2f-26e1-46e6-9fd8-a5d05ca3ef37

📥 Commits

Reviewing files that changed from the base of the PR and between 92164db and e6238e1.

📒 Files selected for processing (2)
  • src/client/index.ts
  • src/client/utils.ts

📝 Walkthrough

Walkthrough

Added a new hasSuccessfulToolCall stop condition helper and updated tool completion detection to recognize both tool-error and tool-result content as completed tool execution, enabling proper distinction between attempted and successful tool calls.

Changes

Cohort / File(s) Summary
Export Configuration
src/client/index.ts
Added import and re-export of the new hasSuccessfulToolCall helper function to the public API.
Tool Execution Detection
src/client/utils.ts
Added hasSuccessfulToolCall(toolName) function that returns true only when the last step contains a successful tool result matching the tool name. Updated willContinue() to count both tool-result and tool-error content parts as completed outputs, and changed control-flow comparison to check toolCalls.length against completedOutputs.

Possibly related issues

Poem

🐰 Hops of joy through tool-call land,
Now success and failure we understand!
No more stopping when validation fails,
Only true completions ring success bells! 🔔

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately describes the main fix: handling tool validation errors in stopWhen and willContinue functions.
Linked Issues check ✅ Passed The PR fully addresses issue #172 by adding hasSuccessfulToolCall and fixing willContinue to properly handle tool-error parts.
Out of Scope Changes check ✅ Passed All changes in src/client/index.ts and src/client/utils.ts are directly related to fixing the tool validation error handling.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Agent stopWhen stops even when tool call fails Zod validation

1 participant