Enhance error handling for Mistral API responses#1769
Open
jka236 wants to merge 1 commit into
Open
Conversation
Signed-off-by: Jonghyeok Kim <jka236@sfu.ca>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Previously,
MistralGenerator._call_modeltreated allSDKErrorexceptions the same way — wrapping them inGeneratorBackoffTriggerand retrying indefinitely. This meant auth failures (401/403) would silently retry forever with no actionable feedback, and rate limit errors (429) were not distinguished from other transient failures.This PR:
BadGeneratorExceptionwith a clear message for 401/403 auth failures so runs fail fast instead of retryingRateLimitHitfor 429 responses and adds it to the backoff exception list so rate limits trigger the existing backoff/retry logicGeneratorBackoffTriggerfor all otherSDKErrorcodes (preserving existing retry behavior for 5xx etc.)test_mistral_chatwhich was passing a rawMessagetogenerate()instead of the requiredConversationobject#1356
Verification
MISTRAL_API_KEYto a valid key and rungarak --model_type mistral --model_name mistral-small --probes dan.Ablation_Dan_11_0MISTRAL_API_KEYto an invalid key — confirm the run fails immediately withBadGeneratorExceptionand a clear auth error message, rather than retryingpython -m pytest tests/generators/test_mistral.py -vtest_mistral_403_raises_bad_generator_exceptionandtest_mistral_429_raises_rate_limit_hitboth pass without a real API key