Skip to content

Enhance error handling for Mistral API responses#1769

Open
jka236 wants to merge 1 commit into
NVIDIA:mainfrom
jka236:fix/mistral-error-handling
Open

Enhance error handling for Mistral API responses#1769
jka236 wants to merge 1 commit into
NVIDIA:mainfrom
jka236:fix/mistral-error-handling

Conversation

@jka236
Copy link
Copy Markdown

@jka236 jka236 commented May 12, 2026

Description

Previously, MistralGenerator._call_model treated all SDKError exceptions the same way — wrapping them in GeneratorBackoffTrigger and retrying indefinitely. This meant auth failures (401/403) would silently retry forever with no actionable feedback, and rate limit errors (429) were not distinguished from other transient failures.

This PR:

  • Raises BadGeneratorException with a clear message for 401/403 auth failures so runs fail fast instead of retrying
  • Raises RateLimitHit for 429 responses and adds it to the backoff exception list so rate limits trigger the existing backoff/retry logic
  • Falls through to GeneratorBackoffTrigger for all other SDKError codes (preserving existing retry behavior for 5xx etc.)
  • Fixes test_mistral_chat which was passing a raw Message to generate() instead of the required Conversation object

#1356

Verification

  • Set MISTRAL_API_KEY to a valid key and run garak --model_type mistral --model_name mistral-small --probes dan.Ablation_Dan_11_0
  • Set MISTRAL_API_KEY to an invalid key — confirm the run fails immediately with BadGeneratorException and a clear auth error message, rather than retrying
  • Run the tests: python -m pytest tests/generators/test_mistral.py -v
  • Confirm test_mistral_403_raises_bad_generator_exception and test_mistral_429_raises_rate_limit_hit both pass without a real API key

Signed-off-by: Jonghyeok Kim <jka236@sfu.ca>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant