Skip to content

fix(llmmultiroundrouter): increase decomposition token budget#178

Open
panpan0000 wants to merge 1 commit into
ulab-uiuc:mainfrom
panpan0000:fix/llmmultiroundrouter-decomp-tokens
Open

fix(llmmultiroundrouter): increase decomposition token budget#178
panpan0000 wants to merge 1 commit into
ulab-uiuc:mainfrom
panpan0000:fix/llmmultiroundrouter-decomp-tokens

Conversation

@panpan0000

@panpan0000 panpan0000 commented Jun 10, 2026

Copy link
Copy Markdown

Description

This PR fixes a routing fallback problem in llmmultiroundrouter.
For some reasoning-heavy prompts, 512 tokens is not enough for the decomposition + routing step. When the output is cut early, the router may fall back to the first model in llm_data.

Related Issues

  • Fixes #
  • Related to #

Type of Change

  • Bug fix (non-breaking change that fixes an issue)
  • New feature (non-breaking change that adds functionality)
  • New router implementation
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Documentation update
  • Performance improvement
  • Code refactoring
  • Test addition or update

Changes Made

  • add decomposition_max_tokens for llmmultiroundrouter
  • change the default decomposition token budget from 512 to 2048
  • update the example config and router README
  • add tests for the default value and config override

Router Affected (if applicable)

  • knnrouter
  • svmrouter
  • mlprouter
  • mfrouter
  • elorouter
  • routerdc
  • automix
  • hybrid_llm
  • graphrouter
  • causallm_router
  • router_r1
  • gmtrouter
  • knnmultiroundrouter
  • llmmultiroundrouter
  • Other / Core framework

Checklist

  • I have tested my changes locally
  • I have reviewed my own code
  • I have added/updated documentation where necessary
  • I have added tests for my changes (if applicable)
  • My changes do not introduce new warnings or errors
  • Any dependent changes have been merged and published

Test Plan

python3 -m pytest tests/test_llmmultiroundrouter_decomposition.py -q

Screenshots (if applicable)

Additional Notes

This is a small fix to make routing output more stable for longer decomposition prompts.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant