Skip to content

fix(graph): report the effective token window in execution info - #1147

Open
Ebenezer-03 wants to merge 1 commit into
ScrapeGraphAI:pre/betafrom
Ebenezer-03:fix/1121-expose-token-fallback-in-execution-info
Open

fix(graph): report the effective token window in execution info#1147
Ebenezer-03 wants to merge 1 commit into
ScrapeGraphAI:pre/betafrom
Ebenezer-03:fix/1121-expose-token-fallback-in-execution-info

Conversation

@Ebenezer-03

@Ebenezer-03 Ebenezer-03 commented Sep 9, 2026

Copy link
Copy Markdown

Part of #1121.

When a model isn't in models_tokens, _create_llm falls back to an 8192 token window. That value becomes the chunk_size used for splitting, so on a long page the model never sees the part that mattered. The run still succeeds and the JSON still validates, so the answer just comes back wrong.

The only signal today is a logger.warning on stderr. That gets lost in batch jobs, workers and anything async, so if all you have is the returned object, you can't tell a truncating fallback from a model whose real limit happens to be 8192.

#1126 exposed this as graph.model_tokens_defaulted. Suggestion (2) in the issue asked for it in the result itself, so downstream code can check it per run without reaching into the graph object. This PR does that half.

What changed

Two new keys in the "TOTAL RESULT" entry of the execution info:

  • effective_model_tokens: the input token window the run actually used for chunking
  • model_tokens_defaulted: true when no limit was known and the 8192 fallback kicked in
result = graph.run()
total = graph.get_execution_info()[-1]

total["effective_model_tokens"]   # 8192
total["model_tokens_defaulted"]   # True, so the answer may be truncated

How it's wired:

  • BaseGraph.__init__ declares model_token = None and model_tokens_defaulted = False.
  • AbstractGraph.__init__ passes both to the graph right after _create_graph(), so every graph picks them up without per-graph changes. _create_llm can be overridden and doesn't set model_token on every path, so it reads through getattr.
  • _execute_standard adds the two keys to the "TOTAL RESULT" entry.
  • get_execution_info documents them.

Existing keys are untouched, so anything already reading that entry keeps working.

Tests

Four in tests/graphs/abstract_graph_test.py, next to the ones from #1126:

  • the defaulted window shows up in "TOTAL RESULT"
  • a known window shows up without the flag
  • AbstractGraph propagates a defaulted window to its graph
  • AbstractGraph propagates a known window with the flag off

What I ran

uv run pytest over the same unit suite as .github/workflows/test-suite.yml: 95 passed.

ruff check and isort --check-only are clean on the files I touched. black --check is clean on my changes, but it also wants to reformat the CLICKABLE_URL constant at the top of base_graph.py. That one predates this PR and AGENTS.md says not to reformat untouched files, so I left it alone. Happy to fold it in if you'd rather.

Two tests already fail on pre/beta before my changes: test_llm_missing_tokens and test_script_creator_multi_graph.py::test_entry_point_node. I checked both on a clean checkout, didn't touch either, and neither one runs in CI.

No dependency changes, so uv.lock is untouched.

The rest of #1121

(4) landed in #1140. (1), raising instead of defaulting, is the breaking option and isn't in here. If that's the direction you want, or an opt-in strict flag, I can do it as a follow-up.

Fixes #1121

An unknown model falls back to an 8192-token window, which chunks long
pages and can change the answer without failing. The only signal was a
warning on stderr, which is lost in batch, worker and async contexts, so
a caller holding the returned object could not tell a truncating
fallback from a real limit.

The "TOTAL RESULT" entry of the execution info now carries
effective_model_tokens and model_tokens_defaulted. AbstractGraph passes
both to the graph it builds, so every graph reports them.

Fixes ScrapeGraphAI#1121
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant