Skip to content

feat(parse): warn when the answer looks one link away - #1148

Open
Ebenezer-03 wants to merge 1 commit into
ScrapeGraphAI:pre/betafrom
Ebenezer-03:feat/1120-warn-answer-behind-link
Open

feat(parse): warn when the answer looks one link away#1148
Ebenezer-03 wants to merge 1 commit into
ScrapeGraphAI:pre/betafrom
Ebenezer-03:feat/1120-warn-answer-behind-link

Conversation

@Ebenezer-03

Copy link
Copy Markdown

Fixes #1120.

SmartScraperGraph reads one page. When the evidence for the prompt lives on a page that one links to, a privacy policy or terms or an "our team" page, the model doesn't say it can't tell. Asked whether a fact holds, it returns a confident negative that looks exactly like a real "this is not true of this site". Nothing in the run says the answer was never fetched.

The issue reports it on two live sites where the notice is real but sits one link away, and both come back no_transfer_notice: False. For compliance questions that error has a direction: a false "no restriction found" is the answer someone acts on.

What this adds

ParseNode now warns when a term the user asked about is missing from the page's own prose but shows up in the label or target of a link:

The terms ['policy', 'privacy'] appear only in links on this page
(e.g. ['https://example.com/privacy/']), not in its text. This graph reads the
single page it was given, so if the answer lives on one of those linked pages
the model will answer from the page it did see, and a negative answer here may
mean the evidence was never fetched rather than that it does not exist.
Consider DepthSearchGraph to follow the links.

This is suggestion (2) from the issue. It's deterministic and LLM-free, and follows the check already in this file for content that holds no trace of the request (#1136), reusing its _collect_expected_terms.

Keeping it quiet

A hint that fires on every page with a footer link would be worse than no hint. Two things hold it back:

  • The term has to be absent from the page's prose. Links are stripped before that test, so a word that only ever appears as anchor text doesn't count as the page answering. If the page can answer, nothing is logged.
  • _warn_if_content_lacks_requested_fields now returns whether it fired, and the link hint only runs when it didn't. A page with no trace of the request gets one warning, not two.

Tests

Five, in tests/test_error_page_detection.py next to the existing ParseNode ones:

  • warns on the reported shape, a contact page linking to /privacy/
  • silent when the page holds the terms itself
  • silent when the page has no links
  • the two warnings don't stack on a page with no trace of the request
  • parsed chunks are unchanged, the hint only logs

Checks

uv run pytest on the same suite as .github/workflows/test-suite.yml: 100 passed, up from 95 on pre/beta.

ruff check, black --check and isort --check-only all clean on both files I touched.

Two tests fail on pre/beta before this branch, test_llm_missing_tokens and test_script_creator_multi_graph.py::test_entry_point_node. I checked them on a clean checkout, didn't touch either, and neither runs in CI.

No dependency changes, uv.lock untouched.

Not in this PR

Suggestion (1), a separate "not found on this page" state, changes the returned shape and is worth its own discussion. Suggestion (3), docs, would mean editing the README plus nine translations, so I kept this to one logical change. Happy to do either as a follow-up.

A single-page graph answers only about the text it was handed. When the
evidence lives on a page this one links to, a privacy policy or a terms
or team page, the model does not stay silent: asked whether a fact
holds, it returns a confident negative that reads exactly like a genuine
"this is not true of this site". Nothing in the run says the answer was
never fetched.

ParseNode now warns when a term the user asked about is absent from the
page's own prose but present in the label or target of a link, and
points at DepthSearchGraph. The check is deterministic and LLM-free,
following the existing check for content that holds no trace of the
request. Requiring the term to be missing from the prose keeps it quiet
whenever the page can answer, and the two warnings do not stack.

Fixes ScrapeGraphAI#1120
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant