From decfc6bb6eb10a29ed6aaabb07244b8915042604 Mon Sep 17 00:00:00 2001
From: semantic-release-bot <semantic-release-bot@martynus.net>
Date: Fri, 21 Aug 2026 08:21:39 +0000
Subject: [PATCH 1/8] ci(release): 2.2.0-beta.7 [skip ci]

## [2.2.0-beta.7](https://github.com/ScrapeGraphAI/Scrapegraph-ai/compare/v2.2.0-beta.6...v2.2.0-beta.7) (2026-08-21)

### Bug Fixes

* resolve markdown links from the document URL ([875385b](https://github.com/ScrapeGraphAI/Scrapegraph-ai/commit/875385b56a3b2ce33b0bddc7baa194017d0293e2))

### Docs

* fix timeout documentation links ([4763fdc](https://github.com/ScrapeGraphAI/Scrapegraph-ai/commit/4763fdcc08af079a7e5a386ce58e7ae2d989155d))
---
 CHANGELOG.md   | 12 ++++++++++++
 pyproject.toml |  2 +-
 2 files changed, 13 insertions(+), 1 deletion(-)

diff --git a/CHANGELOG.md b/CHANGELOG.md
index 6c08454ac..7b6793442 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -1,3 +1,15 @@
+## [2.2.0-beta.7](https://github.com/ScrapeGraphAI/Scrapegraph-ai/compare/v2.2.0-beta.6...v2.2.0-beta.7) (2026-08-21)
+
+
+### Bug Fixes
+
+* resolve markdown links from the document URL ([875385b](https://github.com/ScrapeGraphAI/Scrapegraph-ai/commit/875385b56a3b2ce33b0bddc7baa194017d0293e2))
+
+
+### Docs
+
+* fix timeout documentation links ([4763fdc](https://github.com/ScrapeGraphAI/Scrapegraph-ai/commit/4763fdcc08af079a7e5a386ce58e7ae2d989155d))
+
 ## [2.2.0-beta.6](https://github.com/ScrapeGraphAI/Scrapegraph-ai/compare/v2.2.0-beta.5...v2.2.0-beta.6) (2026-08-19)
 
 
diff --git a/pyproject.toml b/pyproject.toml
index 20e58930d..051608dd3 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -1,7 +1,7 @@
 [project]
 name = "scrapegraphai"
 
-version = "2.2.0b6"
+version = "2.2.0b7"
 
 description = "A web scraping library based on LangChain which uses LLM and direct graph logic to create scraping pipelines."
 authors = [

From f91478eacf86485f6b9efcf843fc0c815dde1ec5 Mon Sep 17 00:00:00 2001
From: Marco Vinciguerra <mvincig11@gmail.com>
Date: Sun, 23 Aug 2026 15:11:48 +0200
Subject: [PATCH 2/8] fix(fetch): surface HTTP errors and missing content
 instead of answering NA
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

A page that could not be scraped as intended was indistinguishable from one
that could. FetchNode's default path (ChromiumLoader -> ascrape_playwright)
dropped the Response returned by page.goto(), so a 404, 403, 500, captcha wall
or login redirect reached the LLM as ordinary content and the model answered
"NA" with nothing in the logs to explain why. Reported in #1102, where
en.wikipedia.org/wiki/Timpson_(company) 404s (the article is at
Timpson_(retailer)) and the run still looked clean.

Two deterministic, LLM-free guards, both warnings so existing behaviour is
unchanged for anyone deliberately scraping error pages:

- ChromiumLoader keeps the Response from every page.goto() call site
  (ascrape_playwright, ascrape_playwright_scroll, ascrape_with_js_support) and
  warns on status >= 400. This mirrors what the opt-in use_soup=True path in
  FetchNode has always done.
- ParseNode warns when the parsed content contains none of the terms the user
  asked about — schema field names plus the significant words of the prompt.
  A 200 response can still reach the LLM without the requested data: content
  behind JavaScript that never rendered, a field inside a <script> blob the
  parser drops, or a document truncated beyond the model window. Zero matches
  is a deliberately conservative bar, so the warning stays quiet when the page
  simply phrases the answer differently. The graphs now pass their schema to
  ParseNode so it has the field names available.

Verified against the URLs from the issue: the 404 now logs "Received HTTP 404
for .../Timpson_(company); the scraped content is likely an error page" before
returning NA, while the corrected URL stays silent and answers 1865.

Also drops three dead imports from smart_scraper_multi_batch_graph.py, which
ruff blocks on now that the file is touched.

Fixes #1102

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
---
 .github/workflows/test-suite.yml              |   1 +
 scrapegraphai/docloaders/chromium.py          |  35 +-
 scrapegraphai/graphs/code_generator_graph.py  |   6 +-
 .../graphs/document_scraper_graph.py          |   1 +
 scrapegraphai/graphs/omni_scraper_graph.py    |   1 +
 scrapegraphai/graphs/script_creator_graph.py  |   1 +
 scrapegraphai/graphs/smart_scraper_graph.py   |   7 +-
 .../graphs/smart_scraper_lite_graph.py        |   6 +-
 .../graphs/smart_scraper_multi_batch_graph.py |   5 +-
 scrapegraphai/graphs/speech_graph.py          |   6 +-
 scrapegraphai/nodes/parse_node.py             | 256 +++++++++++++-
 tests/test_error_page_detection.py            | 326 ++++++++++++++++++
 12 files changed, 640 insertions(+), 11 deletions(-)
 create mode 100644 tests/test_error_page_detection.py

diff --git a/.github/workflows/test-suite.yml b/.github/workflows/test-suite.yml
index e6dda4123..d8c88a6dc 100644
--- a/.github/workflows/test-suite.yml
+++ b/.github/workflows/test-suite.yml
@@ -42,6 +42,7 @@ jobs:
           tests/test_batch_api.py
           tests/test_csv_scraper_multi_graph.py
           tests/test_depth_search_graph.py
+          tests/test_error_page_detection.py
           tests/test_json_scraper_graph.py
           tests/test_minimax_models.py
           tests/test_scrape_do.py
diff --git a/scrapegraphai/docloaders/chromium.py b/scrapegraphai/docloaders/chromium.py
index a522582c3..a452af1ea 100644
--- a/scrapegraphai/docloaders/chromium.py
+++ b/scrapegraphai/docloaders/chromium.py
@@ -10,6 +10,32 @@
 logger = get_logger("web-loader")
 
 
+def _warn_on_error_status(response: Any, url: str) -> None:
+    """Log a warning when a navigation returned an HTTP error status.
+
+    Playwright's ``page.goto()`` returns the main-frame ``Response``, but the
+    scrapers only keep ``page.content()``. Without this check an error page
+    (404, 403, 500, a captcha wall, a login redirect) is indistinguishable
+    from the intended document once it reaches the LLM, which then produces a
+    confidently wrong answer with no signal that anything went wrong.
+
+    This mirrors the behaviour of the ``use_soup=True`` path in ``FetchNode``:
+    it warns rather than raising, so scraping error pages on purpose keeps
+    working.
+
+    Args:
+        response: The ``Response`` returned by ``page.goto()``; may be ``None``
+            (for example on a same-document navigation) or lack a usable status.
+        url: The URL that was requested, used in the warning message.
+    """
+    status = getattr(response, "status", None)
+    if isinstance(status, int) and status >= 400:
+        logger.warning(
+            f"Received HTTP {status} for {url}; the scraped content is likely "
+            "an error page, not the intended document."
+        )
+
+
 class ChromiumLoader:
     """Scrapes HTML pages from URLs using a (headless) instance of the
     Chromium web driver with proxy protection.
@@ -251,7 +277,8 @@ async def ascrape_playwright_scroll(
                     context = await browser.new_context()
                     await Malenia.apply_stealth(context)
                     page = await context.new_page()
-                    await page.goto(url, wait_until="domcontentloaded")
+                    response = await page.goto(url, wait_until="domcontentloaded")
+                    _warn_on_error_status(response, url)
                     await page.wait_for_load_state(self.load_state)
 
                     previous_height = None
@@ -364,7 +391,8 @@ async def ascrape_playwright(self, url: str, browser_name: str = "chromium") ->
                     )
                     await Malenia.apply_stealth(context)
                     page = await context.new_page()
-                    await page.goto(url, wait_until="domcontentloaded")
+                    response = await page.goto(url, wait_until="domcontentloaded")
+                    _warn_on_error_status(response, url)
                     await page.wait_for_load_state(self.load_state)
                     results = await page.content()
                     logger.info("Content scraped")
@@ -421,7 +449,8 @@ async def ascrape_with_js_support(
                         storage_state=self.storage_state
                     )
                     page = await context.new_page()
-                    await page.goto(url, wait_until="networkidle")
+                    response = await page.goto(url, wait_until="networkidle")
+                    _warn_on_error_status(response, url)
                     results = await page.content()
                     logger.info("Content scraped after JavaScript rendering")
                     return results
diff --git a/scrapegraphai/graphs/code_generator_graph.py b/scrapegraphai/graphs/code_generator_graph.py
index 506b87a1b..21cb51e07 100644
--- a/scrapegraphai/graphs/code_generator_graph.py
+++ b/scrapegraphai/graphs/code_generator_graph.py
@@ -93,7 +93,11 @@ def _create_graph(self) -> BaseGraph:
         parse_node = ParseNode(
             input="doc",
             output=["parsed_doc"],
-            node_config={"llm_model": self.llm_model, "chunk_size": self.model_token},
+            node_config={
+                "llm_model": self.llm_model,
+                "chunk_size": self.model_token,
+                "schema": self.schema,
+            },
         )
 
         generate_validation_answer_node = GenerateAnswerNode(
diff --git a/scrapegraphai/graphs/document_scraper_graph.py b/scrapegraphai/graphs/document_scraper_graph.py
index 012ab0b72..4470a099e 100644
--- a/scrapegraphai/graphs/document_scraper_graph.py
+++ b/scrapegraphai/graphs/document_scraper_graph.py
@@ -76,6 +76,7 @@ def _create_graph(self) -> BaseGraph:
                 "parse_html": False,
                 "chunk_size": self.model_token,
                 "llm_model": self.llm_model,
+                "schema": self.schema,
             },
         )
         generate_answer_node = GenerateAnswerNode(
diff --git a/scrapegraphai/graphs/omni_scraper_graph.py b/scrapegraphai/graphs/omni_scraper_graph.py
index ef55dc758..177dc6d75 100644
--- a/scrapegraphai/graphs/omni_scraper_graph.py
+++ b/scrapegraphai/graphs/omni_scraper_graph.py
@@ -83,6 +83,7 @@ def _create_graph(self) -> BaseGraph:
                 "chunk_size": self.model_token,
                 "parse_urls": True,
                 "llm_model": self.llm_model,
+                "schema": self.schema,
             },
         )
 
diff --git a/scrapegraphai/graphs/script_creator_graph.py b/scrapegraphai/graphs/script_creator_graph.py
index 4d373a28a..2b03db0bc 100644
--- a/scrapegraphai/graphs/script_creator_graph.py
+++ b/scrapegraphai/graphs/script_creator_graph.py
@@ -82,6 +82,7 @@ def _create_graph(self) -> BaseGraph:
                 "chunk_size": self.model_token,
                 "parse_html": False,
                 "llm_model": self.llm_model,
+                "schema": self.schema,
             },
         )
 
diff --git a/scrapegraphai/graphs/smart_scraper_graph.py b/scrapegraphai/graphs/smart_scraper_graph.py
index 27c6cb172..b29d038ae 100644
--- a/scrapegraphai/graphs/smart_scraper_graph.py
+++ b/scrapegraphai/graphs/smart_scraper_graph.py
@@ -110,7 +110,11 @@ def _create_graph(self) -> BaseGraph:
         parse_node = ParseNode(
             input="doc",
             output=["parsed_doc"],
-            node_config={"llm_model": self.llm_model, "chunk_size": self.model_token},
+            node_config={
+                "llm_model": self.llm_model,
+                "chunk_size": self.model_token,
+                "schema": self.schema,
+            },
         )
 
         generate_answer_node = GenerateAnswerNode(
@@ -152,6 +156,7 @@ def _create_graph(self) -> BaseGraph:
                 node_config={
                     "llm_model": self.llm_model,
                     "chunk_size": self.model_token,
+                    "schema": self.schema,
                 },
             )
 
diff --git a/scrapegraphai/graphs/smart_scraper_lite_graph.py b/scrapegraphai/graphs/smart_scraper_lite_graph.py
index a845dc7ad..0254d0a82 100644
--- a/scrapegraphai/graphs/smart_scraper_lite_graph.py
+++ b/scrapegraphai/graphs/smart_scraper_lite_graph.py
@@ -74,7 +74,11 @@ def _create_graph(self) -> BaseGraph:
         parse_node = ParseNode(
             input="doc",
             output=["parsed_doc"],
-            node_config={"llm_model": self.llm_model, "chunk_size": self.model_token},
+            node_config={
+                "llm_model": self.llm_model,
+                "chunk_size": self.model_token,
+                "schema": self.schema,
+            },
         )
 
         return BaseGraph(
diff --git a/scrapegraphai/graphs/smart_scraper_multi_batch_graph.py b/scrapegraphai/graphs/smart_scraper_multi_batch_graph.py
index 399e18dc8..89db09c53 100644
--- a/scrapegraphai/graphs/smart_scraper_multi_batch_graph.py
+++ b/scrapegraphai/graphs/smart_scraper_multi_batch_graph.py
@@ -5,9 +5,8 @@
 providing 50% cost savings compared to real-time API calls.
 """
 
-import asyncio
 from copy import deepcopy
-from typing import Dict, List, Optional, Type
+from typing import List, Optional, Type
 
 from pydantic import BaseModel
 
@@ -17,7 +16,6 @@
 from ..utils.copy import safe_deepcopy
 from .abstract_graph import AbstractGraph
 from .base_graph import BaseGraph
-from .smart_scraper_graph import SmartScraperGraph
 
 
 class _FetchParseOnlyGraph(AbstractGraph):
@@ -57,6 +55,7 @@ def _create_graph(self) -> BaseGraph:
             node_config={
                 "llm_model": self.llm_model,
                 "chunk_size": self.model_token,
+                "schema": self.schema,
             },
         )
 
diff --git a/scrapegraphai/graphs/speech_graph.py b/scrapegraphai/graphs/speech_graph.py
index 6622b1213..14aea5ed2 100644
--- a/scrapegraphai/graphs/speech_graph.py
+++ b/scrapegraphai/graphs/speech_graph.py
@@ -70,7 +70,11 @@ def _create_graph(self) -> BaseGraph:
         parse_node = ParseNode(
             input="doc",
             output=["parsed_doc"],
-            node_config={"chunk_size": self.model_token, "llm_model": self.llm_model},
+            node_config={
+                "chunk_size": self.model_token,
+                "llm_model": self.llm_model,
+                "schema": self.schema,
+            },
         )
 
         generate_answer_node = GenerateAnswerNode(
diff --git a/scrapegraphai/nodes/parse_node.py b/scrapegraphai/nodes/parse_node.py
index 44cd5896c..b4a4a6f7b 100644
--- a/scrapegraphai/nodes/parse_node.py
+++ b/scrapegraphai/nodes/parse_node.py
@@ -3,7 +3,7 @@
 """
 
 import re
-from typing import List, Optional, Tuple
+from typing import List, Optional, Set, Tuple, get_args
 from urllib.parse import urljoin
 
 from langchain_community.document_transformers import Html2TextTransformer
@@ -37,6 +37,93 @@ class ParseNode(BaseNode):
     )
     relative_url_pattern = re.compile(r"[\(](/[^\(\)\s]*)")
 
+    # Words carrying no discriminative signal, dropped before checking whether the
+    # parsed document contains any evidence of what the user asked for.
+    prompt_stopwords = frozenset(
+        {
+            "about",
+            "after",
+            "all",
+            "also",
+            "and",
+            "any",
+            "are",
+            "been",
+            "both",
+            "but",
+            "can",
+            "content",
+            "data",
+            "does",
+            "each",
+            "every",
+            "extract",
+            "find",
+            "for",
+            "from",
+            "get",
+            "give",
+            "has",
+            "have",
+            "here",
+            "how",
+            "html",
+            "info",
+            "information",
+            "into",
+            "its",
+            "json",
+            "list",
+            "many",
+            "more",
+            "much",
+            "must",
+            "not",
+            "only",
+            "out",
+            "page",
+            "please",
+            "provide",
+            "retrieve",
+            "return",
+            "scrape",
+            "site",
+            "some",
+            "such",
+            "text",
+            "that",
+            "the",
+            "their",
+            "them",
+            "then",
+            "there",
+            "these",
+            "they",
+            "this",
+            "those",
+            "url",
+            "was",
+            "web",
+            "webpage",
+            "website",
+            "were",
+            "what",
+            "when",
+            "where",
+            "which",
+            "who",
+            "why",
+            "will",
+            "with",
+            "would",
+            "you",
+            "your",
+        }
+    )
+
+    word_pattern = re.compile(r"[a-zA-Z][a-zA-Z0-9]{2,}")
+    camel_case_pattern = re.compile(r"(?<=[a-z0-9])(?=[A-Z])")
+
     def __init__(
         self,
         input: str,
@@ -58,6 +145,7 @@ def __init__(
 
         self.llm_model = node_config.get("llm_model")
         self.chunk_size = node_config.get("chunk_size")
+        self.schema = node_config.get("schema")
 
     def execute(self, state: dict) -> dict:
         """
@@ -119,6 +207,8 @@ def execute(self, state: dict) -> dict:
                     text=docs_transformed, chunk_size=chunk_size
                 )
 
+        self._warn_if_content_lacks_requested_fields(chunks, state.get("user_prompt"))
+
         state.update({self.output[0]: chunks})
         state.update({"parsed_doc": chunks})
 
@@ -128,6 +218,170 @@ def execute(self, state: dict) -> dict:
 
         return state
 
+    def _warn_if_content_lacks_requested_fields(
+        self, chunks: List[str], user_prompt: Optional[str]
+    ) -> None:
+        """
+        Warns when the parsed content holds no trace of what the user asked for.
+
+        An HTTP status check catches error pages, but a perfectly valid 200 page
+        can still reach the LLM without the requested data: content behind
+        JavaScript that never rendered, a field living in a ``<script>`` blob the
+        parser drops, or a document truncated beyond the model window. In each
+        case the LLM answers ``NA`` and the run looks clean.
+
+        This is a deterministic, LLM-free check: it collects the terms the user
+        asked about (schema field names and the significant words of the prompt)
+        and warns only when *none* of them appear in the parsed text. Zero
+        matches is a deliberately conservative bar, so the warning stays quiet
+        for legitimate runs where the answer is phrased differently from the
+        question.
+
+        Args:
+            chunks (List[str]): The parsed content chunks about to be handed downstream.
+            user_prompt (Optional[str]): The user's request, when available in the state.
+        """
+        texts = [chunk for chunk in chunks if isinstance(chunk, str)]
+        total_length = sum(len(text) for text in texts)
+
+        if not any(text.strip() for text in texts):
+            self.logger.warning(
+                "The parsed content is empty; the model will be asked to answer "
+                "from nothing. Check that the source was fetched correctly."
+            )
+            return
+
+        expected_terms = self._collect_expected_terms(user_prompt)
+        if not expected_terms:
+            return
+
+        # Chunks overlap, so a term split across a boundary is still found in one
+        # of them; searching chunk by chunk avoids rebuilding the whole document.
+        for text in texts:
+            lowered = text.lower()
+            if any(term in lowered for term in expected_terms):
+                return
+
+        self.logger.warning(
+            f"None of the requested terms {sorted(expected_terms)} appear in the "
+            f"parsed content ({total_length} chars). The source may be an error "
+            "page, may render its content with JavaScript, or the relevant "
+            "section may have been dropped while parsing; the model will most "
+            "likely answer NA."
+        )
+
+    def _collect_expected_terms(self, user_prompt: Optional[str]) -> Set[str]:
+        """
+        Builds the set of lowercase terms that evidence the user's request.
+
+        Args:
+            user_prompt (Optional[str]): The user's request, when available.
+
+        Returns:
+            Set[str]: Significant terms drawn from the schema field names and the
+                prompt; empty when nothing discriminative could be derived.
+        """
+        terms = self._schema_field_terms(self.schema)
+
+        if isinstance(user_prompt, str):
+            terms |= self._significant_words(user_prompt)
+
+        return terms
+
+    @classmethod
+    def _significant_words(cls, text: str) -> Set[str]:
+        """
+        Extracts the discriminative words of a piece of text.
+
+        Args:
+            text (str): The text to tokenize.
+
+        Returns:
+            Set[str]: Lowercase words at least three characters long that are not
+                generic scraping vocabulary.
+        """
+        words = cls.word_pattern.findall(cls.camel_case_pattern.sub(" ", text))
+        return {
+            word.lower() for word in words if word.lower() not in cls.prompt_stopwords
+        }
+
+    @classmethod
+    def _schema_field_terms(cls, schema, _depth: int = 0) -> Set[str]:
+        """
+        Extracts the field names of an output schema, recursing into nested ones.
+
+        Supports the schema flavours the library accepts: Pydantic models, plain
+        JSON Schema dictionaries, and lists of either.
+
+        Args:
+            schema: The output schema, in any of the supported forms; may be None.
+            _depth (int): Internal recursion guard for deeply nested schemas.
+
+        Returns:
+            Set[str]: Lowercase terms derived from the field names, with
+                ``snake_case`` and ``camelCase`` names split into their parts.
+        """
+        if schema is None or _depth > 5:
+            return set()
+
+        names: List[str] = []
+        terms: Set[str] = set()
+
+        if isinstance(schema, dict):
+            properties = schema.get("properties")
+            if isinstance(properties, dict):
+                names.extend(str(key) for key in properties)
+                for value in properties.values():
+                    terms |= cls._schema_field_terms(value, _depth + 1)
+            items = schema.get("items")
+            if items is not None:
+                terms |= cls._schema_field_terms(items, _depth + 1)
+        elif isinstance(schema, (list, tuple, set)):
+            for entry in schema:
+                terms |= cls._schema_field_terms(entry, _depth + 1)
+        else:
+            model_fields = getattr(schema, "model_fields", None)  # pydantic v2
+            if model_fields is None:
+                model_fields = getattr(schema, "__fields__", None)  # pydantic v1
+            if isinstance(model_fields, dict):
+                names.extend(str(key) for key in model_fields)
+                for field in model_fields.values():
+                    annotation = getattr(field, "annotation", None)
+                    for nested in cls._nested_models(annotation):
+                        terms |= cls._schema_field_terms(nested, _depth + 1)
+
+        for name in names:
+            terms |= cls._significant_words(name.replace("_", " "))
+
+        return terms
+
+    @staticmethod
+    def _nested_models(annotation) -> List:
+        """
+        Finds the Pydantic models reachable from a field annotation.
+
+        Unwraps the typing containers schemas commonly use — ``List[Item]``,
+        ``Optional[Item]``, ``Dict[str, Item]`` — so nested field names are not
+        lost.
+
+        Args:
+            annotation: The annotation of a Pydantic field; may be None.
+
+        Returns:
+            List: The Pydantic model classes found in the annotation.
+        """
+        if annotation is None:
+            return []
+
+        if hasattr(annotation, "model_fields") or hasattr(annotation, "__fields__"):
+            return [annotation]
+
+        return [
+            arg
+            for arg in get_args(annotation)
+            if hasattr(arg, "model_fields") or hasattr(arg, "__fields__")
+        ]
+
     def _extract_urls(self, text: str, source: str) -> Tuple[List[str], List[str]]:
         """
         Extracts URLs from the given text.
diff --git a/tests/test_error_page_detection.py b/tests/test_error_page_detection.py
new file mode 100644
index 000000000..3c402ebd6
--- /dev/null
+++ b/tests/test_error_page_detection.py
@@ -0,0 +1,326 @@
+"""
+Tests for the silent-error-page guards added for issue #1102.
+
+Two independent signals are covered:
+
+1. ``ChromiumLoader`` warns when ``page.goto()`` reports an HTTP error status,
+   so a 404/403/500 page is no longer handed to the LLM as if it were the
+   intended document.
+2. ``ParseNode`` warns when the parsed content contains no trace of what the
+   user asked for, which also catches 200 pages whose content never rendered.
+"""
+
+import asyncio
+from typing import List
+from unittest.mock import AsyncMock, MagicMock, patch
+
+import pytest
+from langchain_core.documents import Document
+from pydantic import BaseModel
+
+from scrapegraphai.docloaders.chromium import ChromiumLoader, _warn_on_error_status
+from scrapegraphai.nodes import ParseNode
+from scrapegraphai.utils.logging import set_propagation, unset_propagation
+
+# --------------------------------------------------------------------------- #
+# ChromiumLoader: HTTP status awareness on every page.goto() call site
+# --------------------------------------------------------------------------- #
+
+
+class _MockPage:
+    def __init__(self, status):
+        response = MagicMock()
+        response.status = status
+        self.goto = AsyncMock(return_value=response)
+        self.wait_for_load_state = AsyncMock()
+        self.content = AsyncMock(return_value="<html>error page</html>")
+        self.evaluate = AsyncMock(return_value=1000)
+        self.mouse = MagicMock()
+        self.mouse.wheel = AsyncMock()
+
+
+@pytest.fixture
+def playwright_with_status():
+    """Patch playwright so page.goto() returns a response with a given status."""
+
+    def _factory(status):
+        page = _MockPage(status)
+        context = MagicMock()
+        context.new_page = AsyncMock(return_value=page)
+        browser = MagicMock()
+        browser.new_context = AsyncMock(return_value=context)
+        browser.close = AsyncMock()
+        pw = MagicMock()
+        pw.chromium.launch = AsyncMock(return_value=browser)
+        pw.firefox.launch = AsyncMock(return_value=browser)
+
+        cm = MagicMock()
+        cm.__aenter__ = AsyncMock(return_value=pw)
+        cm.__aexit__ = AsyncMock(return_value=False)
+        return cm, page
+
+    return _factory
+
+
+@pytest.mark.parametrize("status", [404, 403, 500, 503])
+def test_ascrape_playwright_warns_on_error_status(
+    playwright_with_status, status, caplog
+):
+    cm, _ = playwright_with_status(status)
+    loader = ChromiumLoader(["https://example.com/missing"], backend="playwright")
+
+    with (
+        patch("playwright.async_api.async_playwright", return_value=cm),
+        patch("undetected_playwright.Malenia.apply_stealth", new=AsyncMock()),
+        caplog.at_level("WARNING"),
+    ):
+        asyncio.run(loader.ascrape_playwright("https://example.com/missing"))
+
+    assert f"Received HTTP {status}" in caplog.text
+    assert "likely an error page" in caplog.text
+
+
+def test_ascrape_playwright_silent_on_success(playwright_with_status, caplog):
+    cm, _ = playwright_with_status(200)
+    loader = ChromiumLoader(["https://example.com"], backend="playwright")
+
+    with (
+        patch("playwright.async_api.async_playwright", return_value=cm),
+        patch("undetected_playwright.Malenia.apply_stealth", new=AsyncMock()),
+        caplog.at_level("WARNING"),
+    ):
+        asyncio.run(loader.ascrape_playwright("https://example.com"))
+
+    assert "Received HTTP" not in caplog.text
+
+
+def test_ascrape_with_js_support_warns_on_error_status(playwright_with_status, caplog):
+    cm, _ = playwright_with_status(404)
+    loader = ChromiumLoader(
+        ["https://example.com/missing"], backend="playwright", requires_js_support=True
+    )
+
+    with (
+        patch("playwright.async_api.async_playwright", return_value=cm),
+        caplog.at_level("WARNING"),
+    ):
+        asyncio.run(loader.ascrape_with_js_support("https://example.com/missing"))
+
+    assert "Received HTTP 404" in caplog.text
+
+
+def test_ascrape_playwright_scroll_warns_on_error_status(
+    playwright_with_status, caplog
+):
+    cm, page = playwright_with_status(404)
+    # Stop the scroll loop immediately: same height twice means "bottom reached".
+    page.evaluate = AsyncMock(return_value=1000)
+    loader = ChromiumLoader(["https://example.com/missing"], backend="playwright")
+
+    with (
+        patch("playwright.async_api.async_playwright", return_value=cm),
+        patch("undetected_playwright.Malenia.apply_stealth", new=AsyncMock()),
+        caplog.at_level("WARNING"),
+    ):
+        asyncio.run(
+            loader.ascrape_playwright_scroll(
+                "https://example.com/missing", scroll=5000, sleep=0.01, timeout=1
+            )
+        )
+
+    assert "Received HTTP 404" in caplog.text
+
+
+def test_warn_on_error_status_tolerates_missing_response(caplog):
+    """page.goto() returns None for same-document navigations; that is not an error."""
+    with caplog.at_level("WARNING"):
+        _warn_on_error_status(None, "https://example.com")
+        _warn_on_error_status(MagicMock(status=None), "https://example.com")
+
+    assert "Received HTTP" not in caplog.text
+
+
+# --------------------------------------------------------------------------- #
+# ParseNode: warn when the parsed content holds no trace of the request
+# --------------------------------------------------------------------------- #
+
+
+@pytest.fixture
+def library_logs_propagate():
+    """Let caplog see records from the library root logger.
+
+    ``scrapegraphai`` disables propagation by default so it does not pollute the
+    host application's logging; the nodes log through that root logger.
+    """
+    set_propagation()
+    yield
+    unset_propagation()
+
+
+class Company(BaseModel):
+    company_name: str
+    foundingYear: int
+
+
+class Employee(BaseModel):
+    employee_name: str
+    salary: str
+
+
+class Payroll(BaseModel):
+    employees: List[Employee]
+
+
+def _parse_node(**node_config):
+    config = {"chunk_size": 4096, "verbose": False}
+    config.update(node_config)
+    return ParseNode(input="doc", output=["parsed_doc"], node_config=config)
+
+
+def _run(node, html, user_prompt):
+    state = {"doc": [Document(page_content=html)], "user_prompt": user_prompt}
+    return node.execute(state)
+
+
+WIKIPEDIA_404 = (
+    "<html><body><p>Jump to content. Main menu. Navigation. "
+    "Wikipedia does not have an article with this exact name.</p></body></html>"
+)
+
+TIMPSON_PAGE = (
+    "<html><body><p>Timpson is a British retailer founded in 1865 "
+    "by William Timpson.</p></body></html>"
+)
+
+# A shell page whose real content is rendered client-side: HTTP 200, no error,
+# and nothing for the LLM to work with.
+JS_SHELL_PAGE = "<html><body><div id='root'>Loading...</div></body></html>"
+
+STRUCTURED_PAGE = (
+    "<html><body><p>Company name: Timpson</p>"
+    "<p>Founding year: 1865</p></body></html>"
+)
+
+
+def test_warns_when_no_requested_term_is_present(library_logs_propagate, caplog):
+    node = _parse_node()
+
+    with caplog.at_level("WARNING"):
+        _run(node, WIKIPEDIA_404, "What is the founding year of Timpson?")
+
+    assert "None of the requested terms" in caplog.text
+
+
+def test_silent_when_the_content_holds_the_answer(library_logs_propagate, caplog):
+    node = _parse_node()
+
+    with caplog.at_level("WARNING"):
+        _run(node, TIMPSON_PAGE, "What is the founding year of Timpson?")
+
+    assert "None of the requested terms" not in caplog.text
+
+
+def test_schema_field_names_count_as_requested_terms(library_logs_propagate, caplog):
+    """A page that never rendered is a 200, so only the schema can flag it."""
+    node = _parse_node(schema=Company)
+
+    with caplog.at_level("WARNING"):
+        _run(node, JS_SHELL_PAGE, None)
+
+    assert "None of the requested terms" in caplog.text
+    # snake_case and camelCase names are split into their parts
+    assert "founding" in caplog.text
+    assert "company" in caplog.text
+    assert "year" in caplog.text
+
+
+def test_schema_match_keeps_the_check_quiet(library_logs_propagate, caplog):
+    node = _parse_node(schema=Company)
+
+    with caplog.at_level("WARNING"):
+        _run(node, STRUCTURED_PAGE, None)
+
+    assert "None of the requested terms" not in caplog.text
+
+
+def test_schema_terms_can_rescue_an_unspecific_prompt(library_logs_propagate, caplog):
+    """The union of prompt and schema terms only ever makes the check quieter."""
+    node = _parse_node(schema=Company)
+
+    with caplog.at_level("WARNING"):
+        _run(node, STRUCTURED_PAGE, "Extract everything you can find")
+
+    assert "None of the requested terms" not in caplog.text
+
+
+def test_nested_pydantic_models_are_unwrapped(library_logs_propagate, caplog):
+    """List[Item] and friends must not hide the nested field names."""
+    node = _parse_node(schema=Payroll)
+
+    with caplog.at_level("WARNING"):
+        _run(node, JS_SHELL_PAGE, None)
+
+    assert "salary" in caplog.text
+    assert "employee" in caplog.text
+
+
+def test_json_schema_dict_is_supported(library_logs_propagate, caplog):
+    schema = {
+        "type": "object",
+        "properties": {
+            "founding_year": {"type": "integer"},
+            "locations": {
+                "type": "array",
+                "items": {
+                    "type": "object",
+                    "properties": {"postcode": {"type": "string"}},
+                },
+            },
+        },
+    }
+    node = _parse_node(schema=schema)
+
+    with caplog.at_level("WARNING"):
+        _run(node, JS_SHELL_PAGE, None)
+
+    assert "postcode" in caplog.text
+    assert "locations" in caplog.text
+
+
+def test_no_prompt_and_no_schema_produces_no_warning(library_logs_propagate, caplog):
+    node = _parse_node()
+
+    with caplog.at_level("WARNING"):
+        _run(node, WIKIPEDIA_404, None)
+
+    assert "None of the requested terms" not in caplog.text
+
+
+def test_generic_prompt_words_alone_do_not_trigger_the_warning(
+    library_logs_propagate, caplog
+):
+    """A prompt made only of scraping vocabulary carries no signal to check."""
+    node = _parse_node()
+
+    with caplog.at_level("WARNING"):
+        _run(node, WIKIPEDIA_404, "Extract all the information from this webpage")
+
+    assert "None of the requested terms" not in caplog.text
+
+
+def test_empty_parsed_content_is_reported(library_logs_propagate, caplog):
+    node = _parse_node()
+
+    with caplog.at_level("WARNING"):
+        _run(node, "", "What is the founding year of Timpson?")
+
+    assert "parsed content is empty" in caplog.text
+
+
+def test_state_is_unchanged_by_the_guard():
+    """The guard only logs; the parsed chunks must reach the state as before."""
+    node = _parse_node(schema=Company)
+    state = _run(node, TIMPSON_PAGE, "What is the founding year of Timpson?")
+
+    assert state["parsed_doc"]
+    assert "1865" in "".join(state["parsed_doc"])

From d59c3dfceecdacbba4e17f237b017117cf7f1cee Mon Sep 17 00:00:00 2001
From: semantic-release-bot <semantic-release-bot@martynus.net>
Date: Sun, 23 Aug 2026 13:17:42 +0000
Subject: [PATCH 3/8] ci(release): 2.2.0-beta.8 [skip ci]

## [2.2.0-beta.8](https://github.com/ScrapeGraphAI/Scrapegraph-ai/compare/v2.2.0-beta.7...v2.2.0-beta.8) (2026-08-23)

### Bug Fixes

* **fetch:** surface HTTP errors and missing content instead of answering NA ([f91478e](https://github.com/ScrapeGraphAI/Scrapegraph-ai/commit/f91478eacf86485f6b9efcf843fc0c815dde1ec5)), closes [#1102](https://github.com/ScrapeGraphAI/Scrapegraph-ai/issues/1102) [#1102](https://github.com/ScrapeGraphAI/Scrapegraph-ai/issues/1102)
---
 CHANGELOG.md   | 7 +++++++
 pyproject.toml | 2 +-
 2 files changed, 8 insertions(+), 1 deletion(-)

diff --git a/CHANGELOG.md b/CHANGELOG.md
index 7b6793442..0f4785663 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -1,3 +1,10 @@
+## [2.2.0-beta.8](https://github.com/ScrapeGraphAI/Scrapegraph-ai/compare/v2.2.0-beta.7...v2.2.0-beta.8) (2026-08-23)
+
+
+### Bug Fixes
+
+* **fetch:** surface HTTP errors and missing content instead of answering NA ([f91478e](https://github.com/ScrapeGraphAI/Scrapegraph-ai/commit/f91478eacf86485f6b9efcf843fc0c815dde1ec5)), closes [#1102](https://github.com/ScrapeGraphAI/Scrapegraph-ai/issues/1102) [#1102](https://github.com/ScrapeGraphAI/Scrapegraph-ai/issues/1102)
+
 ## [2.2.0-beta.7](https://github.com/ScrapeGraphAI/Scrapegraph-ai/compare/v2.2.0-beta.6...v2.2.0-beta.7) (2026-08-21)
 
 
diff --git a/pyproject.toml b/pyproject.toml
index 051608dd3..76ea41644 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -1,7 +1,7 @@
 [project]
 name = "scrapegraphai"
 
-version = "2.2.0b7"
+version = "2.2.0b8"
 
 description = "A web scraping library based on LangChain which uses LLM and direct graph logic to create scraping pipelines."
 authors = [

From c21af206862c13be1848eac75b4c04250718c8d9 Mon Sep 17 00:00:00 2001
From: AMIR SHAHZAD HASHMI <amirshahzadhashmi7145@gmail.com>
Date: Mon, 31 Aug 2026 02:27:14 +0500
Subject: [PATCH 4/8] fix(models): add Gemini 2.5 token limits so they are not
 truncated to 8192

---
 scrapegraphai/helpers/models_tokens.py |   9 ++
 tests/test_models_tokens.py            | 163 +++++++++++++++----------
 2 files changed, 106 insertions(+), 66 deletions(-)

diff --git a/scrapegraphai/helpers/models_tokens.py b/scrapegraphai/helpers/models_tokens.py
index b5aef2f9b..e64b1225b 100644
--- a/scrapegraphai/helpers/models_tokens.py
+++ b/scrapegraphai/helpers/models_tokens.py
@@ -140,6 +140,11 @@
         "gemini-2.0-flash-latest": 1000000,
         "gemini-2.0-flash-exp": 1000000,
         "gemini-2.0-pro-exp": 2000000,
+        "gemini-2.5-flash": 1000000,
+        "gemini-2.5-flash-latest": 1000000,
+        "gemini-2.5-flash-lite": 1000000,
+        "gemini-2.5-pro": 1000000,
+        "gemini-flash-latest": 1000000,
         "models/embedding-001": 2048,
     },
     "google_vertexai": {
@@ -150,6 +155,10 @@
         "gemini-2.0-flash-exp": 1048576,
         "gemini-2.0-pro": 2000000,
         "gemini-2.0-pro-exp": 2000000,
+        "gemini-2.5-flash": 1048576,
+        "gemini-2.5-flash-lite": 1048576,
+        "gemini-2.5-pro": 1048576,
+        "gemini-flash-latest": 1048576,
     },
     "ollama": {
         "command-r": 12800,
diff --git a/tests/test_models_tokens.py b/tests/test_models_tokens.py
index bfde8df9c..9b40e5fcd 100644
--- a/tests/test_models_tokens.py
+++ b/tests/test_models_tokens.py
@@ -7,9 +7,9 @@ class TestModelsTokens:
     def test_openai_tokens(self):
         """Test that the 'openai' provider exists and its tokens are valid positive integers."""
         openai_models = models_tokens.get("openai")
-        assert openai_models is not None, (
-            "'openai' key should be present in models_tokens"
-        )
+        assert (
+            openai_models is not None
+        ), "'openai' key should be present in models_tokens"
         for model, token in openai_models.items():
             assert isinstance(model, str), "Model name should be a string"
             assert isinstance(token, int), "Token limit should be an integer"
@@ -30,19 +30,50 @@ def test_google_providers(self):
         assert google_genai is not None, "'google_genai' key should be present"
         assert google_vertexai is not None, "'google_vertexai' key should be present"
         # Check a specific key from google_genai
-        assert "gemini-pro" in google_genai, (
-            "'gemini-pro' should be in google_genai models"
-        )
+        assert (
+            "gemini-pro" in google_genai
+        ), "'gemini-pro' should be in google_genai models"
         # Validate token values types
         for provider in [google_genai, google_vertexai]:
             for token in provider.values():
                 assert isinstance(token, int), "Token limit must be an integer"
 
+    def test_gemini_2_5_models_are_registered(self):
+        """Gemini 2.5 / flash-latest must be in the table so they are not truncated to 8192.
+
+        #1121: an unknown model silently falls back to an 8192-token window.
+        google_genai/gemini-2.5-flash is the reported case; gemini-flash-latest
+        is the current flash alias. Both have a 1M input context.
+        """
+        google_genai = models_tokens["google_genai"]
+        google_vertexai = models_tokens["google_vertexai"]
+
+        for model in (
+            "gemini-2.5-flash",
+            "gemini-2.5-flash-latest",
+            "gemini-2.5-flash-lite",
+            "gemini-2.5-pro",
+            "gemini-flash-latest",
+        ):
+            assert (
+                google_genai.get(model) == 1000000
+            ), f"Expected 1M context for {model} in google_genai"
+
+        for model in (
+            "gemini-2.5-flash",
+            "gemini-2.5-flash-lite",
+            "gemini-2.5-pro",
+            "gemini-flash-latest",
+        ):
+            assert (
+                google_vertexai.get(model) == 1048576
+            ), f"Expected 1M context for {model} in google_vertexai"
+
     def test_non_existent_provider(self):
         """Test that a non-existent provider returns None."""
-        assert models_tokens.get("non_existent") is None, (
-            "Non-existent provider should return None"
-        )
+        assert (
+            models_tokens.get("non_existent") is None
+        ), "Non-existent provider should return None"
 
     def test_total_model_keys(self):
         """Test that the total number of models across all providers is above an expected count."""
@@ -59,120 +90,120 @@ def test_non_empty_model_keys(self):
         """Ensure that model token names are non-empty strings."""
         for provider, model_dict in models_tokens.items():
             for model in model_dict.keys():
-                assert model != "", (
-                    f"Model name in provider '{provider}' should not be empty."
-                )
+                assert (
+                    model != ""
+                ), f"Model name in provider '{provider}' should not be empty."
 
     def test_token_limits_range(self):
         """Test that token limits for all models fall within a plausible range (e.g., 1 to 300000)."""
         for provider, model_dict in models_tokens.items():
             for model, token in model_dict.items():
-                assert 1 <= token <= 1100000, (
-                    f"Token limit for {model} in provider {provider} is out of plausible range."
-                )
+                assert (
+                    1 <= token <= 1100000
+                ), f"Token limit for {model} in provider {provider} is out of plausible range."
 
     def test_provider_structure(self):
         """Test that every provider in models_tokens has a dictionary as its value."""
         for provider, models in models_tokens.items():
-            assert isinstance(models, dict), (
-                f"Provider {provider} should map to a dictionary, got {type(models).__name__}"
-            )
+            assert isinstance(
+                models, dict
+            ), f"Provider {provider} should map to a dictionary, got {type(models).__name__}"
 
     def test_non_empty_provider(self):
         """Test that each provider dictionary is not empty."""
         for provider, models in models_tokens.items():
-            assert len(models) > 0, (
-                f"Provider {provider} should contain at least one model."
-            )
+            assert (
+                len(models) > 0
+            ), f"Provider {provider} should contain at least one model."
 
     def test_specific_model_token_values(self):
         """Test specific expected token values for selected models from various providers."""
         # Verify a token for a selected model from the 'openai' provider
         openai = models_tokens.get("openai")
-        assert openai.get("gpt-3.5-turbo-0125") == 16385, (
-            "Expected token limit for gpt-3.5-turbo-0125 in openai to be 16385"
-        )
+        assert (
+            openai.get("gpt-3.5-turbo-0125") == 16385
+        ), "Expected token limit for gpt-3.5-turbo-0125 in openai to be 16385"
 
         # Verify a token for a selected model from the 'azure_openai' provider
         azure = models_tokens.get("azure_openai")
-        assert azure.get("gpt-3.5") == 4096, (
-            "Expected token limit for gpt-3.5 in azure_openai to be 4096"
-        )
+        assert (
+            azure.get("gpt-3.5") == 4096
+        ), "Expected token limit for gpt-3.5 in azure_openai to be 4096"
 
         # Verify a token for a selected model from the 'anthropic' provider
         anthropic = models_tokens.get("anthropic")
-        assert anthropic.get("claude_instant") == 100000, (
-            "Expected token limit for claude_instant in anthropic to be 100000"
-        )
+        assert (
+            anthropic.get("claude_instant") == 100000
+        ), "Expected token limit for claude_instant in anthropic to be 100000"
 
     def test_providers_count(self):
         """Test that the total number of providers is as expected (at least 15)."""
-        assert len(models_tokens) >= 15, (
-            "Expected at least 15 providers in models_tokens"
-        )
+        assert (
+            len(models_tokens) >= 15
+        ), "Expected at least 15 providers in models_tokens"
 
     def test_non_existent_model(self):
         """Test that a non-existent model within a valid provider returns None."""
         openai = models_tokens.get("openai")
-        assert openai.get("non_existent_model") is None, (
-            "Non-existent model should return None from a valid provider."
-        )
+        assert (
+            openai.get("non_existent_model") is None
+        ), "Non-existent model should return None from a valid provider."
 
     def test_no_whitespace_in_model_names(self):
         """Test that model names do not contain leading or trailing whitespace."""
         for provider, model_dict in models_tokens.items():
             for model in model_dict.keys():
                 # Assert that stripping whitespace does not change the model name
-                assert model == model.strip(), (
-                    f"Model name '{model}' in provider '{provider}' contains leading or trailing whitespace."
-                )
+                assert (
+                    model == model.strip()
+                ), f"Model name '{model}' in provider '{provider}' contains leading or trailing whitespace."
 
     def test_specific_models_additional(self):
         """Test specific token values for additional models across various providers."""
         # Check some models in the 'ollama' provider
         ollama = models_tokens.get("ollama")
-        assert ollama.get("llama2") == 4096, (
-            "Expected token limit for 'llama2' in ollama to be 4096"
-        )
-        assert ollama.get("llama2:70b") == 4096, (
-            "Expected token limit for 'llama2:70b' in ollama to be 4096"
-        )
+        assert (
+            ollama.get("llama2") == 4096
+        ), "Expected token limit for 'llama2' in ollama to be 4096"
+        assert (
+            ollama.get("llama2:70b") == 4096
+        ), "Expected token limit for 'llama2:70b' in ollama to be 4096"
 
         # Check a specific model from the 'mistralai' provider
         mistralai = models_tokens.get("mistralai")
-        assert mistralai.get("open-codestral-mamba") == 256000, (
-            "Expected token limit for 'open-codestral-mamba' in mistralai to be 256000"
-        )
+        assert (
+            mistralai.get("open-codestral-mamba") == 256000
+        ), "Expected token limit for 'open-codestral-mamba' in mistralai to be 256000"
 
         # Check a specific model from the 'deepseek' provider
         deepseek = models_tokens.get("deepseek")
-        assert deepseek.get("deepseek-chat") == 28672, (
-            "Expected token limit for 'deepseek-chat' in deepseek to be 28672"
-        )
+        assert (
+            deepseek.get("deepseek-chat") == 28672
+        ), "Expected token limit for 'deepseek-chat' in deepseek to be 28672"
 
         # Check a model from the 'ernie' provider
         ernie = models_tokens.get("ernie")
-        assert ernie.get("ernie-bot") == 4096, (
-            "Expected token limit for 'ernie-bot' in ernie to be 4096"
-        )
+        assert (
+            ernie.get("ernie-bot") == 4096
+        ), "Expected token limit for 'ernie-bot' in ernie to be 4096"
 
     def test_nvidia_specific(self):
         """Test specific token value for 'meta/codellama-70b' in the nvidia provider."""
         nvidia = models_tokens.get("nvidia")
         assert nvidia is not None, "'nvidia' provider should exist"
         # Verify token for 'meta/codellama-70b' equals 16384 as defined in the nvidia dictionary
-        assert nvidia.get("meta/codellama-70b") == 16384, (
-            "Expected token limit for 'meta/codellama-70b' in nvidia to be 16384"
-        )
+        assert (
+            nvidia.get("meta/codellama-70b") == 16384
+        ), "Expected token limit for 'meta/codellama-70b' in nvidia to be 16384"
 
     def test_groq_specific(self):
         """Test specific token value for 'claude-3-haiku-20240307\'' in the groq provider."""
         groq = models_tokens.get("groq")
         assert groq is not None, "'groq' provider should exist"
         # Note: The model name has an embedded apostrophe at the end in its name.
-        assert groq.get("claude-3-haiku-20240307'") == 8192, (
-            "Expected token limit for 'claude-3-haiku-20240307\\'' in groq to be 8192"
-        )
+        assert (
+            groq.get("claude-3-haiku-20240307'") == 8192
+        ), "Expected token limit for 'claude-3-haiku-20240307\\'' in groq to be 8192"
 
     def test_togetherai_specific(self):
         """Test specific token value for 'meta-llama/Meta-Llama-3.1-70B-Instruct-Turbo' in the toghetherai provider."""
@@ -180,15 +211,15 @@ def test_togetherai_specific(self):
         assert togetherai is not None, "'toghetherai' provider should exist"
         expected = 128000
         model_name = "meta-llama/Meta-Llama-3.1-70B-Instruct-Turbo"
-        assert togetherai.get(model_name) == expected, (
-            f"Expected token limit for '{model_name}' in toghetherai to be {expected}"
-        )
+        assert (
+            togetherai.get(model_name) == expected
+        ), f"Expected token limit for '{model_name}' in toghetherai to be {expected}"
 
     def test_ernie_all_values(self):
         """Test that all models in the 'ernie' provider have token values exactly 4096."""
         ernie = models_tokens.get("ernie")
         assert ernie is not None, "'ernie' provider should exist"
         for model, token in ernie.items():
-            assert token == 4096, (
-                f"Expected token limit for '{model}' in ernie to be 4096, got {token}"
-            )
+            assert (
+                token == 4096
+            ), f"Expected token limit for '{model}' in ernie to be 4096, got {token}"

From 8769c3bddd7c865963cc7e245eefb496f55dc519 Mon Sep 17 00:00:00 2001
From: HKlabworks <hemaabhkhurana.1990@gmail.com>
Date: Mon, 31 Aug 2026 23:52:54 -0400
Subject: [PATCH 5/8] =?UTF-8?q?fix:=20=F0=9F=90=9B=20read=20SCRAPEGRAPHAI?=
 =?UTF-8?q?=5FTELEMETRY=5FENABLED=20from=20the=20environment,=20not=20the?=
 =?UTF-8?q?=20config=20file?=
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

`_check_config_and_environ_for_telemetry_flag` checked that the environment
variable existed and then read its value out of the config file:

    if os.environ.get("SCRAPEGRAPHAI_TELEMETRY_ENABLED") is not None:
        try:
            telemetry_enabled = config_obj.getboolean("DEFAULT", "telemetry_enabled")
        except Exception:
            pass

With no `telemetry_enabled` key in `~/.scrapegraphai.conf`, `getboolean` raises,
the bare `except` swallows it, and the flag keeps its default of `True`. So the
opt-out documented in the README leaves telemetry on for anyone who has not also
written a config file.

Now parses the variable's own value, reusing `configparser`'s BOOLEAN_STATES so the
environment variable and the config file accept the same spellings (true/false,
yes/no, on/off, 1/0). An unparseable value logs a warning and leaves the flag alone
rather than failing silently.

Adds tests/test_telemetry_flag.py covering the config path, the environment path,
precedence between them, and an unparseable value.

Verified with SCRAPEGRAPHAI_TELEMETRY_ENABLED=false and no config key:
  before: True   after: False
---
 scrapegraphai/telemetry/telemetry.py | 28 +++++++++--
 tests/test_telemetry_flag.py         | 70 ++++++++++++++++++++++++++++
 2 files changed, 93 insertions(+), 5 deletions(-)
 create mode 100644 tests/test_telemetry_flag.py

diff --git a/scrapegraphai/telemetry/telemetry.py b/scrapegraphai/telemetry/telemetry.py
index 07fa6e080..fba586b33 100644
--- a/scrapegraphai/telemetry/telemetry.py
+++ b/scrapegraphai/telemetry/telemetry.py
@@ -36,6 +36,19 @@ def _load_config(config_location: str) -> configparser.ConfigParser:
     return config
 
 
+def _parse_bool(value: str) -> bool:
+    """Parse a boolean from a string using configparser's accepted spellings.
+
+    Accepts the same values as the config file does, so
+    ``SCRAPEGRAPHAI_TELEMETRY_ENABLED=false`` and ``telemetry_enabled = false``
+    behave identically. Raises ValueError on anything unrecognised.
+    """
+    try:
+        return configparser.ConfigParser.BOOLEAN_STATES[value.strip().lower()]
+    except KeyError:
+        raise ValueError(f"invalid boolean value: {value!r}")
+
+
 def _check_config_and_environ_for_telemetry_flag(default_value: bool, config_obj):
     telemetry_enabled = default_value
     if "telemetry_enabled" in config_obj["DEFAULT"]:
@@ -44,13 +57,18 @@ def _check_config_and_environ_for_telemetry_flag(default_value: bool, config_obj
         except Exception:
             pass
 
-    if os.environ.get("SCRAPEGRAPHAI_TELEMETRY_ENABLED") is not None:
+    env_value = os.environ.get("SCRAPEGRAPHAI_TELEMETRY_ENABLED")
+    if env_value is not None:
         try:
-            telemetry_enabled = config_obj.getboolean(
-                "DEFAULT", "telemetry_enabled"
+            telemetry_enabled = _parse_bool(env_value)
+        except ValueError:
+            logger.warning(
+                "SCRAPEGRAPHAI_TELEMETRY_ENABLED is set to %r, which is not a "
+                "recognised boolean. Telemetry is left at %s. Use one of: "
+                "true/false, yes/no, on/off, 1/0.",
+                env_value,
+                telemetry_enabled,
             )
-        except Exception:
-            pass
 
     return telemetry_enabled
 
diff --git a/tests/test_telemetry_flag.py b/tests/test_telemetry_flag.py
new file mode 100644
index 000000000..c3743c1f4
--- /dev/null
+++ b/tests/test_telemetry_flag.py
@@ -0,0 +1,70 @@
+"""Tests for the telemetry opt-out flag.
+
+These cover the environment variable path, which previously read its value from
+the config file instead of from the variable, so `SCRAPEGRAPHAI_TELEMETRY_ENABLED=false`
+left telemetry enabled.
+"""
+
+import configparser
+
+import pytest
+
+from scrapegraphai.telemetry.telemetry import (
+    _check_config_and_environ_for_telemetry_flag,
+    _parse_bool,
+)
+
+
+def _config(**defaults):
+    cfg = configparser.ConfigParser()
+    cfg["DEFAULT"] = {k: str(v) for k, v in defaults.items()}
+    return cfg
+
+
+class TestParseBool:
+    @pytest.mark.parametrize("value", ["false", "False", "FALSE", "no", "off", "0", "  false  "])
+    def test_falsey_spellings(self, value):
+        assert _parse_bool(value) is False
+
+    @pytest.mark.parametrize("value", ["true", "True", "yes", "on", "1"])
+    def test_truthy_spellings(self, value):
+        assert _parse_bool(value) is True
+
+    def test_rejects_nonsense(self):
+        with pytest.raises(ValueError):
+            _parse_bool("maybe")
+
+
+class TestTelemetryFlag:
+    def test_defaults_to_the_given_default(self):
+        assert _check_config_and_environ_for_telemetry_flag(True, _config()) is True
+
+    def test_config_file_can_disable(self):
+        cfg = _config(telemetry_enabled="False")
+        assert _check_config_and_environ_for_telemetry_flag(True, cfg) is False
+
+    def test_env_var_disables_with_no_config_key(self, monkeypatch):
+        """The regression. Previously returned True, because the value was read
+        out of the config file rather than out of the environment variable."""
+        monkeypatch.setenv("SCRAPEGRAPHAI_TELEMETRY_ENABLED", "false")
+        assert _check_config_and_environ_for_telemetry_flag(True, _config()) is False
+
+    def test_env_var_overrides_the_config_file(self, monkeypatch):
+        monkeypatch.setenv("SCRAPEGRAPHAI_TELEMETRY_ENABLED", "false")
+        cfg = _config(telemetry_enabled="True")
+        assert _check_config_and_environ_for_telemetry_flag(True, cfg) is False
+
+    def test_env_var_can_also_enable(self, monkeypatch):
+        monkeypatch.setenv("SCRAPEGRAPHAI_TELEMETRY_ENABLED", "true")
+        cfg = _config(telemetry_enabled="False")
+        assert _check_config_and_environ_for_telemetry_flag(True, cfg) is True
+
+    def test_unparseable_env_var_leaves_the_flag_alone(self, monkeypatch):
+        monkeypatch.setenv("SCRAPEGRAPHAI_TELEMETRY_ENABLED", "banana")
+        cfg = _config(telemetry_enabled="False")
+        assert _check_config_and_environ_for_telemetry_flag(True, cfg) is False
+
+    def test_unset_env_var_leaves_the_config_in_charge(self, monkeypatch):
+        monkeypatch.delenv("SCRAPEGRAPHAI_TELEMETRY_ENABLED", raising=False)
+        cfg = _config(telemetry_enabled="False")
+        assert _check_config_and_environ_for_telemetry_flag(True, cfg) is False

From 3047ef8eda694d19c6fe4654777ea6343744acba Mon Sep 17 00:00:00 2001
From: semantic-release-bot <semantic-release-bot@martynus.net>
Date: Mon, 7 Sep 2026 13:38:25 +0000
Subject: [PATCH 6/8] ci(release): 2.2.0-beta.9 [skip ci]

## [2.2.0-beta.9](https://github.com/ScrapeGraphAI/Scrapegraph-ai/compare/v2.2.0-beta.8...v2.2.0-beta.9) (2026-09-07)

### Bug Fixes

* **models:** add Gemini 2.5 token limits so they are not truncated to 8192 ([c21af20](https://github.com/ScrapeGraphAI/Scrapegraph-ai/commit/c21af206862c13be1848eac75b4c04250718c8d9))
---
 CHANGELOG.md   | 7 +++++++
 pyproject.toml | 2 +-
 2 files changed, 8 insertions(+), 1 deletion(-)

diff --git a/CHANGELOG.md b/CHANGELOG.md
index 0f4785663..1aef7560f 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -1,3 +1,10 @@
+## [2.2.0-beta.9](https://github.com/ScrapeGraphAI/Scrapegraph-ai/compare/v2.2.0-beta.8...v2.2.0-beta.9) (2026-09-07)
+
+
+### Bug Fixes
+
+* **models:** add Gemini 2.5 token limits so they are not truncated to 8192 ([c21af20](https://github.com/ScrapeGraphAI/Scrapegraph-ai/commit/c21af206862c13be1848eac75b4c04250718c8d9))
+
 ## [2.2.0-beta.8](https://github.com/ScrapeGraphAI/Scrapegraph-ai/compare/v2.2.0-beta.7...v2.2.0-beta.8) (2026-08-23)
 
 
diff --git a/pyproject.toml b/pyproject.toml
index 76ea41644..c66f2ca3c 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -1,7 +1,7 @@
 [project]
 name = "scrapegraphai"
 
-version = "2.2.0b8"
+version = "2.2.0b9"
 
 description = "A web scraping library based on LangChain which uses LLM and direct graph logic to create scraping pipelines."
 authors = [

From 0bb8bc935028b4f0a91444db2866ec0142f97199 Mon Sep 17 00:00:00 2001
From: semantic-release-bot <semantic-release-bot@martynus.net>
Date: Mon, 7 Sep 2026 13:41:42 +0000
Subject: [PATCH 7/8] ci(release): 2.2.0-beta.10 [skip ci]
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

## [2.2.0-beta.10](https://github.com/ScrapeGraphAI/Scrapegraph-ai/compare/v2.2.0-beta.9...v2.2.0-beta.10) (2026-09-07)

### Bug Fixes

* 🐛 read SCRAPEGRAPHAI_TELEMETRY_ENABLED from the environment, not the config file ([8769c3b](https://github.com/ScrapeGraphAI/Scrapegraph-ai/commit/8769c3bddd7c865963cc7e245eefb496f55dc519))
---
 CHANGELOG.md   | 7 +++++++
 pyproject.toml | 2 +-
 2 files changed, 8 insertions(+), 1 deletion(-)

diff --git a/CHANGELOG.md b/CHANGELOG.md
index 1aef7560f..592f64c89 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -1,3 +1,10 @@
+## [2.2.0-beta.10](https://github.com/ScrapeGraphAI/Scrapegraph-ai/compare/v2.2.0-beta.9...v2.2.0-beta.10) (2026-09-07)
+
+
+### Bug Fixes
+
+* 🐛 read SCRAPEGRAPHAI_TELEMETRY_ENABLED from the environment, not the config file ([8769c3b](https://github.com/ScrapeGraphAI/Scrapegraph-ai/commit/8769c3bddd7c865963cc7e245eefb496f55dc519))
+
 ## [2.2.0-beta.9](https://github.com/ScrapeGraphAI/Scrapegraph-ai/compare/v2.2.0-beta.8...v2.2.0-beta.9) (2026-09-07)
 
 
diff --git a/pyproject.toml b/pyproject.toml
index c66f2ca3c..2c9c99dd8 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -1,7 +1,7 @@
 [project]
 name = "scrapegraphai"
 
-version = "2.2.0b9"
+version = "2.2.0b10"
 
 description = "A web scraping library based on LangChain which uses LLM and direct graph logic to create scraping pipelines."
 authors = [

From 8b3a97c3b41aec29df0512e71f186a98ad747aa1 Mon Sep 17 00:00:00 2001
From: semantic-release-bot <semantic-release-bot@martynus.net>
Date: Mon, 7 Sep 2026 13:46:32 +0000
Subject: [PATCH 8/8] ci(release): 2.2.4-beta.1 [skip ci]
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

## [2.2.4-beta.1](https://github.com/ScrapeGraphAI/Scrapegraph-ai/compare/v2.2.3...v2.2.4-beta.1) (2026-09-07)

### Bug Fixes

* 🐛 read SCRAPEGRAPHAI_TELEMETRY_ENABLED from the environment, not the config file ([8769c3b](https://github.com/ScrapeGraphAI/Scrapegraph-ai/commit/8769c3bddd7c865963cc7e245eefb496f55dc519))
* **models:** add Gemini 2.5 token limits so they are not truncated to 8192 ([c21af20](https://github.com/ScrapeGraphAI/Scrapegraph-ai/commit/c21af206862c13be1848eac75b4c04250718c8d9))
* **fetch:** surface HTTP errors and missing content instead of answering NA ([f91478e](https://github.com/ScrapeGraphAI/Scrapegraph-ai/commit/f91478eacf86485f6b9efcf843fc0c815dde1ec5)), closes [#1102](https://github.com/ScrapeGraphAI/Scrapegraph-ai/issues/1102) [#1102](https://github.com/ScrapeGraphAI/Scrapegraph-ai/issues/1102)

### CI

* **release:** 2.2.0-beta.10 [skip ci] ([0bb8bc9](https://github.com/ScrapeGraphAI/Scrapegraph-ai/commit/0bb8bc935028b4f0a91444db2866ec0142f97199))
* **release:** 2.2.0-beta.7 [skip ci] ([decfc6b](https://github.com/ScrapeGraphAI/Scrapegraph-ai/commit/decfc6bb6eb10a29ed6aaabb07244b8915042604))
* **release:** 2.2.0-beta.8 [skip ci] ([d59c3df](https://github.com/ScrapeGraphAI/Scrapegraph-ai/commit/d59c3dfceecdacbba4e17f237b017117cf7f1cee)), closes [#1102](https://github.com/ScrapeGraphAI/Scrapegraph-ai/issues/1102) [#1102](https://github.com/ScrapeGraphAI/Scrapegraph-ai/issues/1102)
* **release:** 2.2.0-beta.9 [skip ci] ([3047ef8](https://github.com/ScrapeGraphAI/Scrapegraph-ai/commit/3047ef8eda694d19c6fe4654777ea6343744acba))
---
 CHANGELOG.md   | 17 +++++++++++++++++
 pyproject.toml |  2 +-
 2 files changed, 18 insertions(+), 1 deletion(-)

diff --git a/CHANGELOG.md b/CHANGELOG.md
index eb0c8126e..21c0717c5 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -1,3 +1,20 @@
+## [2.2.4-beta.1](https://github.com/ScrapeGraphAI/Scrapegraph-ai/compare/v2.2.3...v2.2.4-beta.1) (2026-09-07)
+
+
+### Bug Fixes
+
+* 🐛 read SCRAPEGRAPHAI_TELEMETRY_ENABLED from the environment, not the config file ([8769c3b](https://github.com/ScrapeGraphAI/Scrapegraph-ai/commit/8769c3bddd7c865963cc7e245eefb496f55dc519))
+* **models:** add Gemini 2.5 token limits so they are not truncated to 8192 ([c21af20](https://github.com/ScrapeGraphAI/Scrapegraph-ai/commit/c21af206862c13be1848eac75b4c04250718c8d9))
+* **fetch:** surface HTTP errors and missing content instead of answering NA ([f91478e](https://github.com/ScrapeGraphAI/Scrapegraph-ai/commit/f91478eacf86485f6b9efcf843fc0c815dde1ec5)), closes [#1102](https://github.com/ScrapeGraphAI/Scrapegraph-ai/issues/1102) [#1102](https://github.com/ScrapeGraphAI/Scrapegraph-ai/issues/1102)
+
+
+### CI
+
+* **release:** 2.2.0-beta.10 [skip ci] ([0bb8bc9](https://github.com/ScrapeGraphAI/Scrapegraph-ai/commit/0bb8bc935028b4f0a91444db2866ec0142f97199))
+* **release:** 2.2.0-beta.7 [skip ci] ([decfc6b](https://github.com/ScrapeGraphAI/Scrapegraph-ai/commit/decfc6bb6eb10a29ed6aaabb07244b8915042604))
+* **release:** 2.2.0-beta.8 [skip ci] ([d59c3df](https://github.com/ScrapeGraphAI/Scrapegraph-ai/commit/d59c3dfceecdacbba4e17f237b017117cf7f1cee)), closes [#1102](https://github.com/ScrapeGraphAI/Scrapegraph-ai/issues/1102) [#1102](https://github.com/ScrapeGraphAI/Scrapegraph-ai/issues/1102)
+* **release:** 2.2.0-beta.9 [skip ci] ([3047ef8](https://github.com/ScrapeGraphAI/Scrapegraph-ai/commit/3047ef8eda694d19c6fe4654777ea6343744acba))
+
 ## [2.2.3](https://github.com/ScrapeGraphAI/Scrapegraph-ai/compare/v2.2.2...v2.2.3) (2026-09-07)
 
 
diff --git a/pyproject.toml b/pyproject.toml
index 2c9c99dd8..1348b0788 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -1,7 +1,7 @@
 [project]
 name = "scrapegraphai"
 
-version = "2.2.0b10"
+version = "2.2.4b1"
 
 description = "A web scraping library based on LangChain which uses LLM and direct graph logic to create scraping pipelines."
 authors = [