Skip to content

fix(polyglot): bound recursive archive extraction and unify error handling - #325

Open
feiiiiii5 wants to merge 1 commit into
trailofbits:masterfrom
feiiiiii5:fix/polyglot-extraction-limits
Open

feiiiiii5 wants to merge 1 commit into
trailofbits:masterfrom
feiiiiii5:fix/polyglot-extraction-limits

Conversation

@feiiiiii5

Copy link
Copy Markdown
Contributor

Root cause

find_file_properties_recursively exists to inspect untrusted archives, but its zip and tar paths extracted members to disk with no size limit and no error handling: zipped_file.read(fname) streams the full decompressed output, so a compressed-expansion bomb (tiny archive, huge inflated member) exhausts storage during scanning. The 7z path already handled both concerns (BytesIOFactory(limit=...) + except (OSError, ArchiveError) graceful degradation) — the zip and tar paths matched neither.

A second defect in the same loop: entries written by zipfile.writestr carry permission bits without file-type bits (external_attr = 0o600 << 16, so mode == 0 is false and stat.S_ISREG(mode) is false) — those members were silently never scanned.

Also removes a stray debug print("check_if_legacy_format") that fired on every tar-file format check through identify_pytorch_file_format → PyTorchModelWrapper.validate_file_format.

Fix

  • Zip/tar member extraction streams in chunks up to MAX_MEMBER_EXTRACTION_SIZE, raising ResourceExhaustionError past the cap — declared archive sizes lie, so the bound is enforced while streaming rather than read from metadata; this aligns the polyglot scanner with the expansion-attack protection the analysis path already has
  • Corrupt members degrade gracefully (BadZipFile/TarError/OSError → member skipped), matching the 7z path
  • Zip member filter keys off the file-type bits (S_IFMT), so permission-only entries (writestr style) are scanned
  • Stray debug print removed

Validation

  • pytest test/test_polyglot.py | sample size: N=21 | key metrics: 4 new tests fail pre-fix (extraction uncapped on a 4 MiB-of-zeros member with a 1 KiB cap; stray print present; writestr members skipped) → post-fix 21 passed | result: pass
  • Full suite pytest test/ → 153 passed
  • ruff@0.16.0 check + format --check: clean

Risk / Compatibility

  • Well-formed archives behave identically (the cap default is 1 GiB per member); hostile archives now fail loudly with ResourceExhaustionError instead of filling the disk, and corrupt members degrade instead of crashing — both align zip/tar with the 7z path's existing contract.

…dling

find_file_properties_recursively extracts archive members to temporary
files with no size limit: a compressed-expansion bomb (tiny zip, huge
decompressed output) streams to disk without bound, exhausting storage
during scanning. The 7z path already degrades gracefully with a
BytesIOFactory size limit; the zip and tar paths had neither the bound
nor the error handling.

- Stream zip/tar member extraction in chunks up to
  MAX_MEMBER_EXTRACTION_SIZE, raising ResourceExhaustionError past the
  cap (declared archive sizes lie, so the bound is enforced while
  streaming, not from metadata)
- Degrade gracefully on corrupt members (BadZipFile/TarError), matching
  the 7z path, instead of crashing the scan
- Only skip zip entries whose type bits name a non-regular type: entries
  written by zipfile.writestr carry permission bits without file-type
  bits (mode 0o600, S_ISREG false) and were previously never scanned
- Remove a stray debug print from check_if_legacy_format
@feiiiiii5
feiiiiii5 requested a review from ESultanik as a code owner September 5, 2026 05:53
@feiiiiii5

Copy link
Copy Markdown
Contributor Author

Hi @thomas-chauchefoin-tob, the #325 branch is still focused on bounded archive extraction and consistent error handling, with no changes since the last update. Is there a remaining concern you’d like addressed before review, or should I leave the current scope unchanged?

@feiiiiii5

Copy link
Copy Markdown
Contributor Author

Second bump on this one, sorry — the only review activity so far has been my own first ping on 24 September.

The scope has not changed since then, so rather than repeat it: the two things I would most appreciate a maintainer ruling on are

  1. the extraction limit — the 7z path already used BytesIOFactory(limit=...), and I matched that shape for zip and tar. If the project wants a different ceiling, or wants it configurable rather than fixed, that is a one-line change on my side and I would rather match your convention than guess at a number.
  2. the permission-bits hunk. Entries written by zipfile.writestr with external_attr = 0o600 << 16 have no file-type bits, so stat.S_ISREG(mode) is false and those members were silently never scanned. I fixed the test rather than the reader. If you would rather the reader treat "no file-type bits but readable" as a regular file, say so — that is arguably the more correct fix and it is not much larger.

If neither is a concern, I would also rather you close it than leave it hanging — happy to hand the bounded-extraction fix over as-is if the permission part is contentious, or vice versa.

Either way, thanks for the time.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant