Repository navigation
Conversation
…XT_BUFFER, READONLY_BUFFER) Fickling's OPCODES_BY_NAME registry covered 64 of the 68 standard pickle opcodes. The four missing ones raise NotImplementedError from Opcode.__new__ at load time — before any analysis runs — so Pickled.load() crashes with a traceback instead of producing a verdict on perfectly benign pickles: any float literal in a legacy protocol-0 file (FLOAT), any bytearray at protocol 5 (BYTEARRAY8), and out-of-band buffers (NEXT_BUFFER/READONLY_BUFFER). Implement the four opcodes mirroring the existing families: - Float(ConstantOpcode): proto-0 newline-terminated literal, like Int/Long - ByteArray8(BinBytes8): 8-byte length prefix pushing a bytearray constant - NextBuffer(Opcode): pushes an opaque out_of_band_buffer name — the buffer contents live outside the stream, nothing to inspect - ReadOnlyBuffer(Opcode): marks the top of stack read-only; a documented no-op for the AST
|
Noting an overlap I should have caught when opening this, so it doesn't cost the maintainers a discovery exercise later. #320 (opened 2026-08-21, earlier than this PR) also implements
Two consequences if both were merged as they stand, from the current code rather than opinion:
So: I'm happy to narrow this PR to the three opcodes #320 does not touch — For orientation: this PR's regression coverage is the four cases in |
|
CI is green here, including the optional RE2 mixed-engine job and the 3.11/3.12/3.13 matrix. The scope is four opcode classes in Is there anything you would like adjusted — naming, the proto-0 float edge cases, or splitting the buffer opcodes into a separate PR — before this can go in? |
|
Hi @trailofbits-fickling maintainers — CI is green (2 checks) and the branch is still rebased on Recap: One question rather than a bump: |
Root cause
Fickling's
OPCODES_BY_NAMEregistry covered 64 of the 68 standard pickle opcodes. The four missing ones raiseNotImplementedErrorfromOpcode.__new__at load time — before any analysis runs — soPickled.load()crashes with a traceback instead of producing a verdict on perfectly benign pickles:FLOATBYTEARRAY8bytearray(pickle.dumps(bytearray(...))uses it by default)NEXT_BUFFER/READONLY_BUFFERbuffer_callbackreturning a relocated buffer)Reproduced pre-fix with benign inputs:
pickle.dumps(1.5, protocol=0)andpickle.dumps(bytearray(b"hello"), protocol=5)both crash atPickled.load(). A security scanner that crashes on certain inputs is itself a problem: in embedding contexts (thefickling.import_hook/ scan-API paths) the crash either aborts the host or is swallowed by a caller that then skips the file — analysis is denied exactly where it matters.Fix
Four opcode subclasses mirroring the existing families:
Float(ConstantOpcode)— proto-0 newline-terminated literal, likeInt/LongByteArray8(BinBytes8)— 8-byte length prefix, pushing abytearrayconstant, likeBinBytes8does forbytesNextBuffer(Opcode)— pushes an opaqueout_of_band_buffername (the buffer contents live outside the stream, so there is nothing to inspect)ReadOnlyBuffer(Opcode)— marks the top of stack read-only; a documented no-op for the ASTValidation
pytest test/test_pickle.py| sample size:N=31| key metrics: 4 new regression tests fail pre-fix (NotImplementedError: TODO: Add support for Opcode BYTEARRAY8/FLOATraised at load) → post-fix31 passed| result:passpytest test/→149 passedruff@0.16.0 check+format --checkon both touched files: cleanPickled.encode()of the new opcodes re-loads to equal values viapickle.loadsRisk / Compatibility