Implement zero-dependency Wasm binary parser

[bbyalcinkaya]

This PR adds a new zero-dependency binary parser in pykwasm that parses Wasm bytecode and produces KAst terms of wasm-semantics. The implementation closely follows the WebAssembly 3.0 binary format specification.

Previously, the parser relied on a py-wasm fork, as described in issue #524. That dependency has been unmaintained since 2019 and blocked support for WASM 2.0+ opcodes.

Changes

• Implement the Wasm binary parser
• Enable all binary parsing integration tests for the new parser
• Add unit tests for individual parsing components
• Removed the unused BlockMetaData field, which was previously used for coverage tracking
• Implement differential testing to compare the old py-wasm-based parser with the new parser

Spec Divergences in the Binary Parser

The parser follows the current WebAssembly spec (WASM 3.0) for decoding, but the K semantics predates several additions. In the places below, bytes are decoded correctly but information is discarded because the semantics has no representation for it.

Blocktype type-index form (instructions.py:47) — A blocktype can be a type index encoded as i33. Only the empty and single-valtype cases are handled; the i33 case raises a parse error. The K semantics block types cannot reference the type section.

Typed select (instructions.py:83) — The typed select variant (0x1C) carries a value-type vector. It is parsed and discarded; the untyped SELECT node is emitted. The K semantics has a single untyped select.

Memory load/store memarg (instructions.py:170) — memarg encodes an alignment hint and an optional memory index alongside the offset. Only the offset is forwarded to the AST; alignment and memory index are discarded. Affects all 23 load/store opcodes. The K semantics models a single memory with offset-only instructions.

memory.size / memory.grow memory index (instructions.py:241) — Both instructions encode a memory index with the multi-memory proposal. It is parsed and discarded; the K semantics has a single implicit memory.

Recursive types (module.py:78) — The spec wraps all type definitions in rectype groups that can be mutually recursive. We only accept singleton groups; actual recursion raises a parse error. The K semantics has no recursive type construct.

Table inline initializer (module.py:119) — The spec allows an optional inline initializer for tables (prefix 0x40 0x00). Encountering it raises a parse error. No counterpart exists in the K semantics.

Address type in limits (types.py:117) — Limit encodings 0x02/0x03 signal 64-bit addressing. The address type is parsed but discarded by every caller (tablesec, memsec, externtype_as_import_desc). The K semantics assumes 32-bit addressing throughout.

Import section externtype (types.py:174) — Import descriptors are encoded as WASM 3.0 externtype but returned as WASM 1.0 ImportDesc nodes. The tag type variant (0x04) raises a parse error. The K ImportDefn sort still matches the WASM 1.0 structure.

Only two reference types are supported: funcref and externref (types.py:90,197) — The GC proposal added further reference types (any, eq, i31, struct, array, none, noextern, nofunc, exn, noexn) and a general 0x63/0x64 encoding for (ref null? <heap type>). None of these are recognized; only funcref (0x70) and
externref (0x6F) parse. v128 (SIMD) is likewise unrecognized wherever a value type is expected. The K semantics predates the GC and SIMD proposals and has no construct for any of these types.

[ehildenb]

Wasm 3.0 compliance review — binary parser (PR #754)

Review of the in-repo binary parser in pykwasm/src/pykwasm/binary/ against the WebAssembly 3.0 binary-format specification.

The PR body declares a set of intentional divergences (typed select, blocktype i33, memarg alignment/memidx discard, recursive types, table inline initializer, address-type discard, tag imports).
Those all check out and are excluded here.
The findings below are bugs or gaps beyond what the PR body declares.

Summary

#	Severity	Location	Issue
1	High	binary/types.py:118	limits address-type flag bytes are wrong (0x02/0x03 vs spec 0x04/0x05)
2	High	binary/module.py:44	Custom sections are not skipped — stream is left misaligned
3	Medium	binary/instructions.py:146	0xFC sub-opcodes 0–11 missing (trunc_sat, memory.init/copy/fill, data.drop)
4	Low	binary/types.py:90,197	reftype/heaptype only decode the 2.0 subset (undeclared)
5	Low	binary/floats.py:12	NaN bit patterns may not round-trip

Verified correct while reviewing: signed/unsigned LEB128 range logic, tabletype field order (reftype-then-limits), call_indirect operand order, elem/data segment variant tables, section ordering (datacount before code), and sized exact-length enforcement.

Finding 1 — limits address-type flag bytes (High)

Location: binary/types.py:118-135

The Wasm 3.0 binary grammar (memory64) encodes the address type in the limits flag byte as:

limits ::= 0x00 n:u64        ⇒ (i32, [n .. ε])
         | 0x01 n:u64 m:u64  ⇒ (i32, [n .. m])
         | 0x04 n:u64        ⇒ (i64, [n .. ε])
         | 0x05 n:u64 m:u64  ⇒ (i64, [n .. m])

The parser uses 0x02/0x03 for the i64 (64-bit) cases and never handles 0x04/0x05.
Consequences:

• A valid memory64/table64 module (flag 0x04/0x05) raises Invalid limit descriptor — a false rejection of spec-compliant input.
• 0x02/0x03 are not i64 in the core spec (they're the shared-memory/threads encodings); accepting them as i64 mis-decodes.

Since the address type is discarded downstream anyway, the fix is just to relabel the flag values: 0x00/0x01 → i32, 0x04/0x05 → i64 (and reject 0x02/0x03 unless shared memory is intended to be accepted-and-discarded).

Finding 2 — custom sections not consumed (High)

Location: binary/module.py:44-63

def parse_custom_section(s):
    try:
        n = peek_byte(s)
    except WasmEOFError:
        return b''
    if n != 0:
        return b''
    skip(1, s)
    return read_bytes(n, s)   # n is 0 here!

When a custom section is present, n == 0 (the section id).
The code skips the 1 id byte, then calls read_bytes(0, s), reading nothing — it never reads the section's size:u32 or its payload.
It returns b'' (falsy), so parse_custom_sections immediately breaks, leaving the stream positioned right after the id byte, before the size.

Every subsequent section(...) call then peeks the wrong byte, doesn't match its expected id, and silently returns its default — so any module containing a custom section anywhere (name section, producers section, DWARF/debug info, etc.) drops everything after that section.
Test modules produced by wat2wasm without --debug-names have no custom sections, which is likely why differential testing passes.

A custom section is 0x00 size:u32 name:name bytes*.
The fix is to read the size after skipping the id and consume that many bytes:

skip(1, s)
size = u32(s)
return read_bytes(size, s)

Finding 3 — missing 0xFC opcodes (Medium)

Location: binary/instructions.py:146-165

The 0xFC handler only covers sub-opcodes 12–17 (table.*).
Missing sub-opcodes:

• 0–7: i32/i64.trunc_sat_{s,u}_{f32,f64}
• 8: memory.init, 9: data.drop, 10: memory.copy, 11: memory.fill

These fall through to case _ → Unsupported opcode.
Notably, memory.copy and memory.fill are already implemented in the semantics (aCopy/aFill, see wasm.md) and reachable via the text frontend, so the binary parser can't reach a capability the semantics already has.
Given the PR's stated motivation (unblocking Wasm 2.0+ opcodes), these core 2.0 instructions seem in scope.

When adding them, remember the trailing immediates must be consumed: memory.fill is 0xFC 11 0x00, memory.copy is 0xFC 10 0x00 0x00, memory.init is 0xFC 8 x:dataidx 0x00, data.drop is 0xFC 9 x:dataidx.

If trunc_sat / memory.init / data.drop are intentionally out of scope, they belong in the "Spec Divergences" list in the PR body alongside the others.

Finding 4 — reftype/heaptype subset (Low)

Location: binary/types.py:90-97, 197-205

reftype accepts only 0x70 (funcref) / 0x6F (externref) and heaptype only 0x70/0x6F.
This is a reasonable scope limit for a pre-GC semantics, but the 3.0 grammar also has the 0x63 ht/0x64 ht prefixed (ref null? ht) forms and the other abstract heap types (any, eq, i31, struct, array, none, noextern, nofunc, exn, noexn).
Anything using those (e.g. ref.null any) raises.
Same story for vectype/v128 in valtype (line 73, # TODO implement vectypes).
Suggest adding these to the PR's "Spec Divergences" list so the scope is explicit.

Finding 5 — NaN payloads (Low)

Location: binary/floats.py:12-21

struct.unpack('<f'/'<d', ...) yields a Python float, which collapses distinct NaN encodings (sign bit, canonical vs. arithmetic, payload bits).
If the downstream K f32.const/f64.const representation doesn't preserve the raw bits, const instructions with non-canonical NaNs won't match the spec's NaN semantics.
Worth a quick check that a f64.const NaN payload survives parse → K → execution; may be pre-existing behavior inherited from the old parser.

Minor note — memarg flag bit

Location: binary/instructions.py:51-61

memarg detects the memidx flag with a range check (n < 2**6 / elif n < 2**7) rather than testing bit 6 (n & 0x40) and masking the alignment.
For all valid modules the alignment exponent is tiny, so this never triggers a spurious error — noted only for completeness.

[bbyalcinkaya]

Wasm 3.0 compliance review — binary parser (PR #754)

Review of the in-repo binary parser in pykwasm/src/pykwasm/binary/ against the WebAssembly 3.0 binary-format specification.

...

Went through each finding:

• 1 (limits address-type flags) — fixed in 70ae762, with regression tests. Relabeled to 0x04/0x05 for i64 per spec; 0x02/0x03 (shared memory) now correctly rejected instead of misdecoded.
• 2 (custom sections not consumed) — fixed in cab1047, with a regression test. Was reading 0 bytes instead of the section's actual size, misaligning everything after it.
• 3 (missing 0xFC opcodes) — memory.copy/memory.fill now implemented in 072a674, with tests. trunc_sat/memory.init/data.drop remain unimplemented since they have no backing K construct at all.
• 4 (reftype/heaptype subset) — no change. Only funcref/externref supported in the semantics; GC ref types and SIMD v128 unsupported since the K semantics predates both proposals.
• 5 (NaN payloads) — no change. Pre-existing K semantics limitation (not introduced by this parser), treating as an accepted corner case.
• memarg flag-bit note — no change. Checked the actual spec grammar and the existing range-check code the definition in the spec, which doesn't use bitwise comparison.

[bbyalcinkaya]

Fixed after self review:

• No end-of-stream check in parse_module: trailing/garbage bytes, or any unrecognized section, were silently ignored rather than rejected. parse_module now raises WasmParseError if any bytes remain after the last section (binary/module.py), with regression tests in TestTrailingData (test_binary_parser.py).
• Dead code in binary/combinators.py: iterate was defined but never called anywhere in the parser.
• parse_module raised a bare Python ValueError (from zip(..., strict=True)) on func/code section length mismatch instead of the parser's own WasmParseError. Now checked explicitly before the zip, with regression tests in TestFuncCodeLengthMismatch (test_binary_parser.py).