Skip to content

linux-x86_64 target: MySQL result rows silently lose every column after the 8th — full-app context only; Node, macOS-arm64, and every minimal linux binary decode correctly #6462

Description

@proggeramlug

Symptom

In a production fastify + @perryts/mysql service cross-compiled for x86_64-unknown-linux-gnu, every result row of a 16-column SELECT comes back with only its first 8 properties populated; columns 9–16 are undefined. JSON.stringify then drops the keys, so API responses are silently missing fields — no error, no warning at any stage. In our app this emptied an approval queue (a UI filtered on a now-undefined field) and broke its approve action (String(undefined) routing).

  • Deterministic: every request, every row, 100% of the time. Not a race, not flaky.
  • Both protocols: text (query without params) and prepared (query with params) truncate identically.
  • Boundary is exactly after column 8 of the SELECT list, independent of value sizes: a synthetic row whose largest value is 2 bytes truncates at the same place as rows with 4 KB JSON columns.
  • Column list where it manifests: CAST(id AS CHAR), CAST(execution_id AS CHAR), varchar, varchar, text, text, json, json, | varchar, json, json, varchar, varchar, datetime, varchar, datetime (the | marks the cut).

Environment

  • perry 0.5.1258 — main @ 889f39eda plus the (unrelated to this issue) PR fix(hir): cross-module Fastify factory — app.listen() compiles, resolves, and silently binds nothing #6436 patch; runtime/stdlib/ext libs built from the same tree.
  • Cross-compiled on macOS arm64 (messense x86_64-unknown-linux-gnu toolchain, Debian-multiarch sysroot), PERRY_LIB_DIR → same-source target/x86_64-unknown-linux-gnu/release.
  • Target box: Ubuntu 26.04 x86_64, MySQL 8.4 on 127.0.0.1.
  • Driver: @perryts/mysql (pinned GitHub tarball), via createPool.
  • Likely a regression after 0.5.1129: the same app's approval flow (same table, same 16-column SELECT) demonstrably worked on a 0.5.1129-era binary on 2026-07-06 and 2026-07-15.

Evidence matrix — why we believe this is context-sensitive codegen

context (all linux binaries built with the identical toolchain/libs) result
Node 26 (tsx), same driver, same SELECT, same prod DB over SSH tunnel 16/16 ✓
macOS-arm64 native build of the full app (its e2e asserts column 9) 16/16 ✓
linux binary, isolated: 12-key object via dynamic obj[k]=v loop / literal / JSON.parse 12/12 ✓
linux binary, small: real @perryts/mysql + the real 16-col SELECT + the real prod table 16/16 ✓
linux binary, small: driver plus fastify linked (instance constructed, routes registered — triggers the well-known shared-tokio ext rebuild) + generic 16×VARCHAR table, both protocols 16/16 ✓
linux binary, full app (~30 kLoC: fastify server + mysql pool + worker/scheduler loops + agent framework) 8/16 ✗ — every row, every request

So: same driver TypeScript, same SQL, same database, same toolchain — correct in every reduced context, wrong only when compiled inside the large application. This matches the pattern of #5432 (Response.headers.forEach SIGSEGV that manifested only inside a full app and not in an isolated binary): codegen appears to specialize differently depending on the surrounding compilation context, and one specialization of the row-materialization path is wrong on the x86_64 target only (the arm64-macOS build of the same app is correct).

Ruled out: value length (2-byte row truncates identically), protocol (both affected), driver logic (Node correct), a general object-property/slot limit (isolated 12-key dynamic objects fine), table/data shape (real table correct in a small binary), pool concurrency alone (the full app on Node runs the same concurrent worker loops and is correct).

Workaround (in case it helps localize)

Selecting the row as one columnSELECT JSON_OBJECT('k1', c1, …) AS r — decodes correctly even in the affected binary, i.e. the JSON-parse object-construction path is fine where the driver's per-column row assembly is not. Every ≤8-column SELECT in the same binary is also correct.

Repro status

I could not reduce it below the full application (each reduction above was an attempt). The failing binary is 31 MB with debug_info, reproduces on every run, and the box has clang/gdb available. Happy to: share the exact ELF binary, run gdb sessions / patched builds against it, share the app source privately, or bisect compiler versions between 0.5.1129 and 0.5.1258 on request.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions