Skip to content

jq: #1643's delimiter check lives only in the CLI printer — .[]/length/keys/library API still accept a missing delimiter #1677

Description

@newhoggy

Summary

#1643's delimiter check (preceding_gap_ok, added to print_json's object/array arms in
jq_runner.rs) lives entirely in the CLI's own printer. It was never added to the actual
jq/eval_generic evaluator that walks objects/arrays to answer .[], length, keys,
add, to_entries, or a plain field lookup that doesn't reach a leaf. Any filter whose
result doesn't require re-serializing the malformed container itself as JSON output never
passes through print_json's object/array arms at all, so it never triggers the check —
even on the ordinary sjq -c . path, no flag required.

#1643's own commit message called this "the same class of which path validates gap #1611
already tracks for a different flag-driven divergence" — but #1611 is specifically about
-a/-S/-C/-s swapping the entire input strategy to a different materializer
(parse_json_streamjson_bytes_to_owned_value, now itself covered by a #1643 follow-up
fix). This issue is a distinct mechanism: it reproduces on the default path, with no
flags at all
, for filters that simply never call print_json's object/array arms on the
malformed container in the first place. It also isn't #1629 (keys_unsorted's positional
fast paths still answering from a malformed object) — that one is about #1194's non-string-
key class; this one is about #1643's delimiter class, and reproduces on plain keys, not
just keys_unsorted.

Repro

Verified against the /usr/bin/jq 1.7.1 pin; every succinctly row is exit 0 with a
plausible-looking answer, every jq row is a parse error at exit 5:

$ echo '{"a" 1, "b": 2}' | sjq -c 'keys'         # ["a","b"]
$ echo '{"a" 1, "b": 2}' | sjq -c 'length'       # 2
$ echo '{"a" 1,"b":2}'   | sjq -c 'to_entries'   # [{"key":"a","value":1},{"key":"b","value":2}]
$ echo '[1 2, 3]'        | sjq -c '.[]'          # 1␊2␊3
$ echo '[1 2, 3]'        | sjq -c 'add'          # 6
$ echo '{"a" 1}'         | sjq -c '.a'           # 1

$ echo '{"a" 1, "b": 2}' | jq -c 'keys'          # parse error, exit 5  (every row above)

sjq -c . (identity, the case #1643's own tests cover) on every one of these inputs already
correctly raises — only these non-reserializing filters see through to the malformed data.

Root cause

The check was added at the wrong layer for full coverage: jq_runner.rs's print_json is a
CLI-only formatter, not part of succinctly::jq::eval/eval_generic's own object/array walk.
#1194's own fix (the non-string-key / odd-child-count class) did land in the evaluator
layer — to_owned_at_depth in eval_generic.rs returns Err directly from its own field
walk — which is why that class is caught uniformly everywhere, including keys/length/
.[]/library use. #1643's delimiter class never got the equivalent placement, so it inherited
none of that uniform coverage.

This also means succinctly::jq::eval/eval_generic used as a library API (not through
the CLI at all) has no delimiter validation whatsoever — json_bytes_to_owned_value's own doc
comment already says plainly that it "performs no validation of its own" and expects the
caller to pre-validate, but nothing in the evaluator's own walk enforces that for a Cursor-
backed value reached via ordinary filter evaluation.

Suggested direction

Not attempted here. Move (or duplicate) the ,/: gap check into eval_generic.rs's object/
array walk (to_owned_at_depth and the lazy eval_with_cursor paths), alongside the existing
#1194 checks (field.key_str(), ends_unpaired()) rather than only in the CLI printer —
mirrors exactly how #1194's fix achieved uniform coverage. Would need the same
known_text_pos-style care #1643 took to avoid a double text_position() lookup on the
already-hot evaluator paths (--slurp/-S are not those; the default lazy path is).

Scope

JSON only, same as #1643. Not blocked on the sibling trailing-comma-gap issue filed alongside
this one — the two are independent limitations of the same underlying check.

Refs #1643, #1611, #1629.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions