Skip to content

Let a caller decide, per element, whether to read the value at all - #86

Merged
xbmlz merged 1 commit into
mainfrom
read-onelement-hook
Sep 7, 2026
Merged

xbmlz merged 1 commit into
mainfrom
read-onelement-hook

Conversation

@xbmlz

@xbmlz xbmlz commented Sep 7, 2026

Copy link
Copy Markdown
Contributor

Closes #72 — streaming as a first-class capability (#51 §12).

What a caller could not do before

Indexing a study means reading a handful of tags out of every file and
ignoring the rest. godicom had two ways to narrow a read, and neither has
that shape:

  • SpecificTags names the tags to keep, but it filtered at every depth —
    naming a sequence kept the sequence element and emptied its items.
  • StopBeforePixels stops at one hardcoded tag.

Neither can say walk this file, hand me the headers, let me decide.

What this adds

ds, err := godicom.ReadFile("ct.dcm", &godicom.ReadOptions{
	OnElement: func(h godicom.RawDataElement, path []godicom.PathStep) godicom.ElementAction {
		if len(path) > 0 || want[h.Tag] {
			return godicom.ElementKeep
		}
		return godicom.ElementSkip
	},
})

The hook is called once per element, after the header is decoded and before
the value is read. h carries the Tag, the VR in effect, the declared Length
and the ValueTell the value would start at — but not the value, not
having read it being the point. path names the enclosing sequences and the
item of each, so the decision can be made at any depth: an oversized nested
sequence can be stepped over element by element, which SpecificTags cannot
express at all.

Three returns: ElementKeep (the zero value, so a hook that falls off the end
of its own logic parses the file the way godicom always has), ElementSkip,
ElementStop.

API sign-off

Both API decisions were put to the user before any code was written, per
AGENTS.md (以创造接口为耻,以复用现有为荣 / 以模糊执行为耻,以寻求确认为荣):

  1. Shape — ReadOptions.OnElement returning a three-valued
    ElementAction, rather than a boolean stop_when mirroring pydicom's.
    Three values subsume both existing options; a boolean cannot express skip.
  2. Depth — called at every depth, and fix SpecificTags while we are
    here. Chosen over a top-level-only hook.

BREAKING: SpecificTags no longer empties the sequences it keeps

This came out of a probe, not a hunch. SpecificTags: []Tag{(3006,0010)} on
rtstruct.dcm returned the sequence with one item holding zero of its two
elements
— a dataset no file could have produced.

pydicom does not do this. read_sequence_item calls read_dataset without
passing specific_tags along (filereader.py), so a sequence that survives
the filter is parsed whole. SpecificTags now filters the top-level dataset
only, matching that. TestSpecificTagsKeepsSequenceItems pins it.

Callers who want the old behaviour pass an OnElement returning
ElementSkip when len(path) > 0.

Convergence

The three element loops — two over a byte slice, one over a ReaderAt — each
carried their own copy of the keep/stop conditions, twelve checks in all. They
now call one decideElement at the same point, so the three agree by
construction rather than by three copies staying in step.
TestOnElementStopMatchesStopBeforePixels holds a hook stopping at
(7FE0,0010) against the option, tag for tag.

Details worth a look

  • ValueTell is in file coordinates. The streaming reader parses a
    sequence out of a chunk it copied, so the offset its loop holds is relative
    to that chunk. Reporting it unadjusted handed the caller a number pointing
    into the wrong element — decideElement adds rc.baseOffset, the same way
    readContext.report does for Diagnostic.Offset.
    TestOnElementValueTellInsideSequence reads the bytes at the reported offset
    and compares them to the value from a full read.
  • Skipping an undefined-length value is real work. It declares no length,
    so the end can only be found by walking the item stream — but walking it is
    cheap and copying it is the whole cost of the element.
    TestOnElementSkipUndefinedLengthDoesNotCopyIt measures it in bytes
    allocated; reverting the branch drops the saving from 6128 bytes to 512.
  • Diagnostics are not suppressed for a skipped element. They describe the
    file, not the subset the caller kept. ElementStop is the exception, because
    the parse ends there. Both directions are pinned
    (TestOnElementSkipStillDiagnoses, TestOnElementStopEndsDiagnostics).
  • path is the reader's own slice, valid only for the duration of the call.
    Documented rather than copied, matching bufio.Scanner.Bytes() — the hook
    runs once per element and most hooks only read it. (Diagnostic.Path does
    copy, because diagnostics are rare.)
  • Two exemptions: group 0x0002 and (0008,0005) Specific Character Set are
    never offered. Skipping either changes what the rest of the file means.

Verification

Every behavioural claim was checked by reverting the code that implements it
and watching the new test fail:

Reverted Failure
valueStart += rc.baseOffset ValueTell 16 points at "\x00\x00…", want "ABCD1234"
len(path) == 0 scope guard item 0 has 0 elements, want 2
undefined-length skip branch the saving is 512, want at least 6128

TestOnElementSkipWholeFileAgreesWithFullRead is the round-trip that catches a
skip landing on the wrong byte: for each of CT_small / MR_small / rtstruct, one
read per top-level tag keeping only that tag, and the value has to match the
full read. TestOnElementSkipOnEveryReadPath runs the same hook through
ReadFile, ReadBytes, a *bytes.Reader and a non-seekable reader, because
each takes a different loop.

Gates: go build, go vet, staticcheck -checks=all, gofmt -l (clean),
go test -count=1 -p 4 ./..., and GOARCH=386 CGO_ENABLED=0 go test ./...
(which caught MustTag(0xFFFCFFFC) overflowing int on a 32-bit build).

Follow-up

docs/godicom/datasets.md and its zh counterpart describe the old
SpecificTags sequence behaviour and need a docs-site PR.

🤖 Generated with Claude Code

A caller after seven index tags out of a 200 MB file had two options, and
both were the wrong shape. SpecificTags names the tags to keep, but it
filters at every depth, so naming a sequence kept the sequence and emptied
its items. StopBeforePixels stops at one hardcoded tag. Neither can say
"walk this file, hand me the headers, and let me decide" -- which is what
indexing a study actually needs.

ReadOptions.OnElement is handed each element's header after it is decoded
and before its value is read, and returns ElementKeep, ElementSkip or
ElementStop. The RawDataElement it gets carries the Tag, the VR in effect,
the declared Length and the ValueTell -- but not the value, not having read
it being the entire point. The []PathStep alongside it names the enclosing
sequences and the item of each, so the decision can be made at any depth:
an oversized nested sequence can be stepped over element by element, which
SpecificTags cannot express at all.

The three element loops -- two over a byte slice, one over a ReaderAt --
each carried their own copy of the keep/stop conditions, twelve checks in
all. They now call one decideElement at the same point, so the answer is
the same by construction rather than by three copies staying in step.
StopBeforePixels and SpecificTags are implemented on top of it, and
TestOnElementStopMatchesStopBeforePixels holds a hook stopping at
(7FE0,0010) against the option, tag for tag.

Two elements are never offered: group 0x0002, which the transfer syntax is
read from, and (0008,0005) Specific Character Set, which decides how every
text value after it decodes. Skipping either changes what the rest of the
file means rather than how much of it is kept.

BREAKING: SpecificTags no longer empties the sequences it keeps. It now
filters the top-level dataset only, which is where pydicom applies it --
read_sequence_item calls read_dataset without passing specific_tags along
(filereader.py) -- so a sequence that survives the filter is parsed whole.
(3006,0010) used to come back with one item holding zero of its two
elements, a dataset no file could have produced. An OnElement hook
returning ElementSkip when len(path) > 0 restores the old behaviour.

Skipping is real work, not a filter applied after the fact. A defined-length
value or a defined-length SQ declares its own extent, so stepping over it is
arithmetic. An undefined-length value declares nothing, so the item stream
still has to be walked to find the end -- but walking it is cheap and copying
it is the whole cost of the element, and
TestOnElementSkipUndefinedLengthDoesNotCopyIt measures the difference in
bytes allocated.

Diagnostics are deliberately not suppressed for a skipped element: they
describe the file, not the subset the caller chose to keep. ElementStop is
the exception, because the parse ends there.

Closes #72.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@xbmlz
xbmlz merged commit c8e28e9 into main Sep 7, 2026
6 checks passed
@xbmlz
xbmlz deleted the read-onelement-hook branch September 7, 2026 03:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Streaming should be a first-class capability (#51 §12)

1 participant