Summary
Make valid aggregate queries succeed when participating storage groups have different projected schemas. Follow-up to #14066 under #13998.
Size audit and dependencies
Classification: tracking parent, not an executable leaf or automated implementation task.
Boundary: multi-group DistributedPlan.Execute through executeAgg, ReduceRawFrames, and ReducePartialBatches.
Schema alignment and storage-group-aware replica handling are independently testable changes; split them into executable tasks before implementation.
Blocked by: #14066 via BanyanDB PR apache/skywalking-banyandb#1326, to finalize the remaining boundary after legacy removal.
Proposed initial slice
Normalize missing projected, non-grouping tags to null before partial reduction, rather than rejecting the query because partial layouts differ. Do not simply disable structural schema validation.
Concrete oracle
- Old schema:
svc,value; rows (old,2) and (old,3).
- New schema:
svc,zone,value; rows (new,west,7) and (new,west,11).
- Query:
GROUP BY svc, SUM(value), project svc,zone.
Expected unordered output:
old, NULL, 5
new, west, 18
Different service keys deliberately avoid ambiguity over representative values of non-grouping tags. Currently differently shaped partial batches can fail with partial ... schema mismatch.
Completion criteria
Relevant code: pkg/query/vectorized/measure/reduce.go, pkg/query/vectorized/measure/plan/distributed.go; extend distributed-plan and topology-matrix coverage.
go test ./pkg/query/vectorized/measure/...
go test ./test/integration/distributed/query/...
Out of scope of the first slice
Missing GroupBy keys: these fail earlier during planning and need a separately specified change, not an assumed reducer-only fix. Also exclude changing aggregate/operator semantics or replacing the ordinary multi-group row merger.
Parent: #13998.
Compatibility and review policy
- Preserve existing on-disk and wire formats and mixed-version compatibility. Any necessary protocol change requires a separately reviewed compatibility contract.
- Use explicit, hand-calculated expected results; do not depend on the removed row engine as the test oracle.
- Human review enforces the no-new-row-path policy. A row-call-site lint guard is not required.
Summary
Make valid aggregate queries succeed when participating storage groups have different projected schemas. Follow-up to #14066 under #13998.
Size audit and dependencies
Classification: tracking parent, not an executable leaf or automated implementation task.
Boundary: multi-group
DistributedPlan.ExecutethroughexecuteAgg,ReduceRawFrames, andReducePartialBatches.Schema alignment and storage-group-aware replica handling are independently testable changes; split them into executable tasks before implementation.
Blocked by: #14066 via BanyanDB PR apache/skywalking-banyandb#1326, to finalize the remaining boundary after legacy removal.
Proposed initial slice
Normalize missing projected, non-grouping tags to null before partial reduction, rather than rejecting the query because partial layouts differ. Do not simply disable structural schema validation.
Concrete oracle
svc,value; rows(old,2)and(old,3).svc,zone,value; rows(new,west,7)and(new,west,11).GROUP BY svc,SUM(value), projectsvc,zone.Expected unordered output:
Different service keys deliberately avoid ambiguity over representative values of non-grouping tags. Currently differently shaped partial batches can fail with
partial ... schema mismatch.Completion criteria
apifrom two different storage groups must total 23, even when both groups use shard 0; duplicate replicas of either contribution must not increase that sum. This requires its own focused implementation/test slice.Relevant code:
pkg/query/vectorized/measure/reduce.go,pkg/query/vectorized/measure/plan/distributed.go; extend distributed-plan and topology-matrix coverage.Out of scope of the first slice
Missing GroupBy keys: these fail earlier during planning and need a separately specified change, not an assumed reducer-only fix. Also exclude changing aggregate/operator semantics or replacing the ordinary multi-group row merger.
Parent: #13998.
Compatibility and review policy