Repository navigation
[core] Allow primary-key-only deletes for deduplicate tables - #9978
Merged
Merged
Conversation
Validate only primary keys and sequence fields for DELETE rows in non-cross-partition deduplicate tables while preserving full nullability checks for other row kinds and merge engines. Use a nullable internal value type so omitted delete payload fields can be serialized safely, including lookup merge paths. Signed-off-by: Akash Reddy Jammula <akashreddyjammula@gmail.com>
Cover direct and generated deletes, lookup merge, sequence ordering, non-delete row kinds, partial-update tables, and cross-partition validation. Co-authored-by: GitHub Copilot <copilot@github.com> Signed-off-by: Akash Reddy Jammula <akashreddyjammula@gmail.com>
JingsongLi
reviewed
Sep 20, 2026
JingsongLi
left a comment
Contributor
There was a problem hiding this comment.
Requirement fit: SUPPORTED (triage: GO)
Implementation: CLEAN
The change enables direct primary-key-only DELETE records without weakening validation for inserts/updates: only delete rows use the reduced required-field set, routing keys remain present, and sequence fields are retained when needed. The tests cover lookup, cross-partition constraints, sequence ordering, row-kind fields, and rejection of incomplete non-delete rows. I did not find an actionable regression, and CI is green.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Purpose
Allow callers of
TableWriteto delete a row from a primary-key table without supplying values for every non-keyNOT NULLcolumn.Previously,
TableWriteImplapplied the table's complete nullability validation to every row kind. A direct delete such as the following was therefore rejected whenvwas declaredNOT NULL, even though the deduplicate merge engine only needs the key to identify the row being deleted:This differs from SQL delete paths that first read the existing row and consequently submit a fully populated delete record. Direct TableWrite users should not need that additional read when the merge semantics do not consume the omitted fields.
Closes #4702.
Changes
Row-kind-aware nullability validation
TableWriteImplnow determines the effective row kind before selecting the nullability checks to apply.For normal rows, the existing behavior remains unchanged: every field declared NOT NULL must still be present.
For eligible
DELETErows,TableWriteImplcan use a restricted set of required fields instead of validating all non-null columns. The original constructor is preserved and continues to perform full validation, so existing callers do not change behavior unless they explicitly provide the delete-specific field set.The effective row kind is derived from the default-wrapped row, which preserves support for rowkind.field . Nullability is still checked against the original input row so configured defaults do not unintentionally bypass input validation.
Conservative scope
The relaxed behavior is enabled only for primary-key tables that satisfy both conditions:
For an eligible delete, the following fields remain required when declared NOT NULL :
Sequence fields remain mandatory because they participate in ordering. A delete with an older sequence value must not remove a newer row.
Full nullability validation is retained for:
Internal serialization support
Skipping the explicit nullability check alone is insufficient. The serializer generated from a logical NOT NULL primitive type directly invokes getters such as getInt and getLong ; attempting to serialize a Java null through those getters can fail.
For eligible deduplicate tables, the internal key-value value type therefore marks fields that may be omitted from a delete as nullable. Primary keys and sequence fields retain their original nullability.
This is an internal storage/serialization type only. The table's public logical schema and its NOT NULL constraints are unchanged, and non-delete rows continue to be validated against the complete logical schema.
The same internal nullable value type is also supplied to lookup merge processing so buffered or spilled delete records are serialized consistently.
Behavior examples
Given this table:
pt INT NOT NULL
k INT NOT NULL
v BIGINT NOT NULL
PRIMARY KEY (pt, k)
Changelog considerations
This change does not reconstruct the previous non-key values for a direct delete. A changelog producer that exposes the input delete record may therefore contain null values for omitted non-key fields.
That is expected for this API path: the final table semantics are preserved, but callers that require a complete before-image must still provide the previous values or use a changelog mode capable of reconstructing them.
Tests
Added coverage for:
Notes for reviewers
The restriction to non-cross-partition DEDUPLICATE tables is intentional:
The internal nullable value type is necessary to serialize the omitted fields safely; changing only TableWriteImpl.checkNullability would allow the row through validation but could fail later in primitive field serialization.