Repository navigation

[SPARK-60716][CONNECT] Convert explicit pandas date columns through Arrow - #138

Closed

dbtsai wants to merge 53 commits into

apache:masterfrom

dbtsai:codex/spark-60716-columnar-pandas-date

Closed

[SPARK-60716][CONNECT] Convert explicit pandas date columns through Arrow#138
dbtsai wants to merge 53 commits into
apache:masterfrom
dbtsai:codex/spark-60716-columnar-pandas-date

Commits on Nov 5, 2026
- [SPARK-60676][CONNECT] Use Arrow columns for signed int64 pandas inputs
  dbtsai
  committed
  dc29a19
- [SPARK-60676][CONNECT] Validate integer upload scales against mock RPC
  dbtsai
  committed
  3e79f91
- [SPARK-60676][CONNECT] Cover leading null inference and non-nullable schemas
  dbtsai
  committed
  8c705c0
- [SPARK-60676][CONNECT] Exercise named schemas through mock rename relations
  dbtsai
  committed
  fb996d9
- [SPARK-60676][CONNECT] Avoid Arrow configuration overhead for small NumPy frames
  dbtsai
  committed
  79d3faf
- [SPARK-60676][CONNECT] Retain IPC row fallback for NUL field names
  dbtsai
  committed
  e266e8e
- [SPARK-60676][CONNECT] Include pandas regressions in the existing Arrow CI suite
  dbtsai
  committed
  9942f21
- [SPARK-60676][CONNECT] Preserve integer pandas schema and fallback semantics
  
  dbtsai
  and
  Isaac
  committed
  5edd700
- [SPARK-60676][TESTS] Preserve subclass column labels on pandas 3
  
  dbtsai
  and
  Isaac
  committed
  080aa16
- [SPARK-60676][CONNECT] Preserve customized pandas columns hooks
  
  dbtsai
  and
  Isaac
  committed
  abbc23a
- [SPARK-60676][CONNECT] Preserve rebound pandas index storage ordering
  
  dbtsai
  and
  Isaac
  committed
  7961bde
- Preserve integer pandas fallback with missing Arrow aliases
  
  dbtsai
  and
  Isaac
  committed
  3e3a8f5
- [SPARK-60676][TESTS] Retain current master coverage after rebase
  
  dbtsai
  and
  Isaac
  committed
  f058ac9
- Preserve current pandas label iteration and fallback order after rebase
  
  dbtsai
  and
  Isaac
  committed
  121a141
- Keep canonical string storage guard module identity
  
  dbtsai
  and
  Isaac
  committed
  aca1d02
Commits on Nov 6, 2026
- [CONNECT] Reconcile remaining change with current inference architecture
  
  dbtsai
  and
  Isaac
  committed
  ecdd67a
- [CONNECT] Reconcile remaining change with current inference architecture
  
  dbtsai
  and
  Isaac
  committed
  e7ae71a
- [CONNECT] Reconcile remaining change with current inference architecture
  
  dbtsai
  and
  Isaac
  committed
  40e5266
- [SPARK-60685][CONNECT] Fast-path exact builtin date inputs
  dbtsai
  committed
  7b40f32
- [SPARK-60685][CONNECT] Include date tests in the CI offline suite
  dbtsai
  committed
  4a50058
- [SPARK-60698][CONNECT] Batch Arrow input normalization config reads
  
  dbtsai
  and
  Isaac
  committed
  fc7373c
- [SPARK-60716][CONNECT] Convert explicit pandas date columns through Arrow
  
  dbtsai
  and
  Isaac
  committed
  1fc2bab
- [SPARK-60716][CONNECT] Keep small date frames on the row path
  
  dbtsai
  and
  Isaac
  committed
  5727e18
- [SPARK-60716][CONNECT] Respect the optional protoc test fixture
  
  dbtsai
  and
  Isaac
  committed
  bd725f9
- [SPARK-60716][CONNECT] Preserve custom pandas date fallback
  
  dbtsai
  and
  Isaac
  committed
  21ca8f1
- [SPARK-60716][CONNECT] Retain direct pandas date normalization hooks
  
  dbtsai
  and
  Isaac
  committed
  bbd3c15
- [SPARK-60716][CONNECT] Preserve date column-label conversion hooks
  
  dbtsai
  and
  Isaac
  committed
  b9d1a5d
- [SPARK-60716][CONNECT] Preserve current Arrow alias fallback semantics
  
  dbtsai
  and
  Isaac
  committed
  1d9869c
- Align date label regression tests with current native iteration
  
  dbtsai
  and
  Isaac
  committed
  5380896
- [SPARK-60727][CONNECT] Keep canonical double input on inline transport
  
  dbtsai
  and
  Isaac
  committed
  280bdee
- [SPARK-60727][CONNECT] Normalize Float helper terminal newline
  
  dbtsai
  and
  Isaac
  committed
  11ff3b7
- Reconcile current master interfaces and formatting after rebase
  
  dbtsai
  and
  Isaac
  committed
  b9498d1
- Align float label regression tests with current native iteration
  
  dbtsai
  and
  Isaac
  committed
  2bd4638
- [CONNECT] Reconcile remaining change with current inference architecture
  
  dbtsai
  and
  Isaac
  committed
  ccc1b48
- [SPARK-60720][CONNECT] Encode guarded pandas String input inline
  
  dbtsai
  and
  Isaac
  committed
  26be6ab
- [SPARK-60720][CONNECT] Remove extra EOF blank lines
  
  dbtsai
  and
  Isaac
  committed
  09d0b13
- Reconcile current master interfaces and formatting after rebase
  
  dbtsai
  and
  Isaac
  committed
  4304062
- Align string label regression tests with current native iteration
  
  dbtsai
  and
  Isaac
  committed
  b2274e4
- [CONNECT] Reconcile remaining change with current inference architecture
  
  dbtsai
  and
  Isaac
  committed
  cfde2b6
- [CONNECT] Reconcile remaining change with current inference architecture
  
  dbtsai
  and
  Isaac
  committed
  e31aad6
- [CONNECT] Reconcile remaining change with current inference architecture
  
  dbtsai
  and
  Isaac
  committed
  053bbcf
- [CONNECT] Reconcile remaining change with current inference architecture
  
  dbtsai
  and
  Isaac
  committed
  9135b9b
- Merge remote-tracking branch 'apache/master' into SPARK-60727-pandas-float64
  
  HyukjinKwon
  and
  Isaac
  committed
  b415494
- [SPARK-60727][CONNECT] Drop the converter guards that the float64 path never reaches, and document the path
  
  HyukjinKwon
  and
  Isaac
  committed
  489a663
- [SPARK-60727][CONNECT] Do not pin the reads of master's pandas row path in the float64 test
  
  HyukjinKwon
  and
  Isaac
  committed
  9fbcd15
- Merge branch 'SPARK-60727-pandas-float64' into SPARK-60720-pandas-strings
  
  HyukjinKwon
  and
  Isaac
  committed
  121a7a8
- [SPARK-60720][CONNECT] Encode each pandas string once, and drop the converter checks the string path never reaches
  
  HyukjinKwon
  and
  Isaac
  committed
  6eca4a4
- [SPARK-60720][CONNECT] Do not pin the reads of master's pandas row path in the string test
  
  HyukjinKwon
  and
  Isaac
  committed
  6104f54
- Merge remote-tracking branch 'apache/master' into SPARK-60727-pandas-float64
  
  HyukjinKwon
  and
  Isaac
  committed
  a31846b
- Merge branch 'SPARK-60727-pandas-float64' into SPARK-60720-pandas-strings
  
  HyukjinKwon
  and
  Isaac
  committed
  74bf14b
- Merge branch 'SPARK-60720-pandas-strings' into upd-138
  
  HyukjinKwon
  and
  Isaac
  committed
  1aa7100
- [SPARK-60716][CONNECT] Serve the Arrow importer's settings in the exact date capture mock, and do not pin the reads of master's pandas row path
  
  HyukjinKwon
  and
  Isaac
  committed
  adc009b
- Merge remote-tracking branch 'apache/master' into upd-138
  
  HyukjinKwon
  and
  Isaac
  committed
  281188e

Commits

Commits on Nov 5, 2026

Commits on Nov 6, 2026