Repository navigation
executemany() sporadically fails to insert rows, throws no exception #670
Description
Describe the bug
Short version:
When inserting data from a pandas dataframe through executemany(), the insert will sporadically fail (0 rows inserted) and sporadically work (86 rows inserted), even if everything else remains the same.
No exceptions are thrown, neither on the SQL side, nor in Python.
Longer version / additional context:
I import an .xpt file using pd.readsas(). The file contains 86 rows and 19 columns. For 2 rows, the data is partially incomplete and information for some of the columns is missing.
The bug is correlated with this particular file – other files import successfully – but, for runs in which all 86 rows are imported successfully, the data is then completely fine and properly stored in the DB.
When I omit the two rows with incomplete data, all rows are always inserted correctly. Likewise, when I omit one particular column, all rows are always inserted correctly.
When I iterate through many execute() commands, rather than using executemany(), all rows are always inserted correctly.
When I convert the datatypes to string for all columns ahead of the executemany() command, all rows are always inserted correctly.
Exception message: None
Stack trace:
SQL Server Profiler:
A successful run will show:
- 1 sp_prepare statement
- 86 exec sp_execute statements
- 1 exec sp_unprepare statement
A non successful run will show:
- 1 sp_prepare statement
- 1 exec sp_unprepare statement
MSSQL Python Logs:
Complete logs attached.
A successful run will end in:
2026-07-09 10:28:51.194, 19736, DEBUG, ddbc_bindings.cpp:2658, DDBC, SQLExecuteMany: PARAMSET_SIZE set to 86
2026-07-09 10:28:51.202, 19736, DEBUG, ddbc_bindings.cpp:2665, DDBC, SQLExecuteMany: SQLExecute completed - rc=0
2026-07-09 10:28:51.203, 19736, DEBUG, ddbc_bindings.cpp:1611, DDBC, SQLGetAllDiagRecords: Retrieving all diagnostic records for handle 0000026954E2BE60, handleType=3
2026-07-09 10:28:51.203, 19736, DEBUG, ddbc_bindings.cpp:5708, DDBC, SQLRowCount_wrap: Get number of rows affected by last execute
2026-07-09 10:28:51.203, 19736, DEBUG, ddbc_bindings.cpp:5721, DDBC, SQLRowCount_wrap: SQLRowCount returned 86
2026-07-09 10:28:51.203, 19736, DEBUG, ddbc_bindings.cpp:2787, DDBC, SQLDescribeCol: Getting column descriptions for statement_handle=0000026954E2BE60
2026-07-09 10:28:51.203, 19736, DEBUG, ddbc_bindings.cpp:5708, DDBC, SQLRowCount_wrap: Get number of rows affected by last execute
2026-07-09 10:28:51.203, 19736, DEBUG, ddbc_bindings.cpp:5721, DDBC, SQLRowCount_wrap: SQLRowCount returned 86
2026-07-09 10:28:51.203, 19736, DEBUG, connection.cpp:192, DDBC, Committing transaction
2026-07-09 10:28:51.206, 19736, INFO, connection.py:1567, Python, Transaction committed successfully.
An unsuccessful run will end in:
2026-07-09 10:19:14.412, 11020, DEBUG, ddbc_bindings.cpp:2600, DDBC, BindParameterArray: Successfully bound all parameters - total_params=21, buffer_count=39
2026-07-09 10:19:14.412, 11020, DEBUG, ddbc_bindings.cpp:2658, DDBC, SQLExecuteMany: PARAMSET_SIZE set to 86
2026-07-09 10:19:14.413, 11020, DEBUG, ddbc_bindings.cpp:2665, DDBC, SQLExecuteMany: SQLExecute completed - rc=99
2026-07-09 10:19:14.413, 11020, DEBUG, ddbc_bindings.cpp:1611, DDBC, SQLGetAllDiagRecords: Retrieving all diagnostic records for handle 0000024BFF3FBE60, handleType=3
2026-07-09 10:19:14.413, 11020, DEBUG, ddbc_bindings.cpp:5708, DDBC, SQLRowCount_wrap: Get number of rows affected by last execute
2026-07-09 10:19:14.413, 11020, DEBUG, ddbc_bindings.cpp:5721, DDBC, SQLRowCount_wrap: SQLRowCount returned -1
2026-07-09 10:19:14.414, 11020, DEBUG, ddbc_bindings.cpp:2787, DDBC, SQLDescribeCol: Getting column descriptions for statement_handle=0000024BFF3FBE60
2026-07-09 10:19:14.414, 11020, DEBUG, ddbc_bindings.cpp:5708, DDBC, SQLRowCount_wrap: Get number of rows affected by last execute
2026-07-09 10:19:14.414, 11020, DEBUG, ddbc_bindings.cpp:5721, DDBC, SQLRowCount_wrap: SQLRowCount returned -1
2026-07-09 10:19:14.414, 11020, DEBUG, connection.cpp:192, DDBC, Committing transaction
2026-07-09 10:19:14.416, 11020, INFO, connection.py:1567, Python, Transaction committed successfully.
log_failure.txt
log_success.txt
To reproduce
Very difficult, because I can't easily share the source data...
Python code attached:
Expected behavior
Ideally, the data is always inserted correctly. But as a minimum, I’d expect an error to be thrown, rather than the transaction being judged as a success.
Further technical details
Python version: Python 3.14.5
MSSQL Python version: 1.8.0
SQL Server version: SQL Server 2025
Operating system: Client: Windows 11; Server: Debian GNU/Linux 12
Activity
github-actions commentedon Jul 9, 2026
Hi Mapkinasekinase, thank you for opening this issue!
Our team will review it shortly. We aim to triage all new issues within 24-48 hours and get back to you.
If you have additional information to share, please feel free to update the issue.
Thank you for your patience!
gargsaumya commentedon Jul 13, 2026
Hi Mapkinasekinase, thanks for the detailed report. It looks like it stems from how the batch (executemany) path sets up the NULL length/indicator values for numeric columns when a NULL follows non-null values in the same column - which leaves some indicators uninitialized.
We'll dig in further and share an update here shortly.
Merge branch 'main' into saumya/gh-670
Merge branch 'main' into saumya/gh-670
Merge branch 'main' into saumya/gh-670
Merge branch 'main' into saumya/gh-670
FIX: Initialize numeric array indicators in executemany (#702)
gargsaumya commentedon Aug 6, 2026
Thanks for reporting this issue. The fix has been merged and will be included in the next release on 7 August. If you continue to experience the issue after upgrading, please reopen this issue.
Thanks!
RELEASE:1.13.0 (#707)