Skip to content

Commit c4e647e

Browse files
dlevy-msft-sqldlevy-msft-sql
authored
FIX: Align date/time type code mappings with ODBC 18 driver source (microsoft#352) (microsoft#355)
### Work Item / Issue Reference <!-- IMPORTANT: Please follow the PR template guidelines below. For mssql-python maintainers: Insert your ADO Work Item ID below For external contributors: Insert Github Issue number below Only one reference is required - either GitHub issue OR ADO Work Item. --> <!-- External contributors: GitHub Issue --> > GitHub Issue: microsoft#352 ------------------------------------------------------------------- ### Summary Fixes `polars.read_database()` `ComputeError` on DATE columns by aligning `cursor.py` type code mappings with the ODBC 18 driver's actual reported SQL type codes. **Root Cause:** The ODBC 18 driver reports ODBC 3.x type codes (`SQL_TYPE_DATE=91`, `SQL_TYPE_TIMESTAMP=93`, `SQL_SS_TIME2=-154`) but `_map_data_type` only had ODBC 2.x constants (`SQL_DATE=9`, `SQL_TIME=10`, `SQL_TIMESTAMP=11`). DATE columns fell through to the `str` default, causing a schema mismatch when polars expected `pl.Date`. **Fix (verified against ODBC 18 driver C++ source):** - `_map_data_type`: Rewritten with driver-verified ODBC 3.x type codes organized by category (string, integer, float, decimal, date/time, boolean, binary, UUID, XML). Removed phantom ODBC 2.x entries that the driver never reports. - `_get_c_type_for_sql_type`: Updated C-type bindings from ODBC 2.x to 3.x codes for all date/time types. - `constants.py`: Added `SQL_SS_TIME2`, `SQL_SS_XML`, `SQL_C_SS_TIME2`. **Zero breaking changes.** `cursor.description[i][1]` continues to return Python type objects (`datetime.date`, `datetime.datetime`, etc.) — now mapped from the correct SQL type codes. ## Changes | File | Change | |------|--------| | `mssql_python/constants.py` | +3 constants (`SQL_SS_TIME2`, `SQL_SS_XML`, `SQL_C_SS_TIME2`) | | `mssql_python/cursor.py` | Rewritten `_map_data_type` and `_get_c_type_for_sql_type` with ODBC 3.x codes | | `tests/test_018_polars_pandas_integration.py` | **NEW** — 14 integration tests | ## Testing 14 new tests across 3 test classes: - **`TestCursorDescriptionTypeCodes`** (7 tests): Verifies all 6 date/time SQL types return correct Python types with `isclass()` checks - **`TestPolarsIntegration`** (4 tests): `polars.read_database()` with DATE, all datetime types, mixed types, NULLs - **`TestPandasIntegration`** (3 tests): `pandas.read_sql()` with DATE, all datetime types, mixed types All 14 tests pass. Existing test suite (448 tests) unaffected.
1 parent 5a2187d commit c4e647e

File tree

3 files changed

+440
-12
lines changed

3 files changed

+440
-12
lines changed

‎mssql_python/constants.py‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -115,6 +115,8 @@ class ConstantsDDBC(Enum):
115115
SQL_FETCH_RELATIVE = 6
116116
SQL_FETCH_BOOKMARK = 8
117117
SQL_DATETIMEOFFSET = -155
118+
SQL_SS_TIME2 = -154
119+
SQL_SS_XML = -152
118120
SQL_C_SS_TIMESTAMPOFFSET = 0x4001
119121
SQL_SCOPE_CURROW = 0
120122
SQL_BEST_ROWID = 1
@@ -365,6 +367,12 @@ def get_valid_types(cls) -> set:
365367
ConstantsDDBC.SQL_DATE.value,
366368
ConstantsDDBC.SQL_TIME.value,
367369
ConstantsDDBC.SQL_TIMESTAMP.value,
370+
ConstantsDDBC.SQL_TYPE_DATE.value,
371+
ConstantsDDBC.SQL_TYPE_TIME.value,
372+
ConstantsDDBC.SQL_TYPE_TIMESTAMP.value,
373+
ConstantsDDBC.SQL_SS_TIME2.value,
374+
ConstantsDDBC.SQL_DATETIMEOFFSET.value,
375+
ConstantsDDBC.SQL_SS_XML.value,
368376
ConstantsDDBC.SQL_GUID.value,
369377
}
370378

‎mssql_python/cursor.py‎

Lines changed: 59 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -844,7 +844,7 @@ def _reset_inputsizes(self) -> None:
844844
self._inputsizes = None
845845

846846
def _get_c_type_for_sql_type(self, sql_type: int) -> int:
847-
"""Map SQL type to appropriate C type for parameter binding"""
847+
"""Map SQL type to appropriate C type for parameter binding."""
848848
sql_to_c_type = {
849849
ddbc_sql_const.SQL_CHAR.value: ddbc_sql_const.SQL_C_CHAR.value,
850850
ddbc_sql_const.SQL_VARCHAR.value: ddbc_sql_const.SQL_C_CHAR.value,
@@ -865,9 +865,19 @@ def _get_c_type_for_sql_type(self, sql_type: int) -> int:
865865
ddbc_sql_const.SQL_BINARY.value: ddbc_sql_const.SQL_C_BINARY.value,
866866
ddbc_sql_const.SQL_VARBINARY.value: ddbc_sql_const.SQL_C_BINARY.value,
867867
ddbc_sql_const.SQL_LONGVARBINARY.value: ddbc_sql_const.SQL_C_BINARY.value,
868+
# ODBC 3.x date/time types (reported by ODBC 18 driver)
869+
ddbc_sql_const.SQL_TYPE_DATE.value: ddbc_sql_const.SQL_C_TYPE_DATE.value,
870+
ddbc_sql_const.SQL_TYPE_TIME.value: ddbc_sql_const.SQL_C_TYPE_TIME.value,
871+
ddbc_sql_const.SQL_TYPE_TIMESTAMP.value: ddbc_sql_const.SQL_C_TYPE_TIMESTAMP.value,
872+
ddbc_sql_const.SQL_SS_TIME2.value: ddbc_sql_const.SQL_C_TYPE_TIME.value,
873+
ddbc_sql_const.SQL_DATETIMEOFFSET.value: ddbc_sql_const.SQL_C_SS_TIMESTAMPOFFSET.value,
874+
# ODBC 2.x aliases (accepted by setinputsizes via SQLTypes)
868875
ddbc_sql_const.SQL_DATE.value: ddbc_sql_const.SQL_C_TYPE_DATE.value,
869876
ddbc_sql_const.SQL_TIME.value: ddbc_sql_const.SQL_C_TYPE_TIME.value,
870877
ddbc_sql_const.SQL_TIMESTAMP.value: ddbc_sql_const.SQL_C_TYPE_TIMESTAMP.value,
878+
# Other types
879+
ddbc_sql_const.SQL_GUID.value: ddbc_sql_const.SQL_C_GUID.value,
880+
ddbc_sql_const.SQL_SS_XML.value: ddbc_sql_const.SQL_C_WCHAR.value,
871881
}
872882
return sql_to_c_type.get(sql_type, ddbc_sql_const.SQL_C_DEFAULT.value)
873883

@@ -1026,34 +1036,71 @@ def _map_data_type(self, sql_type):
10261036
"""
10271037
Map SQL data type to Python data type.
10281038
1039+
Maps the ODBC SQL type code returned by SQLDescribeCol to the
1040+
corresponding Python type for cursor.description[i][1].
1041+
1042+
The ODBC 18 driver for SQL Server reports these type codes:
1043+
Standard ODBC 3.x types:
1044+
SQL_CHAR(1), SQL_VARCHAR(12), SQL_LONGVARCHAR(-1),
1045+
SQL_WCHAR(-8), SQL_WVARCHAR(-9), SQL_WLONGVARCHAR(-10),
1046+
SQL_INTEGER(4), SQL_SMALLINT(5), SQL_TINYINT(-6), SQL_BIGINT(-5),
1047+
SQL_BIT(-7), SQL_FLOAT(6), SQL_REAL(7), SQL_DOUBLE(8),
1048+
SQL_DECIMAL(3), SQL_NUMERIC(2),
1049+
SQL_BINARY(-2), SQL_VARBINARY(-3), SQL_LONGVARBINARY(-4),
1050+
SQL_TYPE_DATE(91), SQL_TYPE_TIME(92), SQL_TYPE_TIMESTAMP(93), SQL_GUID(-11)
1051+
SQL Server-specific types (from msodbcsql.h):
1052+
SQL_SS_TIME2(-154) for time columns
1053+
SQL_DATETIMEOFFSET(-155) for datetimeoffset columns
1054+
SQL_SS_XML(-152) for xml columns
1055+
1056+
ODBC 2.x aliases (9, 10, 11) are also accepted defensively.
1057+
10291058
Args:
1030-
sql_type: SQL data type.
1059+
sql_type: SQL data type code from SQLDescribeCol.
10311060
10321061
Returns:
10331062
Corresponding Python data type.
10341063
"""
10351064
sql_to_python_type = {
1036-
ddbc_sql_const.SQL_INTEGER.value: int,
1037-
ddbc_sql_const.SQL_VARCHAR.value: str,
1038-
ddbc_sql_const.SQL_WVARCHAR.value: str,
1065+
# String types
10391066
ddbc_sql_const.SQL_CHAR.value: str,
1067+
ddbc_sql_const.SQL_VARCHAR.value: str,
1068+
ddbc_sql_const.SQL_LONGVARCHAR.value: str,
10401069
ddbc_sql_const.SQL_WCHAR.value: str,
1070+
ddbc_sql_const.SQL_WVARCHAR.value: str,
1071+
ddbc_sql_const.SQL_WLONGVARCHAR.value: str,
1072+
# Integer types
1073+
ddbc_sql_const.SQL_INTEGER.value: int,
1074+
ddbc_sql_const.SQL_SMALLINT.value: int,
1075+
ddbc_sql_const.SQL_TINYINT.value: int,
1076+
ddbc_sql_const.SQL_BIGINT.value: int,
1077+
# Floating-point types
10411078
ddbc_sql_const.SQL_FLOAT.value: float,
10421079
ddbc_sql_const.SQL_DOUBLE.value: float,
1080+
ddbc_sql_const.SQL_REAL.value: float,
1081+
# Exact numeric types
10431082
ddbc_sql_const.SQL_DECIMAL.value: decimal.Decimal,
10441083
ddbc_sql_const.SQL_NUMERIC.value: decimal.Decimal,
1045-
ddbc_sql_const.SQL_DATE.value: datetime.date,
1046-
ddbc_sql_const.SQL_TIMESTAMP.value: datetime.datetime,
1047-
ddbc_sql_const.SQL_TIME.value: datetime.time,
1084+
# Date/time types — values the ODBC 18 driver actually reports
1085+
ddbc_sql_const.SQL_TYPE_DATE.value: datetime.date, # 91 — date
1086+
ddbc_sql_const.SQL_TYPE_TIME.value: datetime.time, # 92 — time (ODBC 3.x)
1087+
ddbc_sql_const.SQL_TYPE_TIMESTAMP.value: datetime.datetime, # 93 — datetime/datetime2/smalldatetime
1088+
ddbc_sql_const.SQL_SS_TIME2.value: datetime.time, # -154 — time
1089+
ddbc_sql_const.SQL_DATETIMEOFFSET.value: datetime.datetime, # -155 — datetimeoffset
1090+
# ODBC 2.x date/time aliases (defensive, in case any driver reports these)
1091+
ddbc_sql_const.SQL_DATE.value: datetime.date, # 9
1092+
ddbc_sql_const.SQL_TIME.value: datetime.time, # 10
1093+
ddbc_sql_const.SQL_TIMESTAMP.value: datetime.datetime, # 11
1094+
# Boolean
10481095
ddbc_sql_const.SQL_BIT.value: bool,
1049-
ddbc_sql_const.SQL_TINYINT.value: int,
1050-
ddbc_sql_const.SQL_SMALLINT.value: int,
1051-
ddbc_sql_const.SQL_BIGINT.value: int,
1096+
# Binary types
10521097
ddbc_sql_const.SQL_BINARY.value: bytes,
10531098
ddbc_sql_const.SQL_VARBINARY.value: bytes,
10541099
ddbc_sql_const.SQL_LONGVARBINARY.value: bytes,
1100+
# UUID
10551101
ddbc_sql_const.SQL_GUID.value: uuid.UUID,
1056-
# Add more mappings as needed
1102+
# XML — driver reports SQL_SS_XML (-152), fetched as str
1103+
ddbc_sql_const.SQL_SS_XML.value: str,
10571104
}
10581105
return sql_to_python_type.get(sql_type, str)
10591106

0 commit comments

Comments
 (0)