[PECOBLR-1377] SRID in geospatial column type name #1157

sreekanth-db · 2025-12-22T11:30:35Z

Description

This PR enhances geospatial datatype handling to include SRID (Spatial Reference System Identifier) information in column type names and fixes multiple issues related to complex datatype handling across different result formats.

Key Changes

Geospatial Type Name Enhancement
- Column type names now include SRID: GEOMETRY(4326) instead of GEOMETRY
- Applies to both GEOMETRY and GEOGRAPHY types
- Preserves full type information in metadata for better type identification
SEA Inline Mode Complex Type Fix
- Fixed issue where complex types (ARRAY, MAP, STRUCT) were not returned as complex objects in SEA Inline mode (JSON array result format)
- Now properly converts to complex datatype objects when EnableComplexDatatypeSupport=true
Thrift CloudFetch Metadata Enhancement
- Fixed error when extracting type details (e.g., INT from ARRAY<INT>) in Thrift CloudFetch mode
- Enhanced getColumnInfoFromTColumnDesc() to use Arrow schema metadata alongside TColumnDesc
- Arrow schema provides complete type information (e.g., ARRAY<INT>) while TColumnDesc only contains base type (e.g., ARRAY)
Arrow Metadata Extraction
- Added DatabricksThriftUtil.getArrowMetadata() to deserialize Arrow schema from TGetResultSetMetadataResp
- Fixed null arrow metadata issue in DatabricksResultSet constructor for Thrift CloudFetch mode

Testing

Unit Tests

All existing unit tests pass and additional tests are added for new methods

Integration Tests

GeospatialTests.java - Comprehensive E2E integration test
- Tests geospatial types (GEOMETRY and GEOGRAPHY)
- Validates 24 configuration combinations:
  - Protocol: Thrift / SEA
  - Serialization: Arrow / Inline
  - CloudFetch: Enabled / Disabled (only with Arrow, as CloudFetch requires Arrow)
  - GeoSpatial Support: Enabled / Disabled
  - Complex Type Support: Enabled / Disabled
- Validates metadata: column types, type names, class names
- Validates values: WKT representation, SRID
- Validates behavior when geospatial objects are enabled vs. disabled (STRING fallback)
- All 24 tests pass ✅

Additional Notes to the Reviewer

Other required details are mentioned in comments in the diff

Signed-off-by: Sreekanth Vadigi <sreekanth.vadigi@databricks.com>

sreekanth-db · 2025-12-22T13:07:15Z

src/main/java/com/databricks/jdbc/api/impl/converters/ArrowToJavaObjectConverter.java

    return parser.parseJsonStringToDbStruct(object.toString(), arrowMetadata);
  }

-  private static AbstractDatabricksGeospatial convertToGeospatial(


removing duplicate code, using the same logic present in GeospatialConverter

sreekanth-db · 2025-12-22T13:11:08Z

src/main/java/com/databricks/jdbc/api/impl/DatabricksResultSet.java

      long rowSize = executionResult.getRowCount();
-      List<String> arrowMetadata = null;
-      if (executionResult instanceof ArrowStreamResult) {
-        arrowMetadata = ((ArrowStreamResult) executionResult).getArrowMetadata();


((ArrowStreamResult) executionResult).getArrowMetadata() will always be null for cloud fetch mode at this point in the code flow. So getting arrow metadata from TGetResultSetMetadataResp

Will getResultSetMetadata always be present?

Ideally we should have TGetResultSetMetadataResp object in TFetchResultsResp. Anyways we are handling this case in getArrowMetadata method to check for null values

sreekanth-db · 2025-12-22T13:14:14Z

src/main/java/com/databricks/jdbc/api/impl/DatabricksResultSet.java

-   * @return true if the type name starts with ARRAY, MAP, STRUCT, GEOMETRY, or GEOGRAPHY, false
-   *     otherwise
-   */
-  private static boolean isComplexType(String typeName) {


moving this method to DatabricksTypeUtil for using in other classes

sreekanth-db · 2025-12-22T13:15:50Z

src/main/java/com/databricks/jdbc/api/impl/DatabricksResultSet.java

+    return complexDatatypeSupport ? obj : obj.toString();
  }

-  private Object handleComplexDataTypesForSEAInline(Object obj, String columnName)


Updating this method to always return complex datatype, if complex mode is disabled, caller will convert to string

sreekanth-db · 2025-12-22T13:17:12Z

src/main/java/com/databricks/jdbc/api/impl/DatabricksResultSetMetaData.java

            typeText = "STRING";
          }

+          // store base type eg. DECIMAL instead of DECIMAL(7,2) except for geospatial datatypes


For geospatial, we want SRID info (in paranthesis) to be present. eg: GEOMETRY(4326)

Will there be a case that SRID info might be missing?

I don't think so, even for SRID 0, we get GEOMETRY(0)

sreekanth-db · 2025-12-22T13:21:39Z

src/main/java/com/databricks/jdbc/api/impl/DatabricksResultSetMetaData.java

            columnIndex++) {
          TColumnDesc columnDesc = resultManifest.getSchema().getColumns().get(columnIndex);

-          ColumnInfo columnInfo = getColumnInfoFromTColumnDesc(columnDesc);


TColumnDesc doesn't have complete information about the column metadata, hence enhancing it using the arrowMetadata extracted from the TGetResultSetMetadataResp. This is required for methods which use ColumnInfo later in the flow (thrift + cloudfetch + complex)

sreekanth-db · 2025-12-22T13:23:23Z

src/main/java/com/databricks/jdbc/api/impl/DatabricksResultSetMetaData.java

                .columnTypeClassName("java.lang.String")
                .columnType(Types.OTHER)
                .columnTypeText(VARIANT);
-          } else if (isGeometryColumn(arrowMetadata, columnIndex)


This is handled with updates to the column info, no separate handling is required

Signed-off-by: Sreekanth Vadigi <sreekanth.vadigi@databricks.com>

gopalldb · 2026-01-01T08:39:48Z

src/main/java/com/databricks/jdbc/common/util/DatabricksThriftUtil.java

+
+    if (arrowMetadata != null && isComplexType(arrowMetadata)) {
+      typeText = arrowMetadata;
+      if (arrowMetadata.startsWith(GEOMETRY)) {


do we want to check if Geospatial flag is enabled?

This is an inner layer, so flag check is not required here. we are checking for geospatial flags in DatabricksResultSetMetaData which is the interface through which users get metadata info

gopalldb · 2026-01-01T08:41:24Z

src/main/java/com/databricks/jdbc/common/util/DatabricksThriftUtil.java

+          .map(e -> e.get(ARROW_METADATA_KEY))
+          .collect(Collectors.toList());
+    } catch (IOException e) {
+      throw new DatabricksSQLException(


add logging

added error log

…ents

Signed-off-by: Sreekanth Vadigi <sreekanth.vadigi@databricks.com>

sreekanth-db added 2 commits December 22, 2025 16:58

geospatial column type name with srid

d24c48e

Signed-off-by: Sreekanth Vadigi <sreekanth.vadigi@databricks.com>

unit tests

dcb3ee5

Signed-off-by: Sreekanth Vadigi <sreekanth.vadigi@databricks.com>

sreekanth-db commented Dec 22, 2025

View reviewed changes

updated next changelog

48e113d

Signed-off-by: Sreekanth Vadigi <sreekanth.vadigi@databricks.com>

sreekanth-db changed the title ~~geospatial column type name with srid~~ SRID in geospatial column type name Dec 22, 2025

sreekanth-db requested review from gopalldb, jayantsing-db, madhav-db and samikshya-db December 29, 2025 10:24

sreekanth-db changed the title ~~SRID in geospatial column type name~~ [PECOBLR-1377] SRID in geospatial column type name Dec 30, 2025

gopalldb reviewed Jan 1, 2026

View reviewed changes

sreekanth-db added 2 commits January 2, 2026 13:19

Merge remote-tracking branch 'upstream/main' into geospatial-enhancem…

d378d76

…ents

added error log

f3bf0d6

Signed-off-by: Sreekanth Vadigi <sreekanth.vadigi@databricks.com>

[PECOBLR-1377] SRID in geospatial column type name #1157

Are you sure you want to change the base?

[PECOBLR-1377] SRID in geospatial column type name #1157

Uh oh!

Conversation

sreekanth-db commented Dec 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Key Changes

Testing

Unit Tests

Integration Tests

Additional Notes to the Reviewer

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sreekanth-db Dec 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sreekanth-db Dec 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

sreekanth-db commented Dec 22, 2025 •

edited

Loading

sreekanth-db Dec 22, 2025 •

edited

Loading

sreekanth-db Dec 22, 2025 •

edited

Loading