Introduce SQLAlchemy reusable dialog tests (#125)

Signed-off-by: Jim Fulton <jim.fulton@unsupervised.com>
Co-Authored-By: Jesse Whitehouse <jesse.whitehouse@databricks.com>
Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com>

Author: unj1m

CHANGELOG.md

@@ -2,6 +2,8 @@
 
 ## 2.9.4 (Unreleased)
 
+- Other: Introduce SQLAlchemy dialect compliance test suite and enumerate all excluded tests
+
 ## 2.9.3 (2023-08-24)
 
 - Fix: Connections failed when urllib3~=1.0.0 is installed (#206)

CONTRIBUTING.md

@@ -107,6 +107,8 @@ End-to-end tests require a Databricks account. Before you can run them, you must
 export host=""
 export http_path=""
 export access_token=""
+export catalog=""
+export schema=""
 ```
 
 Or you can write these into a file called `test.env` in the root of the repository:
@@ -141,6 +143,21 @@ The `PySQLLargeQueriesSuite` namespace contains long-running query tests and is
 The `PySQLStagingIngestionTestSuite` namespace requires a cluster running DBR version > 12.x which supports staging ingestion commands.
 
 The suites marked `[not documented]` require additional configuration which will be documented at a later time.
+
+#### SQLAlchemy dialog tests
+
+SQLAlchemy provides reusable tests for testing dialect implementations.
+
+To run these tests, assuming the environment variables needed for e2e tests are set:
+
+```
+poetry run python -m pytest tests/sqlalchemy_dialect_compliance --dburi \
+  "databricks://token:$access_token@$host?http_path=$http_path&catalog=$catalog&schema=$schema" 
+```
+
+Some of these of these tests fail currently. We're working on getting
+relavent tests passing and others skipped.
+
 ### Code formatting
 
 This project uses [Black](https://pypi.org/project/black/).

pyproject.toml

@@ -62,3 +62,5 @@ log_cli = "false"
 log_cli_level = "INFO"
 testpaths = ["tests"]
 env_files = ["test.env"]
+addopts = "--ignore=tests/sqlalchemy_dialect_compliance"
+

setup.cfg

@@ -0,0 +1,4 @@
+
+[sqla_testing]
+requirement_cls=databricks.sqlalchemy.dialect.requirements:Requirements
+profile_file=profiles.txt

src/databricks/sqlalchemy/dialect/__init__.py

@@ -71,7 +71,7 @@ class DatabricksDialect(default.DefaultDialect):
 
     # Possible attributes are defined here: https://docs.sqlalchemy.org/en/14/core/internals.html#sqlalchemy.engine.Dialect
     name: str = "databricks"
-    driver: str = "databricks-sql-python"
+    driver: str = "databricks"
     default_schema_name: str = "default"
 
     preparer = DatabricksIdentifierPreparer  # type: ignore

src/databricks/sqlalchemy/dialect/requirements.py

@@ -0,0 +1,32 @@
+"""
+This module is supposedly used by the compliance tests to control which tests are run based on database capabilities.
+However, based on some experimentation that does not appear to be consistently the case. Until we better understand
+when these requirements are and are not implemented, we prefer to manually capture the exact nature of the failures
+and errors.
+
+Once we better understand how to use requirements.py, an example exclusion will look like this:
+
+    import sqlalchemy.testing.requirements
+    import sqlalchemy.testing.exclusions
+
+    class Requirements(sqlalchemy.testing.requirements.SuiteRequirements):
+        @property
+        def __some_example_requirement(self):
+            return sqlalchemy.testing.exclusions.closed
+
+
+The complete list of requirements is provided by SQLAlchemy here:
+
+https://github.com/sqlalchemy/sqlalchemy/blob/main/lib/sqlalchemy/testing/requirements.py
+"""
+
+import sqlalchemy.testing.requirements
+import sqlalchemy.testing.exclusions
+
+import logging
+logger = logging.getLogger(__name__)
+
+logger.warning("requirements.py is not currently employed by Databricks dialect")
+
+class Requirements(sqlalchemy.testing.requirements.SuiteRequirements):
+    pass

tests/sqlalchemy_dialect_compliance/conftest.py

@@ -0,0 +1,13 @@
+from sqlalchemy.dialects import registry
+import pytest
+
+registry.register("databricks", "databricks.sqlalchemy", "DatabricksDialect")
+# sqlalchemy's dialect-testing machinery wants an entry like this.
+# This seems to be based around dialects maybe having multiple drivers
+# and wanting to test driver-specific URLs, but doesn't seem to make
+# much sense for dialects with only one driver.
+registry.register("databricks.databricks", "databricks.sqlalchemy", "DatabricksDialect")
+
+pytest.register_assert_rewrite("sqlalchemy.testing.assertions")
+
+from sqlalchemy.testing.plugin.pytestplugin import *