Cleanup dev setup

- Removes buildout; test_layer now fetches the crate package itself
- Removes setup.py
- Removes stale bin/test
- Removes stale requirements.txt
- Updates DEVELOP.rst
- Removes poethepoet. Most python developers are familiar with `ruff`,
  `mypy` and `pytest`, no need to add an extra DSL on top that
  obfuscates what's being used.
- Removes `bootstrap.sh`. `python -m venv` or `uv venv` are trivial
  enough and Python developers should be familiar with them.
- Removes the `devtools/create_tag.sh` script. It used `python setup.py
  --version` to get the new version, which since
  #757 was broken - running
  into a chicken and egg problem: Can't read the version from git
  when about to create a new version tag. Must use `git tag -s
  <version>` instead.
- Updates the release job. It used `python -m build`, which wasn't
  specified as dependency anywhere. `hatch` is now used which `uv build`
  will pickup.

Author: mfussenegger

.github/workflows/docs.yml

@@ -1,5 +1,5 @@
+---
 name: docs
-
 on:
   workflow_dispatch:
   pull_request: ~
@@ -13,6 +13,11 @@ concurrency:
   group: ${{ github.workflow }}-${{ github.ref }}
   cancel-in-progress: true
 
+permissions:
+  contents: read
+  checks: write
+  statuses: write
+
 jobs:
   documentation:
 
@@ -22,13 +27,20 @@ jobs:
     steps:
       - name: Acquire sources
         uses: actions/checkout@v6
+        with:
+          persist-credentials: false
 
-      - name: Set up Python
-        uses: actions/setup-python@v6
+      - name: Set up uv
+        uses: astral-sh/setup-uv@v7
         with:
-          python-version: '3.12'
-          cache: 'pip'
-          cache-dependency-path: 'setup.py'
+          cache-dependency-glob: |
+            pyproject.toml
+          enable-cache: true
+          activate-environment: true
+          version: "latest"
+
+      - name: Setup env
+        run: uv pip install --group docs
 
       - name: Build docs
         run: |

.github/workflows/nightly.yml

@@ -3,6 +3,11 @@ name: release
 
 on: push
 
+permissions:
+  contents: read
+  checks: write
+  statuses: write
+
 jobs:
   pypi:
     name: Build & publish package to pypi
@@ -15,11 +20,8 @@ jobs:
     if: startsWith(github.event.ref, 'refs/tags')
     steps:
       - uses: actions/checkout@v6
-
-      - name: Set up Python
-        uses: actions/setup-python@v6
         with:
-          python-version: ${{ matrix.python-version }}
+          persist-credentials: false
 
       - name: Set up uv
         uses: astral-sh/setup-uv@v7
@@ -31,10 +33,7 @@ jobs:
           version: "latest"
 
       - name: Build package
-        run: |
-          uv pip install --system build twine wheel
-          python -m build
-          twine check dist/*
+        run: uv build
 
       - name: Publish package to PyPI
         uses: pypa/gh-action-pypi-publish@release/v1

.github/workflows/release.yml

@@ -1,15 +1,24 @@
+---
 name: Tests
 
 on:
   push:
-    branches: [ main ]
+    branches:
+      - main
   pull_request: ~
   workflow_dispatch:
+  schedule:
+    - cron: '0 2 * * *'
 
 concurrency:
   group: ${{ github.workflow }}-${{ github.ref }}
   cancel-in-progress: true
 
+permissions:
+  contents: read
+  checks: write
+  statuses: write
+
 jobs:
   test:
     name: "Python: ${{ matrix.python-version }}
@@ -19,32 +28,24 @@ jobs:
       fail-fast: false
       matrix:
         os: ['ubuntu-22.04']
-        python-version: ['3.9', '3.10', '3.11', '3.12', '3.13', '3.14']
-        cratedb-version: ['nightly']
+        python-version: ['3.10', '3.11', '3.12', '3.13', '3.14']
 
-        # To save resources, only verify the most recent Python versions on macOS.
+        # Save resources - only verify the most recent Python versions on macOS.
         include:
           - os: 'macos-latest'
-            cratedb-version: '5.9.2'
             python-version: '3.11'
           - os: 'macos-latest'
-            cratedb-version: '5.9.2'
             python-version: '3.12'
           - os: 'macos-latest'
-            cratedb-version: '5.9.2'
             python-version: '3.13'
 
     env:
-      CRATEDB_VERSION: ${{ matrix.cratedb-version }}
       CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
 
     steps:
       - uses: actions/checkout@v6
-
-      - name: Set up Python
-        uses: actions/setup-python@v6
         with:
-          python-version: ${{ matrix.python-version }}
+          persist-credentials: false
 
       - name: Set up uv
         uses: astral-sh/setup-uv@v7
@@ -53,30 +54,24 @@ jobs:
             pyproject.toml
           cache-suffix: ${{ matrix.python-version }}
           enable-cache: true
+          activate-environment: true
           version: "latest"
 
       - name: Setup env
-        run: uv sync
+        run: uv pip install --group dev .
 
-      - name: Invoke tests
+      - name: Run Linters
         run: |
-        
-          # Propagate build matrix information.
-          ./devtools/setup_ci.sh
-
-          # Bootstrap environment.
-          source bootstrap.sh
+          ruff check .
+          mypy
 
-          # Report about the test matrix slot.
-          echo "Invoking tests with CrateDB ${CRATEDB_VERSION}"
-
-          # Run linter.
-          poe lint
-
-          # Run tests.
-          poe test
+      - name: Run tests
+        run: |
+          coverage run -m pytest
+          coverage combine
+          coverage xml
+          coverage report
 
-      # https://github.com/codecov/codecov-action
       - name: Upload coverage results to Codecov
         uses: codecov/codecov-action@v5
         env:

.github/workflows/tests.yml

@@ -5,95 +5,49 @@ CrateDB Python developer guide
 Setup
 =====
 
-Optionally install Python package and project manager `uv`_,
-in order to significantly speed up the package installation::
-
-    {apt,brew,pip,zypper} install uv
-    alias pip="uv pip"
-
-To start things off, bootstrap the sandbox environment::
+Clone the repository::
 
     git clone https://github.com/crate/crate-python
     cd crate-python
-    source bootstrap.sh
-
-This command should automatically install all prerequisites for the development
-sandbox and drop you into the virtualenv, ready for invoking further commands.
-
-
-Running tests
-=============
-
-All tests will be invoked using the Python interpreter that was used when
-creating the Python virtualenv. The test runner is `zope.testrunner`_.
-
-Some examples are outlined below. In order to learn about more details,
-see, for example, `useful command-line options for zope-testrunner`_.
-
-Run all tests::
 
-    poe test
+Setup a virtualenv and install the package::
 
-Run specific tests::
+    python -m venv .venv
+    source .venv/bin/activate
+    python -m pip install --group dev --group docs -e .
 
-    # Select modules.
-    bin/test -t test_cursor
-    bin/test -t client
-    bin/test -t testing
+Or if using `uv`_::
 
-    # Select doctests.
-    bin/test -t http.rst
+    uv venv .venv
+    source .venv/bin/activate
+    uv pip install --group dev --group docs -e .
 
-Ignore specific test directories::
 
-    bin/test --ignore_dir=testing
-
-The ``LayerTest`` test cases have quite some overhead. Omitting them will save
-a few cycles (~70 seconds runtime)::
-
-    bin/test -t '!LayerTest'
-
-Invoke all tests without integration tests (~10 seconds runtime)::
-
-    bin/test --layer '!crate.testing.layer.crate' --test '!LayerTest'
-
-Yet ~60 test cases, but only ~1 second runtime::
+Running tests
+=============
 
-    bin/test --layer '!crate.testing.layer.crate' --test '!LayerTest' \
-        -t '!test_client_threaded' -t '!test_no_retry_on_read_timeout' \
-        -t '!test_wait_for_http' -t '!test_table_clustered_by'
+Ensure the virtualenv is active and run tests using `pytest`_::
 
-To inspect the whole list of test cases, run::
+    python -m pytest
 
-    bin/test --list-tests
 
-The CI setup on GitHub Actions (GHA) provides a full test matrix covering
-relevant Python versions. You can invoke the software tests against a specific
-Python interpreter or multiple `Python versions`_ on your workstation using
-`uv`_, by supplying the ``--python`` command-line option, or by defining the
-`UV_PYTHON`_ environment variable prior to invoking ``source bootstrap.sh``.
+See also:
 
-*Note*: Before running the tests, make sure to stop all CrateDB instances which
-are listening on the default CrateDB transport port to avoid side effects with
-the test layer.
+- `How to invoke pytest <https://docs.pytest.org/en/stable/how-to/usage.html>` for more information.
 
 
 Formatting and linting code
 ===========================
 
-To use Ruff for code formatting, according to the standards configured in
-``pyproject.toml``, use::
-
-    poe format
-
-To lint the code base using Ruff and mypy, use::
+Use `ruff`_ for code formatting and linting::
 
-    poe lint
+    ruff format --check .
+    ruff check .
 
-Linting and software testing, all together now::
 
-    poe check
+Use ``mypy`` for type checking::
 
+    mypy
 
 Renew certificates
 ==================
@@ -120,8 +74,8 @@ In the release branch:
 
 - Push to origin/<release_branch>
 
-- Create a tag by running ``./devtools/create_tag.sh``. This will trigger a
-  Github action which releases the new version to PyPi.
+- Create a tag by running ``git tag -s <version>`` and push it ``git push --tags``.
+  This will trigger a Github action which releases the new version to PyPi.
 
 On branch ``main``:
 
@@ -147,7 +101,7 @@ Writing documentation
 
 The docs live under the ``docs`` directory.
 
-The docs are written written with ReStructuredText_ and processed with Sphinx_.
+The docs are written with ReStructuredText_ and processed with Sphinx_.
 
 Build the docs by running::
 
@@ -171,4 +125,4 @@ nothing special you need to do to get the live docs to update.
 .. _uv: https://docs.astral.sh/uv/
 .. _UV_PYTHON: https://docs.astral.sh/uv/configuration/environment/#uv_python
 .. _versions hosted on ReadTheDocs: https://readthedocs.org/projects/crate-python/versions/
-.. _zope.testrunner: https://pypi.org/project/zope.testrunner/
+.. _pytest: https://docs.pytest.org/en/stable/