Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
ecd97fd
chore: spec0 compat (python 3.14 compat, numpy 2 min)
ilan-gold Nov 1, 2025
9ba21fc
fix: point oh
ilan-gold Nov 1, 2025
ae98bdd
fix: `universal_pathlib` min bound
ilan-gold Nov 1, 2025
d5e6d44
fix: new generic syntax (leave `covariant` untouched)
ilan-gold Nov 2, 2025
7ab6c47
feat: `uv` in `hatch`
ilan-gold Nov 2, 2025
98404bd
fix: try covariance handling
ilan-gold Nov 2, 2025
4826f68
Merge branch 'main' into ig/spec0_py314
ilan-gold Nov 3, 2025
86095bb
fix: allow covariance in `ZDType`
ilan-gold Nov 5, 2025
b304658
fix: remove covariance in `common.py`
ilan-gold Nov 5, 2025
e0e362e
fix: remove covariance in `codec.py`
ilan-gold Nov 5, 2025
dac7adc
Merge branch 'ig/spec0_py314' of github.com:ilan-gold/zarr-python int…
ilan-gold Nov 5, 2025
308a900
fix: remove unused `TypeVar`
ilan-gold Nov 5, 2025
528630a
Merge branch 'main' into ig/spec0_py314
ilan-gold Nov 5, 2025
0b66351
fix: bye byte `covariant`!
ilan-gold Nov 5, 2025
6b458b1
Merge branch 'main' into ig/spec0_py314
ilan-gold Nov 6, 2025
9ecb946
Merge branch 'main' into ig/spec0_py314
ilan-gold Nov 20, 2025
abfd77d
Merge branch 'main' into ig/spec0_py314
ilan-gold Nov 25, 2025
eb9121e
fix: merge issue
ilan-gold Nov 25, 2025
1cb2ef3
fix: more merge issues
ilan-gold Nov 25, 2025
00afc6c
fix: universal_pathlib
ilan-gold Nov 25, 2025
b934471
Merge branch 'main' into ig/spec0_py314
ilan-gold Nov 28, 2025
20ba466
Merge branch 'main' into ig/spec0_py314
ilan-gold Jan 2, 2026
ca2d9d1
Merge branch 'main' into ig/spec0_py314
ilan-gold Jan 13, 2026
41d1fe2
fix: pre-commit
ilan-gold Jan 13, 2026
f234ebb
fix: no more needed to pin numpy
ilan-gold Jan 13, 2026
cad6d92
chore: relnote
ilan-gold Jan 13, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/ISSUE_TEMPLATE/bug_report.yml
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,7 @@ body:
value: |
```python
# /// script
# requires-python = ">=3.11"
# requires-python = ">=3.12"
# dependencies = [
# "zarr@git+https://github.com/zarr-developers/zarr-python.git@main",
# ]
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/gpu_test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ jobs:
runs-on: gpu-runner
strategy:
matrix:
python-version: ['3.11']
python-version: ['3.12']
numpy-version: ['2.2']
dependency-set: ["minimal"]

Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/nightly_wheels.yml
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ jobs:
- uses: actions/setup-python@v6
name: Install Python
with:
python-version: '3.13'
python-version: '3.14'

- name: Install build dependencies
run: |
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/releases.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ jobs:
- uses: actions/setup-python@v6
name: Install Python
with:
python-version: '3.11'
python-version: '3.12'

- name: Install PyBuild
run: |
Expand Down
22 changes: 11 additions & 11 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -20,24 +20,24 @@ jobs:

strategy:
matrix:
python-version: ['3.11', '3.12', '3.13']
numpy-version: ['1.26', '2.2']
python-version: ['3.12', '3.13', '3.14']
numpy-version: ['2.0', '2.2']
dependency-set: ["minimal", "optional"]
os: ["ubuntu-latest"]
include:
- python-version: '3.11'
numpy-version: '1.26'
- python-version: '3.12'
numpy-version: '2.0'
dependency-set: 'optional'
os: 'macos-latest'
- python-version: '3.13'
- python-version: '3.14'
numpy-version: '2.2'
dependency-set: 'optional'
os: 'macos-latest'
- python-version: '3.11'
numpy-version: '1.26'
- python-version: '3.12'
numpy-version: '2.0'
dependency-set: 'optional'
os: 'windows-latest'
- python-version: '3.13'
- python-version: '3.14'
numpy-version: '2.2'
dependency-set: 'optional'
os: 'windows-latest'
Expand Down Expand Up @@ -78,12 +78,12 @@ jobs:
runs-on: ubuntu-latest
strategy:
matrix:
python-version: ['3.11', "3.13"]
python-version: ['3.12', "3.14"]
dependency-set: ["upstream", "min_deps"]
exclude:
- python-version: "3.13"
- python-version: "3.14"
dependency-set: min_deps
- python-version: "3.11"
- python-version: "3.12"
dependency-set: upstream
steps:
- uses: actions/checkout@v6
Expand Down
4 changes: 2 additions & 2 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ ci:
default_stages: [pre-commit, pre-push]

default_language_version:
python: python3.11
python: python3.14

repos:
- repo: https://github.com/astral-sh/ruff-pre-commit
Expand Down Expand Up @@ -37,7 +37,7 @@ repos:
- donfig
- numcodecs
- google-crc32c>=1.5
- numpy==2.1 # until https://github.com/numpy/numpy/issues/28034 is resolved
- numpy
- typing_extensions
- universal-pathlib
- obstore>=0.5.1
Expand Down
1 change: 1 addition & 0 deletions changes/3546.misc.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Upgrade to spec0 compat (python 3.14 max, python 3.12 min).
2 changes: 1 addition & 1 deletion docs/contributing.md
Original file line number Diff line number Diff line change
Expand Up @@ -155,7 +155,7 @@ When submitting a pull request, coverage will also be collected across all suppo

### Documentation

Docstrings for user-facing classes and functions should follow the [numpydoc](https://numpydoc.readthedocs.io/en/stable/format.html#docstring-standard) standard, including sections for Parameters and Examples. All examples should run and pass as doctests under Python 3.11.
Docstrings for user-facing classes and functions should follow the [numpydoc](https://numpydoc.readthedocs.io/en/stable/format.html#docstring-standard) standard, including sections for Parameters and Examples. All examples should run and pass as doctests under Python 3.12.

Zarr uses mkdocs for documentation, hosted on readthedocs.org. Documentation is written in the Markdown markup language (.md files) in the `docs` folder. The documentation consists both of prose and API documentation. All user-facing classes and functions are included in the API documentation, under the `docs/api` folder using the [mkdocstrings](https://mkdocstrings.github.io/) extension. Add any new public functions or classes to the relevant markdown file in `docs/api/*.md`. Any new features or important usage information should be included in the user-guide (`docs/user-guide`). Any changes should also be included as a new file in the `changes` directory.

Expand Down
2 changes: 1 addition & 1 deletion docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ Zarr-Python is a Python library for reading and writing Zarr groups and arrays.

## Installation

Zarr requires Python 3.11 or higher. You can install it via `pip`:
Zarr requires Python 3.12 or higher. You can install it via `pip`:

```bash
pip install zarr
Expand Down
4 changes: 2 additions & 2 deletions docs/user-guide/installation.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,9 @@

Required dependencies include:

- [Python](https://docs.python.org/3/) (3.11 or later)
- [Python](https://docs.python.org/3/) (3.12 or later)
- [packaging](https://packaging.pypa.io) (22.0 or later)
- [numpy](https://numpy.org) (1.26 or later)
- [numpy](https://numpy.org) (2.0 or later)
- [numcodecs](https://numcodecs.readthedocs.io) (0.14 or later)
- [google-crc32c](https://github.com/googleapis/python-crc32c) (1.5 or later)
- [typing_extensions](https://typing-extensions.readthedocs.io) (4.9 or later)
Expand Down
2 changes: 1 addition & 1 deletion examples/custom_dtype/custom_dtype.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# /// script
# requires-python = ">=3.11"
# requires-python = ">=3.12"
# dependencies = [
# "zarr @ git+https://github.com/zarr-developers/zarr-python.git@main",
# "ml_dtypes==0.5.1",
Expand Down
27 changes: 15 additions & 12 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -29,11 +29,11 @@ maintainers = [
{ name = "Tom Augspurger", email = "tom.w.augspurger@gmail.com" },
{ name = "Deepak Cherian" }
]
requires-python = ">=3.11"
requires-python = ">=3.12"
# If you add a new dependency here, please also add it to .pre-commit-config.yml
dependencies = [
'packaging>=22.0',
'numpy>=1.26',
'numpy>=2',
'numcodecs>=0.14',
'google-crc32c>=1.5',
'typing_extensions>=4.12',
Expand All @@ -52,9 +52,9 @@ classifiers = [
'Topic :: Software Development :: Libraries :: Python Modules',
'Operating System :: Unix',
'Programming Language :: Python :: 3',
'Programming Language :: Python :: 3.11',
'Programming Language :: Python :: 3.12',
'Programming Language :: Python :: 3.13',
'Programming Language :: Python :: 3.14',
]
license = "MIT"
license-files = ["LICENSE.txt"]
Expand Down Expand Up @@ -162,8 +162,8 @@ COV_CORE_CONFIG = ".coveragerc"
COV_CORE_DATAFILE = ".coverage.eager"

[[tool.hatch.envs.test.matrix]]
python = ["3.11", "3.12", "3.13"]
numpy = ["1.26", "2.2"]
python = ["3.12", "3.13", "3.14"]
numpy = ["2.0", "2.2"]
deps = ["minimal", "optional"]

[tool.hatch.envs.test.overrides]
Expand Down Expand Up @@ -191,8 +191,8 @@ dependencies = [
features = ["test", "gpu"]

[[tool.hatch.envs.gputest.matrix]]
python = ["3.11", "3.12", "3.13"]
numpy = ["1.26", "2.2"]
python = ["3.12", "3.13"]
numpy = ["2.0", "2.2"]
version = ["minimal"]

[tool.hatch.envs.gputest.scripts]
Expand All @@ -205,7 +205,7 @@ list-env = "pip list"

[tool.hatch.envs.upstream]
template = 'test'
python = "3.13"
python = "3.14"
dependencies = [
'packaging @ git+https://github.com/pypa/packaging',
'numpy', # from scientific-python-nightly-wheels
Expand All @@ -230,15 +230,15 @@ description = """Test environment for minimum supported dependencies
See Spec 0000 for details and drop schedule: https://scientific-python.org/specs/spec-0000/
"""
template = "test"
python = "3.11"
python = "3.12"
dependencies = [
'zarr[remote]',
'packaging==22.*',
'numpy==1.26.*',
'numpy==2.0.*',
'numcodecs==0.14.*', # 0.14 needed for zarr3 codecs
'fsspec==2023.10.0',
's3fs==2023.10.0',
'universal_pathlib==0.0.22',
'universal_pathlib==0.2.0',
'typing_extensions==4.12.*',
'donfig==0.8.*',
'obstore==0.5.*',
Expand All @@ -247,6 +247,9 @@ dependencies = [
'zarr[remote_tests]',
]

[tool.hatch.envs.defaults]
installer = "uv"

[tool.hatch.envs.docs]
features = ['docs', 'remote']

Expand Down Expand Up @@ -351,7 +354,7 @@ ignore = [
"tests/**" = ["ANN001", "ANN201", "RUF029", "SIM117", "SIM300"]

[tool.mypy]
python_version = "3.11"
python_version = "3.12"
ignore_missing_imports = true
namespace_packages = false

Expand Down
6 changes: 2 additions & 4 deletions src/zarr/_compat.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
from collections.abc import Callable
from functools import wraps
from inspect import Parameter, signature
from typing import TYPE_CHECKING, Any, TypeVar
from typing import TYPE_CHECKING, Any

import numpy as np
from packaging.version import Version
Expand All @@ -12,12 +12,10 @@
if TYPE_CHECKING:
from numpy.typing import NDArray

T = TypeVar("T")

# Based off https://github.com/scikit-learn/scikit-learn/blob/e87b32a81c70abed8f2e97483758eb64df8255e9/sklearn/utils/validation.py#L63


def _deprecate_positional_args(
def _deprecate_positional_args[T](
func: Callable[..., T] | None = None, *, version: str = "3.1.0"
) -> Callable[..., T]:
"""Decorator for methods that issues warnings for positional arguments.
Expand Down
46 changes: 21 additions & 25 deletions src/zarr/abc/codec.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

from abc import abstractmethod
from collections.abc import Mapping
from typing import TYPE_CHECKING, Generic, TypeGuard, TypeVar
from typing import TYPE_CHECKING, TypeGuard

from typing_extensions import ReadOnly, TypedDict

Expand Down Expand Up @@ -34,13 +34,11 @@
"CodecPipeline",
]

CodecInput = TypeVar("CodecInput", bound=NDBuffer | Buffer)
CodecOutput = TypeVar("CodecOutput", bound=NDBuffer | Buffer)
type CodecInput = NDBuffer | Buffer
type CodecOutput = NDBuffer | Buffer

TName = TypeVar("TName", bound=str, covariant=True)


class CodecJSON_V2(TypedDict, Generic[TName]):
class CodecJSON_V2[TName: str](TypedDict):
"""The JSON representation of a codec for Zarr V2"""

id: ReadOnly[TName]
Expand All @@ -59,7 +57,7 @@ def _check_codecjson_v2(data: object) -> TypeGuard[CodecJSON_V2[str]]:
"""The widest type of JSON-like input that could specify a codec."""


class BaseCodec(Metadata, Generic[CodecInput, CodecOutput]):
class BaseCodec[CI: CodecInput, CO: CodecOutput](Metadata):
"""Generic base class for codecs.

Codecs can be registered via zarr.codecs.registry.
Expand Down Expand Up @@ -137,13 +135,13 @@ def validate(
The array chunk grid
"""

async def _decode_single(self, chunk_data: CodecOutput, chunk_spec: ArraySpec) -> CodecInput:
async def _decode_single(self, chunk_data: CO, chunk_spec: ArraySpec) -> CI:
raise NotImplementedError # pragma: no cover

async def decode(
self,
chunks_and_specs: Iterable[tuple[CodecOutput | None, ArraySpec]],
) -> Iterable[CodecInput | None]:
chunks_and_specs: Iterable[tuple[CO | None, ArraySpec]],
) -> Iterable[CI | None]:
"""Decodes a batch of chunks.
Chunks can be None in which case they are ignored by the codec.

Expand All @@ -154,25 +152,23 @@ async def decode(

Returns
-------
Iterable[CodecInput | None]
Iterable[CI | None]
"""
return await _batching_helper(self._decode_single, chunks_and_specs)

async def _encode_single(
self, chunk_data: CodecInput, chunk_spec: ArraySpec
) -> CodecOutput | None:
async def _encode_single(self, chunk_data: CI, chunk_spec: ArraySpec) -> CO | None:
raise NotImplementedError # pragma: no cover

async def encode(
self,
chunks_and_specs: Iterable[tuple[CodecInput | None, ArraySpec]],
) -> Iterable[CodecOutput | None]:
chunks_and_specs: Iterable[tuple[CI | None, ArraySpec]],
) -> Iterable[CO | None]:
"""Encodes a batch of chunks.
Chunks can be None in which case they are ignored by the codec.

Parameters
----------
chunks_and_specs : Iterable[tuple[CodecInput | None, ArraySpec]]
chunks_and_specs : Iterable[tuple[CI | None, ArraySpec]]
Ordered set of to-be-encoded chunks with their accompanying chunk spec.

Returns
Expand Down Expand Up @@ -460,21 +456,21 @@ async def write(
...


async def _batching_helper(
func: Callable[[CodecInput, ArraySpec], Awaitable[CodecOutput | None]],
batch_info: Iterable[tuple[CodecInput | None, ArraySpec]],
) -> list[CodecOutput | None]:
async def _batching_helper[CI: CodecInput, CO: CodecOutput](
func: Callable[[CI, ArraySpec], Awaitable[CO | None]],
batch_info: Iterable[tuple[CI | None, ArraySpec]],
) -> list[CO | None]:
return await concurrent_map(
list(batch_info),
_noop_for_none(func),
config.get("async.concurrency"),
)


def _noop_for_none(
func: Callable[[CodecInput, ArraySpec], Awaitable[CodecOutput | None]],
) -> Callable[[CodecInput | None, ArraySpec], Awaitable[CodecOutput | None]]:
async def wrap(chunk: CodecInput | None, chunk_spec: ArraySpec) -> CodecOutput | None:
def _noop_for_none[CI: CodecInput, CO: CodecOutput](
func: Callable[[CI, ArraySpec], Awaitable[CO | None]],
) -> Callable[[CI | None, ArraySpec], Awaitable[CO | None]]:
async def wrap(chunk: CI | None, chunk_spec: ArraySpec) -> CO | None:
if chunk is None:
return None
return await func(chunk, chunk_spec)
Expand Down
4 changes: 1 addition & 3 deletions src/zarr/abc/numcodec.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,4 @@
from typing import Any, Self, TypeGuard

from typing_extensions import Protocol
from typing import Any, Protocol, Self, TypeGuard


class Numcodec(Protocol):
Expand Down
4 changes: 2 additions & 2 deletions src/zarr/abc/store.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
if TYPE_CHECKING:
from collections.abc import AsyncGenerator, AsyncIterator, Iterable
from types import TracebackType
from typing import Any, Self, TypeAlias
from typing import Any, Self

from zarr.core.buffer import Buffer, BufferPrototype

Expand Down Expand Up @@ -42,7 +42,7 @@ class SuffixByteRequest:
"""The number of bytes from the suffix to request."""


ByteRequest: TypeAlias = RangeByteRequest | OffsetByteRequest | SuffixByteRequest
type ByteRequest = RangeByteRequest | OffsetByteRequest | SuffixByteRequest


class Store(ABC):
Expand Down
Loading
Loading