Documentation and typing of array types, dtypes, and dimensionality/shape

[mdhaber]

TLDR: Especially as libraries begin to support alternative backends vis the Array API, it might be useful to have a standard format documenting the type, dtype, and dimensionality/shape of array inputs and corresponding outputs (and a way of adding typing information to the code). I thought the summit might be a good place to discuss the topic potentially prepare a SPEC.

---

In SciPy, at least, the term "array-like" (without qualification) is commonly used to document the parameter type of functions. I'll give an example from stats.chatterjeexi, which is about on par with other scipy.stats functions in terms of detail.

"array-like", used throughout SciPy, is far too broad. I don't think it is defined anywhere in our documentation. The most "official" definition of "array_like" I can find is from the NumPy glossary:

Clearly "Any argument accepted by numpy.array is array_like." is not a useful working definition, as almost any Python object is accepted by np.array and coerced to an object array For example, the module numpy is array_like, according to this definition.

import numpy as np
np.array(np)
# array(<module 'numpy' from '/Users/matthaberland/miniforge3/envs/scipy-dev/lib/python3.13/site-packages/numpy/__init__.py'>,
#      dtype=object)

As for the object type, what_ we really mean these days (for many functions in stats, for example) is that the type should be one of the following:

• A Python list, which will be converted to a NumPy array (but this may change in SciPy 2.0)
• An Array-API compatible array subject to limitations that appear in a table in the documentation

For example, a table given in the ttest_ind documentation looks like:

This does not mention two other important pieces of information, which are often omitted: dtype and shape. For dtype, sometimes "real floating" or similar is specified, but many scipy.stats functions assume the reader understands that the input must be real. Many elementwise and reducing functions are pretty flexible about allowed input shape, but even in that case, it would be useful to have a standard way of specifying the relationship between input and output shapes.

We have similar issues with the documentation of return values. Instead of "array-like", we often see float or "scalar or ndarray", which doesn't capture the full story.

I think we need:

• a term that means "array type from a backend that complies with the Python array API standard". (The array API standard docs just use "array", which might be fine, but I can see arguments that it will be confused with an informal, array-like definition.)
• a standard format for documenting allowed dtypes.
• a standard format for documenting the most shape requirements.
• a standard format for the documenting the most common relationships between input and output type, dtype, and shape

The last part is probably the most complex. For instance, in SciPy, it is common to have:

• Elementwise functions, which typically accept array-API compatible arrays of any numerical dtype and shape. Usually:
  • Output type = input type, although sometimes the output may technically be a different type from the same backend that is more or less compatible with other arrays from that backend. (For example, scipy.special functions may accept a 0-d NumPy array and produce a NumPy scalar. Whether this is OK is debatable - see Is Array-In -> Scalar-Out OK? #38 - but there may be other backends which use multiple types to implement the overall array protocol.)
  • The dtype of the output matches the result_type of the input(s) with exceptions (e.g. integers may be promoted to floating point)
  • The shape of the output matches the shape of the input.
• Reducing functions, which follow similar rules as above, but reduce along the axis (or axes) specified by an axis argument; the output shape matches that of the input but eliminates these dimensions
• Generalized ufuncs like those in scipy.linalg, which follow similar rules as above, which preserve the "batch shape" between and outuput but may may have complicated input -> output "core dimension" relationships, which could be specified in terms of shapes like (m,n),(n,p)->(m,p) (but often aren't!)

I'd suggest that we can provide some common language for cases like these, which libraries can adapt to their needs. We would also suggest a way to link to more information within a libary's documentation, since it is very common for input/output rules to be complicated but at least consistent within a certain set of functions. For instance, essentially all scipy.linalg functions now have a standard note that links to a tutorial about batch operations.

I think this is a lot more compact/readable (and TBH, more useful) than to spelll out all the rules in the documentation of every function. This might be a decent pattern to follow for documenting the relationship between input and output shapes and dtypes (e.g. give information for a representative case in the documentation, and link to a full set of common rules).

[mdhaber]

The Generalized Universal Function API and NEP 5 may be useful references.

[mdhaber]

This afternoon, @pfackeldey and I discussed what this might look like.

TLDR: The system attempts to balance the information presented in various places.

1. The one-line type summary of parameters and return values is a concise - but not 100% precise - description of input/output backend/device/dtype/shape.
2. A project-specific class/device/dtype/shape document will include general information for interpreting the one-line type summaries. The SPEC gives an example, which projects will likely modify according to their actual behavior. For instance, the SPEC might define an "array" class to be "a fundamental class used by a backend in its implementation of the Array API Protocol, e.g. numpy.ndarray, torch.tensor." If a project also, say, accepts Python iterables (e.g. lists) and coerces them to NumPy arrays, they would add that information to the project-specific documentation.
3. The full description of a parameter/return value or the Notes section of function/method/class documentation will note any departure from the project-specific documentation.

For example, the one-line type summaries of parameters and return values for a simple elementwise addition function lambda x, y: x + y might look like:

x : array, any device, numeric, (...)
y : array, any device, numeric, (...)
z : array, any device, floating, (...)

A more complex example like scipy.linalg.solve, once it is able to dispatch to GPU backends like CuPy:

A : array, {cpu, gpu}, floating, (..., N, N)
b : array, {cpu, gpu}, floating, (..., N) or (..., N, M)
x : array, {cpu, gpu}, floating, (..., N) or (..., N, M)

A reducing function like scipy.stats.ttest_ind:

x : array, {cpu, gpu}, real numeric, (..., N0, ..., Nm, ...)
y : array, {cpu, gpu}, real numeric, (..., M0, ..., Mm, ...)
axis : tuple[int, ...]
statistic : array, (cpu, gpu), real floating, (...)
pvalue : array, (cpu, gpu), real floating, (...)

The latter functions would need compatibility tables (see below) to describe the compatible combinations of array backend and device.

---

Some preliminary thoughts on the SPEC:

• This document recommends a format and language for array input/output type descriptions.
• input/output type descriptions will document accepted input and corresponding output
  • backend,
  • device,
  • dtype, and
  • shape
    of all arrays as a comma-separated list.
• Unless otherwise specified, the function/method/class accepts
  • arrays of compatible classes from the same backend
  • that reside on the same device
  • with any combination of specified dtypes and shapes.
• Unless otherwise specified, the returned arrays will:
  • be of the same backend as the inputs,
  • reside on the same device as the inputs,
  • have the result dtype (as computed by the backend's result_type function) of all array and Python scalar inputs,
  • have the specified shape.
• backend
  • The names of supported libraries that implement the Array API Standard.
  • Specified as a set; e.g. {numpy, torch, jax, dask}.
  • Alternatively, use "array" to denote "any backend library that implements the Array API Standard protocol".
• device
  • The devices on which an array may reside.
  • Specified as a set; e.g. {cpu, gpu, tpu}
  • Alternatively, use "any device" to denote any device supported by the backend.
• dtype
  • The supported dtypes of arrays.
  • Specified as a set; e.g. {float64, complex128}.
  • Alternatively, use strings specified in the Array API Standard isdtype documentation. Use floating to refer to {real floating, complex floating}, and any dtype.
  • Projects may indicate support for additional dtypes (i.e. not specified in the array API standard) using the compatibility table in the Notes.
• shape
  • The supported shapes of arrays.
  • Specified as a tuple, e.g. the shape (..., M, N, 3) might by used to represent a stack of 2D three-channel images.
  • Use ellipses to represent any number of batch dimensions.
  • Use a variable (e.g. M or N) to represent the integer length of a core dimension. Use the same variable in the shape of another input/output to refer to the same integer length.
  • Use variables separated by ellipses to represent any mixture of core and batch dimensions, e.g. (N0, ..., Nm). Typically, a user provided integer or tuple axis identifies location of core dimensions within the shape tuple. The shape of result arrays is typically calculated according to gufunc rules of eliminating core dimensions and broadcasting the remaining batch dimensions.
• It is not expected that all possible combinations of array backends and devices will be supported. In this case, projects can specify backend "array" and device "any device" but use a "Compatibilty table" like the one below to document the supported combinations of backends, devices, and features in the Notes section of the function/method/class documentation. As desired, projects can extend these tables to document the compatiblility of array backends and devices with backend wrappers (e.g. marray), dtypes, and shapes.
• We do not attempt to make this specification absolutely precise; we currently rely on some implicit understanding of Array API Standard broadcasting rules. Use this SPEC as a guide and respond to user feedback that indicates a lack of clarity by making improvements as needed.

Thoughts:

• I'm a little unsure that "array" corresponds with "backend". Maybe instead of "array", I want to write "any backend".
• Alternatively, we could eliminate separate "backend" and "device" descriptions from the standard and just use the term array to represent "any fundamental class of a library that implements the Array API Standard". The compatibility table would take care of the rest.
• It's time to sleep!

Details

[j-bowhay]

I don't really keep up with the typing world but it seems like a lot of this information must already exist in, for example, scipy-stubs. I wonder if there is a way this could be utilized rather than essentially having two copies of the same information.

[mdhaber]

Sure, I'll take a look at what is going on there.

[mdhaber]

We discussed this today. Present: @rossbar @seberg @pfackeldey @eriknw @j-bowhay @ksunden @mihaimaruseac @kne42. I will finish this comment later.

Some key takeaways:

• no need to write devices in description if they're handled in table
• preference for English-like rather than (incomplete) robotic, typing-like descriptions; prefer "(N, M) array of real or complex floating" vs "array, {real floating, complex floating}, (N, M)" or "array[{real floating, complex floating}, (N, M)]"
• Libraries can do what they want, but be self-consistent, and if you want to be consistent with more libraries in the ecosystem, follow these recommendations.
• Descriptions should be useful to new contributors, who are the ones who actually read the docs?

[mdhaber]

@rossbar @seberg @pfackeldey @eriknw @j-bowhay @ksunden @mihaimaruseac @kne42

Based on the discussion, would you cast a vote (👍 or 👎 ) as to whether I should continue with drafting a SPEC? @rossbar if NetworkX doesn't like the idea of standardization, that's fine - maybe NetworkX just won't endorse this SPEC, and of course, projects that endorse a SPEC don't even have to implement it.

---

I think that based on the discussion, the examples above would change from the comma-separated form to something even more human-oriented. Also, rather than including device information in the type description, we'll only put it in the table at the end. We discussed that the term array_like might actually be OK if:

1. array_like is defined to mean something like "Any array/device combination with support indicated in the compatibility table or something else that np.asarray accepts" and
2. The non-array array_like satisfies the other requirements (especially dtype, which precludes object arrays)

array_like things that are actually arrays are left as arrays; others are eventually converted to NumPy arrays (possibly after a result_type calculation).

I know we talked about something like (M, N) array of real floating, but how about including "real floating" first so the grammar seems more natural? Also, I can live with the idea of not explicitly specifying things (like shape) when there are no limitations. Then the examples become:

> x : numeric array_like
> y : numeric array_like
> z : real floating array_like

> A : real floating (..., N, N) array_like
> b : real floating (..., N) or (..., N, M) array_like
> x : real floating (..., N) or (..., N, M) array_like

# (function is identified to be a reducing function, and input shape requirement / output shape calculation described separately)
> x : real numeric array_like
> y : real numeric array_like
> axis : tuple[int, ...]
> statistic : real floating array_like
> pvalue : real floating array_like

and things that are too cumbersome to describe inline can be described like image, where image might be described elsewhere.

[kne42]

👍 for standardization but I personally prefer that format you proposed of e.g. array[(3, 4, 5), float32]. It may be less human readable at first but it’s way more efficient and less confusing when things get complex e.g. list of tuple of string and string or array of int vs list[tuple[string, string | array[int]]].

[mihaimaruseac]

👍 for standardization. I think typing syntax is now well known, so I'm also leaning towards array<(3,4,5), float32> or similar.

[pfackeldey]

👍 for standardization also from my side.

I can live with the idea of not explicitly specifying things (like shape) when there are no limitations.

I like that approach. It makes it less verbose in most cases, and in specific cases which e.g. work only with specific dtypes it's more verbose - but for good reason.

[rossbar]

tl;dr - 👍 for a SPEC, 👍 for collection of patterns and standardization where it makes sense, so long as flexiblity is maintained (i.e. avoid the word must) and human-readability remains a guiding principle.

Too much exposition below

---

I'm 👍 for a SPEC being drafted as well, especially if it would help with libraries who'd like to adopt patterns that otherwise face resistance to changing parameter type lines!

To be clear, my objections are not so much from NetworkX's perspective (which doesn't have so much array-ness in parameter descriptions) - I was more thinking about how something like this would apply to numpy.

Take np.add for example, which currently has the following parameter description for the operands: x1, x2 : array_like. A lot of my argument during the discussion was based on the fact that it'd be very difficult to provide more information here without getting verbose to the point that it detracts from clarity. Perhaps there is some distinction to be made between types of functions/operations to which these suggestions apply; for example, array operations (or perhaps libraries that implement arrays) may not benefit as much as functions that consume arrays.

I also agree that standardization would be an improvement, but I would personally be more comfortable with a solution that avoids prescription and leaves flexibility to define parameter descriptions as libraries see fit. The collection of examples in the above post seem like nice patterns, but I'd be leery about saying that parameter descriptions must adhere to these patterns. There are also almost certainly going to be cases that don't neatly map to the recommended patterns... here's a parameter description from a library I work with a particularly messy API¹:

img : `array_like` with shape `(W, H)` or `(W, H, C)`, where `C` is 1 or 3

As a docstring author, I'd like to maintain the flexibility to break from more structured information and rely on slightly more descriptive prose when needed. To be clear, I don't think a SPEC prevents this from being possible, so long as it's not couched in terms of imperatives but rather suggestions drawn or derived from successful patterns already in use across the ecosystem.

Finally, I would advocate for resisting the urge to turn parameter type descriptions into type annotations; i.e. imposing structure on the annotations for the goal of programmatic-readability at the expense of human-readability. This is another point where I feel like I may have derailed the discussion at the summit unnecessarily as I think there was more focus on structure/syntax than was perhaps intended, especially early on in the discussion. One point that I think everyone agreed on is that there is a tradeoff between specificity and clarity. Nowhere is this made more clear than in looking at type annotations themselves. One of the reasons scientific Python libraries have resisted the adoption of in line type annotations is that many feel that they actually reduce readability, especially in the cases where the typing gets complex². Also, unlike static type checkers, users have the benefit of context and a function may want to communicate information other than the shape/dtype of an array in the parameter type description. For example, let's say I have the following parameter:

base_pairs : array of nucleotides {"A", "C", "G" or "T"}

Perhaps in the context of my function, I want users to focus on the fact that the values in the array are expected to conform to a certain pattern, rather than emphasis the shape (1D) or dtype ("str"). IMO, there should be the flexibility to do so. So long as a SPEC document has the right scope and maintains sufficient flexibility for such cases, I think it would be a benefit to the ecosystem!

Footnotes

1. You might argue this is an API issue... and I'd agree with you 🙃 ; But! IMO it should still be possible for documentation to clarify around API warts insofar as possible. ↩

2. Hence the preference for stubfiles --- enabling programmatic checking without ruining the readability of function signatures. On a related note, I just find -exec wc -l 'ed numpy's .pyi files: over 40k lines and counting! ↩

[j-bowhay]

I agree some standardization would be great. I think it makes a big difference for beginners if when they switch from package A to B the style of documentation is as similar as possible.

I agree with @rossbar I personally am not a fan of the type hinting style descriptions. To me it is confusing because it is neither especially readable or valid syntax for type hints, rather somewhere in no man's land in between.

Perhaps in order to strike the balance between standardization and allowing projects enough flexibility the spec could do something along the lines of:

• State in the maximalist case all the useful information that might be useful to convey to the user.
• Suggest a pattern for how to present this information to the user and suggested terminology (e.g. array vs array-like, real floating).
• Suggest some examples of when it might be better to omit some parts