From 6029fecb4007d67fecd4126a0ac681aa832f9614 Mon Sep 17 00:00:00 2001 From: Eneg <42005170+Enegg@users.noreply.github.com> Date: Mon, 14 Oct 2024 00:31:09 +0200 Subject: [PATCH 01/19] batman --- peps/pep-0763.rst | 172 ++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 172 insertions(+) create mode 100644 peps/pep-0763.rst diff --git a/peps/pep-0763.rst b/peps/pep-0763.rst new file mode 100644 index 00000000000..8d11b825827 --- /dev/null +++ b/peps/pep-0763.rst @@ -0,0 +1,172 @@ +PEP: 763 +Title: Classes & protocols: Read-only attributes +Author: +Sponsor: +Discussions-To: +Status: Draft +Type: Standards Track +Topic: Typing +Created: 11-Oct-2024 +Python-Version: 3.? + + +Abstract +======== + +:pep:`705` introduced the :external+py3.13:data:`typing.ReadOnly` type qualifier +to allow defining read-only :class:`typing.TypedDict` items. + +This PEP proposes expanding the scope of ``ReadOnly`` to class and protocol +attributes, as a single concise way to mark them read-only. +Akin to :pep:`705`, it makes no Python grammar changes. Correct usage of +read-only attributes is intended to be enforced only by static type checkers. + + +Motivation +========== + +Python type system lacks a single concise way to mark an attribute read-only. +This feature is common in other object-oriented languages (such as C#), and is +useful for restricting attribute mutation at a typechecker level, as well as defining +a very wide interface for structural subtyping. + +Classes +------- + +There are 3 major ways of achieving read-only attributes, honored by typecheckers: + +* annotating the attribute with :class:`~typing.Final`:: + + class Foo: + number: Final[int] + + def __init__(self, number: int) -> None: + self.number = number + + - Supported by :mod:`dataclasses` [#final_in_dataclasses]_. + - Overriding ``number`` is not possible - the specification of ``Final`` + imposes the name cannot be overridden in subclasses. + +* read-only proxy via ``@property``:: + + class Foo: + _number: int + + def __init__(self, number: int) -> None: + self._number = number + + @property + def number(self) -> int: + return self._number + + - Prevents runtime mutation. + - Overriding ``number`` is possible. `Pyright rejects `_ non-``@property`` overrides. + - Requires extra boilerplate. + - Supported by :mod:`dataclasses`, but does not compose well - the synthesized ``__init__`` and ``__repr__`` will use ``_number`` as the parameter/attribute name. + +* using a "freezing" mechanism, such as :func:`~dataclasses.dataclass` or :class:`~typing.NamedTuple`:: + + @dataclass(frozen=True) + class Foo: + number: int + + + class Bar(NamedTuple): + number: int + + - Overriding ``number`` is possible in the ``@dataclass`` case. + - Prevents runtime mutation. + - No per-attribute control - these methods apply to the whole class. + - Frozen dataclasses incur some runtime overhead. + +Protocols +--------- + +There is no way of defining a :class:`~typing.Protocol`, such that the only requirements to satisfy are: + +1. ``hasattr(obj, name)`` +2. ``isinstance(obj.name, T)`` + +The above are satisfiable at runtime by all of the following, regardless of whether the names support mutation: + +1. an object with an attribute ``name: T``, +2. a class with a class variable ``name: ClassVar[T]``, +3. an instance of the class above, +4. an object with a ``@property`` ``def name(self) -> T``, +5. an object with a ``__get__`` descriptor, such as :func:`functools.cached_property`. + +The most common practice is to define such a protocol with a ``@property``:: + + class HasName[T](Protocol): + @property + def name(self) -> T: ... + +Typecheckers special-case this definition, such that objects with plain attributes +are assignable to the type. However, instances with class variables and descriptors other than ``property`` are rejected. + +Covering the extra possibilities involves writing a great amount of repetitive boilerplate, +which gets multiplied for each additional attribute. + + +Rationale +========= + +[Describe why particular design decisions were made.] + + +Specification +============= + +[Describe the syntax and semantics of any new language feature.] + + +Backwards Compatibility +======================= + +This PEP introduces new contexts where ``ReadOnly`` is valid. Programs that inspect those places will have to change to support it. +This is expected to mainly affect type checkers. + + +Security Implications +===================== + +There are no known security consequences arising from this PEP. + + +How to Teach This +================= + +[How to teach users, new and experienced, how to apply the PEP to their work.] + + +Reference Implementation +======================== + +[Link to any existing implementation and details about its state, e.g. proof-of-concept.] + + +Rejected Ideas +============== + +[Why certain ideas that were brought while discussing this PEP were not ultimately pursued.] + + +Open Issues +=========== + +[Any points that are still being decided/discussed.] + + +Footnotes +========= + +.. [#final_in_dataclasses] Final and dataclass interaction has been clarified in https://github.com/python/typing/pull/1669 + +.. _pyright_playground: https://pyright-play.net/?strict=true&code=MYGwhgzhAEBiD28BcBYAUNT0D6A7ArgLYBGApgE5LQCWuALuultACakBmO2t1d22ACgikQ7ADTQCJClVp0AlNAC0APmgA5eLlKoMzLMNEA6PETLloAXklmKjPZgACAB3LxnFOgE8mWNpylzIRF2RVUael19LHJSOnxyXGhDdhNAuzR7UEgYACEwcgEEeHkorHTKCIY0IA + + +Copyright +========= + +This document is placed in the public domain or under the +CC0-1.0-Universal license, whichever is more permissive. From c2bd477a719195a3195d50570238bcaf0cf53293 Mon Sep 17 00:00:00 2001 From: Eneg <42005170+Enegg@users.noreply.github.com> Date: Wed, 16 Oct 2024 21:42:02 +0200 Subject: [PATCH 02/19] refining --- peps/pep-0763.rst | 87 +++++++++++++++++++++++++---------------------- 1 file changed, 46 insertions(+), 41 deletions(-) diff --git a/peps/pep-0763.rst b/peps/pep-0763.rst index 8d11b825827..2979c879907 100644 --- a/peps/pep-0763.rst +++ b/peps/pep-0763.rst @@ -27,15 +27,15 @@ Motivation Python type system lacks a single concise way to mark an attribute read-only. This feature is common in other object-oriented languages (such as C#), and is -useful for restricting attribute mutation at a typechecker level, as well as defining -a very wide interface for structural subtyping. +useful for restricting attribute mutation at a type checker level, as well as +defining a broad interface for structural subtyping. Classes ------- -There are 3 major ways of achieving read-only attributes, honored by typecheckers: +There are 3 major ways of achieving read-only attributes, honored by type checkers: -* annotating the attribute with :class:`~typing.Final`:: +* annotating the attribute with :class:`typing.Final`:: class Foo: number: Final[int] @@ -43,7 +43,7 @@ There are 3 major ways of achieving read-only attributes, honored by typechecker def __init__(self, number: int) -> None: self.number = number - - Supported by :mod:`dataclasses` [#final_in_dataclasses]_. + - Supported by :mod:`dataclasses` (since `typing#1669 `_). - Overriding ``number`` is not possible - the specification of ``Final`` imposes the name cannot be overridden in subclasses. @@ -59,12 +59,13 @@ There are 3 major ways of achieving read-only attributes, honored by typechecker def number(self) -> int: return self._number - - Prevents runtime mutation. - - Overriding ``number`` is possible. `Pyright rejects `_ non-``@property`` overrides. + - Read-only at runtime [#runtime]_. + - Overriding ``number`` is possible. Type checkers disagree about the specific rules [#overriding_property]_. - Requires extra boilerplate. - - Supported by :mod:`dataclasses`, but does not compose well - the synthesized ``__init__`` and ``__repr__`` will use ``_number`` as the parameter/attribute name. + - Supported by :mod:`dataclasses`, but does not compose well - the synthesized + ``__init__`` and ``__repr__`` will use ``_number`` as the parameter/attribute name. -* using a "freezing" mechanism, such as :func:`~dataclasses.dataclass` or :class:`~typing.NamedTuple`:: +* using a "freezing" mechanism, such as :func:`dataclasses.dataclass` or :class:`typing.NamedTuple`:: @dataclass(frozen=True) class Foo: @@ -75,25 +76,28 @@ There are 3 major ways of achieving read-only attributes, honored by typechecker number: int - Overriding ``number`` is possible in the ``@dataclass`` case. - - Prevents runtime mutation. + - Read-only at runtime [#runtime]_. - No per-attribute control - these methods apply to the whole class. - Frozen dataclasses incur some runtime overhead. + - Not all classes qualify to be a ``NamedTuple``. Protocols --------- -There is no way of defining a :class:`~typing.Protocol`, such that the only requirements to satisfy are: +Paraphrasing `discussions#1525 `_, +there is no way of defining a :class:`~typing.Protocol`, such that the only requirements to satisfy are: 1. ``hasattr(obj, name)`` -2. ``isinstance(obj.name, T)`` +2. ``isinstance(obj.name, T)`` [#invalid_typevar]_ -The above are satisfiable at runtime by all of the following, regardless of whether the names support mutation: +The above are satisfiable at runtime by all of the following, +regardless of whether the names support mutation: 1. an object with an attribute ``name: T``, -2. a class with a class variable ``name: ClassVar[T]``, +2. a class with a class variable ``name: ClassVar[T]`` [#invalid_typevar]_, 3. an instance of the class above, 4. an object with a ``@property`` ``def name(self) -> T``, -5. an object with a ``__get__`` descriptor, such as :func:`functools.cached_property`. +5. an object with a custom descriptor, such as :func:`functools.cached_property`. The most common practice is to define such a protocol with a ``@property``:: @@ -101,11 +105,14 @@ The most common practice is to define such a protocol with a ``@property``:: @property def name(self) -> T: ... -Typecheckers special-case this definition, such that objects with plain attributes -are assignable to the type. However, instances with class variables and descriptors other than ``property`` are rejected. +Type checkers special-case this definition, such that objects with plain attributes +are assignable to the type. However, instances with class variables and descriptors +other than ``property`` are rejected. -Covering the extra possibilities involves writing a great amount of repetitive boilerplate, -which gets multiplied for each additional attribute. +Covering the extra possibilities induces a great amount of boilerplate, involving +creation of an abstract descriptor protocol, possibly also accounting for +class vs instance level overloads. +Worse yet, all of that is multiplied for each additional read-only attribute. Rationale @@ -123,8 +130,14 @@ Specification Backwards Compatibility ======================= -This PEP introduces new contexts where ``ReadOnly`` is valid. Programs that inspect those places will have to change to support it. -This is expected to mainly affect type checkers. +This PEP introduces new contexts where ``ReadOnly`` is valid. Programs inspecting +those places will have to change to support it. This is expected to mainly affect type checkers. + +However, caution is advised while using the backported ``typing_extensions.ReadOnly`` +in older versions of Python, especially in conjunction with other type qualifiers; +not all nesting orderings might be treated equal. In particular, the ``@dataclass`` +decorator which looks for ``ClassVar`` will incorrectly treat +``ReadOnly[ClassVar[...]]`` as an instance attribute. Security Implications @@ -139,30 +152,22 @@ How to Teach This [How to teach users, new and experienced, how to apply the PEP to their work.] -Reference Implementation -======================== - -[Link to any existing implementation and details about its state, e.g. proof-of-concept.] - - -Rejected Ideas -============== - -[Why certain ideas that were brought while discussing this PEP were not ultimately pursued.] - - -Open Issues -=========== - -[Any points that are still being decided/discussed.] - - Footnotes ========= -.. [#final_in_dataclasses] Final and dataclass interaction has been clarified in https://github.com/python/typing/pull/1669 +.. [#runtime] + This PEP focuses solely on the type-checking behavior. Nevertheless, its rather + desirable the read-only trait is reflected at runtime. + +.. [#invalid_typevar] + The implied type variable is not valid in this context. It has been used here + for ease of demonstration. -.. _pyright_playground: https://pyright-play.net/?strict=true&code=MYGwhgzhAEBiD28BcBYAUNT0D6A7ArgLYBGApgE5LQCWuALuultACakBmO2t1d22ACgikQ7ADTQCJClVp0AlNAC0APmgA5eLlKoMzLMNEA6PETLloAXklmKjPZgACAB3LxnFOgE8mWNpylzIRF2RVUael19LHJSOnxyXGhDdhNAuzR7UEgYACEwcgEEeHkorHTKCIY0IA +.. [#overriding_property] + Pyright in strict mode disallows non-property overrides. + Mypy does not impose this restriction and allows an override with a plain attribute. + `[Pyright playground] `_ + `[mypy playground] `_ Copyright From 5d2e422962236539fc0f3235f140ad2783a24d82 Mon Sep 17 00:00:00 2001 From: Eneg <42005170+Enegg@users.noreply.github.com> Date: Thu, 17 Oct 2024 22:31:43 +0200 Subject: [PATCH 03/19] Polishing --- peps/pep-0763.rst | 36 ++++++++++++++++++++---------------- 1 file changed, 20 insertions(+), 16 deletions(-) diff --git a/peps/pep-0763.rst b/peps/pep-0763.rst index 2979c879907..ea105ffde97 100644 --- a/peps/pep-0763.rst +++ b/peps/pep-0763.rst @@ -35,7 +35,7 @@ Classes There are 3 major ways of achieving read-only attributes, honored by type checkers: -* annotating the attribute with :class:`typing.Final`:: +* annotating the attribute with :data:`typing.Final`:: class Foo: number: Final[int] @@ -43,6 +43,11 @@ There are 3 major ways of achieving read-only attributes, honored by type checke def __init__(self, number: int) -> None: self.number = number + + class Bar: + def __init__(self, number: int) -> None: + self.number: Final = number + - Supported by :mod:`dataclasses` (since `typing#1669 `_). - Overriding ``number`` is not possible - the specification of ``Final`` imposes the name cannot be overridden in subclasses. @@ -59,8 +64,8 @@ There are 3 major ways of achieving read-only attributes, honored by type checke def number(self) -> int: return self._number - - Read-only at runtime [#runtime]_. - - Overriding ``number`` is possible. Type checkers disagree about the specific rules [#overriding_property]_. + - Overriding ``number`` is possible. *Type checkers disagree about the specific rules*. [#overriding_property]_ + - Read-only at runtime. [#runtime]_ - Requires extra boilerplate. - Supported by :mod:`dataclasses`, but does not compose well - the synthesized ``__init__`` and ``__repr__`` will use ``_number`` as the parameter/attribute name. @@ -76,7 +81,7 @@ There are 3 major ways of achieving read-only attributes, honored by type checke number: int - Overriding ``number`` is possible in the ``@dataclass`` case. - - Read-only at runtime [#runtime]_. + - Read-only at runtime. [#runtime]_ - No per-attribute control - these methods apply to the whole class. - Frozen dataclasses incur some runtime overhead. - Not all classes qualify to be a ``NamedTuple``. @@ -84,17 +89,16 @@ There are 3 major ways of achieving read-only attributes, honored by type checke Protocols --------- -Paraphrasing `discussions#1525 `_, +Paraphrasing `this post `_, there is no way of defining a :class:`~typing.Protocol`, such that the only requirements to satisfy are: 1. ``hasattr(obj, name)`` 2. ``isinstance(obj.name, T)`` [#invalid_typevar]_ -The above are satisfiable at runtime by all of the following, -regardless of whether the names support mutation: +The above are satisfiable at runtime by all of the following: 1. an object with an attribute ``name: T``, -2. a class with a class variable ``name: ClassVar[T]`` [#invalid_typevar]_, +2. a class with a class variable ``name: ClassVar[T]``, [#invalid_typevar]_ 3. an instance of the class above, 4. an object with a ``@property`` ``def name(self) -> T``, 5. an object with a custom descriptor, such as :func:`functools.cached_property`. @@ -155,20 +159,20 @@ How to Teach This Footnotes ========= -.. [#runtime] - This PEP focuses solely on the type-checking behavior. Nevertheless, its rather - desirable the read-only trait is reflected at runtime. - -.. [#invalid_typevar] - The implied type variable is not valid in this context. It has been used here - for ease of demonstration. - .. [#overriding_property] Pyright in strict mode disallows non-property overrides. Mypy does not impose this restriction and allows an override with a plain attribute. `[Pyright playground] `_ `[mypy playground] `_ +.. [#runtime] + This PEP focuses solely on the type-checking behavior. Nevertheless, it should + be desirable the name is read-only at runtime. + +.. [#invalid_typevar] + The implied type variable is not valid in this context. It has been used here + for ease of demonstration. + Copyright ========= From db3d76efc8e793fe519ce91dac5a5833e2f064ee Mon Sep 17 00:00:00 2001 From: Eneg <42005170+Enegg@users.noreply.github.com> Date: Thu, 17 Oct 2024 23:19:23 +0200 Subject: [PATCH 04/19] Rationale & specification --- peps/pep-0763.rst | 162 +++++++++++++++++++++++++++++++++++++++------- 1 file changed, 138 insertions(+), 24 deletions(-) diff --git a/peps/pep-0763.rst b/peps/pep-0763.rst index ea105ffde97..eeebf71436e 100644 --- a/peps/pep-0763.rst +++ b/peps/pep-0763.rst @@ -26,9 +26,9 @@ Motivation ========== Python type system lacks a single concise way to mark an attribute read-only. -This feature is common in other object-oriented languages (such as C#), and is -useful for restricting attribute mutation at a type checker level, as well as -defining a broad interface for structural subtyping. +This feature is common in other object-oriented languages (such as `C# `_), +and is useful for restricting attribute mutation at a type checker level, as well +as defining a broad interface for structural subtyping. Classes ------- @@ -37,16 +37,16 @@ There are 3 major ways of achieving read-only attributes, honored by type checke * annotating the attribute with :data:`typing.Final`:: - class Foo: - number: Final[int] + class Foo: + number: Final[int] - def __init__(self, number: int) -> None: - self.number = number + def __init__(self, number: int) -> None: + self.number = number - class Bar: - def __init__(self, number: int) -> None: - self.number: Final = number + class Bar: + def __init__(self, number: int) -> None: + self.number: Final = number - Supported by :mod:`dataclasses` (since `typing#1669 `_). - Overriding ``number`` is not possible - the specification of ``Final`` @@ -54,15 +54,15 @@ There are 3 major ways of achieving read-only attributes, honored by type checke * read-only proxy via ``@property``:: - class Foo: - _number: int + class Foo: + _number: int - def __init__(self, number: int) -> None: - self._number = number + def __init__(self, number: int) -> None: + self._number = number - @property - def number(self) -> int: - return self._number + @property + def number(self) -> int: + return self._number - Overriding ``number`` is possible. *Type checkers disagree about the specific rules*. [#overriding_property]_ - Read-only at runtime. [#runtime]_ @@ -72,13 +72,13 @@ There are 3 major ways of achieving read-only attributes, honored by type checke * using a "freezing" mechanism, such as :func:`dataclasses.dataclass` or :class:`typing.NamedTuple`:: - @dataclass(frozen=True) - class Foo: - number: int + @dataclass(frozen=True) + class Foo: + number: int - class Bar(NamedTuple): - number: int + class Bar(NamedTuple): + number: int - Overriding ``number`` is possible in the ``@dataclass`` case. - Read-only at runtime. [#runtime]_ @@ -86,6 +86,8 @@ There are 3 major ways of achieving read-only attributes, honored by type checke - Frozen dataclasses incur some runtime overhead. - Not all classes qualify to be a ``NamedTuple``. +.. _protocols: + Protocols --------- @@ -122,13 +124,125 @@ Worse yet, all of that is multiplied for each additional read-only attribute. Rationale ========= -[Describe why particular design decisions were made.] +This problem can be resolved by an attribute-level type qualifier. ``ReadOnly`` +has been chosen for this role, as its name conveys the intent well, and the newly +proposed changes complement its semantics defined in :pep:`705`. + +A class with a read-only instance attribute can be now defined as such:: + + from typing import ReadOnly + + class Member: + id: ReadOnly[int] + + def __init__(self, id: int) -> None: + self.id = id + +...and a protocol as described in :ref:`protocols` is now just:: + + from typing import Protocol, ReadOnly + + class HasName(Protocol): + name: ReadOnly[str] + + def greet(obj: HasName, /) -> str: + return f"Hello, {obj.name}!" + +A subclass of ``Member`` can redefine ``id`` as a ``property`` or writable +attribute, while staying compatible with the base class. + +The ``HasName`` protocol can be implemented by any mechanism allowing for ``.name`` access. + +The ``greet`` function can now accept a wide variety of compatible objects, +while being explicit about no modifications being done to the input. Specification ============= -[Describe the syntax and semantics of any new language feature.] +The :external+py3.13:data:`typing.ReadOnly` type qualifier becomes a valid annotation +for attributes of classes and protocols. + +Classes +------- + +In a class attribute declaration, ``ReadOnly`` indicates that assignment to the +attribute can only occur as a part of the declaration, or within a set of +initializing methods in the same class:: + + from collections import abc + from typing import ReadOnly + + + class Band: + name: str + songs: ReadOnly[list[str]] + + def __init__(self, name: str, songs: abc.Iterable[str] | None = None) -> None: + self.name = name + self.songs = [] + + if songs is not None: + # multiple assignments during initialization are fine + self.songs = list(songs) + + def clear(self) -> None: + self.songs = [] # Type check error: "songs" is read only + + + band = Band("Boa", ["Duvet"]) + band.name = "Emma" # Ok: "name" is not read-only + band.songs = [] # Type check error: "songs" is read-only + band.songs.append("Twilight") # Ok: list is mutable + +The set of initializing methods consists of ``__new__`` and ``__init__``. +However, type checkers may permit assignment in additional `special methods `_ +to facilitate 3rd party mechanisms such as dataclasses' `__post_init__ `_. +It should be non-ambiguous that those methods are not called outside initialization. + +A read-only attribute with an initializer remains assignable to during initialization:: + + class Foo: + number: ReadOnly[int] = 0 + + def __init__(self, number: int | None = None) -> None: + if number is not None: + self.number = number + +Note that this cannot be used in a class defining ``__slots__``, as slots prohibit +the existence of class-level and instance-level attributes of same name. + +Protocols +--------- + +In a protocol attribute declaration, ``name: ReadOnly[T]`` indicates that a structural +subtype must support ``.name`` access, and the returned value is compatible with ``T``. + +Interaction with other special types +------------------------------------ + +``ReadOnly`` can be used with ``ClassVar`` and ``Annotated`` in any nesting order: + +.. code-block:: python + + class Foo: + foo: ClassVar[ReadOnly[str]] = "foo" + bar: Annotated[ReadOnly[int], Gt(0)] + +.. code-block:: python + + class Foo: + foo: ReadOnly[ClassVar[str]] = "foo" + bar: ReadOnly[Annotated[int, Gt(0)]] + +This is consistent with the interaction of ``ReadOnly`` and :class:`typing.TypedDict` +defined in :pep:`705`. + +The combination of ``ReadOnly`` and ``ClassVar`` imposes the attribute must be +initialized in the class scope, as there are no other valid initialization scopes. + +``ReadOnly`` cannot be combined with ``Final``, as the two qualifiers define incompatible +initialization rules, leading to ambiguity and/or significance of ordering. Backwards Compatibility From 7893b78131452f73dcc360197248bcebc43dffce Mon Sep 17 00:00:00 2001 From: Eneg <42005170+Enegg@users.noreply.github.com> Date: Sun, 27 Oct 2024 19:04:44 +0100 Subject: [PATCH 05/19] Syntax --- peps/pep-0763.rst | 140 ++++++++++++++++++++++++++++++++-------------- 1 file changed, 99 insertions(+), 41 deletions(-) diff --git a/peps/pep-0763.rst b/peps/pep-0763.rst index eeebf71436e..5c26e909d5e 100644 --- a/peps/pep-0763.rst +++ b/peps/pep-0763.rst @@ -1,7 +1,7 @@ PEP: 763 Title: Classes & protocols: Read-only attributes Author: -Sponsor: +Sponsor: Carl Meyer Discussions-To: Status: Draft Type: Standards Track @@ -17,7 +17,9 @@ Abstract to allow defining read-only :class:`typing.TypedDict` items. This PEP proposes expanding the scope of ``ReadOnly`` to class and protocol -attributes, as a single concise way to mark them read-only. +attributes, as a single concise way to mark them read-only. Some parity changes +are also made to :class:`typing.Final`. + Akin to :pep:`705`, it makes no Python grammar changes. Correct usage of read-only attributes is intended to be enforced only by static type checkers. @@ -30,6 +32,8 @@ This feature is common in other object-oriented languages (such as `C# `_, -there is no way of defining a :class:`~typing.Protocol`, such that the only requirements to satisfy are: +there is no way of defining a :class:`~typing.Protocol`, such that the only +requirements to satisfy are: 1. ``hasattr(obj, name)`` 2. ``isinstance(obj.name, T)`` [#invalid_typevar]_ @@ -105,6 +111,9 @@ The above are satisfiable at runtime by all of the following: 4. an object with a ``@property`` ``def name(self) -> T``, 5. an object with a custom descriptor, such as :func:`functools.cached_property`. +Note that the attribute being marked ``Final`` or the property defining a setter +do not impact this. + The most common practice is to define such a protocol with a ``@property``:: class HasName[T](Protocol): @@ -117,14 +126,14 @@ other than ``property`` are rejected. Covering the extra possibilities induces a great amount of boilerplate, involving creation of an abstract descriptor protocol, possibly also accounting for -class vs instance level overloads. +class and instance level overloads. Worse yet, all of that is multiplied for each additional read-only attribute. Rationale ========= -This problem can be resolved by an attribute-level type qualifier. ``ReadOnly`` +These problems can be resolved by an attribute-level type qualifier. ``ReadOnly`` has been chosen for this role, as its name conveys the intent well, and the newly proposed changes complement its semantics defined in :pep:`705`. @@ -132,6 +141,7 @@ A class with a read-only instance attribute can be now defined as such:: from typing import ReadOnly + class Member: id: ReadOnly[int] @@ -142,19 +152,19 @@ A class with a read-only instance attribute can be now defined as such:: from typing import Protocol, ReadOnly + class HasName(Protocol): name: ReadOnly[str] + def greet(obj: HasName, /) -> str: return f"Hello, {obj.name}!" -A subclass of ``Member`` can redefine ``id`` as a ``property`` or writable -attribute, while staying compatible with the base class. - -The ``HasName`` protocol can be implemented by any mechanism allowing for ``.name`` access. - -The ``greet`` function can now accept a wide variety of compatible objects, -while being explicit about no modifications being done to the input. +* A subclass of ``Member`` can redefine ``id`` as a ``property`` or writable + attribute, while staying compatible with the base class. +* The ``HasName`` protocol can be implemented by any mechanism allowing for ``.name`` access. +* The ``greet`` function can now accept a wide variety of compatible objects, + while being explicit about no modifications being done to the input. Specification @@ -163,12 +173,46 @@ Specification The :external+py3.13:data:`typing.ReadOnly` type qualifier becomes a valid annotation for attributes of classes and protocols. -Classes -------- +It remains invalid in annotations of global and local variables, as in those contexts +it would have the same meaning as using ``Final``. + +Syntax +------ + +``ReadOnly`` can be used at class-level or within ``__init__`` to declare +an attribute read-only: + +.. code-block:: python + + class Base: + id: ReadOnly[int] + + def __init__(self, id: int, rate: float) -> None: + self.id = id + self.rate: ReadOnly = rate + +The explicit type in ``ReadOnly[]`` can be omitted if an initializing value +is assigned to the attribute. A type checker should apply its usual type inference +rules to determine the type of ``rate``. -In a class attribute declaration, ``ReadOnly`` indicates that assignment to the -attribute can only occur as a part of the declaration, or within a set of -initializing methods in the same class:: +In contexts where an attribute is already implied to be read-only, like in the +frozen :ref:`classes`, it should be valid to explicitly declare it ``ReadOnly``: + +.. code-block:: python + + @dataclass(frozen=True) + class Point: + x: ReadOnly[int] + y: ReadOnly[int] + +Initialization +-------------- + +Assignment to a ``ReadOnly`` attribute can only occur as a part of the declaration, +or within ``__init__`` of the same class. There is no restriction to how many +times the attribute can be assigned to in those contexts. Example: + +.. code-block:: python from collections import abc from typing import ReadOnly @@ -187,37 +231,41 @@ initializing methods in the same class:: self.songs = list(songs) def clear(self) -> None: - self.songs = [] # Type check error: "songs" is read only + # Type check error: assignment to read-only "songs" outside initialization + self.songs = [] - band = Band("Boa", ["Duvet"]) + band = Band(name="Boa", songs=["Duvet"]) band.name = "Emma" # Ok: "name" is not read-only band.songs = [] # Type check error: "songs" is read-only band.songs.append("Twilight") # Ok: list is mutable -The set of initializing methods consists of ``__new__`` and ``__init__``. -However, type checkers may permit assignment in additional `special methods `_ -to facilitate 3rd party mechanisms such as dataclasses' `__post_init__ `_. -It should be non-ambiguous that those methods are not called outside initialization. +Classes which do not define `__slots__ `_ +may give the attribute a default value, overridable at instance level: -A read-only attribute with an initializer remains assignable to during initialization:: +.. code-block:: python class Foo: number: ReadOnly[int] = 0 def __init__(self, number: int | None = None) -> None: if number is not None: - self.number = number + self.number = number -Note that this cannot be used in a class defining ``__slots__``, as slots prohibit -the existence of class-level and instance-level attributes of same name. - -Protocols ---------- +Inheritance +----------- In a protocol attribute declaration, ``name: ReadOnly[T]`` indicates that a structural subtype must support ``.name`` access, and the returned value is compatible with ``T``. +Changes to ``Final`` +-------------------- + +.. TODO + once changes are done, this probably won't be true + ``ReadOnly`` cannot be combined with ``Final``, as the two qualifiers differ in + initialization rules, leading to ambiguity and/or significance of ordering. + Interaction with other special types ------------------------------------ @@ -238,11 +286,8 @@ Interaction with other special types This is consistent with the interaction of ``ReadOnly`` and :class:`typing.TypedDict` defined in :pep:`705`. -The combination of ``ReadOnly`` and ``ClassVar`` imposes the attribute must be -initialized in the class scope, as there are no other valid initialization scopes. - -``ReadOnly`` cannot be combined with ``Final``, as the two qualifiers define incompatible -initialization rules, leading to ambiguity and/or significance of ordering. +``ClassVar`` excludes read-only attributes from being assignable to within +initialization methods. Backwards Compatibility @@ -252,10 +297,10 @@ This PEP introduces new contexts where ``ReadOnly`` is valid. Programs inspectin those places will have to change to support it. This is expected to mainly affect type checkers. However, caution is advised while using the backported ``typing_extensions.ReadOnly`` -in older versions of Python, especially in conjunction with other type qualifiers; -not all nesting orderings might be treated equal. In particular, the ``@dataclass`` -decorator which looks for ``ClassVar`` will incorrectly treat -``ReadOnly[ClassVar[...]]`` as an instance attribute. +in older versions of Python. Mechanisms inspecting annotations may behave incorrectly +when encountering ``ReadOnly``; in particular, the ``@dataclass`` decorator +which `looks for `_ +``ClassVar`` will incorrectly treat ``ReadOnly[ClassVar[...]]`` as an instance attribute. Security Implications @@ -270,6 +315,19 @@ How to Teach This [How to teach users, new and experienced, how to apply the PEP to their work.] +Rejected ideas +============== + +Assignment in ``__post_init__`` +------------------------------- + +An earlier version of this PEP specified that assignment to read-only attributes +may also be permitted within methods augmenting initialization, such as +dataclasses' `__post_init__ `_ +or attrs' `initialization hooks `_. +This has been set aside for now, as defining rules regarding inclusion of +such methods has proven difficult. + Footnotes ========= From a75e977d54a1c2906f45cd8256e1ceeacb5dc77b Mon Sep 17 00:00:00 2001 From: Eneg <42005170+Enegg@users.noreply.github.com> Date: Sun, 27 Oct 2024 20:35:57 +0100 Subject: [PATCH 06/19] Subtyping --- peps/pep-0763.rst | 147 ++++++++++++++++++++++++++++++++++++++++++---- 1 file changed, 134 insertions(+), 13 deletions(-) diff --git a/peps/pep-0763.rst b/peps/pep-0763.rst index 5c26e909d5e..b895c74208a 100644 --- a/peps/pep-0763.rst +++ b/peps/pep-0763.rst @@ -101,7 +101,7 @@ there is no way of defining a :class:`~typing.Protocol`, such that the only requirements to satisfy are: 1. ``hasattr(obj, name)`` -2. ``isinstance(obj.name, T)`` [#invalid_typevar]_ +2. ``isinstance(obj.name, T)`` [#invalid_typevar]_ The above are satisfiable at runtime by all of the following: @@ -231,17 +231,18 @@ times the attribute can be assigned to in those contexts. Example: self.songs = list(songs) def clear(self) -> None: - # Type check error: assignment to read-only "songs" outside initialization + # error: assignment to read-only "songs" outside initialization self.songs = [] band = Band(name="Boa", songs=["Duvet"]) - band.name = "Emma" # Ok: "name" is not read-only - band.songs = [] # Type check error: "songs" is read-only - band.songs.append("Twilight") # Ok: list is mutable + band.name = "Emma" # ok: "name" is not read-only + band.songs = [] # error: "songs" is read-only + band.songs.append("Twilight") # ok: list is mutable Classes which do not define `__slots__ `_ -may give the attribute a default value, overridable at instance level: +retain the ability to initialize a read-only attribute both at declaration and +within ``__init__``: .. code-block:: python @@ -252,19 +253,138 @@ may give the attribute a default value, overridable at instance level: if number is not None: self.number = number -Inheritance ------------ +Type checkers should warn on attributes declared ``ReadOnly`` which may be left +uninitialized after ``__init__`` exits, unless the class is a protocol or an ABC:: + + class Foo: + id: ReadOnly[int] # error: "id" is not initialized on all code paths + name: ReadOnly[str] # error: "name" is never initialized + + def __init__(self) -> None: + if random.random() > 0.5: + self.id = 123 + +This rule stems from the fact the body of the class declaring the attribute is +the only place able to initialize it, in contrast to non-read-only attributes, +which could be initialized outside of the class body. + +Subtyping +--------- + +Read-only attributes are covariant. This has a few subtyping implications. +Borrowing from :pep:`PEP 705 <705#inheritance>`: + +* Read-only attributes can be redeclared as writable attributes, descriptors + and class variables:: + + @dataclass + class HasTitle: + title: ReadOnly[str] + + + @dataclass + class Game(HasTitle): + title: str + year: int + + + game = Game(title="DOOM", year=1993) + game.year = 1994 + game.title = "DOOM II" # ok: attribute is not read-only + + + class TitleProxy(HasTitle): + @functools.cached_property + def title(self) -> str: ... + + + class SharedTitle(HasTitle): + title: ClassVar[str] = "Still Grey" + +* If a read-only attribute is not redeclared, it remains read-only:: + + @dataclass + class Game(HasTitle): + year: int + + + game = Game(title="DOOM", year=1993) + game.title = "DOOM II" # error: attribute is read-only + +* Subtypes can narrow the type of read-only attributes:: + + class GameCollection(Protocol): + games: ReadOnly[abc.Collection[Game]] + + + @dataclass + class GameSeries(GameCollection): + name: str + games: ReadOnly[list[Game]] # ok: list[Game] is assignable to Collection[Game] + +* In nominal subclasses of protocols and ABCs, a read-only attribute should be + considered abstract, unless it is initialized:: + + class MyBase(abc.ABC): + foo: ReadOnly[int] + bar: ReadOnly[str] = "abc" + baz: ReadOnly[float] + + def __init__(self, baz: float) -> None: + self.baz = baz + + @abstractmethod + def do_something(self) -> None: ... + + + @final + class MySubclass(MyBase): + # error: MySubclass does not override "foo" + + def do_something(self) -> None: + print(self.foo, self.bar, self.baz) + +* In a protocol attribute declaration, ``name: ReadOnly[T]`` indicates that a structural + subtype must support ``.name`` access, and the returned value is compatible with ``T``:: + + class HasName(Protocol): + name: ReadOnly[str] + + + class NamedAttr: + name: str + + class NamedProp: + @property + def name(self) -> str: ... + + class NamedClassVar: + name: ClassVar[str] + + class NamedDescriptor: + @cached_property + def name(self) -> str: ... + + # all of the following are ok + has_name: HasName + has_name = NamedAttr() + has_name = NamedProp() + has_name = NamedClassVar + has_name = NamedClassVar() + has_name = NamedDescriptor() -In a protocol attribute declaration, ``name: ReadOnly[T]`` indicates that a structural -subtype must support ``.name`` access, and the returned value is compatible with ``T``. Changes to ``Final`` -------------------- .. TODO - once changes are done, this probably won't be true - ``ReadOnly`` cannot be combined with ``Final``, as the two qualifiers differ in - initialization rules, leading to ambiguity and/or significance of ordering. + - deletability + - final in protocols? + - interaction of Final and ReadOnly - once parity changes are done, the + following shouldn't be true: + ``ReadOnly`` cannot be combined with ``Final``, as the two qualifiers differ in + initialization rules, leading to ambiguity and/or significance of ordering. + - section on "Type consistency"? Interaction with other special types ------------------------------------ @@ -328,6 +448,7 @@ or attrs' `initialization hooks Date: Mon, 18 Nov 2024 22:39:02 +0100 Subject: [PATCH 07/19] References, changes to Final, open issues --- peps/pep-0763.rst | 220 ++++++++++++++++++++++++++++++---------------- 1 file changed, 146 insertions(+), 74 deletions(-) diff --git a/peps/pep-0763.rst b/peps/pep-0763.rst index b895c74208a..75002674082 100644 --- a/peps/pep-0763.rst +++ b/peps/pep-0763.rst @@ -6,7 +6,7 @@ Discussions-To: Status: Draft Type: Standards Track Topic: Typing -Created: 11-Oct-2024 +Created: ?-Nov-2024 Python-Version: 3.? @@ -18,7 +18,7 @@ to allow defining read-only :class:`typing.TypedDict` items. This PEP proposes expanding the scope of ``ReadOnly`` to class and protocol attributes, as a single concise way to mark them read-only. Some parity changes -are also made to :class:`typing.Final`. +are also made to :data:`typing.Final`. Akin to :pep:`705`, it makes no Python grammar changes. Correct usage of read-only attributes is intended to be enforced only by static type checkers. @@ -27,7 +27,7 @@ read-only attributes is intended to be enforced only by static type checkers. Motivation ========== -Python type system lacks a single concise way to mark an attribute read-only. +Python type system lacks a single concise way to mark an :term:`attribute` read-only. This feature is common in other object-oriented languages (such as `C# `_), and is useful for restricting attribute mutation at a type checker level, as well as defining a broad interface for structural subtyping. @@ -52,7 +52,7 @@ There are 3 major ways of achieving read-only attributes, honored by type checke def __init__(self, number: int) -> None: self.number: Final = number - - Supported by :mod:`dataclasses` (since `typing#1669 `_). + - Supported by :mod:`dataclasses` (and type checkers since `typing#1669 `_). - Overriding ``number`` is not possible - the specification of ``Final`` imposes the name cannot be overridden in subclasses. @@ -116,7 +116,7 @@ do not impact this. The most common practice is to define such a protocol with a ``@property``:: - class HasName[T](Protocol): + class HasName(Protocol): @property def name(self) -> T: ... @@ -133,9 +133,9 @@ Worse yet, all of that is multiplied for each additional read-only attribute. Rationale ========= -These problems can be resolved by an attribute-level type qualifier. ``ReadOnly`` -has been chosen for this role, as its name conveys the intent well, and the newly -proposed changes complement its semantics defined in :pep:`705`. +These problems can be resolved by an attribute-level :external+typing:term:`type qualifier`. +``ReadOnly`` has been chosen for this role, as its name conveys the intent well, +and the newly proposed changes complement its semantics defined in the :pep:`705`. A class with a read-only instance attribute can be now defined as such:: @@ -143,12 +143,10 @@ A class with a read-only instance attribute can be now defined as such:: class Member: - id: ReadOnly[int] - def __init__(self, id: int) -> None: - self.id = id + self.id: ReadOnly = id -...and a protocol as described in :ref:`protocols` is now just:: +...and a protocol described in :ref:`protocols` is now just:: from typing import Protocol, ReadOnly @@ -160,8 +158,8 @@ A class with a read-only instance attribute can be now defined as such:: def greet(obj: HasName, /) -> str: return f"Hello, {obj.name}!" -* A subclass of ``Member`` can redefine ``id`` as a ``property`` or writable - attribute, while staying compatible with the base class. +* A subclass of ``Member`` can redefine ``.id`` as a writable attribute or a + :term:`descriptor`. It can also :external+typing:term:`narrow` the type. * The ``HasName`` protocol can be implemented by any mechanism allowing for ``.name`` access. * The ``greet`` function can now accept a wide variety of compatible objects, while being explicit about no modifications being done to the input. @@ -170,47 +168,53 @@ A class with a read-only instance attribute can be now defined as such:: Specification ============= -The :external+py3.13:data:`typing.ReadOnly` type qualifier becomes a valid annotation -for attributes of classes and protocols. +The :external+py3.13:data:`typing.ReadOnly` :external+typing:term:`type qualifier` +becomes a valid annotation for :term:`attributes ` of classes and protocols. +It is used to indicate that an attribute should not be reassigned or ``del``\ eted. + +The deletability rule should be extended to ``Final`` as well, as it is currently +not specified. -It remains invalid in annotations of global and local variables, as in those contexts -it would have the same meaning as using ``Final``. +Akin to ``Final``, read-only attributes do not influence the mutability of +the assigned object. Immutable ABCs and containers may be used in combination with +``ReadOnly`` to prevent mutation of such values. Syntax ------ -``ReadOnly`` can be used at class-level or within ``__init__`` to declare -an attribute read-only: +``ReadOnly`` can be used at class-level or within ``__init__`` to mark individual +attributes read-only: .. code-block:: python - class Base: + class Book: id: ReadOnly[int] - def __init__(self, id: int, rate: float) -> None: + def __init__(self, id: int, name: str) -> None: self.id = id - self.rate: ReadOnly = rate + self.name: ReadOnly = name -The explicit type in ``ReadOnly[]`` can be omitted if an initializing value -is assigned to the attribute. A type checker should apply its usual type inference -rules to determine the type of ``rate``. +The explicit type in ``ReadOnly[]`` can be omitted if the declaration has +an initializing value. A type checker should apply its usual type inference +rules to determine the type of ``name``. -In contexts where an attribute is already implied to be read-only, like in the -frozen :ref:`classes`, it should be valid to explicitly declare it ``ReadOnly``: +If an attribute is already implied to be read-only, like in the frozen :ref:`classes`, +explicit declarations should be permitted and seen as equivalent, except ``Final`` +additionally forbids overriding in subclasses: .. code-block:: python @dataclass(frozen=True) class Point: x: ReadOnly[int] - y: ReadOnly[int] + y: Final[int] Initialization -------------- Assignment to a ``ReadOnly`` attribute can only occur as a part of the declaration, or within ``__init__`` of the same class. There is no restriction to how many -times the attribute can be assigned to in those contexts. Example: +times the attribute can be assigned to. .. code-block:: python @@ -235,28 +239,43 @@ times the attribute can be assigned to in those contexts. Example: self.songs = [] - band = Band(name="Boa", songs=["Duvet"]) - band.name = "Emma" # ok: "name" is not read-only + band = Band(name="Bôa", songs=["Duvet"]) + band.name = "Python" # ok: "name" is not read-only band.songs = [] # error: "songs" is read-only band.songs.append("Twilight") # ok: list is mutable -Classes which do not define `__slots__ `_ -retain the ability to initialize a read-only attribute both at declaration and -within ``__init__``: + + class SubBand(Band): + def __init__(self) -> None: + # error: cannot assign to a read-only attribute of base class + self.songs = [] + +An initializing value at a class level can serve as a `flyweight `_ +default for instances: .. code-block:: python - class Foo: - number: ReadOnly[int] = 0 + class Patient: + number: ReadOnly = 0 def __init__(self, number: int | None = None) -> None: if number is not None: self.number = number -Type checkers should warn on attributes declared ``ReadOnly`` which may be left -uninitialized after ``__init__`` exits, unless the class is a protocol or an ABC:: +This feature should also be supported by ``Final`` attributes. Specifically, +``Final`` attributes initialized in a class body **should no longer** imply ``ClassVar``, +and should remain assignable to within ``__init__``. - class Foo: +.. note:: + Classes defining :data:`~object.__slots__` cannot make use of this feature. + An attribute with a class-level value cannot be included in slots, + effectively making it a class variable. + Type checkers may warn or suggest explicitly marking the attribute as a ``ClassVar``. + +Type checkers should warn on read-only attributes which may be left uninitialized +after ``__init__`` exits, except in :external+typing:term:`stubs `, protocols or ABCs:: + + class Patient: id: ReadOnly[int] # error: "id" is not initialized on all code paths name: ReadOnly[str] # error: "name" is never initialized @@ -264,9 +283,9 @@ uninitialized after ``__init__`` exits, unless the class is a protocol or an ABC if random.random() > 0.5: self.id = 123 -This rule stems from the fact the body of the class declaring the attribute is -the only place able to initialize it, in contrast to non-read-only attributes, -which could be initialized outside of the class body. + + class HasName(Protocol): + name: ReadOnly[str] # ok Subtyping --------- @@ -275,7 +294,7 @@ Read-only attributes are covariant. This has a few subtyping implications. Borrowing from :pep:`PEP 705 <705#inheritance>`: * Read-only attributes can be redeclared as writable attributes, descriptors - and class variables:: + or class variables:: @dataclass class HasTitle: @@ -311,7 +330,7 @@ Borrowing from :pep:`PEP 705 <705#inheritance>`: game = Game(title="DOOM", year=1993) game.title = "DOOM II" # error: attribute is read-only -* Subtypes can narrow the type of read-only attributes:: +* Subtypes can :external+typing:term:`narrow` the type of read-only attributes:: class GameCollection(Protocol): games: ReadOnly[abc.Collection[Game]] @@ -322,8 +341,8 @@ Borrowing from :pep:`PEP 705 <705#inheritance>`: name: str games: ReadOnly[list[Game]] # ok: list[Game] is assignable to Collection[Game] -* In nominal subclasses of protocols and ABCs, a read-only attribute should be - considered abstract, unless it is initialized:: +* Nominal subclasses of protocols and ABCs should redeclare read-only attributes + in order to implement them, unless the base class initializes them in some way:: class MyBase(abc.ABC): foo: ReadOnly[int] @@ -334,14 +353,14 @@ Borrowing from :pep:`PEP 705 <705#inheritance>`: self.baz = baz @abstractmethod - def do_something(self) -> None: ... + def pprint(self) -> None: ... @final class MySubclass(MyBase): # error: MySubclass does not override "foo" - def do_something(self) -> None: + def pprint(self) -> None: print(self.foo, self.bar, self.baz) * In a protocol attribute declaration, ``name: ReadOnly[T]`` indicates that a structural @@ -373,19 +392,6 @@ Borrowing from :pep:`PEP 705 <705#inheritance>`: has_name = NamedClassVar() has_name = NamedDescriptor() - -Changes to ``Final`` --------------------- - -.. TODO - - deletability - - final in protocols? - - interaction of Final and ReadOnly - once parity changes are done, the - following shouldn't be true: - ``ReadOnly`` cannot be combined with ``Final``, as the two qualifiers differ in - initialization rules, leading to ambiguity and/or significance of ordering. - - section on "Type consistency"? - Interaction with other special types ------------------------------------ @@ -409,6 +415,9 @@ defined in :pep:`705`. ``ClassVar`` excludes read-only attributes from being assignable to within initialization methods. +Rules of ``Final`` should take priority when combined with ``ReadOnly``. As such, +type checkers may warn on the redundancy of combining the two type qualifiers. + Backwards Compatibility ======================= @@ -435,18 +444,81 @@ How to Teach This [How to teach users, new and experienced, how to apply the PEP to their work.] -Rejected ideas -============== +Open Issues +=========== + +Assignment in ``__new__`` +------------------------- + +Immutable classes like :class:`fractions.Fraction` often do not define ``__init__``; +instead, they perform initialization in ``__new__`` or classmethods. The proposed +feature won't be useful to them. + +OTOH, allowing assignment within ``__new__`` (and/or classmethods) could open way +to non-trivial bugs: + +.. code-block:: python + + class Foo: + # fully initialized objects + object_cache: ReadOnly[ClassVar[dict[int, Self]]] = {} + + foo: ReadOnly[int] + + def __new__(cls, foo: int) -> Self: + if foo + 1 in cls.object_cache: + # this instance is already initialized + self = cls.object_cache[foo + 1] + + else: + # this instance is not + self = super().__new__(cls) + + # assignment to an object which has been initialized before, + # breaking the invariant a read-only attribute can be assigned to + # only during its initialization? + self.foo = foo + + cls.object_cache[foo] = self + return self + +To my understanding, properly detecting this problem would require type checkers +to keep track of the "level of initialization" of an object. + +This issue doesn't seem to impact ``__init__``, since it's rather uncommon to +ever rebind ``self`` within it to any other object, and type checkers could +flag the action as whole. + + +Extending initialization +------------------------ + +Mechanisms such as :func:`dataclasses.__post_init__` or attrs' `initialization hooks `_ +augment initialization by providing a set of dunder hooks which will be called +once during instance creation. The current rules would disallow assignment in those +hooks. Specifying any single method in the pep isn't enough, as the naming and +functionality differs between mechanisms (``__post_init__`` vs ``__attrs_post_init__``). + +``ReadOnly[ClassVar[...]]`` and ``__init_subclass__`` +----------------------------------------------------- + +Should this be allowed? + +.. code-block:: python + + class URI: + protocol: ReadOnly[ClassVar[str]] = "" + + def __init_subclass__(cls, protocol: str = "") -> None: + cls.foo = protocol + + class File(URI, protocol="file"): ... -Assignment in ``__post_init__`` -------------------------------- +``Final`` in protocols +---------------------- -An earlier version of this PEP specified that assignment to read-only attributes -may also be permitted within methods augmenting initialization, such as -dataclasses' `__post_init__ `_ -or attrs' `initialization hooks `_. -This has been set aside for now, as defining rules regarding inclusion of -such methods has proven difficult. +It's been `suggested `_ +to clarify in this PEP whether ``Final`` should be supported by protocols. Footnotes @@ -463,8 +535,8 @@ Footnotes be desirable the name is read-only at runtime. .. [#invalid_typevar] - The implied type variable is not valid in this context. It has been used here - for ease of demonstration. + The implied type variable is not valid in this context; it has been used for + the ease of demonstration. See `ClassVar `_. Copyright From b179bfb0b112c76028c5df6ef054a7ead0fa47f7 Mon Sep 17 00:00:00 2001 From: Eneg <42005170+Enegg@users.noreply.github.com> Date: Tue, 19 Nov 2024 04:23:54 +0100 Subject: [PATCH 08/19] Placeholder pep number --- peps/{pep-0763.rst => pep-9999.rst} | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) rename peps/{pep-0763.rst => pep-9999.rst} (99%) diff --git a/peps/pep-0763.rst b/peps/pep-9999.rst similarity index 99% rename from peps/pep-0763.rst rename to peps/pep-9999.rst index 75002674082..66a177d60af 100644 --- a/peps/pep-0763.rst +++ b/peps/pep-9999.rst @@ -1,4 +1,4 @@ -PEP: 763 +PEP: 9999 Title: Classes & protocols: Read-only attributes Author: Sponsor: Carl Meyer From dfec951890de3341c954456ee88923d77ce4b77e Mon Sep 17 00:00:00 2001 From: Eneg <42005170+Enegg@users.noreply.github.com> Date: Thu, 21 Nov 2024 01:39:39 +0100 Subject: [PATCH 09/19] Apply suggestions from code review Co-authored-by: Jelle Zijlstra Co-authored-by: Carl Meyer --- peps/pep-9999.rst | 36 ++++++++++++++++++------------------ 1 file changed, 18 insertions(+), 18 deletions(-) diff --git a/peps/pep-9999.rst b/peps/pep-9999.rst index 66a177d60af..388fa6dfecb 100644 --- a/peps/pep-9999.rst +++ b/peps/pep-9999.rst @@ -6,8 +6,8 @@ Discussions-To: Status: Draft Type: Standards Track Topic: Typing -Created: ?-Nov-2024 -Python-Version: 3.? +Created: 18-Nov-2024 +Python-Version: 3.14 Abstract @@ -20,14 +20,14 @@ This PEP proposes expanding the scope of ``ReadOnly`` to class and protocol attributes, as a single concise way to mark them read-only. Some parity changes are also made to :data:`typing.Final`. -Akin to :pep:`705`, it makes no Python grammar changes. Correct usage of +Akin to :pep:`705`, it makes no changes to setting attributes at runtime. Correct usage of read-only attributes is intended to be enforced only by static type checkers. Motivation ========== -Python type system lacks a single concise way to mark an :term:`attribute` read-only. +The Python type system lacks a single concise way to mark an :term:`attribute` read-only. This feature is common in other object-oriented languages (such as `C# `_), and is useful for restricting attribute mutation at a type checker level, as well as defining a broad interface for structural subtyping. @@ -37,7 +37,7 @@ as defining a broad interface for structural subtyping. Classes ------- -There are 3 major ways of achieving read-only attributes, honored by type checkers: +Today, there are three major ways of achieving read-only attributes, honored by type checkers: * annotating the attribute with :data:`typing.Final`:: @@ -54,7 +54,7 @@ There are 3 major ways of achieving read-only attributes, honored by type checke - Supported by :mod:`dataclasses` (and type checkers since `typing#1669 `_). - Overriding ``number`` is not possible - the specification of ``Final`` - imposes the name cannot be overridden in subclasses. + imposes that the name cannot be overridden in subclasses. * read-only proxy via ``@property``:: @@ -97,10 +97,10 @@ Protocols --------- Paraphrasing `this post `_, -there is no way of defining a :class:`~typing.Protocol`, such that the only +there is no way of defining an attribute ``name: T`` on a :class:`~typing.Protocol`, such that the only requirements to satisfy are: -1. ``hasattr(obj, name)`` +1. ``hasattr(obj, "name")`` 2. ``isinstance(obj.name, T)`` [#invalid_typevar]_ The above are satisfiable at runtime by all of the following: @@ -135,9 +135,9 @@ Rationale These problems can be resolved by an attribute-level :external+typing:term:`type qualifier`. ``ReadOnly`` has been chosen for this role, as its name conveys the intent well, -and the newly proposed changes complement its semantics defined in the :pep:`705`. +and the newly proposed changes complement its semantics defined in :pep:`705`. -A class with a read-only instance attribute can be now defined as such:: +A class with a read-only instance attribute can now be defined as:: from typing import ReadOnly @@ -146,7 +146,7 @@ A class with a read-only instance attribute can be now defined as such:: def __init__(self, id: int) -> None: self.id: ReadOnly = id -...and a protocol described in :ref:`protocols` is now just:: +...and the protocol described in :ref:`protocols` is now just:: from typing import Protocol, ReadOnly @@ -170,10 +170,10 @@ Specification The :external+py3.13:data:`typing.ReadOnly` :external+typing:term:`type qualifier` becomes a valid annotation for :term:`attributes ` of classes and protocols. -It is used to indicate that an attribute should not be reassigned or ``del``\ eted. +Type checkers should error on any attempt to reassign or ``del``\ ete an attribute annotated with ``ReadOnly``. -The deletability rule should be extended to ``Final`` as well, as it is currently -not specified. +Type checkers should also error on any attempt to delete an attribute annotated as ``Final``. +(This is not currently specified.) Akin to ``Final``, read-only attributes do not influence the mutability of the assigned object. Immutable ABCs and containers may be used in combination with @@ -198,8 +198,8 @@ The explicit type in ``ReadOnly[]`` can be omitted if the declaration has an initializing value. A type checker should apply its usual type inference rules to determine the type of ``name``. -If an attribute is already implied to be read-only, like in the frozen :ref:`classes`, -explicit declarations should be permitted and seen as equivalent, except ``Final`` +If an attribute is already implied to be read-only, like in frozen :ref:`classes`, +explicit declarations should be permitted and seen as equivalent, except that ``Final`` additionally forbids overriding in subclasses: .. code-block:: python @@ -364,7 +364,7 @@ Borrowing from :pep:`PEP 705 <705#inheritance>`: print(self.foo, self.bar, self.baz) * In a protocol attribute declaration, ``name: ReadOnly[T]`` indicates that a structural - subtype must support ``.name`` access, and the returned value is compatible with ``T``:: + subtype must support ``.name`` access, and the returned value is assignable to ``T``:: class HasName(Protocol): name: ReadOnly[str] @@ -496,7 +496,7 @@ Extending initialization Mechanisms such as :func:`dataclasses.__post_init__` or attrs' `initialization hooks `_ augment initialization by providing a set of dunder hooks which will be called once during instance creation. The current rules would disallow assignment in those -hooks. Specifying any single method in the pep isn't enough, as the naming and +hooks. Specifying any single method in the PEP isn't enough, as the naming and functionality differs between mechanisms (``__post_init__`` vs ``__attrs_post_init__``). ``ReadOnly[ClassVar[...]]`` and ``__init_subclass__`` From cc85100af6a3291f0e300c996db96e69c51a491c Mon Sep 17 00:00:00 2001 From: Eneg <42005170+Enegg@users.noreply.github.com> Date: Thu, 21 Nov 2024 01:44:02 +0100 Subject: [PATCH 10/19] Update PEP number --- peps/{pep-9999.rst => pep-0767.rst} | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) rename peps/{pep-9999.rst => pep-0767.rst} (99%) diff --git a/peps/pep-9999.rst b/peps/pep-0767.rst similarity index 99% rename from peps/pep-9999.rst rename to peps/pep-0767.rst index 388fa6dfecb..d73e7e3d185 100644 --- a/peps/pep-9999.rst +++ b/peps/pep-0767.rst @@ -1,4 +1,4 @@ -PEP: 9999 +PEP: 767 Title: Classes & protocols: Read-only attributes Author: Sponsor: Carl Meyer From e36c19012b6a234bc2d30d8ce68d81246d7506a1 Mon Sep 17 00:00:00 2001 From: Eneg <42005170+Enegg@users.noreply.github.com> Date: Fri, 22 Nov 2024 02:59:22 +0100 Subject: [PATCH 11/19] Add `__new__` and precise initialization, TypeScript, ReadOnly must have a type, better immutability examples, yeet changes to Final --- peps/pep-0767.rst | 222 +++++++++++++++++++++++----------------------- 1 file changed, 113 insertions(+), 109 deletions(-) diff --git a/peps/pep-0767.rst b/peps/pep-0767.rst index d73e7e3d185..c41a4b08a0f 100644 --- a/peps/pep-0767.rst +++ b/peps/pep-0767.rst @@ -1,5 +1,5 @@ PEP: 767 -Title: Classes & protocols: Read-only attributes +Title: Read-only class and protocol attributes Author: Sponsor: Carl Meyer Discussions-To: @@ -20,17 +20,19 @@ This PEP proposes expanding the scope of ``ReadOnly`` to class and protocol attributes, as a single concise way to mark them read-only. Some parity changes are also made to :data:`typing.Final`. -Akin to :pep:`705`, it makes no changes to setting attributes at runtime. Correct usage of -read-only attributes is intended to be enforced only by static type checkers. +Akin to the :pep:`705`, it makes no changes to setting attributes at runtime. Correct +usage of read-only attributes is intended to be enforced only by static type checkers. Motivation ========== The Python type system lacks a single concise way to mark an :term:`attribute` read-only. -This feature is common in other object-oriented languages (such as `C# `_), -and is useful for restricting attribute mutation at a type checker level, as well -as defining a broad interface for structural subtyping. +This feature is present in other statically and gradually typed languages +(such as `C# `_ +or `TypeScript `_), +and is useful for restricting attribute reassignment at a type checker level, +as well as defining a broad interface for structural subtyping. .. _classes: @@ -78,11 +80,11 @@ Today, there are three major ways of achieving read-only attributes, honored by @dataclass(frozen=True) class Foo: - number: int + number: int # implicitly read-only class Bar(NamedTuple): - number: int + number: int # implicitly read-only - Overriding ``number`` is possible in the ``@dataclass`` case. - Read-only at runtime. [#runtime]_ @@ -97,8 +99,8 @@ Protocols --------- Paraphrasing `this post `_, -there is no way of defining an attribute ``name: T`` on a :class:`~typing.Protocol`, such that the only -requirements to satisfy are: +there is no way of defining an attribute ``name: T`` on a :class:`~typing.Protocol`, +such that the only requirements to satisfy are: 1. ``hasattr(obj, "name")`` 2. ``isinstance(obj.name, T)`` [#invalid_typevar]_ @@ -135,7 +137,7 @@ Rationale These problems can be resolved by an attribute-level :external+typing:term:`type qualifier`. ``ReadOnly`` has been chosen for this role, as its name conveys the intent well, -and the newly proposed changes complement its semantics defined in :pep:`705`. +and the newly proposed changes complement its semantics defined in the :pep:`705`. A class with a read-only instance attribute can now be defined as:: @@ -144,7 +146,7 @@ A class with a read-only instance attribute can now be defined as:: class Member: def __init__(self, id: int) -> None: - self.id: ReadOnly = id + self.id: ReadOnly[int] = id ...and the protocol described in :ref:`protocols` is now just:: @@ -170,51 +172,98 @@ Specification The :external+py3.13:data:`typing.ReadOnly` :external+typing:term:`type qualifier` becomes a valid annotation for :term:`attributes ` of classes and protocols. -Type checkers should error on any attempt to reassign or ``del``\ ete an attribute annotated with ``ReadOnly``. +It can be used at class-level or within ``__init__`` to mark individual attributes read-only:: + + class Book: + id: ReadOnly[int] + + def __init__(self, id: int, name: str) -> None: + self.id = id + self.name: ReadOnly[str] = name + +Type checkers should error on any attempt to reassign or ``del``\ ete an attribute +annotated with ``ReadOnly``. Type checkers should also error on any attempt to delete an attribute annotated as ``Final``. (This is not currently specified.) -Akin to ``Final``, read-only attributes do not influence the mutability of -the assigned object. Immutable ABCs and containers may be used in combination with -``ReadOnly`` to prevent mutation of such values. +Akin to ``Final`` [#final_mutability]_, ``ReadOnly`` does not influence how +type checkers perceive the mutability of the assigned object. Immutable :term:`ABCs ` +and :mod:`containers ` may be used in combination with ``ReadOnly`` +to forbid mutation of such values: -Syntax ------- +.. code-block:: python -``ReadOnly`` can be used at class-level or within ``__init__`` to mark individual -attributes read-only: + from collections import abc + from dataclasses import dataclass + from typing import Protocol, ReadOnly -.. code-block:: python - class Book: - id: ReadOnly[int] + @dataclass + class Game: + name: str - def __init__(self, id: int, name: str) -> None: - self.id = id - self.name: ReadOnly = name -The explicit type in ``ReadOnly[]`` can be omitted if the declaration has -an initializing value. A type checker should apply its usual type inference -rules to determine the type of ``name``. + class HasGames[T: abc.Collection[Game]](Protocol): + games: ReadOnly[T] + + + def add_games(shelf: HasGames[list[Game]]) -> None: + shelf.games.append(Game("Half-Life")) # ok: list is mutable + shelf.games[-1].name = "Black Mesa" # ok: "name" is not read-only + shelf.games = [] # error: "games" is read-only + del shelf.games # error: "games" is read-only and cannot be deleted + + + def read_games(shelf: HasGames[abc.Sequence[Game]]) -> None: + shelf.games.append(...) # error: "Sequence" has no attribute "append" + shelf.games[0].name = "Blue Shift" # ok: "name" is not read-only + shelf.games = [] # error: "games" is read-only + -If an attribute is already implied to be read-only, like in frozen :ref:`classes`, -explicit declarations should be permitted and seen as equivalent, except that ``Final`` -additionally forbids overriding in subclasses: +All instance attributes of frozen dataclasses and ``NamedTuple`` should be +implied to be read-only. Type checkers **should not** flag redundant annotations +of such attributes with ``ReadOnly``: .. code-block:: python + from dataclasses import dataclass + from typing import NewType, ReadOnly + + @dataclass(frozen=True) class Point: - x: ReadOnly[int] - y: Final[int] + x: int # implicit read-only + y: ReadOnly[int] # ok, explicit read-only + + + uint = NewType("uint", int) + + + @dataclass(frozen=True) + class UnsignedPoint(Point): + x: ReadOnly[uint] # ok, explicit & narrower type + y: uint # ok, narrower type Initialization -------------- -Assignment to a ``ReadOnly`` attribute can only occur as a part of the declaration, -or within ``__init__`` of the same class. There is no restriction to how many -times the attribute can be assigned to. +Assignment to a read-only attribute can only occur in the class declaring the attribute. +There is no restriction to how many times the attribute can be assigned to. +The assignment can happen only\* in the following contexts: + +1. In the body of ``__init__``, on the instance received as the first parameter (likely, ``self``). +2. In the body of ``__new__``, on instances of the declaring class created via + a direct call to a super-class' ``__new__`` method. +3. In the body of the class. + +\*A type checker may choose to allow assignment to read-only attributes on instances +of the declaring class in ``__new__``, without regard to the origin of the instance. +(This choice trades soundness, as the instance may already be initialized, +for the simplicity of implementation.) + +Note that child classes cannot assign to read-only attributes of parent classes +in any of the aforementioned contexts, unless the attributes are redeclared. .. code-block:: python @@ -240,40 +289,34 @@ times the attribute can be assigned to. band = Band(name="Bôa", songs=["Duvet"]) - band.name = "Python" # ok: "name" is not read-only - band.songs = [] # error: "songs" is read-only + band.name = "Python" # ok: "name" is not read-only + band.songs = [] # error: "songs" is read-only band.songs.append("Twilight") # ok: list is mutable class SubBand(Band): def __init__(self) -> None: - # error: cannot assign to a read-only attribute of base class - self.songs = [] + self.songs = [] # error: cannot assign to a read-only attribute of base class -An initializing value at a class level can serve as a `flyweight `_ +When a class-level declaration has an initializing value, it can serve as a `flyweight `_ default for instances: .. code-block:: python class Patient: - number: ReadOnly = 0 + number: ReadOnly[int] = 0 def __init__(self, number: int | None = None) -> None: if number is not None: self.number = number -This feature should also be supported by ``Final`` attributes. Specifically, -``Final`` attributes initialized in a class body **should no longer** imply ``ClassVar``, -and should remain assignable to within ``__init__``. - .. note:: - Classes defining :data:`~object.__slots__` cannot make use of this feature. - An attribute with a class-level value cannot be included in slots, - effectively making it a class variable. - Type checkers may warn or suggest explicitly marking the attribute as a ``ClassVar``. + This feature conflicts with :data:`~object.__slots__`. An attribute with + a class-level value cannot be included in slots, effectively making it a class variable. -Type checkers should warn on read-only attributes which may be left uninitialized -after ``__init__`` exits, except in :external+typing:term:`stubs `, protocols or ABCs:: +Type checkers can choose to warn on read-only attributes which may be left uninitialized +after an instance is created (except in :external+typing:term:`stubs `, +protocols or ABCs):: class Patient: id: ReadOnly[int] # error: "id" is not initialized on all code paths @@ -322,13 +365,16 @@ Borrowing from :pep:`PEP 705 <705#inheritance>`: * If a read-only attribute is not redeclared, it remains read-only:: - @dataclass class Game(HasTitle): year: int + def __init__(self, title: str, year: int) -> None: + self.title = title # error: cannot assign to a read-only attribute of base class + self.year = year - game = Game(title="DOOM", year=1993) - game.title = "DOOM II" # error: attribute is read-only + + game = Game(title="Robot Wants Kitty", year=2010) + game.title = "Robot Wants Puppy" # error: "title" is read-only * Subtypes can :external+typing:term:`narrow` the type of read-only attributes:: @@ -412,8 +458,8 @@ Interaction with other special types This is consistent with the interaction of ``ReadOnly`` and :class:`typing.TypedDict` defined in :pep:`705`. -``ClassVar`` excludes read-only attributes from being assignable to within -initialization methods. +An attribute annotated as both ``ReadOnly`` and ``ClassVar`` cannot be assigned to +within ``__new__`` or ``__init__``. Rules of ``Final`` should take priority when combined with ``ReadOnly``. As such, type checkers may warn on the redundancy of combining the two type qualifiers. @@ -429,7 +475,10 @@ However, caution is advised while using the backported ``typing_extensions.ReadO in older versions of Python. Mechanisms inspecting annotations may behave incorrectly when encountering ``ReadOnly``; in particular, the ``@dataclass`` decorator which `looks for `_ -``ClassVar`` will incorrectly treat ``ReadOnly[ClassVar[...]]`` as an instance attribute. +``ClassVar`` may incorrectly treat ``ReadOnly[ClassVar[...]]`` as an instance attribute. + +In such circumstances, authors should prefer ``ClassVar[ReadOnly[...]]`` over +``ReadOnly[ClassVar[...]]``. Security Implications @@ -447,49 +496,6 @@ How to Teach This Open Issues =========== -Assignment in ``__new__`` -------------------------- - -Immutable classes like :class:`fractions.Fraction` often do not define ``__init__``; -instead, they perform initialization in ``__new__`` or classmethods. The proposed -feature won't be useful to them. - -OTOH, allowing assignment within ``__new__`` (and/or classmethods) could open way -to non-trivial bugs: - -.. code-block:: python - - class Foo: - # fully initialized objects - object_cache: ReadOnly[ClassVar[dict[int, Self]]] = {} - - foo: ReadOnly[int] - - def __new__(cls, foo: int) -> Self: - if foo + 1 in cls.object_cache: - # this instance is already initialized - self = cls.object_cache[foo + 1] - - else: - # this instance is not - self = super().__new__(cls) - - # assignment to an object which has been initialized before, - # breaking the invariant a read-only attribute can be assigned to - # only during its initialization? - self.foo = foo - - cls.object_cache[foo] = self - return self - -To my understanding, properly detecting this problem would require type checkers -to keep track of the "level of initialization" of an object. - -This issue doesn't seem to impact ``__init__``, since it's rather uncommon to -ever rebind ``self`` within it to any other object, and type checkers could -flag the action as whole. - - Extending initialization ------------------------ @@ -502,7 +508,8 @@ functionality differs between mechanisms (``__post_init__`` vs ``__attrs_post_in ``ReadOnly[ClassVar[...]]`` and ``__init_subclass__`` ----------------------------------------------------- -Should this be allowed? +Should read-only class variables be assignable to within the defining class' +``__init_subclass__``? .. code-block:: python @@ -514,12 +521,6 @@ Should this be allowed? class File(URI, protocol="file"): ... -``Final`` in protocols ----------------------- - -It's been `suggested `_ -to clarify in this PEP whether ``Final`` should be supported by protocols. - Footnotes ========= @@ -538,6 +539,9 @@ Footnotes The implied type variable is not valid in this context; it has been used for the ease of demonstration. See `ClassVar `_. +.. [#final_mutability] + As noted above the second-to-last code example of https://typing.readthedocs.io/en/latest/spec/qualifiers.html#semantics-and-examples + Copyright ========= From 19185777d9b9b71aea6b99ff2226f9ea364e938b Mon Sep 17 00:00:00 2001 From: Eneg <42005170+Enegg@users.noreply.github.com> Date: Fri, 22 Nov 2024 03:21:17 +0100 Subject: [PATCH 12/19] New title --- peps/pep-0767.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/peps/pep-0767.rst b/peps/pep-0767.rst index c41a4b08a0f..36093a8eda6 100644 --- a/peps/pep-0767.rst +++ b/peps/pep-0767.rst @@ -1,5 +1,5 @@ PEP: 767 -Title: Read-only class and protocol attributes +Title: Annotating Read-Only Attributes Author: Sponsor: Carl Meyer Discussions-To: From df4ddc38d7cb57150a4b85ba06ebcf929bbf330a Mon Sep 17 00:00:00 2001 From: Eneg <42005170+Enegg@users.noreply.github.com> Date: Fri, 22 Nov 2024 04:27:49 +0100 Subject: [PATCH 13/19] Clarify that initialization must be allowed --- peps/pep-0767.rst | 33 ++++++++++++++++++--------------- 1 file changed, 18 insertions(+), 15 deletions(-) diff --git a/peps/pep-0767.rst b/peps/pep-0767.rst index 36093a8eda6..6dc94556215 100644 --- a/peps/pep-0767.rst +++ b/peps/pep-0767.rst @@ -250,20 +250,24 @@ Initialization Assignment to a read-only attribute can only occur in the class declaring the attribute. There is no restriction to how many times the attribute can be assigned to. -The assignment can happen only\* in the following contexts: +The assignment must be allowed in the following contexts: -1. In the body of ``__init__``, on the instance received as the first parameter (likely, ``self``). -2. In the body of ``__new__``, on instances of the declaring class created via - a direct call to a super-class' ``__new__`` method. -3. In the body of the class. +1. In ``__init__``, on the instance received as the first parameter (likely, ``self``). +2. In ``__new__``, on instances of the declaring class created via a call + to a super-class' ``__new__`` method. +3. At declaration in the body of the class. -\*A type checker may choose to allow assignment to read-only attributes on instances -of the declaring class in ``__new__``, without regard to the origin of the instance. -(This choice trades soundness, as the instance may already be initialized, -for the simplicity of implementation.) +Additionally, a type checker may choose to allow the assignment: -Note that child classes cannot assign to read-only attributes of parent classes -in any of the aforementioned contexts, unless the attributes are redeclared. +1. In ``__new__``, on instances of the declaring class, without regard + to the origin of the instance. + (This choice trades soundness, as the instance may already be initialized, + for the simplicity of implementation.) +2. In ``@classmethod``\ s, on instances of the declaring class created via + a call to the class' or super-class' ``__new__`` method. + +Note that a child class cannot assign to any read-only attribute of a parent class +in any of the aforementioned contexts, unless the attribute is redeclared. .. code-block:: python @@ -296,7 +300,7 @@ in any of the aforementioned contexts, unless the attributes are redeclared. class SubBand(Band): def __init__(self) -> None: - self.songs = [] # error: cannot assign to a read-only attribute of base class + self.songs = [] # error: cannot assign to a read-only attribute of a base class When a class-level declaration has an initializing value, it can serve as a `flyweight `_ default for instances: @@ -475,10 +479,9 @@ However, caution is advised while using the backported ``typing_extensions.ReadO in older versions of Python. Mechanisms inspecting annotations may behave incorrectly when encountering ``ReadOnly``; in particular, the ``@dataclass`` decorator which `looks for `_ -``ClassVar`` may incorrectly treat ``ReadOnly[ClassVar[...]]`` as an instance attribute. +``ClassVar`` may mistakenly treat ``ReadOnly[ClassVar[...]]`` as an instance attribute. -In such circumstances, authors should prefer ``ClassVar[ReadOnly[...]]`` over -``ReadOnly[ClassVar[...]]``. +To avoid issues with introspection, use ``ClassVar[ReadOnly[...]]`` instead of ``ReadOnly[ClassVar[...]]``. Security Implications From 15b4ed6ce2a32b325acc5b6714ee108608ffb9c0 Mon Sep 17 00:00:00 2001 From: Eneg <42005170+Enegg@users.noreply.github.com> Date: Fri, 22 Nov 2024 05:31:22 +0100 Subject: [PATCH 14/19] Rewrite the `Protocols` section --- peps/pep-0767.rst | 41 ++++++++++++++++++----------------------- 1 file changed, 18 insertions(+), 23 deletions(-) diff --git a/peps/pep-0767.rst b/peps/pep-0767.rst index 6dc94556215..a4b452ed5f5 100644 --- a/peps/pep-0767.rst +++ b/peps/pep-0767.rst @@ -98,39 +98,32 @@ Today, there are three major ways of achieving read-only attributes, honored by Protocols --------- -Paraphrasing `this post `_, -there is no way of defining an attribute ``name: T`` on a :class:`~typing.Protocol`, -such that the only requirements to satisfy are: +A read-only structural attribute ``name: T`` on a :class:`~typing.Protocol` in principle +defines the following two requirements: 1. ``hasattr(obj, "name")`` -2. ``isinstance(obj.name, T)`` [#invalid_typevar]_ +2. ``isinstance(obj.name, T)`` -The above are satisfiable at runtime by all of the following: +Those requirements are satisfiable at runtime by all of the following: 1. an object with an attribute ``name: T``, -2. a class with a class variable ``name: ClassVar[T]``, [#invalid_typevar]_ +2. a class with a class variable ``name: ClassVar[T]``, 3. an instance of the class above, 4. an object with a ``@property`` ``def name(self) -> T``, 5. an object with a custom descriptor, such as :func:`functools.cached_property`. -Note that the attribute being marked ``Final`` or the property defining a setter -do not impact this. - -The most common practice is to define such a protocol with a ``@property``:: +The current `typing spec `_ +defines that read-only protocol variables can be created using (abstract) properties:: class HasName(Protocol): @property def name(self) -> T: ... -Type checkers special-case this definition, such that objects with plain attributes -are assignable to the type. However, instances with class variables and descriptors -other than ``property`` are rejected. - -Covering the extra possibilities induces a great amount of boilerplate, involving -creation of an abstract descriptor protocol, possibly also accounting for -class and instance level overloads. -Worse yet, all of that is multiplied for each additional read-only attribute. - +- The syntax is somewhat verbose. +- It is not obvious that the quality conveyed here is the read-only character of a property. +- Not composable with ``ClassVar`` or ``Annotated``. +- Not all type checkers agree [#property_in_protocol]_ that all of the above five + objects are assignable to this structural type. Rationale ========= @@ -162,7 +155,7 @@ A class with a read-only instance attribute can now be defined as:: * A subclass of ``Member`` can redefine ``.id`` as a writable attribute or a :term:`descriptor`. It can also :external+typing:term:`narrow` the type. -* The ``HasName`` protocol can be implemented by any mechanism allowing for ``.name`` access. +* The ``HasName`` protocol succinctly expresses operations available on its attribute. * The ``greet`` function can now accept a wide variety of compatible objects, while being explicit about no modifications being done to the input. @@ -538,9 +531,11 @@ Footnotes This PEP focuses solely on the type-checking behavior. Nevertheless, it should be desirable the name is read-only at runtime. -.. [#invalid_typevar] - The implied type variable is not valid in this context; it has been used for - the ease of demonstration. See `ClassVar `_. +.. [#property_in_protocol] + Pyright disallows class variable and non-property descriptor overrides. + `[Pyright] `_ + `[mypy] `_ + `[Pyre] `_ .. [#final_mutability] As noted above the second-to-last code example of https://typing.readthedocs.io/en/latest/spec/qualifiers.html#semantics-and-examples From 2df7747132628d4c923a3491a3e86fc10617de81 Mon Sep 17 00:00:00 2001 From: Eneg <42005170+Enegg@users.noreply.github.com> Date: Fri, 22 Nov 2024 05:38:20 +0100 Subject: [PATCH 15/19] Add Carl as the codeowner --- .github/CODEOWNERS | 1 + 1 file changed, 1 insertion(+) diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS index a06a5f3d45d..119861bcbca 100644 --- a/.github/CODEOWNERS +++ b/.github/CODEOWNERS @@ -639,6 +639,7 @@ peps/pep-0758.rst @pablogsal @brettcannon peps/pep-0759.rst @warsaw peps/pep-0760.rst @pablogsal @brettcannon peps/pep-0761.rst @sethmlarson @hugovk +peps/pep-0767.rst @carljm # ... peps/pep-0777.rst @warsaw # ... From 5449b036f8abf5bef998bdd31b5bae0289de5ca4 Mon Sep 17 00:00:00 2001 From: Eneg <42005170+Enegg@users.noreply.github.com> Date: Fri, 22 Nov 2024 07:06:32 +0100 Subject: [PATCH 16/19] Add me as author --- peps/pep-0767.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/peps/pep-0767.rst b/peps/pep-0767.rst index a4b452ed5f5..085978a870c 100644 --- a/peps/pep-0767.rst +++ b/peps/pep-0767.rst @@ -1,6 +1,6 @@ PEP: 767 Title: Annotating Read-Only Attributes -Author: +Author: Eneg Sponsor: Carl Meyer Discussions-To: Status: Draft From 87ab8842cc46c8d8a42b8c60131efcd2485e173d Mon Sep 17 00:00:00 2001 From: Eneg <42005170+Enegg@users.noreply.github.com> Date: Tue, 26 Nov 2024 13:06:10 +0100 Subject: [PATCH 17/19] Note PEP only governs attributes; type checkers may flag redundant ReadOnly; `__new__` and `classmethod` example via simplified Fraction; Note Final can override ReadOnly; Rejected Ideas x2; Expand on Extending Initialization --- peps/pep-0767.rst | 195 +++++++++++++++++++++++++++++++++------------- 1 file changed, 141 insertions(+), 54 deletions(-) diff --git a/peps/pep-0767.rst b/peps/pep-0767.rst index 085978a870c..bd0dba1f701 100644 --- a/peps/pep-0767.rst +++ b/peps/pep-0767.rst @@ -16,23 +16,22 @@ Abstract :pep:`705` introduced the :external+py3.13:data:`typing.ReadOnly` type qualifier to allow defining read-only :class:`typing.TypedDict` items. -This PEP proposes expanding the scope of ``ReadOnly`` to class and protocol -attributes, as a single concise way to mark them read-only. Some parity changes -are also made to :data:`typing.Final`. +This PEP proposes using ``ReadOnly`` in :term:`annotations ` of class and protocol +:term:`attributes `, as a single concise way to mark them read-only. -Akin to the :pep:`705`, it makes no changes to setting attributes at runtime. Correct +Akin to :pep:`705`, it makes no changes to setting attributes at runtime. Correct usage of read-only attributes is intended to be enforced only by static type checkers. Motivation ========== -The Python type system lacks a single concise way to mark an :term:`attribute` read-only. +The Python type system lacks a single concise way to mark an attribute read-only. This feature is present in other statically and gradually typed languages (such as `C# `_ or `TypeScript `_), -and is useful for restricting attribute reassignment at a type checker level, -as well as defining a broad interface for structural subtyping. +and is useful for removing the ability to reassign or ``del``\ ete an attribute +at a type checker level, as well as defining a broad interface for structural subtyping. .. _classes: @@ -98,39 +97,41 @@ Today, there are three major ways of achieving read-only attributes, honored by Protocols --------- -A read-only structural attribute ``name: T`` on a :class:`~typing.Protocol` in principle -defines the following two requirements: +A read-only attribute ``name: T`` on a :class:`~typing.Protocol` in principle +defines two requirements: 1. ``hasattr(obj, "name")`` 2. ``isinstance(obj.name, T)`` Those requirements are satisfiable at runtime by all of the following: -1. an object with an attribute ``name: T``, -2. a class with a class variable ``name: ClassVar[T]``, -3. an instance of the class above, -4. an object with a ``@property`` ``def name(self) -> T``, -5. an object with a custom descriptor, such as :func:`functools.cached_property`. +* an object with an attribute ``name: T``, +* a class with a class variable ``name: ClassVar[T]``, +* an instance of the class above, +* an object with a ``@property`` ``def name(self) -> T``, +* an object with a custom descriptor, such as :func:`functools.cached_property`. The current `typing spec `_ -defines that read-only protocol variables can be created using (abstract) properties:: +allows creation of such protocol members using (abstract) properties:: class HasName(Protocol): @property def name(self) -> T: ... -- The syntax is somewhat verbose. -- It is not obvious that the quality conveyed here is the read-only character of a property. -- Not composable with ``ClassVar`` or ``Annotated``. -- Not all type checkers agree [#property_in_protocol]_ that all of the above five +This syntax has several drawbacks: + +* It is somewhat verbose. +* It is not obvious that the quality conveyed here is the read-only character of a property. +* It is not composable with :external+typing:term:`type qualifiers `. +* Not all type checkers agree [#property_in_protocol]_ that all of the above five objects are assignable to this structural type. Rationale ========= -These problems can be resolved by an attribute-level :external+typing:term:`type qualifier`. +These problems can be resolved by an attribute-level type qualifier. ``ReadOnly`` has been chosen for this role, as its name conveys the intent well, -and the newly proposed changes complement its semantics defined in the :pep:`705`. +and the newly proposed changes complement its semantics defined in :pep:`705`. A class with a read-only instance attribute can now be defined as:: @@ -155,7 +156,8 @@ A class with a read-only instance attribute can now be defined as:: * A subclass of ``Member`` can redefine ``.id`` as a writable attribute or a :term:`descriptor`. It can also :external+typing:term:`narrow` the type. -* The ``HasName`` protocol succinctly expresses operations available on its attribute. +* The ``HasName`` protocol has a more succinct definition, and is agnostic + to the writability of the attribute. * The ``greet`` function can now accept a wide variety of compatible objects, while being explicit about no modifications being done to the input. @@ -176,14 +178,17 @@ It can be used at class-level or within ``__init__`` to mark individual attribut Type checkers should error on any attempt to reassign or ``del``\ ete an attribute annotated with ``ReadOnly``. - Type checkers should also error on any attempt to delete an attribute annotated as ``Final``. (This is not currently specified.) +Use of ``ReadOnly`` in annotations at other sites where it currently has no meaning +(such as local/global variables or function parameters) is considered out of scope +for this PEP. + Akin to ``Final`` [#final_mutability]_, ``ReadOnly`` does not influence how type checkers perceive the mutability of the assigned object. Immutable :term:`ABCs ` and :mod:`containers ` may be used in combination with ``ReadOnly`` -to forbid mutation of such values: +to forbid mutation of such values at a type checker level: .. code-block:: python @@ -215,8 +220,8 @@ to forbid mutation of such values: All instance attributes of frozen dataclasses and ``NamedTuple`` should be -implied to be read-only. Type checkers **should not** flag redundant annotations -of such attributes with ``ReadOnly``: +implied to be read-only. Type checkers may inform that annotating such attributes +with ``ReadOnly`` is redundant, but it should not be seen as an error: .. code-block:: python @@ -227,7 +232,7 @@ of such attributes with ``ReadOnly``: @dataclass(frozen=True) class Point: x: int # implicit read-only - y: ReadOnly[int] # ok, explicit read-only + y: ReadOnly[int] # ok, redundant uint = NewType("uint", int) @@ -235,8 +240,10 @@ of such attributes with ``ReadOnly``: @dataclass(frozen=True) class UnsignedPoint(Point): - x: ReadOnly[uint] # ok, explicit & narrower type - y: uint # ok, narrower type + x: ReadOnly[uint] # ok, redundant; narrower type + y: Final[uint] # not redundant, Final imposes extra restrictions; narrower type + +.. _init: Initialization -------------- @@ -245,21 +252,21 @@ Assignment to a read-only attribute can only occur in the class declaring the at There is no restriction to how many times the attribute can be assigned to. The assignment must be allowed in the following contexts: -1. In ``__init__``, on the instance received as the first parameter (likely, ``self``). -2. In ``__new__``, on instances of the declaring class created via a call - to a super-class' ``__new__`` method. -3. At declaration in the body of the class. +* In ``__init__``, on the instance received as the first parameter (likely, ``self``). +* In ``__new__``, on instances of the declaring class created via a call + to a super-class' ``__new__`` method. +* At declaration in the body of the class. Additionally, a type checker may choose to allow the assignment: -1. In ``__new__``, on instances of the declaring class, without regard - to the origin of the instance. - (This choice trades soundness, as the instance may already be initialized, - for the simplicity of implementation.) -2. In ``@classmethod``\ s, on instances of the declaring class created via - a call to the class' or super-class' ``__new__`` method. +* In ``__new__``, on instances of the declaring class, without regard + to the origin of the instance. + (This choice trades soundness, as the instance may already be initialized, + for the simplicity of implementation.) +* In ``@classmethod``\ s, on instances of the declaring class created via + a call to the class' or super-class' ``__new__`` method. -Note that a child class cannot assign to any read-only attribute of a parent class +Note that a child class cannot assign to any read-only attributes of a parent class in any of the aforementioned contexts, unless the attribute is redeclared. .. code-block:: python @@ -277,8 +284,7 @@ in any of the aforementioned contexts, unless the attribute is redeclared. self.songs = [] if songs is not None: - # multiple assignments during initialization are fine - self.songs = list(songs) + self.songs = list(songs) # multiple assignments are fine def clear(self) -> None: # error: assignment to read-only "songs" outside initialization @@ -295,6 +301,36 @@ in any of the aforementioned contexts, unless the attribute is redeclared. def __init__(self) -> None: self.songs = [] # error: cannot assign to a read-only attribute of a base class +.. code-block:: python + + # a simplified immutable Fraction class + class Fraction: + numerator: ReadOnly[int] + denominator: ReadOnly[int] + + def __new__( + cls, + numerator: str | int | float | Decimal | Rational = 0, + denominator: int | Rational | None = None + ) -> Self: + self = super().__new__(cls) + + if denominator is None: + if type(numerator) is int: + self.numerator = numerator + self.denominator = 1 + return self + + elif isinstance(numerator, Rational): ... + + else: ... + + @classmethod + def from_float(cls, f: float, /) -> Self: + self = super().__new__(cls) + self.numerator, self.denominator = f.as_integer_ratio() + return self + When a class-level declaration has an initializing value, it can serve as a `flyweight `_ default for instances: @@ -311,7 +347,7 @@ default for instances: This feature conflicts with :data:`~object.__slots__`. An attribute with a class-level value cannot be included in slots, effectively making it a class variable. -Type checkers can choose to warn on read-only attributes which may be left uninitialized +Type checkers may choose to warn on read-only attributes which could be left uninitialized after an instance is created (except in :external+typing:term:`stubs `, protocols or ABCs):: @@ -366,6 +402,7 @@ Borrowing from :pep:`PEP 705 <705#inheritance>`: year: int def __init__(self, title: str, year: int) -> None: + super().__init__(title) self.title = title # error: cannot assign to a read-only attribute of base class self.year = year @@ -435,7 +472,7 @@ Borrowing from :pep:`PEP 705 <705#inheritance>`: has_name = NamedClassVar() has_name = NamedDescriptor() -Interaction with other special types +Interaction with Other Special Types ------------------------------------ ``ReadOnly`` can be used with ``ClassVar`` and ``Annotated`` in any nesting order: @@ -455,11 +492,13 @@ Interaction with other special types This is consistent with the interaction of ``ReadOnly`` and :class:`typing.TypedDict` defined in :pep:`705`. -An attribute annotated as both ``ReadOnly`` and ``ClassVar`` cannot be assigned to -within ``__new__`` or ``__init__``. +An attribute annotated as both ``ReadOnly`` and ``ClassVar`` can only be assigned to +at declaration in the class body. -Rules of ``Final`` should take priority when combined with ``ReadOnly``. As such, -type checkers may warn on the redundancy of combining the two type qualifiers. +An attribute cannot be annotated as both ``ReadOnly`` and ``Final``, as the two +qualifiers differ in semantics, and ``Final`` is generally more restrictive. +``Final`` remains allowed as an annotation of attributes that are only implied +to be read-only. It can be also used to redeclare a ``ReadOnly`` attribute of a base class. Backwards Compatibility @@ -489,22 +528,70 @@ How to Teach This [How to teach users, new and experienced, how to apply the PEP to their work.] +Rejected Ideas +============== + +Clarifying Interaction of ``@property`` and Protocols +----------------------------------------------------- + +The :ref:`protocols` section mentions an inconsistency between type checkers in +the interpretation of properties in protocols. The problem could be fixed +by amending the typing specification, clarifying what implements the read-only +quality of such properties. + +This PEP makes ``ReadOnly`` a better alternative for defining read-only attributes +in protocols, superseding the use of properties for this purpose. + + +Assignment Only in ``__init__`` and Class Body +---------------------------------------------- + +An earlier version of this PEP proposed that read-only attributes could only be +assigned to in ``__init__`` and the class' body. A later discussion revealed that +this restriction would severely limit the usability of ``ReadOnly`` within +immutable classes, which typically do not define ``__init__``. + +:class:`fractions.Fraction` is one example of an immutable class, where the +initialization of its attributes happens within ``__new__`` and classmethods. +However, unlike in ``__init__``, the assignment in ``__new__`` and classmethods +is potentially unsound, as the instance they work on can be sourced from +an arbitrary place, including an already finalized instance. + +We find it imperative that this type checking feature is useful to the foremost +use site of read-only attributes - immutable classes. Thus, the PEP has changed +since to allow assignment in ``__new__`` and classmethods under a set of rules +described in the :ref:`init` section. + + Open Issues =========== -Extending initialization +Extending Initialization ------------------------ Mechanisms such as :func:`dataclasses.__post_init__` or attrs' `initialization hooks `_ -augment initialization by providing a set of dunder hooks which will be called -once during instance creation. The current rules would disallow assignment in those -hooks. Specifying any single method in the PEP isn't enough, as the naming and -functionality differs between mechanisms (``__post_init__`` vs ``__attrs_post_init__``). +augment object creation by providing a set of special hooks which are called +during initialization. + +The current initialization rules defined in this PEP disallow assignment to +read-only attributes in such methods. It is unclear whether the rules could be +satisfyingly shaped in a way that is inclusive of those 3rd party hooks, while +upkeeping the invariants associated with the read-only-ness of those attributes. + +The Python type system has a long and detailed `specification `_ +regarding the behavior of ``__new__`` and ``__init__``. It is rather unfeasible +to expect the same level of detail from 3rd party hooks. + +A potential solution would involve type checkers providing configuration in this +regard, requiring end users to manually specify a set of methods they wish +to allow initialization in. This however could easily result in users mistakenly +or purposefully breaking the aforementioned invariants. It is also a fairly +big ask for a relatively niche feature. ``ReadOnly[ClassVar[...]]`` and ``__init_subclass__`` ----------------------------------------------------- -Should read-only class variables be assignable to within the defining class' +Should read-only class variables be assignable to within the declaring class' ``__init_subclass__``? .. code-block:: python From 9c3bc6865f341dc46e6a1a1f99e76efcc0c271fc Mon Sep 17 00:00:00 2001 From: Eneg <42005170+Enegg@users.noreply.github.com> Date: Tue, 26 Nov 2024 13:56:25 +0100 Subject: [PATCH 18/19] How To Teach This --- peps/pep-0767.rst | 19 +++++++++++++++++-- 1 file changed, 17 insertions(+), 2 deletions(-) diff --git a/peps/pep-0767.rst b/peps/pep-0767.rst index bd0dba1f701..1e392571a47 100644 --- a/peps/pep-0767.rst +++ b/peps/pep-0767.rst @@ -367,7 +367,7 @@ Subtyping --------- Read-only attributes are covariant. This has a few subtyping implications. -Borrowing from :pep:`PEP 705 <705#inheritance>`: +Borrowing from :pep:`705#inheritance`: * Read-only attributes can be redeclared as writable attributes, descriptors or class variables:: @@ -525,7 +525,22 @@ There are no known security consequences arising from this PEP. How to Teach This ================= -[How to teach users, new and experienced, how to apply the PEP to their work.] +Suggested changes to the :mod:`typing` module documentation, +following the footsteps of :pep:`705#how-to-teach-this`: + +* Add this PEP to the others listed. +* Link :external+py3.13:data:`typing.ReadOnly` to this PEP. +* Update the description of ``typing.ReadOnly``: + + A special typing construct to mark an attribute of a class or an item of + a ``TypedDict`` as read-only. + +* Add a standalone entry for ``ReadOnly`` under the + `type qualifiers `_ section: + + The ``ReadOnly`` type qualifier in class attribute annotations indicates + that the attribute of the class may be read, but not reassigned or ``del``\ eted. + For usage in ``TypedDict``, see `ReadOnly `_. Rejected Ideas From 0e9f86238ffad85fd0e9b9c6b6893894fa130ee4 Mon Sep 17 00:00:00 2001 From: Eneg <42005170+Enegg@users.noreply.github.com> Date: Tue, 26 Nov 2024 16:16:05 +0100 Subject: [PATCH 19/19] Apply Jelle's suggestions Co-authored-by: Jelle Zijlstra --- peps/pep-0767.rst | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/peps/pep-0767.rst b/peps/pep-0767.rst index 1e392571a47..7adc7cf62ca 100644 --- a/peps/pep-0767.rst +++ b/peps/pep-0767.rst @@ -2,7 +2,7 @@ PEP: 767 Title: Annotating Read-Only Attributes Author: Eneg Sponsor: Carl Meyer -Discussions-To: +Discussions-To: https://discuss.python.org/t/expanding-readonly-to-normal-classes-protocols/67359 Status: Draft Type: Standards Track Topic: Typing @@ -472,8 +472,8 @@ Borrowing from :pep:`705#inheritance`: has_name = NamedClassVar() has_name = NamedDescriptor() -Interaction with Other Special Types ------------------------------------- +Interaction with Other Type Qualifiers +-------------------------------------- ``ReadOnly`` can be used with ``ClassVar`` and ``Annotated`` in any nesting order: