From 2ca626b521e1820be1051a5e7063bfe22c55cb63 Mon Sep 17 00:00:00 2001 From: stan Date: Mon, 24 Feb 2025 17:01:02 +0000 Subject: [PATCH 01/10] Initial addition of pep 775 --- .github/CODEOWNERS | 1 + peps/pep-0775.rst | 134 +++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 135 insertions(+) create mode 100644 peps/pep-0775.rst diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS index 6766f771acd..2e5ae8a7f21 100644 --- a/.github/CODEOWNERS +++ b/.github/CODEOWNERS @@ -653,6 +653,7 @@ peps/pep-0771.rst @pradyunsg peps/pep-0772.rst @warsaw @pradyunsg peps/pep-0773.rst @zooba peps/pep-0774.rst @savannahostrowski +peps/pep-0775.rst @stanfromireland @encukou # ... peps/pep-0777.rst @warsaw # ... diff --git a/peps/pep-0775.rst b/peps/pep-0775.rst new file mode 100644 index 00000000000..473a1e3dbfb --- /dev/null +++ b/peps/pep-0775.rst @@ -0,0 +1,134 @@ +PEP: 775 +Title: Make zlib required to build Python +Author: Stan Ulbrych , + Petr Viktorin +Discussions-To: https://discuss.python.org/t/lets-make-zlib-required-rather-than-optional-to-build-cpython/23062 +Status: Draft +Type: Standards Track +Created: 20-Feb-2025 +Python-Version: 3.14 +Post-History: `23-Jan-2023 `__ + +Abstract +======== + +Building CPython on systems without the ``zlib`` compression library, except WASI, +is no longer supported. + +The ``zlib`` module is made a required part of the standard library, +except on WASI. + +Builds without ``zlib`` will still be possible, but formally unsupported. + + +Motivation +========== + +The ``zlib`` library, which powers the ``zlib`` Python module, +is available on virtually all systems. + +Many wheels on PyPI, including the `pip`_ installer, require ``zlib``. +Users of *pip* would consider CPython without ``zlib`` to be broken, +but mostly don't notice because all major builds of CPython include ``zlib``. + +.. _pip: https://pypi.org/project/pip/ + +CPython developers don't really notice either. It turns out that at the time +of this writing, at least one CPython test fails without ``zlib`` (the “skip” +decorator in ``test_peg_generator.test_c_parser`` is applied too late), +but our CI didn't catch this. + +This PEP treats this as an issue in documentation and messaging. +In practice, we already don't support building CPython without ``zlib``; we +should just say so. + + +Rationale +========= + +There are possible use cases for zlib-less builds, such as embedding and +bootstrapping, as well as unforeseen ones. +Therefore, we don't *remove* support for zlib-less systems; we mark them +unsupported and invite affected users to do their own testing, or to share +use cases that can make us reconsider this decision. + +``zlib`` is not yet used by default on the WASI platform -- mostly because +adding it hasn't yet been a priority there. (Note that `Pyodide`_, the main +“real-world” CPython distribution for WASI, does include ``zlib``.) +We take this as an opportunity to continue testing a platform without +``zlib``, so that we don't unintentionally break unsupported builds yet. + + +Specification +============= + +In standard library modules that use ``zlib`` for optional functionality, +that functionality will raise ``ImportError`` when used. +Code to generate more “friendly” error messages, or to pre-check whether +``zlib`` is available, will be removed. +All non-``zlib``-related functionality will still be usable if ``zlib`` is +missing. + +This affects the following modules, and more that depend on these +transitively: + +* ``shutil`` (``gztar`` and ``zip`` archive formats) +* ``tarfile``, ``zipfile``, ``zipimport``, ``zipapp`` (archive compression) +* ``codecs`` (``zlib_codec``) + +``shutil.get_archive_formats()`` will always include ``zip`` and ``gztar`` +as registered formats, even if they are unusable due to missing ``zlib``. + +The ``configure`` script will issue a warning when `zlib`_ is not found on +platforms other than WASI. + +``test_zlib`` will fail on platforms other than WASI. +All other tests will continue to be skipped -- that is, uses of +``@test.support.requires_zlib`` will be kept in place -- for the benefit +of WASI, unsupported builds, and any possible reverts. + +PEP 11 will be adjusted to mark "Systems without zlib, except WASI" as +unsupported. + + +Backwards Compatibility +======================= + +In practice, nothing major changes, except in error cases -- for example, +attempts to use tar compression withoutt ``zlib`` available will raise +``ImportError`` and not ``CompressionError``. + + +Security Implications +===================== + +None known. + + +How to Teach This +================= + +We don't expect that any instructions will need to change, as ``zlib`` is +already available in all relevant contexts. + + +Reference Implementation +======================== + +https://github.com/python/cpython/pull/130297 + + +Future work +=========== + +In the future, if no use cases for ``zlib``-less builds are found, +``zlib`` may be made fully required. +The main changes needed for that would be making the ``configure`` script +raise a hard error, and removing ``@test.support.requires_zlib``. + + +Copyright +========= + +This document is placed in the public domain or under the +CC0-1.0-Universal license, whichever is more permissive. From ad45795d15870420f8373c29f64cda85a453ee45 Mon Sep 17 00:00:00 2001 From: Stan Ulbrych <89152624+StanFromIreland@users.noreply.github.com> Date: Mon, 24 Feb 2025 17:09:28 +0000 Subject: [PATCH 02/10] Apply suggestions from Petr Co-authored-by: Petr Viktorin --- .github/CODEOWNERS | 2 +- peps/pep-0775.rst | 2 ++ 2 files changed, 3 insertions(+), 1 deletion(-) diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS index 2e5ae8a7f21..7eda0dc1ee8 100644 --- a/.github/CODEOWNERS +++ b/.github/CODEOWNERS @@ -653,7 +653,7 @@ peps/pep-0771.rst @pradyunsg peps/pep-0772.rst @warsaw @pradyunsg peps/pep-0773.rst @zooba peps/pep-0774.rst @savannahostrowski -peps/pep-0775.rst @stanfromireland @encukou +peps/pep-0775.rst @encukou # ... peps/pep-0777.rst @warsaw # ... diff --git a/peps/pep-0775.rst b/peps/pep-0775.rst index 473a1e3dbfb..5d61d8a85fb 100644 --- a/peps/pep-0775.rst +++ b/peps/pep-0775.rst @@ -58,6 +58,8 @@ adding it hasn't yet been a priority there. (Note that `Pyodide`_, the main We take this as an opportunity to continue testing a platform without ``zlib``, so that we don't unintentionally break unsupported builds yet. +.. _Pyodide: https://pyodide.org + Specification ============= From a0743a77bf3d4e36cb8acfaa4fc7f08c61677008 Mon Sep 17 00:00:00 2001 From: stan Date: Mon, 24 Feb 2025 17:14:28 +0000 Subject: [PATCH 03/10] Update links --- peps/pep-0775.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/peps/pep-0775.rst b/peps/pep-0775.rst index 5d61d8a85fb..425c0454b03 100644 --- a/peps/pep-0775.rst +++ b/peps/pep-0775.rst @@ -12,7 +12,7 @@ Post-History: `23-Jan-2023 `__ Abstract ======== -Building CPython on systems without the ``zlib`` compression library, except WASI, +Building CPython on systems without the `zlib `_ compression library, except WASI, is no longer supported. The ``zlib`` module is made a required part of the standard library, @@ -81,7 +81,7 @@ transitively: ``shutil.get_archive_formats()`` will always include ``zip`` and ``gztar`` as registered formats, even if they are unusable due to missing ``zlib``. -The ``configure`` script will issue a warning when `zlib`_ is not found on +The ``configure`` script will issue a warning when `zlib` is not found on platforms other than WASI. ``test_zlib`` will fail on platforms other than WASI. From b954f78a09e247325719123b8728f9dbc43c341a Mon Sep 17 00:00:00 2001 From: stan Date: Mon, 24 Feb 2025 17:16:54 +0000 Subject: [PATCH 04/10] I hate my editor --- peps/pep-0775.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/peps/pep-0775.rst b/peps/pep-0775.rst index 425c0454b03..ba3e8bd8ca1 100644 --- a/peps/pep-0775.rst +++ b/peps/pep-0775.rst @@ -81,7 +81,7 @@ transitively: ``shutil.get_archive_formats()`` will always include ``zip`` and ``gztar`` as registered formats, even if they are unusable due to missing ``zlib``. -The ``configure`` script will issue a warning when `zlib` is not found on +The ``configure`` script will issue a warning when ``zlib`` is not found on platforms other than WASI. ``test_zlib`` will fail on platforms other than WASI. From ea8eaa30d0b5e51cf08ffb0a2dacf5db78642fc5 Mon Sep 17 00:00:00 2001 From: Stan Ulbrych <89152624+StanFromIreland@users.noreply.github.com> Date: Tue, 25 Feb 2025 16:37:06 +0000 Subject: [PATCH 05/10] Apply suggestions from Hugo Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com> --- peps/pep-0775.rst | 52 +++++++++++++++++++++++------------------------ 1 file changed, 25 insertions(+), 27 deletions(-) diff --git a/peps/pep-0775.rst b/peps/pep-0775.rst index ba3e8bd8ca1..1f79fa6fde7 100644 --- a/peps/pep-0775.rst +++ b/peps/pep-0775.rst @@ -5,7 +5,7 @@ Author: Stan Ulbrych , Discussions-To: https://discuss.python.org/t/lets-make-zlib-required-rather-than-optional-to-build-cpython/23062 Status: Draft Type: Standards Track -Created: 20-Feb-2025 +Created: 24-Feb-2025 Python-Version: 3.14 Post-History: `23-Jan-2023 `__ @@ -15,31 +15,29 @@ Abstract Building CPython on systems without the `zlib `_ compression library, except WASI, is no longer supported. -The ``zlib`` module is made a required part of the standard library, +The :mod:`zlib` module is made a required part of the standard library, except on WASI. -Builds without ``zlib`` will still be possible, but formally unsupported. +Builds without zlib will still be possible, but formally unsupported. Motivation ========== -The ``zlib`` library, which powers the ``zlib`` Python module, +The zlib library, which powers the :mod:`!zlib` Python module, is available on virtually all systems. -Many wheels on PyPI, including the `pip`_ installer, require ``zlib``. -Users of *pip* would consider CPython without ``zlib`` to be broken, -but mostly don't notice because all major builds of CPython include ``zlib``. - -.. _pip: https://pypi.org/project/pip/ +Many wheels on PyPI, including the :pypi:`pip` installer, require zlib. +Users of pip would consider CPython without zlib to be broken, +but mostly don't notice because all major builds of CPython include zlib. CPython developers don't really notice either. It turns out that at the time -of this writing, at least one CPython test fails without ``zlib`` (the “skip” +of writing, at least one CPython test fails without zlib (the "skip" decorator in ``test_peg_generator.test_c_parser`` is applied too late), but our CI didn't catch this. This PEP treats this as an issue in documentation and messaging. -In practice, we already don't support building CPython without ``zlib``; we +In practice, we already don't support building CPython without zlib; we should just say so. @@ -52,11 +50,11 @@ Therefore, we don't *remove* support for zlib-less systems; we mark them unsupported and invite affected users to do their own testing, or to share use cases that can make us reconsider this decision. -``zlib`` is not yet used by default on the WASI platform -- mostly because +zlib is not yet used by default on the WASI platform -- mostly because adding it hasn't yet been a priority there. (Note that `Pyodide`_, the main -“real-world” CPython distribution for WASI, does include ``zlib``.) +"real-world" CPython distribution for WASI, does include zlib.) We take this as an opportunity to continue testing a platform without -``zlib``, so that we don't unintentionally break unsupported builds yet. +zlib, so that we don't unintentionally break unsupported builds yet. .. _Pyodide: https://pyodide.org @@ -64,24 +62,24 @@ We take this as an opportunity to continue testing a platform without Specification ============= -In standard library modules that use ``zlib`` for optional functionality, +In standard library modules that use zlib for optional functionality, that functionality will raise ``ImportError`` when used. Code to generate more “friendly” error messages, or to pre-check whether -``zlib`` is available, will be removed. -All non-``zlib``-related functionality will still be usable if ``zlib`` is +zlib is available, will be removed. +All functionality unrelated to zlib will still be usable if zlib is missing. This affects the following modules, and more that depend on these transitively: -* ``shutil`` (``gztar`` and ``zip`` archive formats) -* ``tarfile``, ``zipfile``, ``zipimport``, ``zipapp`` (archive compression) -* ``codecs`` (``zlib_codec``) +* :mod:`shutil` (``gztar`` and ``zip`` archive formats) +* :mod:`tarfile`, :mod:`zipfile`, :mod:`zipimport`, :mod:`zipapp` (archive compression) +* :mod:`codecs` (``zlib_codec``) ``shutil.get_archive_formats()`` will always include ``zip`` and ``gztar`` -as registered formats, even if they are unusable due to missing ``zlib``. +as registered formats, even if they are unusable due to missing zlib. -The ``configure`` script will issue a warning when ``zlib`` is not found on +The ``configure`` script will issue a warning when zlib is not found on platforms other than WASI. ``test_zlib`` will fail on platforms other than WASI. @@ -89,7 +87,7 @@ All other tests will continue to be skipped -- that is, uses of ``@test.support.requires_zlib`` will be kept in place -- for the benefit of WASI, unsupported builds, and any possible reverts. -PEP 11 will be adjusted to mark "Systems without zlib, except WASI" as +:pep:`11` will be adjusted to mark "Systems without zlib, except WASI" as unsupported. @@ -97,7 +95,7 @@ Backwards Compatibility ======================= In practice, nothing major changes, except in error cases -- for example, -attempts to use tar compression withoutt ``zlib`` available will raise +attempts to use tar compression without zlib available will raise ``ImportError`` and not ``CompressionError``. @@ -110,7 +108,7 @@ None known. How to Teach This ================= -We don't expect that any instructions will need to change, as ``zlib`` is +We don't expect that any instructions will need to change, as zlib is already available in all relevant contexts. @@ -123,8 +121,8 @@ https://github.com/python/cpython/pull/130297 Future work =========== -In the future, if no use cases for ``zlib``-less builds are found, -``zlib`` may be made fully required. +In the future, if no use cases for zlib-less builds are found, +zlib may be made fully required. The main changes needed for that would be making the ``configure`` script raise a hard error, and removing ``@test.support.requires_zlib``. From 5f24b6775972f12a7817342a21dc89b2da6c3514 Mon Sep 17 00:00:00 2001 From: Stan Ulbrych <89152624+StanFromIreland@users.noreply.github.com> Date: Thu, 27 Feb 2025 19:01:40 +0000 Subject: [PATCH 06/10] Apply suggestions from Adam Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com> --- peps/pep-0775.rst | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/peps/pep-0775.rst b/peps/pep-0775.rst index 1f79fa6fde7..bf25fe3e4ed 100644 --- a/peps/pep-0775.rst +++ b/peps/pep-0775.rst @@ -2,13 +2,14 @@ PEP: 775 Title: Make zlib required to build Python Author: Stan Ulbrych , Petr Viktorin -Discussions-To: https://discuss.python.org/t/lets-make-zlib-required-rather-than-optional-to-build-cpython/23062 +Discussions-To: https://discuss.python.org/t/23062 Status: Draft Type: Standards Track Created: 24-Feb-2025 Python-Version: 3.14 Post-History: `23-Jan-2023 `__ + Abstract ======== @@ -64,7 +65,7 @@ Specification In standard library modules that use zlib for optional functionality, that functionality will raise ``ImportError`` when used. -Code to generate more “friendly” error messages, or to pre-check whether +Code to generate more "friendly" error messages, or to pre-check whether zlib is available, will be removed. All functionality unrelated to zlib will still be usable if zlib is missing. @@ -73,7 +74,8 @@ This affects the following modules, and more that depend on these transitively: * :mod:`shutil` (``gztar`` and ``zip`` archive formats) -* :mod:`tarfile`, :mod:`zipfile`, :mod:`zipimport`, :mod:`zipapp` (archive compression) +* :mod:`tarfile`, :mod:`zipfile`, :mod:`zipimport`, + :mod:`zipapp` (archive compression) * :mod:`codecs` (``zlib_codec``) ``shutil.get_archive_formats()`` will always include ``zip`` and ``gztar`` @@ -115,7 +117,9 @@ already available in all relevant contexts. Reference Implementation ======================== -https://github.com/python/cpython/pull/130297 +A reference implementation may be found in a pull request to the CPython +repository, `python/cpython#130297 +`_ Future work From 273d8bde34a2fba9f3595bdda23dd510ef5b4fd3 Mon Sep 17 00:00:00 2001 From: Stan Ulbrych <89152624+StanFromIreland@users.noreply.github.com> Date: Fri, 28 Feb 2025 15:26:51 +0000 Subject: [PATCH 07/10] =?UTF-8?q?Apply=20Adam=E2=80=99s=20rewording?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com> --- peps/pep-0775.rst | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/peps/pep-0775.rst b/peps/pep-0775.rst index bf25fe3e4ed..3770a570322 100644 --- a/peps/pep-0775.rst +++ b/peps/pep-0775.rst @@ -13,13 +13,13 @@ Post-History: `23-Jan-2023 `__ Abstract ======== -Building CPython on systems without the `zlib `_ compression library, except WASI, -is no longer supported. - -The :mod:`zlib` module is made a required part of the standard library, -except on WASI. - -Builds without zlib will still be possible, but formally unsupported. +Building CPython without the `zlib `_ compression library +will no be longer supported, and the :mod:`zlib` module will be required in +the standard library. +The only exception is `WASI `_, as zlib is not currently +supported in CPython on WASI. +Building the interpreter without zlib may still be possible, +but formally unsupported. Motivation From b1ea46ef6cbd072c1bb03a3a48e796c6a552b498 Mon Sep 17 00:00:00 2001 From: stan Date: Fri, 28 Feb 2025 15:50:16 +0000 Subject: [PATCH 08/10] Remove 'virtually all' --- peps/pep-0775.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/peps/pep-0775.rst b/peps/pep-0775.rst index 3770a570322..f07d5d76183 100644 --- a/peps/pep-0775.rst +++ b/peps/pep-0775.rst @@ -26,7 +26,7 @@ Motivation ========== The zlib library, which powers the :mod:`!zlib` Python module, -is available on virtually all systems. +is available on all supported systems except WASI. Many wheels on PyPI, including the :pypi:`pip` installer, require zlib. Users of pip would consider CPython without zlib to be broken, From b231b0ea140d8f833fb355490398a143b0a9fefa Mon Sep 17 00:00:00 2001 From: stan Date: Sat, 1 Mar 2025 09:04:48 +0000 Subject: [PATCH 09/10] Add Greg --- peps/pep-0775.rst | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/peps/pep-0775.rst b/peps/pep-0775.rst index f07d5d76183..3883def4400 100644 --- a/peps/pep-0775.rst +++ b/peps/pep-0775.rst @@ -1,6 +1,7 @@ PEP: 775 Title: Make zlib required to build Python -Author: Stan Ulbrych , +Author: Gregory P. Smith , + Stan Ulbrych , Petr Viktorin Discussions-To: https://discuss.python.org/t/23062 Status: Draft From 1f360217eee760ecc009da0b8adca2c994653506 Mon Sep 17 00:00:00 2001 From: Stan Ulbrych <89152624+StanFromIreland@users.noreply.github.com> Date: Sat, 1 Mar 2025 17:28:40 +0000 Subject: [PATCH 10/10] Update title Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com> --- peps/pep-0775.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/peps/pep-0775.rst b/peps/pep-0775.rst index 3883def4400..617971f1d65 100644 --- a/peps/pep-0775.rst +++ b/peps/pep-0775.rst @@ -1,5 +1,5 @@ PEP: 775 -Title: Make zlib required to build Python +Title: Make zlib required to build CPython Author: Gregory P. Smith , Stan Ulbrych , Petr Viktorin