From fe44edd66c5c5bf4f890b6a4d10631db252373ad Mon Sep 17 00:00:00 2001 From: Ed Morley <501702+edmorley@users.noreply.github.com> Date: Mon, 6 Oct 2025 13:10:59 +0100 Subject: [PATCH] Stop compiling Python using `--with-system-expat` The upstream Python test suite (which gets run when compiling with PGO enabled) fails with the `libexpat` in Ubuntu 22.04. In #1661 previously added what I hoped would be a temporarily workaround until the failures were fixed upstream, however, the Python maintainers say they don't guarantee compatibility with distro `expat`, and that it's up to us to test for compatibility and patch if we want to use the distro version. However, this isn't viable given that we're neither a Linux distro maintainer, a CPython maintainer or an expat maintainer. Instead, like the upstream Docker Hub Python images, we will switch the `expat` bundled within the CPython sources, which is actually what the upstream CPython project tests in its CI. This means users won't get security updates for free via the base image, and will instead need to update their Python patch versions instead as newer versions are vendored in CPython. However, this is the least worst alternative for now. I'm doing this now, since otherwise I'll need to generate another patch series for the soon to be released Python 3.14. Note: This change only affects Python versions compiled/released after this merges. Existing Python versions on S3 are unaffected for now (unless they ever get recompiled in the future). For more details, see: https://github.com/python/cpython/issues/125067#issuecomment-2517445480 GUS-W-17414073. --- CHANGELOG.md | 1 + builds/Dockerfile | 2 +- builds/build_python_runtime.sh | 11 --------- ....13-ubuntu-22.04-libexpat-workaround.patch | 24 ------------------- 4 files changed, 2 insertions(+), 36 deletions(-) delete mode 100644 builds/python-3.13-ubuntu-22.04-libexpat-workaround.patch diff --git a/CHANGELOG.md b/CHANGELOG.md index 0a6068f8a..ac8b53a59 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,7 @@ ## [Unreleased] +- Stopped using `--with-system-expat` when compiling new Python versions. ([#1925](https://github.com/heroku/heroku-buildpack-python/pull/1925)) ## [v312] - 2025-10-05 diff --git a/builds/Dockerfile b/builds/Dockerfile index 8e83a738a..e9f0703fc 100644 --- a/builds/Dockerfile +++ b/builds/Dockerfile @@ -17,4 +17,4 @@ RUN apt-get update --error-on=any \ COPY --from=cosign /ko-app/cosign /usr/local/bin/cosign WORKDIR /tmp -COPY build_python_runtime.sh python-3.13-ubuntu-22.04-libexpat-workaround.patch . +COPY build_python_runtime.sh . diff --git a/builds/build_python_runtime.sh b/builds/build_python_runtime.sh index 18a8dc5ea..f78e72b35 100755 --- a/builds/build_python_runtime.sh +++ b/builds/build_python_runtime.sh @@ -83,14 +83,6 @@ cosign verify-blob \ tar --extract --file python.tgz --strip-components=1 --directory "${SRC_DIR}" cd "${SRC_DIR}" -# Work around PGO profile test failures with Python 3.13 on Ubuntu 22.04, due to the tests -# checking the raw libexpat version which doesn't account for Ubuntu backports: -# https://github.com/heroku/heroku-buildpack-python/pull/1661#issuecomment-2405259352 -# https://github.com/python/cpython/issues/125067 -if [[ "${PYTHON_MAJOR_VERSION}" == "3.13" && "${STACK}" == "heroku-22" ]]; then - patch -p1