Skip to content

Commit cf2fcc8

Browse files
author
Release Manager
committed
gh-37237: CI macOS: Fix failure with macos-13-homebrew, add tests on M1 runners, add timeouts <!-- ^^^^^ Please provide a concise, informative and self-explanatory title. Don't put issue numbers in there, do this in the PR body below. For example, instead of "Fixes #1234" use "Introduce new method to calculate 1+1" --> <!-- Describe your changes here in detail --> - Fixing the failure with `macos-13-homebrew`, as seen in https://github.com/sagemath/sage/actions/runs/7761817710/job/21171032513 and upstream uses of macos.yml dimpase/primecountpy#11 (comment), sagemath/cypari2#152 - Generalizing the `ld_classic` workaround from #36599 for the case of full XCode. - This is to fix the failure seen on `homebrew-macos-13-usrlocal- standard-xcode_15.0`, e.g. in https://github.com/scipopt/SCIP- SDP/actions/runs/7793548242/job/21253452882?pr=9#step:10:2766 - Deactivating the `ld_classic` workaround when `LD` is set, as is in a conda environment; as seen e.g. in https://groups.google.com/g/sage- devel/c/1viBzw-ZaoQ/m/S_u9K20xAAAJ - [Apple Silicon runners recently became available](https://github.blog/changelog/2024-01-30-github-actions- introducing-the-new-m1-macos-runner-available-to-open-source/); we add some tests (example run: https://github.com/mkoeppe/sage/actions/runs/7772086965/job/21194110510) - On Apple Silicon, `tox -e local-conda-forge` now uses the correct installer - Intel runners were recently upgraded too, increase parallelism. - Add self-destruct sequence to cancel workflows before the 6h limit (catches linbox-related build hang) - Actually handle the input `extra_sage_packages` (for scipopt/PySCIPOpt#630) <!-- Why is this change required? What problem does it solve? --> <!-- If this PR resolves an open issue, please link to it here. For example "Fixes #12345". --> <!-- If your change requires a documentation PR, please link it appropriately. --> Follow-ups, not addressed here: - conda-forge-macos-latest-minimal tachyon build failure - needs #36969 - adding a config that tests arm64 conda-forge-minimal - would need config.guess updates in numerous packages - arm64 conda-forge-standard - matplotlib build failure (https://github. com/mkoeppe/sage/actions/runs/7810111886/job/21313997438#step:10:2749) - hangs or excessive build time that lead to timeouts, probably from linbox ### :memo: Checklist <!-- Put an `x` in all the boxes that apply. --> <!-- If your change requires a documentation PR, please link it appropriately --> <!-- If you're unsure about any of these, don't hesitate to ask. We're here to help! --> <!-- Feel free to remove irrelevant items. --> - [x] The title is concise, informative, and self-explanatory. - [x] The description explains in detail what this PR is about. - [x] I have linked a relevant issue or discussion. - [ ] I have created tests covering the changes. - [ ] I have updated the documentation accordingly. ### :hourglass: Dependencies <!-- List all open PRs that this PR logically depends on - #12345: short description why this is a dependency - #34567: ... --> <!-- If you're unsure about any of these, don't hesitate to ask. We're here to help! --> URL: #37237 Reported by: Matthias Köppe Reviewer(s): Matthias Köppe, Tobias Diez
2 parents 14a228f + 2165bef commit cf2fcc8

File tree

5 files changed

+46
-17
lines changed

5 files changed

+46
-17
lines changed

‎.github/workflows/ci-macos.yml‎

Lines changed: 6 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -34,6 +34,7 @@ jobs:
3434
uses: ./.github/workflows/macos.yml
3535
with:
3636
stage: "1"
37+
timeout: 14400
3738

3839
stage-2:
3940
uses: ./.github/workflows/macos.yml
@@ -90,7 +91,7 @@ jobs:
9091
run: |
9192
git config --global user.email "nobody@example.com"
9293
git config --global user.name "Sage GitHub CI"
93-
SAGE_ROOT=. SAGE_SRC=./src src/bin/sage-update-version $(git describe --tags) || echo "(ignoring error)"
94+
SAGE_ROOT=. SAGE_SRC=./src src/bin/sage-update-version $(cat src/VERSION.txt).dev0 || echo "(ignoring error)"
9495
- name: make dist
9596
run: |
9697
./configure --enable-download-from-upstream-url && make dist
@@ -108,7 +109,7 @@ jobs:
108109
fail-fast: false
109110
max-parallel: 4
110111
matrix:
111-
os: [ macos-11, macos-12 ]
112+
os: [ macos-11, macos-12, macos-14 ]
112113
tox_system_factor: [macos-nobootstrap]
113114
tox_packages_factor: [minimal]
114115
xcode_version_factor: [default]
@@ -129,7 +130,7 @@ jobs:
129130
if: contains(matrix.tox_system_factor, 'nobootstrap')
130131
- name: Move homebrew away
131132
run: |
132-
(cd /usr/local && for a in bin etc include lib opt sbin share; do sudo mv $a $a-moved; done)
133+
(cd $(brew --prefix) && for a in bin etc include lib opt sbin share; do sudo mv $a $a-moved; done)
133134
- name: Select Xcode version
134135
run: |
135136
if [ ${{ matrix.xcode_version_factor }} != default ]; then sudo xcode-select -s /Applications/Xcode_${{ matrix.xcode_version_factor }}.app; fi
@@ -140,7 +141,8 @@ jobs:
140141
# We use a high parallelization on purpose in order to catch possible parallelization bugs in the build scripts.
141142
# For doctesting, we use a lower parallelization to avoid timeouts.
142143
run: |
143-
MAKE="make -j12" tox -e $TOX_ENV -- SAGE_NUM_THREADS=4 $TARGETS
144+
(sleep 20000; pkill make) &
145+
MAKE="make -j12" tox -e $TOX_ENV -- SAGE_NUM_THREADS=6 $TARGETS
144146
- name: Prepare logs artifact
145147
run: |
146148
mkdir -p "artifacts/$LOGS_ARTIFACT_NAME"; cp -r .tox/*/log "artifacts/$LOGS_ARTIFACT_NAME"

‎.github/workflows/macos.yml‎

Lines changed: 24 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -19,17 +19,21 @@ on:
1919
type: string
2020
# System configuration
2121
osversion_xcodeversion_toxenv_tuples:
22+
# As of 2024-02, "runs-on: macos-latest" is macos-12.
23+
# and "runs-on: macos-14" selects the new M1 runners.
24+
# https://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners/about-github-hosted-runners#standard-github-hosted-runners-for-public-repositories
2225
description: 'Stringified JSON object'
2326
default: >-
24-
[["latest", "", "homebrew-macos-usrlocal-minimal"],
25-
["latest", "", "homebrew-macos-usrlocal-standard"],
26-
["11", "xcode_13.2.1", "homebrew-macos-usrlocal-minimal"],
27+
[["11", "xcode_13.2.1", "homebrew-macos-usrlocal-minimal"],
28+
["12", "", "homebrew-macos-usrlocal-minimal"],
2729
["12", "", "homebrew-macos-usrlocal-standard"],
30+
["12", "", "homebrew-macos-usrlocal-python3_xcode-standard"],
31+
["12", "", "homebrew-macos-usrlocal-maximal"],
2832
["13", "xcode_15.0", "homebrew-macos-usrlocal-standard"],
29-
["latest", "", "homebrew-macos-usrlocal-maximal"],
30-
["latest", "", "homebrew-macos-usrlocal-python3_xcode-standard"],
33+
["14", "", "homebrew-macos-opthomebrew-standard"],
3134
["latest", "", "conda-forge-macos-minimal"],
32-
["latest", "", "conda-forge-macos-standard"]]
35+
["latest", "", "conda-forge-macos-standard"],
36+
["14", "", "conda-forge-macos-standard"]]
3337
type: string
3438
extra_sage_packages:
3539
description: 'Extra Sage packages to install as system packages'
@@ -41,6 +45,10 @@ on:
4145
free_disk_space:
4246
default: false
4347
type: boolean
48+
timeout:
49+
description: 'Elapsed time (seconds) at which to kill the build'
50+
default: 20000
51+
type: number
4452
#
4553
# For use in upstream CIs.
4654
#
@@ -74,10 +82,16 @@ jobs:
7482
repository: ${{ inputs.sage_repo }}
7583
ref: ${{ inputs.sage_ref }}
7684
fetch-depth: 10000
77-
85+
- uses: actions/setup-python@v5
86+
# As of 2024-02-03, the macOS M1 runners do not have preinstalled python or pipx.
87+
# Installing pipx follows the approach of https://github.com/pypa/cibuildwheel/pull/1743
88+
id: python
89+
with:
90+
python-version: "3.8 - 3.12"
91+
update-environment: false
7892
- name: Install test prerequisites
7993
run: |
80-
brew install tox
94+
"${{ steps.python.outputs.python-path }}" -m pip install pipx
8195
- name: Download upstream artifact
8296
uses: actions/download-artifact@v3
8397
with:
@@ -129,7 +143,8 @@ jobs:
129143
*) export TARGETS_PRE="${{ inputs.targets_pre }}" TARGETS="${{ inputs.targets }} TARGETS_OPTIONAL="${{ inputs.targets_optional }}
130144
;;
131145
esac
132-
MAKE="make -j12" tox -e $TOX_ENV -- SAGE_NUM_THREADS=4 $TARGETS
146+
(sleep ${{ inputs.timeout }}; pkill make) &
147+
MAKE="make -j12" EXTRA_SAGE_PACKAGES="${{ inputs.extra_sage_packages }}" "${{ steps.python.outputs.python-path }}" -m pipx run tox -e $TOX_ENV -- SAGE_NUM_THREADS=6 $TARGETS
133148
- name: Prepare logs artifact
134149
run: |
135150
mkdir -p "artifacts/$LOGS_ARTIFACT_NAME"; cp -r .tox/*/log "artifacts/$LOGS_ARTIFACT_NAME"

‎build/pkgs/dsdp/spkg-install.in‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
11
cd src
22
cp ../patches/CMakeLists.txt .
33
sdh_cmake -DCMAKE_BUILD_TYPE=Release \
4+
-DCMAKE_VERBOSE_MAKEFILE=ON \
45
-DBUILD_SHARED_LIBS=ON \
56
-DBLA_VENDOR=OpenBLAS \
67
-DBLAS_LIBRARIES="$(pkg-config --libs blas)" \

‎src/bin/sage-env‎

Lines changed: 9 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -127,7 +127,7 @@ fi
127127
# The compilers are set in order of priority by
128128
# 1) environment variables
129129
# 2) compiler installed by sage
130-
# 3) compiler set at configuration time
130+
# 3) compiler set at configuration time
131131
if [ -z "$CC" ]; then
132132
if [ -n "$SAGE_LOCAL" -a -x "$SAGE_LOCAL/bin/gcc" ]; then
133133
CC=gcc
@@ -373,7 +373,14 @@ if [ -n "$SAGE_LOCAL" ]; then
373373
# On OS X, test whether "ld-classic" is present in the installed
374374
# version of the command-line tools. If so, we add "-ld_classic"
375375
# to LD_FLAGS. See #36599.
376-
if [ "$UNAME" = "Darwin" ] && [ -x "$(xcode-select -p)/usr/bin/ld-classic" ] ; then
376+
# When the full XCode is installed and in use, for example after
377+
# "sudo xcode-select -s /Applications/Xcode.app", then "xcode-select -p"
378+
# gives "/Applications/Xcode.app/Contents/Developer", but "ld-classic"
379+
# is not in the subdirectory "usr/bin/" but rather in the subdirectory
380+
# "Toolchains/XcodeDefault.xctoolchain/usr/bin/". See #37237.
381+
# However, if LD is set explicitly, as it is within conda on macOS,
382+
# do not not do this.
383+
if [ "$UNAME" = "Darwin" ] && [ -z "$LD" ] && [ -x "$(xcode-select -p)/usr/bin/ld-classic" -o -x "$(xcode-select -p)/Toolchains/XcodeDefault.xctoolchain/usr/bin/ld-classic" ] ; then
377384
LDFLAGS="-L$SAGE_LOCAL/lib -Wl,-ld_classic,-rpath,$SAGE_LOCAL/lib $LDFLAGS"
378385
else
379386
LDFLAGS="-L$SAGE_LOCAL/lib -Wl,-rpath,$SAGE_LOCAL/lib $LDFLAGS"

‎tox.ini‎

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -512,6 +512,9 @@ setenv =
512512
# brew caches downloaded files in ${HOME}/Library/Caches. We share it between different toxenvs.
513513
homebrew: SYSTEM=homebrew
514514
local-homebrew: HOMEBREW={envdir}/homebrew
515+
# /opt/homebrew is the default install location on arm64 macOS
516+
local-homebrew-opthomebrew: HOMEBREW=/opt/homebrew
517+
# /usr/local is the default install location on x86_64 macOS
515518
local-{homebrew-usrlocal,nohomebrew}: HOMEBREW=/usr/local
516519
# local-macos-nohomebrew: "best effort" isolation to avoid using a homebrew installation in /usr/local
517520
# We use liblzma from the macOS system - which is available but its headers are not (neither is the xz executable).
@@ -538,10 +541,11 @@ setenv =
538541
local-conda: PATH={env:CONDA_PREFIX}/bin:/usr/bin:/bin:/usr/sbin:/sbin
539542
local-conda: CONDA_PKGS_DIRS={env:SHARED_CACHE_DIR}/conda_pkgs
540543
local-conda: CONDA_OS=$(uname | sed 's/^Darwin/MacOSX/;')
544+
local-conda: CONDA_ARCH=$(uname -m)
541545
local-conda-forge: CONDA_INSTALLER_URL_BASE=https://github.com/conda-forge/miniforge/releases/latest/download/
542-
local-conda-forge: CONDA_INSTALLER_FILE=Miniforge3-{env:CONDA_OS}-x86_64.sh
546+
local-conda-forge: CONDA_INSTALLER_FILE=Miniforge3-{env:CONDA_OS}-{env:CONDA_ARCH}.sh
543547
local-conda-miniconda: CONDA_INSTALLER_URL_BASE=https://repo.anaconda.com/miniconda/
544-
local-conda-miniconda: CONDA_INSTALLER_FILE=Miniconda3-latest-{env:CONDA_OS}-x86_64.sh
548+
local-conda-miniconda: CONDA_INSTALLER_FILE=Miniconda3-latest-{env:CONDA_OS}-{env:CONDA_ARCH}.sh
545549
local-conda: SETENV=. {env:CONDA_PREFIX}/bin/activate base
546550
local-conda-environment: CONDA_SAGE_ENVIRONMENT=sage-build
547551
local-conda-environment: CONDA_SAGE_ENVIRONMENT_DIR=

0 commit comments

Comments
 (0)