Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion .github/PULL_REQUEST_TEMPLATE.md
Original file line number Diff line number Diff line change
@@ -1 +1,3 @@
- [ ] I updated the package version in pyproject.toml and made sure the first 3 numbers match `git describe --tags --abbrev=8` in OpenBLAS at the `OPENBLAS_COMMIT`. If I did not update `OPENBLAS_COMMIT`, I incremented the wheel build number (i.e. 0.3.29.0.0 to 0.3.29.0.1)
- [ ] I updated the package version in pyproject.toml and made sure the first 3 numbers match `./openblas_commit.txt`. If I did not update `./openblas_commit.txt`, I incremented the wheel build number (i.e. 0.3.29.0.0 to 0.3.29.0.1)

Note: update `./openblas_commit.txt` with `cd OpenBLAS; git describe --tags --abbrev=8 > ../openblas_commit.txt`
9 changes: 0 additions & 9 deletions .github/workflows/posix.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,9 +11,6 @@ on:
schedule:
- cron: '0 0 * * 0'

env:
OPENBLAS_COMMIT: "v0.3.30-359-g29fab2b9"

jobs:
build:
runs-on: ${{ matrix.os }}
Expand Down Expand Up @@ -118,12 +115,6 @@ jobs:
path: dist/scipy_openblas*.whl


- name: Upload openblas to artifacts
uses: actions/upload-artifact@v4.3.0
with:
name: openblas-${{ matrix.os }}-${{ matrix.PLAT }}-${{ matrix.INTERFACE64 }}-${{ matrix.MB_ML_LIBC }}-${{ matrix.MB_ML_VER }}
path: dist/openblas*.tar.gz

- uses: conda-incubator/setup-miniconda@v3.2.0
with:
channels: conda-forge
Expand Down
1 change: 0 additions & 1 deletion .github/workflows/windows-arm.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,6 @@ on:
- maintenance/**

env:
OPENBLAS_COMMIT: "v0.3.30-359-g29fab2b9"
OPENBLAS_ROOT: "c:\\opt"
# Preserve working directory for calls into bash
# Without this, invoking bash will cd to the home directory
Expand Down
8 changes: 1 addition & 7 deletions .github/workflows/windows.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,6 @@ on:
workflow_dispatch: null

env:
OPENBLAS_COMMIT: "v0.3.30-359-g29fab2b9"
OPENBLAS_ROOT: "c:\\opt"
# Preserve working directory for calls into bash
# Without this, invoking bash will cd to the home directory
Expand Down Expand Up @@ -71,7 +70,7 @@ jobs:
- name: Build
run: |
git submodule update --init --recursive
& $env:BASH_PATH -lc tools/build_openblas.sh
& $env:BASH_PATH -lc tools/build_steps_windows.sh

- name: Test
run: |
Expand Down Expand Up @@ -148,11 +147,6 @@ jobs:
name: wheels-${{matrix.plat }}-${{ matrix.INTERFACE64 }}
path: dist/scipy_openblas*.whl

- uses: actions/upload-artifact@v4.3.0
with:
name: openblas-${{matrix.plat }}-${{ matrix.INTERFACE64 }}
path: builds/openblas*.zip

- name: Test 64-bit interface wheel
if: matrix.INTERFACE64 == '1'
run: |
Expand Down
3 changes: 0 additions & 3 deletions .gitmodules
Original file line number Diff line number Diff line change
@@ -1,6 +1,3 @@
[submodule "OpenBLAS"]
path = OpenBLAS
url = https://github.com/xianyi/OpenBLAS.git
[submodule "gfortran-install"]
path = gfortran-install
url = https://github.com/MacPython/gfortran-install.git
2 changes: 1 addition & 1 deletion OpenBLAS
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,8 +20,8 @@

# OpenBLAS library build process

First, tarballs are built using `do_build_lib` in `tools/build_steps.sh` (on
posix in a docker and drectly on macos) or `build_openblas.sh` on windows.
First, tarballs are built using `build_lib` in `tools/build_steps.sh` (on
posix in a docker and drectly on macos) or `tools/build_steps_windows.sh` on windows.

Then the shared object and header files from the tarball are used to build the
wheel via `tools/build_wheel.sh`, and the wheels uploaded to
Expand Down
57 changes: 4 additions & 53 deletions build-openblas.sh
Original file line number Diff line number Diff line change
@@ -1,60 +1,11 @@
#! /bin/bash


# Most of the content in this file comes from https://github.com/multi-build/multibuild, with some modifications
# Follow the license below



# .. _license:

# *********************
# Copyright and License
# *********************

# The multibuild package, including all examples, code snippets and attached
# documentation is covered by the 2-clause BSD license.

# Copyright (c) 2013-2024, Matt Terry and Matthew Brett; all rights
# reserved.

# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are
# met:

# 1. Redistributions of source code must retain the above copyright notice,
# this list of conditions and the following disclaimer.

# 2. Redistributions in binary form must reproduce the above copyright
# notice, this list of conditions and the following disclaimer in the
# documentation and/or other materials provided with the distribution.

# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS
# IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
# THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
# PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR
# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
# EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
# PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
# PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
# LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
# NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
# SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.


set -xeo pipefail
source tools/build_steps.sh
echo "------ BEFORE BUILD ---------"
before_build

if [[ "$NIGHTLY" = "true" ]]; then
echo "------ CLEAN CODE --------"
clean_code develop
echo "------ BUILD LIB --------"
build_lib "$PLAT" "$INTERFACE64" "1"
else
echo "------ CLEAN CODE --------"
clean_code $OPENBLAS_COMMIT
echo "------ BUILD LIB --------"
build_lib "$PLAT" "$INTERFACE64" "0"
fi
echo "------ CLEAN CODE --------"
clean_code $OPENBLAS_COMMIT
echo "------ BUILD LIB --------"
build_lib "$PLAT" "$INTERFACE64"
5 changes: 4 additions & 1 deletion ci-before-build.sh
Original file line number Diff line number Diff line change
Expand Up @@ -44,10 +44,13 @@ set -xe
if [[ "$NIGHTLY" = "true" ]]; then
pushd OpenBLAS
git checkout develop
export OPENBLAS_COMMIT=$(git describe --tags --abbrev=8)
# Set the pyproject.toml version: convert v0.3.24-30-g138ed79f to 0.3.34.30
version=$(git describe --tags --abbrev=8 | sed -e "s/^v\(.*\)-g.*/\1/" | sed -e "s/-/./g")
version=$(echo $OPENBLAS_COMMIT | sed -e "s/^v\(.*\)-g.*/\1/" | sed -e "s/-/./g")
popd
sed -e "s/^version = .*/version = \"${version}\"/" -i.bak pyproject.toml
else
export OPENBLAS_COMMIT=$(cat openblas_commit.txt)
fi

if [ "$(uname)" != "Darwin" ]; then
Expand Down
1 change: 0 additions & 1 deletion gfortran-install
Submodule gfortran-install deleted from 3dd38d
1 change: 1 addition & 0 deletions github_commit.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
v0.3.30-392-g5aff62eb
1 change: 1 addition & 0 deletions openblas_commit.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
v0.3.30-359-g29fab2b9
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ build-backend = "setuptools.build_meta"
[project]
name = "scipy-openblas64"
# v0.3.30-359-g29fab2b9
version = "0.3.30.359.0"
version = "0.3.30.359.1"
requires-python = ">=3.7"
description = "Provides OpenBLAS for python packaging"
readme = "README.md"
Expand Down
5 changes: 2 additions & 3 deletions tools/build_prepare.sh
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,7 @@ mkdir -p local/openblas
mkdir -p dist
$PYTHON -m pip install wheel auditwheel

# This will fail if there is more than one file in libs
tar -C local/scipy_openblas64 --strip-components=2 -xf libs/openblas*.tar.gz
tar -C local/scipy_openblas64 --strip-components=2 -xf libs/openblas.tar.gz

# do not package the static libs and symlinks, only take the shared object
find local/scipy_openblas64/lib -maxdepth 1 -type l -delete
Expand Down Expand Up @@ -69,4 +68,4 @@ fi
rm -rf dist/*


echo "The build preparation is done."
echo "The build preparation is done."
93 changes: 12 additions & 81 deletions tools/build_steps.sh
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ ROOT_DIR=$(dirname $(dirname "${BASH_SOURCE[0]}"))
MB_PYTHON_VERSION=3.9

function before_build {
# Manylinux Python version set in build_lib
# install gfortran, objconv on macOS
if [ "$(uname -s)" == "Darwin" ]; then
if [ ! -e /usr/local/lib ]; then
sudo mkdir -p /usr/local/lib
Expand Down Expand Up @@ -86,25 +86,6 @@ function get_plat_tag {
echo "macosx_${target}_${plat}"
}

function build_lib {
# OSX or manylinux build
#
# Input arg
# plat - one of i686, x86_64, arm64
# interface64 - 1 if build with INTERFACE64 and SYMBOLSUFFIX
# nightly - 1 if building for nightlies
#
# Depends on globals
# BUILD_PREFIX - install suffix e.g. "/usr/local"
# MB_ML_VER
set -x
local plat=${1:-$PLAT}
local interface64=${2:-$INTERFACE64}
local nightly=${3:0}
local manylinux=${MB_ML_VER:-1}
do_build_lib "$plat" "$interface64" "$nightly"
}

function patch_source {
# Runs inside OpenBLAS directory
# Make the patches by git format-patch <old commit>
Expand All @@ -114,19 +95,21 @@ function patch_source {
done
}

function do_build_lib {
# Build openblas lib
function build_lib {
# OSX or manylinux build
#
# Input arg
# plat - one of i686, x86_64, arm64
# interface64 (optional) - whether to build ILP64 openblas
# with 64_ symbol suffix
# nightly (optional) - whether to build for nightlies
# interface64 - 1 if build with INTERFACE64 and SYMBOLSUFFIX
#
# Depends on globals
# BUILD_PREFIX - install suffix e.g. "/usr/local"
local plat=$1
local interface64=$2
local nightly=$3
# MB_ML_VER

set -x
local plat=${1:-$PLAT}
local interface64=${2:-$INTERFACE64}

case $(uname -s)-$plat in
Linux-x86_64)
local bitness=64
Expand Down Expand Up @@ -223,11 +206,6 @@ function do_build_lib {
fi
make PREFIX=$BUILD_PREFIX $interface_flags install
popd
if [ "$nightly" = "1" ]; then
local version="HEAD"
else
local version=$(cd OpenBLAS && git describe --tags --abbrev=8)
fi
mv $BUILD_PREFIX/lib/pkgconfig/openblas*.pc $BUILD_PREFIX/lib/pkgconfig/scipy-openblas.pc
local plat_tag=$(get_plat_tag $plat)
if [ "$interface64" = "1" ]; then
Expand All @@ -242,58 +220,11 @@ function do_build_lib {
rm $BUILD_PREFIX/lib/pkgconfig/scipy-openblas.pc.bak
fi

local out_name="openblas${symbolsuffix}-${version}-${plat_tag}.tar.gz"
local out_name="openblas.tar.gz"
tar zcvf libs/$out_name \
$BUILD_PREFIX/include/*blas* \
$BUILD_PREFIX/include/*lapack* \
$BUILD_PREFIX/lib/libscipy_openblas* \
$BUILD_PREFIX/lib/pkgconfig/scipy-openblas* \
$BUILD_PREFIX/lib/cmake/openblas
}


function build_lib_on_travis {
# OSX or manylinux build
#
# Input arg
# plat - one of i686, x86_64, arm64
# interface64 - 1 if build with INTERFACE64 and SYMBOLSUFFIX
# nightly - 1 if building for nightlies
#
# Depends on globals
# BUILD_PREFIX - install suffix e.g. "/usr/local"
# MB_ML_VER
set -x
local plat=${1:-$PLAT}
local interface64=${2:-$INTERFACE64}
local nightly=${3:0}
local manylinux=${MB_ML_VER:-1}

# Manylinux wrapper
local libc=${MB_ML_LIBC:-manylinux}
local docker_image=quay.io/pypa/${libc}${manylinux}_${plat}
docker pull $docker_image
# run `do_build_lib` in the docker image
docker run --rm \
-e BUILD_PREFIX="$BUILD_PREFIX" \
-e PLAT="${plat}" \
-e INTERFACE64="${interface64}" \
-e NIGHTLY="${nightly}" \
-e PYTHON_VERSION="$MB_PYTHON_VERSION" \
-e MB_ML_VER=${manylinux} \
-e MB_ML_LIBC=${libc} \
-v $PWD:/io \
$docker_image /io/tools/docker_build_wrap.sh
}



function build_on_travis {
if [ ${TRAVIS_EVENT_TYPE} == "cron" ]; then
build_lib_on_travis "$PLAT" "$INTERFACE64" 1
version=$(cd OpenBLAS && git describe --tags --abbrev=8 | sed -e "s/^v\(.*\)-g.*/\1/" | sed -e "s/-/./g")
sed -e "s/^version = .*/version = \"${version}\"/" -i.bak pyproject.toml
else
build_lib_on_travis "$PLAT" "$INTERFACE64" 0
fi
}
11 changes: 2 additions & 9 deletions tools/build_steps_win_arm64.bat
Original file line number Diff line number Diff line change
Expand Up @@ -8,11 +8,6 @@
:: If INTERFACE64 environment variable is 1, then if_bits defaults to 64
:: Expects these binaries on the PATH:
:: clang-cl, flang-new, cmake, perl
:: Uses environment variable:
:: OPENBLAS_COMMIT (unspec -> current submodule commit, if contains
:: Windows on ARM build fixes (see below), otherwise earliest commit
:: with those fixes).

:: First commit containing WoA build fixes.
:: Minimum OpenBLAS commit to build; we'll update to this if commit not
:: present.
Expand Down Expand Up @@ -63,13 +58,11 @@ if "%if_bits%"=="64" (
echo Cloning OpenBLAS repository with submodules...
git submodule update --init --recursive OpenBLAS
if errorlevel 1 exit /b 1
set /p OPENBLAS_COMMIT=<openblas_commit.txt

:: Enter OpenBLAS directory and checkout buildable commit
cd OpenBLAS
if defined OPENBLAS_COMMIT (
echo Checking out OpenBLAS commit %OPENBLAS_COMMIT%
git checkout %OPENBLAS_COMMIT%
)
git checkout %OPENBLAS_COMMIT%
git merge-base --is-ancestor %first_woa_buildable_commit% HEAD 2>NUL
if errorlevel 1 (
echo OpenBLAS commit does not contain WoA build fixes.
Expand Down
9 changes: 3 additions & 6 deletions tools/build_openblas.sh → tools/build_steps_windows.sh
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
#!/bin/bash
# Build script for OpenBLAS on Windows
#
# Usage: build_openblas.sh [openblas_root [build_bits [if_bits]]]
# Usage: build_steps_windows.sh [openblas_root [build_bits [if_bits]]]
#
# e.g build_openblas.sh c:\\opt 64 32
# e.g build_steps_windows.sh c:\\opt 64 32
#
# Uses the optional environment variables. We always prefer command line argument
# values above to environment variable values:
Expand All @@ -13,7 +13,6 @@
# INTERFACE64 (1 for 64-bit interface, anything else or undefined for 32,
# This gives the default value if if_bits not specified above).
# START_DIR (directory containing OpenBLAS source, unspec -> .. from here)
# OPENBLAS_COMMIT (unspec -> current submodule commit)
# LDFLAGS (example: "-lucrt -static -static-libgcc")
#
# Expects at leasts these binaries on the PATH:
Expand Down Expand Up @@ -50,9 +49,7 @@ git submodule update --init --recursive

# Get / clean code
git fetch origin
if [ -n "$OPENBLAS_COMMIT" ]; then
git checkout $OPENBLAS_COMMIT
fi
git checkout $(cat ../openblas_commit.txt)
git clean -fxd
git reset --hard
rm -rf $openblas_root/$build_bits
Expand Down
Loading
Loading