Skip to content

Commit ed7229d

Browse files
koukou
committed
GH-20: Add release scripts
Fixes GH-20. This is based on apache/arrow-go#122 . Workflow: Cut a RC: 1. Bump version by `dev/release/bump_version.sh 19.0.0` 2. Run `dev/release/release_rc.sh` 1. `dev/release/release_rc.sh` pushes `v${version}-rc${rc}` tag 2. `.github/workflows/rc.yml` creates `apache-arrow-java-${version}.tar.gz{,.sha256,.sha512}` 3. `.github/workflows/rc.yml` uploads `apache-arrow-java-${version}.tar.gz{,.sha256,.sha512}` to GitHub Releases 4. `dev/release/release_rc.sh` downloads `apache-arrow-java-${version}.tar.gz` from GitHub Releases 5. `dev/release/release_rc.sh` signs `apache-arrow-java-${version}.tar.gz` as `apache-arrow-go-${version}.tar.gz.asc` 6. `dev/release/release_rc.sh` uploads `apache-arrow-java-${version}.tar.gz.asc` to GitHub Releases 3. Start a vote (GitHub Actions instead of https://dist.apache.org/repos/dist/dev/arrow/ is used like ADBC.) Verify a RC: 1. Run `dev/release/verify_rc.sh` Release an approved RC: 1. Run `dev/release/release.sh` 1. `dev/release/release.sh` pushes `v${version}` tag that refers that same commit ID as `v${version}-rc${rc}` 2. `dev/release/release.sh` downloads `apache-arrow-java-${version}.tar.gz{,.asc,.sha256,.sha512}` from GitHub Actions 3. `dev/release/release.sh` uploads `apache-arrow-java-${version}.tar.gz{,.asc,.sha256,.sha512}` to https://dist.apache.org/repos/dist/release/arrow 4. `dev/release/release.sh` removes old releases from https://dist.apache.org/repos/dist/release/arrow 2. Add this release to ASF's report database: https://reporter.apache.org/addrelease.html?arrow Follow-up tasks: * Add support for building binary packages (.jar) * Add support for releasing binary packages (.jar) * Add support for running JNI test in the verification script * Add support for running integration test in the verification script * Add support for verifying binary packages (.jar) in the verification script * Add support for releasing a release note
1 parent 3440633 commit ed7229d

File tree

7 files changed

+761
-0
lines changed

7 files changed

+761
-0
lines changed

‎.github/workflows/rc.yml‎

Lines changed: 141 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,141 @@
1+
# Licensed to the Apache Software Foundation (ASF) under one
2+
# or more contributor license agreements. See the NOTICE file
3+
# distributed with this work for additional information
4+
# regarding copyright ownership. The ASF licenses this file
5+
# to you under the Apache License, Version 2.0 (the
6+
# "License"); you may not use this file except in compliance
7+
# with the License. You may obtain a copy of the License at
8+
#
9+
# http://www.apache.org/licenses/LICENSE-2.0
10+
#
11+
# Unless required by applicable law or agreed to in writing,
12+
# software distributed under the License is distributed on an
13+
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
14+
# KIND, either express or implied. See the License for the
15+
# specific language governing permissions and limitations
16+
# under the License.
17+
18+
name: RC
19+
on:
20+
push:
21+
branches:
22+
- '**'
23+
- '!dependabot/**'
24+
tags:
25+
- '*-rc*'
26+
pull_request:
27+
concurrency:
28+
group: ${{ github.repository }}-${{ github.head_ref || github.sha }}-${{ github.workflow }}
29+
cancel-in-progress: true
30+
permissions:
31+
contents: read
32+
jobs:
33+
archive:
34+
name: Archive
35+
runs-on: ubuntu-latest
36+
timeout-minutes: 5
37+
steps:
38+
- name: Checkout
39+
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
40+
with:
41+
submodules: recursive
42+
- name: Prepare for tag
43+
if: github.ref_type == 'tag'
44+
run: |
45+
version=${GITHUB_REF_NAME%-rc*}
46+
version=${version#v}
47+
rc=${GITHUB_REF_NAME#*-rc}
48+
echo "VERSION=${version}" >> ${GITHUB_ENV}
49+
echo "RC=${rc}" >> ${GITHUB_ENV}
50+
- name: Prepare for branch
51+
if: github.ref_type == 'branch'
52+
run: |
53+
version=$(grep -o '^ <version>.*</version>' "pom.xml" |
54+
sed \
55+
-e 's,^ <version>,,' \
56+
-e 's,</version>$,,')
57+
rc=$(date +%Y%m%d)
58+
echo "VERSION=${version}" >> ${GITHUB_ENV}
59+
echo "RC=${rc}" >> ${GITHUB_ENV}
60+
- name: Archive
61+
run: |
62+
id="apache-arrow-java-${VERSION}"
63+
tar_gz="${id}.tar.gz"
64+
echo "TAR_GZ=${tar_gz}" >> ${GITHUB_ENV}
65+
git archive HEAD --prefix "${id}/" --output "${tar_gz}"
66+
sha256sum "${tar_gz}" > "${tar_gz}.sha256"
67+
sha512sum "${tar_gz}" > "${tar_gz}.sha512"
68+
- name: Audit
69+
run: |
70+
dev/release/run_rat.sh "${TAR_GZ}"
71+
- uses: actions/upload-artifact@6f51ac03b9356f520e9adb1b1b7802705f340c2b # v4.5.0
72+
with:
73+
name: archive
74+
path: |
75+
apache-arrow-java-*
76+
verify:
77+
name: Verify
78+
needs:
79+
- archive
80+
runs-on: ${{ matrix.os }}
81+
strategy:
82+
fail-fast: false
83+
matrix:
84+
os:
85+
- macos-latest
86+
- ubuntu-latest
87+
steps:
88+
- name: Checkout
89+
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
90+
with:
91+
submodules: recursive
92+
- uses: actions/download-artifact@fa0a91b85d4f404e444e00e005971372dc801d16 # v4.1.8
93+
with:
94+
name: archive
95+
- name: Verify
96+
run: |
97+
tar_gz=$(echo apache-arrow-java-*.tar.gz)
98+
version=${tar_gz#apache-arrow-java-}
99+
version=${version%.tar.gz}
100+
# rc isn't used with VERIFY_DOWNLOAD=0
101+
if [ "${GITHUB_REF_TYPE}" = "tag" ]; then
102+
rc="${GITHUB_REF_NAME#*-rc}"
103+
else
104+
rc=$(date +%Y%m%d)
105+
fi
106+
VERIFY_DEFAULT=0 \
107+
VERIFY_SOURCE=1 \
108+
dev/release/verify_rc.sh "${version}" "${rc}"
109+
env:
110+
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
111+
upload:
112+
name: Upload
113+
if: github.ref_type == 'tag'
114+
needs:
115+
- verify
116+
runs-on: ubuntu-latest
117+
permissions:
118+
contents: write
119+
steps:
120+
- name: Checkout
121+
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
122+
with:
123+
submodules: recursive
124+
- uses: actions/download-artifact@fa0a91b85d4f404e444e00e005971372dc801d16 # v4.1.8
125+
with:
126+
name: archive
127+
- name: Upload
128+
run: |
129+
# TODO: Add support for release note
130+
version=${GITHUB_REF_NAME%-rc*}
131+
version=${version#v}
132+
rc=${GITHUB_REF_NAME#*-rc}
133+
gh release create ${GITHUB_REF_NAME} \
134+
--notes "TODO" \
135+
--prerelease \
136+
--title "Apache Arrow Java ${version} RC${rc}" \
137+
--verify-tag \
138+
apache-arrow-java-*.tar.gz \
139+
apache-arrow-java-*.tar.gz.sha*
140+
env:
141+
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}

‎.github/workflows/release.yml‎

Lines changed: 54 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,54 @@
1+
# Licensed to the Apache Software Foundation (ASF) under one
2+
# or more contributor license agreements. See the NOTICE file
3+
# distributed with this work for additional information
4+
# regarding copyright ownership. The ASF licenses this file
5+
# to you under the Apache License, Version 2.0 (the
6+
# "License"); you may not use this file except in compliance
7+
# with the License. You may obtain a copy of the License at
8+
#
9+
# http://www.apache.org/licenses/LICENSE-2.0
10+
#
11+
# Unless required by applicable law or agreed to in writing,
12+
# software distributed under the License is distributed on an
13+
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
14+
# KIND, either express or implied. See the License for the
15+
# specific language governing permissions and limitations
16+
# under the License.
17+
18+
name: RC
19+
on:
20+
push:
21+
tags:
22+
- '*'
23+
- '!*-rc*'
24+
pull_request:
25+
concurrency:
26+
group: ${{ github.repository }}-${{ github.head_ref || github.sha }}-${{ github.workflow }}
27+
cancel-in-progress: true
28+
permissions:
29+
contents: write
30+
env:
31+
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
32+
jobs:
33+
publish:
34+
name: Publish
35+
runs-on: ubuntu-latest
36+
timeout-minutes: 5
37+
steps:
38+
- name: Checkout
39+
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
40+
- name: Download RC contents
41+
run: |
42+
set -x
43+
latest_rc_tag=$(gh release list --json tagName --jq '.[].tagName' | \
44+
grep -F "${GITHUB_REF_NAME}-rc" | \
45+
head -n1)
46+
gh release download ${latest_rc_tag} --dir dists
47+
- name: Create GitHub Release
48+
run: |
49+
version=${GITHUB_REF_NAME#v}
50+
gh release create ${GITHUB_REF_NAME} \
51+
--notes "TODO" \
52+
--title "Apache Arrow Java ${version}" \
53+
--verify-tag \
54+
dists/*

‎dev/release/README.md‎

Lines changed: 122 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,122 @@
1+
<!---
2+
Licensed to the Apache Software Foundation (ASF) under one
3+
or more contributor license agreements. See the NOTICE file
4+
distributed with this work for additional information
5+
regarding copyright ownership. The ASF licenses this file
6+
to you under the Apache License, Version 2.0 (the
7+
"License"); you may not use this file except in compliance
8+
with the License. You may obtain a copy of the License at
9+
10+
http://www.apache.org/licenses/LICENSE-2.0
11+
12+
Unless required by applicable law or agreed to in writing,
13+
software distributed under the License is distributed on an
14+
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
15+
KIND, either express or implied. See the License for the
16+
specific language governing permissions and limitations
17+
under the License.
18+
-->
19+
20+
# Release
21+
22+
## Overview
23+
24+
1. Test the revision to be released
25+
2. Bump version by `dev/release/bump_version.sh X.Y.Z`
26+
3. Prepare RC and vote (detailed later)
27+
4. Publish (detailed later)
28+
5. Bump version by `dev/release/bump_version.sh X.Y.Z-SNAPSHOT`
29+
30+
### Prepare RC and vote
31+
32+
Run `dev/release/release_rc.sh` on a working copy of
33+
`git@github.com:apache/arrow-java` not your fork:
34+
35+
```console
36+
$ git clone git@github.com:apache/arrow-java.git
37+
$ dev/release/release_rc.sh ${RC}
38+
(Send a vote email to dev@arrow.apache.org.
39+
You can use a draft shown by release_rc.sh for the email.)
40+
```
41+
42+
Here is an example to release RC1:
43+
44+
```console
45+
$ GH_TOKEN=${YOUR_GITHUB_TOKEN} dev/release/release_rc.sh 1
46+
```
47+
48+
The argument of `release_rc.sh` is the RC number. If RC1 has a
49+
problem, we'll increment the RC number such as RC2, RC3 and so on.
50+
51+
Requirements to run `release_rc.sh`:
52+
53+
* You must be an Apache Arrow committer or PMC member
54+
* You must prepare your PGP key for signing
55+
56+
If you don't have a PGP key,
57+
https://infra.apache.org/release-signing.html#generate may be helpful.
58+
59+
Your PGP key must be registered to the followings:
60+
61+
* https://dist.apache.org/repos/dist/dev/arrow/KEYS
62+
* https://dist.apache.org/repos/dist/release/arrow/KEYS
63+
64+
See the header comment of them how to add a PGP key.
65+
66+
Apache arrow committers can update them by Subversion client with
67+
their ASF account. e.g.:
68+
69+
```console
70+
$ svn co https://dist.apache.org/repos/dist/dev/arrow
71+
$ cd arrow
72+
$ editor KEYS
73+
$ svn ci KEYS
74+
```
75+
76+
### Publish
77+
78+
We need to do the followings to publish a new release:
79+
80+
* Publish to apache.org
81+
82+
Run `dev/release/release.sh` on a working copy of
83+
`git@github.com:apache/arrow-java` not your fork to publish to
84+
apache.org:
85+
86+
```console
87+
$ GH_TOKEN=${YOUR_GITHUB_TOKEN} dev/release/release.sh ${VERSION} ${RC}
88+
```
89+
90+
Here is an example to release 19.0.0 RC1:
91+
92+
```console
93+
$ GH_TOKEN=${YOUR_GITHUB_TOKEN} dev/release/release.sh 19.0.0 1
94+
```
95+
96+
Add the release to ASF's report database via [Apache Committee Report
97+
Helper](https://reporter.apache.org/addrelease.html?arrow).
98+
99+
### Verify
100+
101+
We have a script to verify a RC.
102+
103+
You must install the following commands to use the script:
104+
105+
* `curl`
106+
* `gpg`
107+
* `shasum` or `sha256sum`/`sha512sum`
108+
* `tar`
109+
110+
To verify a RC, run the following command line:
111+
112+
```console
113+
$ dev/release/verify_rc.sh ${VERSION} ${RC}
114+
```
115+
116+
Here is an example to verify the release 19.0.0 RC1:
117+
118+
```console
119+
$ dev/release/verify_rc.sh 19.0.0 1
120+
```
121+
122+
If the verification is successful, the message `RC looks good!` is shown.

‎dev/release/bump_version.sh‎

Lines changed: 51 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,51 @@
1+
#!/usr/bin/env bash
2+
#
3+
# Licensed to the Apache Software Foundation (ASF) under one
4+
# or more contributor license agreements. See the NOTICE file
5+
# distributed with this work for additional information
6+
# regarding copyright ownership. The ASF licenses this file
7+
# to you under the Apache License, Version 2.0 (the
8+
# "License"); you may not use this file except in compliance
9+
# with the License. You may obtain a copy of the License at
10+
#
11+
# http://www.apache.org/licenses/LICENSE-2.0
12+
#
13+
# Unless required by applicable law or agreed to in writing,
14+
# software distributed under the License is distributed on an
15+
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
16+
# KIND, either express or implied. See the License for the
17+
# specific language governing permissions and limitations
18+
# under the License.
19+
20+
set -eu
21+
22+
SOURCE_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
23+
SOURCE_TOP_DIR="$(cd "${SOURCE_DIR}/../../" && pwd)"
24+
25+
if [ "$#" -ne 1 ]; then
26+
echo "Usage: $0 <version>"
27+
echo " e.g.: $0 19.0.1"
28+
echo " e.g.: $0 20.0.0-SNAPSHOT"
29+
exit 1
30+
fi
31+
32+
version=$1
33+
34+
cd "${SOURCE_TOP_DIR}"
35+
36+
git switch -c bump-version-${version}
37+
mvn versions:set -DnewVersion=${version} -DprocessAllModules -DgenerateBackupPoms=false
38+
case "${version}" in
39+
*-SNAPSHOT)
40+
tag=main
41+
;;
42+
*)
43+
tag=v${version}
44+
;;
45+
esac
46+
mvn versions:set-scm-tag -DnewTag=${tag} -DgenerateBackupPoms=false -pl :arrow-java-root
47+
mvn versions:set-scm-tag -DnewTag=${tag} -DgenerateBackupPoms=false -pl :arrow-bom
48+
git add "pom.xml"
49+
git add "**/pom.xml"
50+
git commit -m "MINOR: Bump version to ${version}"
51+
gh pr create --fill

0 commit comments

Comments
 (0)