Skip to content

Commit 14dcd50

Browse files
kouraulcd
kou
and
raulcd
authored
GH-20: Add release scripts (#496)
Fixes GH-20. This is based on apache/arrow-go#122 . Workflow: Cut a RC: 1. Bump version by `dev/release/bump_version.sh 19.0.0` 2. Run `dev/release/release_rc.sh` 1. `dev/release/release_rc.sh` pushes `v${version}-rc${rc}` tag 2. `.github/workflows/rc.yml` creates `apache-arrow-java-${version}.tar.gz{,.sha256,.sha512}` 3. `.github/workflows/rc.yml` uploads `apache-arrow-java-${version}.tar.gz{,.sha256,.sha512}` to GitHub Releases 4. `dev/release/release_rc.sh` downloads `apache-arrow-java-${version}.tar.gz` from GitHub Releases 5. `dev/release/release_rc.sh` signs `apache-arrow-java-${version}.tar.gz` as `apache-arrow-go-${version}.tar.gz.asc` 6. `dev/release/release_rc.sh` uploads `apache-arrow-java-${version}.tar.gz.asc` to GitHub Releases 3. Start a vote (GitHub Actions instead of https://dist.apache.org/repos/dist/dev/arrow/ is used like ADBC.) Verify a RC: 1. Run `dev/release/verify_rc.sh` Release an approved RC: 1. Run `dev/release/release.sh` 1. `dev/release/release.sh` pushes `v${version}` tag that refers that same commit ID as `v${version}-rc${rc}` 2. `.github/workflows/release.yml` create a GitHub Releases for `v${version}` tag by copying the RC GitHub Releases 3. `dev/release/release.sh` downloads `apache-arrow-java-${version}.tar.gz{,.asc,.sha256,.sha512}` from GitHub Releases 4. `dev/release/release.sh` uploads `apache-arrow-java-${version}.tar.gz{,.asc,.sha256,.sha512}` to https://dist.apache.org/repos/dist/release/arrow 5. `dev/release/release.sh` removes old releases from https://dist.apache.org/repos/dist/release/arrow 2. Add this release to ASF's report database: https://reporter.apache.org/addrelease.html?arrow Follow-up tasks: * Add support for building binary packages (.jar) * Add support for releasing binary packages (.jar) * Add support for running JNI test in the verification script * Add support for running integration test in the verification script * Add support for verifying binary packages (.jar) in the verification script --------- Co-authored-by: Raúl Cumplido <raulcumplido@gmail.com>
1 parent de80cef commit 14dcd50

File tree

7 files changed

+837
-0
lines changed

7 files changed

+837
-0
lines changed

‎.github/workflows/rc.yml‎

Lines changed: 141 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,141 @@
1+
# Licensed to the Apache Software Foundation (ASF) under one
2+
# or more contributor license agreements. See the NOTICE file
3+
# distributed with this work for additional information
4+
# regarding copyright ownership. The ASF licenses this file
5+
# to you under the Apache License, Version 2.0 (the
6+
# "License"); you may not use this file except in compliance
7+
# with the License. You may obtain a copy of the License at
8+
#
9+
# http://www.apache.org/licenses/LICENSE-2.0
10+
#
11+
# Unless required by applicable law or agreed to in writing,
12+
# software distributed under the License is distributed on an
13+
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
14+
# KIND, either express or implied. See the License for the
15+
# specific language governing permissions and limitations
16+
# under the License.
17+
18+
name: RC
19+
on:
20+
push:
21+
branches:
22+
- "**"
23+
- "!dependabot/**"
24+
tags:
25+
- "*-rc*"
26+
pull_request:
27+
schedule:
28+
- cron: "0 0 * * *"
29+
concurrency:
30+
group: ${{ github.repository }}-${{ github.head_ref || github.sha }}-${{ github.workflow }}
31+
cancel-in-progress: true
32+
permissions:
33+
contents: read
34+
jobs:
35+
archive:
36+
name: Archive
37+
runs-on: ubuntu-latest
38+
timeout-minutes: 5
39+
steps:
40+
- name: Checkout
41+
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
42+
with:
43+
submodules: recursive
44+
- name: Prepare for tag
45+
if: github.ref_type == 'tag'
46+
run: |
47+
version=${GITHUB_REF_NAME%-rc*}
48+
version=${version#v}
49+
rc=${GITHUB_REF_NAME#*-rc}
50+
echo "VERSION=${version}" >> ${GITHUB_ENV}
51+
echo "RC=${rc}" >> ${GITHUB_ENV}
52+
- name: Prepare for branch
53+
if: github.ref_type == 'branch'
54+
run: |
55+
version=$(grep -o '^ <version>.*</version>' "pom.xml" |
56+
sed \
57+
-e 's,^ <version>,,' \
58+
-e 's,</version>$,,')
59+
rc=$(date +%Y%m%d)
60+
echo "VERSION=${version}" >> ${GITHUB_ENV}
61+
echo "RC=${rc}" >> ${GITHUB_ENV}
62+
- name: Archive
63+
run: |
64+
id="apache-arrow-java-${VERSION}"
65+
tar_gz="${id}.tar.gz"
66+
echo "TAR_GZ=${tar_gz}" >> ${GITHUB_ENV}
67+
git archive HEAD --prefix "${id}/" --output "${tar_gz}"
68+
sha256sum "${tar_gz}" > "${tar_gz}.sha256"
69+
sha512sum "${tar_gz}" > "${tar_gz}.sha512"
70+
- name: Audit
71+
run: |
72+
dev/release/run_rat.sh "${TAR_GZ}"
73+
- uses: actions/upload-artifact@6f51ac03b9356f520e9adb1b1b7802705f340c2b # v4.5.0
74+
with:
75+
name: archive
76+
path: |
77+
apache-arrow-java-*
78+
verify:
79+
name: Verify
80+
needs:
81+
- archive
82+
runs-on: ${{ matrix.os }}
83+
strategy:
84+
fail-fast: false
85+
matrix:
86+
os:
87+
- macos-latest
88+
- ubuntu-latest
89+
steps:
90+
- name: Checkout
91+
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
92+
with:
93+
submodules: recursive
94+
- uses: actions/download-artifact@fa0a91b85d4f404e444e00e005971372dc801d16 # v4.1.8
95+
with:
96+
name: archive
97+
- name: Verify
98+
run: |
99+
tar_gz=$(echo apache-arrow-java-*.tar.gz)
100+
version=${tar_gz#apache-arrow-java-}
101+
version=${version%.tar.gz}
102+
# rc isn't used with VERIFY_DOWNLOAD=0
103+
if [ "${GITHUB_REF_TYPE}" = "tag" ]; then
104+
rc="${GITHUB_REF_NAME#*-rc}"
105+
else
106+
rc=$(date +%Y%m%d)
107+
fi
108+
VERIFY_DEFAULT=0 \
109+
VERIFY_SOURCE=1 \
110+
dev/release/verify_rc.sh "${version}" "${rc}"
111+
upload:
112+
name: Upload
113+
if: github.ref_type == 'tag'
114+
needs:
115+
- verify
116+
runs-on: ubuntu-latest
117+
permissions:
118+
contents: write
119+
steps:
120+
- name: Checkout
121+
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
122+
with:
123+
submodules: recursive
124+
- uses: actions/download-artifact@fa0a91b85d4f404e444e00e005971372dc801d16 # v4.1.8
125+
with:
126+
name: archive
127+
- name: Upload
128+
run: |
129+
# GH-499: How to create release notes?
130+
version=${GITHUB_REF_NAME%-rc*}
131+
version=${version#v}
132+
rc=${GITHUB_REF_NAME#*-rc}
133+
gh release create ${GITHUB_REF_NAME} \
134+
--generate-notes \
135+
--prerelease \
136+
--title "Apache Arrow Java ${version} RC${rc}" \
137+
--verify-tag \
138+
apache-arrow-java-*.tar.gz \
139+
apache-arrow-java-*.tar.gz.sha*
140+
env:
141+
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}

‎.github/workflows/release.yml‎

Lines changed: 54 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,54 @@
1+
# Licensed to the Apache Software Foundation (ASF) under one
2+
# or more contributor license agreements. See the NOTICE file
3+
# distributed with this work for additional information
4+
# regarding copyright ownership. The ASF licenses this file
5+
# to you under the Apache License, Version 2.0 (the
6+
# "License"); you may not use this file except in compliance
7+
# with the License. You may obtain a copy of the License at
8+
#
9+
# http://www.apache.org/licenses/LICENSE-2.0
10+
#
11+
# Unless required by applicable law or agreed to in writing,
12+
# software distributed under the License is distributed on an
13+
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
14+
# KIND, either express or implied. See the License for the
15+
# specific language governing permissions and limitations
16+
# under the License.
17+
18+
name: Release
19+
on:
20+
push:
21+
tags:
22+
- "*"
23+
- "!*-rc*"
24+
concurrency:
25+
group: ${{ github.repository }}-${{ github.head_ref || github.sha }}-${{ github.workflow }}
26+
cancel-in-progress: true
27+
permissions:
28+
contents: write
29+
env:
30+
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
31+
jobs:
32+
publish:
33+
name: Publish
34+
runs-on: ubuntu-latest
35+
timeout-minutes: 5
36+
steps:
37+
- name: Checkout
38+
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
39+
- name: Download RC contents
40+
run: |
41+
set -x
42+
latest_rc_tag=$(gh release list --json tagName --jq '.[].tagName' | \
43+
grep -F "${GITHUB_REF_NAME}-rc" | \
44+
head -n1)
45+
gh release download ${latest_rc_tag} --dir dists
46+
- name: Create GitHub Release
47+
run: |
48+
# GH-499: How to create release notes?
49+
version=${GITHUB_REF_NAME#v}
50+
gh release create ${GITHUB_REF_NAME} \
51+
--generate-notes \
52+
--title "Apache Arrow Java ${version}" \
53+
--verify-tag \
54+
dists/*

‎dev/release/README.md‎

Lines changed: 174 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,174 @@
1+
<!---
2+
Licensed to the Apache Software Foundation (ASF) under one
3+
or more contributor license agreements. See the NOTICE file
4+
distributed with this work for additional information
5+
regarding copyright ownership. The ASF licenses this file
6+
to you under the Apache License, Version 2.0 (the
7+
"License"); you may not use this file except in compliance
8+
with the License. You may obtain a copy of the License at
9+
10+
http://www.apache.org/licenses/LICENSE-2.0
11+
12+
Unless required by applicable law or agreed to in writing,
13+
software distributed under the License is distributed on an
14+
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
15+
KIND, either express or implied. See the License for the
16+
specific language governing permissions and limitations
17+
under the License.
18+
-->
19+
20+
# Release
21+
22+
## Overview
23+
24+
1. Test the revision to be released
25+
2. Bump version for new release (detailed later)
26+
3. Prepare RC and vote (detailed later)
27+
4. Publish (detailed later)
28+
5. Bump version for new development (detailed later)
29+
30+
### Bump version for new release
31+
32+
Run `dev/release/bump_version.sh` on a working copy of your fork not
33+
`git@github.com:apache/arrow-java`:
34+
35+
```console
36+
$ git clone git@github.com:${YOUR_GITHUB_ACCOUNT}/arrow-java.git arrow-java.${YOUR_GITHUB_ACCOUNT}
37+
$ cd arrow-java.${YOUR_GITHUB_ACCOUNT}
38+
$ GH_TOKEN=${YOUR_GITHUB_TOKEN} dev/release/bump_version.sh ${NEW_VERSION}
39+
```
40+
41+
Here is an example to bump version to 19.0.0:
42+
43+
```
44+
$ GH_TOKEN=${YOUR_GITHUB_TOKEN} dev/release/bump_version.sh 19.0.0
45+
```
46+
47+
It creates a feature branch and adds a commit that bumps version. This
48+
opens a pull request from the feature branch by `gh pr create`. So you
49+
need `gh` command and GitHub personal access token.
50+
51+
See also:
52+
https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens
53+
54+
We need to merge the pull request before we cut a RC. If we try cut a
55+
RC without merging the pull request, the script to cut a RC is failed.
56+
57+
### Prepare RC and vote
58+
59+
Run `dev/release/release_rc.sh` on a working copy of
60+
`git@github.com:apache/arrow-java` not your fork:
61+
62+
```console
63+
$ git clone git@github.com:apache/arrow-java.git
64+
$ cd arrow-java
65+
$ GH_TOKEN=${YOUR_GITHUB_TOKEN} dev/release/release_rc.sh ${RC}
66+
(Send a vote email to dev@arrow.apache.org.
67+
You can use a draft shown by release_rc.sh for the email.)
68+
```
69+
70+
Here is an example to release RC1:
71+
72+
```console
73+
$ GH_TOKEN=${YOUR_GITHUB_TOKEN} dev/release/release_rc.sh 1
74+
```
75+
76+
The argument of `release_rc.sh` is the RC number. If RC1 has a
77+
problem, we'll increment the RC number such as RC2, RC3 and so on.
78+
79+
Requirements to run `release_rc.sh`:
80+
81+
* You must be an Apache Arrow committer or PMC member
82+
* You must prepare your PGP key for signing
83+
84+
If you don't have a PGP key,
85+
https://infra.apache.org/release-signing.html#generate may be helpful.
86+
87+
Your PGP key must be registered to the followings:
88+
89+
* https://dist.apache.org/repos/dist/dev/arrow/KEYS
90+
* https://dist.apache.org/repos/dist/release/arrow/KEYS
91+
92+
See the header comment of them how to add a PGP key.
93+
94+
Apache arrow committers can update them by Subversion client with
95+
their ASF account. e.g.:
96+
97+
```console
98+
$ svn co https://dist.apache.org/repos/dist/dev/arrow
99+
$ cd arrow
100+
$ head KEYS
101+
(This shows how to update KEYS)
102+
$ svn ci KEYS
103+
```
104+
105+
### Publish
106+
107+
We need to do the followings to publish a new release:
108+
109+
* Publish to apache.org
110+
111+
Run `dev/release/release.sh` on a working copy of
112+
`git@github.com:apache/arrow-java` not your fork to publish to
113+
apache.org:
114+
115+
```console
116+
$ GH_TOKEN=${YOUR_GITHUB_TOKEN} dev/release/release.sh ${VERSION} ${RC}
117+
```
118+
119+
Here is an example to release 19.0.0 RC1:
120+
121+
```console
122+
$ GH_TOKEN=${YOUR_GITHUB_TOKEN} dev/release/release.sh 19.0.0 1
123+
```
124+
125+
Add the release to ASF's report database via [Apache Committee Report
126+
Helper](https://reporter.apache.org/addrelease.html?arrow).
127+
128+
### Bump version for new development
129+
130+
We should bump version in the main branch for new development after we
131+
release a new version.
132+
133+
Run `dev/release/bump_version.sh` on a working copy of your fork not
134+
`git@github.com:apache/arrow-java`:
135+
136+
```console
137+
$ git clone git@github.com:${YOUR_GITHUB_ACCOUNT}/arrow-java.git arrow-java.${YOUR_GITHUB_ACCOUNT}
138+
$ cd arrow-java.${YOUR_GITHUB_ACCOUNT}
139+
$ GH_TOKEN=${YOUR_GITHUB_TOKEN} dev/release/bump_version.sh ${NEW_VERSION}-SNAPSHOT
140+
```
141+
142+
Here is an example to bump version to 19.0.1-SNAPSHOT:
143+
144+
```
145+
$ GH_TOKEN=${YOUR_GITHUB_TOKEN} dev/release/bump_version.sh 19.0.0-SNAPSHOT
146+
```
147+
148+
It creates a feature branch and adds a commit that bumps version. This
149+
opens a pull request from the feature branch by `gh pr create`.
150+
151+
## Verify
152+
153+
We have a script to verify a RC.
154+
155+
You must install the following commands to use the script:
156+
157+
* `curl`
158+
* `gpg`
159+
* `shasum` or `sha256sum`/`sha512sum`
160+
* `tar`
161+
162+
To verify a RC, run the following command line:
163+
164+
```console
165+
$ dev/release/verify_rc.sh ${VERSION} ${RC}
166+
```
167+
168+
Here is an example to verify the release 19.0.0 RC1:
169+
170+
```console
171+
$ dev/release/verify_rc.sh 19.0.0 1
172+
```
173+
174+
If the verification is successful, the message `RC looks good!` is shown.

0 commit comments

Comments
 (0)