Skip to content

more/new docker images #1157

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
mxcl merged 1 commit into from
Apr 5, 2025
Merged

more/new docker images #1157

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 0 additions & 16 deletions .github/Dockerfile
View file
Open in desktop

This file was deleted.

25 changes: 19 additions & 6 deletions .github/workflows/cd.docker.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,10 +19,17 @@ permissions:

jobs:
build-and-push-image:
strategy:
matrix:
variety:
- archlinux
- busybox
- debian
- ubuntu
name: docker buildx pkgxdev/pkgx:${{ github.event.release.tag_name || github.event.inputs.tag }}
runs-on: ubuntu-latest
steps:
- uses: robinraju/release-downloader@v1.11
- uses: robinraju/release-downloader@v1.12
with:
releaseId: ${{ github.event.release.id }}
tag: ${{ github.event.inputs.tag }}
Expand All @@ -31,6 +38,10 @@ jobs:
- name: is_latest?
id: latest
run: |
if [ ${{ matrix.variety }} != debian ]; then
echo "value=false" >> "${GITHUB_OUTPUT}"
return
fi
TAG_NAME="${{ github.event.release.tag_name || github.event.inputs.tag }}"
LATEST_RELEASE=$(gh api repos/${{ github.repository }}/releases/latest --jq '.tag_name')
if [ "$TAG_NAME" == "$LATEST_RELEASE" ]; then
Expand Down Expand Up @@ -74,10 +85,11 @@ jobs:
flavor: |
latest=${{ steps.latest.outputs.value }}
tags: |
type=sha
type=semver,pattern=v{{version}},value=${{ github.event.release.tag_name || github.event.inputs.tag }}
type=semver,pattern=v{{major}}.{{minor}},value=${{ github.event.release.tag_name || github.event.inputs.tag }}
type=semver,pattern=v{{major}},value=${{ github.event.release.tag_name || github.event.inputs.tag }}
type=semver,pattern=v{{version}},value=${{ github.event.release.tag_name || github.event.inputs.tag }},enable=${{ matrix.variety == 'debian' }}
type=semver,pattern=v{{major}}.{{minor}},value=${{ github.event.release.tag_name || github.event.inputs.tag }},enable=${{ matrix.variety == 'debian' }}
type=semver,pattern=v{{major}},value=${{ github.event.release.tag_name || github.event.inputs.tag }},enable=${{ matrix.variety == 'debian' }}
type=raw,value=${{ matrix.variety }}
if: ${{ matrix.variety == 'debian' }}

- uses: docker/setup-qemu-action@v3
- uses: docker/setup-buildx-action@v3
Expand All @@ -90,6 +102,7 @@ jobs:
docker buildx build \
--push \
$tags \
--tag pkgxdev/pkgx:${{matrix.variety}} \
--platform linux/amd64,linux/arm64 \
--file ./src/.github/Dockerfile \
--file ./src/.github/Dockerfile.${{ matrix.variety }} \
.
44 changes: 19 additions & 25 deletions .github/workflows/ci.docker.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ name: ci·docker
on:
pull_request:
paths:
- .github/Dockerfile
- docker/Dockerfile.*
- .github/workflows/ci.docker.yml

concurrency:
Expand All @@ -15,51 +15,45 @@ permissions:
packages: write

jobs:
docker-build:
go:
runs-on: ubuntu-latest
container: debian:buster-slim
strategy:
matrix:
variety:
- archlinux
- busybox
- debian
- ubuntu
steps:
- uses: actions/checkout@v4

- run: apt-get update && apt-get install -y curl gcc perl make

- uses: dtolnay/rust-toolchain@stable
- run: cargo build --release

- uses: actions/upload-artifact@v4
with:
name: products
path: ./target/release/pkgx

docker-test:
runs-on: ubuntu-latest
needs: docker-build
steps:
- uses: actions/checkout@v4

- uses: actions/download-artifact@v4
- uses: robinraju/release-downloader@v1.12
with:
name: products
fileName: pkgx-*+linux+x86-64.tar.gz
latest: true

- run: |
mkdir products
mv ./pkgx products/$(uname -m)
tar xzf pkgx*.tar.gz
mv pkgx products/$(uname -m)
curl https://pkgxdev.github.io/pkgm/pkgm.ts -o products/pkgm

- run:
docker build
--tag pkgxdev/pkgx
--file .github/Dockerfile
--tag pkgxdev/${{ matrix.variety }}
--file docker/Dockerfile.${{ matrix.variety }}
.

- run: |
cat <<EoD> Dockerfile
FROM pkgxdev/pkgx
FROM pkgxdev/${{ matrix.variety }}
RUN pkgx --version
RUN if git --version; then exit 1; fi
RUN pkgx git --version
RUN pkgm --version
RUN pkgm install git
RUN if ! git --version; then exit 2; fi
RUN pkgx node --eval "console.log(1)"
EoD

docker build --file Dockerfile .
10 changes: 10 additions & 0 deletions docker/Dockerfile.archlinux
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
FROM archlinux AS stage0
COPY ./products/* /pkgx/
RUN install -m 755 /pkgx/$(uname -m) /usr/local/bin/pkgx
RUN install -m 755 /pkgx/pkgm /usr/local/bin/pkgm

FROM archlinux AS stage1
COPY --from=stage0 /usr/local/bin/pkgx /usr/local/bin/pkgm /usr/local/bin/

CMD ["/bin/bash"]
ENTRYPOINT ["/usr/local/bin/pkgx"]
19 changes: 19 additions & 0 deletions docker/Dockerfile.busybox
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
FROM debian:buster-slim AS stage0
COPY ./products/* /pkgx/
RUN install -m 755 /pkgx/$(uname -m) /usr/local/bin/pkgx
RUN install -m 755 /pkgx/pkgm /usr/local/bin/pkgm

# FIXME a newer glibc would be nice, but the libpthread.so we copy in then fails
FROM busybox:1.31-glibc AS stage1
COPY --from=stage0 /usr/local/bin/pkgx /usr/local/bin/pkgm /usr/local/bin/

# pkgx packages also need libgcc
COPY --from=stage0 /lib/x86_64-linux-gnu/libgcc_s.so.1 /lib/libgcc_s.so.1
# these are part of glibc but for some reason this image doesn’t have them
COPY --from=stage0 /lib/x86_64-linux-gnu/librt-2.28.so /lib/librt.so.1
COPY --from=stage0 /lib/x86_64-linux-gnu/libdl-2.28.so /lib/libdl.so.2
# we need env for `pkgm`
COPY --from=stage0 /usr/bin/env /usr/bin/env

CMD ["/bin/bash"]
ENTRYPOINT ["/usr/local/bin/pkgx"]
10 changes: 10 additions & 0 deletions docker/Dockerfile.debian
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
FROM debian:stable-slim AS stage0
COPY ./products/* /pkgx/
RUN install -m 755 /pkgx/$(uname -m) /usr/local/bin/pkgx
RUN install -m 755 /pkgx/pkgm /usr/local/bin/pkgm

FROM debian:stable-slim AS stage1
COPY --from=stage0 /usr/local/bin/pkgx /usr/local/bin/pkgm /usr/local/bin/

CMD ["/bin/bash"]
ENTRYPOINT ["/usr/local/bin/pkgx"]
10 changes: 10 additions & 0 deletions docker/Dockerfile.ubuntu
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
FROM ubuntu AS stage0
COPY ./products/* /pkgx/
RUN install -m 755 /pkgx/$(uname -m) /usr/local/bin/pkgx
RUN install -m 755 /pkgx/pkgm /usr/local/bin/pkgm

FROM ubuntu AS stage1
Copy link
Contributor

@felipecrs felipecrs Mar 26, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd also recommend you pin the major version of the base image.

At least something like ubuntu:noble and debian:bookwork-slim to give you some control over it.

Copy link
Member Author

@mxcl mxcl Mar 26, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm less keen to do this (at least for now, due to maintenance reasons). Users who want pinning should make their own images. Installing pkgx is trivial.

Copy link
Contributor

@felipecrs felipecrs Mar 26, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

New Ubuntu and Debian major versions happens on every 2 years... it shouldn't be a big problem lol.

Copy link
Contributor

@jhheider jhheider Mar 26, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ubuntu LTS is 2 years. Ubuntu releases a new version every April and October, iirc.

Copy link
Contributor

@felipecrs felipecrs Mar 26, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, but I meant how frequent the ubuntu:latest and debian:latest tag changes, which is Ubuntu LTS and Debian Stable based.

Copy link
Member Author

@mxcl mxcl Apr 2, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Try building node with these images. It will fail.

Copy link
Contributor

@felipecrs felipecrs Apr 2, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see now.

This could be a problem if some user uses or extends from the pkgx docker image to do things like building their own C++ projects.

Maybe we can consider that use case a stretch. Also, doing a FROM pkgxdev/pkgx \n RUN apt-get update && apt-get install -y libstdc++ is probably not a big deal for these users.

But I agree the best would be if it "just works", without having to install extra dependencies.

I'm glad to learn that this is being addressed in the new manifests, which would mean the containers need not to ship things like libstdc++. Very exciting. :)

Copy link
Contributor

@felipecrs felipecrs Apr 2, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For now we are replacing the default image with debian which really is a breaking change

PS: the previous default image was already debian :P

Copy link
Member Author

@mxcl mxcl Apr 2, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the previous image was debian yes, but it preinstalled some apt packages which the new one does not.

I’d like to be able to ship pkgx in alpine or even just a container with a kernel. But we need glibc and probably some logic in pkgx itself so it doesn't install glibc if it doesn't need to since that is excessive.

Copy link
Contributor

@felipecrs felipecrs Apr 2, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I completely agree. It would be awesome (and it would make a lot of sense) to have a distroless image with just pkgx.

COPY --from=stage0 /usr/local/bin/pkgx /usr/local/bin/pkgm /usr/local/bin/

CMD ["/bin/bash"]
ENTRYPOINT ["/usr/local/bin/pkgx"]