Skip to content

Create an official chisel container image to improve UX for creating/extending Chiseled base container images #203

Open

Description

@lbussell

Thanks to the addition of the chisel manifest, we can now use chisel to perform multiple subsequent installations of slices on the same rootfs. This is great for creating and extending Chiseled base container images.

However, when creating or extending a Chiseled container image, the UX is not that great since you need to acquire chisel before you can create a new rootfs.

Ideally, users wouldn't have to know about chisel's dependencies or need to worry about golang versions, etc.
Providing an official Chisel container image would simplify all of this.

Current UX for acquiring Chisel

Binary

Extending a container image
FROM amd64/ubuntu:noble AS chisel
ARG BASE_IMAGE="ubuntu/dotnet-runtime"
ARG CHISEL_VERSION="v1.X.X"

# Install chisel's dependencies
RUN apt-get update \
    && apt-get install -y --no-install-recommends \
        file \
    && rm -rf /var/lib/apt/lists/*

# Install chisel
RUN chisel_url=https://github.com/canonical/chisel/releases/download/v${CHISEL_VERSION}/chisel_v${CHISEL_VERSION}_linux_amd64.tar.gz \
    && curl -fSLOJ ${chisel_url} \
    && curl -fSL ${chisel_url}.sha384 | sha384sum -c - \
    && tar -xzf chisel_v${CHISEL_VERSION}_linux_amd64.tar.gz -C /usr/bin/ chisel

# Install new slices
COPY --from=$BASE_IMAGE / /rootfs/
RUN chisel cut --release ubuntu-24.04 --root /rootfs/ \
            libicu74_libs \
            tzdata-legacy_zoneinfo \
            tzdata_zoneinfo


FROM scratch AS final
COPY --link --from=chisel /rootfs /

Golang

Extending a container image
FROM amd64/golang:1.2X AS chisel
ARG BASE_IMAGE="ubuntu/dotnet-runtime"
ARG CHISEL_VERSION="v1.X.X"

# Install chisel's dependencies
RUN apt-get update \
    && apt-get install -y --no-install-recommends \
        file \
    && rm -rf /var/lib/apt/lists/*

# Install chisel tool
RUN go install github.com/canonical/chisel/cmd/chisel@v${CHISEL_VERSION}

# Install new slices
COPY --from=$BASE_IMAGE / /rootfs/
RUN chisel cut --release ubuntu-24.04 --root /rootfs/ \
            libicu74_libs \
            tzdata-legacy_zoneinfo \
            tzdata_zoneinfo


FROM scratch AS final
COPY --link --from=chisel /rootfs /

Desired UX

Creating a base image

FROM amd64/ubuntu/chisel:v1.X.X AS chisel

# Install slices
RUN mkdir -p /rootfs \
    && chisel cut --release ubuntu-24.04 --root /rootfs \
        base-files_base \
        base-files_chisel \
        base-files_release-info \
        ...


FROM scratch AS final
COPY --link --from=chisel /rootfs /

Extending a base image

FROM amd64/ubuntu/chisel:v1.X.X AS chisel
ARG BASE_IMAGE="ubuntu/dotnet-runtime"

# Install new slices
COPY --from=$BASE_IMAGE / /rootfs/
RUN chisel cut --release ubuntu-24.04 --root /rootfs/ \
            libicu74_libs \
            tzdata-legacy_zoneinfo \
            tzdata_zoneinfo


FROM scratch AS final
COPY --link --from=chisel /rootfs /

Activity

  1. rebornplusplus commented on Feb 5, 2025

    @rebornplusplus

    This is a good point and something I have faced as well! I remember starting to work on this PR: rockcrafters/chisel-rock#1. But this work hasn't been continued for some time. I should probably find some time and pick that up again.

    One question for you though, when you envision a chisel image, do you see it containing just chisel with libc6 or the whole ubuntu base in the back?

  2. lbussell commented on Feb 5, 2025

    @lbussell
    Author

    One question for you though, when you envision a chisel image, do you see it containing just chisel with libc6 or the whole ubuntu base in the back?

    I think it would be natural for it to be as minimal as possible - just chisel and potentially chisel-wrapper. I don't currently see a need for any additional tools like a shell, etc. Can you think of any reasons for using a full Ubuntu base image?

  3. rebornplusplus commented on Feb 6, 2025

    @rebornplusplus

    Huh, good point about the chisel-wrapper. We might need a shell if we include the chisel-wrapper since it's a shell script, but that's about it.

    Can you think of any reasons for using a full Ubuntu base image?

    Unless we want to run some postinst binary that we cannot yet do with Chisel's mutation scripts, I don't think we will need the Ubuntu base. Even if we need to execute binaries, adding another stage in the build process solves the issue.

    In short, no. I agree with you. It should be as minimal as possible.

  4. moved this to Tracking in .NET Dockeron Feb 10, 2025
  5. polarathene commented on Apr 26, 2025

    @polarathene

    Ideally, users wouldn't have to know about chisel's dependencies

    FROM alpine AS get-chisel
    ARG CHISEL_VERSION=1.1.0
    ARG TARGETARCH
    ARG CHISEL_RELEASE="https://github.com/canonical/chisel/releases/download/v${CHISEL_VERSION}/chisel_v${CHISEL_VERSION}_linux_${TARGETARCH}.tar.gz"
    # NOTE: `--no-same-owner` used as `chisel` release has ownership of `1001:128`
    RUN wget -qO - "${CHISEL_RELEASE}" | tar -xz --no-same-owner -C /usr/local/bin chisel

    Huh, good point about the chisel-wrapper. We might need a shell if we include the chisel-wrapper since it's a shell script, but that's about it.

    That would specifically require bash_bins, but also note that:

    • With FROM scratch a typical RUN instruction would fail as the default SHELL is set to /bin/sh -c. It seems only the dash_bins slice provides a /usr/bin/sh symlink to /usr/bin/dash
    • You also need base-files_base for the /bin -> /usr/bin symlink (chisel-wrapper uses #! /bin/bash not #! /usr/bin/bash).

    Dockerfile

    Here's how you'd go about creating the Chisel image discussed thus far:

    FROM alpine AS get-chisel
    ARG CHISEL_VERSION=1.1.0
    ARG TARGETARCH
    ARG CHISEL_RELEASE="https://github.com/canonical/chisel/releases/download/v${CHISEL_VERSION}/chisel_v${CHISEL_VERSION}_linux_${TARGETARCH}.tar.gz"
    # NOTE: `--no-same-owner` used as `chisel` release has ownership of `1001:128`
    RUN wget -qO - "${CHISEL_RELEASE}" | tar -xz --no-same-owner -C /usr/local/bin chisel
    
    
    FROM scratch AS chisel
    # Copy over chisel binary and it's external dependency to `ca-certificates.crt`:
    COPY --link --from=get-chisel /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/ca-certificates.crt
    COPY --link --from=get-chisel /usr/local/bin/chisel /usr/local/bin/chisel
    # Provision bash and make it the default shell for `RUN`:
    # NOTE:
    # - `base-files_base` required for `/bin` -> `/usr/bin` symlink for when scripts use shebang `#! /bin/bash`
    # - The cache mount prevents writing 107MB of cache into image for these two slices.
    RUN --mount=type=cache,target=/.cache/chisel,id="chisel-cache" \
        [ "/usr/local/bin/chisel", "cut", "--release", "ubuntu-24.04", "--root", "/", "base-files_base", "bash_bins" ]
    # Dockerfile that uses this image will inherit this `SHELL` instruction as their default shell for `RUN` instructions:
    SHELL [ "/bin/bash", "-c" ]
    # Likewise set a default command (Alpine uses `CMD ["/bin/sh"]` and Fedora `CMD ["/bin/bash"]`):
    # NOTE: Not using JSON syntax for CMD will preprend string to SHELL (aka CMD becomes `/bin/bash -c /bin/bash`)
    CMD [ "/bin/bash" ]

    NOTES:

    • CA bundle mounting or COPY could be avoided if this gets implemented.
    • If you build that and attempt to shell into it, they'll be some minor UX issues due to lack of /etc/{passwd,group} it seems. Adding the base-passwd_data slice should remedy that AFAIK but it seems that is not presently compatible with --root /.
    • I'm not familiar with chisel-wrapper setup / usage, so that's been omitted from the Dockerfile.
    • The default SHELL instruction could be omitted if installing the dash_bins slice for /bin/sh to resolve to the /usr/bin/sh -> /usr/bin/dash symlink, or alternatively creating that symlink to /usr/bin/bash? 🤷‍♂ (which is what Fedora does)
    # Build and inspect image:
    docker build --tag localhost/chisel .
    docker run --rm -it -v /var/run/docker.sock:/var/run/docker.sock ghcr.io/wagoodman/dive:latest localhost/chisel

    Approx 15.6MB in weight:

    • 220KB for ca-certificates.crt
    • 8.7MB for chisel
    • 6.6MB for the slices installed

    Usage

    If the 15.6MB excess isn't a concern you could just extend from the image (may be less wasted in a future release when you'd want the two slices):

    --root / (1 stage) - Extend the official chisel image

    FROM chisel
    RUN --mount=type=cache,target=/.cache/chisel,id="chisel-cache" \
        chisel cut --release ubuntu-24.04 --root / coreutils_bins

    --root / (1 stage) - Source the chisel binary from official image

    • This will additionally need ca-certificates.crt (unless this fallback gets implemented).
    • Has the benefit of being agnostic to creating/extending (just change the FROM image).

    Extending an image may be simpler when it already provides ca-certificates.crt and a shell 👍

    FROM scratch
    # Nushell is a self-contained binary like chisel, but provides disposable shell functionality
    SHELL ["/nu", "-c"]
    RUN --mount=type=cache,target=/.cache/chisel,id="chisel-cache" \
        --mount=type=bind,from=chisel,source=/usr/local/bin/chisel,target=/chisel \
        --mount=type=bind,from=ghcr.io/nushell/nushell,source=/usr/bin/nu,target=/nu \
        /chisel cut --release ubuntu-24.04 --root / coreutils_bins

    Without a shell you can use RUN with JSON syntax (this is not compatible with ARG/ENV instruction interpolation):

    # As scratch has no shell, use RUN with the "exec" form:
    # https://docs.docker.com/reference/dockerfile/#exec-form
    FROM scratch
    RUN --mount=type=cache,target=/.cache/chisel,id="chisel-cache" \
        --mount=type=bind,from=chisel,source=/usr/local/bin/chisel,target=/chisel \
        [ "/chisel", "cut", "--release", "ubuntu-24.04", "--root", "/", "coreutils_bins" ]

    --root /root-fs + COPY (2 stage)

    You could also still use the /root-fs + COPY dance via a separate stage like @lbussell showed in his original proposal for creating/extending images via an official chisel image 👍

    FROM chisel AS root-fs
    WORKDIR /root-fs
    #COPY --from=${BASE_IMAGE} / /root-fs
    RUN --mount=type=cache,target=/.cache/chisel,id="chisel-cache" \
        chisel cut --release ubuntu-24.04 --root /root-fs coreutils_bins
    
    FROM scratch
    COPY --link --from=root-fs /root-fs /

    Optimizing away the get-chisel stage

    The first stage could instead be skipped in favor of a local COPY, but this would usually be discouraged.

    • Instead you'd have a builder stage, then someone clones the repo so that the Dockerfile has the source to build the Go project in this stage, containerizing the build environment into a stage is ideal.
    • Given the type of project, local builds of chisel from source may be less likely? Preferring a local COPY sourcing a build from prior to building the Dockerfile may be simpler to maintain? 🤷‍♂ (personally I prefer the build environment to be self-contained within the Dockerfile, as a bonus it simplifies troubleshooting / testing with users that aren't familiar with the language tooling / setup)

    One other option is to use the ADD instruction. Unfortunately that wouldn't work with remote .tar.gz assets. The archives are bundling LICENSE + README.md with the chisel binary. If it was just chisel without .tar.gz, you could ADD the binary from GH releases URL directly similar to COPY, there's a separate --checksum arg for this instruction if you wanted to verify integrity. This would likewise remove the need for the separate get-chisel stage.

    If using ADD or COPY alternatives, you may want to include COPY --chown=0:0 to have a consistent and expected ownership for chisel. Presently the CI releases are archived with 1001:128 (CI runner user/group?)

  6. nilsdebruin commented on May 9, 2025

    @nilsdebruin
    Contributor

    For my projects, I have created a public docker container which can be found here:

    https://hub.docker.com/r/wagtaildigital/chisel

    I use it like this:

    FROM wagtaildigital/chisel:v1.1.0 AS chisel
    RUN mkdir -p /rootfs
    RUN chisel cut --release ubuntu-24.04 --root /rootfs --arch amd64 \
        base-files_base \
        ca-certificates_data \
        libc6_libs \
        python3-minimal_bins \
        python3-minimal_libs
    
    # Second stage: Create the final image from the minimal filesystem
    FROM scratch
    COPY --from=chisel /rootfs /
    COPY ./app /app
    WORKDIR /app
    CMD ["python3", "app.py"]
    

    Happy to share or contribute to an official canonical based image if needed!

  7. polarathene commented on May 11, 2025

    @polarathene

    This is just feedback for your published image.

    • Personally, the Dockerfile should be public, especially for unofficial images that appear to just COPY a binary from an unknown source.
    • Your image tagging seems a bit excessive.

    For my projects, I have created a public docker container which can be found here:

    https://hub.docker.com/r/wagtaildigital/chisel

    Your DockerHub page links to here for the source, but that is a 404, presumably it's a private repo?

    Is there a reason APP_REPO defaults to your chisel fork? (I assume it's for the APP_BRANCH which would be redundant once this PR is merged?)

    Defaults should be stable/predictable to upstream, those build ARG defaults you have currently make it a little less convenient to use without providing explicit overrides to prefer upstream sources (assuming the Dockerfile would be public).

    Without seeing the Dockerfile, the DockerHub docs are a bit vague on what APP_VERSION is for since APP_REPO + APP_BRANCH imply source selection already 😅

    wagtaildigital/chisel:v1.1.0 seems to be a Dockerfile that just copies in an external build/download from somewhere?:

    Image


    Small tip, usually it's a more common convention for the semver tag to exclude the v prefix when it's an image tag. While with git tags it's more common to have the v prefix, and I think with Go projects even expected.

    I don't really get the purpose of a :v1.1.0-latest tag? I'm assuming you have a GHA managing that, but slightly misconfigured that it appended -latest?

    For comparison, you also have a latest-hardened tag (but just hardened would suffice? Assuming it's handled as a suffix via GHA, I can see how a suffix would be simpler to configure).

    You are publishing tags with timestamps and digests?? (v1.1.0-20250511-d13e667d6273531e92f641803b8a98f151b0bd0f-hardened)

    There should be no need for that. You can pin an image by it's digest, otherwise update the respective tags as needed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions