Skip to content

Skip unreadable entries in image list - #2157

Merged
jglogan merged 2 commits into
apple:mainfrom
Hokila:fix/2149-image-list-unreadable-entry
Oct 2, 2026
Merged

jglogan merged 2 commits into
apple:mainfrom
Hokila:fix/2149-image-list-unreadable-entry

Conversation

@Hokila

@Hokila Hokila commented Aug 26, 2026 •

Copy link
Copy Markdown
Contributor

Type of Change

  • Bug fix
  • New feature
  • Breaking change
  • Documentation update

Motivation and Context

container image list currently fails completely when one image record references
an unreadable or missing content blob. Healthy images are not rendered, and
structured output such as JSON and YAML is unavailable.

This PR resolves each image independently:

  • healthy images remain visible;
  • unreadable images are reported as warnings;
  • warnings are written to stderr;
  • structured stdout remains valid;
  • the command exits non-zero after rendering a partial listing, so automation can detect it.

Fixes #2149.

Reproduction

Environment

  • macOS 26, arm64
  • one healthy image: docker.io/library/alpine:latest
  • one broken image record: example.invalid/broken:latest
  • the broken record contains a syntactically valid image descriptor, but its
    referenced root content blob is absent

I used an isolated application root so that the reproduction does not affect
the normal container state:

export APP_ROOT=/tmp/container-image-list-repro

container system start \
    --app-root "$APP_ROOT" \
    --disable-kernel-install

container image pull docker.io/library/alpine:latest
container system stop

I then added the following image record to $APP_ROOT/state.json without
creating its referenced content blob:

{
  "example.invalid/broken:latest": {
    "digest": "sha256:ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff",
    "size": 1,
    "mediaType": "application/vnd.oci.image.index.v1+json"
  }
}

The following file was intentionally absent:

$APP_ROOT/content/blobs/sha256/ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff

Behavior Before This PR

With container 1.3.0, before apple/containerization#898, the following
commands fail before rendering the healthy image:

container image list
container image list --format json
container image list --format yaml
container image list --verbose

Each command returns an error similar to:

Error: content with digest sha256:ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff

The command exits with status 1.

container image list -q continues to work because quiet mode does not resolve
the image content:

alpine:latest
example.invalid/broken:latest

Verification Against containerization#898

I repeated the same reproduction with container 1.3.1, which includes the
changes from apple/containerization#898.

The detailed listing commands still fail with the same missing-content-blob
error, while quiet mode remains successful.

This indicates that containerization#898 does not cover this case: the image
store state record is readable, but the content blob referenced by the image
record is missing. The failure occurs later when container image list tries to
resolve the image content.

Behavior With This PR

With this PR, the healthy image remains available and the broken image is
reported as a warning:

skipping unreadable image: ["error": notFound: "content with digest sha256:ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff", "image": example.invalid/broken:latest]
NAME    TAG     DIGEST
alpine  latest  28bd5fe8b56d
Error: failed to read 1 image(s)

The same skip-and-warn behavior applies to table, verbose, JSON, and YAML
output. Only missing content and content decoding errors are skipped; other
errors, including service interruptions, propagate normally.

For example, JSON output can be captured separately from the warning:

container image list --format json > images.json 2> warning.log

The result is:

  • exit status: 1 after the readable images are rendered;
  • images.json contains the healthy image and remains valid JSON;
  • warning.log identifies the skipped image and the underlying error, then
    reports that the listing was incomplete.

The healthy image can be inspected from the JSON output:

jq length images.json
# 1

jq -r '.[0].configuration.name' images.json
# docker.io/library/alpine:latest

Scope and Non-Goals

This PR changes image-listing behavior only.

It does not:

  • repair the missing content blob;
  • remove the broken image record;
  • add an image repair or prune command;
  • change the behavior of container image inspect, pull, or run;
  • hide the underlying error from the user.

The broken image is still identified, and its error is written to stderr. The
change keeps readable entries available in the listing without corrupting
structured stdout, while returning a failure status for incomplete results.

Testing

  • swift test --filter ContainerCommandsTests: 41 tests in 12 suites passed
  • make check: formatting and license checks passed
  • the command-level regression tests cover table, JSON, YAML, and TOML output
  • tests verify that healthy images before and after a broken image remain listed
  • tests verify that partial and fully unreadable listings throw after output
  • tests verify that service errors propagate instead of being skipped
  • the earlier isolated CLI reproduction verified warnings on stderr and valid JSON stdout

Summary

The underlying broken image resource remains unchanged, but container image list can now provide useful results instead of failing completely because of a
single unreadable image entry.


I use Codex help write this description

@Hokila
Hokila force-pushed the fix/2149-image-list-unreadable-entry branch from 3edfdd2 to fbea4e9 Compare August 26, 2026 02:43
@Hokila

Hokila commented Aug 26, 2026 •

Copy link
Copy Markdown
Contributor Author

Hi @katiewasnothere, this PR is ready for review.

  • make test: 760 tests in 83 suites passed
  • make check: formatting and license checks passed
  • The commit is GitHub SSH-signed and shows as Verified

Could you please approve the pending workflow and review this PR when you have a chance? Thank you!

@Hokila
Hokila force-pushed the fix/2149-image-list-unreadable-entry branch from 481bf16 to bb235f9 Compare August 27, 2026 01:47
@Hokila

Hokila commented Aug 27, 2026

Copy link
Copy Markdown
Contributor Author

@egernst
I've rebased this PR onto the latest main. Could you please rerun the workflow?
The previous failure appears to have been caused by a runner timeout, so I believe it should pass on a rerun. Thanks!

@Hokila
Hokila force-pushed the fix/2149-image-list-unreadable-entry branch from bb235f9 to b7be756 Compare August 27, 2026 06:06
@jglogan

jglogan commented Aug 28, 2026 •

Copy link
Copy Markdown
Contributor

@Hokila Can you add a reproduction procedure to your description to show how you're triggering the error?

That will help us determine whether the changes we merged in apple/containerization#898 resolve this issue for the 1.3.1 release.

Once we know how things behave with those changes, let's look at this PR and see what would be the best UX for handling broken resources (images in this case).

@Hokila

Hokila commented Aug 30, 2026

Copy link
Copy Markdown
Contributor Author

Thanks @jglogan. I’ll update the PR description with the exact reproduction procedure and retest it against a build that includes apple/containerization#898.

The current regression test covers per-image error isolation in the CLI, but it does not establish whether #898 already prevents or tolerates the underlying broken image-store state. I’ll document the before/after behavior once I’ve verified it with those changes.

After that, I’m happy to revisit whether the CLI-side skip-and-warn behavior is still needed and what the best UX should be for broken image resources.

@Hokila

Hokila commented Sep 2, 2026

Copy link
Copy Markdown
Contributor Author

Hi @jglogan, I’ve added the reproduction details and retested against 1.3.0, 1.3.1, and this PR.

The issue still reproduces on 1.3.1, while this PR keeps readable images listed and writes the broken image warning to stderr.

Would this skip-and-warn behavior be acceptable, or would you prefer a different UX here?

@Hokila

Hokila commented Sep 14, 2026

Copy link
Copy Markdown
Contributor Author

I’ve moved the complete reproduction procedure and the 1.3.0/1.3.1 comparison
into the PR description, with the repeated command output condensed.

The reproduction still confirms that containerization#898 does not cover the
missing-content-blob case. The PR description now also documents the intended
scope: skip-and-warn during listing, without attempting to repair or remove the
broken image resource.

@Hokila
Hokila force-pushed the fix/2149-image-list-unreadable-entry branch 3 times, most recently from 4e77336 to f123677 Compare September 22, 2026 13:23
@Hokila
Hokila force-pushed the fix/2149-image-list-unreadable-entry branch from f123677 to 71feeac Compare September 25, 2026 15:31
images: images,
containerSystemConfig: containerSystemConfig,
onError: { image, error in
log.warning(

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't like that the image list command now will always exit with 0 even if there was an error.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks let me know, fixed in b54383e: the command now outputs readable images, then exits non-zero if any image was skipped.

@github-actions

github-actions Bot commented Oct 2, 2026

Copy link
Copy Markdown

Code Coverage

Tier Line Coverage
Unit 25.29%
Integration 66.78%
Combined 76.32%

@Hokila
Hokila requested a review from katiewasnothere October 2, 2026 08:36
@jglogan
jglogan merged commit 38fe2f9 into apple:main Oct 2, 2026
3 checks passed
@jglogan

jglogan commented Oct 2, 2026

Copy link
Copy Markdown
Contributor

@Hokila merged, thanks for making the image listing more robust!

@Hokila
Hokila deleted the fix/2149-image-list-unreadable-entry branch October 23, 2026 03:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: container image list fails entirely when one content blob is unreadable

3 participants