Skip to content

refactor(zipapp): port zip_main_maker to Rust and add it to shrubbery - #4440

Open
rickeylev wants to merge 3 commits into
bazel-contrib:mainfrom
rickeylev:copy_zip_main_maker
Open

rickeylev wants to merge 3 commits into
bazel-contrib:mainfrom
rickeylev:copy_zip_main_maker

Conversation

@rickeylev

Copy link
Copy Markdown
Collaborator

Replace the Python zip_main_maker used by py_zipapp's
PyZipAppCreateMainPy action with a Rust implementation. Together with
exe_zip_maker and zipper, all py_zipapp zip actions now run
through shrubbery without invoking a Python interpreter when
shrubbery is resolved.

crates/zip_main_maker mirrors tools/zipapp/zip_main_maker.py: it
parses the same CLI and @flagfile flags, hashes sorted manifest lines
and file contents or symlink targets with SHA-256 across regular,
symlink, inferred-symlink (-1), empty, and explicit-symlink entries,
and applies template substitutions. py_zipapp selects the
zip_main_maker subcommand via get_shrubbery_tool and falls back to
the Python tool when no shrubbery toolchain is resolved.

  • Also extract crates/zip_manifest so zipper and zip_main_maker
    share |-delimited manifest line and is_symlink parsing.
  • Also return std::process::ExitCode instead of i32 or
    process::exit from shrubbery and its subcommand crates.
  • Also document in agent guidance that
    # buildifier: disable=bzl-visibility is allowed on repo-internal
    /private/ helper loads in non-python/ packages such as crates/
    and tools/.

Work towards #4250

Replace the Python `zip_main_maker` used by `py_zipapp`'s
`PyZipAppCreateMainPy` action with a Rust implementation. Together with
`exe_zip_maker` and `zipper`, all `py_zipapp` zip actions now run
through `shrubbery` without invoking a Python interpreter when
`shrubbery` is resolved.

`crates/zip_main_maker` mirrors `tools/zipapp/zip_main_maker.py`: it
parses the same CLI and `@flagfile` flags, hashes sorted manifest lines
and file contents or symlink targets with SHA-256 across regular,
symlink, inferred-symlink (`-1`), empty, and explicit-symlink entries,
and applies template substitutions. `py_zipapp` selects the
`zip_main_maker` subcommand via `get_shrubbery_tool` and falls back to
the Python tool when no `shrubbery` toolchain is resolved.

* Also extract `crates/zip_manifest` so `zipper` and `zip_main_maker`
  share `|`-delimited manifest line and `is_symlink` parsing.
* Also return `std::process::ExitCode` instead of `i32` or
  `process::exit` from `shrubbery` and its subcommand crates.
* Also document in agent guidance that
  `# buildifier: disable=bzl-visibility` is allowed on repo-internal
  `/private/` helper loads in non-`python/` packages such as `crates/`
  and `tools/`.

Work towards bazel-contrib#4250
@rickeylev
rickeylev requested a review from aignas as a code owner November 8, 2026 10:15
@rickeylev

Copy link
Copy Markdown
Collaborator Author

This PR was prepared with AI agent assistance and verified locally by the author. Review comments on this PR will be handled directly by the human author, or reviewed and approved by the human author before any agent-assisted reply is posted.

This is an automatically generated agent comment.

On non-Unix targets, the `#[cfg(unix)]` symlink entry block in
`test_creates_zip_main` is compiled out, leaving `vec![...]` borrowed
only for `.join("\n")` and triggering `-D clippy::useless-vec`.

Join a fixed-size array into a string and append the symlink entry by
rebinding on Unix so Clippy passes on both Unix and Windows.
@rickeylev

Copy link
Copy Markdown
Collaborator Author

@arrdem @Solumin moar rust. Something new i saw in this was 'a. Some sort of indicator to tie the lifetime of inputs to the created object? Sounded rather clever.

@Solumin

Solumin commented Nov 8, 2026

Copy link
Copy Markdown

@arrdem @Solumin moar rust. Something new i saw in this was 'a. Some sort of indicator to tie the lifetime of inputs to the created object? Sounded rather clever.

Yeah! https://doc.rust-lang.org/book/ch10-03-lifetime-syntax.html
It's one of the Rust topics that tends to trip people up, but really, it makes explicit something that is implicit in other languages. You can do some really cool stuff with lifetimes, too, like zero-copy data structures that are completely type safe.

You don't see lifetimes often because the compiler can infer them in a lot of common cases: https://doc.rust-lang.org/nomicon/lifetime-elision.html

@Solumin Solumin left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nothing stands out to me as particularly bad --- which is a bit suspicious in its own right. ;)

@rickeylev

Copy link
Copy Markdown
Collaborator Author

I've been doing pre-reviews interrogating anything I don't entirely understand. Something I asked it about (but haven't looked back at yet) were the string literals for case matching, which seem like easy typos

Replace repeated subcommand, CLI flag, and manifest wire-token string
literals in `match` arms and error messages with module-qualified
constants so mismatches fail to compile instead of falling through to
wildcard arms or binding catch-all pattern variables.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants