Repository navigation
Conversation
CompressionStore::has_with_results reported digest.size_bytes() for a present digest key. That is the decompressed length for a CAS blob, but for an action-cache entry the digest is the Action size and the stored value is the ActionResult. After v1.8, FastSlowStore sizes an ExactSize fill from has() and rejects a length mismatch with DATA_LOSS. Present digest keys now report Footer.uncompressed_data_size, read from its fixed place in the frame footer. A missing footer is an error, not a fallback to the digest size. String keys and missing keys are unchanged.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Member
|
damn. this is good. |
Contributor
Author
|
Thanks @MarcusSorealheis, and thanks for merging main into the branch. |
Member
|
@mvanhorn we have to run more checks on our end because it is touch a few code paths that are sensitive. |
Contributor
Author
|
Makes sense, thanks @MarcusSorealheis. Take the time you need on the sensitive paths, and point me at anything you want changed. |
MarcusSorealheis
temporarily deployed
to
public-ci
November 20, 2026 05:39 — with
GitHub Actions
Inactive
Contributor
|
Build results for this fork PR at
Fork PRs cannot stream build events, so a trusted workflow publishes them after CI finishes and updates this comment as each job is published. Opening a build needs a Trace Machina sign-in; the check run lists the failing tests and their logs. |
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What and why
CompressionStore::has_with_resultsoverwrote a present digest key'ssize with
digest.size_bytes(). For a CAS blob that is the uncompressedlength. For an action-cache entry the digest is the Action, so
has()returned the Action size. After #2974,
FastSlowStoresizes anExactSizefill from that answer andvalidate_sizereturnsDATA_LOSSwhen the stored ActionResult is a different length.
Present digest keys now report
Footer.uncompressed_data_sizefrom itsfixed place in the frame footer. Missing keys stay
None, zero-digestkeys stay 0, and string keys keep the stored framed size. A truncated or
unreadable footer is an
Internalerror, matchingget_part; the storedoes not fall back to the digest size.
Fixes #3194
How was this verified?
On Linux x86_64 with
rustc/cargo1.97.1 (not the Nix flake; Bazeland
pre-commitwere not installed).Untouched
upstream/mainat12e8ae20:18 passed, 0 failed.
The three new tests, run against that same impl with only the test file
added:
has_reports_the_stored_uncompressed_size_when_the_digest_size_differsfailed with
has()=Some(167)instead ofSome(334).compressed_slow_tier_refills_when_digest_size_is_the_action_sizefailed with
DATA_LOSS: Expected 167 upload bytes, received 334.has_errors_when_the_frame_footer_cannot_be_readfailed because apresent garbage object still reported
Some(167).On this branch, the same
cargo +1.97.1 test -p nativelink-store --test compression_store_testcommand is 21 passed, 0 failed, including theexisting CAS
has()size test andcompressed_slow_tier_refills_filesystem_with_the_uncompressed_size.Also ran
cargo +1.97.1 clippy -p nativelink-store --test compression_store_test -- -D warnings(pass) andvale3.12.0 oncompose-stores.mdx(0 errors, 0 warnings, 0 suggestions).rustfmtfrom 1.97.1 on the touched Rust files. Not verified: Bazel
//nativelink-store:integration,bazel test //..., andpre-commit run -a.Risk
If the footer cannot be read,
has()now returnsInternalinstead ofa guessed size. Callers that assumed a present compression-store object
always has a numeric size will see an error on a truncated frame; that
matches
get_part. CAS existence checks for present digest keys nowread 13 bytes from the inner store, so a slow backend sees one extra
ranged read per hit. A wrong footer value would size a
fast_slowfillincorrectly and show up as
DATA_LOSSon the fill, the same symptom theAction size causes today.
AI assistance
An agent wrote it and I reviewed every line.
Demo
Full recording (MP4): https://raw.githubusercontent.com/mvanhorn/pr-demo-assets/main/osc-bulk-1118n/compression-ac-has-size/out.mp4