Repository navigation
Conversation
This comment has been minimized.
This comment has been minimized.
b7r6
force-pushed
the
repro/fastslow-populate-block-rounded
branch
from
October 22, 2026 13:33
0953a63 to
3bd8778
Compare
…stem slow tier Since 086289a (TraceMachina#2974) FastSlowStore's populate checks the bytes it received from the slow tier against the size it declared to the fast tier, and it takes that size from the slow tier's has(). A FilesystemStore reports its block-rounded footprint there (4096 for a 17-byte blob at the default block size), so every read that has to fill the fast tier from a filesystem slow tier fails with DataLoss: Expected 4096 upload bytes, received 17 Slow store ended before the cache fill was complete unless the blob is a block multiple. That is the dedup index store in nativelink-config/examples/filesystem_cas.json5 and the action cache in kubernetes/nativelink/nativelink-config.json5, which shipped broken in v1.8.0. This commit adds only tests, so CI goes red on it; the fix follows. - populate_from_filesystem_slow_tier_at_default_block_size: the shipped shape, with a real FilesystemStore slow tier. - populate_does_not_trust_slow_store_has_size_for_digest: a slow tier whose has() reports less (a compressed stream) or more (a block-rounded footprint) than the blob. - populate_keeps_entry_whose_size_is_not_its_digests: passes today and guards the fix against the obvious simplification of declaring the digest's size, which an AC or dedup index entry does not have. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
b7r6
force-pushed
the
repro/fastslow-populate-block-rounded
branch
from
October 22, 2026 13:34
3bd8778 to
9cc68b3
Compare
This was referenced Oct 22, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What and why
This PR is tests only and is meant to go red. The fix is stacked on it: see the PR based on this branch.
Since upstream TraceMachina#2974 (086289a)
FastSlowStore's populate checks the bytes it received from the slow tier against the size it declared to the fast tier, and it takes that size from the slow tier'shas(). AFilesystemStorereports its block-rounded footprint there (4096 for a 17-byte blob at the default block size), so every read that has to fill the fast tier from a filesystem slow tier fails withunless the blob is a block multiple. That is the dedup index store in
nativelink-config/examples/filesystem_cas.json5and the action cache inkubernetes/nativelink/nativelink-config.json5, and it shipped in v1.8.0.History: TraceMachina#2881 added the exact-size refusal on 10-15, TraceMachina#2953 reverted it the next day because "it needs a follow-up change before it can go into a release", TraceMachina#2974 relanded the check on 10-20, and TraceMachina#2960 covered the compression-store half. The FastSlow half (the follow-up) never landed.
Tests
populate_from_filesystem_slow_tier_at_default_block_size: the shipped shape, a realFilesystemStoreslow tier at the default block size behind a memory fast tier. Failed on v1.8.0; upstream Report content length from filesystem storehas()TraceMachina/nativelink#3006 (filesystemhas()reports the content length) fixes this shape and is in the fork'smainnow, so this one passes here. Kept as the regression test for the shipped bug.populate_does_not_trust_slow_store_has_size_for_digest: a slow tier whosehas()reports less (a compressed stream) or more (a block-rounded footprint) than the blob. Fails on main.populate_keeps_entry_whose_size_is_not_its_digests: passes today; it guards the fix against the obvious simplification of declaring the digest's size, which an AC or dedup index entry does not have.How was this verified?
bazel test //nativelink-store:integration_tests/fast_slow_store_test_teston fork main (upstreamb01417643e+ CI commit): 38 passed, 1 failed (populate_does_not_trust_slow_store_has_size_for_digest), with the DataLoss message above. On v1.8.0 the real-store test failed too.Risk
None to the code: tests only. The risk is in reading the red CI as anything other than the point of this PR; the fix stacked on it takes it green.
AI assistance
Written with Claude Code from a branch b7r6 wrote on 10-15 as the TraceMachina#2881 follow-up; b7r6 reviewed the tests and the history.
🤖 Generated with Claude Code