Repository navigation
Conversation
This reverts commit 3948191.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Member
|
in general, this is a very risky change |
Member
|
but it is an improvement. I'm doing some more tests now. |
Contributor
|
Build results for this fork PR at
Fork PRs cannot stream build events, so a trusted workflow publishes them after CI finishes and updates this comment as each job is published. Opening a build needs a Trace Machina sign-in; the check run lists the failing tests and their logs. |
This branch had an error being deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What and why
Deep input trees can overflow a 2 MiB worker stack during materialization, including with directory caching disabled. Box the filesystem-store emplacement future and poll directory-cache child construction through a scoped work queue instead of recursively polling nested directory futures.
Keep subtree reuse, per-digest construction locks, pinning, and eviction behavior. On construction failure, cancel child work and wait for its descendants to exit before removing the parent's temporary tree.
The latest test-only update keeps hard-link source and destination paths on the same filesystem and streams subprocess phase logs rather than buffering them until exit.
How was this verified?
Separate Linux CI tested source commit
77932f53edd625f8fee640fda2451c0b33996de5:./tools/bazel-retry.sh test //... --config=nl-toolchain --config=nl-quiet --lockfile_mode=error --verbose_failures --nocache_test_results, with a loopback MongoDB test service supplied throughNATIVELINK_TEST_MONGO_URL: 145/145 test targets passed. The stack target completed in 15.518 seconds.smoland unoptimizedtestprofiles: depths 8/128, stacks 2/16 MiB, no/default/subtree caching, cold/hot paths, and fetch budgets 1/2/64. Every case completed setup and materialization; every hot case completed prewarming. Setup and prewarming use a separate 32 MiB stack; the materialization probe uses the selected 2/16 MiB stack.parent completed while a cancelled descendant was still alive.-D warnings, pinned formatting, and directory-cache integration/unit/stress, filesystem-store, fast-slow-store, and S3-store tests pass.The original unoptimized 128-level, 2 MiB stack-overflow reproductions failed during materialization, not setup or prewarming.
Separately, the previous GitHub Linux PR run at
8a523272hit the stack target's overall 60-second timeout. Its cause remains undetermined: it has not reproduced in the separate Linux CI runs, and the old buffered output did not identify the stalled phase. The latest phase logging improves diagnosis if it happens again; it is not claimed to fix that timeout. GitHub checks for this update must be evaluated independently.Risk
This changes a worker hot path. Scoped job accounting adds per-ancestor bookkeeping, and cancellation mistakes could leak work or remove a temporary directory too early. The tests exercise constrained budgets and shared-subtree contention, not every possible scheduling interleaving.
Root cancellation still uses background scratch cleanup. These tests do not establish synchronous cancellation of all underlying filesystem I/O, eliminate every recursive path, or establish safety at arbitrary tree depth. No configuration defaults or wire formats change.
AI assistance
OpenAI Codex implemented the fix and tests, investigated CI failures, and drafted this description. Ping directed the investigation, questioned the cancellation and deadlock behavior, and requested the real-filesystem regression and constrained-concurrency checks.