Repository navigation
Description
| Q | A |
|---|---|
| OS | macOS (arm64) |
| Shell & version | bash 3.2.57 |
| bashunit version | 0.50.1 |
Summary
A test in a file that defines set_up or tear_down costs 3.3x a test in a file that does not. Each hook mints a mktemp scratch file for its output, and the ownership marker that file leaves behind makes the runner rm -rf the test's temp files at exit. Five process forks per test, none of them needed.
.claude/rules/perf-fork-budget.md says "the per-test path is genuinely fork-free". That is true, but only for a hookless file, which is what its census fixture uses. That is why this never showed up.
Measured
200 trivial tests (assert_same 1 1), best of 3, macOS arm64, bash 3.2.57:
| Fixture | Wall | Per test |
|---|---|---|
| 200 tests, no hooks | 1879 ms | 8.5 ms |
200 tests, empty set_up + tear_down |
5808 ms | 28.3 ms |
PATH-shim census over the same two fixtures, using the method in perf-fork-budget.md:
| Binary | Hookless | Hooked |
|---|---|---|
mktemp |
0 | 400 |
rm |
1 | 601 |
awk / base64 / mkdir / perl |
2 / 1 / 1 / 3 | unchanged |
Exactly five binary forks per hooked test: 2 mktemp, 2 rm -f, 1 rm -rf. Hooks cost +19.7 ms per test.
This suite has 1052 tests in files defining set_up or tear_down, out of ~2491. So roughly 20 seconds of a sequential run.
Current behavior
bashunit::runner::execute_test_hook, src/runner/hooks.sh:196-265:
hook_output_file=$(bashunit::temp_file "${hook_name}_output") # :215
...
rm -f "$hook_output_file" # :258bashunit::temp_file (src/api/globals.sh:38-51) forks $MKTEMP, and also writes $BASHUNIT_TEMP_DIR/${BASHUNIT_CURRENT_TEST_ID}_.mark (src/api/globals.sh:25-36). The hook's rm -f removes the output file but not that marker. So bashunit::cleanup_testcase_temp_files (src/api/globals.sh:70-100), reached from the EXIT trap at src/runner/hooks.sh:3-40, sees the marker and runs rm -rf.
A test that creates no temp file of its own still gets a marker, purely because a hook ran.
Same shape once per file in execute_file_hook (src/runner/hooks.sh:82-145): mktemp at :93, rm -f at :141.
Expected behavior
No process fork per hook. The hook still needs a file, because it runs in the current shell on purpose (see the comment at src/runner/hooks.sh:206-212: variables set before a failure have to survive), so only the file's name and lifecycle should change.
Name it arithmetically inside the run directory instead of going through temp_file. The > redirect at :239-241 truncates, so no rm is needed at all. The precedent is already in the tree: src/runner/result.sh:28-62 names the per-test .result file by an ordinal the dispatcher assigns before forking (#851), because a Bash 3 fork inherits both $$ and the RANDOM state and cannot mint a unique token.
Two details for whoever picks this up:
- Key on the ordinal, not
BASHUNIT_CURRENT_TEST_ID.bashunit::helper::generate_id(src/helper/encoding.sh:44-65) produces${fn}_$$, which repeats across data-provider rows. _test_ordinalis bumped only on the--parallelbranches today (src/runner/exec.sh:169-172and:216-219), so a sequential run leaves it at 0. It is also a local ofcall_test_functions, so it restarts per file, and needs combining with the folded file key the wayparallel_suite_dir_to_slotdoes (src/runner/result.sh:36-41).
The RED test belongs in tests/acceptance/bashunit_run_forks_test.sh, with a fixture that defines set_up and tear_down, asserting the mktemp and rm counts. perf-fork-budget.md needs the "fork-free" paragraph corrected to say hookless.
Metadata
Metadata
Assignees
Labels
Type
Projects
- StatusShow more project fieldsDone
Activity
Chemaclass commentedon Sep 9, 2026
The absolute figures in the title need re-measuring; the ratio holds
A second measurement pass on the same host (macOS arm64, bash 3.2.57) did not reproduce the 28.3 ms and 8.5 ms in the title. It got 18.2 ms hooked and 4.1 ms hookless, from a 1-test baseline of 97 ms and a 200-test run of 913 ms.
Both passes agree on everything that matters to the diagnosis:
mktemp, 2rm -f, 1rm -rf.mktempon macOS is expensive in absolute terms: 2815 us per call, against 280 to 580 us on Linux.The absolutes differ by about 2x and I cannot yet account for it, so treat the title's numbers as indicative and re-measure before quoting them in the CHANGELOG.
perf-fork-budget.mdalready warns that single runs vary and that a benchmark must be A/B'd in the same tree; this is that warning landing. The Linux figures for the same fixture are 2 to 4 ms of hook overhead per test, so the win is platform-weighted towards macOS.One correction to the design note above
The issue body says a Bash 3 fork inherits both
$$and theRANDOMstate.$$yes,RANDOMno: measured, subshells get a reseededRANDOMon 3.00.22, 3.2.57, 4.0.44, 5.2.37 and 5.3.15. See #1354.This does not change the recommendation. An ordinal is guaranteed unique, a reseeded
RANDOMis only probably unique, and the ordinal is what #851 already established. Only the stated reason was wrong.Chemaclass commentedon Sep 9, 2026
Done in #1355 (merged).