Skip to content

feat(cli): --order-by=defects to run the last run's failures first #1011

Closed

Description

@Chemaclass

Problem

--rerun-failed replays the last failing tests and drops everything else (src/config/rerun.sh:113 filter_functions). That is the right tool while fixing a specific failure, but it is the wrong tool for CI and for the final check before pushing, where you want the full suite — just with the known-bad tests first, so --stop-on-failure trips in seconds instead of minutes.

The failure cache that makes this possible already exists (.bashunit/last-failed, src/config/rerun.sh); nothing consumes it for ordering.

Proposal

--order-by <mode>     defects | defined (default) | random
  • defects — tests that failed on the last recorded run execute first, in the order recorded; everything else follows in its normal order. The full suite still runs.
  • defined — current behaviour, named so it can be set explicitly in .bashunitrc.
  • random — equivalent to the existing --random-order; keep --random-order and --seed working unchanged as aliases.

With no cache file, defects degrades silently to defined.

Where to change

  • src/config/rerun.sh:68 load, :83 files, :96 allows — read the cache without filtering it.
  • src/helper/discovery.sh:208 load_test_files and :62 get_functions_to_run — ordering hook, same place --random-order applies.
  • src/main/test.sh, src/config/env.sh (BASHUNIT_ORDER_BY), .env.example.

Acceptance criteria

  • --order-by defects runs the full suite with previously failing tests first
  • --order-by defects --stop-on-failure stops on the first known-bad test on a repeat run
  • Missing or empty .bashunit/last-failed degrades to defined order without a warning storm
  • --order-by random behaves identically to --random-order, and --seed still applies
  • An unknown mode exits non-zero with a usage error listing the valid modes
  • --order-by and --rerun-failed together: --rerun-failed still narrows the set, --order-by only orders it
  • Ordering is stable under --parallel in the sense that the cached failures are dispatched first
  • Unit tests in tests/unit/config/rerun_test.sh plus an acceptance test

Repo checklist (agent)

  • TDD: RED → GREEN → REFACTOR. Write the failing test first.
  • Bash 3.0+ only: no printf -v, no += append, no declare -A, no [[ ]], no ${var,,}, no &>>, no ${arr[-1]}. Expanding a possibly-empty array under set -u needs ${arr[@]+"${arr[@]}"}.
  • A new CLI flag must be wired in all of these or a parity test fails:
  • Gates: make sa, make lint, ./bashunit tests/, ./bashunit --parallel tests/. Never run shfmt -w.
  • Docs: update docs/command-line.md. Editing docs/assertions.md invalidates the bashunit doc acceptance snapshot — regenerate it.
  • CHANGELOG.md: add one line under ## Unreleased.
  • Fixtures under tests/acceptance/fixtures/ must not end in *test.sh.
  • One issue = one PR.

Activity

  1. self-assigned this
    on Aug 10, 2026
  2. Chemaclass commented on Aug 10, 2026

    @Chemaclass
    MemberAuthor

    Done in #1039 (merged).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Type

No type

Projects

  • Status
    Done

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions