Skip to content

Replace ttest with a local Welch's t-test - #22075

Merged
calixteman merged 1 commit into
mozilla:masterfrom
calixteman:rm-ttest
Oct 5, 2026
Merged

calixteman merged 1 commit into
mozilla:masterfrom
calixteman:rm-ttest

Conversation

@calixteman

Copy link
Copy Markdown
Contributor

Compute statcmp's two-sided p-values using the regularized incomplete beta function, removing ttest and its three transitive dependencies.

Add Scilab reference tests to gulp externaltest, with the script used to generate the expected values included in the test file.

Compute statcmp's two-sided p-values using the regularized incomplete beta
function, removing `ttest` and its three transitive dependencies.

Add Scilab reference tests to `gulp externaltest`, with the script used to
generate the expected values included in the test file.
@calixteman
calixteman requested review from Snuffleupagus and a balanced review from Copilot October 5, 2026 14:35
@calixteman calixteman added the test label Oct 5, 2026
@codecov-commenter

codecov-commenter commented Oct 5, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 90.28%. Comparing base (df84828) to head (fba07f3).

Additional details and impacted files
@@            Coverage Diff             @@
##           master   #22075      +/-   ##
==========================================
- Coverage   90.28%   90.28%   -0.01%     
==========================================
  Files         269      269              
  Lines       67362    67362              
==========================================
- Hits        60818    60817       -1     
- Misses       6544     6545       +1     
Flag Coverage Δ
browsertest 66.08% <ø> (+0.01%) ⬆️
fonttest 8.95% <ø> (ø)
integrationtest 69.23% <ø> (+0.01%) ⬆️
unittest 59.86% <ø> (-0.02%) ⬇️
unittestcli 58.41% <ø> (+0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟢 Approval recommended

The local implementation is numerically sound, well tested, correctly integrated, and cleanly removes the dependency chain.

Review effort: Balanced
Findings: None

What changed in this PR

Replaces the ttest dependency with a tested local Welch’s t-test implementation for statistical performance comparisons.

Changes:

  • Adds incomplete-beta-based Welch’s t-test calculations and Scilab reference tests.
  • Integrates the implementation into statcmp and external tests.
  • Removes ttest and its transitive dependencies.
File Description
test/​stats/​welch_ttest.js Implements the statistical calculations.
test/​stats/​welch_ttest_spec.js Adds Scilab-based reference tests.
test/​stats/​statcmp.js Uses the local Welch’s t-test.
gulpfile.mjs Runs the new tests in externaltest.
package.json Removes ttest.
package-lock.json Removes obsolete dependency entries.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@Snuffleupagus Snuffleupagus left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

r=me, thanks.

@calixteman
calixteman merged commit fa9d36e into mozilla:master Oct 5, 2026
22 checks passed
@calixteman
calixteman deleted the rm-ttest branch October 5, 2026 17:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants