Skip to content

[WIP] feat: mark and skip flaky test - #746

Closed
marco-ippolito wants to merge 4 commits into
nodejs:mainfrom
marco-ippolito:mark-flaky-test
Closed

marco-ippolito wants to merge 4 commits into
nodejs:mainfrom
marco-ippolito:mark-flaky-test

Conversation

@marco-ippolito

@marco-ippolito marco-ippolito commented Oct 18, 2023 •

Copy link
Copy Markdown
Member

Draft implementation to add flaky tests automatically to the .status file.
Missing:

  • rules to determine which tests we consider flaky
  • tests
  • proper matching for OS and architecture

I'm assuming that if the failed test shows up in at least 5 pr is flaky (debatable) and can be skipped, I'd increase this number to make sure no false positive

@codecov

codecov Bot commented Oct 20, 2023 •

Copy link
Copy Markdown

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (6d68c99) 83.03% compared to head (1a12479) 82.95%.
Report is 15 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #746      +/-   ##
==========================================
- Coverage   83.03%   82.95%   -0.08%     
==========================================
  Files          37       37              
  Lines        4178     4190      +12     
==========================================
+ Hits         3469     3476       +7     
- Misses        709      714       +5     

see 2 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant