Repository navigation
Run tests on wheels - #1977
Run tests on wheels#1977reinecke wants to merge 5 commits into
Conversation
ac4cb5f to
39742e7
Compare
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #1977 +/- ##
=======================================
Coverage 83.62% 83.62%
=======================================
Files 184 184
Lines 13627 13627
Branches 1270 1270
=======================================
Hits 11395 11395
Misses 2059 2059
Partials 173 173
Flags with carried forward coverage won't be shown. Click here to find out more. Continue to review full report in Codecov by Harness.
🚀 New features to boost your workflow:
|
ec030a9 to
742dc6c
Compare
|
I'm ready for feedback on this - just the one open questions about whether or not we really need the smoketests or if I should move the coverage reports to the tests we run on the equivalent wheel. |
9ef3995 to
d9e6e95
Compare
d9e6e95 to
9c1f1da
Compare
Re-tool the Python package workflow to test installed wheels (and the sdist for platforms without a wheel, e.g. mingw) instead of the source checkout. Replace the dev extra with tests/requirements.txt. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Signed-off-by: Eric Reinecke <ereinecke@netflix.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Signed-off-by: Eric Reinecke <ereinecke@netflix.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Signed-off-by: Eric Reinecke <ereinecke@netflix.com>
9c1f1da to
88d7806
Compare
In test_wheels, install the wheel before checking out the source so the tests run against the wheel and not the source tree, matching py_testonly_build. Split the dependency install so the test requirements are installed after the checkout. Rename py_smoketest_build to py_coverage_build and drop its Python matrix. It now builds on the coverage Python only (GH_COV_PY); all Python versions are built and tested as wheels by package_wheels and test_wheels. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Signed-off-by: Eric Reinecke <ereinecke@netflix.com>
Revert the removal of extras_require['dev'] and the related README and Makefile hint changes. Removing the extra is a user-facing packaging change that belongs in a separate PR. tests/requirements.txt stays, because the wheel test jobs need to install the test dependencies without building the package from source. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Signed-off-by: Eric Reinecke <ereinecke@netflix.com>
JeanChristopheMorinPerso
left a comment
There was a problem hiding this comment.
I took a look and saw no major things that needs addressing. I only found minor things.
| run: | | ||
| echo 'OTIO_CXX_DEBUG_BUILD=1' >> $GITHUB_ENV | ||
| sudo apt-get install lcov | ||
| - name: Install ZLIB (Linux) |
There was a problem hiding this comment.
| - name: Install ZLIB (Linux) | |
| - name: Install zlib |
The "Linux" bit is more confusing than anything.
| run: sudo apt-get install -y zlib1g-dev | ||
| - name: Install python build dependencies | ||
| run: | | ||
| python -m pip install --upgrade pip setuptools wheel "flake8>=3.5" check-manifest |
There was a problem hiding this comment.
Do we need flake8 and check manifest if we are only interested in code coverage?
| path: 'wheelhouse' | ||
| merge-multiple: true | ||
| - name: Install wheel for environment | ||
| run: pip install --no-cache-dir --no-index --find-links wheelhouse opentimelineio |
There was a problem hiding this comment.
| run: pip install --no-cache-dir --no-index --find-links wheelhouse opentimelineio | |
| run: python -m pip install --no-cache-dir --no-index --find-links wheelhouse opentimelineio |
That's what we do everywhere else
| include: | ||
| - { os: ubuntu-latest, shell: bash } | ||
| - { os: macos-15, shell: bash } | ||
| - { os: windows-2022, shell: pwsh } |
There was a problem hiding this comment.
I'm curious, why force push? I don't see any powershell code in the workflow.
|
|
||
| env: | ||
| OTIO_CXX_COVERAGE_BUILD: 'ON' | ||
| OTIO_CXX_BUILD_TMP_DIR: ${{ github.workspace }}/build |
There was a problem hiding this comment.
| env: | |
| OTIO_CXX_COVERAGE_BUILD: 'ON' | |
| OTIO_CXX_BUILD_TMP_DIR: ${{ github.workspace }}/build |
This is not needed since we aren't building.

This change re-shuffles the CI so that:
py_coverage_buildbuild runs and generates coverage using the same methodology our previous test builds used..whlbefore pulling the source code for running tests to ensure we don't accidentally rebuild from source.tests/requirements.txtis a new file that CI uses to install test dependencies. It duplicates the dev extra, because the test jobs cannot use the extras otherwise they might install OTIO from source.Note:
ubuntu-24.04-armandmacos-15cp39wheels are built but not tested.Ideally this would take the suggestion from @JeanChristopheMorinPerso and run againstsdist- that change hasn't been made.Update: after investigation, cibuildwheel isn't really meant to happen from sdist sources. I did a pass to make our build and test only phase run off it though - since those platforms would be running off sdist in the wild.
Additionally, if we could capture coverage from one of the cibuildwheel runs, we could probably eliminate the smoke test phase.
Consider dropping the smoke test and just harvest coverage from testing the wheelsFixes #1027
This supersedes #1961 - the changes have been added to this PR
Assisted-by: Claude Code / Claude Sonnet 5.5