Skip to content

Add full python unit testing suite to Github Actions workflow - #508

Merged
nusbaume merged 7 commits into
NCAR:mainfrom
nusbaume:clean_unit_tests
Nov 5, 2026
Merged

nusbaume merged 7 commits into
NCAR:mainfrom
nusbaume:clean_unit_tests

Conversation

@nusbaume

Copy link
Copy Markdown
Collaborator

This PR adds all of the new unit tests to the ADF unit-testing Github Actions workflow, so that the tests are exercised on every future PR into this repo.

This PR also removes a few unit tests which after some analysis were found to offer very little in terms of exercising the actual ADF code (e.g. they tested fundamental python capabilities instead of ADF-specific utilities).

All code modifications in this PR were generated with Claude Opus-5.5

nusbaume and others added 6 commits October 15, 2026 15:43
- full_env_unit_tests: don't end the step with an explicit "exit", which
  in a login shell runs ~/.bash_logout and turns exit code 0 into 1.
- Linting: install pytest in the lint job, and bring all modified unit
  test files up to a pylint score of 10.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@nusbaume nusbaume left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review summary

This PR moves the unit-test workflow from a PyYAML+pytest-only matrix on Python 3.9–3.13 to the real pinned ADF conda environment, where a skipped test fails the job. It adds a non-blocking canary on Python 3.13 with unpinned packages, adds markdown to the env file, makes lib/test pylint-clean, and removes a few low-value tests. All four checks are green. In both the canary steps (the plain run and the one with deprecation warnings as errors), 286 passed, 0 skipped, so the "skip = failure" guard is doing its job. I don't see anything blocking. A few items are worth tightening before merge.

Blocking

None.

Non-blocking

  1. The minimum supported Python version goes up, and the PR description doesn't say so. run_adf_diag:120 now refuses to start on Python < 3.12, and the README says the same. That's reasonable: the env pins 3.12, and CI no longer tests 3.9–3.11. But users running from their own 3.10/3.11 environments will hit a hard stop on their next pull. Please call this out in the PR description and the release notes.

  2. Removing test_AdfConfig_create drops the only test that runs AdfConfig on the shipped config_cam_baseline_example.yaml. Nothing else in lib/test now loads that file. pre-commit's check-yaml only checks that the YAML parses. It doesn't run expand_references() or the single-nesting rule, so a bad ${var} reference or an over-nested key in the example config would no longer be caught. I agree the old test was brittle because it asserted the /glade/campaign/... obs path. I'd suggest keeping a slim version that only constructs AdfConfig(config_cam_baseline_example.yaml) and maybe reads one required key. That checks the shipped config against the real parser, which isn't just "fundamental Python" behaviour.

  3. AGENTS.md §3 is still stale about linting, in the section this PR rewrites. .github/scripts/pr_mod_file_tests.py has linted every modified file under lib/ (minus lib/externals) since 07b292d. That's also why this PR has to add pytest to ADF_linting.yaml and add pylint disables across lib/test. AGENTS.md still says:

    • L56: the hard-coded "testable_files set";
    • L68: "Un-linted by CI: … lib/adf_dataset.py, lib/adf_derive.py, lib/adf_utils.py, lib/plotting_functions.py, lib/plotting_utils.py". This line was rewritten in this PR and is now wrong. Only scripts/ (and lib/externals/) are un-linted.
    • L79, L256, L336: "testable_files" / "the six linted files".

    Since the PR already edits this block, it would be good to fix it here. Otherwise reviewers following AGENTS.md will under-weight lint findings in lib/.

  4. The canary's "newest Python" is hard-coded to 3.13 (ADF_canary_tests.yaml:9 and :35). Python 3.14 has been out for about a year. Either bump the canary (if conda-forge xesmf/esmpy builds exist for it), or reword the comment and AGENTS.md to say "Python 3.13" rather than "newest", and note that the version has to be bumped by hand.

Optional / follow-up

  • jinja2 is imported by AdfWeb.create_website() (lib/adf_web.py:494) and installed explicitly in ADF_linting.yaml, but it isn't in env/conda_environment.yaml. Today it presumably arrives transitively. Since this PR adds markdown for the same reason, consider listing jinja2 too, so website generation doesn't depend on another package's dependency tree.
  • pytest (added via create-args) and pot/joblib are unpinned in the "pinned" environment. With cache-environment: true, the cached pytest only changes when the env file changes. That's fine for now, but worth knowing if a pytest-version difference ever shows up between local and CI runs.
  • The removed colormap_object tests were small, but that helper exists specifically because matplotlib 3.9 removed an API (see its Notes). The canary is now exactly the place a test like that would pay off. prep_contour_plot probably exercises it indirectly; if not, one assertion is cheap to keep.
  • The other removals look fine to me. Each is either duplicated elsewhere (pressure_field_name("lev"/"ilev") is still covered in test_pressure_field.py) or tests trivial logic (len(fils) > 1, isinstance after construction, file-exists).

Not verified

  • I did not rebuild the conda environment or run the suite locally. I relied on this PR's CI runs: full_env_unit_tests passed, and the canary reported 286 passed and 0 skipped in both steps.
  • I did not test the full ADF run_adf_diag end to end. The only runtime-code change is the version check; the lib/adf_file_utils.py and lib/adf_units.py edits only touch docstrings.

Review generated with Claude Code (Claude Opus 5.5), using the repo's AGENTS.md review guidelines.

… on latest stable Python.

- Add test_adfconfig_example_configs, which loads every top-level
  config_*.yaml with AdfConfig and checks that all "${var}"
  references expand.  This replaces the coverage lost when
  test_AdfConfig_create was removed, without hard-coded paths.
- Update AGENTS.md to reflect that pylint runs on every modified
  file under lib/ (except lib/externals), not a fixed file list.
- Have the canary workflow look up the latest stable Python release
  on every run instead of hard-coding 3.13, and update the docs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@nusbaume
nusbaume merged commit 2fcf147 into NCAR:main Nov 5, 2026
4 checks passed
@richling-ucar richling-ucar mentioned this pull request Nov 6, 2026
29 of 52 tasks
richling-ucar added a commit that referenced this pull request Nov 6, 2026
Conflicts:
- env/conda_environment.yaml: keep adf_v1.0.6 and uxarray=2026.10.0,
  take main's alphabetical order and markdown pin.
- test_reference_support_vars.py: keep this branch's
  load_reference_climo_dataset stand-in (main's load_reference_climo_ds
  does not exist here).
- test_adf_file_utils.py: take main's formatting; keep this branch's
  ts_years_covered/missing_year_ranges/years_tag/year_tagged_names tests.
Also point README/AGENTS at adf_v1.0.6.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@richling-ucar richling-ucar mentioned this pull request Nov 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant