Skip to content

Share one seasonal difference helper and stop plotting missing data as 0% - #497

Open
brianpm-ucar wants to merge 1 commit into
NCAR:mainfrom
brianpm-ucar:unify-seasonal-diffs
Open

brianpm-ucar wants to merge 1 commit into
NCAR:mainfrom
brianpm-ucar:unify-seasonal-diffs

Conversation

@brianpm-ucar

Copy link
Copy Markdown
Collaborator

The polar maps end the whole ADF run for any variable whose history file has no units attribute. CAM writes AODVISdn that way, and with scale_factor: 1 the defaults never stamp new_unit, so the field reaches the plotting script with no units:

scripts/plotting/polar_map.py, line 49, in process_seasonal_data
    dseason.attrs['units'] = mseason.attrs['units']
KeyError: 'units'

The lat/lon maps do not fail on the same field because their own copy of process_seasonal_data never reads units. The two copies also differed in how they treat missing data: the polar one (and make_polar_plot itself) set every missing percent difference to 0, so masked points, such as SST over land and sea ice, were drawn as 0% and pulled the panel statistics toward zero. #495 noted this as left for later.

Change

  • New adf_utils.seasonal_diffs(test_data, ref_data, season), used by both global_latlon_map and polar_map in place of their process_seasonal_data copies. It returns the month-length weighted seasonal means, the difference and the percent difference. The difference keeps the units only if the data has them. The percent difference is in % and is missing (NaN) wherever it cannot be computed: a masked point or a zero reference value.
  • Missing percent differences are no longer set to 0 in make_polar_plot, plot_zonal_mean_and_save, plot_meridional_mean_and_save or square_contour_difference. They now plot as blank, as the lat/lon maps already did.
  • The lat/lon and polar colorbars are labeled with units: the mean and difference colorbars with a units entry from the variable defaults if there is one, otherwise the data's units attribute, and the percent colorbar with %. A label set in the defaults under mpl: colorbar: label still takes priority. This replaces the d1.units y-label in make_polar_plot, which would have failed the same way, and fixes the lat/lon percent colorbar label, which assigned set_ylabel = "%" instead of calling it and so never appeared.
  • New tests in lib/test/unit_tests/test_seasonal_diffs.py: values and units, a field with no units, and NaN for masked points and a zero reference.

Effect on real output (b.e30_alpha08o.BHISTC_MTt4s.ne30_t232_wgx3.330_branch vs obs, 1979-1980)

  • AODVISdn vs MODIS: main stops with the KeyError above. This branch makes all lat/lon, polar, zonal and meridional plots.
  • SST vs ERA-Interim, JJA, SH polar: on main the percent panel shows Antarctica and the sea ice region as 0%, and its mean is 0.02. Here those points are blank and the mean is 0.04. The other three panels and their numbers are unchanged.
  • SST, JJA, zonal mean: the percent curve now stops where SST is masked, instead of dropping to 0 south of 65S and north of 80N.

Checking

  • pytest lib/test/unit_tests: 273 passed. black --check lib/ is clean with the pinned black 25.11.0.
  • The runs above, with main run from a separate worktree on the same regridded files for the before/after comparison.
  • square_contour_difference has no caller in scripts/, so it was only checked by calling it directly with a reference that has NaN and zero rows.

Not changed here

  • The percent panel of the zonal mean plot labels its y-axis with the field's units (e.g. K) rather than %. This was already the case.
  • The MODIS AOD obs file MOD08_M3_192x288_AOD_2001-2020_climo.nc has a column at 180 degrees where every value is exactly 0, while its neighbors are missing in the Arctic. It shows as a thin line in the obs and difference panels; the percent panel now leaves it blank. That needs a fix to the file.

Claude-assisted: written with Claude Code.

🤖 Generated with Claude Code

…s 0%

polar_map ended the whole ADF run with KeyError: 'units' for any field
written without a units attribute (e.g. AODVISdn), because its copy of
process_seasonal_data read mseason.attrs['units'] directly.
global_latlon_map had its own copy that never read units, so the two
plot types behaved differently for the same field.

- Add adf_utils.seasonal_diffs() and use it in both scripts in place of
  the two process_seasonal_data copies.  It always uses the month-length
  weighted seasonal mean, carries units only when the data has them,
  and leaves the percent difference missing (NaN) where it cannot be
  computed.
- Stop setting missing percent differences to 0 in make_polar_plot,
  plot_zonal_mean_and_save, plot_meridional_mean_and_save and
  square_contour_difference.  Masked points (e.g. SST over land and sea
  ice) showed as 0% and pulled the regional statistics toward zero.
- Label the colorbars with units in the lat/lon and polar maps.  A units
  entry in the variable defaults is used first, then the data's units
  attribute; a colorbar label set under "mpl" in the defaults still wins.
  This also removes make_polar_plot's d1.units y-label, which failed
  the same way, and fixes the lat/lon percent colorbar label, which was
  assigned to set_ylabel instead of calling it and so never appeared.
- Add unit tests for seasonal_diffs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant