Repository navigation
CESM-compliant time variables and time-average output; output options for an external driver - #92
Merged
Merged
Conversation
In typical CISM runs, most output fields are snapshots of the ice state at a given time. However, some output fields (including most fluxes) are time-averaged. All time-averaged fields have the suffix '_tavg'. Until now, instantanteous and tavg fields could be mixed in the same output file. To be consistent with CESM practice, it is better to separate them. With this commit, the model aborts if the user mixes the two in any output variable list. The model also aborts if write_init = T for a time-average file, since it doesn't make sense to write time averages at t = 0. I considered having the model abort if the user lists more than one time-average file in the config file, but decided this was too strong a requirement. This means there is a risk that the same tavg field will appear in two different output files, which will lead to erroneous averaging. It would take some additional logic to examine multiple tavg variable lists and find repeat variables. One other change: The model aborts if the user sets restart = 1 (standard restart) and there is no file present with the designated restart filename. Previously, the default was to restart with the [CF input] file instead. This could lead to non-transparent fatal errors. These changes have been tested for standalone CISM runs but not in CESM. Most of the changes are in subroutine glimmer_nc_readparams, and they apply only to output files in sections called [CF output] or [CF restart]. Although glimmer_nc_readparams is called in CESM runs, the CISM config files do not have sections called [CF output] or [CF restart].
This commit changes the output time variables in several ways to meet CESM standards:
* The time variables ('internal time' and 'time') are now double-precision.
* These variables now have the long_name attributes 'internal time' and 'time'.
I removed the standard_name attributes.
* Time-average files (i.e., files containing one of more variables with the '_tavg' suffix)
now have the additional output variables internal_time_bounds and time_bounds,
also double precision. These the initial and final time of each averaging interval.
To support clean handling of time-average files, I introduced the requirement that
instantaneous and tavg variables cannot be mixed within a single output file.
At initialization, subroutine glimmer_nc_readparams parses the variable list
(contained in NCO%vars) to make sure that if any variables contain '_tavg',
then all variables contain '_tavg', else the code aborts. Also, the user must set
write_init = .false. for tavg files.
I added some logic using the processed_time property to determine time_bounds
for each tavg file over each averaging period.
Note: It is allowed (though likely not necessary) to have more than one tavg file.
However, a given variable can appear in only one such file, else it is handled incorrectly.
I haven't yet found a simple way to enforce this requirement.
I changed the default internal time units from 'common years since 0001-01-01'
to 'common years since 0000-01-01'.
The old baseline, year 1, implies that t = 0 corresponds to Jan. 1 of year 1,
hence t = 1950.0 is Jan. 1 of year 1951.
With a baseline year of 0, t = 1950.0 corresponds to Jan. 1, 1950, which is more intuitive
and is consistent with most forcing files.
I discovered that as glide_io_create.F90 parses NCO%vars, it removes the variable
substring for every variable it creates. Thus, if all goes well, 'vars' should
contain only empty spaces at the end. I added a warning alerting the reader if
this is not the case. In this way I found and fixed a couple of mistakes in glide_vars.def.
In several places I changed 'processsed' to 'processed'.
I deprecated the use of 'hot' to denote restart files. This was an old Glimmer convention.
…output
This commit continues the work on output time variables and time-average ('_tavg') files.
It refactors subroutine glimmer_nc_checkwrite, in preparation for letting an external
driver (e.g., the CESM wrapper) decide when output is written. It also fixes several bugs,
some new with the recent time-bounds commits and some long-standing.
Most changes are in libglimmer/glimmer_ncio.F90 and libglimmer/ncdf_template.F90.in.
Refactoring (answer-preserving):
* Subroutine glimmer_nc_checkwrite has been split into three steps, each in its own routine:
(1) glimmer_nc_advance_timeslice: leave define mode; advance the time counter after a write
(2) glimmer_nc_write_due: a logical function that decides whether a write is due
(forcewrite, write_init at tstep_count = 0, or the output frequency; not past end_write;
not already written at this time)
(3) glimmer_nc_write_timeslice: write internal_time, time, tstep_count and (for tavg files)
the time bounds, and reset the start of the averaging interval
glimmer_nc_checkwrite keeps its argument list and calls these three routines,
so existing callers are unchanged. This change alone is BFB for standalone runs.
An external driver that controls the timing of output can call step (3) directly.
* glimmer_nc_checkwrite has a new optional output argument, wrote_timeslice,
which is true if a time slice was written during the call (see bug fix 4 below).
* New subroutine glimmer_nc_checkwrite_init(outfile, time, external_time, tstep_count)
sets the start of the first averaging interval (processed_time, processed_external_time)
and the initial value of the new accum_tstep_count (see bug fix 3).
It is called from openall_out for every output file, with time = tstart.
An external driver that passes an external time to glimmer_nc_checkwrite should also
call glimmer_nc_checkwrite_init with the external start time.
Bug fixes:
1. The 'bounds' attribute on internal_time and time is now set only for tavg files.
Previously it was set for all files, so instantaneous files (including restart files)
pointed to time-bounds variables that did not exist, which violates CF conventions.
2. Standard restarts with a tavg output file crashed on the first tavg write
("NetCDF: Variable not found"). When an existing output file is reopened for appending,
glimmer_nc_openappend did not look up the varids of internal_time_bounds and time_bounds.
It now does so for tavg files.
3. Averages were accumulated whenever NAME_io_writeall was called, including calls
at initialization (tstep_count = 0) and in glide_finalise, which do not correspond
to a new timestep. As a result:
* The first averaging interval of a run (startup or hybrid restart) included the initial
state as an extra sample. For fields that are zero at initialization (e.g., acab_applied,
calving_rate), this biased the first average low by a factor n/(n+1),
where n is the number of timesteps in the interval.
* If glide_finalise forced a write of a partial interval (forcewrite_final = T),
the final timestep was counted twice.
Now avg_accumulate does nothing unless tstep_count has increased since the last accumulation.
The new variable accum_tstep_count (in type glimmer_nc_output) holds the tstep_count
of the most recent accumulation. This rule also protects the upcoming CESM design,
in which the wrapper calls writeall every step only to accumulate averages.
4. Long-standing bug: The last tavg slice of each run was overwritten with the
final-timestep value when the run ended on an output time.
NAME_io_writeall wrote the model fields whenever NCO%just_processed = T.
But just_processed remains true for later calls at the same time, e.g. in glide_finalise.
So the finalise call rewrote the fields of every file written at the final time.
For tavg files, the average had already been written and reset; the rewrite used only the
one extra accumulated step (bug 3), replacing the average with the final-timestep value.
Now NAME_io_writeall writes the fields only if wrote_timeslice = T.
This also avoids needless rewriting at finalise of all files written at the final time,
including (potentially large) restart files.
5. Time bounds were wrong for the first averaging interval after a restart, or in any run
with tstart /= 0. processed_time was initialized to 0, so the first interval after a
restart at t = 5 was written as [0,6] instead of [5,6].
This is fixed by calling glimmer_nc_checkwrite_init from openall_out.
6. Output was sometimes skipped at the time specified by 'stop' (end_write)
because of roundoff in the accumulated model time.
For example, with dt = 1/6 yr, the model time at year 8 is 8.00000000000001 > 8.
glimmer_nc_write_due now allows a tolerance of 1.e-3*tinc, the same convention
used for forcing times in NAME_read_forcing.
7. Implicit SAVE: 'logical :: forcewrite = .false.' in NAME_io_writeall (ncdf_template.F90.in)
and in glide_finalise and glide_finalise_all (glide_stop.F90) has an implicit SAVE attribute,
so once forcewrite was set to true, later calls would keep forcing writes.
forcewrite is now initialized in a separate executable statement.
This did not change answers in any tests, since these routines are typically called
only once with forcewrite = T, at the end of the run.
I also added a log message giving the averaging interval and total_time for each tavg write,
and added comments in glimmer_ncparams.F90 noting that the no-mixing rule for
instantaneous and tavg variables does not apply to [CF restart] files.
Testing (standalone):
I tested with an 8-km Antarctic spin-up configuration (DIVA, 256 tasks, dt = 1/6 yr):
a 5-year startup run followed by a 5-year standard restart. Output files were:
* an instantaneous 2D file with frequency = 1 and stop = 3 (stop = 8 on restart)
* a 2D tavg file (acab_applied_tavg, bmlt_applied_tavg, calving_rate_tavg) with frequency = 1
* a scalar tavg file (total_*_flux_tavg) with frequency = 2, so the forced final write
at t = 5 is a partial interval
* an instantaneous scalar file with frequency = 1
* a restart file with frequency = 5
with forcewrite_final = T. I compared each step with nccmp:
* The checkwrite refactoring (with fixes 1 and 2 applied to both the old and new code)
was BFB in all files.
* Fixes 3 through 6 changed only the tavg slices and the stop slice identified above.
Instantaneous and restart files were BFB, apart from the stop fix, which added
the missing year-8 slice.
* After the changes, total_time equals the interval length for every tavg slice,
and the time bounds are continuous across the restart ([4,5], [5,6], ...).
Expected answer changes:
Standalone CISM:
* Instantaneous output fields and restart files: BFB, except files with 'stop' set,
which may gain a slice at the stop time (fix 6).
* Tavg output fields: answer-changing in the first interval of a startup or hybrid run,
in the last interval of a run ending on an output time, and in partial intervals
written with forcewrite_final (fixes 3 and 4).
* Tavg time bounds: the first interval after a restart, or of any run with tstart /= 0
(fix 5).
* Standard restarts with a tavg output file used to abort; they now run (fix 2).
* Metadata: instantaneous files no longer have 'bounds' attributes on internal_time and time
(fix 1).
CESM (CISM-wrapper tests):
The wrapper creates its own history and restart output objects, calls glimmer_nc_createfile
and glimmer_nc_checkwrite (with forcewrite = .true.) directly, and calls glide_io_write itself.
In CESM configurations there are no [CF output] sections, so the only object in CISM's own
output list is the wrapper's oc_tavg_helper, which is never written.
For the changes in this commit, I expect:
* Instantaneous history fields (the default esm_history_vars): BFB.
* Restart files: BFB field values.
* Metadata: 'bounds' attributes are removed from internal_time and time in instantaneous history
files and in restart files. This may show up as metadata or header differences in baseline
comparisons, but not as field differences.
* Tavg history fields (only if the user adds _tavg fields to esm_history_vars):
expected BFB, but please check. The accumulate-once-per-step rule should not matter in CESM,
because oc_tavg_helper is created after the initialization call to glide_io_writeall,
and after a history write nothing reads the extra accumulation in glide_finalise.
tstep_count is incremented every GLAD step, including when evolve_ice = 0.
* Time bounds in wrapper tavg history files: unchanged, and still incorrect,
because the wrapper builds a new output object for each file
(so processed_time = 0 for every file). The wrapper will be fixed in a later commit.
* Exact-restart tests (ERS, ERI): should be unaffected by this commit.
Note for wrapper testing: Commit 8d56fb4 renamed an argument of glimmer_nc_createfile
from baseline_year to external_baseline_year. The wrapper must be updated to use the new name
(e.g., in source_glc/glc_io.F90, glc_io_write_history_tavg_helper and glc_io_write_restart
still pass baseline_year) before it will compile against this branch.
Upcoming work (not in this commit): 'external_control' and 'one file per write' flags
for output objects, and a template routine to write a complete single-slice file,
so the wrapper can use CISM's [CF output] machinery directly for h0i/h0a history files.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…rol flag
This commit adds the option to write each output time slice to a separate file,
while keeping a single persistent output object for each [CF output] section.
It also adds a flag, external_control, which will let an external driver
(e.g., the CESM wrapper) decide when output is written. The external_control flag
is read from the config file but does not yet change model behavior; that will
come in a follow-up commit.
These changes prepare for having CISM-wrapper write CESM history files (h0i and h0a)
through CISM's [CF output] machinery. In CESM, each history file contains one time slice,
whereas standalone CISM output files usually contain many. With one_file_per_write,
the averaging state (accumulated sums, total_time, processed_time for time bounds,
accum_tstep_count) lives in the output object and carries over from one file to the next,
so averages and time bounds are correct without the wrapper's oc_tavg_helper workaround.
New config options (in any [CF output] section; default = .false.):
* one_file_per_write: No file is created at initialization. Each time output is due,
a new file is created, one time slice is written, and the file is closed.
In standalone runs, the file name is built by inserting the time (yr) before '.nc':
e.g., name = out.tavg.nc gives out.tavg.0001.nc, out.tavg.0002.nc, ...
Integer years are written with at least 4 digits (0006, 1861, -21000);
other times with 3 decimal places.
Output timing is set by frequency, write_init, stop and forcewrite_final, as before.
* external_control: Reserved for an external driver. Currently it is read and logged only.
one_file_per_write is not allowed in a [CF restart] section (fatal error), because a
standard restart reads the file named in the config file.
Code changes:
* glimmer_ncdf.F90: New fields external_control, one_file_per_write and base_filename
in type glimmer_nc_output, and file_open in type glimmer_nc_stat.
delete_output closes a file only if it is open.
* glimmer_ncparams.F90: handle_output reads the two new flags; readparams aborts if
one_file_per_write is set for [CF restart].
* glimmer_ncio.F90:
- createfile and openappend set file_open = T.
- New subroutines glimmer_nc_newfile (set up an output object for a new file:
filename, variable list restored from vars_copy, timecounter = 1) and
glimmer_nc_closefile (close the file but keep the object).
- New function glimmer_nc_slice_filename (standalone file name for a given time).
- New function glimmer_nc_output_has_var (see below).
- openall_out creates no file at initialization for one_file_per_write objects,
but still calls glimmer_nc_checkwrite_init.
- glimmer_nc_advance_timeslice leaves define mode only if the file is open,
and for one_file_per_write objects does not increment timecounter or sync the (closed) file.
* ncdf_template.F90.in:
- New subroutine NAME_io_write_slice: create a new file, define the variables, fill the
dimension variables, write the time slice and fields, reset averages, close the file.
The optional arguments external_time, external_baseline_year and external_time_units
are passed through for use by an external driver.
- NAME_io_writeall: for one_file_per_write objects, accumulate averages as usual,
and if a write is due, call NAME_io_write_slice. The default path is unchanged.
- NAME_io_createall: for one_file_per_write objects, skip NAME_io_create at initialization,
but set do_averages from the variable list so averages accumulate before the first write.
* glide_nc_custom.F90: glide_nc_fillall skips one_file_per_write objects
(their dimension variables are filled when each file is created).
* utils/build/generate_ncvars.py: In the generated *_avg_accumulate and *_avg_reset routines,
the test for whether a tavg variable belongs to an output object now calls
glimmer_nc_output_has_var instead of parallel_inq_varid.
Previously the test asked the open netCDF file whether it contained the variable.
With one_file_per_write, no file is open between writes, so nothing was accumulated
and all averages were written as zero. (It also risked querying a stale file ID.)
glimmer_nc_output_has_var checks the open file, as before, if there is one;
otherwise it checks the variable list (vars_copy).
Limitations:
* Standalone one_file_per_write assumes that all variables in the file come from one
I/O module (e.g., glide_io, as in cism_driver). If a GLAD or Glint file set
one_file_per_write without external_control, each module would create its own copy
of the file. For CESM, the wrapper will create and write the files itself,
using external_control.
* As before, a given tavg variable should appear in only one output file.
Testing (standalone):
I used an 8-km Antarctic configuration (DIVA, 256 tasks, dt = 1/6 yr): a 5-year startup run
and a 5-year standard restart, writing an instantaneous 2D file (frequency = 1, stop = 3/8),
a 2D tavg file (frequency = 1), a scalar tavg file (frequency = 2, with a forced partial
interval at the end of the first run), an instantaneous scalar file, and a restart file,
with forcewrite_final = T.
* With the existing config files (no new options), output was BFB with the previous commit
(0a3fe86): all netCDF files identical with nccmp, and log files identical apart from
timestamps. This was checked after the infrastructure changes, after the
one_file_per_write changes, and after the generate_ncvars.py change.
* With one_file_per_write = T for the instantaneous 2D file and both tavg files,
the run produced the expected 23 single-slice files. Each file matched the corresponding
slice of the multi-slice file from the BFB run exactly, for all variables, including
the time variables, tstep_count, the time bounds and the tavg fields.
This confirms that averages and time bounds carry over correctly from file to file
and across the restart. The instantaneous scalar file and the restart file were BFB
(apart from the 'configuration' global attribute, which includes the config file text).
Expected answer changes:
* Standalone CISM: none, unless one_file_per_write is set.
* CESM (CISM-wrapper tests): none expected. The new flags default to .false.,
the wrapper creates its own output objects, and every output object the wrapper uses
has an open file, so the tavg accumulation test (glimmer_nc_output_has_var) behaves
as before.
* Glint: compiles (glint_io and glint_mbal_io are generated from the same template),
but was not run.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The previous commit (96f6083) added the config option external_control, which was read and logged but did not yet change model behavior. With this commit, the option takes effect. For an output object with external_control = .true., subroutine NAME_io_writeall (ncdf_template.F90.in) accumulates averages as usual but does nothing else. It does not decide whether output is due and never writes to the file, including at initialization (write_init) and at finalization (forcewrite_final). An external driver (e.g., the CESM wrapper) decides when to write and writes the output. This separates two responsibilities: * CISM owns WHAT is written: the variable list, metadata, averaging, and time bounds, with the checks already applied in glimmer_nc_readparams (e.g., no mixing of instantaneous and tavg fields; write_init = F for tavg files). * The driver owns WHEN (e.g., based on the CESM clock) and, with one_file_per_write, WHERE (the name of each file, e.g., following CESM conventions). The calling sequence for a driver is documented in a comment in NAME_io_writeall: (1) With one_file_per_write (the intended CESM case, one time slice per file): glimmer_nc_newfile, glimmer_nc_createfile, NAME_io_create (for each I/O module, e.g., glide_io and glad_io), glide_nc_filldvars, glimmer_nc_write_timeslice, NAME_io_write (for each module), NAME_avg_reset, glimmer_nc_closefile. If all the variables belong to one module, NAME_io_write_slice does all of this. The driver can add its own global attributes after the file is created. (2) Without one_file_per_write (the file is created at initialization and holds multiple time slices): glimmer_nc_checkwrite with the optional wrote_timeslice argument; then, if wrote_timeslice = .true., NAME_io_write (for each module) and NAME_avg_reset. In either case, a driver that passes an external time should call glimmer_nc_checkwrite_init at initialization with its external start time, so the first time bounds are in external units. Next step: modify CISM-wrapper to use this machinery for CESM history files (h0i and h0a), with buildnml writing [CF output] sections that set external_control and one_file_per_write. Testing (standalone, same 8-km Antarctic configuration as the previous two commits: 5-year startup run followed by a 5-year standard restart): * With the existing config files, output was BFB with the previous commit: all netCDF files identical with nccmp, and log files identical apart from timestamps. * Smoke test: external_control = T for the 2D tavg file (multi-slice mode), and external_control = T plus one_file_per_write = T for the instantaneous 2D file. As expected, the tavg file was created at initialization with all variables defined but no time slices, was reopened on restart, and still had no time slices at the end; no instantaneous 2D files were created; and the other output files and the restart file were BFB (apart from the 'configuration' global attribute, which includes the config text). The actual writing by an external driver will be tested with CISM-wrapper. Expected answer changes: * Standalone CISM: none, unless external_control is set. * CESM (CISM-wrapper tests): none. The wrapper does not yet set external_control. * Glint: compiles, but was not run. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…utput file
Each time-average ('_tavg') variable is accumulated in a single model array.
If the same variable is listed in two or more output files, it is accumulated once
per file per timestep, and the averages in all of those files are wrong.
Until now this was a known limitation that the code did not check.
This commit adds a private subroutine, check_duplicate_tavg, in glimmer_ncparams.F90.
It is called from glimmer_nc_readparams after all the [CF output] and [CF restart]
sections have been read. For each variable name ending in '_tavg' in an output file's
variable list, it searches the variable lists of the other output files
(whole-name matches only, so that e.g. calving_rate_tavg does not match
total_calving_rate_tavg). If a match is found, the model aborts with a message
naming the variable and the two files.
Notes:
* The check is based on names ending in '_tavg', consistent with the existing rule
that instantaneous and tavg variables cannot be mixed in a [CF output] file.
* Restart files are included in the check, but variables added by expanding the 'restart'
keyword are not checked (the expansion happens later, in NAME_io_create).
This is not an issue in practice, since restart files hold the instantaneous state
and normally contain no tavg variables.
* Listing a non-tavg variable in more than one file is still allowed.
* Once CISM-wrapper writes CESM history files through [CF output] sections, this check
will also apply to CESM runs.
Testing (standalone, 8-km Antarctic configuration, 5-year startup and 5-year restart runs):
* With the existing config files, output was BFB with the previous commit:
all netCDF files identical with nccmp, and log files identical apart from timestamps.
* With acab_applied_tavg added to the variable list of a second tavg file,
the model aborted at initialization with the expected message:
Error: time-average variable acab_applied_tavg is listed in more than one output file:
out_Antarctica_8km.tavg.nc
out_Antarctica_8km.scalars.tavg.nc
FATAL ERROR : Each tavg variable may appear in only one output file
* The name-matching logic was also tested standalone (duplicates anywhere in the list,
substring names, non-tavg duplicates, empty lists).
Expected answer changes: none. Runs that list a tavg variable in more than one output file,
which previously gave incorrect averages, now abort at initialization.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…d with '00:00:00' Two small metadata changes to follow CESM conventions for the time variables (see ESCOMP/CISM-wrapper issue #75): * The dimension of the time bounds variables (internal_time_bounds and time_bounds) is now called 'nbnd' instead of 'tbnd', as in other CESM components. The corresponding netCDF ID in type glimmer_nc_stat is renamed from tbnd_dim to nbnd_dim. (The 'tbnd' dimension was added only recently on this branch, so no existing production files use it.) * The units attributes of the time variables now end with '00:00:00' instead of '0:0:0', e.g. 'common years since 0000-01-01 00:00:00'. For CESM history files, once CISM-wrapper passes time in days since year 1, the units of 'time' will be 'days since 0001-01-01 00:00:00', as requested in issue #75. Both forms are valid under CF/UDUNITS; the new form matches other CESM components. Testing (standalone, 8-km Antarctic configuration, 5-year startup and 5-year restart runs): All netCDF files were BFB in their data compared to the previous commit (nccmp -d). The only header differences were the units strings of internal_time, time and the time bounds, and (in the tavg files) the name of the bounds dimension. Log files were identical apart from timestamps. Expected answer changes: none. Metadata changes only (units strings, and the bounds dimension name in files containing tavg fields). Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
CISM writes the units of internal_time (and, by default, of time and the time bounds) as '<unit> since 0000-01-01 00:00:00', where the unit is a year of exactly 365 days. Until now the unit was written as 'common years', with a space. UDUNITS spells this unit 'common_year' (plural 'common_years'), as noted in an existing comment in glimmer_nc_createfile. With the space, cftime cannot parse the units, so xarray's open_dataset fails on every CISM output file: ValueError: Failed to decode variable 'internal_time': unable to decode time units 'common years since 0000-01-01 00:00:00' with "calendar 'noleap'" With the UDUNITS spelling, 'common_years since 0000-01-01 00:00:00', current versions of cftime (tested 1.6.4) and xarray (tested 2025.7) decode internal_time and time as noleap dates, and open_dataset works with no special options. This resolves the question raised in ESCOMP/CISM-wrapper issue #75 about how to handle internal_time in CESM history files. The discussion there settled on keeping internal_time in both history and restart files, for both standalone and coupled runs, and making sure xarray can open the files. The earlier proposal was to remove 'since' from the internal_time units; testing showed that the failure was caused only by the space, so internal_time can remain a proper CF time coordinate. (In CESM, time will have units 'days since 0001-01-01 00:00:00', supplied by CISM-wrapper, while internal_time keeps CISM's units. The two can differ, e.g. in accelerated runs.) Note: Older versions of cftime that do not support 'common_years' will still fail to decode these units; the workaround is xarray.open_dataset(..., decode_times=False). Testing (standalone, 8-km Antarctic configuration, 5-year startup and 5-year restart runs): * All netCDF files were BFB in their data compared to the previous commit (nccmp -d). The only header differences were the units of internal_time, time and (in tavg files) the two time bounds variables. Log files were identical apart from timestamps. * xarray.open_dataset failed on all five output files from the previous commit and succeeded on all five from this commit, including the restart file and both tavg files, with time, internal_time and time_bounds decoded as noleap dates. Expected answer changes: none. Metadata change only (time units strings). Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Commit 8d56fb4 added a warning in NAME_io_create whose character string was continued onto the next line: write(message,*) 'Warning: glide_io_create may have not have created all the requested& & output variables in file ', trim(NCO%filename) With the gnu compiler, the preprocessor runs in traditional mode and does not recognize Fortran continuation lines. On the second line it sees an unmatched quote and treats the rest of the line as a string, so the NCO macro is not expanded. All the generated *_io.F90 modules then fail to compile: Error: Symbol 'nco' at (1) has no IMPLICIT type The Intel preprocessor handles this case, so standalone (Intel) builds were not affected. The failure appeared in CISM-wrapper tests built with gnu (e.g., ERI_Ly9.f09_g17_ais8gris4.T1850Gag.derecho_gnu.cism-noevolve_ais). The string is now on a single line, with the continuation after it. I also fixed the typo "may have not have" and changed 'glide_io_create' to 'NAME_io_create', so that each generated module reports its own name. A comment explains why the string should not be continued. This was the only line in CISM with a continued string on a line that uses the NCO or NCI macros. Testing: Generated glide_io.F90 from the old and new templates and ran gfortran 14.3 -cpp -E. With the old template, one NCO% reference was left unexpanded (the line above); with the new template, none. Message text change only; no answer changes. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…iles) CISM-wrapper exact-restart tests (e.g., ERS_Ly7.f09_g17_gris4.T1850Gg.derecho_intel) aborted when writing the CESM restart file: FATAL ERROR : (glimmer_ncio.F90:827) NetCDF: Variable not found The CESM restart file is created with the variable list ' restart '. glimmer_nc_createfile looks for '_tavg' in the list before the 'restart' keyword is expanded, so it correctly defines no time bounds variables for this file. Then NAME_io_create expands 'restart' into the full restart variable list, which for GLAD includes the coupling fluxes rofi_tavg, rofl_tavg and hflx_tavg, and sets do_averages = .true. because these names end in '_tavg'. glimmer_nc_write_timeslice wrote the time bounds whenever do_averages was true, so it tried to write variables that did not exist. Standalone CISM was not affected, because CISM's own restart list contains no '_tavg' names. The fix separates "this object accumulates averages" from "this file has time bounds": * New flag NCO%time_bounds (type glimmer_nc_stat), set to .true. by glimmer_nc_createfile when it defines the time bounds variables, and by glimmer_nc_openappend when it finds them in an existing file. * glimmer_nc_write_timeslice writes the time bounds only if NCO%time_bounds is true. For time-average output files the two flags agree, so these files are unchanged. Testing: * Compiled with Intel; checked with the gfortran preprocessor (no unexpanded NCO/NCI macros). * To be tested with the CISM-wrapper test subset and the aux_glc suite (restart tests), and with the standalone 8-km Antarctic test (expected BFB). Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…te a file Subroutine check_for_tempstag (glimmer_ncio.F90) converts 'temp', 'flwa' and 'dissip' in an output variable list to 'tempstag', 'flwastag' and 'dissipstag' (or the reverse for Glide). It is called from NAME_io_create, and it ended with nc%vars_copy = nc%vars NAME_io_create removes each variable name from nc%vars as it creates the variable. In CISM-wrapper, a CESM history file is created by calling glide_io_create and then glad_io_create for the same output object. When check_for_tempstag was called from glad_io_create, nc%vars had already been mostly consumed by glide_io_create, so vars_copy lost most of the variable list. This did not matter as long as each output object wrote a single file. But with the new one_file_per_write option, each file starts from vars_copy (glimmer_nc_newfile). In CISM-wrapper tests with the new h0i history stream, only the first h0i file written by each output object contained the history fields; later files contained only the dimension and time variables. Restart tests (ERS, ERP, ERI) then failed because the base and restart runs had different field lists. Standalone runs were not affected, since standalone CISM calls only glide_io_create for an output object. Now check_for_tempstag applies the same substitutions to nc%vars and to nc%vars_copy separately, and no longer copies vars into vars_copy. vars_copy is padded with spaces first, so that the first and last variable names in the list can also be matched. The log messages are unchanged. Testing: Compiled with Intel (with bounds checking); checked with the gfortran preprocessor. The replacement logic was tested standalone, including the case of a second I/O module calling check_for_tempstag after the first has consumed nc%vars. To be tested with the CISM-wrapper subset (h0i files) and with the standalone 8-km Antarctic test (expected BFB). Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
In files with time bounds (i.e., files containing time-average '_tavg' fields), internal_time and time are now the midpoints of the averaging interval, instead of the end of the interval. The time bounds (internal_time_bounds and time_bounds) still give the start and end of the interval. This follows CF conventions and CESM practice for averaged history fields, and was agreed on ESCOMP/CISM-wrapper issue #75 (resolving CISM-wrapper issue #76). It applies to both standalone and CESM runs. Instantaneous files are unchanged: internal_time and time are the end of the timestep, and there are no time bounds. The change is in glimmer_nc_write_timeslice (glimmer_ncio.F90). The log message "Writing to file ... at time" still gives the end of the interval (the current model time). Testing (standalone, 8-km Antarctic configuration, 5-year startup and 5-year restart runs): * Instantaneous output files and the restart file are BFB with the previous commit (nccmp, data and metadata). Log files are identical apart from timestamps. * In the two tavg files, only internal_time and time differ, and they are exactly the midpoints of the time bounds; e.g., for annual averages, 0.5, 1.5, ..., 9.5 (previously 1, 2, ..., 10), and for a scalar tavg file with 2-year averages and a partial interval at the end of the first run, 1, 3, 4.5, 5.5, 7, 9. All tavg fields and time bounds are unchanged. Expected answer changes: the values of internal_time and time in time-average output files (metadata only). No changes to output fields. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…acters
CISM's config file reader (glimmer_config.F90) limited each value to 400 characters (valuelen),
and read each line with a fixed format of 450 characters. Longer values caused a fatal error
("The value of config option ... is too long to be read fully").
This limit became a problem in CISM-wrapper tests: with a few time-average fields added to the
default CESM history variables, the esm_history_vars line in cism.<icesheet>.config is
427 characters long, and the model aborted at initialization. History variable lists are
expected to grow (e.g., for CMIP7).
Changes:
* valuelen is now 4096, matching len_history_vars in glad_type.F90 (the maximum length of the
CESM history variable list). linelen = valuelen + namelen + 1 increases accordingly.
* Each line is read with format '(a)', i.e., up to the length of the line buffer, instead of the
hardwired '(a450)'. Values that are too long are still caught in InsertValue.
Notes:
* Output variable lists in [CF output] sections can hold up to 1024 characters (glimmer_nc_vars_len);
CISM-wrapper's buildnml checks this limit.
* ConfigAsString writes the whole config to a string of length 10000 (configstring), which is used
only for the 'configuration' global attribute of output files. For a very long config file,
this attribute would be truncated; this is unchanged.
Testing: Compiled with Intel (with bounds checking). To be tested with the CISM-wrapper tests that
use time-average history fields, and with the standalone 8-km Antarctic test (expected BFB).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…tart mid-interval
Previously, the running sums of time-average ('_tavg') fields were not saved in restart files.
After a restart in the middle of an averaging interval, a new interval started at the restart time,
so the first average after the restart did not match the average in a continuous run. This commit
saves the averaging state in restart files and restores it on a standard restart (restart = 1),
so the averages continue exactly (bit-for-bit with a continuous run). This is the CISM part of
ESCOMP/CISM-wrapper issue #75, item 6; the wrapper part (CESM continue runs) will follow.
What is saved:
* Running sums: For each '_tavg' field in a time-average output file, a variable '<name>_tavg_sum'
is added automatically to the restart variable list. It is written from the same array as
'<name>_tavg' (e.g., model%climate%acab_applied_tavg), but without dividing by the total time,
and it is read back into that array. There is no separate '_tavg_sum' array. The units are those
of the field times years (e.g., '(meter/year)*year' for acab_applied_tavg_sum); the scale_factor is
the same as for the field, and the restart read divides it out, so the sums are restored exactly.
The '_tavg_sum' variables are generated by generate_ncvars.py; users should not list them in output
files, and glimmer_nc_readparams aborts if a [CF output] section lists one. See the note on
acab_tavg in glide_types.F90.
* Averaging state of each time-average stream: total_time, the start of the averaging interval
(internal and external time), and the timestep of the most recent accumulation, in new restart
variables tavg_total_time, tavg_start_time, tavg_start_external_time and tavg_accum_tstep_count,
with dimensions (tavgstream, time). The global attribute 'tavg_streams' lists the streams by name
(base_filename for one_file_per_write streams, else the file name), in order.
When it is restored:
* Standard restart (restart = 1; CESM continue runs map to this): the averaging state of each
stream is restored from the restart file, matched by name. If the restart file has averaging state
but not for a current stream, the model aborts, suggesting a hybrid restart (restart = 2) to change
the time-average streams. If the restart file has no averaging state (e.g., an older restart file),
a warning is written and a new averaging interval starts.
* Startup (restart = 0) and hybrid restart (restart = 2): a new averaging interval starts at the
initial time, and the running sums are set to zero, including any read from the input file.
Other changes:
* NAME_io_writeall writes restart files in a second pass, after all other output at the same time,
so the saved averaging state is the state after the time-average files have accumulated this
timestep and (if they wrote a time slice) been reset. Also, if a restart file was written earlier
at the same time (in the last timestep), and a time-average file then writes a slice in the final
call from glide_finalise (forcewrite_final = .true.), the restart time slice is rewritten with the
state after the reset. Otherwise, a standard restart would continue an interval that had already
been written.
* The test for whether an output file holds time averages (do_averages) now matches whole names
ending in '_tavg', so that restart files, which hold '_tavg_sum' variables, do not accumulate
averages themselves (which would double-count, since the arrays are shared).
* New helper routines in glimmer_ncio.F90: glimmer_nc_is_tavg_stream, glimmer_nc_is_restart,
glimmer_nc_stream_name, glimmer_nc_tavg_streams, glimmer_nc_write_tavg_state,
glimmer_nc_restore_tavg_state. New generated routines in each I/O module:
NAME_add_tavg_sums_to_restart and NAME_is_avg_var (new generator token !GENVAR_AVGLIST!).
Glint and Glad: their I/O modules have no '_tavg' fields, so nothing is added for them, but they
are generated from the same template and compile. (Glint may be deprecated; not tested here.)
Testing (standalone CISM, 8-km Antarctic iotest setup, 256 tasks on derecho, intel):
* 5-yr run plus 5-yr standard restart (iotest, forcewrite_final = .true.): all output files are
bit-for-bit with the previous commit. Restart files gain the new variables; all other restart
variables are bit-for-bit. The saved averaging state at t = 5 and t = 10 has zero sums and
total_time, with the intervals starting at the restart time, since the final call wrote and reset
the averages.
* Restart in the middle of averaging intervals (forcewrite_final = .false.; 2-D tavg file with
frequency 3, i.e. interval [3,6], and scalar tavg file with frequency 2, interval [4,6], both
spanning the restart at t = 5): both time-average files are bit-for-bit with a continuous 10-yr
run. The other output files are also bit-for-bit, except for differences in inversion-related
fields that are not new (the previous commit gives exactly the same differences between continuous
and restarted runs):
- bmlt_float_target (and total_bmlt_float_target) is zero after the restart; the field is not in
restart files.
- coulomb_c, and powerlaw_c, which is a function of coulomb_c here (which_ho_powerlaw_c = 4),
differ at some vertices, all with f_ground = 0, where basal friction is not applied. coulomb_c
also differs in the restart file at t = 10.
All other fields are bit-for-bit. In this short test, the differences in coulomb_c do not affect
other answers, but they would if those vertices regrounded later. These restart issues are not
related to this commit and will be revisited later.
A continuous 10-yr run is bit-for-bit with the previous commit (all output files, and all restart
variables other than the new ones).
* Hybrid restart (restart = 2, tstart = 100) from the two restart files above, one with nonzero
partial sums and one with zero sums: all output, including restart files, is bit-for-bit between
the two runs, and the first averaging intervals start at t = 100.
New logical tavg_restored in glimmer_nc_output, set in NAME_io_createall for each time-average output stream: true if the averaging state was restored from the restart file (standard restart, restart = 1), else false. This is for the CESM wrapper, which needs to know whether CISM restored the state: * CESM branch and hybrid runs (with an evolving ice sheet) also use restart = 1, reading the refcase restart file, but the wrapper starts a new averaging interval in those runs, so it must override the restored state. * In a CESM continue run from a restart file written before the previous commit (with no averaging state), the wrapper falls back to setting the start of the averaging interval itself. No answer changes: the flag is not used in standalone CISM. Builds with intel and gnu. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
GLAD read the CESM history variable list (esm_history_vars) from the [esm_output] section of the config file and stored it in glad_instance, but it is no longer used: the CESM wrapper now writes history files through CISM's [CF output] machinery, and the variable lists are in the h0i and h0a [CF output] sections of the config file. This commit removes esm_history_vars and len_history_vars from glad_type.F90. GLAD still reads history_option and history_frequency from [esm_output], for use by the wrapper. Config files that still have esm_history_vars in [esm_output] (e.g., from an older version of the CESM wrapper) still work: CISM checks for unused config sections, but not for unused keys in a section that is used. The maximum length of a config value stays at 4096 characters (glimmer_config.F90), since [CF output] variable lists can be long; the comment there is updated. No answer changes. Builds with intel and gnu. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Katetc
reviewed
Oct 9, 2026
Katetc
left a comment
Contributor
There was a problem hiding this comment.
This looks really nice to me. My only concern is about glide_types
Contributor
Author
|
@Katetc, thanks for bringing this in! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR updates CISM's netCDF output, mainly in
libglimmer(glimmer_ncio.F90,glimmer_ncparams.F90,glimmer_ncdf.F90,ncdf_template.F90.in) andutils/build/generate_ncvars.py:writes output.
It is the CISM side of ESCOMP/CISM-wrapper issue #75 (and CISM-wrapper #76). The companion
CISM-wrapper PR, ESCOMP/CISM-wrapper#150, uses these changes to write CESM history files
(h0i and h0a) through CISM's
[CF output]machinery.Changes that may require config file changes (standalone)
These checks are new and abort at initialization with a clear message:
[CF output]section may not mix instantaneous fields andtime-average (
_tavg) fields. Put the_tavgfields in a separate file.write_initfor averages: time-average files must havewrite_init = .false., since thereis nothing to average at initialization.
_tavgfields: a_tavgfield may be listed in only one output file. Eachaverage is accumulated in a single model array, so listing it twice gave wrong averages.
restart = 1, the model aborts if the file named in[CF restart]does not exist. Previously it fell back silently to the[CF input]file._tavg_sumvariables: variables ending in_tavg_summay not be listed in[CF output].They are added to restart files automatically (see below).
Time variables and time-average files
internal_timeandtimeare double precision, with long namesinternal timeandtime(no
standard_name).common_years since 0000-01-01 00:00:00. This is the UDUNITSspelling (previously
common years, which cftime/xarray could not decode), with base year 0(previously 0001), so that t = 1950.0 is Jan 1, 1950. Files now open with
xarray.open_dataset,with times decoded as noleap dates.
internal_time_boundsandtime_bounds, withdimensions
(time, nbnd), and aboundsattribute on the time variables. Instantaneous andrestart files have no time bounds.
internal_timeandtimeare the midpoints of theaveraging interval (CF and CESM practice). The bounds give the start and end.
time(e.g., days since0001-01-01 in CESM);
internal_timealways uses CISM's units.Bug fixes in time-average output
glide_finalise. This biased the first average of a run, and double-counted the last step of apartial interval written with
forcewrite_final.was overwritten by the final-timestep value in
glide_finalise.tstart /= 0(e.g., [0,6] instead of [5,6]).not found") when the file was reopened.
stop: output at thestoptime was sometimes skipped, because of roundoffin the model time.
forcewritehad an implicit SAVE attribute inNAME_io_writeallandglide_finalise.vars_copy) was lost when more than one I/O module(e.g.,
glide_ioandglad_io) wrote the same file.boundsattribute: instantaneous files had aboundsattribute pointing to time-boundsvariables that did not exist.
ncdf_template.F90.inhad a continued string that the gnu preprocessor couldnot handle.
Exact restart of time averages
Restart files now include the averaging state of each time-average output stream, so averages
continue exactly across a standard restart, including in the middle of an averaging interval:
<field>_tavgin an output file, a<field>_tavg_sumvariable is addedto the restart file automatically. It's the running sum of the field times the timestep, with
units of the field times years. These variables are generated by
generate_ncvars.py.timestep of the last accumulation. These are written in variables with dimensions
(tavgstream, time), and streams are matched by name (global attributetavg_streams).restart = 1):new interval.
restart = 2): a new averaging interval starts, with zero sums.the state after any average written at that time. If the final call from
glide_finalisewritesan average after the restart file was written, the restart time slice is rewritten.
tavg_restored: a new flag that records whether the state was restored, for use by anexternal driver.
New output options for an external driver (
[CF output])one_file_per_write: each write creates a new file with one time slice, while a single outputobject keeps the averaging state from file to file.
out.tavg.0001.nc).[CF restart].external_control: CISM accumulates averages but never writes the file itself. The driverdecides when to write, using a documented calling sequence (see
NAME_io_writeall).glimmer_nc_checkwriteis split intoglimmer_nc_advance_timeslice,glimmer_nc_write_dueandglimmer_nc_write_timeslice, with new routinesglimmer_nc_checkwrite_init,glimmer_nc_newfile,glimmer_nc_closefileandNAME_io_write_slice.Other changes
output variable lists.
esm_history_varsfrom[esm_output]. Older config files withthis key still work.
Expected answer changes (standalone)
stopmay gain the slice at the stop time (fix 5).interval of a run ending on an output time, and in partial intervals written with
forcewrite_final(fixes 1 and 2).corrected after restarts.
nbnddimension, andboundsattributes._tavg_sumand averaging-state variables, if there are time-averageoutput files.
Testing
Standalone (derecho, intel), after each step:
time-average, scalar and restart files): compared each step with nccmp. Answer changes are only
those listed above.
intervals span the restart): averages are bit-for-bit with a continuous run.
restart = 2,tstart = 100) from restart files with and without nonzeropartial sums are bit-for-bit with each other, with new intervals starting at t = 100.
marine_margin = 1,calving_init = 1): restarts from t = 0 andt = 1 are bit-for-bit with a startup run.
*_io.F90files were checked for unexpandedNCO/NCImacros with the gfortran preprocessor.
CESM: tested through the CISM-wrapper branch. The full
aux_glcsuite passes, and all exact-restartcomparisons pass. See ESCOMP/CISM-wrapper#150 for details.
Glint: the generated Glint I/O modules compile. Glint was not run (its deprecation is planned).
Known issues, not changed by this PR
After a standard restart,
bmlt_float_targetis zero, because it is not in restart files. Withinversion,
coulomb_candpowerlaw_cdiffer at vertices withf_ground = 0; they would affectanswers if those cells reground. To be investigated separately.
🤖 Generated with Claude Code