Skip to content

Build-namelist logic to find the finidat file is fragile and needs to be redone #2169

Open
3 of 4 issues completed

Description

@ekluzek

In working on #2008 I'm finding that the logic to fine the finidat file is: complex, hard to understand, fragile, and will require a bunch of duplication. This is another reason for doing #585 which is the long term solution. Although there we should make sure the redesign in python has a better method of getting the finidat files.

This is similar to #549, but slightly different and focussed on just the namelist logic and the namelist default settings for: use_init_interp and init_interp_attributes. This is also related to #2149, as we need to have duplicated lnd_tuning_mode options for all the different clm physics options, as well as the cam options. Right now this is going to require a bunch of duplication.

Definition of Done:

  • Improve the error message so it at least gives you a clue on what's wrong...
  • Add testing for finidat files to the build-namelist unit tester
  • Add a python tester to validate that the namelist_defaults files synchronize: init_interp_sim_years, use_init_interp, init_interp_attributes, and finidat
  • Fix case of finidat files with do_transient_pfts=.true. (example 1979 or 2000 files) so that use_init_interp doesn't need to be true
  • Reduce the required attributes to specify each file (see below)
  • Implement a better solution inside of the current buildnml or build-namelist?
  • Long term: Reimplement build-namelist as in Move CLMBuildNamelist.pm perl code into the python buildnml #585

Activity

  1. added
    code healthimproving internal code structure to make easier to maintain (sustainability)
    on Sep 22, 2023
  2. self-assigned this
    on Sep 22, 2023
  3. added
    enhancementnew capability or improved behavior of existing capability
    on Sep 22, 2023
  4. ekluzek commented on Sep 22, 2023

    @ekluzek
    ContributorAuthor

    What the error looks like when it happens is this:

    Create namelist for component clm
       Calling /glade/work/erik/ctsm_worktrees/ctsm5.2.mksurfdata/cime_config/buildnml
    ERROR: Command /glade/work/erik/ctsm_worktrees/ctsm5.2.mksurfdata/bld/build-namelist failed rc=255
    out=
    err=ERROR : CLM build-namelist::CLMBuildNamelist::setup_logic_initial_conditions() : use_init_interp is NOT synchronized with init_interp_attributes
    

    Which just isn't helpful enough.

  5. ekluzek commented on Sep 22, 2023

    @ekluzek
    ContributorAuthor

    The other error that can happen is:

    use_init_interp is set BUT finidat is NOT, need to set both

    which can happen because the user explicitly set use_init_interp and didn't set finidat on their own. Or it can happen because the default settings aren't setup correctly. This should be improved as well.

  6. ekluzek commented on May 11, 2024

    @ekluzek
    ContributorAuthor

    Working on this now as part of #2501 I see some improvements that could be made:

    • Remove glc_nec and ic_tod (they are all 10 and 0)
    • Possibly use one of ic_ymd and sim_year
    • Replace bgc (sp, bgc, fates) for use_cn, use_cndv, use_fates) use_cndv can be removed as part of Deprecate and remove BGCDV mode #2141
    • use one of use_crop and maxpft

    This would reduce the number of attributes needed to specify a finidat file from 15 to 9.

    Another part is to remove the duplication of the filenames in the XML and make sure that it's only the init_interp_attributes that are duplicated. In general removing duplication in the XML is a good goal and especially in duplicating the full path name to the files.

  7. ekluzek commented on Oct 11, 2024

    @ekluzek
    ContributorAuthor

    Adding to this: https://github.com/ESCOMP/CTSM/pull/2821/files#r1797496942

    so that 1979 and 2000 finidat files will work with do_transient_pfts=.true.

  8. added
    priority: highHigh priority to fix/merge soon, e.g., because it is a problem in important configurations
    b4bbit-for-bit
    on Feb 5, 2025
  9. ekluzek commented on Mar 23, 2026

    @ekluzek
    ContributorAuthor

    One thing that's problematic with the current setup is that the following variables need to be handled together and synchronized appropriately:

    • init_interp_sim_years
    • use_init_interp
    • init_interp_attributes
    • finidat

    A better way to handle this might be to do some checking of the namelist_default file to make sure it's constructed correctly keeping them all in sync.

    Also in many cases the above items are copied and don't need to be. Also it can be constructed so that finidat files are NOT copied, and only init_interp_attributes and use_init_interp need to be copied for some cases. So that's another thing that could be checked for.

    A checking script for this purpose could be implemented in python, which would make it more likely to be useful when we do move away from perl for build-namelist. It could also be run as a github action, which would catch these issues as they come in with each PR.

  10. added a commit that references this issue on Mar 23, 2026
  11. ekluzek commented on Nov 3, 2026

    @ekluzek
    ContributorAuthor

    #4252 is another example of a PR that was much more difficult both to create and review because of this problem.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

b4bbit-for-bitcode healthimproving internal code structure to make easier to maintain (sustainability)enhancementnew capability or improved behavior of existing capabilitypriority: highHigh priority to fix/merge soon, e.g., because it is a problem in important configurations

Type

No type

Projects

Relationships

None yet

Development

No branches or pull requests

Issue actions