Skip to content

FatesColdTwoStream tests rely on Python module in .bash_profile #2335

Closed

Description

@samsrabin

Brief summary of bug

The Izumi FatesColdTwoStream and FatesColdTwoStreamNoCompFixedBioGeo tests from aux_clm fail, if called with run_sys_tests, unless Python is loaded in a user's .bash_profile (or presumably the respective profile files for other shells). The failure happens during the CREATE_NEWCASE step, during the call of shell_commands.

See this error reported by @slevis-lmwg during testing of during #2334, as well as subsequent messages from me.

General bug information

CTSM version you are using: ctsm5.1.dev164

Does this bug cause significantly incorrect results in the model's science? No

Configurations affected: n/a

Details of bug

The user can solve this by adding module load lang/python/3.7.0 to their .bash_profile, but it would be better to solve this permanently rather than relying on the user figuring this out.

The problem is that shell_commands in these tests calls the script $FATESDIR/tools/modify_fates_paramfile.py, which depends on numpy, scipy, etc. It would be more appropriate for that script to be called in the run_phase of a new SystemTest—call it MODIFYFATESPARAMF. This would make it so that, as long as the user calls run_sys_tests with ctsm_pylib loaded, everything necessary will be available. @glemieux, what do you think?

Important details of your setup / configuration so we can reproduce the bug

On Izumi, in bash, with no module load commands in my .bash_profile, I set up the failing test like so:

./run_sys_tests --skip-compare --skip-generate -t ERS_D_Ld15.f45_f45_mg37.I2000Clm50FatesRs.izumi_nag.clm-FatesColdTwoStream

Important output or errors that show the problem

TestStatus.log:

RUN: /scratch/cluster/samrabin/tests_0124-132302iz/ERS_D_Ld15.f45_f45_mg37.I2000Clm50FatesRs.izumi_nag.clm-FatesColdTwoStream.0124-132302iz/shell_commands
FROM: /scratch/cluster/samrabin/tests_0124-132302iz/ERS_D_Ld15.f45_f45_mg37.I2000Clm50FatesRs.izumi_nag.clm-FatesColdTwoStream.0124-132302iz
  stat: 1

  errput: Traceback (most recent call last):
  File "/home/samrabin/ctsm/src/fates/tools/modify_fates_paramfile.py", line 35, in <module>
    from scipy.io import netcdf as nc
ImportError: No module named scipy.io
Leaving broken case dir /scratch/cluster/samrabin/tests_0124-132302iz/ERS_D_Ld15.f45_f45_mg37.I2000Clm50FatesRs.izumi_nag.clm-FatesColdTwoStream.0124-132302iz
ERROR: Command: '/scratch/cluster/samrabin/tests_0124-132302iz/ERS_D_Ld15.f45_f45_mg37.I2000Clm50FatesRs.izumi_nag.clm-FatesColdTwoStream.0124-132302iz/shell_commands' failed with error 'Traceback (most recent call last):
  File "/home/samrabin/ctsm/src/fates/tools/modify_fates_paramfile.py", line 35, in <module>
    from scipy.io import netcdf as nc
ImportError: No module named scipy.io' from dir '/scratch/cluster/samrabin/tests_0124-132302iz/ERS_D_Ld15.f45_f45_mg37.I2000Clm50FatesRs.izumi_nag.clm-FatesColdTwoStream.0124-132302iz'

 ---------------------------------------------------
2024-01-24 13:23:13: CREATE_NEWCASE FAILED for test 'ERS_D_Ld15.f45_f45_mg37.I2000Clm50FatesRs.izumi_nag.clm-FatesColdTwoStream'.
Command: /bin/python3 /home/samrabin/ctsm/cime/CIME/scripts/create_newcase.py --case /scratch/cluster/samrabin/tests_0124-132302iz/ERS_D_Ld15.f45_f45_mg37.I2000Clm50FatesRs.izumi_nag.clm-FatesColdTwoStream.0124-132302iz --res f45_f45_mg37 --compset I2000Clm50FatesRs --test --machine izumi --compiler nag --project P93300641  --output-root /scratch/cluster/samrabin/tests_0124-132302iz  --workflow default --srcroot /home/samrabin/ctsm --user-mods-dir /home/samrabin/ctsm/cime_config/testdefs/testmods_dirs/clm/FatesColdTwoStream
Output: Compset longname is 2000_DATM%GSWP3v1_CLM50%FATES_SICE_SOCN_SROF_SGLC_SWAV
Compset specification file is /home/samrabin/ctsm/cime_config/config_compsets.xml
Automatically adding SESP to compset
Leaving broken case dir /scratch/cluster/samrabin/tests_0124-132302iz/ERS_D_Ld15.f45_f45_mg37.I2000Clm50FatesRs.izumi_nag.clm-FatesColdTwoStream.0124-132302iz
ERROR: locked: cannot make child group in file /scratch/cluster/samrabin/tests_0124-132302iz/ERS_D_Ld15.f45_f45_mg37.I2000Clm50FatesRs.izumi_nag.clm-FatesColdTwoStream.0124-132302iz/env_case.xml

 ---------------------------------------------------
2024-01-24 13:23:15: CREATE_NEWCASE FAILED for test 'ERS_D_Ld15.f45_f45_mg37.I2000Clm50FatesRs.izumi_nag.clm-FatesColdTwoStream'.
Command: /bin/python3 /home/samrabin/ctsm/cime/CIME/scripts/create_newcase.py --case /scratch/cluster/samrabin/tests_0124-132302iz/ERS_D_Ld15.f45_f45_mg37.I2000Clm50FatesRs.izumi_nag.clm-FatesColdTwoStream.0124-132302iz --res f45_f45_mg37 --compset I2000Clm50FatesRs --test --machine izumi --compiler nag --project P93300641  --output-root /scratch/cluster/samrabin/tests_0124-132302iz  --workflow default --srcroot /home/samrabin/ctsm --user-mods-dir /home/samrabin/ctsm/cime_config/testdefs/testmods_dirs/clm/FatesColdTwoStream
Output: Compset longname is 2000_DATM%GSWP3v1_CLM50%FATES_SICE_SOCN_SROF_SGLC_SWAV
Compset specification file is /home/samrabin/ctsm/cime_config/config_compsets.xml
Automatically adding SESP to compset
Leaving broken case dir /scratch/cluster/samrabin/tests_0124-132302iz/ERS_D_Ld15.f45_f45_mg37.I2000Clm50FatesRs.izumi_nag.clm-FatesColdTwoStream.0124-132302iz
ERROR: locked: cannot make child group in file /scratch/cluster/samrabin/tests_0124-132302iz/ERS_D_Ld15.f45_f45_mg37.I2000Clm50FatesRs.izumi_nag.clm-FatesColdTwoStream.0124-132302iz/env_case.xml

 ---------------------------------------------------

Activity

  1. added
    bugsomething is working incorrectly
    nextthis should get some attention in the next week or two. Normally each Thursday SE meeting.
    on Jan 24, 2024
  2. samsrabin commented on Jan 25, 2024

    @samsrabin
    MemberAuthor

    See also discussion from CTSM SE meeting here.

  3. removed
    nextthis should get some attention in the next week or two. Normally each Thursday SE meeting.
    on Feb 1, 2024
  4. added
    good first issuesimple; good for first-time contributors
    and removed on Oct 3, 2024
  5. ekluzek commented on Oct 9, 2024

    @ekluzek
    Contributor

    Note, this starts showing up in ctsm5.3.003 and after for SMS_Lm3_D_Mmpi-serial.1x1_brazil.I2000Clm50FatesCruRsGs.izumi_intel.clm-FatesColdHydro because of a similar addition to shell_commands for it. It worked for @glemieux for ctsm5.3.003 because of his startup shell options.

  6. ekluzek commented on Feb 5, 2025

    @ekluzek
    Contributor

    This was resolved in ctsm5.3.014

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    b4bbit-for-bitbugsomething is working incorrectlygood first issuesimple; good for first-time contributors

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions