Repository navigation
Labels
bugsomething is working incorrectlysomething is working incorrectlypriority: highHigh priority to fix/merge soon, e.g., because it is a problem in important configurationsHigh priority to fix/merge soon, e.g., because it is a problem in important configurationstestingadditions or changes to testsadditions or changes to tests
Milestone
Description
Brief summary of bug
Baseline testing shows the st_archive tests as failing.
General bug information
CTSM version you are using: ctsm5.3.041
Does this bug cause significantly incorrect results in the model's science? No
Configurations affected: st_archive in test lists
Details of bug
Tests when run show the the ST_ARCHIVE fails. For example...
FAIL SMS_Ld10_D_Mmpi-serial.CLM_USRDAT.I1PtClm60SpRs.izumi_nag.clm-default--clm-NEON-TOOL SHORT_TERM_ARCHIVER exception
Important details of your setup / configuration so we can reproduce the bug
Normal aux_clm test suite.
Important output or errors that show the problem
---------------------------------------------------
2025-04-29 14:41:04: Exception during SHORT_TERM_ARCHIVER:
ERROR: Moved file rpointer.lnd_9999.1976-01-03-00000 with dispostion ignore to directory env_archive_test/archive/rest/1976-01-01-00000
Traceback (most recent call last):
File "/fs/cgd/data0/erik/ctsm_worktree/externals_update/cime/CIME/SystemTests/system_tests_common.py", line 788, in _phase_modifying_call
function()
File "/fs/cgd/data0/erik/ctsm_worktree/externals_update/cime/CIME/SystemTests/system_tests_common.py", line 775, in _st_archive_case_test
result = self._case.test_env_archive()
File "/fs/cgd/data0/erik/ctsm_worktree/externals_update/cime/CIME/case/case_st_archive.py", line 1260, in test_env_archive
_check_disposition(testdir)
File "/fs/cgd/data0/erik/ctsm_worktree/externals_update/cime/CIME/case/case_st_archive.py", line 1309, in _check_disposition
_file, root
File "/fs/cgd/data0/erik/ctsm_worktree/externals_update/cime/CIME/utils.py", line 176, in expect
raise exc_type(msg)
CIME.utils.CIMEError: ERROR: Moved file rpointer.lnd_9999.1976-01-03-00000 with dispostion ignore to directory env_archive_test/archive/rest/1976-01-01-00000
---------------------------------------------------
./xmlquery -p DOUT
Results in group run_data_archive
DOUT_S: FALSE
DOUT_S_SAVE_INTERIM_RESTART_FILES: FALSE
Metadata
Metadata
Assignees
Labels
bugsomething is working incorrectlysomething is working incorrectlypriority: highHigh priority to fix/merge soon, e.g., because it is a problem in important configurationsHigh priority to fix/merge soon, e.g., because it is a problem in important configurationstestingadditions or changes to testsadditions or changes to tests
Activity
ekluzek commentedon Apr 29, 2025
This must be due to #3067, but I swear we didn't see this before. Although it the ctsm5.3.041 baselines on both Derecho and Izumi do show it as a problem.
It seems to be doing something with the tests in config_archive.xml, which I didn't think it exercised.
ekluzek commentedon Apr 30, 2025
OK, first I figured out an easy way to test this. In a test case you just do this
./case.st_archive --test-case(and also try --test-all to test the config_archive.xml file)
Next it looks like that cime has to be updated to cime6.1.091 and cmeps to 1.0.47.
There also seems to be a problem with this line in the config_archive.xml file (in the file testing section).
This filename should NOT be copied, but seems to being copied anyway. CAM has some files like this that seem to work correctly. So I'm not clear on what's going on here. But, should be able to get something working with at least a workaround.
ekluzek commentedon May 1, 2025
OK, so this is dependent on cmeps and cime externals update, which we aren't quite ready for on master. And I'm not sure the st_archive is working properly in that I thought it shouldn't move files that weren't for the date from the cpl files of 1976-01-01-00000, so there might be a problem there that we should resolved. And most importantly we need to get the tests working in config_archive.xml so that the st_archive part won't fail.
So we'll revert this both on master and b4b-dev. And then on b4b-dev we'll rebring it in with submodule updates needed as well as changes so that the CTSM config_archive.xml will PASS.
ekluzek commentedon May 7, 2025
I was afraid that this bug meant that st_archive would also fail for production cases that use create_newcase rather than create_test.
So to check in ctsm5.3.041 I ran this test:
./create_newcase --compset I1850Clm60BgcCropCrujra --res f09_f09_mg17 --case st_archive_test_case --run-unsupported ./xmlchange STOP_N=1 ./xmlchange STOP_OPTION=nmonths ./case.setup vi user_nl_clm. # Add use_init_interp = TRUE ./case.build ./case.submitThe st_archive batch log ends with "st_archive completed" and all of the files seem to be archived correctly.
I then continued the run setting CONTINUE_RUN=TRUE to verify it could continue on. And it was fine.
ekluzek commentedon Jul 28, 2025
@jedwards4b and @samsrabin FYI this is the issue that documents the problems I ran into when originally bringing in the rpointer changes. It looks like part of it had to do with cmeps and cime updates that we weren't ready to bring in at the time. But, we've now brought those updates in and are bringing in the very latest cime update as well. So I'm hopeful we'll have things working with this go around.
ekluzek commentedon Jul 28, 2025
Since, the "fix" that came in with #3106 was just doing a revert this issue should NOT have been closed at that point and should have stayed open to document that we still needed to unrevert it.
samsrabin commentedon Jul 29, 2025
Closing as completed with #3354 even though it hasn't yet merged to master. Note that #3351 is still open.