Skip to content

ctsm5.3.082: Update to official CMIP7 population density data #3545

Description

@ekluzek

We need to update to the official CMIP7 population density dataset.

The dataset is documented here:

NCAR/CMIP7_inputdata_processing#20

Activity

  1. added theissue type on Oct 13, 2025
  2. added
    enhancementnew capability or improved behavior of existing capability
    non-b4bChanges answers (incl. adding tests)
    on Oct 13, 2025
  3. ekluzek commented on Oct 13, 2025

    @ekluzek
    ContributorAuthor

    @slevis-lmwg I think of this as something for you to do, if that works for you. I assigned you to the issue in CMIP7.

  4. self-assigned this
    on Oct 13, 2025
  5. wwieder commented on Oct 13, 2025

    @wwieder
    Contributor

    I don't know if this is different from what Fang's already using, @lifang0209 or @lawrencepj1 can you help clarify here?

  6. slevis-lmwg commented on Oct 13, 2025

    @slevis-lmwg
    Contributor

    @wwieder I tried a simple diff between the two files and they seem different.

    I have generated a new file here:
    /glade/work/slevis/git_cmip7/CMIP7_inputdata_processing/code/pop-density

    I can refrain from moving it to /inputdata and from pushing my code changes to the cmip7 repo until we confirm that the new file works.

  7. ekluzek commented on Oct 13, 2025

    @ekluzek
    ContributorAuthor

    I don't know if this is different from what Fang's already using, @lifang0209 or @lawrencepj1 can you help clarify here?

    @wwieder I differenced the input dataset that we had from @lifang0209 and the new CMIP7 version available and there are differences as you can see from cprnc:

    $CPRDIR/cprnc -m /glade/work/fangli/hd/TRENDY/pop-dens_input4MIPs_population_CMIP_PIK-CMIP-1-0-0_gr_1850-2025.nc /glade/campaign/cesm/cesmdata/input4MIPs_raw/input4MIPs/CMIP7/CMIP/PIK/PIK-CMIP-1-0-1/land/yr/pop_dens/gr/v20250928/pop-dens_input4MIPs_population_CMIP_PIK-CMIP-1-0-1_gr_1850-2025.nc
     file 1=/glade/work/fangli/hd/TRENDY/pop-dens_input4MIPs_population_CMIP_PIK-CMIP-1-0-0_gr_1850-2025.nc
     file 2=/glade/campaign/cesm/cesmdata/input4MIPs_raw/input4MIPs/CMIP7/CMIP/PIK/PIK-CMIP-1-0-1/land/yr/pop_dens/gr/v20250928/pop-dens_input4MIPs_population_CMIP_PIK-CMIP-1-0-1_gr_1850-2025.nc
     Attribute creation_date from file1: 2025-06-28T21:26:09Z does not match that found on file2: 2025-09-27T09:21:07Z
     Attribute source_id from file1: PIK-CMIP-1-0-0 does not match that found on file2: PIK-CMIP-1-0-1
     Attribute source_version from file1: 1.0.0 does not match that found on file2: 1.0.1
     Attribute tracking_id from file1: hdl:21.14100/f2bb27c0-dcb9-47bf-b2a0-cae58e33901a does not match that found on file2: hdl:21.14100/9a38ff53-0f2e-4d09-9d85-0539caf43793
     Dimension time matches
     Dimension lat matches
     Dimension lon matches
     Dimension bnds matches
     lat   (lat)
                     360  (   360) (     1)
                     360   0.897500000000000E+02  -0.897500000000000E+02
                     360   0.897500000000000E+02  -0.897500000000000E+02
                     360  (   360) (     1)
              avg abs field values:    4.500000000000000E+01
                                       4.500000000000000E+01
     lat_bnds   (bnds,lat)
                     720  (     0,   361) (     1,     1)
                     720   0.900000000000000E+02  -0.900000000000000E+02
                     720   0.900000000000000E+02  -0.900000000000000E+02
                     720  (     0,   361) (     1,     1)
              avg abs field values:    4.500000000000000E+01
                                       4.500000000000000E+01
     lon   (lon)
                     720  (   720) (     1)
                     720   0.179750000000000E+03  -0.179750000000000E+03
                     720   0.179750000000000E+03  -0.179750000000000E+03
                     720  (   720) (     1)
              avg abs field values:    9.000000000000000E+01
                                       9.000000000000000E+01
     lon_bnds   (bnds,lon)
                    1440  (     0,   721) (     1,     1)
                    1440   0.180000000000000E+03  -0.180000000000000E+03
                    1440   0.180000000000000E+03  -0.180000000000000E+03
                    1440  (     0,   721) (     1,     1)
              avg abs field values:    9.000000000000000E+01
                                       9.000000000000000E+01
     pop_dens   (lon,lat,time)  t_index =      1     1
          53008   259200  (   421,   300,     1) (     1,     1,     1) (   537,   226,     1) (   651,    95,     1)
                  259200   7.325084228515625E+02   0.000000000000000E+00 1.2E+02  6.316968994140625E+02 2.5E-02  2.250654773661154E-07
                  259200   7.334188232421875E+02   0.000000000000000E+00          5.120675048828125E+02          0.000000000000000E+00
                  259200  (   421,   300,     1) (     1,     1,     1)
              avg abs field values:    1.909920454025269E+00    rms diff: 1.1E+00   avg rel diff(npos):  2.5E-02
                                       1.909690260887146E+00                        avg decimal digits(ndif):  1.5 worst:  0.0
     RMS pop_dens                         1.1377E+00            NORMALIZED  5.9570E-01
    
     time   (time)  t_index =      1     1
                       1  (     1) (     1)
                       1   0.181000000000000E+03   0.181000000000000E+03
                       1   0.181000000000000E+03   0.181000000000000E+03
                       1  (     1) (     1)
              avg abs field values:    1.810000000000000E+02
                                       1.810000000000000E+02
     time_bnds   (bnds,time)  t_index =      1     1
    .
    .
    .
     RMS pop_dens                         9.4478E+00            NORMALIZED  7.9308E-01
    
     time   (time)  t_index =    176   176
                       1  (     1) (     1)
                       1   0.640990000000000E+05   0.640990000000000E+05
                       1   0.640990000000000E+05   0.640990000000000E+05
                       1  (     1) (     1)
              avg abs field values:    6.409900000000000E+04
                                       6.409900000000000E+04
     time_bnds   (bnds,time)  t_index =    176   176
                       2  (     2,     1) (     1,     1)
                       2   0.642830000000000E+05   0.639180000000000E+05
                       2   0.642830000000000E+05   0.639180000000000E+05
                       2  (     2,     1) (     1,     1)
              avg abs field values:    6.410050000000000E+04
                                       6.410050000000000E+04
    ************************************************************************************************************************************
      
    SUMMARY of cprnc:
     A total number of    532 fields were compared
              of which    176 had non-zero differences
                   and      0 had differences in fill patterns
                   and      0 had different dimension sizes
                   and      0 had different data types
     A total number of      0 fields could not be analyzed
     A total number of      0 time-varying fields on file 1 were not found on file 2.
     A total number of      0 time-constant fields on file 1 were not found on file 2.
     A total number of      0 time-varying fields on file 2 were not found on file 1.
     A total number of      0 time-constant fields on file 2 were not found on file 1.
      diff_test: the two files seem to be DIFFERENT 
    
  8. moved this from Todo to In Progress in LMWG: Sprint Planning Boardon Oct 13, 2025
  9. lifang0209 commented on Oct 14, 2025

    @lifang0209
    Contributor

    My data were based on pop-dens_input4MIPs_population_CMIP_PIK-CMIP-1-0-0_gr_1850-2025.nc.
    CMIP7 updated the population density dataset from version 1-0-0 to 1-0-1 on Sep 28, 2025.
    It’s better to use the latest version for CESM3 and CLM6. From Erik’s comparison, the two versions show small differences.

  10. added
    nextthis should get some attention in the next week or two. Normally each Thursday SE meeting.
    on Oct 15, 2025
  11. moved this to Ready to start (or start again) in CTSM: Upcoming tagson Oct 15, 2025
  12. moved this from Ready to start (or start again) to In progress - master in CTSM: Upcoming tagson Oct 15, 2025
  13. slevis-lmwg commented on Oct 15, 2025

    @slevis-lmwg
    Contributor

    @wwieder asked that the new hdm/popdens file cover the historical period only. I modified Fang's ncl script and generated this file:
    /glade/work/slevis/git_cmip7/CMIP7_inputdata_processing/code/pop-density/clmforc.Li_2025_CMIP7_hdm_0.5x0.5_simyr1850-2016_c251015.nc
    The new file ends in 2016 because it uses a preexisting file as its template. I am not fluent in ncl and my alternate attempts to make a file ending in 2025 failed. The script is here if anyone would like to help:
    /glade/work/slevis/git_cmip7/CMIP7_inputdata_processing/code/pop-density/popden.ncl

    This is how I modified the file:

    -f=addfile("/glade/campaign/cesm/cesmdata/cseg/inputdata/lnd/clm2/firedata/clmforc.Li_2018_SSP3_CMIP6_hdm_0.5x0.5_AVHRR_simyr1850-2100_c181205.nc","r")
    +f=addfile("/glade/campaign/cesm/cesmdata/cseg/inputdata/lnd/clm2/firedata/clmforc.Li_2017_HYDEv3.2_CMIP6_hdm_0.5x0.5_AVHRR_simyr1850-2016_c180202.nc","r")
     hdm=f->hdm
     
    -f1=addfile("/glade/work/fangli/hd/TRENDY/pop-dens_input4MIPs_population_CMIP_PIK-CMIP-1-0-0_gr_1850-2025.nc","r")
    +f1=addfile("/glade/campaign/cesm/cesmdata/input4MIPs_raw/input4MIPs/CMIP7/CMIP/PIK/PIK-CMIP-1-0-1/land/yr/pop_dens/gr/v20250928/pop-dens_input4MIPs_population_CMIP_PIK-CMIP-1-0-1_gr_1850-2025.nc","r")
     hdmn=f1->pop_dens
     
     f2=addfile("/glade/work/fangli/hd/TRENDY/ctl05.clm2.h0.1850-01.nc","r")
    @@ -23,14 +26,14 @@ landf(:,360:719)=landf1(:,0:359)
     do ilat=0,359
      do ilon=0,719
       if(ismissing(landf(ilat,ilon)))then
    -      hdm(0:175,ilat,ilon)=(/hdmn(:,ilat,ilon)/)
    +     hdm(0:166,ilat,ilon)=(/hdmn(0:166,ilat,ilon)/)
       else
    -     hdm(0:175,ilat,ilon)=(/hdmn(:,ilat,ilon)/landf(ilat,ilon)/)
    +     hdm(0:166,ilat,ilon)=(/hdmn(0:166,ilat,ilon)/landf(ilat,ilon)/)
       end if 
  14. slevis-lmwg commented on Oct 15, 2025

    @slevis-lmwg
    Contributor

    I have another option: I will try nco to change the 1850-2100 file to 1850-2025.

  15. slevis-lmwg commented on Oct 15, 2025

    @slevis-lmwg
    Contributor

    That worked.

  16. slevis-lmwg commented on Oct 15, 2025

    @slevis-lmwg
    Contributor

    @wwieder latest file that can now be tested is
    /glade/work/slevis/git_cmip7/CMIP7_inputdata_processing/code/pop-density/clmforc.Li_2025_CMIP7_hdm_0.5x0.5_simyr1850-2025_c251013.nc

  17. removed
    nextthis should get some attention in the next week or two. Normally each Thursday SE meeting.
    on Oct 16, 2025
  18. changed the title [-]Update to official CMIP7 population density data[/-] [+]alpha-ctsm5.4.CMIP7.15.ctsm5.3.079: Update to official CMIP7 population density data[/+] on Oct 17, 2025
  19. changed the title [-]alpha-ctsm5.4.CMIP7.15.ctsm5.3.079: Update to official CMIP7 population density data[/-] [+]ctsm5.3.081: Update to official CMIP7 population density data[/+] on Oct 20, 2025
  20. changed the title [-]ctsm5.3.081: Update to official CMIP7 population density data[/-] [+]ctsm5.3.082: Update to official CMIP7 population density data[/+] on Oct 20, 2025
  21. moved this from In progress - master to Done (non release/external) in CTSM: Upcoming tagson Oct 24, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

enhancementnew capability or improved behavior of existing capabilitynon-b4bChanges answers (incl. adding tests)

Projects

Relationships

None yet

Development

No branches or pull requests

Issue actions