Update timerange and reference dataset for CMIP7 cloud radiative effects diagnostic by bouweandela · Pull Request #683 · Climate-REF/climate-ref

bouweandela · 2026-05-20T11:05:01Z

Description

Test the recipe from ESMValGroup/ESMValTool#4461

Checklist

Please confirm that this pull request has done the following:

Tests added
Documentation added (where applicable)
Changelog item added to changelog/

bouweandela · 2026-05-20T11:13:53Z

@lewisjared When I run the command

ref test-cases run --provider esmvaltool --diagnostic cloud-radiative-effects --test-case cmip7 --fetch --clean --force-regen

with this branch it ends with

...
  File "/home/bandela/src/Climate-REF/climate-ref/packages/climate-ref/src/climate_ref/cli/test_cases.py", line 92, in _solve_test_case
    executions = list(solve_executions(data_catalog, diagnostic, diagnostic.provider))
                      │                │             │           │          └ <property object at 0x7b48b7862660>
                      │                │             │           └ CloudRadiativeEffects(name='Climatologies and zonal mean profiles of cloud radiative effects')
                      │                │             └ CloudRadiativeEffects(name='Climatologies and zonal mean profiles of cloud radiative effects')
                      │                └ {<SourceDatasetType.CMIP7: 'cmip7'>:   activity_id institution_id source_id experiment_id variant_label  ...             time...
                      └ <function solve_executions at 0x7b48b78d74c0>

  File "/home/bandela/src/Climate-REF/climate-ref/packages/climate-ref/src/climate_ref/solver.py", line 298, in solve_executions
    raise InvalidDiagnosticException(
          └ <class 'climate_ref_core.exceptions.InvalidDiagnosticException'>

climate_ref_core.exceptions.InvalidDiagnosticException: Invalid diagnostic: 'CloudRadiativeEffects(name='Climatologies and zonal mean profiles of cloud radiative effects')'
 No data catalog matches any of the diagnostic's data requirements. Available source types: SourceDatasetType.CMIP7, SourceDatasetType.obs4MIPs

Any ideas on what I'm doing wrong here?

codecov · 2026-05-20T11:28:02Z

Codecov Report

✅ All modified and coverable lines are covered by tests.

Flag	Coverage Δ
core	`92.43% <ø> (+0.01%)`	⬆️
providers	`87.12% <100.00%> (+0.14%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
..._esmvaltool/diagnostics/cloud_radiative_effects.py	`100.00% <100.00%> (ø)`
...ef-esmvaltool/src/climate_ref_esmvaltool/recipe.py	`87.94% <100.00%> (+1.12%)`	⬆️

... and 6 files with indirect coverage changes

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

lewisjared · 2026-05-20T11:49:50Z

Running with ref --verbose suggests that there isn't any overlap

2026-05-20 21:47:14.102 +10:00 | DEBUG    | climate_ref_core.constraints - Constraint RequireTimerange(group_by=('instance_id',), start=PartialDateTime(year=2001, month=1), end=PartialDateTime(year=2021, month=12)) not satisfied because end time 2014-12-16 12:00:00 is before required end time for /Users/jared/Library/Caches/climate_ref/cmip7-converted/MIP-DRS7/CMIP7/CMIP/CCCma/CanESM5/historical/r1i1p1f1/glb/mon/rlut/tavg-u-hxy-u/gn/20190429/rlut_tavg-u-hxy-u_mon_glb_gn_CanESM5_historical_r1i1p1f1_185001-201412.nc
2026-05-20 21:47:14.102 +10:00 | DEBUG    | climate_ref_core.constraints - Constraint RequireTimerange(group_by=('instance_id',), start=PartialDateTime(year=2001, month=1), end=PartialDateTime(year=2021, month=12)) not satisfied because end time 2014-12-16 12:00:00 is before required end time for /Users/jared/Library/Caches/climate_ref/cmip7-converted/MIP-DRS7/CMIP7/CMIP/CCCma/CanESM5/historical/r1i1p1f1/glb/mon/rlutcs/tavg-u-hxy-u/gn/20190429/rlutcs_tavg-u-hxy-u_mon_glb_gn_CanESM5_historical_r1i1p1f1_185001-201412.nc
2026-05-20 21:47:14.103 +10:00 | DEBUG    | climate_ref_core.constraints - Constraint RequireTimerange(group_by=('instance_id',), start=PartialDateTime(year=2001, month=1), end=PartialDateTime(year=2021, month=12)) not satisfied because end time 2014-12-16 12:00:00 is before required end time for /Users/jared/Library/Caches/climate_ref/cmip7-converted/MIP-DRS7/CMIP7/CMIP/CCCma/CanESM5/historical/r1i1p1f1/glb/mon/rsut/tavg-u-hxy-u/gn/20190429/rsut_tavg-u-hxy-u_mon_glb_gn_CanESM5_historical_r1i1p1f1_185001-201412.nc
2026-05-20 21:47:14.103 +10:00 | DEBUG    | climate_ref_core.constraints - Constraint RequireTimerange(group_by=('instance_id',), start=PartialDateTime(year=2001, month=1), end=PartialDateTime(year=2021, month=12)) not satisfied because end time 2014-12-16 12:00:00 is before required end time for /Users/jared/Library/Caches/climate_ref/cmip7-converted/MIP-DRS7/CMIP7/CMIP/CCCma/CanESM5/historical/r1i1p1f1/glb/mon/rsutcs/tavg-u-hxy-u/gn/20190429/rsutcs_tavg-u-hxy-u_mon_glb_gn_CanESM5_historical_r1i1p1f1_185001-201412.nc
2026-05-20 21:47:14.103 +10:00 | DEBUG    | climate_ref_core.constraints - Constraint RequireTimerange(group_by=('instance_id',), start=PartialDateTime(year=2001, month=1), end=PartialDateTime(year=2021, month=12)) not satisfied for 4 rows

A downside of having no real test data?

coderabbitai · 2026-06-30T01:27:47Z

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: cd087878-eb26-40ab-a611-c8934ff6fb1a

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests
Commit unit tests in branch test-diagnostic-cre-cmip7

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands.}

Resolves the `recipe.py` conflict by taking main's `_ESMVALTOOL_COMMIT`. Main has since converged on 12cb5309, and `ref/recipe_ref_cre_cmip7.yml` now exists at that commit with the sha256 this branch already records in `recipes.txt`, so the branch no longer needs its own pin.

Reworks the diagnostic so both source types share `ref/recipe_ref_cre_cmip7.yml`. - Rewrites the recipe timerange per source type in `update_recipe`: 2002-2014 for CMIP6 and 2002-2021 for CMIP7. The base recipe hardcodes the CMIP7 window, and CMIP6 `historical` ends in 2014. - Replaces the three native references (CERES-EBAF Ed4.2, ESACCI-CLOUD, ISCCP-FH) with the obs4MIPs `CERES-EBAF-4-2-1` reference, declared as a data requirement for both source types and added to the ESGF fetch script. - Fabricates CMIP7 coverage through 2021 in the test case via `extend_historical_to`, because CMIP7 `historical` stops in 2014. - Reverts the ESMValCore pin to main's 2cff15ca. The branch had moved it forward, which the new recipe does not need and which applies to every diagnostic. - Bumps `CloudRadiativeEffects.version` to 2, so both baselines need re-minting.

CMIP7 replaces the MIP table with a realm, so `rewrite_mip_for_cmip7` rewrites `mip: Amon` to `mip: atmos` on each variable. Reference datasets are still CMOR-ised the CMIP6 way and are checked against a table, so the pass already pinned the original table onto a variable's own reference datasets. It did not reach references declared against the diagnostic as a whole, which inherited the realm instead. obs4MIPs names no table `atmos`, so the CMOR check failed with an empty frequency. Hoists the pinning so it covers both levels. A diagnostic-level reference applies to every variable, so it is only pinned when the variables agree on one table. This is what broke the cloud radiative effects CMIP7 mint. It also pins `AERmon` onto the two ozone diagnostic-level references, matching what ozone's own variable-level references already carry.

bouweandela added 2 commits May 20, 2026 13:03

Test CMIP7 cloud radiative effects diagnostic

7bec064

Add obs4MIPs data request

21ac2fc

Update timerange and expected observational datasets

e7308d5

bouweandela mentioned this pull request May 20, 2026

REF recipe CRE with CMIP7 data ESMValGroup/ESMValTool#4461

Merged

16 tasks

bouweandela added 3 commits May 28, 2026 16:02

Merge branch 'main' into test-diagnostic-cre-cmip7

ef7e427

Update to use latest recipe

0f1a420

Update regression test data

fc0f27c

bouweandela commented May 29, 2026

View reviewed changes

Comment thread ...ges/climate-ref-esmvaltool/src/climate_ref_esmvaltool/diagnostics/cloud_radiative_effects.py Outdated

bouweandela added 2 commits May 29, 2026 09:50

Update reference dataset name

13b772c

Add changelog

b33686c

bouweandela changed the title ~~Test CMIP7 cloud radiative effects diagnostic~~ Update timerange and reference dataset for CMIP7 cloud radiative effects diagnostic May 29, 2026

Merge branch 'main' into test-diagnostic-cre-cmip7

2e0b818

lewisjared mentioned this pull request Jul 10, 2026

Diagnostic cloud-scatterplots-cli-ta expects mip=realm for OBS #788

Open

lewisjared had a problem deploying to native-baselines July 10, 2026 03:37 — with GitHub Actions Error

lewisjared force-pushed the test-diagnostic-cre-cmip7 branch from 8e076ac to 31e4d13 Compare July 10, 2026 03:59

lewisjared temporarily deployed to native-baselines July 10, 2026 03:59 — with GitHub Actions Inactive

lewisjared had a problem deploying to native-baselines July 10, 2026 04:41 — with GitHub Actions Failure

lewisjared deployed to native-baselines July 10, 2026 05:52 — with GitHub Actions Active

chore(regression): mint native baselines for esmvaltool

a3cdf33

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Update timerange and reference dataset for CMIP7 cloud radiative effects diagnostic#683

Update timerange and reference dataset for CMIP7 cloud radiative effects diagnostic#683
bouweandela wants to merge 13 commits into
mainfrom
test-diagnostic-cre-cmip7

bouweandela commented May 20, 2026

Uh oh!

bouweandela commented May 20, 2026

Uh oh!

codecov Bot commented May 20, 2026 •

edited

Loading

Uh oh!

lewisjared commented May 20, 2026 •

edited

Loading

Uh oh!

Uh oh!

coderabbitai Bot commented Jun 30, 2026 •

edited

Loading

Review skipped

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

bouweandela commented May 20, 2026

Description

Checklist

Uh oh!

bouweandela commented May 20, 2026

Uh oh!

codecov Bot commented May 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

lewisjared commented May 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

coderabbitai Bot commented Jun 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Review skipped

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

codecov Bot commented May 20, 2026 •

edited

Loading

lewisjared commented May 20, 2026 •

edited

Loading

coderabbitai Bot commented Jun 30, 2026 •

edited

Loading