O2.3.10 Improve performance of data handling software (data reading, regridding, etc), given quantity of reanalysis data

[juliasloan25]

The Climate Modeling Alliance

Software Design Issue 📜

Purpose

Given the quantity of reanalysis data needed by ClimaLand for global runs using prescribed forcings, we need to ensure our data reading, regridding, etc is efficient. This code will also be useful in the coupling software and atmosphere model.

It's currently unclear how much of an improvement we require, since there are other performance bottlenecks in ClimaLand globally-forced runs besides the data handling, and there is not yet a global SoilCanopy run in place. See CliMA/ClimaLand.jl#586 for more details.

Note that the land-sea mask has not been implemented yet. After that is in use, the data handling software will likely require updates (e.g. handling interpolation at the mask boundary, not reading in data that will be unused over the ocean, etc). To avoid increasing the scope of this SDI, that will be addressed in the land-sea mask work - see CliMA/ClimaLand.jl#488.

Cost/Benefits/Risks

costs: developer time
benefits: improved performance across CliMA repos

People and Personnel

• Lead: @Sbozzolo @juliasloan25
• Collaborators:
• Reviewers:

Components

Inputs

• Data handling functionality introduced in O5.1.5: Generalize ClimaCoupler common functions to all component models and move to ClimaUtilities.jl #39

Results and Deliverables

Run ClimaLand's SoilCanopyModel globally with 1 degree resolution, reading in reanalysis data every 3 hours, and measure data handling taking up < 10% of the total simulation time (may be measured using flame graphs).

SDI Revision Log

CC

@tapios @simonbyrne @cmbengue

performance improvements

Beta Give feedback

1. Control/clean caches #21
   new feature 💡 +1
   
2. More robust handling of creating folders in an distributed filesystem #38
   bug 🐛 +1
   
3. Pass additional arguments to the regridder from higher level constructors #22
   new feature 💡 +1
   
4. Support composing Time and Space VaryingInputs #41
   +0
   
5. Add support for boundary conditions in time interpolation #73
   +0
   
6. Add FileReader.read! #81
   +0