Add documentation and other things

.github/workflows/CI.yml

@@ -39,3 +39,10 @@ jobs:
           files: lcov.info
         env:
           CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
+      - name: Archive production artifacts
+        uses: actions/upload-artifact@v4
+        with:
+          name: flamegraphs
+          path: |
+            test/flame.html
+            test/allocs.html

.github/workflows/Invalidations.yml

@@ -0,0 +1,40 @@
+name: Invalidations
+
+on:
+  pull_request:
+
+concurrency:
+  # Skip intermediate builds: always.
+  # Cancel intermediate builds: always.
+  group: ${{ github.workflow }}-${{ github.ref }}
+  cancel-in-progress: true
+
+jobs:
+  evaluate:
+    # Only run on PRs to the default branch.
+    # In the PR trigger above branches can be specified only explicitly whereas this check should work for master, main, or any other default branch
+    if: github.base_ref == github.event.repository.default_branch
+    runs-on: ubuntu-latest
+    steps:
+    - uses: julia-actions/setup-julia@v1
+      with:
+        version: '1'
+    - uses: actions/checkout@v4
+    - uses: julia-actions/julia-buildpkg@v1
+    - uses: julia-actions/julia-invalidations@v1
+      id: invs_pr
+
+    - uses: actions/checkout@v4
+      with:
+        ref: ${{ github.event.repository.default_branch }}
+    - uses: julia-actions/julia-buildpkg@v1
+    - uses: julia-actions/julia-invalidations@v1
+      id: invs_default
+
+    - name: Report invalidation counts
+      run: |
+        echo "Invalidations on default branch: ${{ steps.invs_default.outputs.total }} (${{ steps.invs_default.outputs.deps }} via deps)" >> $GITHUB_STEP_SUMMARY
+        echo "This branch: ${{ steps.invs_pr.outputs.total }} (${{ steps.invs_pr.outputs.deps }} via deps)" >> $GITHUB_STEP_SUMMARY
+    - name: Check if the PR does increase number of invalidations
+      if: steps.invs_pr.outputs.total > steps.invs_default.outputs.total
+      run: exit 1

Manifest.toml

@@ -2,7 +2,7 @@
 
 julia_version = "1.10.2"
 manifest_format = "2.0"
-project_hash = "b5cbbfc2ad5b08767926a393e2ce43ef22d93177"
+project_hash = "24609dacff2a805e1985ee1c50ed6e4a693f4a3e"
 
 [[deps.ADTypes]]
 git-tree-sha1 = "016833eb52ba2d6bea9fcb50ca295980e728ee24"
@@ -1140,6 +1140,11 @@ git-tree-sha1 = "6cc9d682755680e0f0be87c56392b7651efc2c7b"
 uuid = "9602ed7d-8fef-5bc8-8597-8f21381861e8"
 version = "0.1.5"
 
+[[deps.UnrolledUtilities]]
+git-tree-sha1 = "b73f7a7c25a2618c5052c80ed32b07e471cc6cb0"
+uuid = "0fe1646c-419e-43be-ac14-22321958931b"
+version = "0.1.2"
+
 [[deps.UnsafeAtomics]]
 git-tree-sha1 = "6331ac3440856ea1988316b46045303bef658278"
 uuid = "013be700-e6cd-48c3-b4a1-df204f14c38f"

Project.toml

@@ -1,36 +1,44 @@
 name = "ClimaDiagnostics"
 uuid = "1ecacbb8-0713-4841-9a07-eb5aa8a2d53f"
 authors = ["Gabriele Bozzola <gbozzola@caltech.edu>"]
-version = "0.0.3"
+version = "0.0.4"
 
 [deps]
 Accessors = "7d9f7c33-5ae7-4f3b-8dc6-eff91059b697"
 ClimaComms = "3a4d1b5c-c61d-41fd-a00a-5873ba7a1b0d"
 ClimaCore = "d414da3d-4745-48bb-8d80-42e94e092884"
+Dates = "ade2ca70-3891-5945-98fb-dc099432e06a"
 NCDatasets = "85f8d34a-cbdd-5861-8df4-14fed0d494ab"
 SciMLBase = "0bca4576-84f4-4d90-8ffe-ffa030f20462"
+UnrolledUtilities = "0fe1646c-419e-43be-ac14-22321958931b"
 
 [compat]
 Accessors = "0.1"
 Aqua = "0.8"
 ClimaComms = "0.5"
 ClimaCore = "0.13"
 ClimaTimeSteppers = "0.7"
+Dates = "1"
 Documenter = "1"
 JuliaFormatter = "1"
 NCDatasets = "0.13, 0.14"
+Profile = "1"
+ProfileCanvas = "0.1"
 SafeTestsets = "0.1"
 SciMLBase = "1, 2"
 Test = "1"
+UnrolledUtilities = "0.1"
 julia = "1.9"
 
 [extras]
 Aqua = "4c88cf16-eb10-579e-8560-4a9242c79595"
 ClimaTimeSteppers = "595c0a79-7f3d-439a-bc5a-b232dc3bde79"
 Documenter = "e30172f5-a6a5-5a46-863b-614d45cd2de4"
 JuliaFormatter = "98e50ef6-434e-11e9-1051-2b60c6c9e899"
+Profile = "9abbd945-dff8-562f-b5e8-e1ebf5ef1b79"
+ProfileCanvas = "efd6af41-a80b-495e-886c-e51b0c7d77a3"
 SafeTestsets = "1bc83da4-3b8d-516f-aca4-4fe02f6d838f"
 Test = "8dfed614-e22c-5e08-85e1-65c5234f0b40"
 
 [targets]
-test = ["Aqua", "ClimaTimeSteppers", "Documenter", "JuliaFormatter", "SafeTestsets", "Test"]
+test = ["Aqua", "ClimaTimeSteppers", "Documenter", "JuliaFormatter", "Profile", "ProfileCanvas", "SafeTestsets", "Test"]

docs/Project.toml

@@ -3,5 +3,5 @@ ClimaDiagnostics = "1ecacbb8-0713-4841-9a07-eb5aa8a2d53f"
 Documenter = "e30172f5-a6a5-5a46-863b-614d45cd2de4"
 
 [compat]
-ClimaDiagnostics = "0.0.1"
+ClimaDiagnostics = "0.0.4"
 Documenter = "0.27"

docs/src/api.md

@@ -3,8 +3,7 @@
 ## `ClimaDiagnostics`
 
 ```@docs
-ClimaDiagnostics.DiagnosticsHandler
-ClimaDiagnostics.DiagnosticsCallback
+ClimaDiagnostics.IntegratorWithDiagnostics
 ```
 
 ## `Callbacks`

docs/src/developer_guide.md

@@ -16,38 +16,23 @@ There are three components needed to add support for `ClimaDiagnostics.jl` in yo
 ## An example for steps 2. and 3.
 
 Let us assume that `scheduled_diagnostics` is the list of `ScheduledDiagnostic`s
-obtained from step 1. (more on this later), `Y` is the simulation initial state,
-`p`, the cache, `t0` the initial time, and `dt` the timestep.
+obtained from step 1. (more on this later), and `integrator` a `SciML`
+integrator.
 
-Schematically, what we need to do is
+All we need to do to add diagnostics is
 ```julia
-import ClimaDiagnostics: DiagnosticsHandler, DiagnosticsCallback
-
-# Initialize the diagnostics, can be expensive
-diagnostic_handler = ClimaDiagnostics.DiagnosticsHandler(
-        scheduled_diagnostics,
-        Y,
-        p,
-        t0;
-        dt,
-    )
-
-# Prepare the SciML callback
-diag_cb = ClimaDiagnostics.DiagnosticsCallback(diagnostic_handler)
+import ClimaDiagnostics: IntegratorWithDiagnostics
 
-SciMLBase.init(args...; kwargs..., callback = diag_cb)
+integrator = IntegratorWithDiagnostics(integrator, scheduled_diagnostics)
 ```
-with `args` and `kwargs` the argument and keyword argument needed to set up the
-target simulation.
-
-In `DiagnosticsHandler`, `dt` is used exclusively for consistency checks.
-Suppose your timestep is `50s` and you request a variable to be output every
-`70s`, if you pass `dt`, `DiagnosticsHandler` will catch this and error out. If
-you don't pass `dt`, `DiagnosticsHandler` will warn you about that.
+Creating an `IntegratorWithDiagnostics` results in calling all the diagnostics
+once. Therefore, the compile and runtime of this function can be significant if
+you have a large number of diagnostics.
 
-Creating a `DiagnosticsHandler` results in calling all the diagnostics once.
-Therefore, the compile and runtime of this function can be significant if you
-have a large number of diagnostics.
+`IntegratorWithDiagnostics` assumes that the state is in `integrator.u` and the
+cache in `integrator.p`. This assumption can be adjusted with keyword arguments.
+For instance, if the state is in `integrator.Y`, pass the `state_name = :Y`
+argument to `IntegratorWithDiagnostics`.
 
 You can learn about what is happening under the hook in the [Internals](@ref)
 page.
@@ -304,14 +289,7 @@ Allowing users to just call `monthly_average("hus", writer, t_start)`.
 > Note: `ClimaDiagnostics` will probably provided these schedules natively at
 > some point in the future.
 
-### Level 2: Provide defaults
-
-Once you built your database of variables, it is a good idea to provide
-reasonable defaults too.
-
-TODO: Fill this
-
-### Level 3: Provide higher-level interfaces (e.g., YAML)
+### Level 2: Provide higher-level interfaces (e.g., YAML)
 
 Finally, you can set in place that parses user input (e.g., from command line or
 text files) into `ScheduledDiagnostics` using the short names in your database.
@@ -432,7 +410,6 @@ YAML-specified ones.
 ## API
 
 ```@docs
-ClimaDiagnostics.DiagnosticsHandler
-ClimaDiagnostics.DiagnosticsCallback
+ClimaDiagnostics.IntegratorWithDiagnostics
 ```
 

docs/src/index.md

@@ -12,10 +12,12 @@ If you are a user of a package that is already using `ClimaDiagnostics.jl`, you
 can jump to the User guide page.
 
 If you are a developer interested in adding support for `ClimaDiagnostics.jl` in
-your package or learning about the internal design of this package, please read the Developer guide page.
+your package or learning about the internal design of this package, please read
+the Developer guide page.
 
 ## Features
 
+- Define diagnostics as function of the integrator state and the cache;
 - Accumulate diagnostics over period of times with associative binary temporal
   reductions (eg, averages);
 - Allow users to define arbitrary new diagnostics;

docs/src/internals.md

@@ -1,7 +1,9 @@
-# Internals
+# Some notes about the internals of `ClimaDiagnostics`
 
-There are multiple moving parts to this package. In this page, we describe the
-internal design.
+There are multiple moving parts to this package. In this page, we provide some
+notes about the internal design. This page also aims at _clarifying the whys_,
+i.e., explaining why things are the way they are. Learning about that might help
+you extend this package further.
 
 ## The callback system
 
@@ -10,7 +12,9 @@ the end of integration steps. `ClimaDiagnostics` implements its own system to
 manage callbacks. The centerpiece of this is the `CallbackOrchestrator`, a
 master callback that is unconditionally executed at the end of _each_ step.
 `CallbackOrchestrator` loops over each registered callback function, executing
-the ones for which the trigger condition is met.
+the ones for which the trigger condition is met. We implement our callback
+system because it gives flexibility to do what we want and because the `SciML`
+ecosystem can be unnecessarily complex for our use case.
 
 The callbacks registered with `CallbackOrchestrator` are [`Callback`](@ref)
 objects (in the [`Callbacks`](@ref) module). `Callback`s are simple `struct`
@@ -29,7 +33,8 @@ struct Callback
     schedule_func::SCHEDULE
 end
 ```
-At the end of each step `CallbackOrchestrator` calls `schedules_func`, when it returns true, it calls `callback_func` too.
+At the end of each step `CallbackOrchestrator` calls `schedules_func`, when it
+returns true, it calls `callback_func` too.
 
 Most often, `schedules_func` are not simple functions, but callable objects
 (subtypes of `AbstractSchedule`). There are two reasons for this:
@@ -42,7 +47,14 @@ An internal detail that is not there is related to names. We define a method for
 ```julia
 Base.show(io::IO, schedule::AbstractSchedule) =  print(io, short_name(schedule))
 ```
-This allows us to set names of `ScheduledDiagnostics` with `"$schedule_func"` in both the case `schedule_func` is a normal function, or an `AbstractSchedule`.
+This allows us to set names of `ScheduledDiagnostics` with `"$schedule_func"` in
+both the case `schedule_func` is a normal function, or an `AbstractSchedule`.
+
+> We might want to move the Callback system to ClimaUtilities if it grows
+> enough.
+
+Each scheduled diagnostic is assigned two callbacks: one for compute and one for
+output.
 
 ## Accumulation
 
@@ -57,14 +69,11 @@ Accumulation is accomplished by the `accumulate!` function. All this function
 does is applying the binary `reduction_time_func` to the previous accumulated
 value and the newly computed one and store the output to the accumulator.
 
-
 After an accumulated variable is output, the accumulator is reset to its natural
-state. This is achieved with the `reset_accumulator!` function.
-
-However, we have to fill the space with something
-that does not affect the reduction. This, by definition, is the identity of the
-operation. The identity of the operation `+` is `0` because `x + 0 = x` for
-every `x`.
+state. This is achieved with the `reset_accumulator!` function. However, we have
+to fill the space with something that does not affect the reduction. This, by
+definition, is the identity of the operation. The identity of the operation `+`
+is `0` because `x + 0 = x` for every `x`.
 
 We have to know the identity for every operation we want to support. Of course,
 users are welcome to define their own by adding new methods to
@@ -78,9 +87,20 @@ end
 ```
 (Or add this to the `reduction_identities.jl` file.)
 
-
-## Add discussion on possible alternative designs
-
-The current design has at least one limitation:
-
-All the diagnostics have to be initialized at the very beginning. 
+### On the design of the `DiagnosticsHandler`
+
+There are two possible choices for accumulation of variables: each scheduled
+diagnostic can carry its accumulator and counters, or all the accumulators and
+counters are managed by a single central handler. `ClimaDiagnostics` implements
+this second approach. The author of this package has not decided whether this is
+a good idea or not. On one side, this allows us to have a concretely typed and
+well defined `DiagnosticsHandler` struct. On the other side, it forces us to
+initialize all the diagnostics at the very beginning of the simulation. It might
+be worth exploring the alternative design where the `ScheduledDiagnostics` get
+their storage space the first time they are called.
+
+Given this restriction, the main entry point for `ClimaDiagnostics` is the
+`IntegratorWithDiagnostics` function. This function is a little dissatisfying
+because it creates a new integrator obtained by copying all the fields of the
+old one and adding the diagnostics (with
+[`Accessors`](https://github.com/JuliaObjects/Accessors.jl)).