Skip to content

Commit

Permalink
Merge pull request #18 from ESMG/dev
Browse files Browse the repository at this point in the history
Release 0.3.2
  • Loading branch information
jr3cermak authored Nov 24, 2021
2 parents ec8da9f + f179248 commit 6764e65
Show file tree
Hide file tree
Showing 42 changed files with 4,628 additions and 615 deletions.
483 changes: 195 additions & 288 deletions conda/gridTools_export-linux-64-RTD.yml

Large diffs are not rendered by default.

397 changes: 323 additions & 74 deletions docs/development/CHANGELOG.md

Large diffs are not rendered by default.

12 changes: 8 additions & 4 deletions docs/development/DEPLOY.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ and the DEPLOYMENT TEMPLATE header above.
- [ ] Ensure `binder/environment.yml` is in sync
with `conda/gridTools_export-linux-64.yml`
- [ ] Ensure release/version is properly updated in `gridtools/__init__.py`
- [ ] Modify any test CI Github Actions
- [ ] Modify any test CI GitHub Actions
- [ ] Update any tests performed by pytest
- [ ] Update TODOs.md, archiving completed TODOs and milestones
- [ ] Add contributors in their own section below for contributions via the pull request or related issues
Expand All @@ -43,14 +43,18 @@ and the DEPLOYMENT TEMPLATE header above.
- [ ] Merge "Release x.y.z" to main
- [ ] Reverify mybinder.org operation
- [ ] Ensure CI/Actions continue to pass (requires manual request)
- [ ] Ensure MDs on github renders correctly
- [ ] Add and commit a tag with x.y.z
- [ ] Check CI/Actions Artifacts for proper results
- [ ] Ensure MDs on GitHub renders correctly
- [ ] Add and commit a tag with x.y.z; add commit hash to archive/x.y.z.md
- [ ] Ensure Read the Docs renders correctly for `stable` (triggered after new tag is pushed)
- [ ] Run through a Release on the github site
- [ ] Run through a Release on the GitHub site
- [ ] Place a release notice on the MOM6 forum

<!-- Delete sections with no content -->
# General Release Notes

# Bug Fixes

# API Changes

# Contributors
7 changes: 6 additions & 1 deletion docs/development/ImportantReferences.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,15 +35,18 @@
* https://github.com/pydata/xarray/issues/2899
* https://github.com/rabernat/intro_to_physical_oceanography
* https://github.com/SciTools/cartopy/issues/787
* https://gitlab.com/remikz/nccmp
* https://guides.github.com/features/mastering-markdown/
* http://holoviews.org/user_guide/Dashboards.html
* https://math.libretexts.org/Bookshelves/Calculus/Book%3A_Calculus_(OpenStax)/12%3A_Vectors_in_Space/12.7%3A_Cylindrical_and_Spherical_Coordinates#:~:text=In%20the%20Cartesian%20coordinate%20system%2C%20the%20location%20of%20a%20point,an%20angle%20measure%20(%CE%B8).
* http://mathworld.wolfram.com
* https://matplotlib.org/stable/gallery/images_contours_and_fields/image_masked.html
* https://matplotlib.org/stable/gallery/images_contours_and_fields/pcolormesh_levels.html#sphx-glr-gallery-images-contours-and-fields-pcolormesh-levels-py
* https://matplotlib.org/stable/gallery/images_contours_and_fields/pcolormesh_grids.html#sphx-glr-gallery-images-contours-and-fields-pcolormesh-grids-py
* https://matplotlib.org/stable/tutorials/colors/colormaps.html
* https://matplotlib.org/stable/tutorials/colors/colorbar_only.html#sphx-glr-tutorials-colors-colorbar-only-py
* https://matplotlib.org/stable/tutorials/colors/colormaps.html
* https://matplotlib.org/stable/tutorials/colors/colormap-manipulation.html
* https://matplotlib.org/stable/tutorials/colors/colors.html
* https://matplotlib.org/stable/users/event_handling.html
* https://matplotlib.org/stable/users/interactive.html
* https://matthewrocklin.com/blog/work/2017/07/03/scaling
Expand All @@ -67,6 +70,7 @@
* https://scitools.org.uk/cartopy/docs/latest/index.html
* https://scitools.org.uk/cartopy/docs/latest/crs/projections.html
* https://scitools.org.uk/cartopy/docs/latest/gallery/effects_of_the_ellipse.html
* https://scitools.org.uk/cartopy/docs/latest/matplotlib/gridliner.html
* https://softwareengineering.stackexchange.com/questions/329348/classes-vs-modules-in-python
* https://www.sphinx-doc.org/en/master/latex.html#latexsphinxsetup
* https://sphinx-rtd-tutorial.readthedocs.io/en/latest/docstrings.html
Expand All @@ -82,6 +86,7 @@
* https://stackoverflow.com/questions/12575421/convert-a-1d-array-to-a-2d-array-in-numpy
* https://stackoverflow.com/questions/13530489/adding-background-color-of-notes-to-sphinx-generated-pdf-files
* https://stackoverflow.com/questions/15411967/how-can-i-check-if-code-is-executed-in-the-ipython-notebook
* https://stackoverflow.com/questions/18717877/prevent-plot-from-showing-in-jupyter-notebook
* https://stackoverflow.com/questions/20180543/how-to-check-version-of-python-modules
* https://stackoverflow.com/questions/20288711/post-install-script-with-python-setuptools
* https://stackoverflow.com/questions/20961287/what-is-pylab
Expand Down
65 changes: 45 additions & 20 deletions docs/development/TODO.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,34 +2,37 @@

## Milestones

- [ ] Release 0.3.2
- [ ] Add dataset subset/preparer function that will satisfy
FRE-NCtools `make_topog`.
- [ ] Release 0.3.x
- [ ] Release 0.x
- [ ] Additional metadata for tile variable: `standard_name`, `tile_spec_version`,
`geometry`, `discretization` and `conformal`.
- [ ] Side by side comparison of bathymetric fields as generated by
current set of tools and FRE-NCtools.
- [ ] Release 0.x
- [ ] Auto extending known grid types to allow flexible calculations.
- [ ] Fix GridUtils.regridTopo() halo effect on outer grid edge
- [ ] Improvements to roughness calculation.
- [ ] Add dataset subset/preparer function that will satisfy
FRE-NCtools `make_topog`.
- [ ] Improvements to roughness calculation
- [ ] Finish port ROMS mask editor as-is to gridtools for MOM6/ROMS grids (pylab)
- [ ] Rewrite/Optimize pylab mask editor
- [ ] Application improvements
- [ ] Boundery condition grid creation and support (OBCs)
- [ ] Save only the points we need instead of the whole supergrid
- [ ] Sponge data preparation
- [ ] Subset existing grids and infrastructure
- [ ] Subsetting
- [ ] Add more flexibility for handling model grids
- [ ] Boundary files
- [ ] Leverage dask (for users that lack access to large memory nodes)
- [ ] More graceful handling of periodic grid edges: +0,+360 vs -180,+180
- [ ] Enhanced grid/plot projection options (non-map based;
e.g. dumbbell; double gyre grids)
- [ ] Enhanced plotting support
- [ ] Allow export of MOM6 grid to ROMS
- [ ] implement ROMS.extend_ROMS_grid()
- [ ] Grid filling options (flooding) (ice9)
- [ ] Enable gridtools library to be installable via conda

# BUGS

- [ ] gridtools applications do not start up properly when jupyter is
started with `--ip=0.0.0.0`
- [ ] Investigate reliably of produced grids between platforms
- [ ] A nested dictionary will clobber other nested elements instead
of updating elements. Recode `setPlotParameters` and
Expand All @@ -39,9 +42,18 @@

# TASKS

- [ ] Generic tool to subset a MOM6 grid for a given area for debugging.
- [X] Name this function subsetGrid().
- [ ] Allow selection of offset, spacing and range.
- [ ] Show example of original roughness calculation
- [ ] grid extension and other operations for `GNOMONIC_ED` grid (FRE)
- [ ] gridutils.readGrid() ROMS grid; spurious breakpoint() removed; missing
condition in if statement
- [ ] Add tile metadata `geometry`; move global `grid_version` into tile
variable
- [ ] Create our own conda repository since there is now a name conflict with
at least PyPi.
- [ ] Sponge data preparation
- [ ] Boundary/sponge data preparation
- [ ] Current scripts generate u,v fields on h-points; this needs
to be changed to C-grid u/v-points instead
- [ ] general documentation
Expand All @@ -59,8 +71,6 @@
- [ ] having tilt may not produce conformal grids
- [X] Niki''s example added; but it may not be correct
- [ ] Niki might have solved lat lon tilt?
- [ ] make Stereographic grids
- [ ] user testing
- [ ] grid generation in other projections (tri-polar, etc)
- [ ] on saveGrid():
- [X] convert lon [+0,+360] to [-180,+180]
Expand All @@ -79,7 +89,7 @@
- [ ] MOM6: Obey `MASKING_DEPTH`, `MINIMUM_DEPTH`, `ALLOW_LANDMASK_CHANGES`,
`MAXIMUM_DEPTH`, `TOPO_EDITS_FILE` MOM6/src/initialization parameters
- [ ] show outline of full grid
- [ ] Example 09a has a wierd artifact showing grid boundary
- [ ] Example 09a has a weird artifact showing grid boundary
- [ ] show other underlying fields - topo?
- [ ] ROMS: write edited mask
- [X] MOM6: write edited mask
Expand All @@ -94,10 +104,10 @@
- [ ] MOM6: write edited mask
- [X] ROMS: write edited mask
- [ ] integration of data sources
- [ ] Using xesmf regridder and other tools to create bathymetry and
other forcing and boundary files
- [ ] generic regridder for creating boundary files (OBCs) from data sources
- [ ] xesmf regridder for bathymetry sources
- [X] option to create ocean mask fraction
- [ ] investigate artifact generated in the grid corners
- [ ] option to use source grid as a supergrid for coarsening
- [ ] refactor function arguments into kwargs
- [ ] refactor print statements to use gridtools logging facility
Expand Down Expand Up @@ -127,14 +137,22 @@
- [ ] update any tool metadata that is appropriate for that grid
- [ ] parse and utilize any available proj string; must be a global
or variable attribute
- [ ] Using xesmf regridder and other tools to create bathymetry and
other forcing and boundary files
- [ ] Develop a field "flood" routine similar to pyroms
- [ ] Perform checks for ensureEvenI and ensureEvenJ everywhere.
This applies only to the grid not the supergrid.

# TODO

- [ ] Improve gridutils.getXYDist() to maybe look at other
derivatives to improve accuracy of appending points along
a non-regularly spaced line.
- [X] Incorporate grid extension (extending) code
- [X] initial automatic grid detection code
- [X] spherical coords
- [ ] test northern hemispheric grids
- [ ] test southern hemispheric grids
- [ ] test grids that overlap the equator
- [X] lat/lon coord
- [X] clip grid to specified size
- [ ] Update operational paths graphic
- [ ] Interoperability with FRE-NCtools
- [ ] Generic plotting of figures
Expand Down Expand Up @@ -165,11 +183,12 @@
- [X] Create a message buffer/system for information.
- [ ] Create a way to monitor a log file;
https://discourse.holoviz.org/t/scrollable-log-text-viewer/317
- [ ] log/display github revision of gridtools used by mybinder.org instances
- [ ] log/display github revision of gridtools used by mybinder.org instances/application
- [ ] For now, the gridParameters are always in reference to a center
point in a grid in the future, one may fix a side or point of
the grid and grow out from that point instead of the center.
- [ ] Grid generation application
- [ ] add option to turn on/off gridline labels
- [ ] enable user configurable plot and widget sizes (hardcoded in __init__)
- [ ] enable user to change ellipsoid, R, `x_0` and `y_0` grid and
plot parameters
Expand All @@ -184,7 +203,6 @@
- [ ] Develop additional GridUtils() functions
- [ ] Run `proj -le` and return the names or display the details
- [ ] Populates the ellps field for the application
- [ ] Detect grid types and auto extend grids to deal with data artifacts
- [ ] Create routine to handle periodic conditions; automatically extend
needed points to cover zero degree and dateline issues.
- [ ] Deploy use of self.gridMade (robTest:PR#1)
Expand All @@ -195,13 +213,20 @@
- [ ] a test fails in `test_trunc.py`
- [ ] CI/Actions test harnesses
- [ ] pytest: Setup some simple projection tests: IBCAO, ....
- [ ] pytest: Test other spherical grids not centered over the pole;
add graphics to artifacts for viewing.
- [ ] pytest: Refactor numpypi into structured tests under pytest
- [ ] pytest: allow certain tests to fail if a module is not
available (issue warnings instead)
- [ ] selenium: Testing interactive methods may be harder.

# WISH

- [ ] Show difference fields for python script examples 3 and 8.
- [ ] Add more features to the ice-9 algorithm
- [ ] zEdits
- [ ] Multiple seed points
- [ ] Periodic grid support
- [ ] Update gridTools.yml with some pinned versions to help package resolution
- [ ] Write example program(s)
- [ ] Example 04 : DASK: mkGridsEample4a.ipynb is incomplete
Expand Down
2 changes: 1 addition & 1 deletion docs/development/archive/0.3.1.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# Retired TODO and Milestones

## Release 0.3.1
* https://github.com/ESMG/gridtools/commit/...
* https://github.com/ESMG/gridtools/commit/194d7b2e73e9365799a25977de6e84f5aaeae20c
* August 12, 2021

### Milestones
Expand Down
34 changes: 34 additions & 0 deletions docs/development/archive/0.3.2.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
# Retired TODO and Milestones

## Release 0.3.2
* https://github.com/ESMG/gridtools/commit/...
* November 16, 2021

### Milestones

- [X] Update meshutils.writeLandMask() and meshutils.writeOceanMask() to search for x and y
coordinates if not provided by the supplied variable.
- [X] GridUtils.plotGrid() allow plotting of one or more grid points.
- [X] Grid filling options (flooding; ice-9 algorithm)
- [X] Plotting model grid: include option to show grid points on the grid.
- [X] Fix GridUtils.regridTopo() halo effect on outer grid edge
- [X] option to create ocean mask fraction
- [X] investigate artifact generated in the grid corners; fix by extending grid and clipping result
- [X] Implemented metadata functions: gridutils.updateGridMetadata(); gridutils.checkGridMetadata()
- [X] Add or append to 'history' global variable.
- [X] Auto extending known grid types to allow workarounds for calculations
with artifacts around the edges.
- [X] Add reference for Niki's `ocean_model_grid_generator`!
- [X] Core model grid generic subsetting is available via gridutils.subsetGrid(); does not subset
boundary conditions

### TODOs

- [X] able to make and create stereographic grids
- [X] Develop a field "flood" routine similar to pyroms; ported ice9 routine

### BUGS

- [X] BUG: Fix example7 scripts so they can run independently of each other as examples.
- [X] BUG: `gridtools.generate_regional_spherical_meters` fix spherical grid generation. Resolves #16
- [X] BUG: app: adjust grid center latitude bounds to -90 to +90. Resolves #16.
9 changes: 9 additions & 0 deletions docs/development/netCDF.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
# netCDF

The netCDF kitchen sink tools append to the global history element by
separating the inserted strings by a linefeed (`\n`).

For example:
```
:history = "Sat Nov 6 15:45:29 2021: ncrename -v xx,x land_mask_Example3.nc\nSat Nov 6 15:45:11 2021: ncrename -v x,xx land_mask_Example3.nc" ;
```
70 changes: 70 additions & 0 deletions docs/development/releases/0.3.2.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
# Release 0.3.2

## Previous Release

Tag: 0.3.1 `194d7b2e73e9365799a25977de6e84f5aaeae20c`

# Checklist

- [X] Operating system checks
- [X] `x86_64`: Ubuntu 20.04.2 LTS (64 GB)
- [ ] `x86_64`: Ubuntu 20.04.2 LTS (12 GB VM) **Skipped**
- [X] `aarch64`: Raspberry Pi 4 (8 GB)
- [ ] `x86_64`: triton node (64 GB) **Cluster offline**
- [X] `x86_64`: chinook node (128 GB)
- [X] Verify operation of example notebooks
- [X] Verify operation of example scripts
- [X] Resync environments
- [X] Pip requirements.txt should closely mirror gridTools.yml
- [X] Update any special needs in requirements.txt
- [X] Resync `gridTools_export-linux-64.yml` without pip modules
- [X] Resync `gridTools_export-linux-64-RTD.yml` without nodejs and pip modules
- [X] Ensure `binder/environment.yml` is in sync
with `conda/gridTools_export-linux-64.yml`
- [X] Ensure release/version is properly updated in `gridtools/__init__.py`
- [X] Modify any test CI GitHub Actions
- [X] Update any tests performed by pytest
- [X] Update TODOs.md, archiving completed TODOs and milestones
- [X] Add contributors in their own section below for contributions via the pull request or related issues
- [X] After submission of PR to main
- [X] Review commit as necessary
- [X] Verify CI/Actions pass
- [X] Verify Read the Docs render correctly
- [X] Verify mybinder.org is functional
- [X] Review, update and/or close any issues
- [X] Merge "Release 0.3.2" to main
- [X] Reverify mybinder.org operation
- [X] Ensure CI/Actions continue to pass (requires manual request)
- [X] Check CI/Actions Artifacts for proper results
- [X] Ensure MDs on GitHub renders correctly
- [X] Add and commit a tag with 0.3.2; add commit hash to archive/0.3.2.md
- [X] Ensure Read the Docs renders correctly for `stable` (triggered after new tag is pushed)
- [X] Run through a Release on the GitHub site
- [X] Place a release notice on the MOM6 forum

# General Release Notes

- This release resolves issues #15 and #16 with fixes to spherical grid support and
the ability to extend a model grid for use in routines that generate artifacts on
the boundaries.
- ReadtheDocs resources were moved around such that the conda environment is a
small subset of packages to get around a memory issue. The remaining resources
are loaded via pip.

# Bug Fixes

- Fix example7 scripts so they can run independently of each other as examples.
- `gridtools.generate_regional_spherical_meters` fix spherical grid generation. Resolves #16
- app: adjust grid center latitude bounds to -90 to +90. Resolves #16.

# API Changes

- For some examples, the **i** verticies are now shown in yellow to show orientation of the grid and
to make sure the grid orientation is correct as requested.
- Updates to computeBathymeticRoughness()
- Option FixByOverlapQHGridShift replaced with two boolean arguments: useQHGridShift, useOverlap

# Contributors

- Jessica Garwood, Kate Hedstrom: debugging and confirmation of problems generating MOM6 exchange
grids using xarray or gridtools.
Loading

0 comments on commit 6764e65

Please sign in to comment.