Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Binary file modified docs/_static/examples/grids/cam_se.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/_static/examples/grids/face_node_edge.png

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider slightly larger dots for nodes, and color code a few nodes and edges. E.g., put nodes in blue and the word "node" in blue, and the pointed-to edge in red and the word "edge" in red

Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/_static/examples/grids/iso_grid.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/_static/examples/grids/lat_lon.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/_static/examples/grids/ocean.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/_static/examples/grids/structured_grid.png

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The numbers are extremely low resolution here; if keeping this file consider a slightly higher-res version of it!

Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/_static/examples/grids/unstructured_grid.png

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The description says "you may end up in a hole" but there are no "holes" in this image, just regions with faces and regions without faces. I think of a hole as "an empty region surrounded by non-empty regions". Maybe add a few more differently-shaped faces connecting the bottom right cell to the second-row left cell?

Additionally, maybe talking about this grid would be easier if you label the faces with letters? E.g., "move one step right from face A and you end up on face B, but move another step right from face B and you fall off the grid entirely" or something like that?

Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
4 changes: 3 additions & 1 deletion docs/tutorials.rst
Original file line number Diff line number Diff line change
Expand Up @@ -7,4 +7,6 @@ Tutorials & Videos

Tutorials
---------
* `Project Pythia Unstructured Grid Visualization Cookbook <https://projectpythia.org/unstructured-grid-viz-cookbook>`_
* `Project Pythia Unstructured Grid Visualization Cookbook <https://projectpythia.org/unstructured-grid-viz-cookbook>`_ - A series of guides in the form of Jupyter notebooks. Shows code, examples, visualizations for various uses of ``uxarray``.
* `UXarray Tutorial at ESDS 2024 <https://www.youtube.com/watch?v=sOCMSLZrwQs>`_ - A YouTube video presentation by one of the lead UXarray developers on the motivation of UXarray, its origins, and uses.
* `UXarray Presentation at SciPy 2023 <https://www.youtube.com/watch?v=qwqJeOO8m6A>`_ - A second YouTube video presentation by another one of the lead UXarray developers on UXarray at SciPy 2023.
9 changes: 6 additions & 3 deletions docs/user-guide/data-structures.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -24,8 +24,8 @@
"The core functionality of UXarray revolves around three data structures, which are used for interacting with unstructured grids and the data variables that reside on them.\n",
"\n",
"1. **[`uxarray.Grid`](https://uxarray.readthedocs.io/en/latest/generated/uxarray.UxDataArray.html)**: Stores the grid representation (i.e. coordinates, connectivity information, etc.)\n",
"2. **[`uxarray.UxDataset`](https://uxarray.readthedocs.io/en/latest/api.html#uxdataset)**: One or more data variable that resided on a grid.\n",
"3. **[`uxarray.UxDataArray`](https://uxarray.readthedocs.io/en/latest/api.html#uxdataarray)**: A single data variable that resides on a grid\n"
"2. **[`uxarray.UxDataset`](https://uxarray.readthedocs.io/en/latest/api.html#uxdataset)**: One or more data variables that resided on a grid. Each variable is stored in a UxDataArray.\n",

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

resided --> reside; they still live on the grid.

"3. **[`uxarray.UxDataArray`](https://uxarray.readthedocs.io/en/latest/api.html#uxdataarray)**: A single data variable that resides on a grid.\n"
]
},
{
Expand Down Expand Up @@ -486,7 +486,10 @@
}
},
"outputs": [],
"source": "uxds = ux.open_dataset(grid_path, data_path)\nuxds"
"source": [
"uxds = ux.open_dataset(grid_path, data_path)\n",
"uxds"
]
},
{
"cell_type": "markdown",
Expand Down
4 changes: 2 additions & 2 deletions docs/user-guide/grid-formats.rst
Original file line number Diff line number Diff line change
Expand Up @@ -242,13 +242,13 @@ Coordinates

<style>
.yes-cell {
background-color: green;
background-color: #6aa84f; /* Light green color */

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did not change actual rendering of table, as far as I can tell. Either revert the color changes to this file, or debug and fix so that the new colors are actually used. (I think light green and light red would be an improvement, if you do manage to get them to work!)

color: white;
text-align: center;
}

.no-cell {
background-color: red;
background-color: #e06666; /* Light red color */
color: white;
text-align: center;
}
Expand Down
5 changes: 5 additions & 0 deletions docs/user-guide/terminology.rst
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,11 @@ made up of nodes and edges. Below is an example of an unstructured grid used in

**Node**: A point within a spherical grid, representing the vertices of the elements (such as the corners of triangles or quadrilaterals)

.. image:: ../_static/examples/grids/face_node_edge.png
:width: 300
:align: center
:alt: A visual of the components of a grid

**Edge**: A segment that connects two nodes within a grid.

**Face**: An individual polygon that is defined by nodes connected by edges.
Expand Down
111 changes: 111 additions & 0 deletions docs/user-guide/unstructured-grids.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,111 @@
.. currentmodule:: uxarray
Comment thread
dylannelson marked this conversation as resolved.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All the figures except ocean.png renders too large in this document, please see https://uxarray--1607.org.readthedocs.build/en/1607/user-guide/unstructured-grids.html

Just a clarification if this was led by my previous comment: I only meant ocean.png was rendering too small previously, not the other(s)


================================
Structured vs Unstructured Grids
================================

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For this section and others in the rest of this file, you may want to check out this section from our UXarray Cookbook: https://projectpythia.org/unstructured-grid-viz-cookbook/notebooks/foundations/unstructured-grids/#structured-vs-unstructured-grids since a lot of text there can be reused here. However, it touched code etc. which may not be needed here.


Before diving into unstructured grids, it is helpful to understand the basic differences
between structured and unstructured grids. Unstructured grids differ from structured grids
in how they are designed, navigated, and used in modeling functions.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can avoid some redundant wording here by combining the last two sentences, and also did you mean "model analysis" instead of "models":

"...understand the basic differences between structured and unstructured grids in how they are designed, navigated, and used in models."



Structured Grids - How They're Designed

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider: "Design of Structured Grids" or "Structured Grid Design" to make more concise / make it easier to skim quickly. Similar for the similarly-named unstructured grid heading below.

=======================================

Structured grids are matrix-like in structure. Just as we can navigate a 2D
plane using simple coordinates like (2, 2) to (2, 3), a structured grid can be
navigated predictably because it follows a repeatable pattern.

.. image:: ../_static/examples/grids/structured_grid.png
:width: 600
:align: center
:alt: A structured grid with a regular, matrix-like arrangement of cells

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While it is technically not false/impossible to define cells being centered on a particular (lon,lat) pair, grid cells in structured grids are defined with their boundary longitudes and latitudes as follows:

Image

So, rather than identifying cell centers with lon, lat pairs, maybe draw a diagram similar to the above but with only nine cells, tag the constant lon and lat lines instead of cell centers, and rephrase the text in this section accordingly?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I was able to capture everything here in my last push, let me know if it captures the vision you had. Tried to keep the same styling as before, fairly minimal and as much glance value as possible

Consider the cell bounded by 19°E–20°E and 19°N–20°N. Its four neighboring cells are bounded by:

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Recommendation: use half-degrees for nodes in this example, so that faces are centered at whole-degree values, and then refer to faces by their center (similar to what you had in an earlier version). Then, the text can say something like "Consider the cell centered (20°E, 20°N). Its four neighboring cells are at (north) (20°E, 21°N), (south) (20°E, 19°N), …". As-is, this section is too verbose to parse quickly (I agree with your thoughts on this from our conversation earlier).

I do think it is okay to stick with basically the image format here though, rather than reverting to your older 5-cell image. If you want to improve quick-glance + skimming-the-text value maybe give slight colors to some of the cells to reference in the text as well? E.g. "(north/light blue) (20°E, 21°N), (south/light green) (20°E, 19°N), …"


- 19°E–20°E and 20°N–21°N (north),
- 19°E–20°E and 18°N–19°N (south),
- 20°E–21°E and 19°N–20°N (east), and
- 18°E–19°E and 19°N–20°N (west)

You can know the above is true without looking at any data.

Moving along a structured grid has predictable outcomes and observations.
There are many assumptions you can make about a different cell that is 1 degree East.
It will have a predictable center, number of sides, and predictable neighboring cells
of the same characteristics.

Unstructured Grids - How They're Designed
=========================================

Unstructured grids do not follow a repeated, predictable pattern, and instead
can vary in shape and organization. Because of this, unstructured grids are
built like network graphs.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not completely sure what you mean by "network graphs". Adding a hyperlink (even if it is to a wikipedia page) here to help show what you mean would be sufficient.


.. image:: ../_static/examples/grids/unstructured_grid.png
:width: 600
:align: center
:alt: An unstructured grid made from hexagons

When you move one degree on a grid like the one above, you may end up in a hole,
a face of a different shape, or something entirely unexpected. That's why
the data is traversed by looking at its neighbors from connected edges.

Use Cases
=========

Many kinds of unstructured grids can exist because of these features.
Comment thread
dylannelson marked this conversation as resolved.

Some grids use hexagons rather than a rectilinear structure:

.. image:: ../_static/examples/grids/iso_grid.png
:width: 600
:align: center
:alt: An unstructured grid built from hexagonal cells

Others are allowed to have holes that cut out regions that aren't of interest in order to improve efficiency:

.. image:: ../_static/examples/grids/ocean.png
:width: 600
:align: center
:alt: An unstructured ocean grid with holes cut out over land regions
Comment thread
dylannelson marked this conversation as resolved.

Here a grid changes the size and shape of its cells to provide higher
resolution in a region of interest and coarser resolution elsewhere:

.. image:: ../_static/examples/grids/cam_se.png
:width: 600
:align: center
:alt: A CAM-SE grid with variable-resolution cells refined over a region of interest

.. note::

More info about each of the models currently supported can be seen on the `Supported Models & Grid Formats <https://uxarray.readthedocs.io/en/latest/user-guide/grid-formats.html>`__ page

Specialization & Efficiency
===========================

Structured grids face scalability limits due to several reasons such as global refinement, polar singularity,

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hyperlinks to pages where these reasons for scalability limits are defined or discussed would be very helpful. Is "global refinement" a reason for scalability limits? I'm not really sure what that means here. A well-chosen hyperlink (even if it is to a wikipedia page) would likely be enough to clarify.

Another option is to give an example of each. The "For instance, as the grid resolution scales…" example below is helpful. Though, I can't tell exactly which reasons this is intended to correspond to - is it global refinement, or polar singularity, or both?

land-ocean boundary fitting, etc. For instance, as the grid resolution scales, a global lat-lon rectilinear grid
creates too many cells near the poles, wasting massive storage/memory and compute resources for both model generation
and post-processing & analysis.

Models focused on the equator, like tropical cyclone models, will have a much higher density of cells near the poles
where they don't need data from, and a much lower density of cells near the equator, due to how the lat-lon grid scales.

On the other hand, models focused on the polar region tend to want more uniform shapes in this region, like the icosahedral

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestion: replace "want" with "perform better", or add some other metric for why it is better to have uniform shapes here. E.g., "models focused on the polar region tend to want more uniform shapes in this region in order to provide equal weights to all data points". I think the language I suggested in this comment could be improved, but hopefully this gives the sense of what I mean

grid, which is an unstructured grid that contains hexagons of similar size uniformly across the whole grid.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestion: "…like the icosahedral grid above, which contains hexagons…"


Some of the value of unstructured grids comes from researchers being able to build or adopt
models that are tailored to their use case. For example, a researcher studying
measurements made across the planet's many oceans does not want to consider
cells on land. They can use a model that has no cells on land,
which creates large holes that structured grids could not run calculations on.

In a traditional lat/lon grid, ~30% of the grid cells in the previous example would be

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestion: add "oceans-only" or some other concise phrase to describe the example, not just "previous example". I was skimming at first and it took me a bit too long to understand what this meant originally.

unused, meaning the data footprint could be roughly 30% smaller. Many
calculations will likely require noticeably fewer resources, and less time will be required to

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestion: "…less time will be required to compute" --> "… reducing compute time and dataset sizes."

compute. Many nodes, edges, and faces are eliminated by the removal of the
land, and even the cells over the sea have fewer connected faces, nodes,

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"the cells over the sea have fewer connected faces, nodes, and edges as a result" is a good reason to remove the land, but I was confused at first because the rest of this page seems to be all about comparing structured & unstructured. This statement is not related to structured vs unstructured. I think it would be okay to remove; the previous sentences already clarify some structured vs unstructured benefits here.

and edges as a result. These efficiency improvements matter a lot in
geospatial models, which tend to already be very resource intensive.
10 changes: 7 additions & 3 deletions docs/userguide.rst
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,9 @@ Introductory Guides

These user guides provide the necessary background information for understanding concepts in UXarray.

`Structured vs Unstructured Grids <user-guide/unstructured-grids.rst>`_
Introduction to the concept of unstructured grids

`Terminology <user-guide/terminology.rst>`_
Core concepts and terminology for working with unstructured grids

Expand All @@ -26,13 +29,13 @@ These user guides provide the necessary background information for understanding
Core Guides
------------

These user guides provide detailed explanations of the core functionality in UXarray.
These user guides provide detailed explanations of the core functionality in UXarray

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Restore the period at end of sentence


`Data Structures <user-guide/data-structures.ipynb>`_
Core data structures for working with unstructured grid and data files

`Custom Grid Topology <user-guide/custom-grid.ipynb>`_
Create a Grid from custom Grid topology and convert existing Xarray data structures to UXarray.
Create a Grid from custom Grid topology and convert existing Xarray data structures to UXarray

`Loading Data using Dask <user-guide/parallel-load-ux-with-dask.ipynb>`_
Read data with chunking and/or in parallel
Expand All @@ -50,7 +53,7 @@ These user guides provide detailed explanations of the core functionality in UXa
Select specific regions of a grid

`Spatial Hashing <user-guide/spatial-hashing.ipynb>`_
Use spatial hashing to locate the faces a list of points reside in.
Use spatial hashing to locate the faces a list of points reside in

`Cross-Sections <user-guide/cross-sections.ipynb>`_
Select cross-sections of a grid
Expand Down Expand Up @@ -109,6 +112,7 @@ These user guides provide additional details about specific features in UXarray.
.. toctree::
:hidden:

user-guide/unstructured-grids.rst
user-guide/terminology.rst
user-guide/representation.rst
user-guide/grid-formats.rst
Expand Down