ESMPy uses a
Field object to represent data variables
built on an underlying spatial discretization, which is represented by a
Fields is accomplished with the
Regrid class. All of these classes are explained in
more detail in the sections provided by the links in the following table.
A manager class to initialize and finalize ESMF
A class to represent a logically rectangular grid
A class to represent an unstructured grid
A class to represent observational data as a collection of disconnected points
The regridding utility
The from file regridding utility
ESMPy follows the ESMF convention of using “named constants” to represent the available options for parameters that expect a variety of specific inputs. The following table lists the available named constants and provides links to pages that further explain the available values.
Specify the coordinate system of a
Specify the extrapolation method
Specify the format of a data file
Specify the mode of a data file
Specify a mask or area item on a
Specify the type of line that connects two points on a sphere
Specify how much logging should be done
Specify the type of the
Specify a nodal or elemental
Specify the type of normalization to use for conservative regridding weights
Specify the type of connection that appears at the poles of the
Specify which type of artificial pole to construct on the source
Specify various regions in the data layout of
Specify which interpolation method to use regridding
Specify the position for data in a
Specify the type and kind of data
Specify which action to take with respect to unmapped destination points
Manager is used by ESMPy to simplify a
number of low-level calls used by the underlying ESMF framework to allocate
resources, enable logging, and control garbage collection.
The ESMF Virtual Machine (VM)
is created at the beginning of each ESMPy execution, and contains information
about the topology and characteristics of the underlying computer. The VM
allocates computational resources in the form of
Persistent Execution Threads, or PETs. These are equivalent to operating
system threads with a lifetime of at least that of the ESMPy execution. In the
simplest, and most common case, a PET is equivalent to an MPI process. The
number of PETs and the current PET can be queried from the
mg = ESMF.Manager() pet_count = mg.pet_count() local_pet = mg.local_pet()
Refer to the VM Class of the ESMF Reference Manual for more information.
Manager is also used to enable logging:
mg = ESMF.Manager(debug=True) local_pet = mg.local_pet()
The output will be logged in files named PET<local_pet>.ESMF_LogFile.
The underlying ESMF framework needs to be initialized and finalized once and
only once per execution. This is handled internally by the
Manager and does not require any explicit
user intervention. However, the ESMF garbage collection feature is not triggered
until the finalization routine is invoked, which may not happen until the
Manager goes out of scope at the end of the
If memory deallocation of ESMPy
objects is required prior to the end of the program, the class level
destroy routines should be invoked:
mg = ESMF.Manager() mg.destroy()
MOAB Mesh backend¶
The Manager can be used to enable the MOAB mesh backend to the Mesh.
The MOAB mesh is an alternative to the native ESMF mesh, and does not yet have full support.
Spatial Discretization Objects¶
There are three different objects used for spatial coordinate representation:
Grids are used to represent logically rectangular
Meshes are used for unstructured collections of polygons, and
LocStreams are used for unstructured collections of individual points. These
objects are nearly identical counterparts to the objects of the same name in
ESMF, with some simplifications for ease of use in the Python environment.
Grid is used to represent the geometry and discretization of logically
rectangular physical grids. The
Grid can also hold information that can used in
calculations involving the
Grid, like a mask or the cell areas. Refer to the Grid Class of the
ESMF Reference Manual for more information.
Staggering is a finite difference technique in which the values of different physical quantities are placed at different locations within a grid cell.
Grid class supports a variety of stagger locations, including cell
centers, corners, and edge centers. The default stagger location in ESMF is the
cell center, and cell counts in
Grid are based on this assumption. Combinations
of the 2D ESMF stagger locations are sufficient to specify any of the Arakawa
staggers. ESMF also supports staggering in 3D and higher dimensions. There are
shortcuts for standard staggers, and interfaces through which users can create
As a default the ESMF
Grid class provides symmetric staggering, so that cell
centers are enclosed by cell perimeter (e.g. corner) stagger locations. This
means the coordinate arrays for stagger locations other than the center will
have an additional element of padding in order to enclose the cell center
locations. However, to achieve other types of staggering, the user may alter or
eliminate this padding by using the appropriate options when adding coordinates
grid = ESMF.Grid(np.array([3,4]), staggerloc=ESMF.StaggerLoc.CENTER)
In the case that the
Grid is on a sphere (coord_sys =
ESMF.api.constants.CoordSys.SPH_RAD) then the coordinates given in the
Grid are interpreted
as latitude and longitude values. The coordinates can either be in degrees or
radians as indicated by the
coord_sys flag set during
Grid creation. As is
true with many global models, this application currently assumes the latitude
and longitude refer to positions on a perfect sphere.
grid = ESMF.Grid(np.array([3,4]), staggerloc=ESMF.StaggerLoc.CENTER, coord_sys=ESMF.CoordSys.SPH_DEG)
A periodic connection can be specified when building
Grids in spherical
num_peri_dims parameter indicates the total number of
periodic dimensions and
periodic_dim is used to identify which dimensions
should be considered periodic. There must always be at least one non-periodic
dimension. For example, to create a global latitude-longitude
Grid there would
be one periodic dimension, dimension 0 (longitude).
grid = ESMF.Grid(np.array([3,4]), staggerloc=ESMF.StaggerLoc.CENTER, coord_sys=ESMF.CoordSys.SPH_DEG, num_peri_dims=1, periodic_dim=0)
Grid can generate an artificial pole by using the
pole_dim parameter. This
can be helpful for regridding operations to smooth out the interpolated values
in the polar region. For the example of creating a global latitude-longitude
Grid, the pole dimension would be 1 (latitude).
grid = ESMF.Grid(np.array([3,4]), staggerloc=ESMF.StaggerLoc.CENTER, coord_sys=ESMF.CoordSys.SPH_DEG, num_peri_dims=1, periodic_dim=0, pole_dim=1)
Masking is the process used to mark parts of a
Grid to be ignored during an
Grid cells as masked can affect the
Field values that are
represented by those cells. Masking is specified by assigning an integer value
Grid cell. This allows many different masks to be defined on the same
any combination of which may be also activated on the
Field by specifying the
corresponding integer values. The activation of
Field masks with respect to the
Grid mask is handled by
Regrid, and a more
general discussion of masking is covered in the masking
In : import numpy as np ...: import ESMF ...: grid = ESMF.Grid(np.array([3,4]), staggerloc=ESMF.StaggerLoc.CENTER, ...: coord_sys=ESMF.CoordSys.SPH_DEG, ...: num_peri_dims=1, periodic_dim=0, pole_dim=1) ...: ...: mask = grid.add_item(ESMF.GridItem.MASK, staggerloc=ESMF.StaggerLoc.CENTER) ...: mask ...: Out: array([[1, 1, 1, 1], [1, 1, 1, 1], [1, 1, 1, 1]], dtype=int32)
In : grid = ESMF.Grid(np.array([3,4]), staggerloc=[ESMF.StaggerLoc.CENTER, ESMF.StaggerLoc.CORNER], ...: coord_sys=ESMF.CoordSys.SPH_DEG, ...: num_peri_dims=1, periodic_dim=0, pole_dim=1) ...: ...: ...: gridLon = grid.get_coords(0) ...: gridLat = grid.get_coords(1) ...: gridLonCorner = grid.get_coords(0, staggerloc=ESMF.StaggerLoc.CORNER) ...: gridLatCorner = grid.get_coords(1, staggerloc=ESMF.StaggerLoc.CORNER) ...: ...: lon = np.linspace(-120,120,3) ...: lat = np.linspace(-67.5, 67.5,4) ...: lon_corner = np.arange(-180,180,120) ...: lat_corner = np.linspace(-90, 90, 5) ...: ...: lonm, latm = np.meshgrid(lon, lat, indexing='ij') ...: lonm_corner, latm_corner = np.meshgrid(lon_corner, lat_corner, indexing='ij') ...: ...: gridLon[:] = lonm ...: gridLat[:] = latm ...: gridLonCorner[:] = lonm_corner ...: gridLatCorner[:] = latm_corner ...: ...: field = ESMF.Field(grid) ...: field.get_area() ...: field.data ...: Out: array([[ 0.32224085, 1.02707409, 1.02707409, 0.32224085], [ 0.32224085, 1.02707409, 1.02707409, 0.32224085], [ 0.32224085, 1.02707409, 1.02707409, 0.32224085]])
Mesh is constructed of nodes and elements. A node, also known as a vertex
or corner, is a part of a
Mesh which represents a single point. An element,
also known as a cell, is a part of a
Mesh which represents a small
region of space. Elements are described in terms of a connected set of nodes
which represent locations along their boundaries.
The dimension of a
Mesh in ESMF is specified with two parameters: the
parametric dimension and the spatial dimension.
The parametric dimension of a
Mesh is the dimension of the topology of the
This can be thought of as the dimension of the elements which make up the
For example, a
Mesh composed of triangles would have a parametric dimension of
2, and a
Mesh composed of tetrahedra would have a parametric dimension of 3.
The spatial dimension of a
Mesh is the dimension of the space in which the
is embedded. In other words, it is the number of coordinate dimensions needed to
describe the location of the nodes making up the
For example, a
Mesh constructed of squares on a plane would have a parametric
dimension of 2 and a spatial dimension of 2. If that same
Mesh were used to
represent the 2D surface of a sphere, then the
Mesh would still have a
parametric dimension of 2, but now its spatial dimension would be 3.
Meshes whose number of coordinate dimensions (spatial dimension) is 2 or 3
are supported. The dimension of the elements in a
Mesh (parametric dimension) must
be less than or equal to the spatial dimension, but also must be either 2 or 3.
This means that a
Mesh may be either 2D elements in 2D space, 3D elements in 3D
space, or a manifold constructed of 2D elements embedded in 3D space.
For a parametric dimension of 2, the native supported element types are
triangles and quadrilaterals. In addition to these, ESMF supports 2D polygons
with any number of sides. Internally these are represented as sets of triangles,
but to the user should behave like any other element. For a parametric dimension
of 3, the supported element types are tetrahedrons and hexahedrons. The
supports any combination of element types within a particular dimension, but
types from different dimensions may not be mixed. For example, a
Mesh cannot be
constructed of both quadrilaterals and tetrahedra.
Mesh as a whole we set its parametric dimension and spatial dimension.
Mesh's parametric dimension can be thought of as the dimension of the elements
which make up the
Mesh's spatial dimension, on the other hand, is the
number of coordinate dimensions needed to describe the location of the nodes
making up the
The structure of the per node and element information used to create a
influenced by the
Mesh distribution strategy. The
Mesh class is distributed by
elements. This means that a node must be present on any PET that contains
an element associated with that node, but not on any other PET (a node
can’t be on a PET without an element “home”). Since a node may be used by
two or more elements located on different PETs, a node may be duplicated
on multiple PETs. When a node is duplicated in this manner, one and only
one of the PETs that contain the node must “own” the node. The user sets
this ownership when they define the nodes during
creation. When a
Field is created on a
Mesh (i.e. on the
on each PET the
Field is only
created on the nodes which are owned by that PET. This means that the size
Field memory on the PET can be smaller than the
number of nodes used to create the
Mesh on that PET.
Three properties need to be defined for each
Mesh node: the global id of the node
node_ids), node coordinates (
node_coords), and which PET owns the node
node_owners). The node id is a unique (across all PETs) integer attached
to the particular node. It is used to indicate which nodes are the same when
connecting together pieces of the
Mesh on different PETs. The node
coordinates indicate the location of a node in space and are used in the
functionality when interpolating. The node owner indicates which PET is in
charge of the node. This is used when creating a
Field on the
Mesh to indicate
which PET should contain a
Field location for the data.
Three properties need to be defined for each
Mesh element: the global id of the
element_ids), the topology type of the element (
which nodes are connected together to form the element (
element id is a unique (across all PETs) integer attached to the
particular element. The element type describes the topology of the element
(e.g. a triangle vs. a quadrilateral). The range of choices for the topology of
the elements in a
Mesh are restricted by the
Mesh's parametric dimension (e.g. a
Mesh can’t contain a 2D element like a triangle, when its parametric dimension
is 3D), but it can contain any combination of elements appropriate to its
dimension. In particular, in 2D ESMF supports two native element types triangle
and quadrilateral, but also provides support for polygons with any number of
sides. These polygons are represented internally as sets of triangles, but to
the user should behave like other elements. To specify a polygon with more than
four sides, the element type should be set to the number of corners of the
polygon (e.g. element type=6 for a hexagon). The element connectivity indicates
which nodes are to be connected together to form the element. The number of
nodes connected together for each element is implied by the elements topology
element_types). It is IMPORTANT to note, that the entries in this list are
NOT the global ids of the nodes, but are indices into the PET local lists
of node info used in the
Mesh creation. In other words, the element connectivity
isn’t specified in terms of the global list of nodes, but instead is specified
in terms of the locally described node info. One other important point about
connectivities is that the order of the nodes in the connectivity list of an
element is important. In general, when specifying an element with parametric
dimension 2, the nodes should be given in counterclockwise order around the
# 2.5 8 10 --------11 # / \ / | # 2.1 7 9 12 # | | 5 / # | 4 | / # | | / # 1.0 4 ------- 5 ------- 6 # | | \ 3 | # | 1 | \ | # | | 2 \ | # -0.1 1 ------- 2 ------- 3 # # -0.1 1.0 2.1 2.5 # # Node Ids at corners # Element Ids in centers # Two parametric dimensions, and two spatial dimensions mesh = ESMF.Mesh(parametric_dim=2, spatial_dim=2, coord_sys=coord_sys) num_node = 12 num_elem = 5 nodeId = np.array([1,2,3,4,5,6,7,8,9,10,11,12]) nodeCoord = np.array([-0.1,-0.1, #node id 1 1.0,-0.1, #node id 2 2.1,-0.1, #node id 3 0.1, 1.0, #node id 4 1.0, 1.0, #node id 5 2.1, 1.0, #node id 6 0.1, 2.1, #node id 7 0.5, 2.5, #node id 8 1.0, 2.1, #node id 9 1.5, 2.5, #node id 10 2.5, 2.5, #node id 11 2.5, 2.1]) #node id 12 nodeOwner = np.zeros(num_node) elemId = np.array([1,2,3,4,5]) elemType=np.array([ESMF.MeshElemType.QUAD, ESMF.MeshElemType.TRI, ESMF.MeshElemType.TRI, 5, 6]) elemConn=np.array([0,1,4,3, # elem id 1 1,2,4, # elem id 2 2,5,4, # elem id 3 3,4,8,7,6, # elem id 4 4,5,11,10,9,8]) # elem id 5 mesh.add_nodes(num_node,nodeId,nodeCoord,nodeOwner) mesh.add_elements(num_elem,elemId,elemType,elemConn)
There are two types of masking available in
Mesh: node masking and element
masking. These both work in a similar manner, but vary slightly in the details
of setting the mask information during
For node masking, the mask information is set using the
Regrid object is created the mask values arguments
dst_mask_values can then be used to indicate which particular values set in
node_mask array indicate that the node should be masked. For example, if
dst_mask_values has been set to 1, then any node in the destination
node_mask value is 1 will be masked out (a node with any other
value than 1 will not be masked).
For element masking, the mask information is set using the
parameter when adding elements to the
Mesh. In a similar manner to node masking,
the mask values parameters to
can then be used to indicate which particular values set in the
array indicate that the element should be masked. For example, if
dst_mask_values has been set to 1, then any element in the destination
element_mask value is 1 will be masked out (an element
with any other value than 1 will not be masked).
If cell areas are not specified by the user they can be calculated by ESMPy
LocStream can be used to represent the locations of a set of
data points. For example, in the data assimilation world,
LocStreams can be used
to represent a set of observations. The values of the data points are stored
Field created using the
Refer to the LocStream Class of the
ESMF Reference Manual
for more information.
The locations are generally described using Cartesian (x, y, z), or
(lat, lon, radius) coordinates. The coordinates are stored using constructs
called keys. A key is essentially a list of point descriptors, one for each data
point. They may hold other information besides the coordinates - a mask, for
example. They may also hold a second set of coordinates. Keys are referenced by
name. Each key must contain the same number of elements as there are data points
LocStream. While there is no assumption in the ordering of the points,
the order chosen must be maintained in each of the keys.
LocStream is similar to a
Mesh in that both are collections of irregularly
positioned points. However, the two structures differ because a
Mesh also has
connectivity: each data point represents either a center or corner of a cell.
There is no requirement that the points in a
LocStream have connectivity, in
fact there is no requirement that any two points have any particular spatial
relationship at all.
locstream = ESMF.LocStream(16, coord_sys=coord_sys) deg_rad = pi if coord_sys == ESMF.CoordSys.SPH_DEG: deg_rad = 180 locstream["ESMF:Lon"] = [0.0, 0.5*deg_rad, 1.5*deg_rad, 2*deg_rad, 0.0, 0.5*deg_rad, 1.5*deg_rad, 2*deg_rad, 0.0, 0.5*deg_rad, 1.5*deg_rad, 2*deg_rad, 0.0, 0.5*deg_rad, 1.5*deg_rad, 2*deg_rad] locstream["ESMF:Lat"] = [deg_rad/-2.0, deg_rad/-2.0, deg_rad/-2.0, deg_rad/-2.0, -0.25*deg_rad, -0.25*deg_rad, -0.25*deg_rad, -0.25*deg_rad, 0.25*deg_rad, 0.25*deg_rad, 0.25*deg_rad, 0.25*deg_rad, deg_rad/2.0, deg_rad/2.0, deg_rad/2.0, deg_rad/2.0] if domask: locstream["ESMF:Mask"] = np.array([1, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], dtype=np.int32)
Create a Grid or Mesh from File¶
ESMPy can create
Mesh objects from NetCDF files in a variety
of formats. A
Mesh can be created from files in
Grid files can be in
This file format is used by the
SCRIP , package, grid files that
work with that package should also work here.
SCRIP format files are
capable of storing either 2D logically rectangular grids or 2D
unstructured grids. More information can be found in the
ESMF Reference Manual.
ESMF has a custom unstructured grid file format for describing
This format is more compatible than the
SCRIP format with the methods
used to create a
Mesh object, so less conversion needs to be done to
ESMFMESH format is thus more efficient than
used with ESMPy. More information can be found in the
ESMF Reference Manual.
GRIDSPEC is an extension to the Climate and Forecast (CF) metadata
conventions for the representation of gridded data for Earth System
Models. ESMPy supports NetCDF files that follow the CF
convention to support logically rectangular lat/lon grids. More
information can be found in the
ESMF Reference Manual.
UGRID is an extension to the CF metadata
conventions for the unstructured grid data model. ESMPy support
NetCDF files that follow the CF
UGRID convention for unstructured grids.
More information can be found in the
ESMF Reference Manual.
Meshes from File¶
When creating a
Mesh from a
SCRIP format file, there are a number of
options to control the output
Mesh. The data is located at the center
of the grid cell in a
SCRIP grid. Therefore, when the
Mesh will be
part of a conservative regridding operation, the
flag must be set to True to properly generate coordinates at the the
Mesh may also be created with boolean flags to specify whether or not to
add an area property to the
add_user_area, or to add a mask
add_mask held by the NetCDF variable indicated in the optional argument,
varname. These argument are only valid for
UGRID formatted files.
The mask generated for a
Mesh created from file will
have 0 for the masked values and 1 for the unmasked values.
Grids from File¶
A number of optional boolean arguments are also supported to create a
Grid from a file. These include
is_sphere to indicate whether
the grid is spherical or regional,
add_corner_stagger to add the corner
stagger information to the
Grid for conservative regridding, and
add_user_area to specify whether to read in the cell area from the
NetCDF file or to calculate them.
GRIDSPEC formated files
there is the
add_mask optional argument
to add a mask held by the NetCDF variable indicated in optional
varname, and the
coord_names argument to specify the longitude
and latitude variable names in a
GRIDSPEC file containing multiple sets of
SCRIP formated files the integer array
grid_imask is used to mask out grid cells which should not participate in the regridding.
The mask generated for a
Grid created from
file (any format) will have 0 for the masked values and 1 for the unmasked values.
The following table describe the regridding methods and options that are available in ESMPy, the flag that is required to use it and a short description. More information can be found on these options in the ESMF Reference Manual.
Linear regridding in two dimensions
Higher-order least squares method
Nearest source point used for each destination
Nearest destination point used for each source
Normalization options for integral conservation
Line types for spherical and Cartesian space
Unmapped destination point handling options
Spherical grids and pole handling
Great Circle Cells¶
For Grids and Meshes on a sphere some combinations of interpolation options (e.g. first and second-order conservative methods) use cells whose edges are great circles. This section describes some behavior that the user may not expect from these cells and some potential solutions. A great circle edge isn’t necessarily the same as a straight line in latitude longitude space. For small edges, this difference will be small, but for long edges it could be significant. This means if the user expects cell edges as straight lines in latitude longitude space, they should avoid using one large cell with long edges to compute an average over a region (e.g. over an ocean basin).
Also, the user should also avoid using cells that contain one edge that runs half way or more around the earth, because the regrid weight calculation assumes the edge follows the shorter great circle path. There isn’t a unique great circle edge defined between points on the exact opposite side of the earth from one another (antipodal points). However, the user can work around both of these problem by breaking the long edge into two smaller edges by inserting an extra node, or by breaking the large target grid cells into two or more smaller grid cells. This allows the application to resolve the ambiguity in edge direction.
Masking is the process whereby parts of a
LocStream can be marked to be ignored
during an operation, such as when they are used in regridding. Masking can be used on a
created from a regridding source to indicate that certain portions should not be used to generate
regridded data. This is useful, for example, if a portion of the source contains unusable values.
Masking can also be used on a
Field created from a regridding destination to indicate that a certain
portion should not receive regridded data. This is useful, for example, when part of the destination
isn’t being used (e.g. the land portion of an ocean grid).
The user may mask out points in the source
Field or destination
Field or both. To do masking the user
sets mask information in the
Fields passed into the
Regrid call are built. The
dst_mask_values arguments to that call can then be used to specify which values in that mask information
indicate that a location should be masked out. For example, if
dst_mask_values is set to [1,2], then any
location that has a value of 1 or 2 in the mask information of the
LocStream upon which
Field is built will be masked out.
Masking behavior differs slightly between regridding methods. For non-conservative regridding methods
(e.g. bilinear or high-order patch), masking is done on points. For these methods, masking a destination
point means that the point will not participate in regridding. For these
methods, masking a source point means that the entire source cell using that point is masked out.
In other words, if any corner point making up a source cell is masked then the cell is masked.
For conservative regridding methods masking is done on cells.
Masking a destination cell means that the cell won’t participate in regridding.
Similarly, masking a source cell means that the cell won’t participate in regridding.
For any type of interpolation method (conservative or non-conservative)
the masking is set on the location upon which the
Fields passed into the regridding call are built.
For example, if
Fields built on
StaggerLoc.CENTER are passed into
then the masking should also be set on
Region.SELECT flag to the
zero_region parameter of
Regrid can be used to
Fields values on locations that do not
participate in the regridding operation. This is useful when setting an
uninitialized value to help identify masked locations within the
Numpy Slicing and Indexing¶
Numpy arrays are used to represent
LocStream coordinates and
among other things. Standard numpy conventions for array indexing
and slicing can be expected. There are some exceptions when it comes to fancy
indexing, index arrays, and multi-dimensional slicing. Significant effort has
been put into raising exceptions where inappropriate indexing or slicing
operations are attempted.
It is very important to remember that all indexing
and slicing operations apply ONLY to the ESMPy level objects, and these operations
do not propagate down to the lower-level Fortran- and C-based representations
of the ESMF objects. One example of where this could come up is when passing
Field slice into regridding. The entire original
Field will still be run
through the ESMF regridding engine, and only the appropriate portion of
Field slice will be updated with the regridded values.
The underlying ESMF library is built with a mix of Fortran and C/C++ and follows Fortran conventions with respect to array indexing and dimension ordering. Some effort has been made to make ESMPy feel more natural to the Python user where possible. This means that ESMPy uses 0-based indexing, which is translated to the 1-based indexing used by the ESMPy backend. However, the dimension ordering still follows Fortran conventions. Namely, longitude comes before latitude, which also comes before temporal dimensions when in use.
In : import numpy as np ...: import ESMF ...: ...: grid = ESMF.Grid(np.array([3,4]), staggerloc=ESMF.StaggerLoc.CENTER) ...: ...: gridLon = grid.get_coords(0) ...: gridLat = grid.get_coords(1) ...: ...: lon = np.linspace(-120,120,3) ...: lat = np.linspace(-67.5, 67.5,4) ...: ...: lonm, latm = np.meshgrid(lon, lat, indexing='ij') ...: ...: gridLon[:] = lonm ...: gridLat[:] = latm ...: In : grid.coords[ESMF.StaggerLoc.CENTER].shape Out: (3, 4) In : lon.shape Out: (3,) In : lat.shape Out: (4,) In : grid.coords[ESMF.StaggerLoc.CENTER] Out: array([[-120., -120., -120., -120.], [ 0., 0., 0., 0.], [ 120., 120., 120., 120.]]) In : grid.coords[ESMF.StaggerLoc.CENTER] Out: array([[-67.5, -22.5, 22.5, 67.5], [-67.5, -22.5, 22.5, 67.5], [-67.5, -22.5, 22.5, 67.5]]) In : field = ESMF.Field(grid, ndbounds=) # create a Field with a time dimension In : field.data.shape Out: (3, 4, 10)
ESMPy is a thin wrapper on top of ESMF, which was designed for high performance and scalable computing. The ESMF virtual machine is used to manage the available resources of the execution environment in a layer that is transparent to the ESMPy user. This allows the full power of the high performance computing environment to be utilized by the ESMPy user with little use of specialized parallel programming techniques.
ESMPy objects will be distributed across the available computing resources with
no additional parameters required. The
will all be transparently “parallelized” with no need for user calls to a
message passing interface. Likewise, the
Regrid class will compute and apply
the interpolation weights using all available computing resources with no need
for user intervention.
However, it is useful to remember that resulting
Field values will only be
accessible on certain PETs. The mpi4py package may be necessary for post
processing tasks that require access to global
mpirun vs. MPI.Spawn¶
There are a few different options for using ESMPy in a parallel environment. Using mpirun to specify the desired number of computing cores is probably the easiest way to start a parallel ESMPy job. Another option is to call the MPI.Spawn() function from the mpi4py Python package from within a serial Python script or interpreter. It has been observed that MPI.Spawn() may not work properly when mpi4py is built with an underlying mpich library, openmpi has seen better success. A third option is to call mpirun using a system call from within a serial Python script or interpreter, however this method is not highly recommended.
The following two examples demonstrate how to execute an ESMPy script in parallel. Any of the scripts found in the examples directory of the ESMPy source code can be run in parallel using mpirun as well as in serial mode.
mpirun -n 4 python hello_world.py
import sys from mpi4py import MPI # Parent if len(sys.argv) == 1: # Spawn workers comm = MPI.COMM_WORLD.Spawn( sys.executable, args=[sys.argv, 'worker'], maxprocs=4) # Shutdown comm.Disconnect() # Worker elif sys.argv == 'worker': # Connect to parent try: comm = MPI.Comm.Get_parent() rank = comm.Get_rank() except: raise ValueError('Could not connect to parent - ' + usage) # worker code goes here, regridding etc.. print "Hello World from PET #"+str(rank) # Shutdown comm.Disconnect() # Catch else: raise ValueError('Program should be started without arguments')
A more detailed example of using MPI.Spawn() can be found in the Tutorials section of the documentation.