next up previous contents
Next: 5 Infrastructure: Utilities Up: ESMC_crefdoc Previous: 3 Superstructure   Contents

Subsections

4 Infrastructure: Fields and Grids

14 Overview of Infrastructure Data Handling

The ESMF infrastructure data classes are part of the framework's hierarchy of structures for handling Earth system model data and metadata on parallel platforms. The hierarchy is in complexity; the simplest data class in the infrastructure represents a distributed data array and the most complex data class represents a bundle of physical fields that are discretized on the same grid. However, the current C API does not support bundled data structures yet. Array and Field are the two data classes offered by the ESMF C language binding. Data class methods are called both from user-written code and from other classes internal to the framework.

Data classes are distributed over DEs, or Decomposition Elements. A DE represents a piece of a decomposition. A DELayout is a collection of DEs with some associated connectivity that describes a specific distribution. For example, the distribution of a grid divided into four segments in the x-dimension would be expressed in ESMF as a DELayout with four DEs lying along an x-axis. This abstract concept enables a data decomposition to be defined in terms of threads, MPI processes, virtual decomposition elements, or combinations of these without changes to user code. This is a primary strategy for ensuring optimal performance and portability for codes using the ESMF for communications.

ESMF data classes are useful because they provide a standard, convenient way for developers to collect together information related to model or observational data. The information assembled in a data class includes a data pointer, a set of attributes (e.g. units, although attributes can also be user-defined), and a description of an associated grid. The same set of information within an ESMF data object can be used by the framework to arrange intercomponent data transfers, to perform I/O, for communications such as gathers and scatters, for simplification of interfaces within user code, for debugging, and for other functions. This unifies and organizes codes overall so that the user need not define different representations of metadata for the same field for I/O and for component coupling.

Since it is critical that users be able to introduce ESMF into their codes easily and incrementally, ESMF data classes can be created based on native Fortran pointers. Likewise, there are methods for retrieving native Fortran pointers from within ESMF data objects. This allows the user to perform allocations using ESMF, and to retrieve Fortran arrays later for optimized model calculations. The ESMF data classes do not have associated differential operators or other mathematical methods.

For flexibility, it is not necessary to build an ESMF data object all at once. For example, it's possible to create a field but to defer allocation of the associated field data until a later time.


Key Features
Hierarchy of data structures designed specifically for the Earth system domain and high performance, parallel computing.
Multi-use ESMF structures simplify user code overall.
Data objects support incremental construction and deferred allocation.
Native Fortran arrays can be associated with or retrieved from ESMF data objects, for ease of adoption, convenience, and performance.

14.1 Infrastructure Data Classes

The main classes that are used for model and observational data manipulation are as follows:

14.2 Design and Implementation Notes

  1. In communication methods such as Regrid, Redist, Scatter, etc. the Field code cascades down through the Array code, so that the actual implementation exist in only one place in the source.

15 Field Class

15.1 Description

An ESMF Field represents a physical field, such as temperature. The motivation for including Fields in ESMF is that bundles of Fields are the entities that are normally exchanged when coupling Components.

The ESMF Field class contains distributed and discretized field data, a reference to its associated grid, and metadata. The Field class stores the grid staggering for that physical field. This is the relationship of how the data array of a field maps onto a grid (e.g. one item per cell located at the cell center, one item per cell located at the NW corner, one item per cell vertex, etc.). This means that different Fields which are on the same underlying ESMF Grid but have different staggerings can share the same Grid object without needing to replicate it multiple times.

Fields can be added to States for use in inter-Component data communications.

Field communication capabilities include: data redistribution, regridding, scatter, gather, sparse-matrix multiplication, and halo update. These are discussed in more detail in the documentation for the specific method calls. ESMF does not currently support vector fields, so the components of a vector field must be stored as separate Field objects.

15.2 Constants


15.2.1 ESMC_REGRIDMETHOD

DESCRIPTION:
Specify which interpolation method to use during regridding.

The type of this flag is:

type(ESMC_RegridMethod_Flag)

The valid values are:

ESMC_REGRIDMETHOD_BILINEAR
Bilinear interpolation. Destination value is a linear combination of the source values in the cell which contains the destination point. The weights for the linear combination are based on the distance of destination point from each source value.
ESMC_REGRIDMETHOD_PATCH
Higher-order patch recovery interpolation. Destination value is a weighted average of 2D polynomial patches constructed from cells surrounding the source cell which contains the destination point. This method typically results in better approximations to values and derivatives than bilinear. However, because of its larger stencil, it also results in a much larger interpolation matrix (and thus routeHandle) than the bilinear.
ESMC_REGRIDMETHOD_CONSERVE
First order conservative interpolation. Value of a destination cell is the weighted sum of the values of the source cells that it overlaps. The weights are determined by the amount the source cell overlaps the destination cell. Will typically give less accurate approximations to values than the other interpolation methods, however, will do a much better job preserving the integral of the value between the source and destination. Needs corner coordinate values to be provided in the Grid. Currently only works for Fields created on the Grid center stagger (or the Mesh element location).

15.3 Use and Examples

A Field serves as an annotator of data, since it carries a description of the grid it is associated with and metadata such as name and units. Fields can be used in this capacity alone, as convenient, descriptive containers into which arrays can be placed and retrieved. However, for most codes the primary use of Fields is in the context of import and export States, which are the objects that carry coupling information between Components. Fields enable data to be self-describing, and a State holding ESMF Fields contains data in a standard format that can be queried and manipulated.

The sections below go into more detail about Field usage.

15.3.1 Field create and destroy

Fields can be created and destroyed at any time during application execution. However, these Field methods require some time to complete. We do not recommend that the user create or destroy Fields inside performance-critical computational loops.

All versions of the ESMC_FieldCreate() routines require a Mesh object as input. The Mesh contains the information needed to know which Decomposition Elements (DEs) are participating in the processing of this Field, and which subsets of the data are local to a particular DE.

The details of how the create process happens depends on which of the variants of the ESMC_FieldCreate() call is used.

When finished with an ESMC_Field, the ESMC_FieldDestroy method removes it. However, the objects inside the ESMC_Field created externally should be destroyed separately, since objects can be added to more than one ESMC_Field. For example, the same ESMF_Mesh can be referenced by multiple ESMC_Fields. In this case the internal Mesh is not deleted by the ESMC_FieldDestroy call.

15.4 Class API

15.4.1 ESMC_FieldCreateGridArraySpec - Create a Field from Grid and ArraySpec


INTERFACE:

 ESMC_Field ESMC_FieldCreateGridArraySpec(
   ESMC_Grid grid,                           // in
   ESMC_ArraySpec arrayspec,                 // in
   enum ESMC_StaggerLoc staggerloc,          // in
   ESMC_InterfaceInt *gridToFieldMap,         // in
   ESMC_InterfaceInt *ungriddedLBound,        // in
   ESMC_InterfaceInt *ungriddedUBound,        // in
   const char *name,                         // in
   int *rc                                   // out
 );
RETURN VALUE:
    Newly created ESMC_Field object.
DESCRIPTION:

Creates a ESMC_Field object.

The arguments are:

grid
A ESMC_Grid object.
arrayspec
A ESMC_ArraySpec object describing data type and kind specification.
staggerloc
Stagger location of data in grid cells. The default value is ESMF_STAGGERLOC_CENTER.
gridToFieldMap
List with number of elements equal to the grid's dimCount. The list elements map each dimension of the grid to a dimension in the field by specifying the appropriate field dimension index. The default is to map all of the grid's dimensions against the lowest dimensions of the field in sequence, i.e. gridToFieldMap = (/1,2,3,.../). The values of all gridToFieldMap entries must be greater than or equal to one and smaller than or equal to the field rank. It is erroneous to specify the same gridToFieldMap entry multiple times. The total ungridded dimensions in the field are the total field dimensions less the dimensions in the grid. Ungridded dimensions must be in the same order they are stored in the field. If the Field dimCount is less than the Mesh dimCount then the default gridToFieldMap will contain zeros for the rightmost entries. A zero entry in the gridToFieldMap indicates that the particular Mesh dimension will be replicating the Field across the DEs along this direction.
ungriddedLBound
Lower bounds of the ungridded dimensions of the field. The number of elements in the ungriddedLBound is equal to the number of ungridded dimensions in the field. All ungridded dimensions of the field are also undistributed. When field dimension count is greater than grid dimension count, both ungriddedLBound and ungriddedUBound must be specified. When both are specified the values are checked for consistency. Note that the the ordering of these ungridded dimensions is the same as their order in the field.
ungriddedUBound
Upper bounds of the ungridded dimensions of the field. The number of elements in the ungriddedUBound is equal to the number of ungridded dimensions in the field. All ungridded dimensions of the field are also undistributed. When field dimension count is greater than grid dimension count, both ungriddedLBound and ungriddedUBound must be specified. When both are specified the values are checked for consistency. Note that the the ordering of these ungridded dimensions is the same as their order in the field.
[name]
The name for the newly created field. If not specified, i.e. NULL, a default unique name will be generated: "FieldNNN" where NNN is a unique sequence number from 001 to 999.
[rc]
Return code; equals ESMF_SUCCESS if there are no errors.

15.4.2 ESMC_FieldCreateGridTypeKind - Create a Field from Grid and typekind


INTERFACE:

 ESMC_Field ESMC_FieldCreateGridTypeKind(
   ESMC_Grid grid,                           // in
   enum ESMC_TypeKind_Flag typekind,              // in
   enum ESMC_StaggerLoc staggerloc,          // in
   ESMC_InterfaceInt *gridToFieldMap,         // in
   ESMC_InterfaceInt *ungriddedLBound,        // in
   ESMC_InterfaceInt *ungriddedUBound,        // in
   const char *name,                         // in
   int *rc                                   // out
 );
RETURN VALUE:
    Newly created ESMC_Field object.
DESCRIPTION:

Creates a ESMC_Field object.

The arguments are:

grid
A ESMC_Grid object.
typekind
The ESMC_TypeKind_Flag that describes this Field data.
staggerloc
Stagger location of data in grid cells. The default value is ESMF_STAGGERLOC_CENTER.
gridToFieldMap
List with number of elements equal to the grid's dimCount. The list elements map each dimension of the grid to a dimension in the field by specifying the appropriate field dimension index. The default is to map all of the grid's dimensions against the lowest dimensions of the field in sequence, i.e. gridToFieldMap = (/1,2,3,.../). The values of all gridToFieldMap entries must be greater than or equal to one and smaller than or equal to the field rank. It is erroneous to specify the same gridToFieldMap entry multiple times. The total ungridded dimensions in the field are the total field dimensions less the dimensions in the grid. Ungridded dimensions must be in the same order they are stored in the field. If the Field dimCount is less than the Mesh dimCount then the default gridToFieldMap will contain zeros for the rightmost entries. A zero entry in the gridToFieldMap indicates that the particular Mesh dimension will be replicating the Field across the DEs along this direction.
ungriddedLBound
Lower bounds of the ungridded dimensions of the field. The number of elements in the ungriddedLBound is equal to the number of ungridded dimensions in the field. All ungridded dimensions of the field are also undistributed. When field dimension count is greater than grid dimension count, both ungriddedLBound and ungriddedUBound must be specified. When both are specified the values are checked for consistency. Note that the the ordering of these ungridded dimensions is the same as their order in the field.
ungriddedUBound
Upper bounds of the ungridded dimensions of the field. The number of elements in the ungriddedUBound is equal to the number of ungridded dimensions in the field. All ungridded dimensions of the field are also undistributed. When field dimension count is greater than grid dimension count, both ungriddedLBound and ungriddedUBound must be specified. When both are specified the values are checked for consistency. Note that the the ordering of these ungridded dimensions is the same as their order in the field.
[name]
The name for the newly created field. If not specified, i.e. NULL, a default unique name will be generated: "FieldNNN" where NNN is a unique sequence number from 001 to 999.
[rc]
Return code; equals ESMF_SUCCESS if there are no errors.

15.4.3 ESMC_FieldCreateMeshArraySpec - Create a Field from Mesh and ArraySpec


INTERFACE:

 ESMC_Field ESMC_FieldCreateMeshArraySpec(
   ESMC_Mesh mesh,                           // in
   ESMC_ArraySpec arrayspec,                 // in
   ESMC_InterfaceInt *gridToFieldMap,         // in
   ESMC_InterfaceInt *ungriddedLBound,        // in
   ESMC_InterfaceInt *ungriddedUBound,        // in
   const char *name,                         // in
   int *rc                                   // out
 );
RETURN VALUE:
    Newly created ESMC_Field object.
DESCRIPTION:

Creates a ESMC_Field object.

The arguments are:

mesh
A ESMC_Mesh object.
arrayspec
A ESMC_ArraySpec object describing data type and kind specification.
gridToFieldMap
List with number of elements equal to the grid's dimCount. The list elements map each dimension of the grid to a dimension in the field by specifying the appropriate field dimension index. The default is to map all of the grid's dimensions against the lowest dimensions of the field in sequence, i.e. gridToFieldMap = (/1,2,3,.../). The values of all gridToFieldMap entries must be greater than or equal to one and smaller than or equal to the field rank. It is erroneous to specify the same gridToFieldMap entry multiple times. The total ungridded dimensions in the field are the total field dimensions less the dimensions in the grid. Ungridded dimensions must be in the same order they are stored in the field. If the Field dimCount is less than the Mesh dimCount then the default gridToFieldMap will contain zeros for the rightmost entries. A zero entry in the gridToFieldMap indicates that the particular Mesh dimension will be replicating the Field across the DEs along this direction.
ungriddedLBound
Lower bounds of the ungridded dimensions of the field. The number of elements in the ungriddedLBound is equal to the number of ungridded dimensions in the field. All ungridded dimensions of the field are also undistributed. When field dimension count is greater than grid dimension count, both ungriddedLBound and ungriddedUBound must be specified. When both are specified the values are checked for consistency. Note that the the ordering of these ungridded dimensions is the same as their order in the field.
ungriddedUBound
Upper bounds of the ungridded dimensions of the field. The number of elements in the ungriddedUBound is equal to the number of ungridded dimensions in the field. All ungridded dimensions of the field are also undistributed. When field dimension count is greater than grid dimension count, both ungriddedLBound and ungriddedUBound must be specified. When both are specified the values are checked for consistency. Note that the the ordering of these ungridded dimensions is the same as their order in the field.
[name]
The name for the newly created field. If not specified, i.e. NULL, a default unique name will be generated: "FieldNNN" where NNN is a unique sequence number from 001 to 999.
[rc]
Return code; equals ESMF_SUCCESS if there are no errors.

15.4.4 ESMC_FieldCreateMeshTypeKind - Create a Field from Mesh and typekind


INTERFACE:

 ESMC_Field ESMC_FieldCreateMeshTypeKind(
   ESMC_Mesh mesh,                           // in
   enum ESMC_TypeKind_Flag typekind,              // in
   enum ESMC_MeshLoc_Flag meshloc,           // in
   ESMC_InterfaceInt *gridToFieldMap,         // in
   ESMC_InterfaceInt *ungriddedLBound,        // in
   ESMC_InterfaceInt *ungriddedUBound,        // in
   const char *name,                         // in
   int *rc                                   // out
 );
RETURN VALUE:
    Newly created ESMC_Field object.
DESCRIPTION:

Creates a ESMC_Field object.

The arguments are:

mesh
A ESMC_Mesh object.
typekind
The ESMC_TypeKind_Flag that describes this Field data.
meshloc
The ESMC_MeshLoc_Flag that describes this Field data.
gridToFieldMap
List with number of elements equal to the grid's dimCount. The list elements map each dimension of the grid to a dimension in the field by specifying the appropriate field dimension index. The default is to map all of the grid's dimensions against the lowest dimensions of the field in sequence, i.e. gridToFieldMap = (/1,2,3,.../). The values of all gridToFieldMap entries must be greater than or equal to one and smaller than or equal to the field rank. It is erroneous to specify the same gridToFieldMap entry multiple times. The total ungridded dimensions in the field are the total field dimensions less the dimensions in the grid. Ungridded dimensions must be in the same order they are stored in the field. If the Field dimCount is less than the Mesh dimCount then the default gridToFieldMap will contain zeros for the rightmost entries. A zero entry in the gridToFieldMap indicates that the particular Mesh dimension will be replicating the Field across the DEs along this direction.
ungriddedLBound
Lower bounds of the ungridded dimensions of the field. The number of elements in the ungriddedLBound is equal to the number of ungridded dimensions in the field. All ungridded dimensions of the field are also undistributed. When field dimension count is greater than grid dimension count, both ungriddedLBound and ungriddedUBound must be specified. When both are specified the values are checked for consistency. Note that the the ordering of these ungridded dimensions is the same as their order in the field.
ungriddedUBound
Upper bounds of the ungridded dimensions of the field. The number of elements in the ungriddedUBound is equal to the number of ungridded dimensions in the field. All ungridded dimensions of the field are also undistributed. When field dimension count is greater than grid dimension count, both ungriddedLBound and ungriddedUBound must be specified. When both are specified the values are checked for consistency. Note that the the ordering of these ungridded dimensions is the same as their order in the field.
[name]
The name for the newly created field. If not specified, i.e. NULL, a default unique name will be generated: "FieldNNN" where NNN is a unique sequence number from 001 to 999.
[rc]
Return code; equals ESMF_SUCCESS if there are no errors.

15.4.5 ESMC_FieldDestroy - Destroy a Field


INTERFACE:

 int ESMC_FieldDestroy(
   ESMC_Field *field     // inout
 );
RETURN VALUE:
    Return code; equals ESMF_SUCCESS if there are no errors.
DESCRIPTION:

Releases all resources associated with this ESMC_Field. Return code; equals ESMF_SUCCESS if there are no errors.

The arguments are:

field
Destroy contents of this ESMC_Field.

15.4.6 ESMC_FieldGetArray - Get the internal Array stored in the Field


INTERFACE:

 ESMC_Array ESMC_FieldGetArray(
   ESMC_Field field,     // in
   int *rc               // out
 );
RETURN VALUE:
    The ESMC_Array object stored in the ESMC_Field.
DESCRIPTION:

Get the internal Array stored in the ESMC_Field.

The arguments are:

field
Get the internal Array stored in this ESMC_Field.
[rc]
Return code; equals ESMF_SUCCESS if there are no errors.

15.4.7 ESMC_FieldGetMesh - Get the internal Mesh stored in the Field


INTERFACE:

 ESMC_Mesh ESMC_FieldGetMesh(
   ESMC_Field field,     // in
   int *rc               // out
 );
RETURN VALUE:
    The ESMC_Mesh object stored in the ESMC_Field.
DESCRIPTION:

Get the internal Mesh stored in the ESMC_Field.

The arguments are:

field
Get the internal Mesh stored in this ESMC_Field.
[rc]
Return code; equals ESMF_SUCCESS if there are no errors.

15.4.8 ESMC_FieldGetPtr - Get the internal Fortran data pointer stored in the Field


INTERFACE:

 void *ESMC_FieldGetPtr(
   ESMC_Field field,     // in
   int localDe,          // in
   int *rc               // out
 );
RETURN VALUE:
    The Fortran data pointer stored in the ESMC_Field.
DESCRIPTION:

Get the internal Fortran data pointer stored in the ESMC_Field.

The arguments are:

field
Get the internal Fortran data pointer stored in this ESMC_Field.
localDe
Local DE for which information is requested. [0,..,localDeCount-1].
[rc]
Return code; equals ESMF_SUCCESS if there are no errors.

15.4.9 ESMC_FieldPrint - Print the internal information of a Field


INTERFACE:

 int ESMC_FieldPrint(
   ESMC_Field field      // in
 );
RETURN VALUE:
    Return code; equals ESMF_SUCCESS if there are no errors.
DESCRIPTION:

Print the internal information within this ESMC_Field.

The arguments are:

field
Print contents of this ESMC_Field.

15.4.10 ESMC_FieldRegridGetArea - Get the area of the cells used for

conservative interpolation


INTERFACE:

 int ESMC_FieldRegridGetArea(
   ESMC_Field field      // in
 );
RETURN VALUE:
    Return code; equals ESMF_SUCCESS if there are no errors.
DESCRIPTION:

This subroutine gets the area of the cells used for conservative interpolation for the grid object associated with areaField and puts them into areaField. If created on a 2D Grid, it must be built on the ESMF_STAGGERLOC_CENTER stagger location. If created on a 3D Grid, it must be built on the ESMF_STAGGERLOC_CENTER_VCENTER stagger location. If created on a Mesh, it must be built on the ESMF_MESHLOC_ELEMENT mesh location.

The arguments are:

areaField
The Field to put the area values in.

15.4.11 ESMC_FieldRegridStore - Precompute a Field regridding operation and return a RouteHandle


INTERFACE:

 int ESMC_FieldRegridStore( 
     ESMC_Field srcField,                       // in
     ESMC_Field dstField,                       // in
     ESMC_InterfaceInt *srcMaskValues,          // in
     ESMC_InterfaceInt *dstMaskValues,          // in
     ESMC_RouteHandle *routehandle,             // inout
     enum ESMC_RegridMethod_Flag *regridmethod,       // in
     enum ESMC_UnmappedAction_Flag *unmappedaction,   // in
     ESMC_Field *srcFracField,                  // out
     ESMC_Field *dstFracField);                 // out
RETURN VALUE:
     Return code; equals ESMF_SUCCESS if there are no errors.
DESCRIPTION:

Creates a sparse matrix operation (stored in routehandle) that contains the calculations and communications necessary to interpolate from srcField to dstField. The routehandle can then be used in the call ESMC_FieldRegrid() to interpolate between the Fields.

The arguments are:

srcField
ESMC_Field with source data.
dstField
ESMC_Field with destination data.
srcMaskValues
List of values that indicate a source point should be masked out. If not specified, no masking will occur.
dstMaskValues
List of values that indicate a destination point should be masked out. If not specified, no masking will occur.
routehandle
The handle that implements the regrid, to be used in ESMC_FieldRegrid().
regridmethod
The type of interpolation. If not specified, defaults to ESMF_REGRIDMETHOD_BILINEAR.
unmappedaction
Specifies what should happen if there are destination points that can't be mapped to a source cell. Options are ESMF_UNMAPPEDACTION_ERROR or ESMF_UNMAPPEDACTION_IGNORE. If not specified, defaults to ESMF_UNMAPPEDACTION_ERROR.
[srcFracField]
The fraction of each source cell participating in the regridding. Only valid when regridmethod is ESMC_REGRIDMETHOD_CONSERVE. This Field needs to be created on the same location (e.g staggerloc) as the srcField.
[dstFracField]
The fraction of each destination cell participating in the regridding. Only valid when regridmethod is ESMF_REGRIDMETHOD_CONSERVE. This Field needs to be created on the same location (e.g staggerloc) as the dstField.

15.4.12 ESMC_FieldRegrid - Compute a regridding operation


INTERFACE:

   int ESMC_FieldRegrid( 
     ESMC_Field srcField,                // in
     ESMC_Field dstField,                // inout
     ESMC_RouteHandle routehandle,       // in
     enum ESMC_Region_Flag *zeroregion);  // in
RETURN VALUE:
    Return code; equals ESMF_SUCCESS if there are no errors.
DESCRIPTION:

Execute the precomputed regrid operation stored in routehandle to interpolate from srcField to dstField. See ESMF_FieldRegridStore() on how to precompute the routehandle. It is erroneous to specify the identical Field object for srcField and dstField arguments. This call is collective across the current VM.

The arguments are:

srcField
ESMC_Field with source data.
dstField
ESMC_Field with destination data.
routehandle
Handle to the precomputed Route.
[zeroregion]
If set to ESMC_REGION_TOTAL (default) the total regions of all DEs in dstField will be initialized to zero before updating the elements with the results of the sparse matrix multiplication. If set to ESMC_REGION_EMPTY the elements in dstField will not be modified prior to the sparse matrix multiplication and results will be added to the incoming element values. Setting zeroregion to ESMC_REGION_SELECT will only zero out those elements in the destination Array that will be updated by the sparse matrix multiplication.

15.4.13 ESMC_FieldRegridRelease - Free resources used by a regridding operation


INTERFACE:

   int ESMC_FieldRegridRelease(ESMC_RouteHandle *routehandle);  // inout
RETURN VALUE:
    Return code; equals ESMF_SUCCESS if there are no errors.
DESCRIPTION:

Free resources used by regrid object

The arguments are:

routehandle
Handle carrying the sparse matrix

16 Array Class

16.1 Description

The Array class is an alternative to the Field class for representing distributed, structured data. Unlike Fields, which are built to carry grid coordinate information, Arrays can only carry information about the indices associated with grid cells. Since they do not have coordinate information, Arrays cannot be used to calculate interpolation weights. However, if the user can supply interpolation weights, the Array sparse matrix multiply operation can be used to apply the weights and transfer data to the new grid. Arrays can also perform redistribution, scatter, and gather communication operations.

Like Fields, Arrays can be added to a State and used in inter-Component data communications.

From a technical standpoint, the ESMF Array class is an index space based, distributed data storage class. It provides DE-local memory allocations within DE-centric index regions and defines the relationship to the index space described by the ESMF DistGrid. The Array class offers common communication patterns within the index space formalism.

16.2 Class API

16.2.1 ESMC_ArrayCreate - Create an Array


INTERFACE:

 ESMC_Array ESMC_ArrayCreate(
   ESMC_ArraySpec arrayspec,   // in
   ESMC_DistGrid distgrid,     // in
   const char* name,           // in
   int *rc                     // out
 );
RETURN VALUE:
    Newly created ESMC_Array object.
DESCRIPTION:

Create an ESMC_Array object.

The arguments are:

arrayspec
ESMC_ArraySpec object containing the type/kind/rank information.
distgrid
ESMC_DistGrid object that describes how the Array is decomposed and distributed over DEs. The dimCount of distgrid must be smaller or equal to the rank specified in arrayspec, otherwise a runtime ESMF error will be raised.
[name]
The name for the Array object. If not specified, i.e. NULL, a default unique name will be generated: "ArrayNNN" where NNN is a unique sequence number from 001 to 999.
[rc]
Return code; equals ESMF_SUCCESS if there are no errors.

16.2.2 ESMC_ArrayDestroy - Destroy an Array


INTERFACE:

 int ESMC_ArrayDestroy(
   ESMC_Array *array           // inout
 );
RETURN VALUE:
    Return code; equals ESMF_SUCCESS if there are no errors.
DESCRIPTION:

Destroy an ESMC_Array object.

The arguments are:

array
ESMC_Array object to be destroyed.

16.2.3 ESMC_ArrayGetName - Get the name of an Array


INTERFACE:

 const char *ESMC_ArrayGetName(
   ESMC_Array array,           // in
   int *rc                     // out
 );
RETURN VALUE:
    Pointer to the Array name string.
DESCRIPTION:

Get the name of the specified ESMC_Array object.

The arguments are:

array
ESMC_Array object to be queried.
[rc]
Return code; equals ESMF_SUCCESS if there are no errors.

16.2.4 ESMC_ArrayGetPtr - Get pointer to Array data.


INTERFACE:

 void *ESMC_ArrayGetPtr(
   ESMC_Array array,           // in
   int localDe,                // in
   int *rc                     // out
 );
RETURN VALUE:
    Pointer to the Array data.
DESCRIPTION:

Get pointer to the data of the specified ESMC_Array object.

The arguments are:

array
ESMC_Array object to be queried.
localDe
Local De for which to data pointer is queried.
[rc]
Return code; equals ESMF_SUCCESS if there are no errors.

16.2.5 ESMC_ArrayPrint - Print an Array


INTERFACE:

 int ESMC_ArrayPrint(
   ESMC_Array array            // in
 );
RETURN VALUE:
    Return code; equals ESMF_SUCCESS if there are no errors.
DESCRIPTION:

Print internal information of the specified ESMC_Array object.

The arguments are:

array
ESMC_Array object to be printed.

17 ArraySpec Class

17.1 Description

An ArraySpec is a very simple class that contains type, kind, and rank information about an Array. This information is stored in two parameters. TypeKind describes the data type of the elements in the Array and their precision. Rank is the number of dimensions in the Array.

The only methods that are associated with the ArraySpec class are those that allow you to set and retrieve this information.

17.2 Class API

17.2.1 ESMC_ArraySpecGet - Get values from an ArraySpec


INTERFACE:

 int ESMC_ArraySpecGet(
   ESMC_ArraySpec arrayspec,         // inout
   int *rank,                        // in
   enum ESMC_TypeKind_Flag *typekind      // in
 );
RETURN VALUE:
    Return code; equals ESMF_SUCCESS if there are no errors.
DESCRIPTION:

Returns information about the contents of an ESMC_ArraySpec.

The arguments are:

arrayspec
The ESMC_ArraySpec to query.
rank
Array rank (dimensionality - 1D, 2D, etc). Maximum allowed is 7D.
typekind
Array typekind. See section ?? for valid values.

17.2.2 ESMC_ArraySpecSet - Set values for an ArraySpec


INTERFACE:

 int ESMC_ArraySpecSet(
   ESMC_ArraySpec *arrayspec,         // inout 
   int rank,                          // in
   enum ESMC_TypeKind_Flag typekind        // in
 );
RETURN VALUE:
    Return code; equals ESMF_SUCCESS if there are no errors.
DESCRIPTION:

Set an Array specification - typekind, and rank.

The arguments are:

arrayspec
The ESMC_ArraySpec to set.
rank
Array rank (dimensionality - 1D, 2D, etc). Maximum allowed is 7D.
typekind
Array typekind. See section ?? for valid values.

18 Grid Class

18.1 Description

The ESMF Grid class is used to describe the geometry and discretization of logically rectangular physical grids. It also contains the description of the grid's underlying topology and the decomposition of the physical grid across the available computational resources. The most frequent use of the Grid class is to describe physical grids in user code so that sufficient information is available to perform ESMF methods such as regridding.


Key Features
Representation of grids formed by logically rectangular regions, including uniform and rectilinear grids (e.g. lat-lon grids), curvilinear grids (e.g. displaced pole grids), and grids formed by connected logically rectangular regions (e.g. cubed sphere grids).
Support for 1D, 2D, 3D, and higher dimension grids.
Distribution of grids across computational resources for parallel operations - users set which grid dimensions are distributed.
Grids can be created already distributed, so that no single resource needs global information during the creation process.
Options to define periodicity and other edge connectivities either explicitly or implicitly via shape shortcuts.
Options for users to define grid coordinates themselves or call prefabricated coordinate generation routines for standard grids [NO GENERATION ROUTINES YET].
Options for incremental construction of grids.
Options for using a set of pre-defined stagger locations or for setting custom stagger locations.

18.1.1 Grid Representation in ESMF

ESMF Grids are based on the concepts described in A Standard Description of Grids Used in Earth System Models [Balaji 2006]. In this document Balaji introduces the mosaic concept as a means of describing a wide variety of Earth system model grids. A mosaic is composed of grid tiles connected at their edges. Mosaic grids includes simple, single tile grids as a special case.

The ESMF Grid class is a representation of a mosaic grid. Each ESMF Grid is constructed of one or more logically rectangular Tiles. A Tile will usually have some physical significance (e.g. the region of the world covered by one face of a cubed sphere grid).

The piece of a Tile that resides on one DE (for simple cases, a DE can be thought of as a processor - see section on the DELayout) is called a LocalTile. For example, the six faces of a cubed sphere grid are each Tiles, and each Tile can be divided into many LocalTiles.

Every ESMF Grid contains a DistGrid object, which defines the Grid's index space, topology, distribution, and connectivities. It enables the user to define the complex edge relationships of tripole and other grids. The DistGrid can be created explicitly and passed into a Grid creation routine, or it can be created implicitly if the user takes a Grid creation shortcut. The DistGrid used in Grid creation describes the properties of the Grid cells. In addition to this one, the Grid internally creates DistGrids for each stagger location. These stagger DistGrids are related to the original DistGrid, but may contain extra padding to represent the extent of the index space of the stagger. These DistGrids are what are used when a Field is created on a Grid.

18.1.2 Supported Grids

The range of supported grids in ESMF can be defined by:


18.1.3 Grid Topologies and Periodicity

ESMF has shortcuts for the creation of standard Grid topologies or shapes up to 3D. In many cases, these enable the user to bypass the step of creating a DistGrid before creating the Grid. There are two sets of methods which allow the user to do this. These two sets of methods cover the same set of topologies, but allow the user to specify them in different ways.

The first set of these are a group of overloaded calls broken up by the number of periodic dimensions they specify. With these the user can pick the method which creates a Grid with the number of periodic dimensions they need, and then specify other connectivity options via arguments to the method. The following is a description of these methods:


ESMF_GridCreateNoPeriDim()
Allows the user to create a Grid with no edge connections, for example, a regional Grid with closed boundaries.

ESMF_GridCreate1PeriDim()
Allows the user to create a Grid with 1 periodic dimension and supports a range of options for what to do at the pole (see Section [*]. Some examples of Grids which can be created here are tripole spheres, bipole spheres, cylinders with open poles.

ESMF_GridCreate2PeriDim()
Allows the user to create a Grid with 2 periodic dimensions, for example a torus, or a regional Grid with doubly periodic boundaries.

More detailed information can be found in the API description of each.


The second set of shortcut methods is a set of methods overloaded under the name ESMF_GridCreate(). These methods allow the user to specify the connectivites at the end of each dimension, by using the ESMF_GridConn_Flag flag. The table below shows the ESMF_GridConn_Flag settings used to create standard shapes in 2D using the ESMF_GridCreate() call. Two values are specified for each dimension, one for the low end and one for the high end of the dimension's index values.


2D Shape connflagDim1(1) connflagDim1(2) connflagDim2(1) connflagDim2(2)
Rectangle NONE NONE NONE NONE
Bipole Sphere POLE POLE PERIODIC PERIODIC
Tripole Sphere POLE BIPOLE PERIODIC PERIODIC
Cylinder NONE NONE PERIODIC PERIODIC
Torus PERIODIC PERIODIC PERIODIC PERIODIC


If the user's grid shape is too complex for an ESMF shortcut routine, or involves more than three dimensions, a DistGrid can be created to specify the shape in detail. This DistGrid is then passed into a Grid create call.


18.1.4 Grid Distribution

ESMF Grids have several options for data distribution (also referred to as decomposition). As ESMF Grids are cell based, these options are all specified in terms of how the cells in the Grid are broken up between DEs.

The main distribution options are regular, irregular, and arbitrary. A regular distribution is one in which the same number of contiguous grid cells are assigned to each DE in the distributed dimension. A irregular distribution is one in which unequal numbers of contiguous grid cells are assigned to each DE in the distributed dimension. An arbitrary distribution is one in which any grid cell can be assigned to any DE. Any of these distribution options can be applied to any of the grid shapes (i.e., rectangle) or types (i.e., rectilinear). Support for arbitrary distribution is limited in v5.2.0, See section [*] for more detail descriptions.

Figure 7 illustrates options for distribution.

Figure 7: Examples of regular and irregular decomposition of a grid a that is 6x6, and an arbitrary decomposition of a grid b that is 6x3.
\scalebox{0.9}{\includegraphics{GridDecomps}}

A distribution can also be specified using the DistGrid, by passing object into a Grid create call.


18.1.5 Grid Coordinates

Grid Tiles can have uniform, rectilinear, or curvilinear coordinates. The coordinates of uniform grids are equally spaced along their axes, and can be fully specified by the coordinates of the two opposing points that define the grid's physical span. The coordinates of rectilinear grids are unequally spaced along their axes, and can be fully specified by giving the spacing of grid points along each axis. The coordinates of curvilinear grids must be specified by giving the explicit set of coordinates for each grid point. Curvilinear grids are often uniform or rectilinear grids that have been warped; for example, to place a pole over a land mass so that it does not affect the computations performed on an ocean model grid. Figure 8 shows examples of each type of grid.

Figure 8: Types of logically rectangular grid tiles. Red circles show the values needed to specify grid coordinates for each type.
\scalebox{0.9}{\includegraphics{LogRectGrids}}

Each of these coordinate types can be set for each of the standard grid shapes described in section 18.1.3.

The table below shows how examples of common single Tile grids fall into this shape and coordinate taxonomy. Note that any of the grids in the table can have a regular or arbitrary distribution.


  Uniform Rectilinear Curvilinear
Sphere Global uniform lat-lon grid Gaussian grid Displaced pole grid
Rectangle Regional uniform lat-lon grid Gaussian grid section Polar stereographic grid section

18.1.6 Coordinate Specification and Generation

There are two ways of specifying coordinates in ESMF. The first way is for the user to set the coordinates. The second way is to take a shortcut and have the framework generate the coordinates.

No ESMF generation routines are currently available.

18.1.7 Staggering

Staggering is a finite difference technique in which the values of different physical quantities are placed at different locations within a grid cell.

The ESMF Grid class supports a variety of stagger locations, including cell centers, corners, and edge centers. The default stagger location in ESMF is the cell center, and cell counts in Grid are based on this assumption. Combinations of the 2D ESMF stagger locations are sufficient to specify any of the Arakawa staggers. ESMF also supports staggering in 3D and higher dimensions. There are shortcuts for standard staggers, and interfaces through which users can create custom staggers.

As a default the ESMF Grid class provides symmetric staggering, so that cell centers are enclosed by cell perimeter (e.g. corner) stagger locations. This means the coordinate arrays for stagger locations other than the center will have an additional element of padding in order to enclose the cell center locations. However, to achieve other types of staggering, the user may alter or eliminate this padding by using the appropriate options when adding coordinates to a Grid.

18.1.8 Masking

Masking is the process whereby parts of a grid can be marked to be ignored during an operation, such as regridding. Masking can be used on a source grid to indicate that certain portions of the grid should not be used to generate regridded data. This is useful, for example, if a portion of the source grid contains unusable values. Masking can also be used on a destination grid to indicate that the portion of the field built on that part of the Grid should not receive regridded data. This is useful, for example, when part of the grid isn't being used (e.g. the land portion of an ocean grid).

ESMF regrid currently supports masking for Fields built on structured Grids and element masking for Fields built on unstructured Meshes. The user may mask out points in the source Field or destination Field or both. To do masking the user sets mask information in the Grid or Mesh upon which the Fields passed into the ESMC_FieldRegridStore() call are built. The srcMaskValues and dstMaskValues arguments to that call can then be used to specify which values in that mask information indicate that a location should be masked out. For example, if dstMaskValues is set to (/1,2/), then any location that has a value of 1 or 2 in the mask information of the Grid or Mesh upon which the destination Field is built will be masked out.

Masking behavior differs slightly between regridding methods. For non-conservative regridding methods (e.g. bilinear or high-order patch), masking is done on points. For these methods, masking a destination point means that that point won't participate in regridding (e.g. won't be interpolated to). For these methods, masking a source point means that the entire source cell using that point is masked out. In other words, if any corner point making up a source cell is masked then the cell is masked. For conservative regridding methods (e.g. first-order conservative) masking is done on cells. Masking a destination cell means that the cell won't participate in regridding (e.g. won't be interpolated to). Similarly, masking a source cell means that the cell won't participate in regridding (e.g. won't be interpolated from). For any type of interpolation method (conservative or non-conservative) the masking is set on the location upon which the Fields passed into the regridding call are built. For example, if Fields built on ESMC_STAGGERLOC_CENTER are passed into the ESMC_FieldRegridStore() call then the masking should also be set on ESMC_STAGGERLOC_CENTER.

18.2 Constants


18.2.1 ESMC_COORDSYS

DESCRIPTION:
A set of values which indicates in which system the coordinates in the Grid are. This value is useful both to indicate to other users the type of the coordinates, but also to control how the coordinates are interpreted in regridding methods (e.g. ESMC_FieldRegridStore()).

The type of this flag is:

type(ESMC_CoordSys_Flag)

The valid values are:

ESMC_COORDSYS_CART
Cartesian coordinate system. In this system, the cartesian coordinates are mapped to the Grid coordinate dimensions in the following order: x,y,z. (E.g. using coordDim=2 in ESMC_GridGetCoord() references the y dimension)

ESMC_COORDSYS_SPH_DEG
Spherical coordinates in degrees. In this system, the spherical coordinates are mapped to the Grid coordinate dimensions in the following order: longitude, latitude, radius. (E.g. using coordDim=2 in ESMC_GridGetCoord() references the latitude dimension) Note, however, that ESMC_FieldRegridStore() currently just supports longitude and latitude (i.e. with this system, only Grids of dimension 2 are supported in the regridding).

ESMC_COORDSYS_SPH_RAD
Spherical coordinates in radians. In this system, the spherical coordinates are mapped to the Grid coordinate dimensions in the following order: longitude, latitude, radius. (E.g. using coordDim=2 in ESMC_GridGetCoord() references the latitude dimension) Note, however, that ESMC_FieldRegridStore() currently just supports longitude and latitude (i.e. with this system, only Grids of dimension 2 are supported in the regridding).


18.2.2 ESMC_GRIDITEM

DESCRIPTION:
The ESMC Grid can contain other kinds of data besides coordinates. This data is referred to as Grid ``items''. Some items may be used by ESMC for calculations involving the Grid. The following are the valid values of ESMC_GridItem_Flag.

The type of this flag is:

type(ESMC_GridItem_Flag)

The valid values are:
Item Label Type Restriction Type Default ESMC Uses Controls
ESMC_GRIDITEM_MASK ESMC_TYPEKIND_I4 ESMC_TYPEKIND_I4 YES Masking in Regrid
ESMC_GRIDITEM_AREA NONE ESMC_TYPEKIND_R8 YES Conservation in Regrid


18.2.3 ESMC_GRIDSTATUS

DESCRIPTION:
The ESMC Grid class can exist in two states. These states are present so that the library code can detect if a Grid has been appropriately setup for the task at hand. The following are the valid values of ESMC_GRIDSTATUS.

The type of this flag is:

type(ESMC_GridStatus_Flag)

The valid values are:

ESMC_GRIDSTATUS_EMPTY:
Status after a Grid has been created with ESMC_GridEmptyCreate. A Grid object container is allocated but space for internal objects is not. Topology information and coordinate information is incomplete. This object can be used in ESMC_GridEmptyComplete() methods in which additional information is added to the Grid.
ESMC_GRIDSTATUS_COMPLETE:
The Grid has a specific topology and distribution, but incomplete coordinate arrays. The Grid can be used as the basis for allocating a Field, and coordinates can be added via ESMC_GridCoordAdd() to allow other functionality.


18.2.4 ESMC_POLEKIND

DESCRIPTION:
This type describes the type of connection that occurs at the pole when a Grid is created with ESMC_GridCreate1PeriodicDim().

The type of this flag is:

type(ESMC_PoleKind_Flag)

The valid values are:

ESMC_POLEKIND_NONE
No connection at pole.

ESMC_POLEKIND_MONOPOLE
This edge is connected to itself. Given that the edge is n elements long, then element i is connected to element i+n/2.

ESMC_POLEKIND_BIPOLE
This edge is connected to itself. Given that the edge is n elements long, element i is connected to element n-i-1.


18.2.5 ESMC_STAGGERLOC

DESCRIPTION:
In the ESMC Grid class, data can be located at different positions in a Grid cell. When setting or retrieving coordinate data the stagger location is specified to tell the Grid method from where in the cell to get the data. Although the user may define their own custom stagger locations, ESMC provides a set of predefined locations for ease of use. The following are the valid predefined stagger locations.


Figure 9: 2D Predefined Stagger Locations
\scalebox{0.75}{\includegraphics{GridStaggerLoc2D}}

The 2D predefined stagger locations (illustrated in figure 9) are:

ESMC_STAGGERLOC_CENTER:
The center of the cell.
ESMC_STAGGERLOC_CORNER:
The corners of the cell.
ESMC_STAGGERLOC_EDGE1:
The edges offset from the center in the 1st dimension.
ESMC_STAGGERLOC_EDGE2:
The edges offset from the center in the 2nd dimension.


Figure 10: 3D Predefined Stagger Locations
\scalebox{1.0}{\includegraphics{GridStaggerLoc3D}}

The 3D predefined stagger locations (illustrated in figure 10) are:

ESMC_STAGGERLOC_CENTER_VCENTER:
The center of the 3D cell.
ESMC_STAGGERLOC_CORNER_VCENTER:
Half way up the vertical edges of the cell.
ESMC_STAGGERLOC_EDGE1_VCENTER:
The center of the face bounded by edge 1 and the vertical dimension.
ESMC_STAGGERLOC_EDGE2_VCENTER:
The center of the face bounded by edge 2 and the vertical dimension.
ESMC_STAGGERLOC_CORNER_VFACE:
The corners of the 3D cell.
ESMC_STAGGERLOC_EDGE1_VFACE:
The center of the edges of the 3D cell parallel offset from the center in the 1st dimension.
ESMC_STAGGERLOC_EDGE2_VFACE:
The center of the edges of the 3D cell parallel offset from the center in the 2nd dimension.
ESMC_STAGGERLOC_CENTER_VFACE:
The center of the top and bottom face. The face bounded by the 1st and 2nd dimensions.

18.3 Restrictions and Future Work

18.4 Design and Implementation Notes

18.4.1 Grid Topology

The ESMF_Grid class depends upon the ESMF_DistGrid class for the specification of its topology. That is, when creating a Grid, first an ESMF_DistGrid is created to describe the appropriate index space topology. This decision was made because it seemed redundant to have a system for doing this in both classes. It also seems most appropriate for the machinary for topology creation to be located at the lowest level possible so that it can be used by other classes (e.g. the ESMF_Array class). Because of this, however, the authors recommend that as a natural part of the implementation of subroutines to generate standard grid shapes (e.g. ESMF_GridGenSphere) a set of standard topology generation subroutines be implemented (e.g. ESMF_DistGridGenSphere) for users who want to create a standard topology, but a custom geometry.

18.5 Class API: General Grid Methods

18.5.1 ESMC_GridCreateNoPeriDim - Create a Grid with no periodic dimensions


INTERFACE:

 ESMC_Grid ESMC_GridCreateNoPeriDim(
   ESMC_InterfaceInt maxIndex,  // in
   enum ESMC_CoordSys_Flag *coordSys,      // in
   enum ESMC_TypeKind_Flag *coordTypeKind, // in
   int *rc                      // out
 );
RETURN VALUE:
    type(ESMC_Grid)
DESCRIPTION:

This call creates an ESMC_Grid with no periodic dimensions.

The arguments are:

maxIndex
The upper extent of the grid array.
coordSys
The coordinated system of the grid coordinate data. If not specified then defaults to ESMF_COORDSYS_SPH_DEG.
coordTypeKind
The type/kind of the grid coordinate data. If not specified then the type/kind will be 8 byte reals.
rc
Return code; equals ESMF_SUCCESS if there are no errors.

18.5.2 ESMC_GridCreate1PeriDim - Create a Grid with 1 periodic dimension


INTERFACE:

 ESMC_Grid ESMC_GridCreate1PeriDim(
   ESMC_InterfaceInt maxIndex,  // in
   enum ESMC_CoordSys_Flag *coordSys,      // in
   enum ESMC_TypeKind_Flag *coordTypeKind, // in
   enum ESMC_PoleKind_Flag *poleKind, // in
   int *rc                      // out
 );
RETURN VALUE:
    type(ESMC_Grid)
DESCRIPTION:

This call creates an ESMC_Grid with 1 periodic dimension.

The arguments are:

maxIndex
The upper extent of the grid array.
coordSys
The coordinated system of the grid coordinate data. If not specified then defaults to ESMF_COORDSYS_SPH_DEG.
coordTypeKind
The type/kind of the grid coordinate data. If not specified then the type/kind will be 8 byte reals.
poleKind
Two item array which specifies the type of connection which occurs at the pole. polekindflag(1) the connection that occurs at the minimum end of the index dimension. polekindflag(2) the connection that occurs at the maximum end of the index dimension. If not specified, the default is ESMF_POLETYPE_MONOPOLE for both.
rc
Return code; equals ESMF_SUCCESS if there are no errors.

18.5.3 ESMC_GridDestroy - Destroy a Grid


INTERFACE:

 int ESMC_GridDestroy(
   ESMC_Grid *grid             // in
 );
RETURN VALUE:
    Return code; equals ESMF_SUCCESS if there are no errors.
DESCRIPTION:

Destroy the Grid.

The arguments are:

grid
Grid object whose memory is to be freed.

18.5.4 ESMC_GridAddItem - Add items to a Grid


INTERFACE:

 int ESMC_GridAddItem(
   ESMC_Grid grid,                   // in
   enum ESMC_GridItem_Flag itemflag, // in
   enum ESMC_StaggerLoc staggerloc   // in
 );
RETURN VALUE:
    Return code; equals ESMF_SUCCESS if there are no errors.
DESCRIPTION:

Add coordinates to the Grid.

The arguments are:

grid
Grid object to which the coordinates will be added
itemflag
The grid item to add.
staggerloc
The stagger location to add. If not present, defaults to ESMC_STAGGERLOC_CENTER.

18.5.5 ESMC_GridGetItem - Get item from a Grid


INTERFACE:

 void * ESMC_GridGetItem(
   ESMC_Grid grid,                         // in
   enum ESMC_GridItem_Flag itemflag,       // in
   enum ESMC_StaggerLoc staggerloc,        // in
   int *rc                                 // out
 );
RETURN VALUE:
    Return code; equals ESMF_SUCCESS if there are no errors.
DESCRIPTION:

Get coordinatess from the Grid.

The arguments are:

grid
Grid object from which to obtain the coordinates.
itemflag
The grid item to add.
staggerloc
The stagger location to add. If not present, defaults to ESMC_STAGGERLOC_CENTER.
rc
Return code; equals ESMF_SUCCESS if there are no errors.

18.5.6 ESMC_GridAddCoord - Add coordinates to a Grid


INTERFACE:

 int ESMC_GridAddCoord(
   ESMC_Grid grid,                   // in
   enum ESMC_StaggerLoc staggerloc   // in
 );
RETURN VALUE:
    Return code; equals ESMF_SUCCESS if there are no errors.
DESCRIPTION:

Add coordinates to the Grid.

The arguments are:

grid
Grid object to which the coordinates will be added
staggerloc
The stagger location to add. If not present, defaults to ESMC_STAGGERLOC_CENTER.

18.5.7 ESMC_GridGetCoord - Get coordinates from a Grid


INTERFACE:

 void * ESMC_GridGetCoord(
   ESMC_Grid grid,                         // in
   int coordDim,                           // in
   enum ESMC_StaggerLoc staggerloc,        // in
   int *exclusiveLBound,                   // out
   int *exclusiveUBound,                   // out
   int *rc                                 // out
 );
RETURN VALUE:
    Return code; equals ESMF_SUCCESS if there are no errors.
DESCRIPTION:

Get coordinatess from the Grid.

The arguments are:

grid
Grid object from which to obtain the coordinates.
coordDim
The coordinate dimension from which to get the data.
staggerloc
The stagger location to add. If not present, defaults to ESMC_STAGGERLOC_CENTER.
exclusiveLBound
Upon return this holds the lower bounds of the exclusive region. This bound must be allocated to be of size equal to the coord dimCount.
exclusiveUBound
Upon return this holds the upper bounds of the exclusive region. This bound must be allocated to be of size equal to the coord dimCount.
rc
Return code; equals ESMF_SUCCESS if there are no errors.

18.5.8 ESMC_GridWrite - Write a Grid to a VTK file


INTERFACE:

 int ESMC_GridWrite(
   ESMC_Grid grid,                  // in
   enum ESMC_StaggerLoc staggerloc, // in
   const char* fname                // in
 );
RETURN VALUE:
    Return code; equals ESMF_SUCCESS if there are no errors.
DESCRIPTION:

Write a grid to VTK file.

The arguments are:

grid
The grid.
staggerloc
The stagger location to add. If not present, defaults to ESMC_STAGGERLOC_CENTER.
filename
The name of the output file.

19 Mesh Class

19.1 Description

Unstructured grids are commonly used in the computational solution of Partial Differential equations. These are especially useful for problems that involve complex geometry, where using the less flexible structured grids can result in grid representation of regions where no computation is needed. Finite element and finite volume methods map naturally to unstructured grids and are used commonly in hydrology, ocean modeling, and many other applications.

In order to provide support for application codes using unstructured grids, the ESMF library provides a class for representing unstructured grids called the Mesh. Fields can be created on a Mesh to hold data. In Fortran, Fields created on a Mesh can also be used as either the source or destination or both of an interpolaton (i.e. an ESMF_FieldRegridStore() call). This capability is currently not supported with the C interface, however, if the C Field is passed via a State to a component written in Fortran then the regridding can be performed there. The rest of this section describes the Mesh class and how to create and use them in ESMF.

19.1.1 Mesh Representation in ESMF

A Mesh in ESMF is described in terms of nodes and elements. A node is a point in space which represents where the coordinate information in a Mesh is located. An element is a higher dimensional shape constructed of nodes. Elements give a Mesh its shape and define the relationship of the nodes to one another. Field data may be located on a Mesh's nodes.

19.1.2 Supported Meshes

The range of Meshes supported by ESMF are defined by several factors: dimension, element types, and distribution.

ESMF currently only supports Meshes whose number of coordinate dimensions (spatial dimension) is 2 or 3. The dimension of the elements in a Mesh (parametric dimension) must be less than or equal to the spatial dimension, but also must be either 2 or 3. This means that an ESMF mesh may be either 2D elements in 2D space, 3D elements in 3D space, or a manifold constructed of 2D elements embedded in 3D space.

ESMF currently supports two types of elements for each Mesh parametric dimension. For a parametric dimension of 2 the supported element types are triangles or quadrilaterals. For a parametric dimension of 3 the supported element types are tetrahedrons and hexahedrons. See Section [*] for diagrams of these. The Mesh supports any combination of element types within a particular dimension, but types from different dimensions may not be mixed, for example, a Mesh cannot be constructed of both quadrilaterals and tetrahedra.

ESMF currently only supports distributions where every node on a PET must be a part of an element on that PET. In other words, there must not be nodes without an element on a PET.

19.2 Constants


19.2.1 ESMC_MESHELEMTYPE

DESCRIPTION:
An ESMF Mesh can be constructed from a combination of different elements. The type of elements that can be used in a Mesh depends on the Mesh's parameteric dimension, which is set during Mesh creation. The following are the valid Mesh element types for each valid Mesh parametric dimension (2D or 3D) .


                     3                          4 ---------- 3
                    / \                         |            |  
                   /   \                        |            |
                  /     \                       |            |
                 /       \                      |            |
                /         \                     |            |
               1 --------- 2                    1 ---------- 2

           ESMC_MESHELEMTYPE_TRI            ESMC_MESHELEMTYPE_QUAD

2D element types (numbers are the order for elementConn during 
                  Mesh create)

For a Mesh with parametric dimension of 2 the valid element types (illustrated above) are:

Element Type Number of Nodes Description
ESMC_MESHELEMTYPE_TRI 3 A triangle
ESMC_MESHELEMTYPE_QUAD 4 A quadrilateral (e.g. a rectangle)



                                            
                 3                               8---------------7
                /|\                             /|              /|
               / | \                           / |             / |
              /  |  \                         /  |            /  |
             /   |   \                       /   |           /   |
            /    |    \                     5---------------6    |
           4-----|-----2                    |    |          |    |
            \    |    /                     |    4----------|----3
             \   |   /                      |   /           |   /
              \  |  /                       |  /            |  /
               \ | /                        | /             | /
                \|/                         |/              |/
                 1                          1---------------2

       ESMC_MESHELEMTYPE_TETRA             ESMC_MESHELEMTYPE_HEX  

3D element types (numbers are the order for elementConn during 
                  Mesh create)

For a Mesh with parametric dimension of 3 the valid element types (illustrated above) are:

Element Type Number of Nodes Description
ESMC_MESHELEMTYPE_TETRA 4 A tetrahedron (CAN'T BE USED IN REGRID)
ESMC_MESHELEMTYPE_HEX 8 A hexahedron (e.g. a cube)

19.3 Class API


19.3.1 ESMC_MeshAddElements - Add elements to a Mesh


INTERFACE:

 int ESMC_MeshAddElements(
   ESMC_Mesh mesh,           // inout 
   int  elementCount,        // in
   int *elementIds,          // in
   int *elementTypes,        // in
   int *elementConn,         // in
   int *elementMask,         // in
   double *elementArea       // in
 );
RETURN VALUE:
    Return code; equals ESMF_SUCCESS if there are no errors.
DESCRIPTION:

This call is the third and last part of the three part mesh create sequence and should be called after the mesh is created with ESMF_MeshCreate() (19.3.3) and after the nodes are added with ESMF_MeshAddNodes() (19.3.2). This call adds the elements to the mesh and finalizes the create. After this call the Mesh is usable, for example a Field may be built on the created Mesh object and this Field may be used in a ESMF_FieldRegridStore() call.

The parameters to this call elementIds, elementTypes, and elementConn describe the elements to be created. The description for a particular element lies at the same index location in elementIds and elementTypes. Each entry in elementConn consists of the list of nodes used to create that element, so the connections for element $e$ in the elementIds array will start at $number\_of\_nodes\_in\_element(1) + number\_of\_nodes\_in\_element(2) +
\cdots + number\_of\_nodes\_in\_element(e-1) + 1$ in elementConn.

mesh
Mesh object.
elementCount
The number of elements on this PET.
elementIds
An array containing the global ids of the elements to be created on this PET. This input consists of a 1D array of size elementCount.
elementTypes
An array containing the types of the elements to be created on this PET. The types used must be appropriate for the parametric dimension of the Mesh. Please see Section [*] for the list of options. This input consists of a 1D array of size elementCount.
elementConn
An array containing the indexes of the sets of nodes to be connected together to form the elements to be created on this PET. The entries in this list are NOT node global ids, but rather each entry is a local index (1 based) into the list of nodes which were created on this PET by the previous ESMC_MeshAddNodes() call. In other words, an entry of 1 indicates that this element contains the node described by nodeIds(1), nodeCoords(1), etc. passed into the ESMC_MeshAddNodes() call on this PET. It is also important to note that the order of the nodes in an element connectivity list matters. Please see Section [*] for diagrams illustrating the correct order of nodes in a element. This input consists of a 1D array with a total size equal to the sum of the number of nodes in each element on this PET. The number of nodes in each element is implied by its element type in elementTypes. The nodes for each element are in sequence in this array (e.g. the nodes for element 1 are elementConn(1), elementConn(2), etc.).
[elementMask]
An array containing values which can be used for element masking. Which values indicate masking are chosen via the srcMaskValues or dstMaskValues arguments to ESMF_FieldRegridStore() call. This input consists of a 1D array the size of the number of elements on this PET. If not specified (i.e. NULL is passed in), then no masking will occur.
[elementArea]
An array containing element areas. This input consists of a 1D array the size of the number of elements on this PET. If not specified (i.e. NULL is passed in), the element areas are internally calculated.


19.3.2 ESMC_MeshAddNodes - Add nodes to a Mesh


INTERFACE:

 int ESMC_MeshAddNodes(
   ESMC_Mesh mesh,          // inout
   int nodeCount,           // in
   int *nodeIds,            // in
   double *nodeCoords,      // in
   int *nodeOwners          // in
 );
RETURN VALUE:
    Return code; equals ESMF_SUCCESS if there are no errors.
DESCRIPTION:

This call is the second part of the three part mesh create sequence and should be called after the mesh's dimensions are set using ESMC_MeshCreate(). This call adds the nodes to the mesh. The next step is to call ESMC_MeshAddElements() (19.3.3).

The parameters to this call nodeIds, nodeCoords, and nodeOwners describe the nodes to be created on this PET. The description for a particular node lies at the same index location in nodeIds and nodeOwners. Each entry in nodeCoords consists of spatial dimension coordinates, so the coordinates for node $n$ in the nodeIds array will start at $(n-1)*spatialDim+1$.

mesh
Mesh object.
nodeCount
The number of nodes on this PET.
nodeIds
An array containing the global ids of the nodes to be created on this PET. This input consists of a 1D array the size of the number of nodes on this PET (i.e. nodeCount).
nodeCoords
An array containing the physical coordinates of the nodes to be created on this PET. The coordinates in this array are ordered so that the coordinates for a node lie in sequence in memory. (e.g. for a Mesh with spatial dimension 2, the coordinates for node 1 are in nodeCoords(0) and nodeCoords(1), the coordinates for node 2 are in nodeCoords(2) and nodeCoords(3), etc.). This input consists of a 1D array the size of nodeCount times the Mesh's spatial dimension (spatialDim).
nodeOwners
An array containing the PETs that own the nodes to be created on this PET. If the node is shared with another PET, the value may be a PET other than the current one. Only nodes owned by this PET will have PET local entries in a Field created on the Mesh. This input consists of a 1D array the size of the number of nodes on this PET (i.e. nodeCount).


19.3.3 ESMC_MeshCreate - Create a Mesh as a 3 step process


INTERFACE:

 ESMC_Mesh ESMC_MeshCreate(
   int parametricDim,         // in
   int spatialDim,            // in
   int *rc                    // out
 );
RETURN VALUE:
    type(ESMC_Mesh)         :: ESMC_MeshCreate
DESCRIPTION:

This call is the first part of the three part mesh create sequence. This call sets the dimension of the elements in the mesh (parametricDim) and the number of coordinate dimensions in the mesh (spatialDim). The next step is to call ESMC_MeshAddNodes() (19.3.2) to add the nodes and then ESMC_MeshAddElements() (19.3.1) to add the elements and finalize the mesh.

The arguments are:

parametricDim
Dimension of the topology of the Mesh. (E.g. a mesh constructed of squares would have a parametric dimension of 2, whereas a Mesh constructed of cubes would have one of 3.)
spatialDim
The number of coordinate dimensions needed to describe the locations of the nodes making up the Mesh. For a manifold, the spatial dimesion can be larger than the parametric dim (e.g. the 2D surface of a sphere in 3D space), but it can't be smaller.
rc
Return code; equals ESMF_SUCCESS if there are no errors.


19.3.4 ESMC_MeshCreateFromFile - Create a Mesh from a NetCDF grid file


INTERFACE:

 ESMC_Mesh ESMC_MeshCreateFromFile(
 				  char *filename,     // in (required)
 				  int fileTypeFlag,   // in (required)
 				  int *convert3D,     // in (optional)
 				  int *convertToDual, // in (optional)
 				  int *addUserArea,   // in (optional)
 				  char *meshname,     // in (optional)
 				  int *addMask,       // in (optional)
 				  char *varname,      // in (optional)
 				  int *rc             // out
 );
RETURN VALUE:
    type(ESMC_Mesh)         :: ESMC_MeshCreateFromFile
DESCRIPTION:

Method to create a Mesh object from a NetCDF file in either SCRIP, UGRID, or ESMF file formats.

The required arguments are:

filename
The name of the grid file
filetypeflag
The file type of the grid file to be read, please see Section [*] for a list of valid options.
[convert3D]
if 1, the node coordinates will be converted into 3D Cartisian, which is required for a global grid. If not specified, defaults to 0.
[convertToDual]
if 1, the mesh will be converted to its dual. If not specified, defaults to 1. Converting to dual is only supported with file type ESMF_FILEFORMAT_SCRIP.
[addUserArea]
if 1, the cell area will be read in from the GRID file. This feature is only supported when the grid file is in the SCRIP or ESMF format. If not specified, defaults to 0.
[meshname]
The dummy variable for the mesh metadata in the UGRID file if the filetypeflag is ESMF_FILEFORMAT_UGRID. If not specified, defaults to empty string.
[addMask]
If 1, generate the mask using the missing_value attribute defined in 'varname' This flag is only supported when the grid file is in the UGRID format. If not specified, defaults to 0.
[varname]
If addMask is 1, provide a variable name stored in the UGRID file and the mask will be generated using the missing value of the data value of this variable. The first two dimensions of the variable has to be the the longitude and the latitude dimension and the mask is derived from the first 2D values of this variable even if this data is 3D, or 4D array. If not specified, defaults to empty string.
[rc]
Return code; equals ESMF_SUCCESS if there are no errors.

19.3.5 ESMC_MeshDestroy - Destroy a Mesh


INTERFACE:

 int ESMC_MeshDestroy(
   ESMC_Mesh *mesh             // in
 );
RETURN VALUE:
    Return code; equals ESMF_SUCCESS if there are no errors.
DESCRIPTION:

Destroy the Mesh. This call removes all internal memory associated with mesh. After this call mesh will no longer be usable.

The arguments are:

mesh
Mesh object whose memory is to be freed.

19.3.6 ESMC_MeshFreeMemory - Remove a Mesh and its memory


INTERFACE:

 int ESMC_MeshFreeMemory(
   ESMC_Mesh mesh            // in
 );
RETURN VALUE:
    Return code; equals ESMF_SUCCESS if there are no errors.
DESCRIPTION:

This call removes the portions of mesh which contain connection and coordinate information. After this call, Fields build on mesh will no longer be usable as part of an ESMF_FieldRegridStore() operation. However, after this call Fields built on mesh can still be used in an ESMF_FieldRegrid() operation if the routehandle was generated beforehand. New Fields may also be built on mesh after this call.

The arguments are:

mesh
Mesh object whose memory is to be freed.

19.3.7 ESMC_MeshGetLocalElementCount - Get the number of elements in a Mesh on the current PET


INTERFACE:

 int ESMC_MeshGetLocalElementCount(
   ESMC_Mesh mesh,           // in
   int *elementCount         // out
 );
RETURN VALUE:
    Return code; equals ESMF_SUCCESS if there are no errors.
DESCRIPTION:

Query the number of elements in a mesh on the local PET. The arguments are:

mesh
The mesh
elementCount
The number of elements on this PET.

19.3.8 ESMC_MeshGetLocalNodeCount - Get the number of nodes in a Mesh on the current PET


INTERFACE:

 int ESMC_MeshGetLocalNodeCount(
   ESMC_Mesh mesh,          // in
   int *nodeCount           // out
 );
RETURN VALUE:
    Return code; equals ESMF_SUCCESS if there are no errors.
DESCRIPTION:

Query the number of nodes in a mesh on the local PET. The arguments are:

mesh
The mesh
nodeCount
The number of nodes on this PET.

19.3.9 ESMC_MeshGetOwnedElementCount - Get the number of elements in a Mesh owned by the current PET


INTERFACE:

 int ESMC_MeshGetOwnedElementCount(
   ESMC_Mesh mesh,           // in
   int *elementCount         // out
 );
RETURN VALUE:
    Return code; equals ESMF_SUCCESS if there are no errors.
DESCRIPTION:

Query the number of elements in a mesh owned by the local PET. This number will be equal or less than the local element count. The arguments are:

mesh
The mesh
elementCount
The number of elements owned by this PET.

19.3.10 ESMC_MeshGetOwnedNodeCount - Get the number of nodes in a Mesh owned by the current PET


INTERFACE:

 int ESMC_MeshGetOwnedNodeCount(
   ESMC_Mesh mesh,          // in
   int *nodeCount           // out
 );
RETURN VALUE:
    Return code; equals ESMF_SUCCESS if there are no errors.
DESCRIPTION:

Query the number of nodes in a mesh owned by the local PET. This number will be equal or less than the local node count. The arguments are:

mesh
The mesh
nodeCount
The number of nodes owned by this PET.

20 DistGrid Class

20.1 Description

The ESMF DistGrid class sits on top of the DELayout class (not currently directly accessible through the ESMF C API) and holds domain information in index space. A DistGrid object captures the index space topology and describes its decomposition in terms of DEs. Combined with DELayout and VM the DistGrid defines the data distribution of a domain decomposition across the computational resources of an ESMF Component.

The global domain is defined as the union or ``tilework'' of logically rectangular (LR) sub-domains or tiles. The DistGrid create methods allow the specification of such a tilework global domain and its decomposition into exclusive, DE-local LR regions according to various degrees of user specified constraints. Complex index space topologies can be constructed by specifying connection relationships between tiles during creation.

The DistGrid class holds domain information for all DEs. Each DE is associated with a local LR region. No overlap of the regions is allowed. The DistGrid offers query methods that allow DE-local topology information to be extracted, e.g. for the construction of halos by higher classes.

A DistGrid object only contains decomposable dimensions. The minimum rank for a DistGrid object is 1. A maximum rank does not exist for DistGrid objects, however, ranks greater than 7 may lead to difficulties with respect to the Fortran API of higher classes based on DistGrid. The rank of a DELayout object contained within a DistGrid object must be equal to the DistGrid rank. Higher class objects that use the DistGrid, such as an Array object, may be of different rank than the associated DistGrid object. The higher class object will hold the mapping information between its dimensions and the DistGrid dimensions.

20.2 Class API

20.2.1 ESMC_DistGridCreate - Create a DistGrid


INTERFACE:

 ESMC_DistGrid ESMC_DistGridCreate(
   ESMC_InterfaceInt minIndexInterfaceArg,   // in
   ESMC_InterfaceInt maxIndexInterfaceArg,   // in
   int *rc                                   // out
 );
RETURN VALUE:
    Newly created ESMC_DistGrid object.
DESCRIPTION:

Create an ESMC_DistGrid from a single logically rectangular (LR) tile with default decomposition. The default decomposition is deCount $ \times 1 \times ... \times 1$, where deCount is the number of DEs in a default DELayout, equal to petCount. This means that the default decomposition will be into as many DEs as there are PETs, with 1 DE per PET.

The arguments are:

minIndex
Global coordinate tuple of the lower corner of the tile.
maxIndex
Global coordinate tuple of the upper corner of the tile.
[rc]
Return code; equals ESMF_SUCCESS if there are no errors.

20.2.2 ESMC_DistGridDestroy - Destroy a DistGrid


INTERFACE:

 int ESMC_DistGridDestroy(
   ESMC_DistGrid *distgrid         // inout
 );
RETURN VALUE:
    Return code; equals ESMF_SUCCESS if there are no errors.
DESCRIPTION:

Destroy an ESMC_DistGrid object.

The arguments are:

distgrid
ESMC_DistGrid object to be destroyed.

20.2.3 ESMC_DistGridPrint - Print a DistGrid


INTERFACE:

 int ESMC_DistGridPrint(
   ESMC_DistGrid distgrid          // in
 );
RETURN VALUE:
    Return code; equals ESMF_SUCCESS if there are no errors.
DESCRIPTION:

Print internal information of the specified ESMC_DistGrid object.

The arguments are:

distgrid
ESMC_DistGrid object to be destroyed.

21 RouteHandle Class

21.1 Description

The ESMF RouteHandle class provides a unified interface for all route-based communciation methods across the Field, FieldBundle, Array, and ArrayBundle classes. All route-based communication methods implement a pre-computation step, returning a RouteHandle, an execution step, and a release step. Typically the pre-computation, or Store() step will be a lot more expensive (both in memory and time) than the execution step. The idea is that once precomputed, a RouteHandle will be executed many times over during a model run, making the execution time a very preformance critical piece of code. In ESMF, Regridding, Redisting, and Haloing are implemented as route-based communication methods. The following sections discuss the RouteHandle concepts that apply uniformly to all route-based communication methods, across all of the above mentioned classes.

21.2 Use and Examples

The user interacts with the RouteHandle class through the route-based communication methods of Field, FieldBundle, Array, and ArrayBundle. The usage of these methods are described in detail under their respective class documentation section. The following examples focus on the RouteHandle aspects common across classes and methods.

21.3 Restrictions and Future Work

21.4 Design and Implementation Notes

Internally all route-based communication calls are implemented as sparse matrix multiplications. The precompute step for all of the supported communication methods can be broke into three steps:

  1. Construction of the sparse matrix for the specific communication method.
  2. Generation of the communication pattern according to the sparse matrix.
  3. Encoding of the communication pattern for each participating PET in form of an XXE stream.

21.5 Class API

21.5.1 ESMC_RouteHandlePrint - Print a RouteHandle


INTERFACE:

 int ESMC_RouteHandlePrint(
   ESMC_RouteHandle rh            // in
 );
RETURN VALUE:
    Return code; equals ESMF_SUCCESS if there are no errors.
DESCRIPTION:

Print internal information of the specified ESMC_RouteHandle object.

The arguments are:

rh
ESMC_RouteHandle object to be printed.


next up previous contents
Next: 5 Infrastructure: Utilities Up: ESMC_crefdoc Previous: 3 Superstructure   Contents
esmf_support@list.woc.noaa.gov