ESMF Releases

ESMF releases are listed below in reverse chronological order. Links to matching documentation, the release date, release notes, and known bugs are provided for each release.

IMPORTANT: Starting with release 8.0.0, no distinction is made between "internal" and "public" releases. All releases 8.0.0 and up are considered public. The version number is defined as major.minor.patch, and completely specifies a release. The old "r" and "rpX" suffix, where X is a patch number, is no longer used. Release 7.1.0r was the last release that followed the old versioning scheme and nomenclature.



Version Date Release Notes Known Issues
HEAD

Reference Manual for Fortran
built from Git Develop HEAD
(html) (pdf)

Reference Manual for C
built from Git Develop HEAD
(html) (pdf)

User's Guide
built from Git Develop HEAD
(html) (pdf)

NUOPC Layer Reference Manual
(html) (pdf)

Building a NUOPC Model Manual
(html) (pdf)

ESMPy Documentation
(html) (pdf)
Current The Git Develop HEAD represents the most recently checked in source code and does not always compile nor run. Users will generally want to download the most recent release with a specific version number below. The documentation available at left is auto-generated daily and may be missing or have formatting errors. Known issues are only listed for official releases.
v8.6.0

Reference Manual for Fortran
(html) (pdf)

Reference Manual for C
(html) (pdf)

User's Guide
(html) (pdf)

NUOPC Layer Reference Manual
(html) (pdf)

Building a NUOPC Model Manual
(html) (pdf)

ESMPy Documentation
(html) (pdf)

Regridding Status

Supported Platforms

API Changes since v8.5.0

API Test Coverage
11/14/23
  • The public Fortran API in this release is backward compatible with the previous release, ESMF 8.5.0. There were a few API changes, none of which require user code changes. The complete list of API changes is summarized in a table showing interface changes since ESMF 8.5.0. The table includes the rationale and impact for each change.
  • No bit-for-bit changes were observed for this release compared to ESMF 8.5.0. This is based on test runs with the Intel compilers using options "-O2 -fp-model precise". However, changes were made to the implementation of the conservative weight calculation to remove a problem caused by the same calculation being optimized differently in two places in the code. Bit-for-bit changes compared to ESMF 8.5.0 from this change are possible when using regridding methods ESMF_REGRIDMETHOD_CONSERVE or ESMF_REGRIDMETHOD_CONSERVE_2ND and not using strict floating point compiler options.
  • No changes that affect the status of existing regridding methods were made to the ESMF regrid weight generation methods and applications. However, the tables summarizing the ESMF regridding status were extended to cover the spherical vector regridding option added in this release.
  • Keys to match all of the ESMF_RUNTIME_* environment variables were added to the App Options of the ESMX run configuration. This feature provides a convenient way to control the run-time behavior of the ESMX executable from its standard configuration file.
  • Basic accelerator device management for multi-component applications was implemented. This feature allows the user to assign accelerator devices via the "devList" option to individual components. "devList" is available as a new argument to ESMF_GridCompCreate() and ESMF_CplCompCreate(), and is also accessible on the ESMX level as Component Label Option. Once specified, ESMF handles the device bookkeeping, and provides the user with context specific device information through the ESMF_VMGet() API. This feature can be leveraged in connection with popular accelerator programming paradigms like OpenACC, OpenMP, or standard language approaches to offload component code to the desired devices.
  • Basic support for vector regridding was added to the ESMF_FieldRegridStore() method. The initial implementation of this feature is limited to 2D tangential vectors (expressed in terms of east and north components) on a spherical geometry (e.g. an ESMF_Grid with ESMF_COORDSYS_SPH_DEG) and requires that the vector components are stored in an ungridded Field dimension. The advantage of using this capability, which maps vectors through 3D Cartesian space, instead of regridding both components separately, is that it provides more accurate results, particularly in the polar regions.
  • The "normalization" attribute in mapping files written by the ESMF regrid weight generation methods and applications was changed to now correctly set "N/A" for anything other than the conservative regridding methods. User code that accesses the "normalization" attribute in ESMF mapping files might need to be adjusted accordingly.
  • A new entry point was added to the ESMF_FieldCreate() interface allowing creation from an existing Field object. The interface supports slicing with respect to trailing ungridded dimensions.
  • The ESMF_ArrayCreate() entry point that allows creation from an existing Array object was extended to support slicing with respect to trailing undistributed dimensions.
  • The Read and Write operations for multi-tile Arrays and Fields (e.g., for representing a cubed sphere grid as a six-tile grid) have been extended to permit I/O for Arrays / Fields with layouts other than 1 DE per PET. This change applies to the ESMF_ArrayRead(), ESMF_ArrayWrite(), ESMF_ArrayBundleRead(), ESMF_ArrayBundleWrite(), ESMF_FieldRead(), ESMF_FieldWrite(), ESMF_FieldBundleRead(), and ESMF_FieldBundleWrite() methods. The remaining limitations of the implementation are discussed in the Restrictions and Future Work section of the I/O Capability in the reference manual.
  • A basic ESMF C API was added to provide access to the ESMF tracing and profiling capability from code written in C.
  • Crashes that were observed with ESMF applications during MPI_Finalize() with the Darshan or Cray Performance Analysis Tool active have been resolved. (The issue was traced back to an interaction with the MPI Tools interface that used to be initialized during ESMF_Initialize(). ESMF no longer initializes the MPI Tools interface by default to avoid this problem.)
  • The issues noted in previous releases with regard to using the ESMF tracing and profiling feature for MPI calls have been resolved. This capability is now fully functional on all the supported platforms.
  • NVIDIA's NVHPC compiler suite is now fully supported by ESMF for both ESMF_OS=Linux and ESMF_OS=Unicos, and setting ESMF_COMPILER=nvhpc. This is particularly important for users looking at exploring ESMF accelerator device support on systems with NVIDIA GPUs.
  • Using vector regridding (vectorRegrid=.true.) through the ESMF_FieldRegridStore() interface in combination with either conservative regridding method (regridmethod=ESMF_REGRIDMETHOD_CONSERVE or regridmethod=ESMF_REGRIDMETHOD_CONSERVE_2ND) currently results in an error.
  • Attempting to write weight files from the ESMPy Regrid object when using filemode=FileMode.WITHAUX currently crashes.

    Platform-specific issues:

  • Compiling ESMF with GCC version 8.1.0 triggers an internal compiler error in ESMF_HConfig.F90 due to the use of allocatable character variables. Earlier and later versions of GCC do not have this issue.
  • On Darwin, with version 15 of the clang C compiler, when building under Rosetta, it is sometimes necessary to add "-Wl,-ld_classic" to environment variables ESMF_CXXLINKOPTS, ESMF_CLINKOPTS, and ESMF_F90LINKOPTS to work around link errors. (For more details, see the related GitHub issue.)
v8.5.0

Reference Manual for Fortran
(html) (pdf)

Reference Manual for C
(html) (pdf)

User's Guide
(html) (pdf)

NUOPC Layer Reference Manual
(html) (pdf)

Building a NUOPC Model Manual
(html) (pdf)

ESMPy Documentation
(html) (pdf)

Regridding Status

Supported Platforms

API Changes since v8.4.0

API Test Coverage
07/26/23
  • The public Fortran API in this release is backward compatible with the last release of ESMF, 8.4.0, and patch releases 8.4.1 and 8.4.2. There were a few minor API changes, most of which do not require user code changes. The exception is a change to the Read() and Write() methods for multi-tile Arrays or Fields, where the tile number placeholder character in the fileName argument was changed from "#" to "*". A number of new interfaces were added. The complete list of API changes is summarized in a table showing interface changes since ESMF 8.4.0, including the rationale and impact for each change.
  • Some bit-for-bit changes are expected for this release compared to ESMF 8.4.0, and patch releases 8.4.1 and 8.4.2. This is based on test runs with the Intel compiler suite using options "-O2 -fp-model precise", and due to the following changes:
    • Fixed a problem in the first-order conservative weight calculation that might occur in some situations when a destination cell is a concave quadrilateral. The fix was made to both the 2D Cartesian and 2D spherical weight conservative calculation. Expected bit-for-bit changes:
      • Small changes to first-order conservative weights for destination cells that are concave. These changes can occur for either 2D Cartesian or 2D spherical Grids or Meshes.
    • Updated the ESMF cubed sphere coordinate generation algorithm to use R8 reals. This makes the algorithm more accurate and also aligns it better with the same algorithm used by NASA. Expected bit-for-bit changes:
      • Small changes to the coordinate values produced by ESMF_GridCreateCubedSphere() or ESMF_MeshCreateCubedSphere(), and also changes to anything which depends on those values (e.g. regridding weights to/from an ESMF generated cube sphere Grid).
  • No changes that affect regridding status were made to the ESMF regrid weight generation methods and applications. The ESMF tables summarizing the ESMF regridding status are unchanged.
  • All of the fixes and improvements released with patches 8.4.1 and 8.4.2 are included in release 8.5.0.
  • Significant improvements have been made to the Earth System Model eXecutable (ESMX) layer:
    • Component building has been integrated into the ESMX build system.
    • The build configuration has been enhanced through new build options.
    • The ESMX_Builder script has been added to simplify building ESMX applications.
    • The ESMX run configuration file has been updated to YAML formatting.
    • The names of the ESMX build and run configuration files can be customized.
    • The ESMX Data Component, which allows users to define import/export fields and prescribe data, has been added to the ESMX Layer.
  • A hierarchical configuration class (ESMF_HConfig) with Fortran API was implemented. ESMF_HConfig covers the YAML 1.2 standard and integrates with the existing configuration class ESMF_Config for backward compatibility.
  • The ESMF_Initialize() method now supports reading startup configurations from a YAML file.
  • The ESMF_InfoSet() interface was overloaded to allow setting ESMF_Info key value pairs from an ESMF_HConfig object.
  • The NUOPC methods that implement ingestion of PetList, RunSequence and Attribute information have been overloaded to support the YAML format through the ESMF_HConfig class.
  • A new named constant, ESMF_STATEINTENT_INTERNAL, for type(ESMF_StateIntent_Flag) was added. This new option allows a state to be marked exclusively for component internal access.
  • A generic geometry class (ESMF_Geom) was added to ESMF. A Geom object can be created from any object of the existing geometry classes (ESMF_Grid, ESMF_Mesh, ESMF_LocStream, and ESMF_XGrid). Having this class allows a user to pass a generic geometry object through a coupled system without knowing its exact type. To support greater flexibility in coupled systems, some ESMF operations are now supported on Geom (e.g. Field creation). More will be added as the need arises.
  • Creating a Mesh, or adding nodes to an existing Mesh, without specifying the nodeOwners argument now works, even if there are PETs with zero nodes. Previously this condition led to a hang inside the respective ESMF method.
  • Creating a Mesh without specifying node information now works, even if there are PETs with zero elements. Previously this condition led to a hang inside the respective ESMF method.
  • Grids that contain DEs of zero width are now supported in regridding. Previously using a Grid of this type with some regridding methods (e.g. ESMF_REGRIDMETHOD_CONSERVE) would result in an error.
  • The srcTermProcessing argument was added to the version of ESMF_FieldRegridStore() that operates on an XGrid. This fixes the issue where in some cases bit-for-bit reproducibility wasn't available in that version of ESMF_FieldRegridStore().
  • The RouteHandle reuse optimization in ESMF_FieldBundleRegridStore() was extended to include the Grid-to/from-Mesh and Mesh-to/from-Mesh combinations. This optimization is leveraged by the NUOPC_Connector and results in a significant reduction of cost (both time and memory) during model initialization when Fields on Meshes are present in the import- and/or exportStates.
  • All of the RedistStore() methods that take the optional srcToDstTransposeMap argument now support negative entries. The order of elements along each negative dimension entry is reversed.
  • Two new dataFillScheme options, "nan" and "snan", have been added to the ESMF_FieldFill() method. These options fill a Field with IEEE quiet and signaling NaN ("Not a Number") values, respectively.
  • An optional logical flag (isESMFAllocated) was added to the ESMF_FieldGet(), ESMF_ArrayGet(), and ESMF_LocalArrayGet() methods to allow the user to query whether the respective data allocation is held by ESMF.
  • A bug in the auto-detection of file type in the ESMF_RegridWeightGen application was fixed. Previously SCRIP grid files could sometimes be incorrectly identified as GRIDSPEC files.
  • The -t, --src_type, and --dst_type arguments of the ESMF_RegridWeightGen application that allow grid file types to be specified explicitly, have been re-enabled.
  • The Read and Write operations for multi-tile Arrays and Fields (e.g., for representing a cubed sphere grid as a six-tile grid) have been extended to permit I/O for Arrays / Fields with undistributed / ungridded dimensions. This change applies to the ESMF_ArrayRead(), ESMF_ArrayWrite(), ESMF_ArrayBundleRead(), ESMF_ArrayBundleWrite(), ESMF_FieldRead(), ESMF_FieldWrite(), ESMF_FieldBundleRead(), and ESMF_FieldBundleWrite() methods.
  • For Read() and Write() operations for multi-tile Arrays and Fields (e.g., for representing a cubed sphere grid as a six-tile grid), the file name placeholder character in the fileName string that stands for the tile number has been changed from "#" to "*". This change was needed to support obtaining the fileName string from ESMF Config file, where the "#" character is interpreted as the start of a comment.
  • Creating a cubed sphere grid under ESMPy with corner stagger by calling ESMF.Grid() with add_corner_stagger=True now works correctly. Previously this condition would lead to a run-time error.
  • ESMF is available through Spack, a popular package manager for High Performance Computing. A user can build ESMF, and its dependencies, in a convenient and standard way using the Spack tools. ESMF core team members are now maintainers of the official ESMF Spack package. To ensure a robust integration, the ESMF build procedure through Spack is automatically tested using GitHub Actions.
  • Intel oneAPI compiler support was improved and additional test combinations were added. A new ESMF User's Guide section discussing Intel compiler support was added to provide guidance with the classic to oneAPI transition.
  • Fujitsu compiler support was added under ESMF_OS=Linux. Use ESMF_COMPILER=fujitsu to access the option.
  • Attempting to write weight files from the ESMPy Regrid object when using filemode=FileMode.WITHAUX currently crashes.

    Platform-specific issues:

  • The GNU and Intel compilers require GCC>=4.8 for C++11 support (Intel uses the GCC headers). By default, ESMF uses the C++11 standard and cannot be downgraded. If you run into build issues due to the C++11 dependency, you must make sure a GCC>=4.8 is loaded.
  • Compiling ESMF with GCC version 8.1.0 triggers an internal compiler error in ESMF_HConfig.F90 due to the use of allocatable character variables. Earlier and later versions of GCC do not have this issue.
  • On Darwin, with the GNU gfortran+gcc combination, when building MPICH3 from source, it is important to specify the "--enable-two-level-namespace" configure option. By default, i.e. without this option, on Darwin, the produced MPICH compiler wrappers include a linker flag (-flat_namespace) that causes issues with C++ exception handling. Building and linking ESMF applications with MPICH compiler wrappers that specify this linker option leads to "mysterious" application aborts during execution.
  • On Darwin, with the Intel Fortran compiler, command line arguments cannot be accessed from ESMF applications when linked against the shared library version of libesmf. There is no issue when linked against the static libesmf.a version. Setting the environment variable ESMF_SHARED_LIB_BUILD=OFF, during the ESMF build, can be used as a work around for this issue.
  • There is an issue with intercepting the MPI calls for profiling on some of the supported platforms. This results in a single FAIL reported for ESMF_TraceMPIUTest.F90. The affected platforms are:
    • Catania: Darwin+GNU+mpich
    • Green: Darwin+gfortranclang+mpich/openmpi
    • Gaea: Unicos+GNU+cray-mpich
  • There is an issue with loading the libesmftrace_preload.so library on some of the supported platforms. This results in a reported CRASH for ESMF_TraceIOUTest.F90 and ESMF_TraceMPIUTest.F90. The affected platforms are:
    • Discover: Linux+GNU+intelmpi
    • Gaea: Unicos+Intel+cray-mpich
    • Gaea: Unicos+Intel+mpiuni
v8.4.2

Reference Manual for Fortran
(html) (pdf)

Reference Manual for C
(html) (pdf)

User's Guide
(html) (pdf)

NUOPC Layer Reference Manual
(html) (pdf)

Building a NUOPC Model Manual
(html) (pdf)

ESMPy Documentation
(html) (pdf)

Regridding Status

Supported Platforms
04/25/23
  • This patch release is backward compatible with ESMF 8.4.0 and 8.4.1
  • No bit-for-bit changes are expected for this release compared to ESMF 8.4.0 and 8.4.1.
  • No changes were made to the ESMF regrid weight generation methods and applications. The ESMF tables summarizing the ESMF regridding status are unchanged.
  • The ESMF profiler mode (activated by setting environment variable ESMF_RUNTIME_PROFILE=ON) has been made compatible with user code that calls the standard ESMF component methods from within OpenMP threaded regions. This is a rare scenario. However, it can be encountered e.g. under NOAA's Unified Forecast System (UFS) when enabling the threading option for NASA's GOCART chemistry component.
  • A linking issue reported for the Darwin.gfortranclang.default build configuration, when building ESMF under the Spack package manager, has been fixed.
  • Same as ESMF 8.4.1.
v8.4.1

Reference Manual for Fortran
(html) (pdf)

Reference Manual for C
(html) (pdf)

User's Guide
(html) (pdf)

NUOPC Layer Reference Manual
(html) (pdf)

Building a NUOPC Model Manual
(html) (pdf)

ESMPy Documentation
(html) (pdf)

Regridding Status

Supported Platforms
03/06/23
  • This patch release is backward compatible with ESMF 8.4.0.
  • No bit-for-bit changes are expected for this release compared to ESMF 8.4.0.
  • No changes were made to the ESMF regrid weight generation methods and applications. The ESMF tables summarizing the ESMF regridding status are unchanged.
  • A bug in the implementation of method ESMF_FieldGet() was fixed. The problematic code was accessing the optional, intent(out) "name" argument without the proper present() check. As a consequence, code not specifying an actual "name" argument when making this call was at risk of suffering from memory corruption issues. Due to the fact that the ESMF library internally is making calls to ESMF_FieldGet() without passing the "name" argument, it must be assumed that all user code is at risk of memory corruption issues when using ESMF 8.4.0.
  • A problem of incorrect ESMPy project metadata was fixed. This issue was causing failures during installation when using python's setuptools module v67.1.0 or later: ("configuration error: `project.maintainers[{data__maintainers_x}]` must not contain {'author'} properties"). The fix allows ESMPy installation to work with the latest setuptools. We recommend this release for anyone doing a new installation of ESMPy.
  • The NUOPC_Driver code responsible for stepping components through the initialization protocol has been optimized to reduce the amount of synchronization needed between components. This allows for a greater level of concurrency between components during initialization, resulting in a reduction of execution time.
  • A number of code adjustments were made to support Intel's LLVM based OneAPI compiler suite under the common ESMF_COMPILER=intel setting. Classic and OneAPI Intel compiler flavors are now supported under the same setting.
  • Same as ESMF 8.4.0.
v8.4.0

Reference Manual for Fortran
(html) (pdf)

Reference Manual for C
(html) (pdf)

User's Guide
(html) (pdf)

NUOPC Layer Reference Manual
(html) (pdf)

Building a NUOPC Model Manual
(html) (pdf)

ESMPy Documentation
(html) (pdf)

Regridding Status

Supported Platforms

API Changes since v8.3.0
10/28/22
  • The public Fortran API in this release is backward compatible with the last release of ESMF 8.3.0 and patch release ESMF 8.3.1. There were a few minor API changes, none of which require user code changes. The list of API changes is summarized in a table showing interface changes since ESMF 8.3.0, including the rationale and impact for each change.
  • The Python module name for ESMPy was changed in this release from "ESMF" to "esmpy" for better alignment with Python Enhancement Proposal (PEP) guidelines. This change will require user code changes! ESMPy users will have to do a global search and replace on their scripts to adjust to the module name change. Also notice that the internal ESMF source tree directory structure has changed, moving ./src/addon/ESMPy to ./src/addon/esmpy. This might affect anybody keeping an ESMF repository clone, and maintaining scripts that depend on the internal directory naming.
  • No bit-for-bit changes were observed for this release compared to release ESMF 8.3.0. This is based on test runs with the Intel compilers using options "-O2 -fp-model precise".
  • No changes were made to the ESMF regrid weight generation methods and applications. The ESMF tables summarizing the ESMF regridding status are unchanged.
  • All of the fixes and improvements released with patch 8.3.1 are also included in release 8.4.0. Of particular note is support for messages above the 2GiB limit by the ESMF VM communication layer.
  • The Earth System Model eXecutable (ESMX) layer was added to ESMF. ESMX removes technical hurdles that impede hierarchical NUOPC model testing. It includes a unified executable capable of driving multiple configurations of model components: single component, one-way forced, and fully dynamic two-way coupled. A simple YAML file is used to list the component build dependencies (CMake based). A standard NUOPC run configuration file is used to specify the processor layout, run sequence, the field dictionary, and model component options. ESMX supports hierarchical testing by allowing decentralized testing at the component level before integration into larger coupling systems, as well as testing on the integration level.
  • The concept of "Named Aliases" has been introduced. Regular aliases of deep ESMF objects continue being created using a simple assignment operator. Changing the name of a regular alias affects the object and thus all other aliases. Named aliases are created using the new ESMF_NamedAlias() function. Named aliases manage their own private name. Changing the name of a named alias does not affect the object or other aliases.
  • Using a Mesh that contains one or more elements with greater than 4 sides when creating an XGrid is now supported. Previously doing so would result in an error when data was transferred into or out of the XGrid.
  • LocStreams can now be created from a Cartesian Mesh file. Previously the attempt to do so would result in an error.
  • Attempting to create a Grid from a GRIDSPEC mosaic file that doesn't contain variables with standard_name set to "grid_tile_spec", "geographic_latitude", or "geographic_longitude" now triggers clear ERROR log messages before returning with an error. Previously no ERROR messages were logged, making this situation hard to debug.
  • Read and Write operations are now permitted for multi-tile Arrays and Fields. A common use case for this is representing a cubed sphere grid as a six-tile grid. This change applies to ESMF_ArrayRead(), ESMF_ArrayWrite(), ESMF_ArrayBundleRead(), ESMF_ArrayBundleWrite(), ESMF_FieldRead(), ESMF_FieldWrite(), ESMF_FieldBundleRead(), and ESMF_FieldBundleWrite(). Each of these methods reads or writes to multiple files in the multi-tile case. See the respective API documentation for details of how the tile-specific file names are constructed.
    Current limitations are:
    • For I/O of ArrayBundles and FieldBundles, all Arrays / Fields in the bundle must contain the same number of tiles;
    • I/O is not yet permitted for multi-tile Arrays / Fields with ungridded / undistributed dimensions;
    • I/O is currently only permitted for multi-tile Arrays / Fields with 1 DE per PET.
  • ESMF profiles now include a summary of the full application run time for convenience. Previously user instrumentation was required to introduce an end-to-end profiling level.
  • The configuration for ESMF_COMPILER=gfortranclang now works on Darwin systems (Mac OS X). Note that on Darwin, the system-level g++ invokes clang++, and unless you ensure that you have a true g++ early in your path (or build the MPI compiler wrappers to ensure that they wrap the true g++), you will end up using clang++ even if you think you are using the GNU C++ compiler. Setting ESMF_COMPILER=gfortranclang is correct for this typical situation. Attempting to use ESMF_COMPILER=gfortran when the C++ compiler is actually clang++ now issues an error message.
  • Attempting to write weight files from the ESMPy Regrid object when using filemode=FileMode.WITHAUX currently crashes.

    Platform-specific issues:

  • The GNU and Intel compilers require GCC>=4.8 for C++11 support (Intel uses the GCC headers). By default, ESMF uses the C++11 standard and cannot be downgraded. If you run into build issues due to the C++11 dependency, you must make sure a GCC>=4.8 is loaded.
  • On Darwin, with the GNU gfortran+gcc combination, when building MPICH3 from source, it is important to specify the "--enable-two-level-namespace" configure option. By default, i.e. without this option, on Darwin, the produced MPICH compiler wrappers include a linker flag (-flat_namespace) that causes issues with C++ exception handling. Building and linking ESMF applications with MPICH compiler wrappers that specify this linker option leads to "mysterious" application aborts during execution.
  • On Darwin, with the Intel Fortran compiler, command line arguments cannot be accessed from ESMF applications when linked against the shared library version of libesmf. There is no issue when linked against the static libesmf.a version. Setting the environment variable ESMF_SHARED_LIB_BUILD=OFF, during the ESMF build, can be used as a work around for this issue.
  • There is an issue with intercepting the MPI calls for profiling on some of the supported platforms. This results in a single FAIL reported for ESMF_TraceMPIUTest.F90. The affected platforms are:
    • Catania: Darwin+GNU+mpich
    • Green: Darwin+gfortranclang+mpich/openmpi
    • Gaea: Unicos+GNU+cray-mpich
  • There is an issue with loading the libesmftrace_preload.so library on some of the supported platforms. This results in a reported CRASH for ESMF_TraceIOUTest.F90 and ESMF_TraceMPIUTest.F90. The affected platforms are:
    • Discover: Linux+GNU+intelmpi
    • Gaea: Unicos+Intel+cray-mpich
    • Gaea: Unicos+Intel+mpiuni
v8.3.1

Reference Manual for Fortran
(html) (pdf)

Reference Manual for C
(html) (pdf)

User's Guide
(html) (pdf)

NUOPC Layer Reference Manual
(html) (pdf)

Building a NUOPC Model Manual
(html) (pdf)

ESMPy Documentation
(html) (pdf)

Supported Platforms

Regridding Status
09/21/22
  • This patch release is backward compatible with ESMF 8.3.0.
  • No bit-for-bit changes are expected for this release compared to ESMF 8.3.0.
  • No changes were made to the ESMF regrid weight generation methods and applications. The ESMF tables summarizing the ESMF regridding status are unchanged.
  • The ESMF communication layer now supports single messages that are above the previous 2GiB limit. This applies to the direct usage of ESMF_VM communication calls, but also extends to ESMF_RouteHandle based communication methods: Regrid, Redist, Halo, and SMM. User applications have been observed to push over the previous 32-bit message limit when each PET addresses a substantial amount of memory and calls into the ESMF communication methods. Before this fix, a user could experience application crashes in the MPI layer due to ESMF attempting to send messages that exceed the 32-bit size limit.
  • An issue inside ESMF_Info was fixed in which the data type and precision of attributes was not properly set when querying for an attribute through ESMF_InfoGet(). This led to a downstream issue of writing NetCDF attributes with a precision inconsistent with the attribute's precision inside the ESMF_Info object.
  • Fixed an issue in which the IO layer was incorrectly querying for the number of compute cores on a node. In some cases, the bug led to application hangs during IO operations (e.g. ArrayWrite) that span multiple nodes.
  • An issue was fixed in the IO layer that occurred when reading through PNetCDF into a destination with repeating elements (e.g. halo points). When used to read a Mesh from file, this problem caused bad coordinate values to be set in the created mesh. This in turn led to very poor performance when subsequently using the Mesh in a regrid weight generation operation.
  • The IO performance and memory requirement for the Read() operation was significantly improved by eliminating a costly check only relevant for Write() operations.
  • An issue observed under Darwin M1 systems with ESMP_Initailize() called by ESMPy was fixed.
  • The ESMPy syntax was corrected to allow for calls with property accessor Manager.local_pet.
  • The internal MOAB library included with ESMF now builds under old GCC 5.4.x.
  • Same as ESMF 8.3.0.
v8.3.0

Reference Manual for Fortran
(html) (pdf)

Reference Manual for C
(html) (pdf)

User's Guide
(html) (pdf)

NUOPC Layer Reference Manual
(html) (pdf)

Building a NUOPC Model Manual
(html) (pdf)

ESMPy Documentation
(html) (pdf)

Supported Platforms

Regridding Status

API Changes since v8.2.0
06/08/22
  • This release is backward compatible with the last release ESMF 8.2.0, for all the interfaces that are marked as backward compatible in the Reference Manual. There were API changes to a few unmarked methods that may require minor modifications to user code that uses these methods. The entire list of API changes is summarized in a table showing interface changes since ESMF 8.2.0, including the rationale and impact for each change.
  • No bit-for-bit changes were observed for this release compared to release ESMF v8.2.0 with Intel compilers using "-O2 -fp-model precise".
  • Tables summarizing the ESMF regridding status have been updated. These include supported grids and capabilities of the offline and integrated regridding.
  • A new section was added to the NUOPC Reference Manual describing the use of NUOPC_AddNestedState() for the coupling of multiple nests or multiple data sets between components.
  • The option to profile the execution time of each individual iteration through a NUOPC run sequence has been implemented in the Driver Component Metadata Profiling attribute. Setting the appropriate profiling bit results in a profile where the timing for each individual run sequence iteration is reported in the timing profile under a unique label. This information can be helpful for cases where the cost per iteration changes throughout the execution.
  • An issue in the ESMF_StateReconcile() method used by ESMF and NUOPC to generate a consistent object view across multiple components was fixed. The optimization implemented in v8.1.0 introduced the unintended behavior of switching out geom objects (Grid, Mesh, etc.) for Fields contained in States that are used in multiple ESMF_StateReconcile() operations. The incorrect association of geom objects with Fields resulted in unexpected results during subsequent operations using those Fields, such as creating a RouteHandle for regridding.
  • Progress was made in full adoption of MOAB as the internal mesh representation in ESMF. This includes updating the internal MOAB library included with ESMF to version 5.3.1 and the addition of combinatorial testing designed to ensure consistency and backward compatibility between the native mesh implementation and the MOAB-based implementation. Several consistency issues were resolved as a result of the new testing.
  • The previous version of ESMF_MeshCreate() from a file used to read all the node coordinate information on every processor. For large mesh files this global read can lead to high memory consumption and prevent reading in certain large meshes entirely. To reduce the memory footprint, a fully distributed read of mesh coordinate information was implemented. This change allows the creation of much larger meshes from file.
  • The nodeOwners argument for method ESMF_MeshCreate() and ESMF_MeshAddNodes() was made optional. This allows the user to defer specification of node ownership to ESMF in cases where a specific ownership assignment is not needed to match the application data distribution. When this argument is absent, ESMF generates a consistent assignment of node owners.
  • The ESMF_GridCreate() method that creates a new Grid from an existing Grid with new DistGrid was extended to optionally return a RouteHandle object. The RouteHandle allows subsequent calls into the new ESMF_GridRedist() method to redistribute the coordinate values from the original source grid to the new destination grid. This feature supports efficient handling of dynamically changing grids between components.
  • The implementation of the exchange grid (ESMF_XGrid) class that supports efficient conservative regridding between multiple grids on source and destination sides has been improved:
    • Element areas can now be set in a Field built on an XGrid by using the ESMF_FieldRegridGetArea() method. Previously, this capability was only supported for a Field built on a Grid or Mesh.
    • The algorithm used to generate interpolation weights was improved to guarantee that exchange grid cells are exactly overlapping exactly one cell on each side. Prior to this change, small numerical errors prevented this property from holding.
  • Added the optional --checkFlag argument to ESMF_RegridWeightGen application. This flag allows the user to turn on more expensive error checking that may not be appropriate for an operational run. Initially this flag turns on a check for grid self-intersection during conservative regridding.
  • The VM Epoch implementation now provides an option to reduce the memory pressure on the sending side PETs. By default, internal send buffers, once allocated, are kept until the VM is destroyed. This can lead to high memory pressure for cases where the same sending PETs participate in communication with multiple sets of receiving PETs. Setting keepAlloc=.false. when calling ESMF_VMEpochEnter(), instructs ESMF to immediately deallocate internal send buffers once the data has been transferred. This is in analogy to the handling of internal receive buffers with keepAlloc=.false. when calling ESMF_VMEpochExit(). The default remains .false. for both sides for efficiency.
  • Two internal fixed size buffers that caused issues when precomputing RouteHandles (e.g. via RegridStore()) for high-resolution, high PET count cases (~10,000 and above) were modified. The size of one of the buffers was doubled, while the other fixed size limitation was removed. The symptom of the first buffer size issue (now increased in size) was an error trace in the ESMF Log starting with "ESMCI_DELayout.C:9616 ESMCI::XXE::storeBufferInfo() Internal error: Bad condition - bufferInfoList overflow!!!". The second buffer size issue (now eliminated) was an error trace starting with "ESMCI_DELayout.C:8416 ESMCI::XXE::execReady() Internal error: Bad condition - sendnbCount out of range".
  • ESMF uses a library called ParallelIO (PIO) for its internal I/O operations, such as reading in mesh files and writing out fields. During this release, the version of PIO used internally was upgraded from a very outdated 1.x version to version 2.5. As a result, the binary output option ESMF_IOFMT_BIN is no longer supported and has been removed. A new option was also added to the ESMF build system to allow linking to an external build of the PIO library, as long as the external build is at least version 2.5.8. The upgrade eliminates the need to pass the compiler flags "-fallow-argument-mismatch -fallow-invalid-boz" when building with GNU 10.x or newer compilers. If the internal build of PIO is used, CMake version 2.8.12 or newer must be available in the system path. See the User's Guide for information about the environment variables used to configure PIO build and linking options.
  • An extra column was added to the ESMF profiler summary output, reporting the number of PEs (CPU cores) associated with the executing PETs. This information is helpful for example when profiling components that run with ESMF-managed threading. In the single-threaded case, each PET is associated with a single PE, and the number of PEs equals that of PETs. However, for the multi-threaded case, where N threads (e.g. OpenMP) are spawned under each PET, the number of PEs will be N times the number of PETs.
  • The ESMF_COMM build setting for MPICH has been reworked to better align with the current state of the MPICH project, and other ESMF_COMM settings. ESMF_COMM=mpich now covers the current MPICH versions 3 and 4. ESMF_COMM=mpich3 is still supported for backward compatibility. The old MPICH2 continues to be supported via ESMF_COMM=mpich2.
  • A problem with ESMF library installation linking for dylibs under Darwin was fixed. Previously the installed ESMF library remained dependent on files under the src directory of the ESMF build tree.
  • The FindESMF.cmake file included with ESMF, which is provided as a convenience to users that use CMake in their projects, has been updated. The module now searches ESMF_ROOT if ESMFMKFILE is not provided by the environment. Option USE_ESMF_STATIC_LIBS has been added to use the static ESMF library when building executables. This module requires CMake v3.12 and above.


Platform-specific issues:

  • The GNU and Intel compilers require GCC>=4.8 for C++11 support (Intel uses the GCC headers). By default, ESMF uses the C++11 standard and cannot be downgraded. If you run into build issues due to the C++11 dependency, you must make sure a GCC>=4.8 is loaded.
  • On Darwin, with the GNU gfortran+gcc combination, when building MPICH3 from source, it is important to specify the "--enable-two-level-namespace" configure option. By default, i.e. without this option, on Darwin, the produced MPICH compiler wrappers include a linker flag (-flat_namespace) that causes issues with C++ exception handling. Building and linking ESMF applications with MPICH compiler wrappers that specify this linker option leads to "mysterious" application aborts during execution.
  • On Darwin, with the Intel Fortran compiler, command line arguments cannot be accessed from ESMF applications when linked against the shared library version of libesmf. There is no issue when linked against the static libesmf.a version. Setting the environment variable ESMF_SHARED_LIB_BUILD=OFF, during the ESMF build, can be used as a work around for this issue.
  • There is an issue with intercepting the MPI calls for profiling on some of the supported platforms. This results in a single FAIL reported for ESMF_TraceMPIUTest.F90. The affected platforms are:
    • Catania: Darwin+GNU+MPICH3
    • Gaea: Unicos+GNU+cray-mpich
  • There is an issue with loading the libesmftrace_preload.so library on some of the supported platforms. This results in a reported CRASH for ESMF_TraceIOUTest.F90 and ESMF_TraceMPIUTest.F90. The affected platforms are:
    • Cori: Unicos+Intel+cray-mpich
    • Cori: Unicos+Intel+mpiuni
    • Discover: Linux+GNU+intelmpi
    • Gaea: Unicos+Intel+cray-mpich
    • Gaea: Unicos+Intel+mpiuni
    • Hera: Linux+GNU+intelmpi
    • Orion: Linux+GNU+mpiuni
ESMF_8_2_0

Reference Manual for Fortran
(html) (pdf)

Reference Manual for C
(html) (pdf)

User's Guide
(html) (pdf)

NUOPC Layer Reference Manual
(html) (pdf)

Building a NUOPC Model Manual
(html) (pdf)

ESMPy Documentation
(html) (pdf)

Supported Platforms

Regridding Status
10/28/21
  • This release is backward compatible with the last major release update, ESMF 8.1.0 and patch release ESMF 8.1.1, for all the interfaces that are marked as backward compatible in the Reference Manual. There were API changes to a few unmarked methods that may require minor modifications to user code that uses these methods. The entire list of API changes is summarized in a table showing interface changes since ESMF_8_1_0, including the rationale and impact for each change.
  • No bit-for-bit changes were observed for this release compared to release ESMF 8.1.0 and patch release ESMF 8.1.1, with Intel compilers using "-O2 -fp-model precise". However, the release contains code changes to the regridding implementation that have the potential to lead to bit-for-bit changes in regridding weights. Any release item with the potential to introduce a bit-for-bit change is indicated in the respective release note.
  • Tables summarizing the ESMF regridding status have been updated. These include supported grids and capabilities of the offline and integrated regridding.
  • The NUOPC RunSequence syntax was extended to support Alarm Blocks. An alarm block specifies the time interval at which the elements within the block are executed. This adds additional flexibility to the RunSequence approach, e.g. to write restart files at certain intervals that are multiples of the parent timestep.
  • Fields created on XGrids can now be used as either source, destination, or both when calling the general ESMF regrid methods (ESMF_FieldRegridStore(), ESMF_FieldRegrid(), ESMF_FieldBundleRegridStore(), ESMF_FieldBundleRegrid()). This enables the use of all ESMF regridding methods (bilinear, patch, etc.) and options (extrapolation, regridding status, etc.) when regridding to or from Fields on an XGrid. Prior to this release, regridding to or from Fields on an XGrid was only supported when going from one of the grids used to originally create the XGrid. Also, only conservative methods were supported.
  • A change in the 3D spherical bilinear weight calculation to handle more complex cells lead to a decrease in performance in releases 8.0.0, 8.1.0, and 8.1.1. The current release restores the performance to the level of ESMF 7.1.0r, and better, while retaining support for the complex cells. (Note that this change has the potential to introduce round off level changes in weights calculated for the 3D spherical bilinear method compared to previous ESMF releases. However, bit-for-bit testing with the Intel compiler using "-O2 -fp-model precise" did not detect any changes.)
  • A number of issues that were found with ESMF-managed threading under real application usage, as released with ESMF 8.1.0, have been addressed: (1) PETs that execute a threaded component are no longer instantiated as Pthreads by default but instead execute under the original MPI process. This resolves the issue of not being able to set an unlimited stack size. (2) Issues within the automatic garbage collection of ESMF objects have been resolved, which lead to memory corruption issues during ESMF_Finalize() when Grids or Meshes were transferred between threaded components. (3) Thread affinities and number of OpenMP threads are reset when exiting from a threaded component method, and global resource control can be turned on/off via the optional argument globalResourceControl during ESMF_Initialize().
  • It is now possible to override the defaults of a number of global ESMF settings by specifying an ESMF_Config file during ESMF_Initialize(). This is particularly useful for adjusting log specific settings, or to turn on/off resource control on the global VM.
  • A new section was added to the ESMF Reference Manual that discusses use of VMEpoch for asynchronous RouteHandle communications.
  • The VMEpoch feature allows sending PETs to fill the message queue up to the limit set by the MPI implementation. For message sizes where an MPI implementation chooses to use the EAGER protocol, this can lead to memory exhaustion on the receiving PETs. To prevent this issue, VMEpoch now limits the number of outstanding send cycles to ten by default. This default can be overridden by the user through the optional argument throttle to ESMF_VMEpochEnter().
  • The process of replacing the native ESMF mesh implementation with the MOAB library is continuing. The MOAB mesh backend is now available to ESMPy by calling Manager.set_moab(). This allows the user to test ESMPy regridding features with the new MOAB backend in preparation for MOAB becoming the default. Manager.moab returns a boolean value to indicate if the MOAB backend is currently in use. The default is to use the native ESMF mesh backend.
  • The ESMF_XGrid construction can lead to degenerate cells in cases where the source and destination grids have edges that are almost the same. Often these cells don't produce weights and are benign, but when weights are produced, they can lead to low accuracy results when transferring data to/from the XGrid.
  • Attempting to write weight files from the ESMPy Regrid object when using filemode=FileMode.WITHAUX currently crashes.


Platform-specific issues:

  • The GNU and Intel compilers require GCC>=4.8 for C++11 support (Intel uses the GCC headers). By default, ESMF uses the C++11 standard and cannot be downgraded. If you run into build issues due to the C++11 dependency, you must make sure a GCC>=4.8 is loaded.
  • For GNU compilers GCC>=10.x, the default Fortran argument mismatch checking has become stricter. This results in build failures in some of the code that comes with ESMF. Setting environment variable ESMF_F90COMPILEOPTS="-fallow-argument-mismatch -fallow-invalid-boz", during the ESMF build, can be used as a work-around for this issue.
  • On Darwin, with the GNU gfortran+gcc combination, when building MPICH3 from source, it is important to specify the "--enable-two-level-namespace" configure option. By default, i.e. without this option, on Darwin, the produced MPICH compiler wrappers include a linker flag (-flat_namespace) that causes issues with C++ exception handling. Building and linking ESMF applications with MPICH compiler wrappers that specify this linker option leads to "mysterious" application aborts during execution.
  • On Darwin, with the Intel Fortran compiler, command line arguments cannot be accessed from ESMF applications when linked against the shared library version of libesmf. There is no issue when linked against the static libesmf.a version. Setting the environment variable ESMF_SHARED_LIB_BUILD=OFF, during the ESMF build, can be used as a work around for this issue.
  • The ESMF_ArrayIOUTest unit test fails the binary read test on the S4 test system (Linux+Intel+IntelMPI).
  • There is an issue with intercepting the MPI calls for profiling on some of the supported platforms. This results in a single FAIL reported for ESMF_TraceMPIUTest.F90. The affected platforms are:
    • Catania: Darwin+GNU+MPICH3
    • Gaea: Unicos+GNU+cray-mpich
  • There is an issue with loading the libesmftrace_preload.so library on some of the supported platforms. This results in a reported CRASH for ESMF_TraceIOUTest.F90 and ESMF_TraceMPIUTest.F90. The affected platforms are:
    • Cori: Unicos+Intel+cray-mpich
    • Cori: Unicos+Intel+mpiuni
    • Discover: Linux+GNU+intelmpi
    • Gaea: Unicos+Intel+cray-mpich
    • Gaea: Unicos+Intel+mpiuni
    • Hera: Linux+GNU+intelmpi
    • Orion: Linux+GNU+mpiuni
ESMF_8_1_1

Reference Manual for Fortran
(html) (pdf)

Reference Manual for C
(html) (pdf)

User's Guide
(html) (pdf)

NUOPC Layer Reference Manual
(html) (pdf)

Building a NUOPC Model Manual
(html) (pdf)

ESMPy Documentation
(html) (pdf)

Supported Platforms

Regridding Status
04/23/21
  • This patch release is backward compatible with ESMF 8.1.0.
  • No bit-for-bit changes are expected for this release compared to ESMF 8.1.0.
  • No changes were made to the ESMF regrid weight generation methods and applications. The ESMF tables summarizing the ESMF regridding status are unchanged.
  • This patch release corrects a regression in the handling of user level OpenMP threads. See the Known Bugs column for details.
  • Same as ESMF_8_1_0 with the following exceptions:
    • The issue with OpenMP thread count being reset to 1 within all ESMF components has been fixed.
    • The PETs of all ESMF components, and any potentially created OpenMP threads under such PETs, are no longer pinned automatically to any specific PEs.
Platform-specific issues:
  • The same as ESMF_8_1_0.
ESMF_8_1_0

Reference Manual for Fortran
(html) (pdf)

Reference Manual for C
(html) (pdf)

User's Guide
(html) (pdf)

NUOPC Layer Reference Manual
(html) (pdf)

Building a NUOPC Model Manual
(html) (pdf)

ESMPy Documentation
(html) (pdf)

Supported Platforms

Regridding Status
03/29/21
  • This release is backward compatible with the last major release update, ESMF 8.0.0, for all the interfaces that are marked as backward compatible in the Reference Manual. There were API changes to a few unmarked methods that may require minor modifications to user code that uses these methods. The entire list of API changes is summarized in a table showing interface changes since ESMF_8_0_0, including the rationale and impact for each change.
  • Some bit-for-bit changes are expected for this release compared to release ESMF 8.0.0 and patch release ESMF 8.0.1. We observe the following impact with Intel compilers using "-O2 -fp-model precise":
    • Fixed a problem that could result in erroneously unmapped destinations when going from a very fine source grid to a coarse destination grid (e.g. 1/20 degree to 10x15 degree). Expected bit-for-bit changes:
      • ESMF_REGRIDMETHOD_CONSERVE_2ND: roundoff level changes in weight values because of a change in the order of calculation.
      • All regrid methods: Missing weights being added for very fine source grid to coarse destination grid regridding cases as this fix comes into play.
    • Fixed a problem where using the bilinear method between two identical grids doesn't result in an identity matrix for the regridding weights. It also improves the efficiency of the code when using bilinear or patch between identical grids. Expected bit-for-bit changes:
      • ESMF_REGRIDMETHOD_BILINEAR: small changes in regridding weights when a destination point exactly matches a source point.
      • ESMF_REGRIDMETHOD_PATCH: small changes in regridding weights when a destination point exactly matches a source point.
    • Fixed a problem where a set of points with latitudes set at exactly -90.0 are not all mapped to the same point. Expected bit-for-bit changes:
      • All regrid methods: Small weight changes when a point in the grid lies at exactly -90.0 latitude.
    • Optimized the creep fill so the memory doesn't increase as quickly for large numbers of creep levels. Expected bit-for-bit changes:
      • ESMF_EXTRAPMETHOD_CREEP: Small weight changes for the extrapolated destination locations.
    • Fixed an issue where in some cases creep fill weights can trigger an assertion in the code that redistributes weights to their final decomposition. Expected bit-for-bit changes:
      • ESMF_EXTRAPMETHOD_CREEP: Small weight changes for the extrapolated destination locations.
  • Tables summarizing the ESMF regridding status have been updated. These include supported grids and capabilities of the offline and integrated regridding, and numerical results of some specific test cases.
  • The ESMF_Info class was introduced as a replacement for ESMF_Attribute. ESMF_Info is based on a modern C++ JSON implementation to provide efficient key-value pair storage. Most of the legacy ESMF_Attribute API is preserved for backward compatibility.
  • ESMF is in the process of upgrading the internal mesh representation to use the MOAB mesh library developed by the U.S. Department of Energy. In this release, ESMF capabilities using MOAB have been significantly optimized and expanded, allowing for application-level testing of ESMF with MOAB as the underlying mesh representation. MOAB is built into the ESMF library by default, but its use must be enabled at run-time by calling ESMF_MeshSetMOAB(). When MOAB is activated, the following new capabilities are supported in this release:
    • The Mesh creation, conservative regridding, and bilinear regridding algorithms when MOAB is active have been optimized to reduce their memory use and expand the size of Meshes they can be used on.
    • Grids can now be explicitly converted to a Mesh when MOAB is active, using the ESMF_MeshCreate() method.
    • Grids can be used to do first-order conservative regridding using MOAB.
    • Grids can be used for bilinear regridding on cell centers or corners using MOAB.
  • The NUOPC Layer contained in this release has been improved in the following specific technical areas:
    • The NUOPC API has been simplified through the introduction of semantic specialization labels. The new approach leads to clearer and more concise NUOPC "cap" implementations that do not require specifying an Initialize Phase Definition (IPD) version or using the IPDvXXpY nomenclature when registering methods in the SetServices() method. Existing caps do not have to be re-written or updated, although updating to the new semantic specialization labels is recommended for existing and new NUOPC caps. The older IPD version based approach is supported for backward compatibility.
    • The NUOPC layer now provides features for resource control and handling of threaded components. This mechanism supports mixing of hybrid MPI+OpenMP components with different threading levels and mixing with standard MPI components on the same processing elements (PEs), i.e. cores. It allows each component to fully utilize HPC resources independently. Coupling between threaded components is supported automatically via the standard NUOPC Connectors.
    • The external NUOPC interface that supports interaction between an entire NUOPC application and a layer outside of NUOPC (e.g. a Data Assimilation system) was further refined. The associated prototype code (ExternalDriverAPIProto) has been updated to reflect the current status.
    • The NUOPC Profiling attribute, available in the Driver Metadata, Model Metadata, Mediator Metadata, and Connector Metadata, has been re-implemented. The NUOPC layer profiling features now integrate with the ESMF profiling infrastructure.
    • The NUOPC transfer protocol for geometry objects (Grid, Mesh, LocStream) has been made more efficient. Geometries used for multiple Fields are only transferred once, reducing the initialization overhead associated with the transfer.
    • Several optimizations were implemented in the NUOPC layer to reduce overhead. All applications using NUOPC benefit from these optimizations without requiring code changes.
    • Added the creep_nrst_d value to the extrapMethod NUOPC connection options. This is equivalent to the ESMF_EXTRAPMETHOD_CREEP_NRST_D option in ESMF_FieldRegridStore() discussed below.
  • Added the extrapolation option: CREEP_FILL to ESMPy. This option fills unmapped destination points by repeatedly moving data from mapped locations to neighboring unmapped locations.
  • The implementation of the ESMF_StateReconcile() method was redesigned to improve performance, and scalability. Most users do not interact with this method directly, however, the NUOPC initialization time has been reduced as a consequence, which is most pronounced in applications with large Field and PET counts.
  • Added a new extrapolation method called "nearest destination" to the regrid weight generation system. This capability fills destination points that were not filled by an initial regridding by using the nearest regridded destination point. The nearest destination method is accessible by specifying the extrapMethod=ESMF_EXTRAPMETHOD_NEAREST_D option in any of the ESMF_*RegridStore() methods or the --extrap_method nearestd option when using the ESMF_RegridWeightGen application).
  • Added a new extrapolation method called "creep nearest destination" to the regrid weight generation system. This capability fills destination points that were not filled by an initial regridding by first applying a creep fill extrapolation and then filling the remaining unmapped destination points using nearest destination extrapolation. The creep nearest destination method is accessible by specifying the extrapMethod=ESMF_EXTRAPMETHOD_CREEP_NRST_D option in any of the ESMF_*RegridStore() methods or the --extrap_method creepnrstd option when using the ESMF_RegridWeightGen application).
  • The creep fill extrapolation has been optimized so that it uses less memory per level extrapolated. This allows the creep fill method to extrapolate several times further into unmapped parts of the destination Field.
  • Added the optional checkFlag argument to ESMF_FieldRegridStore(). The intention behind this flag is to allow the user to turn on more expensive error checking that may not be appropriate for an operational run. Initially this flag turns on a check for grid self-intersection during conservative regridding.
  • The ESMF_MeshGet() call has been expanded to allow the user to query a full set of information for most Meshes. The exception is 2D Meshes with cells of more than four sides for which the element information (e.g. element connectivity) is not yet available.
  • A new shared memory feature was introduced that allows sharing of decomposition elements (DEs) between PETs that execute on the same single system image (SSI), i.e. node. This feature provides an efficient way to access data by reference between components that are running on the same PEs (cores), but with different threading levels. Both Field-level DE sharing and Array-level DE sharing are supported.
  • Key-value pair storage was added to the ESMF_Mesh and ESMF_LocStream classes through the overloaded ESMF_InfoGetFromHost() method.
  • A section discussing "Debugging of ESMF User Applications" has been added to the User's Guide. This section is designed to help users interpret error traces and efficiently locate issues in failing ESMF applications.
  • FieldBundles don't currently enforce that every contained Field is built on the same Grid, Mesh, LocStream, or XGrid object, although the documentation says that this should be so.
  • The packed FieldBundle implementation uses a concatenated string to create a base object. When this string has more than 255 characters, e.g. a large number of Fields with long individual names is packed, the base object is not created correctly resulting in incorrect behavior at the FieldBundle level.
  • When the ESMF regrid weight generation methods and applications are used with nearest destination to source interpolation method, the unmapped destination point detection does not work. Even if the option is set to return an error for unmapped destination points (the default) no error will be returned.
  • The ESMF regrid weight generation methods and applications do not currently work for source Fields created on Grids which contain a DE of width less than 2 elements. For conservative regridding the destination Field also has this restriction.
  • The ESMF regrid weight generation methods and applications do not currently work on Fields created on Grids with arbitrary distribution.
  • When using the ESMF_RegridWeightGen application to generate conservative weights for a Mesh with > 16 million cells, the weight file produced has some of its factors scrambled in a subtle way. This can lead to higher than usual conservation error (>1.0E-7).
  • When pole extrapolation is used during regridding operations and a quadrilateral cell that degenerates into a triangle is in the top or bottom row of the grid, the error "Condition {etopo->num_nodes == 4} failed..." is incorrectly returned.
  • The ESMF_GridCreate() interface that allows the user to create a copy of an existing Grid with a new distribution will give incorrect results when used on a Grid with 3 or more dimensions and whose coordinate arrays are less than the full dimension of the Grid (i.e. it contains factorized coordinates).
  • The ESMF_XGrid construction can lead to degenerate cells for cases where the source and destination grids have edges that are almost the same. Often these cells don't produce weights and are benign, but when weights are produced can lead to low accuracy results when transferring data to/from the XGrid.
  • When any of the Redist() methods are used inside a VMEpoch, the execution crashes with an MPI error.
  • The OpenMP thread count is being reset to 1 within all ESMF components. This affects user code that leverages OpenMP threading inside of components, and uses the OMP_NUM_THREADS environment variable to set the desired number of OpenMP threads. As a consequence the expected speed up from OpenMP threading in user code will not be present.
  • The PETs of all ESMF components, and any potentially created OpenMP threads under such PETs, are pinned to the PE on the respective shared memory node, corresponding to the PET number. As a consequence, even if a user overcomes the OpenMP thread count reset to 1 bug, e.g. by using omp_set_num_threads() API directly, the performance of OpenMP threaded user code is far below that of the expected speed up.
Platform-specific issues:
  • The GNU and Intel compilers require GCC>=4.8 for C++11 support (Intel uses the GCC headers). By default ESMF now uses the C++11 standard and cannot be downgraded. If you run into build issues due to the C++11 dependency, you must make sure a GCC>=4.8 is loaded.
  • For GNU compilers GCC>=10.x, the default Fortran argument mismatch checking has become stricter. This results in build failures in some of the code that comes with ESMF. Setting environment variable ESMF_F90COMPILEOPTS="-fallow-argument-mismatch -fallow-invalid-boz", during the ESMF build, can be used as a work around for this issue.
  • On Darwin, with the GNU gfortran+gcc combination, when building MPICH3 from source, it is important to specify the "--enable-two-level-namespace" configure option. By default, i.e. without this option, on Darwin, the produced MPICH compiler wrappers include a linker flag (-flat_namespace) that causes issues with C++ exception handling. Building and linking ESMF applications with MPICH compiler wrappers that specify this linker option leads to "mysterious" application aborts during execution.
  • On Darwin, with the Intel Fortran compiler, command line arguments cannot be accessed from ESMF applications when linked against the shared library version of libesmf. There is no issue when linked against the static libesmf.a version. Setting environment variable ESMF_SHARED_LIB_BUILD=OFF, during the ESMF build, can be used as a work around for this issue.
  • Currently the ESMPy interface to retrieve regridding weights from Python is only supported under the GNU compiler. On all other compilers the method will flag an error.
  • There is an issue with intercepting the MPI calls for profiling on some of the supported platforms. This results in a single FAIL reported for ESMF_TraceMPIUTest.F90. The affected platforms are:
    • Catania: Darwin+GNU+MPICH3
    • Gaea: Unicos+GNU+cray-mpich
  • There is an issue with loading the libesmftrace_preload.so library on some of the supported platforms. This results in a reported CRASH for ESMF_TraceIOUTest.F90 and ESMF_TraceMPIUTest.F90. The affected platforms are:
    • Discover: Linux+GNU+intelmpi
    • Gaea: Unicos+Intel+cray-mpich
    • Gaea: Unicos+Intel+mpiuni
    • Orion: Linux+GNU+mpiuni
ESMF_8_0_1

Reference Manual for Fortran
(html) (pdf)

Reference Manual for C
(html) (pdf)

User's Guide
(html) (pdf)

NUOPC Layer Reference Manual
(html) (pdf)

Building a NUOPC Model Manual
(html) (pdf)

ESMPy Documentation
(html) (pdf)

Supported Platforms

Regridding Status
05/21/20
  • This patch release is backward compatible with ESMF 8.0.0. Two new interfaces were added to the Fortran API: ESMF_VMEpochEnter() and ESMF_VMEpochExit().
  • No bit-for-bit changes are expected for this release compared to ESMF 8.0.0. This has been verified for a large number of regridding tests with Intel compilers using flags "-O2 -fp-model precise".
  • No changes were made to the ESMF regrid weight generation methods and applications. The ESMF tables summarizing the ESMF regridding status are unchanged.
  • The ESMF Virtual Machine (ESMF_VM) now supports message aggregation to improve performance for some very common communication patterns. Two new methods, ESMF_VMEpochEnter() and ESMF_VMEpochExit(), allow explicit use of this feature.
  • The generic NUOPC_Connector automatically takes advantage of VMEpoch message aggregation when used between components on disjoint petLists.
  • The ability to write/read ESMF RouteHandles to/from file was added to the ESMPy layer. This allows a user to perform the costly RouteHandle generation once, and re-use it in subsequent runs. This provides a more efficient approach to Regridding in the situation where the number of PETs does not change between runs.
  • The pole_kind parameter was added to allow specification of pole behavior when creating an ESMPy Grid.
  • Same as ESMF_8_0_0 with the following exceptions:
    • The race condition in the ESMF_FileRegrid() method and ESMF_Regrid application has been fixed. Now if the destination grid is a multi-tile grid in GRIDSPEC MOSAIC format and the tile is distributed into multiple PETs, the regridded field is written correctly into the output file.
    • Applying the sparse matrix multiplication to cases where the local data allocation is above the 32-bit limit now works reliably.
    • The ESMF_GridCreate1PeriDim() method can now be used to create a bipole connection on the lower side.
    • The ESMF_ArrayCreate() method now succeeds when called from inside a component, requesting DE sharing.
    • The ESMF_DistGridGet() method now correctly returns "de" and "tile" information for a "localDe".
    • ESMF now correctly links against the Fortran bindings of NetCDF, even when the C and Fortran bindings of NetCDF are provided in different locations.
    • The ESMF install target now reliably works when installing ESMF into a location that is shared with other library installations.
Platform-specific issues:
  • The same as ESMF_8_0_0 with the following exceptions:
    • The GNU and Intel compilers require GCC>=4.8 for C++11 support (Intel uses the GCC headers). By default ESMF now uses the C++11 standard and cannot be downgraded. If you run into build issues due to the C++11 dependency, you must make sure a GCC>=4.8 is loaded.
    • The Darwin.intelclang.default build configuration now works correctly.
ESMF_8_0_0

Reference Manual for Fortran
(html) (pdf)

Reference Manual for C
(html) (pdf)

User's Guide
(html) (pdf)

NUOPC Layer Reference Manual
(html) (pdf)

Building a NUOPC Model Manual
(html) (pdf)

ESMPy Documentation
(html) (pdf)

Supported Platforms

Regridding Status
10/15/19
  • This release is backward compatible with the last release, ESMF 7.1.0r, for all the interfaces that are marked as backward compatible in the Reference Manual. There were API changes to a few unmarked methods that may require minor modifications to user code that uses these methods. A number of new interfaces were added. The entire list of API changes is summarized in a table showing interface changes since ESMF_7_1_0r, including the rationale and impact for each change.
  • Some bit-for-bit changes are expected for this release compared to the last release, ESMF 7.1.0r. We observe the following impact with Intel compilers using "-O2 -fp-model precise":
    • Roundoff level differences in conservative regridding due to an improvement in an area calculation algorithm
    • Roundoff level differences in regridding when used on a Mesh created from a SCRIP format file that contains longitudes <=0 degrees. This change was due to removing a conversion for non-positive longitudes to improve consistency
    • Minor differences in 2nd order conservative regridding for cells that protrude outside their neighbors (e.g. a peninsula made up of a single cell) due to a bug fix in the weight calculation algorithm for that regridding method
  • Tables summarizing the ESMF regridding status have been updated. These include supported grids and capabilities of the offline and integrated regridding, and numerical results of some specific test cases.
  • Added an option to output component timing profiles in text format by setting the ESMF_RUNTIME_PROFILE environment variable. ESMF and NUOPC component phases are automatically instrumented and user-defined timed regions are also supported. Timing profiles can be written to the end of the ESMF PET log files, to separate per-PET text files, and/or to a single timing summary file. The summary file provides timing statistics across all the PETs. This provides a simple way to understand the relative cost (in terms of wall clock time) of each component in a coupled application.
  • The NUOPC Layer contained in this release has been improved in the following specific technical areas:
    • The NUOPC Field Dictionary can now be ingested from a community-based YAML file which resides in a dedicated public repository and can evolve independently of the NUOPC Layer. YAML Ain't Markup Language (YAML) is a human friendly, Unicode-based data serialization language for all programming languages.
    • A NUOPC_Driver can now be called from a higher level driver outside of NUOPC, going through an ESMF API. This is useful for systems that come with their own driver layer, but need to drive a NUOPC system (e.g. Data Assimilation).
    • The NUOPC run sequences now supports the "*" wildcard character in the "@" timestep syntax. This allows the timestep length to be set in code via driver specialization for run sequences ingested from a text file.
    • NUOPC now allows switching between different run sequence sections during execution. For example, this capability allows changing which components are active at different stages during a run. To enable this, the NUOPC_DriverIngestRunSequence() method now supports specification of a run duration, run sequence concatenation, and component run sequence elements outside loops.
    • ConnectionOptions is now an official NUOPC level Connector attribute. The attribute is read during RunSequence ingestion (for each Connector line), and appended by default to all of the CplList entries of the Connector.
    • The NUOPC_DriverAddComp() method now supports adding components with a SetVM() method, allowing the component to configure its own VM, e.g. for PET idling for PE reuse under threaded PETs.
    • Fields that are mirrored now arrive on the acceptor side with attributes set to reflect information about the provider side Field (TypeKind, GeomLoc, MinIndex, MaxIndex, ArbDimCount, GridToFieldMap, UngriddedLBound, UngriddedUBound). These attributes can then be used when creating Fields on the acceptor side.
    • The NUOPC Mesh transfer protocol was extended to correctly transfer either the node DistGrid, the element DistGrid, or both if present on the provider Mesh.
    • Completed the implementation of the sharing protocol. Whether to share the GeomObject and/or Field are now independent decisions, and all four possible combinations are supported. Sharing is available through component hierarchies and supports nested states (e.g. to use the Namespace and/or CplSet features of NUOPC).
    • The sharing protocol now checks whether the combination of provider and acceptor VMs allows Fields and/or GeomObjects to be shared between them and properly sets the share status attributes.
    • The generic Connector was optimized to also reuse Redist RouteHandles for Fields built on Meshes and LocStreams. Previously only the Redist for Fields built on Grids was optimized in this manner. Interleaving Redist and Regrid Field pairs is supported as before.
    • Implemented a communication optimized approach to propagating Field timestamps during Connector Run() to remove unnecessary synchronization between PETs. Only real data dependencies remain.
    • The NUOPC timestamp definition has been extended to include the ESMF calendar kind for more accurate timestamp validation between components.
    • Improved, standardized, and documented Verbosity attribute handling across all NUOPC generic component kinds. Preset Verbosity levels are available: "off", "low", "high", and "max".
    • The Diagnostic attribute option has been implemented across all NUOPC generic component kinds. This attribute allows the user to specify when Fields contained in the importState and exportState of a component are dumped to file when entering and exiting component methods (Initialize/Run/Finalize).
    • Several new NUOPC prototype (example) codes have been added:
      • AtmOcnCplSetProto demonstrates the use of the CplSet feature to support coupling multiple independent sets of Fields between components (e.g., multi-domain coupling).
      • AtmOcnLogNoneProto demonstrates the option to turn off ESMF PET log files completely.
      • AtmOcnMirrorFieldsProto demonstrates the NUOPC Field mirroring protocol.
      • CustomFieldDictionaryProto demonstrates the use of an external YAML file to populate the NUOPC Field Dictionary.
      • ExternalDriverAPIProto demonstrates how an external layer can drive a NUOPC driver component going through ESMF.
      • SingleModelOpenMPProto demonstrates the use of OpenMP threading inside a NUOPC model component and resource allocation through the generic SetVM method.
  • The following capabilities were added to the ESMF Python interface (ESMPy):
    • Added an in-memory weight generation option to the Regrid class, allowing re-use of weight vectors without writing them to netCDF. The weight arrays can be returned as NumPy objects or Python dictionary of weight vectors. This allows retrieval of the weights by source and destination key/value pairs.
  • The ESMF_Regrid application now supports additional options, and one option was removed:
    • Added --srcdatafile, --dstdatafile, and --tilefile_path options to support the GRIDSPEC Mosaic file format.
    • Added the --dst_loc option to support regridding on different stagger locations when the destination grid is in UGRID format and the regridding method is non-conservative.
    • Added the --check option to check the regridding results using a synthetic field, generated by an analytic function.
    • Removed the --user_areas option because none of the currently supported file formats provide user areas.
  • Added new features to the ESMF_FileRegrid() method and the ESMF_Regrid application:
    • Regridding of multiple variables
    • Multi-tile GRIDSPEC Mosaic file format with data stored in separate files, one per tile
    • 2nd order conservative regridding
    • Regridding to the corner stagger location if the regridding method is non-conservative and the destination file is in UGRID format
  • Added new features to the ESMF_RegridWeightGen application:
    • 1D network topology support in UGRID format
    • 2D Cartesian grid in CF Single Tile file format
    • Creep fill added as an extrapolation method
  • Added a new extrapolation method called "creep fill" to the entire regrid weight generation system. This capability allows the user to spread data from mapped destination points to neighboring unmapped destination points. This action can be repeated for a number of levels where at each level the data is spread from filled to neighboring unfilled points. The creep fill method is accessible by specifying the extrapMethod=ESMF_EXTRAPMETHOD_CREEP option in any of the ESMF_*RegridStore() methods or the --extrap_method creep option when using the ESMF_RegridWeightGen application).
  • Additional regridding methods are now supported when ESMF_Mesh is switched to be based on DOE's MOAB mesh library. The new supported regrid methods are: bilinear (when the source Field is not built on Grid), and nearest neighbor. All extrapolation methods, except for "creep fill" are supported.
  • RouteHandles can now be written to file via ESMF_RouteHandleWrite() and read back into memory (on the same number of PETs) via ESMF_RouteHandleCreate(). This provides an opportunity to lower the initialization cost e.g. for short production runs that repeatedly require regridding between the same grids.
  • The ESMF_GridCreateCubedSphere() and ESMF_GridCreateMosaic() methods now support irregular decompositions.
  • The ESMF_GridCreateCubedSphere() method can now apply the Schmidt transformation on the coordinates.
  • The ESMF_GridCreateMosaic() and the ESMF_GridCreate() method that reads a grid from file now support different coordinate typekinds.
  • The ESMF_GridCreate() method now automatically determines the correct file format if it was not explicitly specified as an argument.
  • The ESMF_GridCreate() method now supports grid creation from a GRIDSPEC Mosaic supergrid tile file.
  • The ESMF_LocStreamCreate() method now supports 1D network topology in UGRID format.
  • Added the capability to create an unstructured Mesh object from a structured Grid object. This allows components that internally work with a Mesh (e.g. a generic Mesh based mediator) to construct a Mesh from a transferred Grid.
  • The ESMF_MeshGet() method has been extended to allow the user to get mask and area information from an unstructured Mesh object.
  • Wrapped the ESMF_XGrid methods with ESMC bindings, to make them easily available to applications implemented in C.
  • Added a new ESMF_FieldBundleCreate() method that allows the creation of a packed FieldBundle. This is an initial capability with some limitations that will be addressed in a future release. The method takes a pre-allocated Fortran array pointer containing the memory of a set of interleaved fields. This is often how sets of fields are structured in model code and this capability allows ESMF to reference this memory directly. Interleaving along any dimension is supported. Packed FieldBundles support communication methods including redistribution, sparse matrix multiplication, and regridding. Currently the number of fields on source and destination must be the same, and permutations of fields are not supported. This means that the order of fields on source and destination must agree.
  • Implemented the ESMF_DECOMP_SYMMEDGEMAX option for cases where the number of elements is not evenly divisible by the number of decomposition elements. This option assigns the largest number of elements to the two edge DEs. It then progresses by assigning a descending number of elements to DEs as the center of the decomposition is approached from both sides. This produces a decomposition that is identical to the decomposition the FV3 model uses for this situation.
  • The ESMF_Array class now allows sharing of DEs between PETs that run on the same single system image (SSI). This feature can be used for shared memory data access between components running on different number of PETs (e.g. for threading) but are located on the same SSI.
  • The ESMF_Config class API has been extended. The ESMF_ConfigCreate() method now supports creating a Config object from a subsection of an existing Config object. This feature allows the consolidation of the content of multiple Config objects into a single configuration file.
  • The YAML-CPP parser has been included in the ESMF distribution. It is active by default, but can be turned off by setting ESMF_YAMLCPP=OFF.
  • Added support for C++11 standard. ESMF builds with the compiler default, but switches to C++11 if ESMF_YAMLCPP is enabled (default). The new ESMF_CXXSTD environment variable can be used to explicitly switch to specific C++ standards.
  • Simplified the use of CMake for projects using ESMF by providing a FindESMF.cmake file. This file is located under the new cmake subdirectory. It parses the esmf.mk file of an ESMF installation, exposing ESMF build variables as global variables accessible by other CMake modules.
  • The ESMF regression test suite can now be used to validate a pre-installed ESMF installation. This can be done by setting the ESMF_TESTESMFMKFILE environment variable to ON, and pointing ESMFMKFILE to the esmf.mk file of the ESMF installation to be validated.
  • Added an external demo ESMF_RegridCheck to test the ESMF_Regrid application with a set of test grid and data sets. In all the test cases, the input variables were constructed using an analytic function and the regridded destination variables were compared with that function to calculate the mean relative errors.
  • FieldBundles don't currently enforce that every contained Field is built on the same Grid, Mesh, LocStream, or XGrid object, although the documentation says that this should be so.
  • When the ESMF regrid weight generation methods and applications are used with nearest destination to source interpolation method, the unmapped destination point detection does not work. Even if the option is set to return an error for unmapped destination points (the default) no error will be returned.
  • The ESMF regrid weight generation methods and applications do not currently work for source Fields created on Grids which contain a DE of width less than 2 elements. For conservative regridding the destination Field also has this restriction.
  • The ESMF regrid weight generation methods and applications do not currently work on Fields created on Grids with arbitrary distribution.
  • There is a race condition in the ESMF_FileRegrid() method and the ESMF_Regrid application when the destination grid is of GRIDSPEC Mosaic format and >=12 PETs are used. This issue leads to intermittent failures in the external_demos tests for the GRIDSPEC_1x1_time_to_C48_mosaic_bilinear case when run on the 16 PET configuration.
  • When regridding from a very fine source grid (e.g. 1/20th degree) to a fairly coarse grid (e.g. 15 degree) using conservative regridding, source and destination fractions can be erroneously less than they should be.
  • When using creep fill extrapolation sometimes the extrapolation will not cross from a piece of a Field on one PET to a piece on another. This can lead to places in the Field not being properly extrapolated to.
  • The exact identity matrix is not returned when using bilinear regridding from a grid that extends all the way to the poles, to an identical grid.
  • The regridding can hang when doing bilinear or patch regridding on the centers of a mesh and the mesh contains a cell whose center is at the exact same position as one of the corners of the cell.
  • When using the ESMF_RegridWeightGen application to generate conservative weights for a Mesh with > 16 million cells, the weight file produced has some of its factors scrambled in a subtle way. This can lead to higher than usual conservation error (>1.0E-7).
  • When pole extrapolation is used during regridding operations and a quadrilateral cell that degenerates into a triangle is in the top or bottom row of the grid, the error "Condition {etopo->num_nodes == 4} failed..." is incorrectly returned.
  • Applying the sparse matrix multiplication to cases where the local data allocation is above the 32-bit limit will fail with a memory allocation error. This affects all Regrid(), Redist(), Halo(), and SMM() calls.
  • The ESMF_GridCreate() interface that allows the user to create a copy of an existing Grid with a new distribution will give incorrect results when used on a Grid with 3 or more dimensions and whose coordinate arrays are less than the full dimension of the Grid (i.e. it contains factorized coordinates).
  • Using the ESMF_GridCreate1PeriDim() method to create a grid with a bipole connection on the lower side (typically referring to the southern hemisphere) resulted in no connection there.
  • The ESMF_XGrid construction can lead to degenerate cells for cases where the source and destination grids have edges that are almost the same. Often these cells don't produce weights and are benign, but when weights are produced can lead to low accuracy results when transferring data to/from the XGrid.
  • The ESMF_ArrayCreate() crashes when used with pinflag=ESMF_PIN_DE_TO_SSI or pinflag=ESMF_PIN_DE_TO_SSI_CONTIG from within a component. The crash is from inside MPI with "invalid communicator". The "pinflag" option works correctly from the application level, i.e. in the context of the global VM.
  • Querying the ESMF_DistGridGet() method for "de" or "tile" information for a "localDe" will return incorrect results, and/or crash.
  • ESMF_AttributeWrite() has only been verified to work for ESMF standard Attribute packages. Non-standard Attribute packages may trigger a crash inside the ESMF_AttributeWrite() implementation.
  • For NetCDF installations that have the C and Fortran bindings installed in different locations, a NetCDF enabled build of ESMF does not correctly include the Fortran NetCDF library during linking.
  • When installing ESMF into a location that is shared with other libraries, it can happen that executing the ESMF install target fails with a "permission denied" error.
Platform-specific issues:
  • The GNU and Intel compilers require GCC>=4.8 for C++11 support (Intel uses the GCC headers). By default ESMF now uses the C++11 standard. If you run into build issues due to the C++11 dependency, you can either (1) make sure a GCC>=4.8 is loaded, or (2) set ESMF_YAMLCPP=OFF. In the latter case the YAML-dependent features in ESMF will not be available.
  • For GNU compilers GCC>=10.x, the default Fortran argument mismatch checking has become stricter. This will result in build failures. Setting environment variable ESMF_F90COMPILEOPTS="-fallow-argument-mismatch -fallow-invalid-boz", during the ESMF build, can be used as a work around for this issue.
  • On some systems with the PGI compiler, there is an issue with shared memory pointers between PETs on the same SSI. We see failures or crashes for Array tests that exercise this feature (ESMF_ArraySharedDeSSISTest.F90, ESMF_ArrayCreateGetUTest.F90) on the following platforms:
    • Hera/PGI-18.10.1
    • Gaea/PGI-16.5.0
    • Electra/PGI-17.1.0
    • Pleiades/PGI-17.1.0
    • Summitdev/PGI-19.7.0
    However, we do not observe these failures or crashes on:
    • Discover/PGI-14.1.0
    • Discover/PGI-17.7.0
  • On Summitdev/PGI-19.7.0 we see ESMF_XGridUTest.F90 unit test failures due to erroneously produce weights for source and destination grids that have edges that are almost the same.
  • On Discover/PGI-14.1.0 the ESMF_FieldRegridUTest.F90 and ESMF_FieldBundleCrGetUTest.F90 unit tests are failing.
  • Currently the ESMPy interface to retrieve regridding weights from Python is only supported under the GNU compiler. On all other compilers the method will flag an error.
  • On Darwin, with the Intel Fortran compiler, command line arguments cannot be accessed from ESMF applications when linked against the shared library version of libesmf. There is no issue when linked against the static libesmf.a version. Setting environment variable ESMF_SHARED_LIB_BUILD=OFF, during the ESMF build, can be used as a work around for this issue.
  • On some systems with the Cray compiler (CCE version 8.x), the ESMF library fails to build. The error can be prevented by setting the ESMF build environment variable ESMF_MOAB=OFF.
  • The Darwin.intelclang.default build configuration is broken.

In the old versioning scheme, used by all the releases below, only releases with an "r" or "rpX" after the version number were considered "public". All other releases were considered "internal". Public releases were shown in colored cells.

ESMF_7_1_0r

Reference Manual for Fortran
(html) (pdf)

Reference Manual for C
(html) (pdf)

User's Guide
(html) (pdf)

NUOPC Layer Reference Manual
(html) (pdf)

Building a NUOPC Model Manual
(html) (pdf)

ESMPy Documentation
(html) (pdf)

Supported Platforms

Regridding Status
03/08/18
  • This release is backward compatible with ESMF 7.0.2 and the last public release, ESMF 6.3.0rp1, for all the interfaces that are marked as backward compatible in the Reference Manual. There were API changes to a few unmarked methods that may require minor modifications to user code that uses these methods. The entire list of API changes is summarized in a table showing interface changes since ESMF_7_0_0, including the rationale and impact for each change.
  • Tables summarizing the ESMF regridding status have been updated. These include supported grids and capabilities of the offline and integrated regridding, and numerical results of some specific test cases.
  • Added a new tracing capability that outputs an application trace including entry and exit times of all ESMF and NUOPC run phases as well as user-defined code regions. Tracing is enabled at runtime by setting the ESMF_RUNTIME_TRACE environment variable to "ON". Traces can be imported into Cupid in order to visualize component execution sequences and produce timing profiles.
  • The NUOPC Layer contained in this release has been improved in the following specific technical areas:
    • Run sequences can now be read in from config file, or ingested from a text string in the driver.
    • Support for component hierarchies that can propagate field dependencies.
    • Support for Field reference sharing has been implemented. Sequential components can now share data by accessing the same memory allocation.
    • Grid transfer now supports different decomposition options, as well as multi-tile Grids like cubed sphere.
  • ESMF now offers methods to query if the framework has been initialized or finalized.
  • ESMF_GridComp is now able to hold multiple objects of type Grid, Mesh, LocStream, and XGrid as a convenience for the user.
  • A new create method was added for the ESMF Mesh class. This method allows the user to create a Mesh by specifying the coordinates of the corners for each element. This removes the need to specify node information (e.g. node id, ownership, etc.) when creating a Mesh. A Mesh created by this method can only be used in first-order conservative regridding.
  • The ESMF_FieldRegridStore() method now optionally outputs a regrid destination status. This destination status reports what happened to each destination point during the regridding (e.g. point not masked due to source mask).
  • The ESMF regrid weight generation methods and applications have been updated to support source and destination Grids which consist of more than one logically rectangular tile (e.g. a cubed sphere grid with six tiles).
  • The second-order conservative regrid method was added to the ESMF regrid weight generation methods and applications. Like the first-order conservative method, this method's main purpose is to preserve the integral of the field across the interpolation from source to destination. The second-order conservative method yields a smoother destination field compared to the first-order, and typically matches the source field more closely. This is especially beneficial when regridding from a coarse grid to a finer grid.
  • Added extrapolation to the ESMF regrid weight generation methods and applications. This capability is enabled by a new flag that indicates which extrapolation method should be applied to unmapped destination points. Currently there are two choices:
    • Nearest neighbor, in which an unmapped destination point's value is set to the same value as the closest source location.
    • Inverse distance weighted average, in which an unmapped destination point's value is set to a weighted average of the N closest source points' values. The weight for the later option is the reciprocal of the distance of the source point from the destination point raised to a power P. The user can choose N and P, but defaults are provided.
  • Added the ESMF_GridCreateCubedSphere() method to create a six tile cubed sphere ESMF_Grid object.
  • Added the ESMF_MeshCreateCubedSphere() method to create an ESMF_Mesh object with a cubed sphere topology.
  • Added the ESMF_GridCreateMosaic() method to create a multi-tile ESMF_Grid based on the definition from a GRIDSPEC mosaic file and its associated tile files.
  • Augmented the ESMF_RegridWeightGen application and the ESMF_RegridWeightGen() method to support multi-tile grids defined in GRIDSPEC mosaic file format.
  • Implemented automatic detection of the grid file type in the ESMF_RegridWeightGen() application and the ESMF_RegridWeightGen() method. The user no longer needs to provide this information explicitly.
  • The ESMF_RegridWeightGen application and the ESMF_RegridWeightGen() method now support the option to output a simple weight file that only contains weights and the source and destination indices.
  • Expanded the ESMF Unstructured Grid file format to allow the element connections to be stored in a 1D array. This reduces the file size significantly when some of the elements in the grid have a large number of connections.
  • Added the ability to write regridding weight files from memory using the ESMF_SparseMatrixWrite() method.
  • Added support to read regridding weights from file when generating ESMF_RouteHandles via ESMF_ArraySMMStore(), ESMF_ArrayBundleSMMStore(), ESMF_FieldSMMStore(), and ESMF_FieldBundleSMMStore().
  • RouteHandle based communications are now able to handle ungridded/undistributed dimensions that are interleaved with up to two gridded/distributed dimensions.
  • RouteHandles can now be transferred between different sets of PETs. This reduces initialization cost for applications that connect multiple components with the same data layout by allowing a single RouteHandle to be reused.
  • Dynamic masking has been implemented for the ESMF_FieldRegrid() and ESMF_ArraySMM() methods. User specified mask information is evaluated by the communication methods during RouteHandle execution, and a user provided handler method is called to modify the results of the operation.
  • Sequence indices, used to enumerate the elements of ESMF distributed data objects (Fields and Arrays), now offer 64-bit support. Unless specified by the user, ESMF automatically switches from 32-bit to 64-bit indices for objects with element counts that require the larger index type.
  • The temporary memory requirement of RouteHandle creation methods was reduced. Typical cases are expected to see a 25% reduction for regrid, and 75% reduction for halo operations.
  • Support for writing 64-bit NetCDF files has been added as an option to ESMF_ArrayWrite(), ESMF_ArrayBundleWrite(), ESMF_FieldWrite(), and ESMF_FieldBundleWrite().
  • Support for multiple DEs per PET was added to the ESMF_ArrayRead(), ESMF_ArrayWrite(), ESMF_ArrayBundleRead(), ESMF_ArrayBundleWrite(), ESMF_FieldRead(), ESMF_FieldWrite(), ESMF_FieldBundleRead(), and ESMF_FieldBundleWrite() methods.
  • The user now can specify the dimension names used when writing Fields, FieldBundles, Arrays, or ArrayBundles to NetCDF files, by setting predefined Attributes "ESMF_ATT_GRIDDED_DIM_LABELS" on Grid or DistGrid, and "ESMF_ATT_UNGRIDDED_DIM_LABELS" on Field or Array.
  • The user now can specify per-variable attributes when writing Fields, or Arrays to NetCDF files by setting Attributes on the objects.
  • The user now can specify global attributes when writing FieldBundles, or ArrayBundles to NetCDF files by setting Attributes on the objects.
  • Added Attributes to the LocStream class.
  • The ESMF_AttributeCopy() method can now be called on Grid, Field and FieldBundle objects.
  • The ESMF_LOGKIND_MULTI_ON_ERROR log option was added to suppress creation of ESMF log files unless an error was encountered. This is important for large PET counts where generating a large number of log files can impact performance.
  • Created ESMF_MOABCheck external demo to compare regrid weights generating using ESMF native mesh and a mesh built on the MOAB software.
  • The build system now allows use of the nc-config and nf-config NetCDF tools. This simplifies NetCDF enabled builds.
  • The build system now allows use of the pnetcdf-config PNetCDF tool. This simplifies PNetCDF enabled builds.
  • The Gfortran/gcc build configuration was added for the Windows/MinGW combination. This is now the default compiler configuration for this platform.
  • The PGI build configuration was added for the Darwin platform.
  • The following capabilities were added to the ESMF Python interface (ESMPy)
    • The ability to create a cubed sphere Grid object on 6 PETs and use it in regridding operations.
    • Support for Python 3.5/3.6.
    • The second-order conservative regrid method was added to the Regrid class. Like the first-order conservative method, this method's main purpose is to preserve the integral of the field across the interpolation from source to destination. This is especially beneficial when regridding from a coarse grid to a finer grid.
    • The extrapolation option was added to the Regrid class. This capability is enabled by a new flag that indicates which extrapolation method should be applied to unmapped destination points. Currently there are two choices:
      • Nearest neighbor, in which an unmapped destination point's value is set to the same value as the closest source location.
      • Inverse distance weighted average, in which an unmapped destination point's value is set to a weighted average of the N closest source points' values. The weight for the later option is the reciprocal of the distance of the source point from the destination point raised to a power P. The user can choose N and P, but defaults are provided.
    • The ability to write regridding weight files from memory using the Regrid class. This improves performance for cases where the RouteHandle is not needed.
    • The ability to read regridding weights from file when generating a RouteHandle with the RegridFromFile class.
  • FieldBundles don't currently enforce that every contained Field is built on the same Grid, Mesh, LocStream, or XGrid object, although the documentation says that this should be so.
  • The ESMF regrid weight generation methods and applications currently stop with an error if the source and destination grids are completely disjoint or if the source grid is completely masked, and the weight generation is set to ignore unmapped points. The correct behavior is to return successfully with an empty weight matrix.
  • When the ESMF regrid weight generation methods and applications are used with nearest destination to source interpolation method, the unmapped destination point detection does not work. Even if the option is set to return an error for unmapped destination points (the default) no error will be returned.
  • The ESMF regrid weight generation methods and applications do not currently work for source Fields created on Grids which contain a DE of width less than 2 elements. For conservative regridding the destination Field also has this restriction.
  • The ESMF regrid weight generation methods and applications do not currently work on Fields created on Grids with arbitrary distribution.
  • The ESMF_Regrid application and the ESMF_FileRegrid() method abort with an error when the destination file is in NetCDF4 format and the destination variable does not exist.
  • The ESMF_GridCreate() interface that allows the user to create a copy of an existing Grid with a new distribution will give incorrect results when used on a Grid with 3 or more dimensions and whose coordinate arrays are less than the full dimension of the Grid (i.e. it contains factorized coordinates).
  • ESMF_AttributeWrite() has only been verified to work for ESMF standard Attribute packages. Non standard Attribute packages may trigger a crash inside the ESMF_AttributeWrite() implementation.
Platform-specific issues:
  • When building the library with the PGI C++ compiler, the ESMF_MOAB=OFF environment variable may need to be set. The issue seems to be with the PGI pgc++ compiler front-end, and the associated libraries. The symptoms are errors during linking. The older (but deprecated) PGI pgCC compiler front-end does not seem to be affected.
  • Linux with PGI (Discover): Fails the ESMF_FieldBundleCrGetUTest.F90 unit test with ESMF_COMM=mvapich2 and ESMF_BOPT=O.
ESMF_7_0_2

Reference Manual for Fortran
(html) (pdf)

Reference Manual for C
(html) (pdf)

User's Guide
(html) (pdf)

NUOPC Layer Reference Manual
(html) (pdf)

Building a NUOPC Model Manual
(html) (pdf)

ESMPy Documentation
(html)

Supported Platforms

Regridding Status
10/31/17
  • The same as ESMF_7_0_1.
Platform-specific issues:
  • Same as ESMF_7_0_1 with the following exceptions:
    • All platforms using GNU g++: The library build failure with GNU g++ 6.1 (and newer) has been corrected. The default option of ESMF_MOAB=internal can safely be used.
    • Linux with intel (Theia): The problem of applications built with optimization level -O2 or higher, and linked against ESMF using the linker flags provided by esmf.mk, failing after the October system upgrade has been mitigated. The linker flags provided by esmf.mk no longer explicitly contain the -lsvml option.

Note for Platforms Using MPICH 3:
The mpich 3 series split apart the Fortran and C bindings. This requires ESMF_CXXLINKLIBS=-lmpifort to be set for a successful ESMF build using mpich. If this doesn't resolve your build issue with mpich, please contact ESMF support at esmf_support@list.woc.noaa.gov.
ESMF_7_0_1

Reference Manual for Fortran
(html) (pdf)

Reference Manual for C
(html) (pdf)

User's Guide
(html) (pdf)

NUOPC Layer Reference Manual
(html) (pdf)

Building a NUOPC Model Manual
(html) (pdf)

ESMPy Documentation
(html)

Supported Platforms

Regridding Status
12/12/16
  • Same as ESMF_7_0_0 with the following exceptions:
    • The overestimation of the internal buffer requirement in ESMF_StateReconcile() was fixed. It is now possible to run beyond the 10,000 core limit that previously caused ESMF_StateReconcile() to run out of memory on typical configurations.
Platform-specific issues:
  • The same as ESMF_7_0_0.

Note for Platforms Using MPICH 3:
The mpich 3 series split apart the Fortran and C bindings. This requires ESMF_CXXLINKLIBS=-lmpifort to be set for a successful ESMF build using mpich. If this doesn't resolve your build issue with mpich, please contact ESMF support at esmf_support@list.woc.noaa.gov.
ESMF_7_0_0

Reference Manual for Fortran
(html) (pdf)

Reference Manual for C
(html) (pdf)

User's Guide
(html) (pdf)

NUOPC Layer Reference Manual
(html) (pdf)

Building a NUOPC Model Manual
(html) (pdf)

ESMPy Documentation
(html)

Supported Platforms

Regridding Status
01/29/16
  • This release is backward compatible with the last public release, ESMF 6.3.0rp1, for all the interfaces that are marked as backward compatible in the Reference Manual. There were API changes to some of the unmarked methods that may require modifications to user code that uses these methods. The entire list of API changes is summarized in a table showing interface changes since ESMF_6_3_0rp1, including the rationale and impact for each change.
  • Tables summarizing the ESMF regridding status have been updated to include the new capability to perform regridding to/from a set of disconnected points (i.e. an ESMF LocStream) and to perform bilinear regridding on 3D spherical grids. These tables include supported grids and capabilities of the offline and integrated regridding, and numerical results of some specific test cases.
  • The NUOPC Layer contained in this release is a refinement of the previously released reference API. One of the design goals was to provide a more object oriented interface that clearly reflects the NUOPC concepts of Driver, Model, Connector, Mediator. The following specific technical areas were also improved:
    • The NUOPC Driver now allows child components to be dynamically added without prior setting of the total number of components in the driver. This flexibility simplifies coding of highly configurable coupled systems.
    • The consistent use of labels to reference components and their initialize/run/finalize phases simplifies interaction with the NUOPC Layer and makes application code more readable and less error prone.
    • The introduction of namespaces provides more control over the automatic field pairing implemented by the NUOPC Connector.
    • The NUOPC Compliance Checker has become more context aware and as a consequence produces more usable output.
    • The transfer protocol of Geom objects (e.g. ESMF Grids and Meshes) between components during initialization has been extended to cover ESMF LocStreams (a geometric type that can be used to represent a set of disconnected points).
  • The following capabilities were added to the ESMF Python interface (ESMPy):
    • Added ability to represent tripole Grids.
    • Added ability to create Fields on ungridded dimensions of a Grid or Mesh.
    • Added Field.read() which allows a Field to be filled with data from a CF-compliant NetCDF file.
    • Added LocStream class to represent observational data.
    • Added support for numpy array slicing for Grid, Mesh, LocStream, and Field (extraction of certain array elements into an alias of the original array).
    • Added support for Mesh cells with more than 4 sides.
    • Added public <object>.destroy() functions to provide explicit control over memory usage of persistent Python objects.
    • Added norm_type flag to allow the destination fraction to be included when calculating conservative regridding weights.
    • Added ignore_degenerate flag to allow users to ignore degenerate cells in Grids and Meshes.
    • Added coord_sys flag to Mesh.create() to allow switching between Cartesian and spherical coordinate systems.
    • Added read-only properties for all object attributes.
    • Added copy() for Grid, Mesh, LocStream, and Field to produce a shallow copy of the object.
    • Added ability to return Mesh element coordinates.
    • Added Anaconda packages for easy installation of ESMPy on Linux and Darwin. Specific package availability is recorded in a table of NESII Anaconda packages.
  • Added a new application ESMF_Regrid to support file-based regridding. This application regrids a data field defined in a grid file to another field defined in a different grid file. The regridded field will be written into the destination grid file. The grid files have to be in either UGRID or GRIDSPEC format and are limited to 2D grids.
  • ESMF LocStreams can now be used in regridding. This is useful for operations involving observational data, for example in data assimilation.
  • ESMF now offers the option to build the unstructured Mesh class on DOE's MOAB mesh library as an alternative to its native implementation. Meshes built on MOAB support a limited number of operations. They can be created, destroyed, queried for information, and used in the creation of an ESMF Field. The standard ESMF Field operations are expected to work on a Field build on a MOAB Mesh, however testing of these has been limited. In the case of ESMF regridding only the first order conservative regridding method is supported.
  • Added the ability for an element in an ESMF unstructured Mesh to consist of more than one polygon. This allows the representation of geographically disconnected areas (e.g. Hawaiian islands) as a single element. Meshes containing elements of this type will also work in the ESMF regrid weight generation methods and applications.
  • Added a new algorithm to compute bilinear interpolation weights in 3D. To use this algorithm set lineType=ESMF_LINETYPE_GREAT_CIRCLE when calling into the ESMF regrid weight generation methods and applications. Weights are then calculated along lines that follow the curve of a sphere instead of straight lines. This significantly reduces the error in 3D spherical grids with highly attenuated cells.
  • Expanded ESMF's first-order conservative interpolation algorithm to work on quadrilateral concave cells. Previously an error was returned under this condition.
  • Removed a restriction in the unstructured Mesh class so that now element coordinates can be set and retrieved for elements with more than four sides. Previously an error was returned under this condition.
  • Removed a restriction in the ESMF regrid weight generation methods and applications so that now non-conservative regridding can be done on Mesh elements with more than four sides. Previously an error was returned under this condition.
  • Extended the ESMF_RegridWeightGen application and the ESMF_MeshCreate() method to support 3D unstructured grid definitions provided in the UGRID file format.
  • The ESMF_RegridWeightGen application now supports additional options:
    • Added --src_loc and --dst_loc options to allow non-conservative regridding on the cell center locations of an unstructured grid defined in the UGRID or the ESMF unstructured file format.
    • Added --ignore_degenerate to ignore degenerate cells in either the source or the destination grid. This can be useful when cells that collapse into a point or line aren't a concern (e.g. when they are located in a region not used in regridding).
    • Added --line_type or -l to allow users to choose between great circle or Cartesian paths for the line between two points on a sphere during the calculation of bilinear interpolation weights.
  • Added two new shortcut Grid create methods: ESMF_GridCreate1PeriDimUfrm() and ESMF_GridCreateNoPeriDimUfrm(). Each of these methods creates a Grid and fills the coordinate arrays with uniformly spaced coordinate values.
  • Added a new interface to create an ESMF LocStream from a grid file. Currently, SCRIP, UGRID and ESMF unstructured file formats are supported. For a grid in either UGRID or ESMF format, the center coordinates or the corner coordinates can be chosen to construct the LocStream. For a grid in SCRIP format, the center coordinates will be used to construct the LocStream.
  • Expanded the ESMF_MeshCreate() method to support creation of a dual Mesh using the center coordinates defined in a UGRID or ESMF unstructured grid file.
  • The ESMF I/O methods that depend on the Parallel I/O (PIO) library are now turned on by default on platforms where certain Fortran 2003 features are supported (compilers not supported are Absoft and Lahey). The affected methods are ESMF_FieldBundleRead(), ESMF_FieldBundleWrite(), ESMF_FieldRead(), ESMF_FieldWrite(), ESMF_ArrayBundleRead(), ESMF_ArrayBundleWrite(), and ESMF_ArrayRead(), ESMF_ArrayWrite(). On platforms that do not fulfill the above system requirements, the PIO dependent methods will return an error when invoked.
  • Extended FieldBundle, Field, ArrayBundle, and Array I/O to support multiple DEs per PET and undistributed dimensions.
  • Extended the Log class to allow closing of a Log object (including the default Log) and reopening it under different name.
  • All of the distributed data classes (FieldBundle, Field, ArrayBundle, Array, Grid, DistGrid, DELayout) now offer the "noGarbage" option in their Destroy() method. This allows the user to deallocate all memory associated with an object, even the reference held by the ESMF garbage collection. Use of this new option has shown to be necessary to prevent out-of-memory conditions in applications that use many temporary objects (e.g. create new temporary objects each time step).
  • The ESMF VM class is now accelerator aware. The user can query the VM for information about available accelerator hardware. This feature is implemented on top of four different lower level software stacks: OpenCL, OpenACC, Intel MIC, and OpenMP (version 4). The appropriate software stack must be chosen when building the ESMF library by setting the ESMF_ACC_SOFTWARE_STACK environment variable in order to activate this new feature. See the User's Guide for details.
  • Added functionality to retrieve names of all ESMF deep objects using ESMF_AttributeGet(). This is the next step in implementing full access to object information through the ESMF Attribute interface.
  • Added the attnestflag argument to ESMF_AttributeGet() to allow users to specify whether they want to search through the entire Attribute hierarchy when retrieving an Attribute.
  • When transferring Mesh objects between components using the NUOPC_Connector, the ungridded dimensions of the Field object are ignored. This leads to errors in the NUOPC_Connector phase that precomputes the data transfer for the Field.
  • Reference sharing requests are ignored by the NUOPC_Connector for Fields with ungridded dimensions. The result is the use of the default copy behavior instead.
  • When a Mesh is transferred between components, e.g. via the NUOPC_Connector, where the destination component's PET list is not simply a truncated version of the source component's PET list, then the resulting Mesh will have internal inconsistencies. A Regrid or a second Mesh transfer operation involving this Mesh will result in an error.
  • The ESMF_StateReconcile() call overestimates the size requirement of an internal temporary buffer. The estimation error increases with core count and causes ESMF_StateReconcile() to run out of memory for large core counts (~10,000) on typical configurations.
  • FieldBundles don't currently enforce that every contained Field is built on the same Grid, Mesh, LocStream, or XGrid object, although the documentation says that this should be so.
  • The ESMF_FieldCompleteEmpty() function does not error out when trying to construct a Field with incorrect ungridded dimensions. Instead ESMF_SUCCESS, and an internally inconsistent Field object is returned.
  • When the ESMF regrid weight generation methods and applications are used with nearest destination to source interpolation method, the unmapped destination point detection does not work. Even if the option is set to return an error for unmapped destination points (the default) no error will be returned.
  • The ESMF regrid weight generation methods and applications do not currently work for source Fields created on Grids which contain a DE of width less than 2 elements. For conservative regridding the destination Field also has this restriction.
  • The ESMF regrid weight generation methods and applications do not currently work on Fields created on Grids with arbitrary distribution.
  • When the ESMF regrid weight generation methods and applications are used to do conservative interpolation from a concave quadrilateral source cell to a destination cell which only partially overlaps the source cell, a weight of infinity can sometimes be generated.
  • When the ESMF regrid weight generation methods and applications are used for bilinear regridding with an extremely coarse spherical source grid (e.g. 5 cells to cover the entire Earth), then destination points can be mapped to the wrong side of the sphere leading to incorrect weights.
  • ESMF_FieldRegridStore() that takes XGrid as an argument does not support regridding between Fields that are constructed on Grids for which ESMF_GridMatch() locally returns a value of ESMF_GRIDMATCH_EXACT or greater. This may result in unexpected local errors.
  • Using first-order conservative regridding through the XGrid does not ensure bit-for-bit results when the PET count changes.
  • The ESMF_Regrid application and the ESMF_FileRegrid() method abort with an error when the destination file is in NetCDF4 format and the destination variable does not exist.
  • The destination field written by the ESMF_Regrid application and the ESMF_FileRegrid() method is incorrect when the field contains one or more undistributed dimension.
  • The ESMF_Regrid application and the ESMF_FileRegrid() method does not set the destination field's initial value to the _FillValue of the source field when the destination field does not exist in the destination grid file. The unmapped points in the destination field are filled with zeros instead of the correct _FillValue.
  • The ESMF_GridCreate() interface which allows the user to create a copy of an existing Grid with a new distribution will give incorrect results when used on a Grid with 3 or more dimensions and whose coordinate arrays are less than the full dimension of the Grid (i.e. it contains factorized coordinates).
  • The ESMF_Grid creation interfaces that take the petMap argument (e.g. ESMF_GridCreate1PeriDim()) mistakenly return an error for inputs that put more than one DE on a PET, if the DEs are not mapped onto the PETs in sequential order.
  • The ESMF_GridCreate() from a UGRID file crashes when the node index is 0-based and the number of elements in a PET is less than four.
  • When the gridAlign argument is used during Grid creation to set the index space alignment to 1 without explicitly using the gridEdgeLWidth argument to set the index space lower edge width, then an incorrect lower edge width of 0 is used. This results in the Grid not having the correct index space padding for non-corner staggers.
  • The ESMF_XGridCreate() fails with error when created on Meshes or Grids where a source cell shares a single vertex with a destination cell.
  • The ESMF_<class>Read() and ESMF_<class>Write() methods for Field, FieldBundle, Array, and ArrayBundle are known to not work correctly when ungridded/undistributed dimensions are present. The results are situation dependent and range from errors returned by the method, incorrectly written or read data, and crashes.
  • ESMF_AttributeWrite() has only been verified to work for ESMF standard Attribute packages. Non standard Attribute packages may trigger a crash inside the ESMF_AttributeWrite() implementation.
  • ESMF does not link correctly with the Fortran libraries of the latest MPICH 3 series due to a separation of the libraries holding C and Fortran symbols.
Platform-specific issues:
  • All platforms using GNU g++: The library build fails with GNU g++ 6.1 (and newer) by default. The offending source code is in a section of code that can be disabled by setting ESMF_MOAB=OFF.
  • Linux with g95 (Fusion): Fails the ESMF_ArrayIOUTest.F90 unit test.
  • Linux with gfortran (Discover, Endeavour, Pleiades, Pluto): Fails the ESMF_ArrayIOUTest.F90 unit test.
  • Linux with gfortran (Pluto): Fails the ESMF_ArrayBundleIOUTest.F90 and the ESMF_ArrayIOUTest.F90 unit tests.
  • Linux with intel (Theia): After a system upgrade on Theia in October 2017, applications built with optimization level -O2 or higher, and linked against ESMF using the linker flags provided by esmf.mk, started failing. The symptoms were similar to memory corruption issues. It was determined that the problem could be triggered for a very simple Fortran program (not depending on ESMF), using ifort -O2 (or higher optimization), and explicitly linking against Intel's small vector math library via -lsvml. This finding was reported to Cray and Intel and a reproducer was made available to the vendor.
  • Linux with intel (Pleiades): Fails the ESMF_ArrayIOUTest.F90 unit test.
  • Linux with intel (Yellowstone): Fails the ESMF_DELayoutWorkQueueUTest.F90 unit test intermittently.
  • Linux with PGI (Discover, Fusion): Fails the ESMF_DELayoutWorkQueue.F90 unit test intermittently with ESMF_COMM=mvapich2.
  • Linux with PGI (Discover): Fails the ESMF_FieldBundleCrGetUTest.F90 unit test with ESMF_COMM=mvapich2 and ESMF_BOPT=O.
  • Linux with PGI (Discover): Fails the ESMF_ArrayIOUTest.F90 unit test with ESMF_COMM=openmpi and ESMF_BOPT=O.
  • Unicos with PGI (Titan): Fails the ESMF_ArrayIOUTest.F90 unit test with ESMF_BOPT=O.
  • Windows/Cygwin with g95 (Nix): Fails the ESMF_UtilUTest.F90 and TestHarness unit tests.
  • Windows/Cygwin with g95 (Nix): The command line tools ESMF_RegridWeightGen and ESMF_Regrid do not work.

Note for Platforms Using MPICH 3:
The mpich 3 series split apart the Fortran and C bindings. This requires ESMF_CXXLINKLIBS=-lmpifort to be set for a successful ESMF build using mpich. If this doesn't resolve your build issue with mpich, please contact ESMF support at esmf_support@list.woc.noaa.gov.
ESMF_6_3_0rp1

Reference Manual for Fortran
(html) (pdf)

Reference Manual for C
(html) (pdf)

User's Guide
(html) (pdf)

NUOPC Layer Reference Manual
(html)

ESMPy Documentation
(html)

Supported Platforms

Regridding Status
07/11/14
  • This release is backward compatible with ESMF 6.3.0r. There were two small API changes compared to 6.3.0r: the ESMF_NormType option was added to the ESMF regrid weight generation methods and applications, and a mistake in the intent of a dummy argument in ESMF_UtilIOGetCWD() was corrected. Both changes are listed in the table showing interface changes since ESMF_6_3_0r, including the rationale and impact of the changes.
  • Tables summarizing the ESMF regridding status have been updated to include the new ESMF_NormType option. These tables include supported grids and capabilities of the offline and integrated regridding, and numerical results of some specific test cases.
  • An optional command line argument "--no_log" was added to the ESMF_RegridWeightGen application. This allows the user to turn off the ESMF log files that are written by default.
  • The ESMF regrid weight generation methods and applications now allow the user to choose between two normalization types when generating conservative interpolation weights: destination area normalization (default) and destination fraction normalization.
  • The ESMF Python interface (ESMPy) is now part of the ESMF source distribution. The build and installation process for both ESMF and ESMPy remain separate in this release.
  • When building ESMF with NetCDF support enabled, the NetCDF C++ interface is no longer required.
  • New build configurations were added to support the Apple Clang/LLVM C++ compiler on both Linux and Darwin systems.
  • Support was added to compile the ESMF library for the Intel MIC architecture under Linux. This allows ESMF applications to execute natively on the Intel Xeon Phi accelerator hardware.
  • Same as ESMF_6_3_0r with the following exceptions:
    • The ESMF_RegridWeightGen application now correctly handles cases of SCRIP unstructured grid files that contain corner coordinates with small round off errors. Including those where for some corners the round off error along one dimension is above the tolerance value while it is below the tolerance for the other dimension.
    • The issue with ESMF_RegridWeightGen() crashing when either of the optional arguments srcFileType or dstFileType is not given has been fixed. Now it is okay to not specify the arguments.
    • The ESMF_AttributeUpdate() method now works correctly for the following situations:
      • Re-ordering objects in the container classes (State, FieldBundle, ArrayBundle) no longer causes ESMF_AttributeUpdate() to hang.
      • Making a small number of changes to a large Attribute package not longer leads to failures due to buffer issues.
      • A Grid shared by multiple Fields now correctly updates all Grid attributes on all PETs.
      • A State that is reconciled more than once to Components on exclusive petLists now correctly updates on all Components.
      • Any Attribute that is added during runtime and then modified will be correctly updated.
    • The debug print statement inside of the ESMF library that was triggered when calling ESMF_AttributeGet() has been removed. Calling ESMF_AttributeGet() many times will no longer cause issues with large volume of output.
    • The intent of the "pathName" argument in the ESMF_UtilIOGetCWD() method is now correctly specified as intent(out). This makes the method dependable.
    • The issue of building the unit tests and examples when build option ESMF_ARRAY_LITE is set to TRUE has been resolved. All test and example codes now build with this option.
Platform-specific issues:
  • Linux with PGI (Discover, Fusion, Jet): Systems tests hang intermittently with ESMF_COMM=mvapich2 due to a system level issue with the mvapich2/PGI combination.
  • Linux with PGI (Discover): Fails the ESMF_FieldBundleCrGetUTest.F90 unit test with ESMF_BOPT=O.
  • Linux with PGI (Discover, Fusion): Fails the ESMF_DELayoutWorkQueue.F90 unit test intermittently with ESMF_COMM=mvapich2.
  • Windows/Cygwin with g95: Fails the ESMF_UtilUTest.F90 and TestHarness unit tests when linked against the shared library version of ESMF.

Note for Platforms Using MPICH 3:
The mpich 3 series split apart the Fortran and C bindings. This requires ESMF_CXXLINKLIBS=-lmpifort to be set for a successful ESMF build using mpich. If this doesn't resolve your build issue with mpich, please contact ESMF support at esmf_support@list.woc.noaa.gov.
ESMF_6_3_0r

Reference Manual for Fortran
(html) (pdf)

Reference Manual for C
(html) (pdf)

User's Guide
(html) (pdf)

NUOPC Layer Reference Manual
(html) (pdf)

Supported Platforms

Regridding Status
01/31/14
  • This release is backward compatible with ESMF 6.2.0 and the last public release, ESMF 5.2.0rp3, for all the interfaces that are marked as backward compatible in the Reference Manual. There were API changes to a few unmarked methods that may require minor modifications to user code that uses these methods. The entire list of API changes is summarized in a table showing interface changes since ESMF_6_2_0, including the rationale and impact for each change.
  • Tables summarizing the ESMF regridding status have been updated. These include supported grids and capabilities of the offline and integrated regridding, and numerical results of some specific test cases.
  • The NUOPC Layer contained in this release has been improved in the following specific technical areas:
    • The NUOPC Component Explorer was added. This is a run-time tool that can be used to analyze the compliance status of a Component.
    • Standard Component dependencies have been formalized and documented, making Components more interoperable on the build level.
    • The resolution of data dependencies during initialization was extended to cover components that run on different petLists.
    • The transfer of Grid and Mesh objects between Components is now supported during initialization.
  • The tools that allow Components to be made available as web-based services now provide the capability to use a job scheduler to manage the execution and resource allocation for the service.
  • The capability to create logically rectangular Grid objects from NetCDF files in SCRIP format was added to ESMPy, the ESMF Python interface. Documentation and downloads are available on the ESMPy web page.
  • Distributed objects Fields, Arrays, Grids, and Meshes created in one Component can now be transferred to another Component, even when the Components are defined on different petLists.
  • The "--check" option was added to the ESMF_RegridWeightGen application to calculate the interpolation and conservation error from regridding with the weights in the output file. The error is calculated using an analytical test field.
  • The ESMF_RegridWeightGen() interface was overloaded to allow regridding between grids with user specified element and optional nodal distribution. The grids must be supplied in the SCRIP format.
  • The ESMF_FieldRegridStore() call has been extended to work on Fields built on Meshes containing elements with more than four sides (e.g. pentagons and hexagons).
  • The ESMF_FieldRegridStore() call has been extended so that non-conservative interpolation methods (bilinear, patch, and nearest-neighbor) can be performed on Fields built on the Mesh element location (ESMF_MESHLOC_ELEMENT).
  • The ESMF_FieldRegridStore() call now allows the user to choose between great circle or Cartesian paths for the line between two points on a sphere during the calculation of bilinear interpolation weights.
  • The ESMF_FieldEmptySet() method now allows the Grid, Mesh, LocStream, or XGrid object to be replaced in a partially completed Field.
  • The ESMF_MeshCreate() interfaces have been extended to allow the user to specify a coordinate system. This information is used by the ESMF_FieldRegridStore() call to automatically determine the geometry of the source and destination Meshes.
  • The ESMF_MeshCreate() and ESMF_MeshAddElements() methods have been extended to allow the user to create a Mesh containing elements with more than four sides (e.g. pentagons and hexagons).
  • The ESMF_MeshCreate() and ESMF_MeshAddElements() methods have been extended to allow the user to optionally set element center coordinates. This information is used when doing non-conservative regridding on Fields built on the Mesh element location (ESMF_MESHLOC_ELEMENT).
  • The ESMF_MeshCreate() which allows the user to to create a copy of an existing Mesh, but with a different distribution, now allows the user to optionally omit the node distribution.
  • The ESMF_MeshCreate() method has been extended to allow generation of the nodal mask when reading from a UGRID file.
  • The ESMF_MeshCreate() method has been updated to support the new UGRID version 0.9.0 conventions.
  • The ESMF_MeshCreate() method has been extended to allow Meshes to be created from a grid file with user specified element and optional nodal distribution.
  • The ESMF_GridCreate() interface was overloaded to allow Grids to be created from a grid file with user specified distribution.
  • The ESMF_AttPack type was introduced, allowing a user to retrieve a handle to an ESMF Attribute package. This is a more compact way of working with Attribute packages.
  • The ESMF_UtilIOGetCWD() interface was added to allow retrieval of the absolute path and name of the current working directory.
  • A new parallel application ESMF_Scrip2Unstruct was added to convert a SCRIP format grid file to an unstructured ESMF format or a UGRID format grid file. This allows the user to execute the potentially expensive file format conversion operation once instead of every time the ESMF_RegridWeightGen application is called. The SCRIP file can be either unstructured or logically rectangular. The application has an option to generate a dual mesh, which is needed to support non-conservative regridding on cell centers.
  • When transferring Mesh objects between components using the NUOPC_Connector, the ungridded dimensions of the Field object are ignored. This leads to errors in the NUOPC_Connector phase that precomputes the data transfer for the Field.
  • Reference sharing requests are ignored by the NUOPC_Connector for Fields with ungridded dimensions. The result is the use of the default copy behavior instead.
  • When a Mesh is transferred between components, e.g. via the NUOPC_Connector, where the destination component's PET list is not simply a truncated version of the source component's PET list, then the resulting Mesh will have internal inconsistencies. A Regrid or a second Mesh transfer operation involving this Mesh will result in an error.
  • The ESMF_StateReconcile() call overestimates the size requirement of an internal temporary buffer. The estimation error increases with core count and causes ESMF_StateReconcile() to run out of memory for large core counts (~10,000) on typical configurations.
  • FieldBundles don't currently enforce that every contained Field is built on the same Grid, Mesh, LocStream, or XGrid object, although the documentation says that this should be so.
  • The ESMF_FieldCompleteEmpty() function does not error out when trying to construct a Field with incorrect ungridded dimensions. Instead ESMF_SUCCESS, and an internally inconsistent Field object is returned.
  • When the ESMF regrid weight generation methods and applications are used with nearest destination to source interpolation method, the unmapped destination point detection does not work. Even if the option is set to return an error for unmapped destination points (the default) no error will be returned.
  • The ESMF regrid weight generation methods and applications do not currently work on Fields created on Grids which contain a DE of width less than 2 elements.
  • The ESMF regrid weight generation methods and applications do not currently work on Fields created on Grids of arbitrary distribution.
  • The ESMF regrid weight generation methods and applications can erroneously report unmapped cells for the conservative regridding method when the destination side was constructed on an unstructured Mesh containing elements with more than four sides.
  • When the ESMF regrid weight generation methods and applications are used for bilinear regridding with an extremely coarse spherical source grid (e.g. 5 cells to cover the entire Earth), then destination points can be mapped to the wrong side of the sphere leading to incorrect weights.
  • The ESMF_RegridWeightGen application has issues with some cases of SCRIP unstructured grid files that contain corner coordinates with small round off errors. Previously a small tolerance was introduced to determine if corner points of separate cells match to connect together neighboring cells. However, a problem can still occur in cases where for some corners the round off error along one dimension is above the tolerance value while it is below the tolerance for the other dimension. This can result in unexpected unmapped destination points in the regridding.
  • The ESMF_RegridWeightGen() method crashes when either or both of the optional arguments srcFileType or dstFileType are not given.
  • ESMF_FieldRegridStore() that takes XGrid as an argument does not support regridding between Fields that are constructed on Grids for which ESMF_GridMatch() locally returns a value of ESMF_GRIDMATCH_EXACT or greater. This may result in unexpected local errors.
  • ESMF_FieldRegridStore() can not be used with non-conservative interpolation methods (bilinear, patch, or nearest-neighbor) for Fields built on the element location of a Mesh (ESMF_MESHLOC_ELEMENT) that contains elements with more than four sides. The call will return with error if used this way.
  • A larger than expected difference in integrals (8.7E-10) is occurring when using the conservative regridding between a 1 deg lat lon grid and a 2.5 deg lat lon grid. This problem appears isolated to a subset of the cases when grid cell edges align, but not exactly due to round off. It is recommended that users test the integration of their conservative interpolation under these circumstances.
  • The ESMF_FieldRegridStore() call does not return an error when the regrid method is bilinear or patch and an input Field is built on a Grid or Mesh that contains a degenerate cell (i.e. one that collapses to a line or a point).
  • Using first-order conservative regridding through the XGrid does not ensure bit-for-bit results when the PET count changes.
  • The ESMF_Grid creation interfaces that take the petMap argument (e.g. ESMF_GridCreate1PeriDim()) mistakenly return an error for inputs that put more than one DE on a PET, if the DEs are not mapped onto the PETs in sequential order.
  • The ESMF_GridCreate() interface which allows the user to create a copy of an existing Grid with a new distribution will give incorrect results when used on a Grid with 3 or more dimensions and whose coordinate arrays are less than the full dimension of the Grid (i.e. it contains factorized coordinates).
  • The ESMF_GridCreate() from a UGRID file crashes when the node index is 0-based and the number of elements in a PET is less than four.
  • When the gridAlign argument is used during Grid creation to set the index space alignment to 1 without explicitly using the gridEdgeLWidth argument to set the index space lower edge width, then an incorrect lower edge width of 0 is used. This results in the Grid not having the correct index space padding for non-corner staggers.
  • The ESMF_MeshCreate() interface that allows the user to to create a copy of an existing Mesh, but with a different distribution will return an error if the element distribution is not specified.
  • The ESMF_XGridCreate() fails with error when created on Meshes or Grids where a source cell shares a single vertex with a destination cell.
  • The ESMF_<class>Read() and ESMF_<class>Write() methods for Field, FieldBundle, Array, and ArrayBundle are known to not work correctly when ungridded/undistributed dimensions are present. The results are situation dependent and range from errors returned by the method, incorrectly written or read data, and crashes.
  • The ESMF_AttributeGet() interface can return incorrect values when using the ESMF_AttGetCountFlag set to ESMF_ATTGETCOUNT_ATTRIBUTE.
  • The ESMF_AttributeUpdate() method does not work correctly for the following situations:
    • Re-ordering objects in the container classes (State, FieldBundle, ArrayBundle) can cause ESMF_AttributeUpdate() to hang.
    • When a small number of changes is made to a large Attribute package a failure can occur with a "buffer too short" log message.
    • A Grid shared by multiple Fields may not have all Grid attributes updated on all PETs.
    • A State that is reconciled more than once to Components on exclusive petLists will not be updated on any but the first Component.
    • Any Attribute that is added during runtime and then modified will not be updated correctly.
  • ESMF_AttributeWrite() only works for ESMF standard Attribute packages.
  • When calling the ESMF_AttributeGet() method to query a character string attribute, a debugging print statement is triggered inside of the ESMF library. This can lead to a problematic volume of output when ESMF_AttributeGet() is called many times (e.g. from within a loop).
  • The intent of the "pathName" argument in the ESMF_UtilIOGetCWD() method is incorrectly specified as intent(in). This makes the method undependable.
  • The unit tests and examples fail to compile when build option ESMF_ARRAY_LITE is set to TRUE.
Platform-specific issues:
  • Linux with PGI (Discover, Fusion, Jet): Systems tests hang intermittently with ESMF_COMM=mvapich2 due to a system level issue with the mvapich2/PGI combination.
  • Darwin with g95 (Haumea): Fails the ESMF_LocStreamEx.F90 example with ESMF_BOPT=g for ESMF_COMM=mpiuni.
  • Darwin with gfortran (Haumea): Fails the ESMF_LocStreamUTest.F90 and ESMF_MeshUTest.F90 unit tests and the ESMF_LocStreamEx.F90 example with ESMF_BOPT=O for ESMF_COMM=mpich2 and ESMF_COMM=mvapich2.
  • Linux with PGI (Fusion): Fails the ESMF_FieldRegridUTest.F90 unit test with ESMF_COMM=mvapich2.
  • Windows/Cygwin with g95: Fails the ESMF_UtilUTest.F90 and TestHarness unit tests when linked against the shared library version of ESMF.
ESMF_5_2_0rp3

Reference Manual for Fortran
(html) (pdf)

Reference Manual for C
(html) (pdf)

User's Guide
(html) (pdf)

Supported Platforms

Regridding Status
09/30/13
  • This release is backward compatible with ESMF 5.2.0rp2. There are no API changes.
  • The ESMF regridding status tables have not changed since 5.2.0rp1/rp2.
  • Same as ESMF_5_2_0rp2 with the following exceptions:
    • The ESMF_GridCreate() call, which allows the user to create a Grid from a SCRIP format file and a DistGrid, no longer creates the Grid using the wrong coordSys (Cartesian instead of spherical). A Grid created with this call will now work fine in methods (e.g. ESMF_FieldRegridStore()) which use the coordSys to determine the shape of the Grid.
    • The instability that led to undefined behavior (e.g. segmentation fault) for all the sparse matrix store calls (i.e., ESMF_FieldBundleSMMStore(), ESMF_FieldSMMStore(), ESMF_ArrayBundleSMMStore(), and ESMF_ArraySMMStore()) for very large problem sizes (~15 million factors in the sparse matrix) has been fixed.
    • The decomposition scheme of the original DistGrid (indicated by the ESMF_Decomp_Flag) is now correctly duplicated by the ESMF_DistGridCreate() methods that take a DistGrid input argument.
  • Also see SourceForge bugs list.
Platform-specific issues:
  • The same as ESMF_5_2_0rp2.
ESMF_6_2_0

Reference Manual for Fortran
(html) (pdf)

Reference Manual for C
(html) (pdf)

User's Guide
(html) (pdf)

NUOPC Layer Reference Manual
(html) (pdf)

Supported Platforms

Regridding Status
5/17/13
  • This release is backward compatible with ESMF 6.1.1 for all the interfaces that were previously marked as backward compatible. There were API changes to unmarked methods, and new methods were added. All of these changes are backward compatible with ESMF 6.1.1. The entire list of API changes is summarized in a table showing interface changes since ESMF_6_1_1, including the rationale and impact for each change.
  • Tables summarizing the ESMF regridding status have been updated. These include supported grids and capabilities of the offline and integrated regridding, and numerical results of some specific test cases.
  • This release contains a reference version of the NUOPC Layer software. The NUOPC Layer provides generic components that represent Drivers, Models, Mediators, and Connectors. These pre-fabricated, customizable components allow users to easily build interoperable ESMF applications. New web pages, a Reference Manual, and many examples are available to help users learn about the software.
  • Specific technical improvements to the NUOPC Layer in this release include:
    • The default entries in the Field Dictionary were reduced to those entries that are compliant with the Climate and Forecast (CF) Convention. It is the intention to grow the number of default entries over time, based on a clear relationship with CF.
    • Support for dynamic resolution of data-dependencies between components during initialization was added.
    • Support for compatibility checking of components that interact on multiple timescales was added.
  • The capability of retrieving coordinates from an unstructured Mesh object created from a NetCDF file in SCRIP or UGRID format was added to the ESMF Python interface. More information about the Python interface is available at the ESMP web page.
  • Support to write out weight files in the NetCDF4 format was added to the ESMF_RegridWeightGen application and the ESMF_RegridWeightGen() interface.
  • The ESMF_RegridWeightGen application and the ESMF_MeshCreate() interface have been extended to support 3D unstructured grids provided in the UGRID convention.
  • The ESMF_FieldRegridStore() call has been enhanced so that now regridding takes into account Mesh node masking. This will allow the user to mask out portions of an unstructured Mesh.
  • The user can now get the coordinate system that a Grid was created on from the ESMF_GridGet() interface.
  • A new ESMF_MeshCreate() interface was added. This interface allows a user to create a copy of an existing Mesh, but with a different distribution of the nodes and elements from that of the original Mesh.
  • The ESMF_MeshCreate() and ESMF_MeshAddNodes() methods have been extended to allow the user to optionally set node masks.
  • The ESMF_UtilIOMkDir() interface was added to allow creation of a directory in the file system. The complementary ESMF_UtilIORmDir() interface was also added that allows removal of an empty directory.
  • A new demonstration program, ESMF_Regrid, was added to the ESMF external_demos repository. This program demonstrates regridding between Fields built on Grids created from GRIDSPEC formatted files.
  • Same as ESMF_6_1_1 with the following exceptions:
    • The ESMF_RegridWeightGen application now correctly outputs the grid mask array in the output weight file when either input grid is supplied following the UGRID convention.
    • The ESMF regrid weight generation methods and applications now consistently give the correct answer when a destination point falls very close to the edge of a source cell when using the bilinear method on 3D Grids or 3D Meshes.
    • The ESMF regrid weight generation methods and applications no longer suffer from performance issues when going from a very high resolution grid to a coarse grid (e.g. a 2km global grid to a 7.5x1.875 degree global grid).
    • The ESMF_RegridWeightGen application and interface no longer fail if an input grid contains a polygon with more than 20 sides. The allocation is now done dynamically.
    • The ESMF_GridCreate() interface which allows the user to create a copy of an existing Grid with a new distribution now gives correct results if the Grid contains items (e.g. areas or mask values). The new Grid contains correctly redistributed versions of the original Grid items.
    • The ESMF_GridCreate() interface which allows the user to create a copy of an existing Grid with a new distribution now correctly works if the Grid contains stagger locations other than ESMF_STAGGERLOC_CENTER, ESMF_STAGGERLOC_EDGE1, and ESMF_STAGGERLOC_EDGE2. The new Grid will contain correctly redistributed versions of the original Grid stagger locations.
    • ESMF_AttributeUpdate() now works correctly even if the only changes have been Attribute removals.
  • The ESMF_GridCreate() interface which allows the user to create a copy of an existing Grid with a new distribution will give incorrect results when used on a Grid with 3 or more dimensions and whose coordinate arrays are less than the full dimension of the Grid (i.e. it contains factorized coordinates).
  • The code that allows the compliance checker to report memory usage on Linux is not thread-safe. The ESMF_ComplianceCheckerSTest.F90 system test fails or hangs on Linux systems when ESMF_TESTWITHTHREADS is set to ON.
  • Also see SourceForge bugs list.
Platform-specific issues:
  • Linux with PGI (Discover, Fusion, Jet): Systems tests hang intermittently with ESMF_COMM=mvapich2 due to a system level issue with the mvapich2/PGI combination.
  • Linux/SGI Altix ICE with gfortran (Pleiades): Fails the ESMF_field_1_NP4UTest test harness test with ESMF_COMM=mvapich2.
  • Darwin with g95 (Haumea): Fails the ESMF_LocStreamEx.F90 example with ESMF_BOPT=g for ESMF_COMM=mpiuni.
  • Darwin with gfortran (Haumea): Fails the ESMF_LocStreamUTest.F90 and ESMF_MeshUTest.F90 unit tests and the ESMF_LocStreamEx.F90 example with ESMF_BOPT=O for ESMF_COMM=mpich2 and ESMF_COMM=mvapich2.
  • Linux with PGI (Fusion): Fails the ESMF_FieldRegridUTest.F90 unit test with ESMF_COMM=mvapich2.
  • Windows/Cygwin with g95: Fails the ESMF_UtilUTest.F90 unit test when linked against the shared library version of ESMF.
  • Windows/MinGW with intel: The ESMF_CompTunnelUTest.F90 unit test fails.
ESMF_6_1_1

Reference Manual for Fortran
(html) (pdf)

Reference Manual for C
(html) (pdf)

User's Guide
(html) (pdf)

Supported Platforms

Regridding Status
1/04/13
  • This release is backward compatible with ESMF 6.1.0. There are no API changes.
  • The regridding capabilities in this release are identical to ESMF 6.1.0. Tables summarizing the ESMF regridding status are available. These include supported grids and capabilities of the offline and integrated regridding, and numerical results of some specific test cases.
  • Same as ESMF_6_1_0 with the following exceptions:
    • The bug in the NUOPC Layer that lead to an "Input argument out of range" error during Component initialization has been fixed. Full functionality of the NUOPC Layer has been restored.
  • Also see SourceForge bugs list.
ESMF_6_1_0

Reference Manual for Fortran
(html) (pdf)

Reference Manual for C
(html) (pdf)

User's Guide
(html) (pdf)

Supported Platforms

Regridding Status
11/30/12
  • This release is backward compatible with ESMF 5.3.0 for all the interfaces that were previously marked to remain backward compatible in future ESMF releases. However, there were API changes to unmarked methods that may require user code modifications. In FieldBundle, Field, ArrayBundle, and Array, changes were made to the Read/Write methods as a consequence of reworking the I/O capabilities. In Grid, an unmarked method was removed as obsolete. In XGrid, several unmarked methods were modified to provide a cleaner, more general interface. In Attribute the convention and purpose of a few Attribute packages were updated. In the NUOPC Layer, there were changes in the initialize phase and run sequence definitions. The entire list of API changes is summarized in a table showing interface changes since ESMF_5_3_0, including the rationale for each change.
  • Tables summarizing the ESMF regridding status have been updated. These include supported grids and capabilities of the offline and integrated regridding, and numerical results of some specific test cases.
  • The capability of creating an unstructured Mesh object from a NetCDF file in SCRIP or UGRID format was added to the ESMF Python interface. More information about the Python interface is available at the ESMP web page.
  • The NUOPC_Model component now supports nesting. Parent and nested domains are separate instances of the same model component, running as siblings under the same NUOPC_Driver component.
  • A Science Component class, named ESMF_SciComp, has been implemented. The ESMF_SciComp is intended to represent scientific domains, or realms, in an Earth Science Model. Its primary purpose is to provide a means for representing Component metadata within a hierarchy of Components by functioning as a container for Attributes as well as other Components.
  • The implementation of the ESMF_StateReconcile() method was redesigned to improve performance, and scalability. There were no changes to the user interface.
  • A new option was introduced to the container Get() methods to support the retrieval of container elements in the order in which they were added, instead of returning them in the default alphabetical order. All container classes (ESMF_State, ESMF_FieldBundle, and ESMF_ArrayBundle) support this new functionality.
  • Added two new arguments to the ESMF_RegridWeightGen application: --src_coordinates and --dst_coordinates to allow users to choose the coordinate variables if multiple sets of coordinates are defined in a GRIDSPEC file.
  • The ESMF_FieldRegridStore() method, and the offline regrid weight generation application (ESMF_RegridWeightGen), have been extended to support two new nearest neighbor regridding methods:
    • The nearest source to destination method allows the user to map each destination point to the closest source point.
    • The nearest destination to source method allows the user to map each source point to the closest destination point.
  • The ESMF_FieldRegridStore() call has been extended to allow the user to obtain a list of unmapped destination points. This list is available via an optional argument passed into the method.
  • An option was added to the ESMF_FieldRegridStore() call to allow users to ignore degenerate cells when doing first-order conservative regridding. Cells are considered degenerate if enough corners are closer than 1E-15 degrees, and practically collapse into a line or point. Previously these cells caused the method to return with an error.
  • A new method was added to the library that supports regrid weight generation from grid files. The new interface is called ESMF_RegridWeightGen().
  • The capability to support bit-for-bit reproducibility was added to the sparse matrix multiply methods. This capability is also available for the internal sparse matrix multiply part of the regrid method, although the regrid weight generation part is not yet guaranteed to be bit-for-bit. Three levels of bit-for-bit reproducibility are supported: strict reproducibility across different number of PETs, relaxed reproducibility when the number of PETs does not change, and no guarantee of bit-for-bit reproducibility when performance is most important.
  • The interfaces and the implementation of the ESMF I/O methods for ESMF_Field, ESMF_FieldBundle, ESMF_Array, and ESMF_ArrayBundle were redesigned to be more consistent and complete. None of these interfaces have been marked as backward compatible yet. The interface changes are detailed in the table showing interface changes since ESMF_5_3_0.
  • The ESMF_GridCreate() method that creates Grids from SCRIP or GRIDSPEC files has been parallelized to improve performance.
  • XGrid creation has been extended to support flexible ways of supplying Grids and Meshes on either side of the XGrid. Weights generated by regridding through XGrid can be renormalized by user supplied source or destination area. The user can retrieve the area and centroid information from the XGrid.
  • An Attribute package implementing the METAFOR Common Information Model (CIM) of version 1.5.1 was added to allow a CIM 1.5.1 compliant XML file to be written from the internal information stored in an ESMF Grid.
  • The ESMF_AttributeWrite() method was modified and is now a collective call. This means that all PETs that are part of the current VM must issue this call.
  • The ESMF_UtilSort() utility was added for sorting arrays of data. Currently supported data types include: character string, integer, long integer, real, and double precision.
  • Same as ESMF_5_3_0 with the following exceptions:
    • Added a small tolerance (1E-10 degrees) when determining whether two corner coordinates are identical in a SCRIP unstructured grid file. This allows the ESMF_RegridWeightGen application to correctly handle SCRIP unstructured grid files that have small round off errors in the corner coordinate arrays.
    • The ESMF_RegridWeightGen application now correctly handles bilinear and patch regridding on SCRIP unstructured grid files containing concave cells.
    • The instability that led to undefined behavior (e.g. segmentation fault) for all the sparse matrix store calls (i.e., ESMF_FieldBundleSMMStore(), ESMF_FieldSMMStore(), ESMF_ArrayBundleSMMStore(), and ESMF_ArraySMMStore()) for very large problem sizes (~15 million factors in the sparse matrix) has been fixed.
    • The decomposition scheme of the original DistGrid (indicated by the ESMF_Decomp_Flag) is now correctly duplicated by the ESMF_DistGridCreate() methods that take a DistGrid input argument.
  • A bug in the NUOPC Layer leads to an "Input argument out of range" error during Component initialization. This issue renders the NUOPC Layer practically nonfunctional in this ESMF release.
  • When the ESMF_FieldRegridStore() call or the ESMF_RegridWeightGen application is used with nearest destination to source method, the unmapped destination point detection does not work. Even if the option is set to return an error for unmapped destination points (the default) no error will be returned.
  • Also see SourceForge bugs list.
Platform-specific issues:
  • AIX/IBM (Bluefire): Fails the ESMF_ArrayRedistOpenMPSTest.F90 system test intermittently.
  • Linux with PGI (Discover, Fusion, Jet): Systems tests hang intermittently with ESMF_COMM=mvapich2 due to a system level issue with the mvapich2/PGI combination.
  • Linux/SGI Altix ICE with gfortran (Pleiades): Fails the ESMF_field_1_NP4UTest test harness test with ESMF_COMM=mvapich2.
  • Linux with PGI (Jet): Fails the ESMF_DELayoutWorkQueueUTest.F90 unit test intermittently with ESMF_COMM=mvapich2.
  • Linux with PGI (Pleiades): Fails the ESMF_DELayoutWorkQueueUTest.F90 unit test intermittently with ESMF_COMM=mpi.
  • Darwin with g95 (Haumea): Fails the ESMF_LocStreamEx.F90 example with ESMF_BOPT=g for ESMF_COMM=mpiuni.
  • Darwin with gfortran (Haumea): Fails the ESMF_LocStreamUTest.F90 and ESMF_MeshUTest.F90 unit tests and the ESMF_LocStreamEx.F90 example with ESMF_BOPT=O for ESMF_COMM=mpich2 and ESMF_COMM=mvapich2.
  • Linux with PGI (Fusion): Fails the ESMF_FieldRegridUTest.F90 unit test with ESMF_COMM=mvapich2.
  • Windows/Cygwin with g95: Fails the ESMF_UtilUTest.F90 unit test when linked against the shared library version of ESMF.
  • Windows/MinGW with intel: The ESMF_AttributeWriteInternalUTest.F90 unit test fails.
  • Windows/MinGW with intel: The ESMF_CompTunnelUTest.F90 unit test and the ESMF_CompTunnelEx.F90 example fail.
  • Windows/MinGW with intel: The ESMF_FieldBundleSMMSTest.F90 and ESMF_FieldSparseMatMulSTest.F90 system tests fail during VM shutdown.
  • Windows/MinGW with intel: Several ESMF_Attribute unit tests (ABundle, Array, CplComp, FBundle, Field, Grid, Sci, State) fail due to file comparison differences between Windows and Linux line terminators.
ESMF_5_3_0

Reference Manual for Fortran
(html) (pdf)

Reference Manual for C
(html) (pdf)

User's Guide
(html) (pdf)

Supported Platforms

Regridding Status
04/30/12
  • This release is backward compatible with ESMF 5.2.0rp2 for all the interfaces that were previously marked to remain backward compatible in future ESMF releases. However, there were API changes to unmarked methods in Mesh and Attachable Methods. These changes require user code modifications unless argument keyword syntax was used. The entire list of API changes is summarized in a table showing interface changes since ESMF_5_2_0rp2, including the rationale for each change.
  • Tables summarizing the ESMF regridding status have been updated. These include supported grids and capabilities of the offline and integrated regridding, and numerical results of some specific test cases.
  • A Python interface to ESMF, currently called ESMP, has been implemented. It is available for download at the ESMP web page. ESMP provides support for Grids, Meshes, Fields, and Field regridding. The regridding capability includes bilinear, higher-order patch recovery and first-order conservative regridding methods and the choice to ignore unmapped points. There is support for applying masking or user areas to the Grid or Mesh, and retrieving areas or fractions from a Field. There is also limited access to the ESMF VM and Log capability through ESMP.
  • A fault-tolerant extension to the ESMF_GridComp and ESMF_CplComp interfaces has been implemented. This extension allows ESMF Components, running in separate executables, to be linked via a socket based connection. A user controlled "timeout" mechanism allows either side of this link to continue with normal execution even after the other side of the connection has become unresponsive. The practical use cases of this new feature are large ensemble configurations where individual ensemble members may crash during the course of a simulation run. The fault-tolerant Component mechanism allows the ensemble driver, under these circumstances, to complete the run in a controlled manner with the remaining ensemble members. This mechanism was implemented by adding a "timeout" argument to ESMF_GridCompDestroy(), ESMF_GridCompFinalize(), ESMF_GridCompInitialize(), ESMF_GridCompReadRestart(), ESMF_GridCompRun(), ESMF_GridCompSetServices(), ESMF_GridCompWait(), and ESMF_GridCompWriteRestart(), as well as the equivalent ESMF_CplComp methods. New methods ESMF_GridCompServiceLoop(), and ESMF_CplCompServiceLoop() were added with the "timeout" argument.
  • The generic NUOPC_Driver component now supports petLists for all model and connector components. This enables these components to be run concurrently if desired.
  • The generic NUOPC_Driver and NUOPC_Model components now support simple explicit, semi-implicit, as well as complex implicit component run sequences.
  • The ESMF_StateReconcile() method can now be called repeatedly on the same State during the course of execution. This can be used to re-reconcile the State after adding to or removing objects from it.
  • Added an optional "index" argument to all of the ESMF_MethodAdd(), ESMF_MethodExecute(), and ESMF_MethodRemove() calls to support multiple methods under the same "label". The NUOPC layer utilizes this feature to implement phase-specific Component method specializations.
  • An ESMF_FieldCopy() method was added. This method copies data values from a source Field to a destination Field. The underlying data arrays in the source and destination Fields must have the same shape and distribution (i.e., their DistGrids must be identical).
  • The ESMF_FieldRegridStore() call has been extended to support first order conservative regridding involving Fields built on 3D Grids or 3D Meshes. The 3D Meshes this capability will work on are ones composed of hexahedrons and tetrahedrons. (In contrast to this, bilinear interpolation is supported on 3D Meshes composed just of hexahedrons and higher-order patch interpolation isn't supported in 3D.)
  • The ESMF_FieldRegridStore() call has been enhanced so that now conservative regridding takes into account Mesh cell masking. This will allow the user to mask out portions of an unstructured Mesh.
  • The ESMF_FieldRegridStore() call has been extended so that now conservative regridding is adjusted to use the cell areas set in the Grid or Mesh object upon which the source and destination Fields are built. If these areas are set, the conservation will now hold for these areas. If not set, the conservation will hold for the cell areas as calculated by ESMF.
  • The offline regrid weight generation application (ESMF_RegridWeightGen) now has the capability to generate conservative weights for 3D unstructured Meshes specified in the ESMF unstructured file format.
  • The ESMF_RegridWeightGen application supports two new grid file formats: Climate and Forecast (CF) UGRID convention for 2D unstructured grids and CF GRIDSPEC convention for single-tile logically rectangular 2D grids.
  • The ESMF_RegridWeightGen application allows users to construct a grid mask by using the missing values of a data variable defined in the UGRID or the GRIDSPEC file.
  • A new command line option (--users_area) was added to the ESMF_RegridWeightGen application. For conservative regridding this option causes the weight generation to use the grid areas provided in the grid files instead of calculating the areas internally. If these areas are read in, the conservation will now hold for these areas. If the option is not set, the conservation will hold for the cell areas as calculated by ESMF.
  • An ESMF_ArrayCopy() method was added. This method takes two existing Array objects and copies the data from one Array into the other, after verifying that both have the same shape and distribution (i.e., the DistGrids must be identical).
  • The ESMF_GridCreate() method that supports creation of a Grid from a SCRIP format NetCDF file was extended to support files using the CF GRIDSPEC convention. Options for user area and masking were also added to the interface.
  • The ESMF_MeshCreate() and ESMF_MeshAddElements() methods have been extended to allow the user to optionally set cell areas and cell masks.
  • The ESMF_MeshCreate() method that supports creation of a Mesh from a SCRIP format or ESMF unstructured Mesh format NetCDF file was extended to support files using the CF UGRID convention. Options for user area and masking were also added to the interface.
  • The ESMF_MeshCreate() method that supports creation of a Mesh from an ESMF unstructured Mesh format NetCDF file was extended to support 3D unstructured Meshes.
  • The ESMF_XGridCreate() method was extended to support lists of Grids on either side of the XGrid (exchange grid). The flux exchange calculated through the XGrid is first order conservative. Masking and merging when creating an XGrid are not supported in this release.
  • The ability to access Grid object information through the Attribute class with string valued input parameters was added to ESMF_AttributeGet(). This is an initial example of how information can be drawn directly from data objects, and does not need to be entered by the user.
  • Implemented the custom calendar kind, ESMF_CALKIND_CUSTOM, in Time Manager. This can be used to define calendars for other planets, such as Mars, as well as custom Earth calendars.
  • Added support for Fortran logical data type in ESMF_VMGather() and ESMF_VMAllToAllV().
  • Added support for Fortran character array in ESMF_VMAllGatherV() and ESMF_VMAllToAllV().
  • Implemented ESMF_VMAllToAll() for real and integer data types.
  • Support for the Grid, Fields built on Grids and Field regridding was added to the C interface. There is support for applying masking or user areas to the Grid or Mesh, and retrieving areas or fractions from a Field.
  • By default, LAPACK dependencies are now satisfied using internal code. Linking against an external LAPACK library is no longer required to use the higher order patch interpolation.
  • The ESMF_StateReconcile() call overestimates the size requirement of an internal temporary buffer. The estimation error increases with core count and causes ESMF_StateReconcile() to run out of memory for large core counts (~10,000) on typical configurations.
  • FieldBundles don't currently enforce that every contained Field is built on the same Grid, Mesh, LocStream, or XGrid object, although the documentation says that this should be so.
  • ESMF_FieldRegridStore() doesn't currently work on Fields created on Grids which contain a DE of width less than 2 elements.
  • ESMF_FieldRegridStore() doesn't currently work on Fields created on Grids of arbitrary distribution.
  • ESMF_FieldRegridStore() that takes XGrid as an argument does not support regridding between Fields that are constructed on Grids for which ESMF_GridMatch() locally returns a value of ESMF_GRIDMATCH_EXACT or greater. This may result in unexpected local errors.
  • A larger than expected difference in integrals (8.7E-10) is occurring when using the conservative regridding between a 1 deg lat lon grid and a 2.5 deg lat lon grid. This problem appears isolated to a subset of the cases when grid cell edges align, but not exactly due to round off. It is recommended that users test the integration of their conservative interpolation under these circumstances.
  • The ESMF_Grid creation interfaces that take the petMap argument (e.g. ESMF_GridCreate1PeriDim()) mistakenly return an error for inputs that put more than one DE on a PET, if the DEs are not mapped onto the PETs in sequential order.
  • The ESMF_RegridWeightGen application does not correctly handle SCRIP unstructured grid files that have small round off errors in the corner coordinate arrays. This problem leads to an abort with error about degenerated cells.
  • The ESMF_RegridWeightGen application does not correctly handle bilinear and patch regridding on SCRIP unstructured grid files containing concave cells. This problem leads to an abort with error about degenerated cells.
  • The ESMF_RegridWeightGen application does not output the grid mask array in the output weight file when either input grid is supplied following the UGRID convention. As a result, the output weight file produced under this condition does not work with the scrip_test utility program provided by the SCRIP package.
  • The ESMF regrid weight generation methods and applications can give wrong answers for the bilinear method on 3D Grids or 3D Meshes when a point in the destination falls very close to the edge of a source cell. In this case the destination point can sometimes be mapped to a wrong cell.
  • The ESMF regrid weight generation methods and applications take noticeably longer to generate regridding weights when going from a very high resolution grid to a coarse grid (e.g. a 2km global grid to a 7.5x1.875 degree global grid) than when going to another high resolution grid.
  • The ESMF_RegridWeightGen application and interface will fail with an error if an input grid contains a polygon with more than 20 sides.
  • All of the sparse matrix store calls (i.e., ESMF_FieldBundleSMMStore(), ESMF_FieldSMMStore(), ESMF_ArrayBundleSMMStore(), and ESMF_ArraySMMStore()) suffer from an instability that can lead to undefined behavior (e.g. segmentation fault) for very large problem sizes (~15 million factors in the sparse matrix).
  • The ESMF_GridCreate() interface which allows the user to create a copy of an existing Grid with a new distribution will give incorrect results if the Grid contains items (e.g. areas or mask values). The new Grid is created without items.
  • The ESMF_GridCreate() interface which allows the user to create a copy of an existing Grid with a new distribution will give incorrect results if the Grid contains stagger locations other than ESMF_STAGGERLOC_CENTER, ESMF_STAGGERLOC_EDGE1, and ESMF_STAGGERLOC_EDGE2. The new Grid will not contain any stagger locations other than the above.
  • The decomposition scheme of the original DistGrid (indicated by the ESMF_Decomp_Flag) is not duplicated by the ESMF_DistGridCreate() methods that take a DistGrid input argument. Instead, the duplicate DistGrid is always constructed using ESMF_DECOMP_BALANCED. This issue is most noticeable when building Grids on DistGrids with decomposition schemes other than the default ESMF_DECOMP_BALANCED. A Field built on such a Grid will incorrectly use a balanced decomposition.
  • ESMF_AttributeWrite() only works for ESMF standard Attribute packages.
  • ESMF_AttributeUpdate() will not work correctly if the only changes have been Attribute removals. In this situation the call will return success but no updating will have occurred. This situation can occur during the removal of Attribute packages as well as single Attributes.
  • Also see SourceForge bugs list.
Platform-specific issues:
  • AIX/IBM (Bluefire): Fails the ESMF_ArrayRedistOpenMPSTest.F90 system test intermittently.
  • Darwin with gfortran (Haumea): Fails the ESMF_CompTunnelUTest.F90 unit test with ESMF_COMM=mpich2 and ESMF_COMM=mvapich2.
  • Linux with Intel (Columbia): Fails the ESMF_FieldRegridUTest.F90 unit test with ESMF_COMM=mpi and mpiuni + ESMF_BOPT=O. It also fails the ESMF_FieldSphereRegridEx.F90 example with ESMF_COMM=mpi + ESMF_BOPT=O.
  • Linux with Intel (Discover): Fails the ESMF_CompTunnelUTest.F90 unit test with ESMF_COMM=mvapich2 + ESMF_BOPT=O. It also fails the ESMF_CompTunnelEx.F90 example intermittently with ESMF_COMM=mvapich2 + ESMF_BOPT=O.
  • Linux with g95 (Pluto): Fails the ESMF_CompTunnelUTest.F90 unit test intermittently with ESMF_COMM=mvapich2 and mpich2. It also fails the ESMF_CompTunnelEx.F90 example intermittently with ESMF_COMM=mvapich2 + ESMF_BOPT=O.
  • Linux with gfortran (Pluto): Fails the ESMF_CompTunnelUTest.F90 unit test intermittently with ESMF_COMM=mpich2 and openmpi. It also fails the ESMF_CompTunnelEx.F90 example intermittently with ESMF_COMM=mpich2 + ESMF_BOPT=O.
  • Linux with Intel (Pluto): Fails the ESMF_CompTunnelUTest.F90 unit test with ESMF_COMM=mvapich2 + ESMF_BOPT=g.
  • Linux with NAG (Pluto): Fails the ESMF_DELayoutWorkQueueUTest.F90 unit test intermittently with ESMF_COMM=openmpi.
  • Linux with NAG (Pluto): Fails the ESMF_CompTunnelUTest.F90 unit test with ESMF_COMM=mpich2 + ESMF_BOPT=O and ESMF_COMM=mvapich2 + ESMF_BOPT=g.
  • Linux with PGI (Discover, Fusion, Jet): Systems tests hang intermittently with ESMF_COMM=mvapich2 due to a system level issue with the mvapich2/PGI combination.
  • Linux/SGI Altix ICE with Intel (Pleiades): Fails the ESMF_field_1_NP4UTest test harness test with ESMF_COMM=openmpi.
  • Linux/SGI Altix ICE with Intel (Pleiades): Fails the ESMF_DELayoutWorkQueueUTest.F90 unit test intermittently with ESMF_COMM=intelmpi.
  • Linux/SGI Altix ICE with Intel (Pleiades): Fails the ESMF_CompTunnelEx.F90 example intermittently with ESMF_COMM=mvapich2 + ESMF_BOPT=g.
  • Linux/SGI Altix ICE with gfortran (Pleiades): Fails the ESMF_field_1_NP4UTest test harness test with ESMF_COMM=mvapich2.
  • Linux/Sun Fire X4640 with PGI (Euclid): Fails the ESMF_DELayoutWorkQueueUTest.F90 unit test intermittently with ESMF_COMM=openmpi.
  • Linux with PGI (Jet): Fails the ESMF_DELayoutWorkQueueUTest.F90 unit test intermittently with ESMF_COMM=mvapich2.
  • Windows/Cygwin with g95: Fails the ESMF_UtilUTest.F90 unit test when linked against the shared library version of ESMF.
  • Windows/MinGW with intel and intelcl: The ESMF_CompTunnelUTest fails and the ESMF_CompTunnelEx hangs.
  • Windows/MinGW with intel: The ESMF_FieldBundleSMMSTest fails during VM shutdown.
  • Windows/MinGW with intelcl: The ESMF_FieldSparseMatMulSTest fails during VM shutdown.
ESMF_5_2_0rp2

Reference Manual
(html) (pdf)

Reference Manual for C
(html) (pdf)

User's Guide
(html) (pdf)

Supported Platforms

Regridding Status
04/10/12
  • This release is backward compatible with ESMF 5.2.0rp1. There are no API changes.
  • Extended ESMF_FieldRegridStore() and the offline regrid weight generation application (ESMF_RegridWeightGen) to support conservative regridding involving Fields built on Grids or Meshes which contain cells whose corners are in clockwise order.
  • The ESMF regridding status tables have not changed since 5.2.0rp1.
Platform-specific issues:
  • Same as ESMF_5_2_0rp1 with the following exceptions:
    • Cray with PGI: Linker flags in the esmf.mk now support that a different PGI version is used during application build than was used for the ESMF installation.
ESMF_5_2_0rp1

Reference Manual for Fortran
(html) (pdf)

Reference Manual for C
(html) (pdf)

User's Guide
(html) (pdf)

Supported Platforms

Regridding Status
01/24/12
  • This release is backward compatible with ESMF 5.2.0r. However, for four methods, it was found that issues with consistency were missed. A small number of argument deprecations were introduced in order to address this without breaking backward compatibility. The changes are summarized in a table showing interface changes since ESMF_5_2_0r, including the rationale for each change.
  • Tables summarizing the ESMF regridding status have been updated to reflect minor decreases in interpolation errors. These tables include supported grids and capabilities of the offline and integrated regridding, and numerical results of some specific test cases.
  • The Field and FieldComm API sections in the Fortran Reference Manual were consolidated into a single Field API section. Now all class API sections are structured this way.
  • Extended ESMF_FieldRegridStore() to support conservative regridding involving Fields built on Grids containing only a single cell.
  • Extended the offline regrid weight generation application (ESMF_RegridWeightGen) to support generation of conservative interpolation weights for grid files containing only a single cell.
  • The ESMF_RegridWeightGen application now supports conservative interpolation on Meshes which contain concave cells of five or more sides.
  • The "--64bit_offset" option was added to the ESMF_RegridWeightGen application. This option allows weight files to be generated in the NetCDF 64bit offset format. This option is available for ESMF built against NetCDF 3.6.0 or later.
  • The "--version" option was added to the ESMF_RegridWeightGen application. This option prints the ESMF license together with the exact ESMF version information.
  • XML output of Attributes in the METAFOR Common Information Model (CIM) format can now be done without the external library Xerces, by not setting the ESMF_XERCES environment variables before building ESMF. See the "Third Party Libraries" section in the ESMF User's Guide for details.
  • Enabled the CIM "General Component Properties Description" Attribute package to hold one or more values for each Attribute name.
  • Added a new CIM "Scientific Properties Description" Attribute package which can hold any user-defined name/value sets.
  • Enhanced the CIM "General Component Properties Description" Attribute package to output the XML attribute 'type="custom" ' as part of the CIM component property element, e.g. . This denotes it as a custom component property, to distinguish it from a scientific property, which is written similarly, but without the 'type="custom" ' designation.
  • Concatenated the value set in Attribute "URL", part of the CIM Citation Attribute package, to the value of Attribute "LongTitle" upon output to a CIM XML file. This is a temporary measure while waiting for the CIM to have a separate spot for holding the URL value.
  • Created a new use test case, ESMF_AttributeCIM2ESG, to show how to set Attributes, and how CIM XML output can be ingested into a gateway such as Earth System Grid (ESG).
  • Added automatically written comments to CIM XML generated by ESMF which document the sources of a ESMF CIM Attribute package/Attributes.
  • The convention specification for CIM Attribute packages is now just "CIM"; previously it was "CIM 1.5". This allows user code to remain unchanged as new ESMF/CIM versions are developed.
  • CIM XML output of Attributes validates against the CIM 1.7 release, as well as the previous CIM 1.5 release.
  • Same as ESMF_5_2_0r with the following exceptions:
    • The performance issue in ESMF_Finalize() that was most noticeable for applications with large numbers of DELayout objects (~1000 or more) has been fixed.
    • The API documentation for ESMF_StateAdd(), ESMF_StateAddReplace(), and ESMF_StateReplace() in the Fortran Reference Manual now correctly uses the "nestedStateList" keyword to match the actual implementation.
    • Calling ESMF_FieldEmptyComplete() on a Field object for which previously ESMF_FieldEmptySet() was called with a Grid no longer fails.
    • The ESMF_GridCreate1PeriDim() and ESMF_GridCreate2PeriDim() methods that create Grids with irregular distribution now correctly set up Grid connections. Consequently ESMF_FieldRegridStore() now correctly maps all destination points for these Grids.
    • The mistakes in section 29.3.5 of the Fortran Reference Manual, with the title "Create a Mesh from a SCRIP Grid file or an ESMF unstructured Grid file", have been corrected.
    • ESMF_FieldRegridStore() now works on Fields created on Grids with indexflag equal to ESMF_INDEX_DELOCAL. This is the default.
    • The ESMF_FieldRegridStore() no longer fails with unmapped destination points if the source Field is built on a Mesh or Grid which contains a quadrilateral cell in which particular pairs of nodes are identical so that the quadrilateral collapses to a triangle.
    • The ESMF_FieldRegridStore() no longer uses coordinates in the masked region. Errors in the coordinates in a masked region will not lead to ESMF_FieldRegridStore() returning with errors.
    • ESMF_FieldRegridStore() now returns with error for Fields built on a Mesh containing clockwise elements or a Grid with a left handed coordinate system.
    • The ESMF_RegridWeightGen application now works correctly when source or destination SCRIP file contains less than 2 cells in any dimension.
    • Running the ESMF_RegridWeightGen application on enough processors that the decomposition of the destination Mesh does not produce entries on all processors now works. This bug is also fixed when calling ESMF_FieldRegridStore().
    • The problem in the ESMF_RegridWeightGen application which lead to incorrect interpolation weights when using non-conservative interpolation on an unstructured SCRIP format file with the following conditions has been fixed. The problem occured if a corner node with coordinates (lon, lat) in the SCRIP file followed another corner with the same lon, but only strictly smaller lats, while at the same time another corner (lon2, lat2) had already been read where lat=lat2, but lon2 was the next larger longitude after lon that had been in the file so far.
    • The ESMF_RegridWeigthGen application now generates correct weights for the case where either the source or the destination grid is in the ESMF unstructured grid format and the node coordinate unit is "radians".
    • The interpolation weight file generated by the ESMF_RegridWeigthGen application now has correct dimension and values for the center coordinate variables (i.e., xc_a, yc_a or xc_b, yc_b) for the case where the source/destination grid file is in the ESMF unstructured file format and the regridding method is either bilinear or patch.
    • Using the ESMF_RegridWeightGen application on a SCRIP unstructured grid which has an element with one of the nodes on the North Pole (i.e. the latitude of the node is 90 degree north) now works.
    • The weight file produced by the ESMF_RegridWeightGen application no longer contains multiple entries for the same source and destination pair. (This problem did not affect the accuracy of the regridding, but it did increase the file size.)
    • The "src_grid_dims" and "dst_grid_dims" variables in the SCRIP weight file generated by ESMF_RegridWeightGen are now correctly set to the "elementCount" value of the source or destination input grid file, respectively.
    • Halo updates are now correct for Fields, FieldBundles, Arrays, and ArrayBundles with more than one DE per PET.
    • For the CIM "Inputs Description" Attribute package, values set for the CouplingSource and CouplingTarget Attributes now appear correctly in ESG.
    • The API documentation for the ESMF_ATTGETCOUNT_FLAG in the Fortran Reference Manual now correctly describes the purpose of this flag.
    • The previously ambiguous API documentation for ESMF_TimeSet() in the Fortran Reference Manual now documents which arguments are allowed for Gregorian and Julian calendars.
    • The default MPI tags used by the ESMF VM layer are now bounded by the largest supported MPI tag. This allows calls like ESMF_ArrayRedistStore() to execute on large processor counts (~16K and larger).
    • The inconsistency in the internal use of the ESMF Attribute API in the NUOPC addon layer that lead to failure of correct NUOPC user code has been fixed.
    • The ESMF install target now succeeds following a failed attempt to build the ESMF documentation for the ESMF source tarball.
    • The "ESMF_NETCDF_INCLUDE" environment variable now is correctly documented in the NETCDF section of the ESMF User's Guide.
  • Also see SourceForge bugs list.
Platform-specific issues:
  • Same as ESMF_5_2_0r with the following exceptions:
    • g95: Now builds the ESMF library with ESMF_PIO=internal.
    • Cray/Intel-12.0.x: Applications now link successfully against the ESMF library.
  • Windows/MinGW with intel and intelcl: The ESMF_FieldRegridUTest.F90 and ESMF_FieldArbGridUTest.F90 fail. With ESMF_BOPT=g, the ESMF_ArrayRedistOpenMPSTest.F90 also fails.
  • SGI Altix ICE with intel: Fails the ESMF_field_1_NP4UTest test harness test with ESMF_COMM=openmpi.
  • SGI Altix ICE with intel: Fails the ESMF_DELayoutWorkQueueUTest.F90 unit test intermittently with ESMF_COMM=intelmpi.
  • SGI Altix ICE with gfortran : Fails the ESMF_field_1_NP4UTest test harness test with ESMF_COMM=mvapich2.
  • Linux with PGI: System tests hang intermittently with ESMF_COMM=mvapich2 due to a system level issue with the mvapich2/PGI combination.
ESMF_3_1_0rp5

Reference Manual
(html) (pdf)

User's Guide
(html) (pdf)

Supported Platforms
8/22/11
  • Implemented a work-around for a bug in the Component return code handling which lead to segmentation faults under some conditions.
ESMF_5_2_0r

Reference Manual for Fortran
(html) (pdf)

Reference Manual for C
(html) (pdf)

User's Guide
(html) (pdf)

Supported Platforms

Regridding Status
7/15/11
  • The majority of the interfaces in established ESMF classes (e.g. Component, State, Field, Grid) will be backward compatible starting with this release. Interfaces that are backward compatible indicate this in the "STATUS:" section of their API description in the ESMF Reference Manual. Throughout the framework, interfaces were changed to improve consistency. Major system-wide changes are noted in other items below. All changes since ESMF_5_1_0 are summarized in two tables including the rationale for and the impact of each change:
  • Tables summarizing the ESMF regridding status have been updated. These include supported grids and capabilities of the offline and integrated regridding, and numerical results of some specific test cases.
  • In order to be more consistent with other software packages, the access to the ESMF Fortran language binding has switched from "use ESMF_Mod" to "use ESMF".
  • Keywords are now required for specifying optional arguments when calling ESMF interfaces (e.g. rc=rc).
  • Added "attachable methods" to ESMF Components. This allows users to attach other than standard Component methods to a Component.
  • Added tools that allow component developers to make their components available as web-based services. A client can access these components through provided Simple Object Access Protocol (SOAP) interfaces.
  • State "placeholder names" have been removed. This functionality had become redundant with the ability to create empty Fields and FieldBundles.
  • ESMF State, FieldBundle and ArrayBundle classes are now based on a unified and more efficient container class implementation. As a consequence the user experiences consistent ESMF container class interfaces and behavior.
  • Field partial creation now allows an intermediate state in which the underlying grid is specified. The new interfaces are: ESMF_FieldEmptyCreate(), ESMF_FieldEmptySet() and ESMF_FieldEmptyComplete().
  • An argument which allows the user to specify a coordinate system when creating a Grid has been added to the Grid create interfaces. This information is used by the ESMF_FieldRegridStore() call to automatically determine the geometry of the source and destination Grids.
  • Added new shortcut Grid interfaces broken up by the number of periodic dimensions. These allow the user to specify connection information when creating a Grid.
  • The connection information stored inside the Grid is now used by the ESMF_FieldRegridStore() call to automatically determine the topology of the source and destination Grids.
  • XGrids can now be created by providing Grids for both sides without needing to specify sparse matrix parameters.
  • Added the ability to get Attributes package information by name.
  • Added the ability to query for the presence of Attributes.
  • The Attribute packages that implement METAFOR's Common Information Model (CIM) now allow more complete ingestion into Earth System Grid (ESG). This includes the CIM 1.5 "Inputs Description" package.
  • A new standard CIM 1.5 "General Component Properties Description" package was added. This Attribute package allows custom, user-defined name/value Attribute pairs to be represented and output in standard CIM 1.5 XML format.
  • The National Unified Operational Prediction Capability (NUOPC) Layer prototype code is now part of the ESMF source distribution. It builds automatically with the ESMF library and is available along side ESMF. ESMF application makefiles can be used unchanged to link against the NUOPC layer.
  • The time spent in ESMF_Finalize() increases super-linearly with the number of DELayout objects. DELayout objects are created by the default method of creating Arrays and Fields.
  • The API documentation for ESMF_StateAdd(), ESMF_StateAddReplace(), and ESMF_StateReplace() in the Fortran Reference Manual incorrectly uses the "stateList" keyword. The actual implementation of these calls uses the "nestedStateList" keyword for this argument.
  • The ESMF_StateRemove() method fails to correctly unlink Attributes from the Attribute hierarchy.
  • FieldBundles don't currently enforce that every contained Field is built on the same Grid, although the documentation says that this should be so.
  • Calling ESMF_FieldEmptyComplete() on a Field object with a user defined Fortran array pointer for which previously ESMF_FieldEmptySet() was called with a Grid fails with an error, while it should return success.
  • The ESMF_GridCreate1PeriDim() and ESMF_GridCreate2PeriDim() methods that create Grids with irregular distribution fail to set up correct Grid connections. Consequently ESMF_FieldRegridStore() fails unexpectedly with unmapped destination points for these Grids.
  • The ESMF_GridCreate() call, which allows the user to create a Grid from a SCRIP format file and a DistGrid, creates the Grid using the wrong coordSys. Instead of using spherical coordinates (which are implied by the SCRIP format), the call creates the Grid with Cartesian coordinates. This can cause unexpected problems in methods (e.g. ESMF_FieldRegridStore() which use the coordSys to determine the shape of the Grid.
  • The ESMF_Grid creation interfaces that take the petMap argument (e.g. ESMF_GridCreate1PeriDim()) mistakenly return an error for inputs that put more than one DE on a PET, if the DEs are not mapped onto the PETs in sequential order.
  • There are mistakes in section 29.3.5 of the Fortran Reference Manual, with the title "Create a Mesh from a SCRIP Grid file or an ESMF unstructured Grid file", that could be misleading. In the SCRIP file header example the "grid_imask" variable should be of type "int" instead of "double". In the ESMF unstructured grid file example the "units" attribute of the "nodeCoords" variable should be "degrees" instead of "degrees,degrees". Further, in the same example, variables "numElement" and "numNode" should be "elementCount" and "nodeCount", respectively.
  • ESMF_FieldRegridStore() doesn't currently work on Fields created on Meshes which contain tetrahedra.
  • ESMF_FieldRegridStore() doesn't currently work on Fields created on Grids which contain a DE of width less than 2 elements.
  • ESMF_FieldRegridStore() doesn't currently work on Fields created on Grids of arbitrary distribution.
  • ESMF_FieldRegridStore() only works on Fields created on Grids with indexflag equal to ESMF_INDEX_GLOBAL. This is not the default.
  • Masking is currently not supported for unstructured grids (Mesh) for any regridding method.
  • ESMF_FieldRegridStore() that takes XGrid as an argument does not support regridding between Fields that are constructed on Grids for which ESMF_GridMatch() locally returns a value of ESMF_GRIDMATCH_EXACT or greater. This may result in unexpected local errors.
  • A larger than expected difference in integrals (8.7E-10) is occurring when using the conservative regridding between a 1 deg lat lon grid and a 2.5 deg lat lon grid. This problem appears isolated to a subset of the cases when grid cell edges align, but not exactly due to round off. It is recommended that users test the integration of their conservative interpolation under these circumstances.
  • The ESMF_FieldRegridStore() call fails unexpectedly with unmapped destination points if the source Field is built on a Mesh or Grid which contains a quadrilateral cell in which particular pairs of nodes are identical so that the quadrilateral collapses to a triangle.
  • The ESMF_FieldRegridStore() mistakenly uses coordinates in the masked region. Errors in the coordinates in a masked region will lead to ESMF_FieldRegridStore() returning unexpectedly with errors.
  • ESMF_FieldRegridStore() will - without warning or error - produce wrong interpolation weights for Fields built on a Mesh containing clockwise elements or a Grid with a left handed coordinate system.
  • The ESMF_RegridWeightGen application crashes when source or destination SCRIP file contains less than 2 cells in any dimension.
  • Running the ESMF_RegridWeightGen application on enough processors that the decomposition of the destination Mesh does not produce entries on all processors leads to undefined behavior (segmentation fault). This bug will also occur when calling ESMF_FieldRegridStore() with the indices or weights arguments.
  • There is problem in the ESMF_RegridWeightGen application which can lead to incorrect interpolation weights when using non-conservative interpolation on an unstructured SCRIP format file. The problem occurs if a corner node with coordinates (lon, lat) in the SCRIP file follows another corner with the same lon, but only strictly smaller lats, while at the same time another corner (lon2, lat2) has already been read where lat=lat2, but lon2 is the next larger longitude after lon that has been in the file so far.
  • The ESMF_RegridWeigthGen application does not generate correct weights if either the source or the destination grid is in the ESMF unstructured grid format and the node coordinate unit is "radians".
  • The interpolation weight file generated by the ESMF_RegridWeigthGen application has wrong dimension and values for the center coordinate variables (i.e., xc_a, yc_a or xc_b, yc_b) if the source/destination grid file is in the ESMF unstructured file format and the regridding method is either bilinear or patch.
  • Using the ESMF_RegridWeightGen application on a SCRIP unstructured grid which has an element with one of the nodes on the North Pole (i.e. the latitude of the node is 90 degree north) results in wrong interpolation weights or other unexpected behavior, such as a segmentation fault.
  • The weight file produced by the ESMF_RegridWeightGen application may contain multiple entries for the same source and destination pair. This does not affect the accuracy of the regridding, but it does increase the file size.
  • The "src_grid_dims" and "dst_grid_dims" variables in the SCRIP weight file generated by ESMF_RegridWeightGen are incorrectly set to 1 if the source or destination input grid files are in the ESMF unstructured file format, respectively.
  • Halo updates are incorrect for Fields, FieldBundles, Arrays, and ArrayBundles with more than one DE per PET.
  • All of the sparse matrix store calls (i.e., ESMF_FieldBundleSMMStore(), ESMF_FieldSMMStore(), ESMF_ArrayBundleSMMStore(), and ESMF_ArraySMMStore()) will fail with an error for sparse matrices that contain multiple entries for the same source and destination pair when there are undistributed dimension in the source or destination array.
  • All of the sparse matrix store calls (i.e., ESMF_FieldBundleSMMStore(), ESMF_FieldSMMStore(), ESMF_ArrayBundleSMMStore(), and ESMF_ArraySMMStore()) suffer from an instability that can lead to undefined behavior (e.g. segmentation fault) for very large problem sizes (~15 million factors in the sparse matrix).
  • The decomposition scheme of the original DistGrid (indicated by the ESMF_Decomp_Flag) is not duplicated by the DistGridCreate() methods that take a DistGrid input argument. Instead, the duplicate DistGrid is always constructed using ESMF_DECOMP_BALANCED. This issue is most noticeable when building Grids on DistGrids with decomposition schemes other than the default ESMF_DECOMP_BALANCED. A Field built on such a Grid will incorrectly use a balanced decomposition.
  • ESMF_AttributeWrite() only works for ESMF standard Attribute packages.
  • For the CIM "Inputs Description" Attribute package, values set for the CouplingSource and CouplingTarget Attributes appear incorrectly in ESG.
  • ESMF_AttributeUpdate() will not work correctly if the only changes have been Attribute removals. In this situation the call will return success but no updating will have occurred. This situation can occur during the removal of Attribute packages as well as single Attributes.
  • The API documentation for the ESMF_ATTGETCOUNT_FLAG in the Fortran Reference Manual incorrectly uses the description for the ESMF_ATTTREE_FLAG.
  • The API documentation for ESMF_TimeSet() in the Fortran Reference Manual is ambiguous about allowing the updating of a Gregorian calendar object using Julian date arguments.
  • The default MPI tags used by the ESMF VM layer are unbounded. This leads to MPI errors on large processor counts (~16K), e.g. for ESMF_ArrayRedistStore().
  • An inconsistency in the internal use of the ESMF Attribute API in the NUOPC addon layer leads to unexpected failure of correct NUOPC user code.
  • The ESMF install target fails following a failed attempt to build the ESMF documentation for the ESMF source tarball.
  • The "ESMF_NETCDF_INCLUDE" environment variable is incorrectly documented as "ESMF_NETLIB_INCLUDE" in the NETCDF section of the ESMF User's Guide.
  • Also see SourceForge bugs list.
Platform-specific issues:
  • IBM Bluefire AIX: Fails the ESMF_ArrayRedistOpenMPSTest.F90 system test intermittently.
  • Linux Pluto with NAG: Fails the ESMF_DELayoutWorkQueueUTest.F90 unit test intermittently with ESMF_COMM=openmpi.
  • Sun Fire X4640 Euclid with PGI: Fails the ESMF_DELayoutWorkQueueUTest.F90 unit test intermittently with ESMF_COMM=openmpi.
  • Windows/Cygwin with g95: Fails the ESMF_LogErrUTest.F90 unit test intermittently.
  • Windows/Cygwin with g95: Fails the ESMF_UtilUTest.F90 unit test when linked against the shared library version of ESMF.
  • Windows/Cygwin with g95: ESMF_MeshUTest.F90 and ESMF_FieldBundleUTest.F90 unit tests crash.
  • PGI 11.6: Fails to build the ESMF library.
  • PGI 11.7: Fails to build the ESMF tests.
  • g95: Fails to build the ESMF library with ESMF_PIO=internal.
  • Cray with Intel-12.0.x: The ESMF library builds successfully, but applications fail to link.
  • Cray with PGI: Linker flags in the esmf.mk require that the same PGI version is used during application build as was used for the ESMF installation. According to Cray and PGI, this restriction is not necessary.
ESMF_5_2_0p1

Reference Manual for Fortran
(html) (pdf)

Reference Manual for C
(html) (pdf)

User's Guide
(html) (pdf)

Supported Platforms

Regridding Status
5/31/11
  • Added support for the PGI 11.x compiler version on Linux/PC and the Cray XT platform.
  • The ESMF regridding status have not changed since 5.2.0.
ESMF_3_1_0rp4

Reference Manual
(html) (pdf)

User's Guide
(html) (pdf)

Supported Platforms
4/29/11
  • Added support for the Intel compiler suite on the Cray XT platform.
  • Added support for the PGI 10.x and 11.x compiler versions.
ESMF_5_2_0

Reference Manual for Fortran
(html) (pdf)

Reference Manual for C
(html) (pdf)

User's Guide
(html) (pdf)

Supported Platforms

Regridding Status
2/28/11
  • An intensive effort to make interfaces consistent in preparation for future backward compatibility resulted in significant interface changes for this release. These are summarized in a table showing interface changes since ESMF_5_1_0, including the rationale for and the impact of each change.
  • Tables summarizing the ESMF regridding status have been updated. These include supported grids and capabilities of the offline and integrated regridding, and numerical results of some specific test cases.
  • Added ESMF_StateReplace() method, which replaces an existing State item with a new item of the same name.
  • Added ESMF_StateRemove() method, which removes an item from a State.
  • Masking is now supported for conservative interpolation, for logically rectangular grids, for both the offline regrid weight generation application and the ESMF_FieldRegridStore() method.
  • Support was added for regridding a regional Grid on a sphere. This allows regridding between two regional grids on a sphere, between a regional Grid on a sphere and spherical global Grid and between a spherical global Grid and a regional Grid on a sphere. This capability is available in both the offline regrid weight generation application and ESMF_FieldRegridStore().
  • Enabled a Grid constructed using DistGrid connections to work with ESMF_FieldRegridStore(). For example, this means that a user constructed tripole Grid can now participate in a regrid operation.
  • Optimized ESMF_FieldBundleRegridStore() to be faster when Fields are built on the same Grid. When computing the sparse matrices for the regridding, if the Grids in the source and destination Fields match the Grids in the previous set of source and destination Fields in their respective FieldBundle lists, then that same sparse matrix is used instead of recomputing a new one. In particular, this means that if all the Fields in the source FieldBundle are built on the same Grid and all the Fields in the destination FieldBundle are built on the same Grid, then the sparse matrix between those Grids only needs to be computed once. This can save a substantial amount of time for FieldBundles containing many Fields.
  • Support for options from ESMF_FieldRegridStore() were added to ESMF_FieldBundleRegridStore(). Masking, pole options and the ability to chose what happens when destination points can't be mapped were added to the ESMF_FieldBundleRegridStore() interface.
  • Added ability to create multiple instances of the Attribute packages ISO 191115/Responsible Party Description and/or ISO 191115/Citation Description, nested within a standard CIM 1.0/Model Component Simulation Description package for Components.
  • The Attribute values within the ESG/General and CF/Extended standard packages, nested within the standard CIM 1.0/Inputs Description package for Fields, now appear in the Component Properties section of the CIM XML output file. This allows input and output Fields to be described for components other than the top level component in CIM output files.
  • The ESMF Coupled Flow demonstration program has been updated to work with current interfaces. It is now part of the external demos module.
  • Added ESMF_UtilGetArgC() subroutine, which returns the number of command line arguments.
  • Added ESMF_UtilGetArg() subroutine, which returns a given command line argument.
  • Added ESMF_UtilGetArgIndex() subroutine, which searches for a specified command line argument (e.g., '-filename') and, if found, returns its index.
  • An online tutorial based on the Coupled Flow demonstration program was added to the ESMF website.
  • The prototype Exchange Grid implementation has been extended to work with multiple PETS.
  • Same as ESMF_5_1_0 with the following exceptions:
    • The argument attPackInstanceName, introduced in ESMF_5_1_0, in ESMF_AttributeGet(), ESMF_AttributeSet(), and ESMF_AttributeRemove() is now implemented, supporting multiple instances of Attribute packages as described in the release notes.
    • The offline regrid weight generation application, ESMF_RegridWeightGen, is not limited to 2D global grids.
    • The offline regrid weight generation application, ESMF_RegridWeightGen, now supports masking.
    • The coordinate unit attribute is now read correctly on all machines, so grid files with coordinates in radians will be interpreted correctly.
    • Masking is now supported for conservative regridding.
    • The Attribute Table in Section 33.2.6 of the pdf version of the Fortran Reference Manual is not complete, please refer to HTML version of the document for a complete table. This table was in Section 34.2.6 in release ESMF_5_1_0.
  • A larger than expected difference in integrals (8.7E-10) is occurring when using the conservative regridding between a 1 deg lat lon grid and a 2.5 deg lat lon grid. This problem appears isolated to a subset of the cases when grid cell edges align, but not exactly due to round off. It is recommended that users test the integration of their conservative interpolation under these circumstances.
  • The ESMF_FieldRegridStore() call or the ESMF_RegridWeigthGen application may erroneously report destination points as unmapped when running on multiple processors.
  • The matrix coming out of ESMF_FieldRegridStore() needs to be transposed before it can be used in ESMF_FieldSMMStore().
  • The weight file produced by the ESMF_RegridWeightGen application may contain multiple entries for the same source and destination pair. This does not affect the accuracy of the regridding, but it does increase the file size.
  • Also see SourceForge bugs list.
Platform-specific issues:
  • IBM Bluefire AIX: Fails the ESMF_ArrayRedistOpenMPSTest.F90 system test with ESMF_BOPT=O.
  • Linux Pluto with g95: Fails the ESMF_FieldRegridXGUTest.F90 unit test with ESMF_COMM=mvapich2.
  • PC Xeon(64) Cluster Jet with PGI: Fails the ESMF_GridCreateUTest.F90 and ESMF_UtilUTest.F90 unit tests.
  • Cray XT5 Jaguarpf with PGI: Fails the ESMF_GridCreateUTest.F90 and ESMF_UtilUTest.F90 unit tests.
  • Windows/Cygwin g95: Fails the ESMF_LogErrUTest.F90 and ESMF_UtilUTest.F90 unit tests.
ESMF_5_1_0

Reference Manual for Fortran
(html) (pdf)

Reference Manual for C
(html) (pdf)

User's Guide
(html) (pdf)

Supported Platforms

Regridding Status
10/29/10
  • List of ESMF interface changes since ESMF_5_0_0.
  • An integrated ESMF compliance checker has been implemented. It can be activated by setting the runtime environment variable ESMF_RUNTIME_COMPLIANCECHECK=ON. The compliance checker is currently geared toward the NUOPC requirements. The current version is described in more detail in the "NUOPC Layer: ESMF Compliance Checker - ESMF v5.1.0" document.
  • Tables summarizing the ESMF regridding status have been compiled. These include supported grids and capabilities of the offline and integrated regridding, and numerical results of some specific test cases.
  • ESMF_StateGet() has been enhanced to allow additional nested State capabilities. These include:
    • Items can now be searched by name using the itemSearch argument.
    • The nestedFlag argument controls whether the search traverses the entire State hierarchy.
    • State items can be accessed directly by using a path style naming convention.
  • The methods ESMF_StateSetNeeded(), ESMF_StateIsNeeded(), and ESMF_StateGetNeeded() that were removed in release ESMF_5_0_0 have been reinstated. This was necessary to reenable the ESMF COUPLED_FLOW demo.
  • The first-order conservative regridding algorithm has been reworked. This new version is based on the overlap of source cells with destination cells, and is less prone to interpolation artifacts and diffusion than the previous version. It is supported for regridding between any combination of Fields built on the center-stagger of 2D logically rectangular Grids and Fields built on the element location of 2D unstructured Meshes. Grids and Meshes can be either regional or mapped to the sphere (using the ESMF_REGRID_SCHEME_FULL3D option to ESMF_FieldRegridStore()). This functionality is available either offline (via the ESMF_RegridWeightGen application) or through the library (via ESMF_FieldRegridStore()).
  • The ESMF_FieldRegridStore(), ESMF_FieldRegrid() and ESMF_FieldRegridRelease() methods have been overloaded to support Fields created on XGrids.
  • The two offline weight generation applications ESMC_RegridWgtGenEx and ESMF_CubedSphereRegridEx have been merged into a single application named ESMF_RegridWeightGen. The application is documented under the "Applications" section of the ESMF Reference Manual. This application generates interpolation weights using either bilinear, finite element patch recovery or first order conservation methods. The input grids can be either 2D global logically rectangular or 2D global unstructured.
  • The new location argument in ESMF_FieldCreate() allows Fields to be created on Mesh elements or Mesh nodes. Conservative regridding is only supported between Fields created on Mesh elements.
  • Added ESMF_FieldCreate() method that supports creation of a Field on an XGrid.
  • ESMF_FieldCreate() and ESMF_ArrayCreate() have been overloaded to support typekind and rank arguments in addition to the ArraySpec versions.
  • Implemented halo support for arbitrarily decomposed FIeld and Array objects.
  • This release includes new parallel I/O capabilities introduced by using the Parallel I/O (PIO) library within ESMF. PIO development was led by John Dennis of NCAR and it has evolved under a NCAR/DOE laboratory partnership. ESMF data classes Field, FieldBundle, Array and ArrayBundle can now read and write NetCDF files in parallel using methods such as ESMF_FieldRead(), and ESMF_FieldWrite(). Please see the "IO Capabilties" section in the ESMF Reference Manual for how to enable PIO in the ESMF build.
  • Added ESMF_GridCreate() method that supports creation of a logically rectangular 2D Grid from a SCRIP format NetCDF file.
  • Added ESMF_MeshCreate() method that supports creation of an unstructured 2D Mesh from a SCRIP format or ESMF unstructured Mesh format NetCDF file.
  • A first implementation of the Exchange Grid (XGrid) class has been added. An XGrid object can be created from user supplied lists of Grids and sparse matrix parameters. The functionality of ths initial implementation is currently limited to single PET execution. This is not a limitation of the XGrid design but reflects the current implementation status.
  • Added standard ESMF-supplied Attribute packages for Components and Fields, whose XML output conforms to the Climate and Forecast Conventions (CF) and METAFOR's Common Information Model (CIM) schema. The output is ingestable into the Earth System Grid (ESG) web portal for display. For details and examples please see the Attribute section of the ESMF Reference Manual.
  • Added ability for Attribute packages to contain multiple nested Attribute packages of different types.
  • The new msgAllow argument in ESMF_LogSet() permits users to specify the type of Log messages allowed for output. Other Log messages are ignored.
  • The ESMF COUPLED_FLOW demo application has been restored but is currently limited to single PET execution. The documentation in the User's Guide has been updated.
  • Starting with this release a collection of external demonstration programs is available in form of a separate CVS module. These external demos have been written to serve as good examples to show how users would interact with an ESMF installation. A description of the available demos and download instructions are available on the ESMF website under Users->Code Examples->External Demos.
  • Added ESMF_ComplianceCheckerSTest.F90 system test which demonstrates the operation and output of the compliance checker within a multi-Component, multi-PET application.
  • Added ESMF_AttributeCIMSTest.F90 system test which demonstrates the use of the new standard ESMF-supplied, METAFOR CIM Attribute packages for Components and Fields, within a multi-Component, multi-PET application.
C interfaces:
  • ESMF now offers a Reference Manual for the ESMF C API. This document is a work in progress.
  • The available C interfaces are a subset of the Fortran API. This subset was constructed to fullfil the need of a specific coupling project.
  • Changes were made to improve interface consistency.
  • Same as ESMF_5_0_0 with the following exceptions:
    • The Field description in the reference manual now correctly states that the halo update operation is supported.
    • Conservative regridding has been reworked so that its accuracy is now within the expected range (15 decimal places).
    • Resolution differences between source and destination grids no longer cause problems in conservative regridding.
    • Suspicious interpolation weights (large, outside [0,1]) that occurred for cubed sphere regridding no longer occur for bilinear and conservative interpolation.
  • The offline regrid weight generation application, ESMF_RegridWeightGen, only works for 2D global grids.
  • The offline regrid weight generation application, ESMF_RegridWeightGen, does not support masking, and thus any mask information in the grid files will be ignored.
  • On some machines (e.g. bluefire), the coordinate unit attribute is not read correctly by the ESMF_RegridWeightGen application, so grid files with coordinates in radians won't be interpreted correctly and will produce incorrect weights or errors.
  • Masking is currently not supported for conservative regridding.
  • Masking is currently not supported for unstructured grids (Mesh) for any regridding method.
  • ESMF_ArrayWrite() and ESMF_FieldWrite() will fail with an internal PIO error if a NetCDF output file already exists from a previous run.
  • XGrid does not support regridding between Fields that are constructed on Grids for which ESMF_GridMatch() locally returns a value of true. This may result in unexpected local errors.
  • Fields built on XGrids cannot be added to FieldBundles.
  • Only single-PET XGrids are supported. Creating and using multi-PET XGrids will result in undefined behavior.
  • ESMF_AttributeUpdate() will not work correctly if the only changes have been Attribute removals. In this situation the call will return success but no updating will have occurred. This situation can occur during the removal of Attribute packages as well as single Attributes.
  • The new argument attPackInstanceName in ESMF_AttributeGet(), ESMF_AttributeSet(), and ESMF_AttributeRemove() is not fully implemented. Consequently multiple instances of an Attribute package of the same type (same convention and purpose) within a nested Attribute package are not supported.
  • The Attribute Table in Section 34.2.6 of the pdf version of the Fortran Reference Manual is not complete, please refer to HTML version of the document for a complete table.
  • Also see SourceForge bugs list.
  • Platform-specific issues:
    • Discover with intel: Fails ESMF_CompCreateSTest.F90 with ESMF_COMM=mvapich2 and ESMF_BOPT=O.
    • Discover with pgi: the ESMF_DELayoutWorkQueueUTest.F90 fails.
    • IBM Frost Bluegene: Fails ESMC_ConfigUTest.C unit test with ESMF_BOPT=g.
    • IBM Bluefire AIX: Fails ESMC_ConfigUTest.C unit test with ESMF_BOPT=g and ESMF_ArrayRedistOpenMPSTest.F90 with ESMF_ABI=64.
    • Pluto with nag.: Fails system test ESMF_FieldBundleSMMSTest.F90 with ESMF_COMM=mvapich2 and mpich2 intermittently.
    • Windows/MinGW with intel and intelcl: the ESMF_ArrayRedistOpenMPSTest.F90, the ESMF_FieldRegridUTest.F90, and ESMF_FieldArbGridUTest.F90 fail.
    • Windows/Cygwin g95: the ESMF_MeshUTest.F90 fails.
    • Windows/Cygwin g95: ESMF_LogErrUTest.F90 fails intermittently.
    ESMF_5_0_0

    Reference Manual
    (html) (pdf)

    User's Guide
    (html) (pdf)

    Supported Platforms

    Regridding Status
    5/28/10
    • List of ESMF interface changes since ESMF_4_0_0rp2.
    • Tables summarizing the ESMF regridding status have been compiled. These include supported grids and capabilities in the offline and online regridding and numerical results of some specific test cases.
    • ESMF_StatePrint() has a new nestedFlag argument for printing nested States. The options argument now supports 'short' and 'long' printouts. (Default is short.)
    • ESMF_StateValidate() has a new nestedFlag argument for validating nested States.
    • ESMF_StateGet() has a new nestedFlag argument for inquiry requests, which allows returning lists of nested objects and their types.
    • The following methods were removed: ESMF_StateSetNeeded(), ESMF_StateIsNeeded(), and ESMF_StateGetNeeded().
      The reason for removal is that the semantics surrounding these routines were not well-defined, and they were rarely used. These methods may be revisited as part of an ongoing investigation into broader interoperability issues.
    • Implemented halo communication operation for Field, FieldBundle, Array and ArrayBundle classes. Variable and asymmetric halo depths are supported.
    • Added regridding support for FieldBundle objects. First order conservative regridding has been added as a modification to the existing Field regridding methods in both the offline and online and regridding. This modification is based on a finite element L2 projection method. In this initial version quantities are conserved globally. Further work is required for satisfactory local conservation.
    • Improved the algorithm in ESMF_FieldRegridStore() to fix problems with interpolation on a sphere. The change fixes incidents of destination points that are lying close to the boundary between two source elements from being mapped into the wrong element. This led to errors in the interpolation weights.
    • The offline weight generation application for structured grids has been more accurately renamed to ESMC_RegridWgtGenEx. The bilinear interpolation option, as well as a conservative option was added. For details please see the "Structured Grid to Structured Grid" subsection under the "File Based Regrid Weight Applications" section in the "Infrastructure: Fields and Grids" part of the ESMF Reference Manual.
    • Added a new offline weight generation application called ESMF_CubedSphereRegridEx. This application supports regridding between a cubed sphere and a 2D logically rectangular grid mapped to a sphere. The user may use higher-order patch recovery interpolation or bilinear interpolation between the grids. For details please see the "Cubed Sphere to Structured Grid" subsection under the "File Based Regrid Weight Applications" section in the "Infrastructure: Fields and Grids" part of the ESMF Reference Manual.
    • Support the reuse of RouteHandles for Fields and Arrays where the size of the undistributed dimensions that come before the first distributed dimension is different from the size used during the precompute.
    • RouteHandle-based Array communication calls now provide a non-blocking option. This option allows user code to overlap computation with communication.
    • Improved the implementation of ESMF_GridMatch(). Previously the method just checked to see if the Grids had identical pointers, now it checks to see if the two Grids' internal data is the same (dimensions, coordinates, items, etc.). This means that two Grids constructed independently of each other are checked more reliably.
    • Added an option to ESMF_MeshGet() to allow users to retrieve coordinates for just the locally owned Mesh nodes. These coordinates will correspond exactly with the data elements stored in a Field created on the Mesh. This is useful for setting data in the Field or performing calculations involving coordinates on the data in a Field.
    • Added the ability to validate Attributes read from an XML file by specifying a corresponding XSD file in method ESMF_AttributeRead(), via new optional argument schemaFileName. For standard ESMF-supplied Component, Field, and Grid Attribute packages validation is performed automatically, without the need to specify schemaFileName. For examples see the Attribute section of the ESMF Reference Manual.
    • XML Attribute file writing is now done with the open source third party Xerces C++ library, replacing the previous internally hard-coded fprintf() technique. For reading and/or writing XML Attribute files, ESMF must be built against the Xerces library, which requires the setting of additional ESMF environment variables. See the "Third Party Libraries" section in the ESMF User's Guide for details.
    • A new system test, ESMF_FieldRegridMeshToMesh, was added to demonstrate regridding between Fields built on unstructured Meshes.
    • FieldBundles don't currently enforce that every contained Field is built on the same Grid, although the documentation says that this should be so.
    • The Field description in the reference manual incorrectly states that the halo update operation is not supported. Field halo is available in this release.
    • ESMF_FieldRegridStore() doesn't currently work on Fields created on Meshes which contain tetrahedra.
    • ESMF_FieldRegridStore() doesn't currently work on Fields created on Grids which contain a DE of width less than 2 elements.
    • ESMF_FieldRegridStore() doesn't currently work on Fields created on Grids of arbitrary distribution.
    • ESMF_FieldRegridStore() only works on Fields created on Grids with indexflag equal to ESMF_INDEX_GLOBAL. This is not the default.
    • Grid edge connections don't work in this release and will have no effect on the actions of ESMF_FieldRegridStore().
    • he ESMF_Grid creation interfaces that take the petMap argument (e.g. ESMF_GridCreateShapeTile()) mistakenly return an error for inputs that put more than one DE on a PET, if the DEs are not mapped onto the PETs in sequential order.
    • Testing on a constant field has shown that the integration weights that are generated for the conservative regridding are accurate to only 9 decimal places, rather than the expected 15 decimal places. This limits the precision of the conservative regridding method.
    • The conservative regridding may generate errors when the destination grid is of higher resolution than the source grid, specifically:
      • Values of destination points will not be set when the destination cell lies completely within a single source cell.
      • There can be regions of high interpolation error where shifting grid alignment causes different numbers of source points to contribute to the destination point.
      These drawbacks are due to the fact that this conservative method does not have a constraint on local conservation.
    • The ESMF_CubedSphereRegridEx produces some suspicious interpolation weights (large, outside [0,1]). These should be further investigated to see if they are within the bounds of expected results for non-monotonic remapping methods or represent true errors.
    • ESMF_AttributeWrite() only works for ESMF standard Attribute packages.
    • Also see SourceForge bugs list.
    Platform-specific issues:
    • Discover: ESMF_ArrayRedistOpenMPSTest.F90 fails with intelmpi when run by an automatic test script, but runs correctly when executed manually.
    • SunOS test platform: the C interfaces do not work (seg fault).
    • SunOS test platform: the ESMF_AttributeSTest.F90 and ESMF_AttributeUpdateEx.F90 fail.
    • SunOS test platform: the ESMF_FieldRegridDisjointSTest.F90 system test fails.
    • SunOS test platform: the Field test harness results are ignored.
    • Windows/MinGW with intel and intelcl: the ESMF_ArrayRedistOpenMPSTest.F90, and ESMF_FieldArbGridUTest.F90 fail.
    • Windows/Cygwin g95: the ESMF_MeshUTest.F90 fails.
    • Windows/Cygwin g95: ESMF_LogErrUTest.F90 fails intermittently.
    ESMF_4_0_0rp2

    Reference Manual
    (html) (pdf)

    User's Guide
    (html) (pdf)

    Supported Platforms
    5/11/10
    • Added support for the Intel compiler suite on the Cray XT platform.
    • Added support for the PGI 10.x compiler version.
    • The same as ESMF_4_0_0rp1 with the following exceptions:
      • Negative whole numbers specified as 8-byte floating point values for *_r8 time units in ESMF_TimeSet() and ESMF_TimeIntervalSet() methods now retain their sign, remaining negative.
      • Proxy Field objects, internally created during ESMF_StateReconcile(), now return the correct dimCount.
      • Also see SourceForge bugs list.
    Platform-specific issues:
    • Windows/Cygwin g95: ESMF_MeshUTest.F90 fails with what appears to be an incompatibility between C++ exceptions and the g95 driver on Cygwin.
    ESMF_4_0_0rp1

    Reference Manual
    (html) (pdf)

    User's Guide
    (html) (pdf)

    Supported Platforms
    3/23/10
    • List of ESMF interface changes since ESMF_4_0_0r.
    • The performance of ESMF_StateReconcile() for large processor counts has been improved dramatically.
    • During ESMF_FieldRegridStore() pole elements are no longer created if the source Grid extends all the way to the pole. Previously degenerate zero-sized elements were erroneously created around the pole for this case.
    • Added ESMF_LocStreamCreate() that allows a user to create a LocStream from a background Grid or Mesh. This call creates a new LocStream that is a redistribution of an existing LocStream. The new LocStream is created so that an element with a given set of coordinates is located on the PET with the piece of background Grid (or Mesh) that contains those coordinates.
    • Added support for replicated dimensions in Field along all Grid dimensions. A gridToFieldMap argument containing all zeros in ESMF_FieldCreate() and ESMF_FieldSetCommit() indicates that no Grid dimensions are part of the Field. Instead the Field is replicated across all the DEs along all the Grid dimensions.
    • An "advice to installers" paragraph has been added to the Installation section of the User Guide. The paragraph recommends that a standard environment variable named ESMFMKFILE be set by the system to point to the "esmf.mk" file of the ESMF installation. Standardizing this aspect of the ESMF installation process promotes the design of portable ESMF application build systems.
    • The linker flags that are provided to the user through "esmf.mk" on the ORNL Cray XT systems are now portable between jaguar (XT4) and jaguarpf (XT5).
    • The "mvapich2" option was added to build environment variable ESMF_COMM. Please refer to the second table in the Supported Platforms section of the User Guide to see where it can be used.
    • A number of known bugs have been fixed, including issues with repeating sticky and non-sticky Alarms. Please see the known bugs list for more information.
    • The same as ESMF_4_0_0r with the following exceptions:
      • The problem that caused segmentation faults in ESMF_StateReconcile() has been fixed.
      • Repeating sticky Alarms whose ringInterval is less than or equal to the Clock's timeStep now ring properly.
      • Repeating sticky Alarms that are not turned off during a waiting period of a Clock run now ring properly after the waiting period is over, Alarms are cleared, and turning off of subsequent Alarms resumes.
      • Repeating non-sticky Alarms now ring properly for a Clock that goes into ESMF_MODE_REVERSE at the Clock's stopTime, while an Alarm is ringing.
      • Windows/Cygwin gfortran and g95 no longer require the setting of environment variables ESMF_F90LINKOPTS, ESMF_CXXLINKOPTS and ESMF_F90LINKLIBS.
      • The build issue with GCC 4.4.1 has been resolved.
      • The problem that caused segmentation faults on a user's Linux system with Intel compilers and OpenMPI has been identified and corrected in the ESMF library code.
    • Also see SourceForge bugs list.

    • Platform-specific issues:
    • Windows/Cygwin g95: ESMF_LogErrUTest.F90 fails intermittently.
    ESMF_3_1_0rp3

    Reference Manual
    (html) (pdf)

    User's Guide
    (html) (pdf)

    Supported Platforms
    2/19/10 No new features.
    • The same as ESMF_3_1_0rp2 with the following exceptions:
      • Repeating sticky Alarms whose ringInterval is less than or equal to the Clock's timeStep now ring properly.
      • Repeating sticky Alarms that are not turned off during a waiting period of a Clock run now ring properly after the waiting period is over, Alarms are cleared, and turning off of subsequent Alarms resumes.
      • Repeating non-sticky Alarms now ring properly for a Clock that goes into ESMF_MODE_REVERSE at the Clock's stopTime, while an Alarm is ringing.
    • Also see SourceForge bugs list.
    ESMF_4_0_0r

    Reference Manual
    (html) (pdf)

    User's Guide
    (html) (pdf)

    Supported Platforms
    10/30/09
    • List of ESMF interface changes since ESMF_4_0_0.
    • Component methods necessary to support user-level threading were added. These methods (such as ESMF_GridCompSetMaxPEs(); see the list of interface changes for others) allow a Component to set the characteristics of its VM before it is started up. One application of these additional methods is to share the Processing Elements (PEs), provided by the Persistent Execution Threads (PETs) of the parent Component, across a lesser number of child PETs. These PEs can then be used in user-level OpenMP threading during Component method execution.
    • Implemented the underpinnings of a Component-level automatic garbage collection scheme. An initial effect is that ESMF objects that were created within a Component will be destroyed when the Component is destroyed. Additional garbage collection features will be finished and documented in future releases.
    • Corrected ESMF_StateReconcile() code to correctly handle nested States.
    • Removed the (previously undocumented) restriction that prevented States, originating from partially overlapping Components, from being reconciled with ESMF_StateReconcile().
    • The internal buffer in ESMF_StateReconcile() is now dynamically allocated. Applications with a large number of objects contained in a State should no longer run up against internal buffer size limitations.
    • Added ESMF_StateRead() and ESMF_StateWrite() methods to read and write undistributed Arrays serially from a NetCDF file. See example in the State section of the ESMF Reference Manual.
    • The Internal State implementation has been changed so that ESMF_GridCompGetInternalState() always returns the Internal State set by the last ESMF_GridCompSetInternalState() call. The previous behavior was that it would always return the Internal State set by the first ESMF_GridCompSetInternalState() call. The same is true with equivalent CplComp methods.
    • FieldBundles can now be created from Fields built on LocStreams or Meshes.
    • Fields built on LocStreams and Meshes now work with ESMF_StateReconcile() and can therefore be transferred between Components running on different sets of PETs.
    • The farray argument in ESMF_FieldGet() method has been renamed to farrayPtr to better reflect the nature of the returned parameter, and to be consistent with the ESMF_ArrayGet() method.
    • Regrid weight generation can be performed either through an offline application or through an online interface. The offline application is a separate executable which takes in two NetCDF grid files and outputs a NetCDF interpolation weight file. The online interface (ESMF_FieldRegridStore()) takes ESMF Field descriptions and generate weights. The weights can either be returned to the user through Fortran arrays, or can be applied with a parallel sparse matrix multiply in an ESMF_FieldRegrid() call.
    • The ESMF offline weight generation application supports:
      • Regridding between a pair of 2D logically rectangular Grids mapped to a sphere.
      • Multiple pole options: 1) Pole value is an average of source values around pole, 2) Pole value is an average of n-point stencil of source values, or 3) No pole.
      • Higher-order patch recovery interpolation.
    • For details please see the "File Based Regrid Weight Application" section in the "Infrastructure: Fields and Grids" part of the ESMF reference manual.
    • The ESMF online weight generation interface supports:
      • Regridding between any combination of 2D Meshes composed of triangles and quadrilaterals, and 2D Grids composed of a single logically rectangular tile with no edge connections.
      • Regridding between any combination of 3D Meshes composed of hexahedrons, and 3D Grids composed of a single logically rectangular region.
      • Regridding between a pair of 2D logically rectangular Grids mapped to a sphere.
      • One pole option: Pole value is an average of source values around pole.
      • Source and destination point masking of Grids.
      • The ability to choose what happens to unmapped destination points: either 1) return an error if any are found, or 2) ignore and don't interpolate them.
      • Two interpolation options, 1) bilinear or 2) higher order.
      For details please see the "Field Regrid" section in the "Infrastructure: Fields and Grids" part of the ESMF reference manual.
    • A more efficient tree-based search has been added to the ESMF_FieldRegridStore() method which improves its average running time by an order of magnitude.
    • A zeroflag and a checkflag have been added to the ESMF_FieldRegrid() call. The zeroflag allows the user to control how the destination Field should be initialized before regrid, and the checkflag allows the user to turn on more extensive error checking.
    • Optimized performance of ESMF_ArraySparseMatMulStore() for cases where the sparse matrix elements (i.e. factorList and factorIndexList arguments) are provided in parallel across the calling PETs.
    • Added an overloaded method for ESMF_GridCreate() to read the specification for a 2D regularly distributed rectilinear Grid from an XML file. See example in the Grid section of the ESMF Reference Manual.
    • The Grid class now contains a DistGrid per stagger location. This change allows the memory within a Field to exactly match the size of the exclusive region (the DE local portion) of the DistGrid associated with each stagger location. Previously the exclusive region of every stagger location needed to be sized to contain the exclusive region of all the stagger locations. This change affects some interfaces, such as ESMF_GridGet(), which now can return a DistGrid per stagger location.
    • Methods in the Mesh class now generate error messages through the standard ESMF LogErr system.
    • Mesh methods can now operate correctly in child Components which are running on fewer PETs than their parents.
    • The documentation for Meshes, and Fields created on Meshes, has been vastly improved, including multiple examples of Mesh creation and improved method descriptions. In addition, more extensive input checking has been added to Mesh to make its use easier.
    • Added bounds checking to the defaultvalueList argument in ESMF_AttributeGet().
    • Added a standard Attribute package to Array.
    • Added Attribute hierarchy linking to Array and ArrayBundle.
    • Time Manager interfaces have been made more robust to gracefully handle uninitialized objects and to eliminate memory leaks.
    • Two additional system tests were added to demonstrate the sparse matrix multiply operation for Fields based on LocStreams and Meshes.
    • A new system test, ESMF_ArrayRedistOpenMP, was added to demonstrate support for user-implemented OpenMP parallelism in ESMF Components.
    • Expanded the ESMF_CompFortranAndC system test to demonstrate the inter-language usage of Mesh and Field objects through States.
    • Reworked the VM documentation section in the Reference Manual to be more user oriented.
    • Removed the remaining dependency on explicit knowledge of the shape of the Fortran dope vector. This makes the library more portable across platforms, and more robust with compiler upgrades.
    • Implemented a clearer OpenMP support strategy. Two new environment variables were introduced that can be set to ON/OFF. ESMF_OPENMP, when set to ON during the library build, will compile all library sources using OpenMP compiler flags. In this case the OpenMP compiler flags automatically become part of the compiler flags exported by esmf.mk. If the ESMF library was compiled with ESMF_OPENMP set to OFF, a second environment variable, ESMF_TESTFORCEOPENMP, can be set to force the test and example codes to build with OpenMP compiler flags.
    • A new Windows option, intelcl, has been added. This allows building ESMF with the Intel Fortran compiler and the Microsoft cl (Visual C++) compiler, using the MinGW toolchain.
    C interfaces:
    • Added to the C interface the ability to add and retrieve Field objects from a State.
    • Added C interfaces for the Field class. These enable a Field to be created on a Mesh, and allow the Mesh to be retrieved.
    • Segmentation faults have been reported for ESMF_StateReconcile().
    • ESMF_FieldRegridStore() doesn't currently work on Fields created on Meshes which contain tetrahedra.
    • ESMF_FieldRegridStore() doesn't currently work on Fields created on Grids which contain a DE of width less than 2 elements.
    • ESMF_FieldRegridStore() doesn't currently work on Fields created on Grids of arbitrary distribution.
    • ESMF_FieldRegridStore() only works on Fields created on Grids with indexflag equal to ESMF_INDEX_GLOBAL. This is not the default.
    • Grid edge connections don't work in this release and will have no effect on the actions of ESMF_FieldRegridStore().
    • The ESMF_Grid creation interfaces that take the petMap argument (e.g. ESMF_GridCreateShapeTile()) mistakenly return an error for inputs that put more than one DE on a PET, if the DEs are not mapped onto the PETs in sequential order.
    • FieldBundles don't currently enforce that every contained Field is built on the same Grid, although the documentation says that this should be so.
    • Proxy Field objects, internally created during ESMF_StateReconcile(), return the wrong dimCount.
    • Repeating sticky Alarms whose ringInterval is less than or equal to the Clock's timeStep do not ring properly.
    • Repeating sticky Alarms that are not turned off during a waiting period of a Clock run do not ring properly after the waiting period is over, Alarms are cleared, or turning off of subsequent Alarms resumes.
    • Repeating non-sticky Alarms do not ring properly for a Clock that goes into ESMF_MODE_REVERSE at the Clock's stopTime, while an Alarm is ringing.
    • Negative whole numbers specified as 8-byte floating point values for *_r8 time units in ESMF_TimeSet() and ESMF_TimeIntervalSet() methods incorrectly lose their sign, becoming positive.
    • ESMF_AttributeWrite() only works for ESMF standard Attribute packages.
    • Also see SourceForge bugs list.
    Platform-specific issues:
    • IBM Bluefire: all the ArrayHarness unit tests fail with ESMF_ABI=32.
    • Discover: ESMF_ArrayRedistOpenMPSTest.F90 fails with intelmpi when run by an automatic test script, but runs correctly when executed manually.
    • Linux ia32 test platform: ESMF_ConfigOverviewEx.F90 and ESMF_ConfigUTest.F90 fail with PGI.
    • Linux ia32 test platform: ESMF_MeshUTest.F90 fails with PGIGCC configuration.
    • Linux x86_64 test platform: Fails 15 system tests with ESMF_ABI=x86_64_medium.
    • SunOS test platform: the C interfaces do not work (seg fault).
    • SunOS test platform: the ESMF_AttributeSTest.F90 and ESMF_AttributeUpdateEx.F90 fail.
    • SunOS test platform: the ESMF_FieldRegridDisjointSTest.F90 system test fails.
    • Windows/MinGW test platform: the ESMF_ArrayRedistOpenMPSTest.F90 and ESMF_FieldArbGridUTest.F90 fail, using the intel configuration.
    • Windows/MinGW test platform: the ESMF_ArrayRedistOpenMPSTest.F90, ESMF_FieldArbGridUTest.F90, and ESMF_FieldCreateGetUTest.F90 fail, using the intelcl configuration.
    • Windows/Cygwin gfortran: the following environment variables may need to be set on some systems in order to properly link applications:
      • export ESMF_F90LINKOPTS=-Wl,--enable-auto-import
      • export ESMF_CXXLINKOPTS=-Wl,--enable-auto-import
    • Windows/Cygwin g95: the ESMF_MeshUTest.F90 fails.
    • Windows/Cygwin g95: the following environment variables may need to be set on some systems in order to properly link applications:
      • export ESMF_F90LINKLIBS=-lgcc_s
      • export ESMF_F90LINKOPTS=-Wl,--enable-auto-import
      • export ESMF_CXXLINKOPTS=-Wl,--enable-auto-import
    • The ESMF build has been reported failing for GCC 4.4.1.
    • Segmentation faults have been reported for some of the bundled test codes on a user's Linux system with Intel compilers and OpenMPI.
    ESMF_4_0_0

    Reference Manual
    (html) (pdf)

    User's Guide
    (html) (pdf)

    Supported Platforms
    5/15/09
    • List of ESMF interface changes since ESMF_3_1_1.
    • Added public C interfaces for select methods of the Mesh class. The prefix for these methods is ESMC. We do not yet have a Reference Manual that describes these interfaces.
    • Modifications were made to selected GridComp and CplComp methods that enable use of keyword syntax for procedure arguments. This was not permitted before because they did not have explicit interfaces. Using keyword syntax ensures that the compiler will check argument types. We strongly recommend that users take advantage of this feature. All of the ESMF calls that take dummy procedure arguments, or call into previously registered user procedures, now provide explicit interfaces: ESMF_GridCompSetVM(), ESMF_GridCompSetServices(), ESMF_GridCompSetEntryPoint(), ESMF_GridCompInitialize(), ESMF_GridCompRun(), ESMF_GridCompFinalize(), ESMF_GridCompReadRestart(), ESMF_GridCompWriteRestart(), ESMF_CplCompSetVM(), ESMF_CplCompSetServices(), ESMF_CplCompSetEntryPoint(), ESMF_CplCompInitialize(), ESMF_CplCompRun(), ESMF_CplCompFinalize(), ESMF_CplCompReadRestart(), ESMF_CplCompWriteRestart().
    • Those methods that call into a user supplied routine provide two separate (optional) return codes - the standard "rc" return code for the ESMF library, and the "userRc" return code set by the user routine. If keyword syntax is not used, the two return codes can get confused, with dire results. Specifically, using the old interface argument order, without keyword syntax, would lead to incorrect rc association. Users are strongly encouraged to make appropriate changes to their codes. See the API changes page for details.
    • ESMF_GridCompSetServices() and ESMF_CplCompSetServices() now support runtime loading of ESMF components via shared objects. This allows users to maintain separate build systems for individual components in the same application. The newly added ESMF_ArrayRedistSharedObj system test demonstrates this feature.
    • Changed the behavior of ESMF_GridCompSetServices() and ESMF_CplCompSetServices() to support the MAPL/GEOS-5 Component hierarchy paradigm. For most user code the SetServices change is expected to be completely transparent. All previous versions of ESMF executed the specified SetServices routine in the context of the parent VM. Now, the specified public child routine is executed within the context of the child VM. This means that calls that would affect the child VM must be issued before the child SetServices routine is called. Two new interfaces, ESMF_GridCompSetVM() and ESMF_CplCompSetVM(), have been added that allow child component code to be called and executed from within the context of the parent VM. The benefit of the SetServices change is that the child SetServices routine may now contain calls that require execution from within the context of the child VM, e.g. creation of sub-components.
    • Added standard ReadRestart and WriteRestart methods to the ESMF_GridComp and ESMF_CplComp interfaces.
    • The ESMF_GridCompGet() and ESMF_CplCompGet() interfaces now offer access to the current method (for example, initialize, run, ...) and current phase of a Component.
    • Added "attachable methods" to ESMF States. User supplied methods can be attached, executed and removed. Currently attached user methods are ignored during ESMF_StateReconcile(), and attached methods cannot be copied or moved between States. For details see the new "Attachable Methods" section in the "Infrastructure: Utilities" part of the ESMF Reference Manual.
    • The InternalState documentation in the Reference Manual has been reworked to be more applicable to real world situations.
    • Added support in ESMF_FieldCreate() methods for Fields on arbitrarily distributed Grids. ESMF_FieldRegrid() does not yet work for arbitrarily distributed Grids. Overall, communications using Fields on arbitrarily distributed Grids have not been well tested.
    • The ESMF_FieldRegridStore() call now supports source and destination masking. The user can use this masking to tell the regrid to ignore source and/or destination points.
    • The ESMF_FieldRegridStore() call now has an unmappedDstAction flag. This allows the user to choose if they want unmapped destination points to cause an error or to be ignored.
    • A number of bug fixes and tweaks have improved the accuracy and stability of ESMF_FieldRegridStore(). This is in particular true of the higher-order interpolation weight generation.
    • Added a new public method ESMF_FieldGetBounds() to allow a user to query localDe specific data bounds.
    • Grids now have the capability to store other types of data besides coordinates. This data is referred to as Grid items. A set of new interfaces which mirror the ESMF_GridCoord interfaces has been added to Grid to support this functionality. The Grid currently supports four items and a set of new flags (ESMF_GRIDITEM_MASK, ESMF_GRIDITEM_AREA, ESMF_GRIDITEM_AREAM, ESMF_GRIDITEM_FRAC) has been added to be used in the interface to indicate the item being accessed. The ESMF_GRIDITEM_MASK item is used to store mask information for use by ESMF_FieldRegridStore().
    • Added support to create arbitrarily distributed Grids using ESMF_GridCreateShapeTile(), ESMF_GridCreate(), and ESMF_GridSetCommitShapeTile(). Other Grid interfaces have been modified as needed. Grid items do not work for arbitrarily distributed Grids.
    • Changed ESMF_GridGet() interface to get the information defined for the arbitrarily distributed Grid, including arbDim (the dimension in the DistGrid that represents the arbitrarily distributed dimension(s)), localCount (the number of PET-local arbitrarily distributed index locations), and localIndices (a list of PET-local arbitrarily distributed index locations). The names of these arguments are likely to change in future releases.
    • Added ability to read Attributes from an XML file via new method ESMF_AttributeRead(), which then attaches the Attributes to a Component, Field, or Grid.
    • XML Attribute file reading is done with the open source third party Xerces C++ library. ESMF must be built against the Xerces library, which requires the setting of additional ESMF environment variables. See the "Third Party Libraries" section in the ESMF User's Guide for details.
    • Implemented ability to specify time units as 8-byte floating point values in Time Manager methods.
    • Implemented ESMF standard Attribute packages for Components, State, Field, and Grid.
    • Added Attributes to ArrayBundle and DistGrid.
    • Allow Field Attribute hierarchy to connect to Grid Attribute hierarchy.
    • Allow ESMF_AttributeWrite() to be called from Components, State, FieldBundle, and Field.
    • Allow ESMF_AttributeCopy() to locally copy Attribute values between Components.
    • Added ESMF_AttributeUpdate() routine to build consistent Attribute structure and values for objects defined across a VM. One use of this feature is to use it in conjunction with ESMF_AttributeCopy() during the CplComp run method in order to transfer Attributes between GridComps defined on mutually exclusive sets of PETs.
    • Overloaded ESMF_AttributeRemove() to remove entire Attribute packages, Attributes in an Attribute package, or just single Attributes.
    • Added the ability to nest Attribute packages inside of each other.
    • ESMF_StateReconcile() now has a flag to allow optional Attribute reconciliation. The default is currently that Attribute reconciliation is off, due to concerns about performance and robustness.
    • Added a set of overloaded ESMF_AttributeLinkRemove() interfaces. These detach an object's Attribute hierarchy from that of another object.
    • Modified the ESMF_Attribute system test to use ESMF_AttributeUpdate(), ESMF_AttributeCopy(), and ESMF_StateReconcile() on both States and Components where appropriate.
    • The range of Fortran unit numbers used within ESMF can be overridden via the ESMF_Initialize() call by using the new IOUnitLower and IOUnitUpper optional arguments. A new ESMF_IOUnitGet() method to find an unused unit number within the ESMF unit number range is provided as a service to application developers.
    • ESMF_IOUnitFlush() is provided as a common method to flush Fortran unit numbers. This is intended for use as a service to application developers.
    • Added ESMF_ConcurrentEnsemble system test demonstrating an ensemble in which different ensemble members, which are distinctly different components, run concurrently on distinct sets of PETs.
    • Added ESMF_InternalStateEnsemble system test demonstrating how different ensemble members representing the same Component can be written using a single Component object which indexes different elements of an ESMF internal State.
    • Activated TestHarness for Array and Field testing.
    • Added support for Windows MinGW/MSYS, using the Intel Fortran and C++ compiler and Microsoft MPI. This was developed and tested under Windows Compute Cluster Server (CCS) 2003.
    • Standardized 3rd party library support (NETCDF, PNETCDF, LAPACK, XERCES).
    • An offline file-to-file regridding application has been developed as part of ESMF. This application takes two netcdf grid files and outputs interpolation weights generated using ESMF's higher-order interpolation. This application supports a number of options for handling the pole: no pole, treating the pole as the average of the values surrounding the pole, and an n-point stencil average of the points surrounding the pole. The format of the files makes this application a suitable replacement for SCRIP. For details see the new "File Based Regrid Weight Application" section in the "Infrastructure: Fields and Grids" part of the ESMF Reference Manual.
    • Furthermore, all the features listed in the release notes for ESMF_3_1_0rp2 apply to this release.
    • The ESMF_StateReconcile() call, generally used by couplers dealing with Components running on exclusive petLists, has an internal size limit (which will be addressed in a subsequent release). There is no hard-coded limit to how many elements, i.e. States, Bundles, Fields, Arrays can be added to the State. However, there is a limit, roughly 20, on the maximum number of sub-elements contained by any element in the State passed into ESMF_StateReconcile()
    • The ESMF_StateReconcile() call fails for nested States.
    • ESMF_LOG_SINGLE is known to be broken. The symptoms are platform dependent and may appear non-deterministic.
    • Fields built on LocStreams and Meshes will not work with ESMF_StateReconcile() and therefore cannot be transfered between Components running concurrently.
    • ESMF_FieldRegrid() is only implemented for Fields created on Grids with indexflag equal to ESMF_INDEX_GLOBAL. This is not the default.
    • Proxy Field objects, internally created during ESMF_StateReconcile(), return the wrong dimCount.
    • The ESMF_Grid creation interfaces that take the petMap argument (e.g. ESMF_GridCreateShapeTile()) mistakenly return an error for inputs that put more than one DE on a PET, if the DEs are not mapped onto the PETs in sequential order.
    • Grid connections don't work in this release and will have no effect on the actions of ESMF_FieldRegridStore().
    • Repeating sticky Alarms whose ringInterval is less than or equal to the Clock's timeStep do not ring properly.
    • Repeating sticky Alarms that are not turned off during a waiting period of a Clock run do not ring properly after the waiting period is over, Alarms are cleared, or turning off of subsequent Alarms resumes.
    • Repeating non-sticky Alarms do not ring properly for a Clock that goes into ESMF_MODE_REVERSE at the Clock's stopTime, while an Alarm is ringing.
    • Negative whole numbers specified as 8-byte floating point values for *_r8 time units in ESMF_TimeSet() and ESMF_TimeIntervalSet( ) methods incorrectly lose their sign, becoming positive.
    • Array and ArrayBundle Attribute hierarchies are not implemented yet.
    • ESMF_AttributeWrite() only works for ESMF standard Attribute packages.
    • In the Reference Manual, "Figure 29: The structure of the Attribute class" shows an attrCount variable which is no longer present, and does not show two vectors of Attribute pointers called packList and linkList.
    • In the API description of ESMF_AttributeLink() and ESMF_AttributeLinkRemove(), in the ESMF Reference Manual, the argument keyword should be "comp1" instead of "comp".
    • Also see SourceForge bugs list.
    Platform-specific issues:
    • The Mesh and ESMF_FieldRegrid() code is unstable when compiled with PGI.
    • ESMF_DistGridCreateGetUTest.F90 fails on IBM Lightning with PGI.
    • ESMF_ClockUTest.F90 fails on IBM/AIX in ESMF_BOPT=O mode.
    • ESMF_ClockUTest.F90 fails on Linux with NAG.
    • All the ArrayHarness unit tests fail on IBM Bluefire with ESMF_ABI=32 and ESMF_BOPT=g.
    • ESMC_ConfigUTest.C fails on IBM Bluefire with ESMF_ABI=64 and ESMF_BOPT=O.
    • ESMF_GridCoordUTest.F90 fails on Darwin with Absoft and with ESMF_BOPT=O.
    • The Attribute unit tests fail with Lahey and ESMF_BOPT=O.
    • ESMF_ConfigOverviewEx.F90 and ESMF_ConfigUTest.F90 fail on ia32 with PGI.
    • Examples ESMF_LogErrEx.F90, ESMF_AttReadGridCompEx.F90, ESMF_AttReadGridEx.F90, and ESMF_AttributePackageEx.F90 fail on Linux with the g95 v0.5 compiler.
    • The C unit tests and examples fail to link on wJet with the Intel compiler.
    ESMF_3_1_0rp2

    Reference Manual
    (html) (pdf)

    User's Guide
    (html) (pdf)

    Supported Platforms
    2/06/09
    • List of ESMF interface changes since ESMF_3_1_0rp1.
    • Added support for replicated dimensions in Field. A "0" entry in the gridToFieldMap argument in ESMF_FieldCreate() and ESMF_FieldSetCommit() indicates that a Grid dimension is not part of the Field. Instead the Field is replicated across all DEs along the corresponding Grid dimension.
    • Overloaded ESMF_FieldCreate() to accept a Fortran array pointer and preserve its properties (such as bounds and ability to deallocate).
    • Overloaded ESMF_FieldSetCommit() to accept a Fortran array pointer and preserve its properties (such as bounds and ability to deallocate).
    • The Grid now supports an ESMF_INDEX_USER option for the indexflag. This allows the user to explicitly set the lower bound of their local index space. This is necessary when creating a Field from a Fortran array pointer with non-standard indexing.
    • Overloaded ESMF_ArrayCreate() to accept a Fortran array pointer and preserve its properties (such as bounds and ability to deallocate).
    • Optimized ESMF_ArraySMM() and ESMF_ArrayBundleSMM() runtime performance, scalability and memory usage.
    • Added support for optimized communications for Arrays with different distributions, dimensions and sizes in the same ArrayBundle.
    • Added support for DEs with zero elements in DistGrid, Array and Field.
    • Made localDe argument optional in ESMF_ArrayGet() and ESMF_FieldGet() methods where not prohibited by overloading restrictions.
    • Alarms set to ring at timeStep 0, upon ESMF_AlarmCreate(), now ring properly.
    • Alarms now ring properly for a Clock in ESMF_MODE_REVERSE.
    • Alarms now ring properly for a Clock using a negative timeStep.
    • Added default pointer size override feature to build system utilizing new environment variables ESMF_F90_PTR_BASE_SIZE and ESMF_F90_PTR_PLUS_RANK.
    • The same as ESMF_3_1_0rp1 with the following exceptions:
      • The failure of ESMF_VMSendVMRecvUTest.F90 on IBM BlueGene/L in uni-PET mode has been fixed.
      • On the SunOS Sparc set ESMF_OPTLEVEL=2, otherwise the ESMF_DELayoutWorkQueueUTest.F90 unit test hangs running with ESMF_BOPT=O.
      • The ESMC_Component examples fail on the PC Xeon(64) Cluster with the Intel 9.1 compiler and ESMF_COMM=mpich2.
      • A discrepancy in the names of ESMF_FieldBundle arguments in the ESMF_AttributeGet()/Set() interface documentation was corrected. The ESMF Reference Manual was updated to reflect this change.
    • Also see SourceForge bugs list.
    ESMF_3_1_1

    Reference Manual
    (html) (pdf)

    User's Guide
    (html) (pdf)

    Supported Platforms
    9/11/08
    • List of ESMF interface changes since ESMF_3_1_0rp1.
    • Added public C interfaces for select methods of the following classes:
      • GridComp, CplComp
      • State
      • Clock, TimeInterval, Time, Calendar
      • Config
      • Array
      • ArraySpec
      • DistGrid
      • VM
      • The prefix for these methods is ESMC. We do not yet have a Reference Manual that describes these interfaces.
    • Restored most Field and FieldBundle communication methods except the halo update methods.
    • Added a method to perform a bilinear or higher order regridding between two Fields constructed on Grids. The Fields must be based on Grids with indexflag equal to ESMF_INDEX_GLOBAL (not the default) and must have data at cell centers (the default). This method will not work for Fields built on LocStreams or Meshes. Interpolation weights are computed in parallel.
    • Added a LocStream (Location Stream) class to represent and manipulate observational data streams. Fields can be created using a LocStream instead of a Grid.
    • Added ability to represent an unstructured Mesh in ESMF. Fields can be created using a Mesh instead of a Grid.
    • The DistGrid interface changes in this release reflect modifications that are necessary to support factorized sequence indices in the future. This feature is required for the ongoing arbitrary Grid development and is not fully supported yet.
    • Added Attributes to the CplComp and GridComp classes.
    • Added the ability to build Attribute hierarchies.
    • Added Attribute packages to CplComp, GridComp, and Field. Either standard or custom Attribute packages can be created.
    • Added a method to locally copy Attributes between States called ESMF_AttributeCopy().
    • Overloaded ESMF_AttributeGet() and ESMF_AttributeSet() to allow the manipulation of Attributes in Attribute packages.
    • Added an optional default value to ESMF_AttributeGet().
    • Added an interface to write out an Attribute hierarchy in either tab delimited or XML format called ESMF_AttributeWrite().
    • Added character lists as an option for Attribute type.
    • Standardized the use of count and itemCount in Attribute calls and made them optional in most cases.
    • Added support for the Modified Julian Day calendar, ESMF_CAL_MODJULIANDAY.
    • Alarms now ring properly for a clock in ESMF_MODE_REVERSE.
    • Argument lists for Fortran-callable entry points which contain logical data have been regularized to always use Fortran LOGICAL. Previously, some routines used the internal ESMF_Logical derived type.
    • Parallel build of the ESMF library supported via GNU make -j option.
    • Added ESMF_CompFortranAndC system test that tests and demonstrates passing of State objects between Fortran and C components.
    • Added ESMF_Attribute system test demonstrating Attribute packages and hierarchies.
    • Added ESMF_FieldBundleSMM system test demonstrating the sparse matrix multiply using FieldBundles.
    • Added ESMF_ArrayRedist3D system test demonstrating a three dimensional redistribution operation for Arrays.
    • Added ESMF_SeqEnsemEx system test demonstrating an ensemble in which different ensemble members, which are distinctly different components, run on the same set of PETs.
    • The same as ESMF_3_1_0rp1 with the following exception:
      • The problem with the ESMC Component interfaces on XLF and PGI compilers has been fixed.
    • For higher-order interpolation, ESMF_FieldRegrid() can give wrong answers for multiple processors. We recommend only using this on a single processor.
    • In this release the accuracy of the higher-order interpolation has been degraded by a bug. For better results we recommend using ESMF v4.0.0. or later.
    • ESMF_FieldRegrid() has problems with the pole. When using ESMF_REGRID_SCHEME_FULL3D, we recommend not using any destination points which either lie in a cell next to the pole (for both bilinear and higher-order) or two cells away from the pole (for just higher-order).
    • Fields built on LocStreams and Meshes cannot be transferred between Components running concurrently.
    • ESMF_FieldRegrid() is only implemented for Fields with data located at cell centers.
    • ESMF_FieldRegrid() is only implemented for Fields created on Grids with indexflag equal to ESMF_INDEX_GLOBAL. This is not the default.
    • The Grid connections don't work in this release and will have no effect on the actions of ESMF_FieldRegrid().
    • New Attribute methods are unreliable across a range of platforms. They should be considered an early prototype and used with caution.
    • On IBM Bluegene the ESMF_FieldRegridEx.F90 and ESMF_FieldSphereRegridEx.F90 fail with ESMF_BOPT=O.
    • A problem in the Config C interface leads to failures in the ESMC_ConfigUTest.C on the XLF compiler.
    • ESMF_AttributeWrite() does not write out the values in the header of the .xml file generated when calling with ESMF_AttWriteFlag=ESMF_ATTWRITE_XML.
    • The Use Test Cases were not updated or tested for this release.
    • The ESMF_ArrayHarness2UTest is incorrectly reported as failing when executed through any of ESMF's batch mode mpirun scripts.
    • Also see SourceForge bugs list.
    ESMF_3_1_0rp1

    Reference Manual
    (html) (pdf)

    User's Guide
    (html) (pdf)

    Supported Platforms
    7/25/08
    • List of ESMF interface changes since ESMF_3_1_0r.
    • Added Attributes to the Grid class.
    • Added Attributes to the Array class.
    • Added minIndex and maxIndex arguments to ESMF_GridGet(). These arguments apply to the first tile.
    • Added minIndexPDimPDe and maxIndexPDimPDe arguments to ESMF_DistGridGet().
    • Added ESMF_GridMatch() method to compare two Grid objects.
    • A new section in the User's Guide entitled "Setting up ESMF to run Test Suite Applications" was added that outlines how users can specify their own mpirun scripts. ESMF now consistently uses the setting of environment variable ESMF_MPIRUN to launch the bundled test suite and example applications.
    • A number of known bugs have been fixed. Please see the known bugs list.
    • The same as ESMF_3_1_0r with the following exceptions:
      • The problem with ESMF_ArrayRedist() and ESMF_ArraySMM() methods hanging in mpiuni mode has been fixed.
      • The problem with ESMF_ArrayRedist, ESMF_DirectCoupling and ESMF_RecursiveComponent system tests failing with a segmentation fault in ESMF_BOPT=O mode on x86_64 Darwin systems with Intel compilers has been fixed.
      • States containing FieldBundles can now be reconciled, so FieldBundles can be transferred between concurrently operating components.
      • Reconciliation errors for Fields created with user supplied arguments for gridToFieldMap, maxHaloWidths, and ungriddedBounds have been fixed.
      • Proxy Field objects, internally created during ESMF_StateReconcile(), are now properly destroyed during ESMF_StateDestroy().
      • The ESMF_AttributeGet() and ESMF_AttributeSet() Reference Manual entries that did not show a required "count" argument have been fixed.
    • Also see SourceForge bugs list.
    ESMF_3_1_0r

    Reference Manual
    (html) (pdf)

    User's Guide
    (html) (pdf)

    Supported Platforms
    5/21/08
    • List of ESMF interface changes since ESMF_3_1_0p1.
    • Shortened the names of State add and get methods so that they do not refer to the item being retrieved. For example, ESMF_StateGetField() is now just ESMF_StateGet().
    • Fixed problem in the InternalState implementation that rendered the returned Fortran pointer to the user defined derived type incomplete and unsuitable for memory deallocation across a wide range of platforms.
    • Implemented ArrayBundle class with limited support for sparse matrix and redist communication methods.
    • Renamed Bundle to FieldBundle.
    • Shortened ESMF_FieldBundleGetField() and ESMF_FieldBundleAddField() method names to ESMF_FieldBundleGet() and ESMF_FieldBundleAdd(), respectively.
    • State, Field, and FieldBundle Attribute methods were consolidated into a single set of Attribute methods that are overloaded for object type. For example, ESMF_AttributeGet() is overloaded to get Attributes from States, Fields, and Bundles. This reduces the number of documented calls in the interface significantly.
    • Field, Grid, and Array classes now interpret their width and bound arguments in a consistent way. The order of dimensions is defined by their index sequence in memory.
    • Restored the ESMF_FieldCreate() method that allows Field creation from an existing Grid and Array.
    • Restored the ESMF_FieldGet() that returns the native Fortran data pointer to Field data.
    • Implemented ESMF_FieldCreateEmpty(), which allocates the shell of a Field, and ESMF_FieldSetCommit(), which finishes a Field started by ESMF_FieldCreateEmpty().
    • Implemented a ESMF_FieldGet() method that provides precomputed bounds and counts from Grid and other Field creation parameters. This method helps users allocate native Fortran arrays to be used in ESMF_FieldCreate() and ESMF_FieldSetCommit() methods.
    • Switched Grid and Field localDe arguments to non-optional to conform to Array.
    • Grid undistributed dimensions are now always DistGrid dimensions distributed across 1 DE. This simplified interfaces and implementation. Field interfaces also reflect this change.
    • Grid now destroys internally created DELayouts, DistGrids, and Arrays.
    • Regular decompositions now behave more intuitively. Previously stagger edge padding could cause cells to be divided up oddly.
    • Added method to get GridStatus from a Grid.
    • Renamed ArraySparseMatMul methods ArraySMM to avoid method name size limits on some compilers.
    • Implemented more flexible zeroflag option in ESMF_ArraySMM() that supports zero out of only those destination elements that will be updated by the sparse matrix multiplication. This preserves special fill values if they exist.
    • Added ESMF_ArraySMMRelease() and ESMF_ArrayRedistRelease() interfaces for consistency.
    • Added ESMF_ConcurrentComp system test that demonstrates a very simple concurrent coupling scenario.
    • Added ESMF_ArrayBundleSparseMatMul system test.
    • Added ESMF_ArrayRedistMPMD system test to test and demonstrate ESMF's multiple executable support.
    • Added ESMF_RecursiveComponent system test to test and demonstrate the recursive use of ESMF Components.
    • Implemented FieldCreateGetUTest, FieldStressUTest, FieldBundleCrGetUTest to help diagnose/fix various memory associated issues.
    • Added petascale performance and memory optimization of Component creation.
    • Added support for SunOS/sparc.
    • Added support for new Cray XT4 compiler wrappers.
    • Added support for Darwin on x86 and x86_64 systems with Intel compilers.
    • Added support for gfortran version 4.3.
    • Added explicit interface blocks that ensure conformance to ESMF interfaces in user-written routines called by ESMF_GridCompSetServices() or ESMF_CplCompSetServices() and registered via the ESMF_GridCompSetEntryPoint() or ESMF_CplCompSetEntryPoint() methods.
    • Changed environment variable ESMF_EXHAUSTIVE to ESMF_TESTEXHAUSTIVE.
    • Added environment variable ESMF_TESTMPMD to turn on/off MPMD system tests.
    • Implemented running only single processor system tests from the run_system_test_uni target.
    • Removed netcdf_stubs and hdf_stubs.
    • Checked for and fixed memory leaks.
    • Reviewed and updated introductory and general sections of the Reference Manual.
    • Stagger location and bounds diagrams were added to the Grid documentation.
    • The ESMF_StateReconcile() call, generally used by couplers dealing with Components running on exclusive petLists, has an internal size limit (which will be addressed in a subsequent release). There is no hard-coded limit to how many elements, i.e. States, Bundles, Fields, Arrays can be added to the State. However, there is a limit, roughly 20, on the maximum number of sub-elements contained by any element in the State passed into ESMF_StateReconcile() .
    • FieldBundles contained inside States cannot be reconciled. This means that FieldBundles cannot be transferred between components running concurrently.
    • There are reconciliation errors for Fields created with user supplied arguments for gridToFieldMap, maxHaloWidths, and ungriddedBounds.
    • Proxy Field objects, internally created during ESMF_StateReconcile(), are not properly destroyed during ESMF_StateDestroy(). In applications that reconcile States across exclusive sets of PETs this will lead to a small memory leak for each contained Field object. This should not cause significant problems if ESMF_StateReconcile() is called once during initialization as recommended.
    • ESMF_ArrayRedist() and ESMF_ArraySMM() methods will hang in mpiuni mode. We expect this to be fixed in an upcomng patch release.
    • The ESMF_Grid creation interfaces that take the petMap argument (e.g. ESMF_GridCreateShapeTile()) mistakenly return an error for inputs that put more than one DE on a PET, if the DEs are not mapped onto the PETs in sequential order.
    • Repeating sticky Alarms whose ringInterval is less than or equal to the Clock's timeStep do not ring properly.
    • Repeating sticky Alarms that are not turned off during a waiting period of a Clock run do not ring properly after the waiting period is over, Alarms are cleared, or turning off of subsequent Alarms resumes.
    • Repeating non-sticky Alarms do not ring properly for a Clock that goes into ESMF_MODE_REVERSE at the Clock's stopTime, while an Alarm is ringing.
    • The documentation for ESMF_AttributeGet() and ESMF_AttributeSet() calls for multi-valued list Attributes does not show a required integer input "count" argument after the "name" argument. This has been corrected in the Reference Manual for this release shown at left. However a user who builds the documentation themselves will still see this error.
    • ESMF_LOG_SINGLE is known to be broken. The symptoms are platform dependent and may appear non-deterministic.
    • On Cray X1 the ESMF_DistDir system test fails.
    • On Cray X1 the ESMF_String unit test fails.
    • On IBM BlueGene/L ESMF_VMSendVMRecvUTest.F90 fails in uni-PET mode.
    • On x86_64 Darwin systems with Intel compilers the ESMF_ArrayRedist, ESMF_DirectCoupling and ESMF_RecursiveComponent system tests fail with segmentation fault in ESMF_BOPT=O mode.
    • ESMC_Component examples fail with the XLF and PGI compilers as well as for the Linux/absoftintel combination.
    • All NAG compiler combinations we tested (on Linux as well as on Darwin) show random run-time failures when ESMF is compiled in optimized mode and also when user code that links with ESMF is compiled in optimized mode. Consequently ESMF sets ESMF_OPTLEVEL to 0 (zero) by default when compiling ESMF with NAG and we advise NAG users to not use optimization on user code that links against the ESMF library.
    • Last time we were able to test with PGI 6.2.5 on Cray XT3/4 it failed to compile ESMF with an internal compiler error. The same version of PGI works fine on non-Cray machines. We were unable to test with this compiler version on Cray XT3/4 to determine the status of this issue with 3.1.0r.
    • Also see SourceForge bugs list.
    ESMF_3_1_0p1

    Reference Manual
    (html) (pdf)

    User's Guide
    (html) (pdf)

    Supported Platforms
    2/27/08
    • List of ESMF interface changes since ESMF_3_1_0.
    • Added garbage collection of proxy objects to ESMF_StateDestroy(). This fixes several memory leaks in system tests.
    • Re-enabled the ESMF_FieldPrint() method.
    • Changed ESMF_GridSetShapeTile() to ESMF_GridSetCommitShapeTile(). In addition to just setting data the newly renamed Grid method now also does an internal commit.
    • Added deep copy overload to ESMF_ArrayCreate().
    • Implemented transpose mode for Array redistribution in ESMF_ArrayRedistStore().
    • Implemented factor option for Array redistribution in ESMF_ArrayRedistStore(). Now the user can multiply the source Array by a constant factor as part of the redistribution operation.
    • Implemented support for direct coupling between components via RouteHandle-based communications (e.g. ESMF_ArraySparseMatMul(), ESMF_ArrayRedist()).
    • Added ESMF_DirectCoupling system test.
    • Implemented general TK support in ESMF_ArraySparseMatMul(). This means that source Array, destination Array, and factor list arguments can be of different type and kind.
    • Implemented general TK support in ESMF_ArrayRedist(). This means that source Array, destination Array, and factor arguments can be of different type and kind.
    • Fixed zeroflag bug in ESMF_ArraySparseMatMul() that did not correctly initialize the destination Array.
    • Fixed bug in ESMF_ArrayScatter()/ESMF_ArrayGather() that would access not allocated or not associated array arguments.
    • Fixed TKR checking in ESMF_LocalArrayGetData().
    • Corrected string passing on internal inter-language interfaces. A symptom of the problem was the failure of ESMF_StringUTest.F90 on some x86_64 machines.
    • Added support for compiler XLF v11.1.
    • Added scalimpi option to Linux.intel.default.
    • Added scalimpi option to Linux.pgi.default.
    • Same as release ESMF_3_1_0 with the following exceptions:
      • Corrected string passing on internal inter-language interfaces. A symptom of the problem was the failure of ESMF_StringUTest.F90 on some x86_64 machines.
      • Fixed zeroflag bug in ESMF_ArraySparseMatMul().
    • On Tempest IRIX64 unit test ESMC_ClockEx.C fails intermittently for ESMF_ABI=64.
    • An XLC 8.0 compiler bug causes the IBM BlueGene/L to fail system test ESMF_DirectCouplingSTest.F90 and unit tests ESMF_ArrayArbIdxSMMUTest.F90 and ESMF_ArrayRedistUTest.F90 for ESMF_BOPT=O.
    • On x86_64 systems using the medium-memory model with PGI 6.2-3 the ESMF_ArrayScatterUTest.F90 unit test fails.
    • Also see SourceForge bugs list.
    ESMF_2_2_2rp3

    Reference Manual
    (html) (pdf)

    User's Guide
    (html) (pdf)

    Supported Platforms
    2/4/08
    • Added support for Cray XT3/4 systems with CNL kernel.
    • Added scalimpi option to Linux.intel.default and Linux.pgi.default configurations. This has only been tested with the intel compiler.
    • Corrected string passing on internal inter-language interfaces. A symptom of the problem was the failure of ESMF_StringUTest.F90 on some x86_64 machines.
    • Same as release ESMF_2_2_2r.
    • PGI compiler 7.1-3 fails 4 examples and several TimeManager and LogErr unit tests.
    • Also see SourceForge bugs list.
    ESMF_3_1_0

    Reference Manual
    (html) (pdf)

    User's Guide
    (html) (pdf)

    Supported Platforms
    11/29/07
    • List of ESMF interface changes since ESMF_3_0_3.
    • Field, Bundle, State, and Component classes are now built on new Grid and Array classes. Field and Bundle method arguments have significantly changed so that they more closely resemble those of Grid and Array.
    • The following classes were removed:
      • IGrid (in the InternGrid directory, which was also removed)
      • FieldDataMap
      • BundleDataMap
    • The InternArray class is no longer visible at public interfaces. It will be removed in a future release along with InternDG.
    • Added system test FieldSparseMatMul. This creates Fields and gets Arrays from them in order to perform an ESMF_ArraySparseMatMul() operation.
    • The following were disabled:
      • Field and Bundle communication methods and corresponding unit tests and examples
      • Many Field and Bundle get and set methods and corresponding unit tests and examples
      • System tests *except* for CompCreate, DistDir, ArraySparseMatMul, FieldSparseMatMul, ArrayRedist, and ArrayScatterGather. Other system tests have not yet been modified to work with updated Field and Bundle classes, and will be restored in a future release.
    • Further work on the new Grid class:
      • Grids can be created or set with a regular distribution.
      • Coordinate arrays can be less than the Grid's rank (e.g. rectilinear coordinates)
      • Grids can be serialized and deserialized (e.g. as part of a state reconcile operation)
      • Grids can be queried for information about a stagger location including local DE exclusive and computational bounds, computationalEdgeWidths, and undistributed bounds.
      • Grids can be queried for element counts in addition to lower and upper bound information.
    • The Grids have been altered to live entirely within the Array exclusive region to allow for smoother interaction with Array methods.
    • Stagger locations have also been changed to allow:
      • Comparisons between staggerlocs
      • The output of a string describing the staggerloc
    • Implemented ESMF_ArrayGather()
    • Implemented ESMF_ArrayRedist()
    • Implemented ESMF_ArraySet()
    • Support the use of precomputed ArraySparseMatMul RouteHandles with different actual Arrays.
    • Optimized performance of ESMF_ArraySparseMatMulStore().
    • Optimized performance of ESMF_ArraySparseMatMul().
    • Implemented Array replication along specified DistGrid dimensions.
    • Generalized ArraySparseMatMul to support general Array - DistGrid dimension mapping.
    • Generalized ArraySparseMatMul to support tensor term mixing.
    • Implemented a more flexible Array computational bounds scheme that allows a computational region that is smaller than the exclusive region.
    • Changed Array total widths to be relative to the Array computational region.
    • Added computationalEdgeLWidth and computationalEdgeUWidth arguments to ESMF_ArrayCreate() interfaces to allow special treatment of the edge.
    • Changed DistGrid basic elements from cells to elements.
    • Changed Array's dimmap and inverseDimmap arguments to distgridToArrayMap and arrayToDistGridMap, respectively.
    • Added LocalArray section to ESMF Reference Manual.
    • Added support for Intel compiler version 10.
    • Increased robustness of g95 and gfortran configurations on Linux and Darwin.
    • Added support for Cray XT3/4 systems with CNL kernel.
    • The ESMF_Grid creation interfaces that take the petMap argument (e.g. ESMF_GridCreateShapeTile()) mistakenly return an error for inputs that put more than one DE on a PET, if the DEs are not mapped onto the PETs in sequential order.
    • The only Use Test Case that currently builds is ESMF_SparseMatMul. Due to failures of the other UTCs it must be built and run separately, not from the collective UTC makefile.
    • The zeroflag=ESMF_TRUE option in the ESMF_ArraySparseMatMul() call is broken for Arrays that have total regions that are larger than the exclusive region. The symptom of this bug is that not all destination elements are reset to zero before adding the sparse matrix multiplication terms.
    • The ESMF_StateReconcile() call, generally used by couplers dealing with Components running on exclusive petLists, has an internal size limit (which will be addressed in a subsequent release). There is no hard-coded limit to how many elements, i.e. States, Bundles, Fields, Arrays can be added to the State. However, there is a limit, roughly 20, on the maximum number of sub-elements contained by any element in the State passed into ESMF_StateReconcile() .
    • C/C++ Component examples fail with the XLF and PGI compilers as well as for the Linux/absoftintel combination.
    • The documentation for ESMF_GetAttribute() and ESMF_SetAttribute() calls for multi-valued list Attributes does not show a required integer input "count" argument after the "name" argument. If the "count" argument, which represents the number of Attribute values, is inserted the call will work.
    • ESMF_LOG_SINGLE is known to be broken. The symptoms are platform dependent and may appear non-deterministic.
    • On IRIX64 the exhaustive unit tests compiled against MPI will intermittently hang when run in uni-PET mode if the ESMF library was built with ESMF_PTHREADS=ON. This issue does not show up when running the MPI version in multi-PET mode or when using the MPIUNI version in uni-PET runs. The issue disappears when the ESMF library is compiled with ESMF_PTHREADS=OFF (which is the default for IRIX64).
    • ESMF fails to build on Cray XT3/XT4 with PGI 6.2.5 with an internal compiler error.
    • On IBM AIX systems in 32-bit mode StateLimitUTest fails in uni-PET mode.
    • On x86_64 systems using the medium-memory model with PGI 6.2-3 or PathScale 2.4 ESMF_ArrayDataUTest.F90 will report two failures. These failures indicate a problem with the F90 dope vector and ESMF must be assumed unreliable under these conditions.
    • On x86_64 systems using the medium-memory model with PGI 6.2-3 the ESMF_ArrayLarrayEx.F90 fails.
    • On Cray X1 the ESMF_DistDir system test fails.
    • On IBM BlueGene/L ESMF_VMSendVMRecvUTest.F90 fails in uni-PET mode.
    • On Linux Lahey, a test in ESMF_ArrayDataUTest.F90 fails with ESMF_BOPT=g.
    • All NAG compiler combinations we tested (on Linux as well as on Darwin) show random run-time failures when ESMF is compiled in optimized mode and also when user code that links with ESMF is compiled in optimized mode. Consequently ESMF sets ESMF_OPTLEVEL to 0 (zero) by default when compiling ESMF with NAG and we advise NAG users to not use optimization on user code that links against the ESMF library.
    • On Linux intel 9.1 with mpich2, unit test ESMF_StringUTest.F90 fails with ESMF_BOPT=O.
    • On x86_64 systems with PGI 6.2-3 or PathScale 2.4, unit test ESMF_VMBarrierUTest.F90 fails intermittently.
    • Also see SourceForge bugs list.
    ESMF_2_2_2rp2

    Reference Manual
    (html) (pdf)

    User's Guide
    (html) (pdf)

    Supported Platforms
    10/22/07
    • Added code to work around a bug in current Intel compiler versions on the NASA Altix system known as Columbia. This leads to failures in State and Grid interfaces when compiled in debug mode. It is unclear at this time if this is a system specific problem or not.
    • Added support for Intel compiler version 10.
    • Added port for gfortran compiler on Linux.
    • Added port for gfortran compiler on Darwin.
    • Added port for g95 compiler on Darwin.
    ESMF_3_0_3

    Reference Manual
    (html) (pdf)

    User's Guide
    (html) (pdf)

    Supported Platforms
    7/27/07
    • List of ESMF interface changes since ESMF_3_0_2.
    • Renamed all references to Grids that existed in previous versions of the framework to IGrid and moved the class to an InternGrid directory. The IGrid class is deprecated and will be removed in a future release.
    • Created a new Grid class, which is still at an early stage of development. The new Grids can:
      • be 1D, 2D, 3D and higher dimensions
      • have multiple undistributed dimensions
      • hold curvilinear coordinates at multiple stagger locations.
      The user can now:
      • create a new Grid either with shortcut methods or incrementally
      • get parameters from the new Grid
      • set and get Grid coordinates from an ESMF Array
      • get a pointer to the memory holding the coordinates in the Grid
      • get array bound information for the coordinate arrays
      • destroy a new Grid.
    • The ESMF_ArraySparseMatMulStore() and ESMF_ArraySparseMatMul() methods were rewritten and improved performance and reduced memory usage are expected. The methods have the following new features:
      • support for distributed definition of factors
      • support for user-supplied arbitrary sequence indices via 1D DistGrid
      • support for ESMF_TYPEKIND_I4/I8/R4/R8
      • default initialization of destination Array to zero (can be overridden)
    • A collective ESMF_DistGridCreate() call was introduced that takes local lists of arbitrary sequence indices and returns a matching 1D DistGrid. This can be used in combination with the Array sparse matrix multiply to regrid unstructured grids.
    • The ESMF_DistGridGet() interface was massively updated to provide complete access to DistGrid information. The names of many of the arguments have changed.
    • The ESMF_ArrayCreate() calls now provide the "name" argument.
    • Many unimplemented methods will return the failure code ESMF_RC_NOT_IMPL. Validate methods are considered implemented if at least initialization checks are present.
    • Cygwin.g95.default and Cygwin.gfortran.default configurations were added.
    • Starting with this version of ESMF, support for PGI compilers has been limited to PGI versions >= 6.2. We will stop reporting ESMF known bugs for older PGI versions.
    • C/C++ Component examples fail with the XLF and PGI compilers as well as for the Linux/absoftintel combination.
    • The ESMF_StateReconcile() call, generally used by couplers dealing with Components running on exclusive petLists, has an internal size limit (which will be addressed in a subsequent release). There is no hard-coded limit to how many elements, i.e. States, Bundles, Fields, Arrays can be added to the State. However, there is a limit, roughly 20, on the maximum number of sub-elements contained by any element in the State passed into ESMF_StateReconcile() .
    • ESMF_BundleGather(), ESMF_FieldGather() and ESMF_ArrayGather() are only expected to work on 2D grids with 2D data.
    • When ESMF_InternArray objects are created with a non-zero halo width, that width is added to all dimensions of the array, not only ones which correspond to the decomposed grid axes.
    • Overall, we have found that regridding using ESMF_InternArrayRegrid(), FieldRegrid(), and BundleRegrid() is not robust. We encourage customers interested in ESMF regridding to move to the ESMF_ArraySparseMatMul() approach while regrid methods are reworked.
    • The conservative regrid algorithm for full spheres, which includes special treatment of the poles, is largely untested and expected to produce errors.
    • The conservative regrid method for both regional and global grids produces incorrect results for PGI in MPIUNI mode.
    • The ESMF_Grid creation interfaces that take the petMap argument (e.g. ESMF_GridCreateShape()) mistakenly return an error for inputs that put more than one DE on a PET, if the DEs are not mapped onto the PETs in sequential order.
    • The documentation for ESMF_GetAttribute() and ESMF_SetAttribute() calls for multi-valued list Attributes does not show a required integer input "count" argument after the "name" argument. If the "count" argument, which represents the number of Attribute values, is inserted the call will work.
    • ESMF_LOG_SINGLE is known to be broken. The symptoms are platform dependent and may appear non-deterministic.
    • On IRIX64 the exhaustive unit tests compiled against MPI will intermittently hang when run in uni-PET mode if the ESMF library was built with ESMF_PTHREADS=ON. This issue does not show up when running the MPI version in multi-PET mode or when using the MPIUNI version in uni-PET runs. The issue disappears when the ESMF library is compiled with ESMF_PTHREADS=OFF (which is the default for IRIX64).
    • On Cray X1, the conservative regrid method does not work when the grid is defined as a uniform horizontal LatLon grid with periodic longitude coordinate and the source and destination grid are of the same size.
    • ESMF fails to build on Cray XT3/XT4 with PGI 6.2.5 with an internal compiler error.
    • On IBM AIX systems in 32-bit mode StateLimitUTest fails in uni-PET mode.
    • On x86_64 systems using the medium-memory model with PGI 6.2-3 or PathScale 2.4 ESMF_ArrayDataUTest.F90 will report two failures. These failures indicate a problem with the F90 dope vector and ESMF must be assumed unreliable under these conditions.
    • On x86_64 systems using the medium-memory model with PGI 6.2-3 the ESMF_ArrayLarrayEx.F90 fails.
    • On Cray X1 the ESMF_DistDir system test fails.
    • On IBM BlueGene/L ESMF_VMSendVMRecvUTest.F90 fails in uni-PET mode.
    • On Linux Lahey, a test in ESMF_ArrayDataUTest.F90 fails with ESMF_BOPT=g.
    • All NAG compiler combinations we tested (on Linux as well as on Darwin) show random run-time failures when ESMF is compiled in optimized mode and also when user code that links with ESMF is compiled in optimized mode. Consequently ESMF sets ESMF_OPTLEVEL to 0 (zero) by default when compiling ESMF with NAG and we advise NAG users to not use optimization on user code that links against the ESMF library.
    • The documentation of the ESMF_ArraySparseMatMulStore() API incorrectly states that the returned "routehandle" argument can be used to perform ESMF_ArraySparseMatMul() on any Array pair that is DistGrid-conforming to the Array pair used during the ESMF_ArraySparseMatMulStore(). This feature is _not_ yet supported, i.e. currently the "routehandle" can only be used to perform ESMF_ArraySparseMatMul() between the Array pair for which it was precomputed.
    • Also see SourceForge bugs list.
    ESMF_2_2_2rp1

    Reference Manual
    (html) (pdf)

    User's Guide
    (html) (pdf)

    Supported Platforms
    6/15/07
    • Added port for x86_64 ABI to Linux.intel.default.
    • Added port for x86_64 ABI to Linux.absoft.default.
    • Added PGI 7.x support for Linux and Cray XT3/4.
    • Fixed build file issue that caused problems with GNU Make 3.81.
    • Fixed memory leak in non-blocking VM communications.
    • Replaced the ESMF_NONBLOCKING option section in the Reference Manual with a more thorough explanation.
    ESMF_3_0_2

    Reference Manual
    (html) (pdf)

    User's Guide
    (html) (pdf)

    Supported Platforms
    5/25/07
    • List of ESMF interface changes since ESMF_3_0_1.
    • The ESMF_ArrayCreate(), ESMF_ArrayGet() and ESMF_ArrayScatter() interfaces were type/kind/rank overloaded.
    • ESMF_VMScatterV() and ESMF_VMGatherV() calls were added.
    • Methods across the framework had their return codes initialized to not implemented (ESMF_RC_NOT_IMPL). This is a step towards ensuring that unimplemented code returns appropriate errors when called.
    • LogErr calls were modified so that they need not set return codes prematurely to ESMF_SUCCESS. This is also a step towards ensuring that unimplemented code returns appropriate errors when called. This new LogErr behavior is not yet turned on, as there are still calls that rely on the former behavior of LogErr.
    • The declaration and usage of real and integer datatypes across the framework was standardized. The most significant change was the combination of type and kind into a single ESMF_TypeKind parameter. The new datatype conventions are included in the Developer's Guide.
    • A policy for the auto-promotion of user data via compile options was developed and is included in the Developer's Guide.
    • An ESMF_ConfigSetAttribute() call was implemented for 4-byte integers.
    • Conventions were established for a public ESMC optional argument API based on the variable-length argument list functionality provided by <stdarg.h>.
    • Added C interfaces for Config class (ESMC_Config). The ESMC_ConfigSetAttribute method is present, but not implemented.
    • An advanced stub for a new testing control system for running exhaustive tests on regrid and redist, called the test_harness, has been added. It runs and parses trivial bundle and field redistribution test configurations.
    • Fixed a bug that required Log files to be explicitly closed by the user before calling ESMF_Finalize.
    • Users may now open, write and close multiple Log files.
    • C/C++ Component examples fail with the XLF and PGI compilers as well as for the Linux/absoftintel combination.
    • The ESMF_StateReconcile() call, generally used by couplers dealing with Components running on exclusive petLists, has an internal size limit (which will be addressed in a subsequent release). There is no hard-coded limit to how many elements, i.e. States, Bundles, Fields, Arrays can be added to the State. However, there is a limit, roughly 20, on the maximum number of sub-elements contained by any element in the State passed into ESMF_StateReconcile() .
    • ESMF_BundleGather(), ESMF_FieldGather() and ESMF_ArrayGather() are only expected to work on 2D grids with 2D data.
    • When ESMF_InternArray objects are created with a non-zero halo width, that width is added to all dimensions of the array, not only ones which correspond to the decomposed grid axes.
    • Overall, we have found that regridding using ESMF_InternArrayRegrid(), FieldRegrid(), and BundleRegrid() is not robust. We encourage customers interested in ESMF regridding to move to the ESMF_ArraySparseMatMul() approach while regrid methods are reworked.
    • The conservative regrid algorithm for full spheres, which includes special treatment of the poles, is largely untested and expected to produce errors.
    • The conservative regrid method for both regional and global grids produces incorrect results for PGI in MPIUNI mode.
    • The documentation for ESMF_GetAttribute() and ESMF_SetAttribute() calls for multi-valued list Attributes does not show a required integer input "count" argument after the "name" argument. If the "count" argument, which represents the number of Attribute values, is inserted the call will work.
    • ESMF_LOG_SINGLE is known to be broken. The symptoms are platform dependent and may appear non-deterministic.
    • On IRIX64 the exhaustive unit tests compiled against MPI will intermittently hang when run in uni-PET mode if the ESMF library was built with ESMF_PTHREADS=ON. This issue does not show up when running the MPI version in multi-PET mode or when using the MPIUNI version in uni-PET runs. The issue disappears when the ESMF library is compiled with ESMF_PTHREADS=OFF (which is the default for IRIX64).
    • On Cray X1, the conservative regrid method does not work when the grid is defined as a uniform horizontal LatLon grid with periodic longitude coordinate and the source and destination grid are of the same size.
    • On Cray XT3/XT4, the ESMF_VMBarrierUTest is incorrectly reported as failing. The origin of this problem seems to be a bug in Cray's implementation of MPI_Wtime(). This problem will also affect the validity of ESMF_VMWtime().
    • ESMF fails to build on Cray XT3/XT4 with PGI 6.2.5 with an internal compiler error.
    • On IBM AIX systems in 32-bit mode StateLimitUTest fails in uni-PET mode.
    • On x86_64 systems using the medium-memory model with PGI 6.2-3 or PathScale 2.4 ESMF_ArrayDataUTest.F90 will report two failures. These failures indicate a problem with the F90 dope vector and ESMF must be assumed unreliable under these conditions.
    • On IBM BlueGene/L, tests in ESMF_ClockUTest.F90 fail for ESMF_BOPT=O/g. Fails ESMF_VMSendVMRecvUTest.F90 in uni-PET mode.
    • On Linux Lahey, a test in ESMF_ArrayDataUTest.F90 fails with ESMF_BOPT=g.
    • All NAG compiler combinations we tested (on Linux as well as on Darwin) show random run-time failures when ESMF is compiled in optimized mode and also when user code that links with ESMF is compiled in optimized mode. Consequently ESMF sets ESMF_OPTLEVEL to 0 (zero) by default when compiling ESMF with NAG and we advise NAG users to not use optimization on user code that links against the ESMF library.
    • On the PGI 5.0-2 compiler with mpiuni, five regriding unit tests fail.
    • On the PGI 6.1-1 compiler with mpiuni, multiple unit tests, examples and 2 system tests fail.
    • Also see SourceForge bugs list.
    ESMF_3_0_1

    Reference Manual
    (html) (pdf)

    User's Guide
    (html) (pdf)

    Supported Platforms
    2/28/07
    • List of ESMF interface changes since ESMF_3_0_0.
    • Standardized the checking of initialization in ESMF. This introduces two new error return codes. The first, ESMF_RC_OBJ_NOT_CREATED, is returned if the user attempts to use a deep class variable without first creating it. The second ESMF_RC_OBJ_DELETED is returned if the user attempts to use a deep class variable after its deletion. Also, before this change, shallow class variables were not initialized on platforms with compilers that didn't support type component initialization. Now this initialization is handled by ESMF. Implementing this change necessitated the change of the intents of shallow class input parameters throughout ESMF from strictly in to inout.
    • Added an ESMF_LOG_NONE option to the "defaultLogType" argument of ESMF_Initialize() to turn off ESMF's default error logging.
    • Added "mpiCommunicator" argument to ESMF_Initialize() to allow ESMF to be initialized on a subset of MPI processes. The added VM example ESMF_VMUserMpiCommEx.F90 tests this new feature.
    • Added an ESMF_KEEPMPI option to the "terminationflag" argument of ESMF_Finalize() to allow ESMF finalization without MPI shutdown.
    • Removed deprecated GridCompCreate() and CplCompCreate() interfaces.
    • Optimized ESMF_BundleRedistStore(), ESMF_BundleRedist(), ESMF_FieldRedistStore(), and ESMF_FieldRedist() for arbitrarily distributed grids. A new algorithm was implemented i n the ESMF Route code to initialize the arbitrary to arbitrary grid redistribution. The redist run routine has been optimized specifically for the Cray X1 machine.
    • Added ESMF_VMAllGatherV() and VMAllToAllV() methods.
    • Added "errorMask" argument to ESMF_LogSet() to allow masking of error codes in LogErr handling.
    • Reworked the build system to be more robust and portable. Some of the previously used environment variables have become obsolete or were replaced by new variables. A full list of the environment variables as well as build system details can be found in the User's Guide. The main README contains enough information to get started quickly.
    • Introduced enhanced support for compiling and linking user applications against ESMF. See the User's Guide or main README for details.
    • Added a section to the User's Guide on how to port ESMF to other platforms.
    • Ported ESMF to the Cray XT3/XT4 and IBM BlueGene/L.
    • Added the following system tests to test the Bundle and Field redistribution methods for both arbitrarily distributed grids and block distributed grids:
      • ESMF_BundleRedistArb2ArbSTest.F90
      • ESMF_BundleRedistBlk2ArbSTest.F90
      • ESMF_BundleRedistBlk2BlkSTest.F90
      • ESMF_FieldRedistArb2ArbSTest.F90
      • ESMF_FieldRedistBlk2ArbSTest.F90
      • ESMF_FieldRedistBlk2BlkSTest.F90
    • Augmented the exhaustive regrid unit test ESMF_RegridToolUTest.F90 to include regrid scenarios with different resolution source grid and destination grid.
    • Fixed ESMF_InternArrayUTest unit test failures on the Linux PGI compiler.
    • The problem with PGI 6.1.1 on Linux was resolved as a consequence of the reworked build system.
    • The following issues appear to be resolved due to compiler upgrades or indirectly as a result of other development:
      • Problem building with XL Fortran compiler version 8.1.1.6, when compiling with the combination ESMF_PREC=64 and ESMF_BOPT=g.
      • Issues with configuration of MPICH with Intel 8.1.
    • The rank check error that incorrectly occurred during redistribution of Fields or Bundles that are transferred from block to arbitrary distribution (and vice versa) has been corrected.
    • The ESMF distribution available for download on the ESMF website has been pruned of many extraneous files, and is no longer bundled with documentation files. The distribution has been reduced in size from about 127MB to 12MB. Users can get documentation files by downloading from SourceForge.
    • This ESMF release is under the University of Illinois-NCSA Open Source License. Licenses are still a subject of discussion.
    • C/C++ Component examples fail with the XLF and PGI compilers as well as for the Linux/absoftintel combination.
    • The ESMF_StateReconcile() call, generally used by couplers dealing with Components running on exclusive petLists, has an internal size limit (which will be addressed in a su bsequent release). There is no hard-coded limit to how many elements, i.e. States, Bundles, Fields, Arrays can be added to the State. However, there is a limit, roughly 20, on the maximum number of sub-elements contained by any element in the State passed into ESMF_StateReconcile().
    • ESMF_BundleGather(), ESMF_FieldGather() and ESMF_ArrayGather() are only expected to work on 2D grids with 2D data.
    • When ESMF_InternArray objects are created with a non-zero halo width, that width is added to all dimensions of the array, not only ones which correspond to the decomposed grid axes.
    • Overall, we have found that regridding using ESMF_InternArrayRegrid(), FieldRegrid(), and BundleRegrid() is not robust. We encourage customers interested in ESMF regridding to move to the ESMF_ArraySparseMatMul() approach while regrid methods are reworked.
    • On IRIX64 the exhaustive unit tests compiled against MPI will intermittently hang when run in uni-PET mode if the ESMF library was built with ESMF_PTHREADS=ON. This issue does not show up when running the MPI version in multi-PET mode or when using the MPIUNI version in uni-PET runs. The issue disappears when the ESMF library is compiled with ESMF_PTHREADS=OFF (which is the default for IRIX64).
    • The conservative regrid algorithm for full spheres, which includes special treatment of the poles, is largely untested and expected to produce errors.
    • The conservative regrid method for both regional and global grids produces incorrect results for PGI in MPIUNI mode.
    • The documentation for ESMF_GetAttribute() and ESMF_SetAttribute() calls for multi-valued list Attributes does not show a required integer input "count" argument after the "name" argument. If the "count" argument, which represents the number of Attribute values, is inserted the call will work.
    • ESMF_LOG_SINGLE is known to be broken. The symptoms are platform dependent and may appear non-deterministic.
    • On Cray X1, the conservative regrid method does not work when the grid is defined as a uniform horizontal LatLon grid with periodic longitude coordinate and the source and destination grid are of the same size.
    • On Cray X1, there are unit test failures across a wide range of classes, including Time classes, Field, State and Array, and we do not consider the release to be supported on this platform.
    • On Cray XT3/XT4, the ESMF_VMBarrierUTest is incorrectly reported as failing. The origin of this problem seems to be a bug in Cray's implementation of MPI_Wtime(). This problem will also affect the validity of ESMF_VMWtime().
    • On Cray XT3/XT4, the system test ESMF_ArraySparseMatMulSTest() and example ESMC_AppMainEx.C fail for ESMF_BOPT=O/g. System test ESMF_FieldHaloSTest() fails with ESMF_BOPT=O.
    • On IBM BlueGene/L, tests in ESMF_ArrayScatterUTest.F90 fail for ESMF_BOPT=g. Tests in ESMF_ClockUTest.F90 fail for ESMF_BOPT=O/g.
    • All NAG compiler combinations we tested (on Linux as well as on Darwin) show random run-time failures when ESMF is compiled in optimized mode and also when user code that links with ESMF is compiled in optimized mode. We advise NAG users to set environment variable ESMF_OPTLEVEL to 0 (zero) when compiling ESMF and to not use optimization on user code that links against the ESMF library.
    • Also see SourceForge bugs list.
    ESMF_2_2_2r

    Reference Manual
    (html) (pdf)

    User's Guide
    (html) (pdf)

    Supported Platforms
    12/1/06
    • List of ESMF interface changes since ESMF_2_2_2.
    • This is the first ESMF release under the University of Illinois-NCSA Open Source License. Subsequent releases will be under this license.
    • Added an ESMF_LOG_NONE option to the "defaultLogType" argument of ESMF_Initialize() to turn off ESMF's default error logging.
    • Added an ESMF_KEEPMPI option to the "terminationflag" argument of ESMF_Finalize() to allow ESMF finalization without MPI shutdown.
    • Fixed the C/C++ interfaces for all but XLF and PGI compilers, and reactivated the C/C++ Component interface examples.
    • Optimized ESMF_BundleRedistStore(), ESMF_BundleRedist(), ESMF_FieldRedistStore(), and ESMF_FieldRedist() for arbitrarily distributed grids. A new algorithm was implemented in the ESMF Route code to initialize the arbitrary to arbitrary grid redistribution. The redist run routine has been optimized specifically for the Cray X1 machine. View performance report.
    • Added an ESMF_VMAllGatherV() method.
    • Reworked the build system to be more robust and portable. Some of the previously used environment variables have become obsolete or were replaced by new variables. A full list of the environmentvariables as well as build system details can be found in the User's Guide. The main README contains enough information to get started quickly.
    • Introduced enhanced support for compiling and linking user applications against ESMF. See the User's Guide or main README for details.
    • Added a section to the User's Guide on how to port ESMF to other platforms.
    • Ported ESMF to the Cray XT3, IBM BlueGene/L and Darwin xlfgcc.
    • Added the following system tests to test the Bundle and Field redistribution methods for both arbitrarily distributed grids and block distributed grids:
      • ESMF_BundleRedistArb2ArbSTest.F90
      • ESMF_BundleRedistBlk2ArbSTest.F90
      • ESMF_BundleRedistBlk2BlkSTest.F90
      • ESMF_FieldRedistArb2ArbSTest.F90
      • ESMF_FieldRedistBlk2ArbSTest.F90
      • ESMF_FieldRedistBlk2BlkSTest.F90
    • Augmented the exhaustive regrid unit test ESMF_RegridToolUTest.F90 to include regrid scenarios with different resolution source grid and destination grid.
    • Fixed unit test failures on the Linux PGI compiler:
      • ESMF_ArrayUTest.F90
      • ESMF_VMBarrierUTest.F90
    • On the Linux Lahey compiler, fixed failure of the unit test ESMF_VMSendNbVMRecvNbUTest.F90.
    • Added support for the LSF batch queuing system.
    • The following issues appear to be resolved due to compiler upgrades or indirectly as a result of other development:
      • Problem building with XL Fortran compiler version8.1.1.6, when compiling with the combination ESMF_PREC=64 and ESMF_BOPT=g.
      • Intermittent failure of the "ESMF_FlowWithCoupling" system test and the "ESMF_COUPLED_FLOW" demo on AIX.
      • On the Linux PGI 5.0-2 compiler, failure of the "ESMF_FlowWithCoupling" system test and the "ESMF_COUPLED_FLOW" demo to compile with -g, resulting in an internal compiler error.
      • Issues with configuration of MPICH with Intel 8.1.
    • C/C++ Component examples fail with the XLF and PGI compilers.
    • The ESMF_StateReconcile() call, generally used by couplers dealing with concurrently running Components, has an internalsize limit (which will be addressed in a subsequent release). There is no hard-coded limit to how many elements, i.e. States, Bundles, Fields, Arrays can be added to the State. However, there is a limit, roughly 20, on the maximum number of sub-elements contained by any element in the State passed into ESMF_StateReconcile().
    • ESMF_BundleGather(), ESMF_FieldGather() and ESMF_ArrayGather() are only expected to work on 2D grids with 2D data.
    • When ESMF_Array objects are created with a non-zero halo width, that width is added to all dimensions of the array, not only ones which correspond to the decomposed grid axes.
    • Overall, we have found that regridding using ArrayRegrid(), FieldRegrid(), and BundleRegrid() is not robust. We encourage customers interested in ESMF regridding to move to the ESMF_ArraySparseMatMul() approach in the ESMF 3 series while regrid methods are reworked.
    • The conservative regrid algorithm for full spheres, which includes special treatment of the poles, is largely untested and expected to produce errors.
    • The conservative regrid method for both regional and global grids produces incorrect results for PGI in MPIUNI mode.
    • On Cray X1, the conservative regrid method does not work when the grid is defined as a uniform horizontal LatLon grid with periodic longitude coordinate and the source and destination grid are of the same size.
    • On Cray XT3, the ESMF_VMBarrierUTest is incorrectly reported as failing. The origin of this problem seems to be a bug in Cray's implementation of MPI_Wtime(). This problem will also affect the validity of ESMF_VMWtime().
    • 'ESMF_LOG_SINGLE' is known to be broken. The symptoms are platform dependent and may appear non-deterministic.
    • All NAG compiler combinations we tested (on Linux as well as on Darwin) show random run-time failures when ESMF is compiled in optimized mode and also when user code that links with ESMF is compiled in optimized mode. We advise NAG users to set environment variable ESMF_OPTLEVEL to 0 (zero) when compiling ESMF and to not use optimization on user code that links against the ESMF library.
    • Darwin absoft version 8.2 fails unit test ESMF_CalRangeUTest.F90.
    • Also see SourceForge bugs list.
    ESMF_3_0_0

    Reference Manual
    (html) (pdf)

    User's Guide
    (html) (pdf)

    Supported Platforms
    5/20/06
    • List of ESMF interface changes since ESMF_2_2_2.
    • The following classes were modified to implement an index space layer according to JST design discussions:
      • Array
      • DELayout
      • DistGrid
    • Adjusted library sources, examples and tests to DELayout changes.
    • Added support for arbitary to arbitrary redistribution of Fields and Bundles.
    • Renamed previous Array class to InternArray. This class is expected to become an internal class in future releases.
    • Adjusted library sources, examples and tests to InternArray. Removed as many InternArray dependencies as possible.
    • Fixed VMBarrierUTest for PGI (Argonne Jazz cluster). This was a known bug in the previous release.
    • Fixed the C++ Component interface for most platforms. PGI and XLF still do not work.
    • Re-enable the C++ Component examples, which were disabled for the previous release.
    • Modified VMKernel and makefiles to allow compilation on NEC SX.
    • Added support for the LSF batch queueing system on the AIX platform. (For NCAR computer users, this supports 'bluevista'.)
    • The ESMF_StateReconcile() call, generally used by couplers dealing with concurrently running Components, has an internal size limit (which will be addressed in a subsequent release). There is no hard-coded limit to how many elements, i.e. States, Bundles, Fields, Arrays can be added to the State. However, there is a limit, roughly 20, on the maximum number of sub-elements contained by any element in the State passed into ESMF_StateReconcile().
    • Redistribution of Fields or Bundles that are transferred from block to arbitrary distribution (and vice versa) fails with a message about mismatched ranks.
    • The C/C++ Component examples fail with the XLF and PGI compilers.
    • ESMF_BundleGather(), ESMF_FieldGather() and ESMF_InternArrayGather() are only expected to work on 2D grids with 2D data.
    • The conservative regrid algorithm for full spheres, which includes special treatment of the poles, is largely untested and expected to produce errors.
    • When ESMF_InternArray objects are created with a non-zero halo width, that width is added to all dimensions of the array, not only ones which correspond to the decomposed grid axes.
    • On AIX systems the ESMF fails to build with the XL Fortran compiler version 8.1.1.6, when compiling with the combination ESMF_PREC=64 and ESMF_BOPT=g. (Compiler version 8.1.1.4 works fine with all compile options, and ESMF_PREC=32 or ESMF_BOPT=O also work with either compiler version.)
    • When compiling with Intel 8.1 and using MPICH, make sure the Intel/MPICH libraries are configured with: #define MPI2CPP_HAVE_BOOL 1 which may not be the default output from configure.
    • On the Linux PGI compiler unit test ESMF_InternArrayUTest.F90 fails with both ESMF_BOPT=O/g
    • Build fails on Longs Linux PGI 6.1.1 compiler with ESMF_BOPT=g.
    • On the Cray X1, several system tests, examples and unit tests fail.
    • All NAG compiler combinations we tested (on Linux as well as on Darwin) show random run-time failures when ESMF is compiled in optimized mode and also when user code that links with ESMF is compiled in optimized mode. We advise NAG users to set environment variable ESMF_OPTLEVEL to 0 (zero) when compiling ESMF and to not use optimization on user code that links against the ESMF library.
    • Also see SourceForge bugs list.
    ESMF_2_2_2

    Reference Manual
    (html) (pdf)

    User's Guide
    (html) (pdf)

    Supported Platforms
    3/16/06
    • The ESMF_FieldExclSTest system test has been modified to demonstrate a simplified run loop for the concurrent case. This simplification was made possible by the modified blocking behavior of Components that was introduced with ESMF release 2.2.1.
    • Fixed a copy in/out problem on the F90/C VM interface. On Linux using the Lahey f95 compiler this problem had caused the non-blocking VM communication calls to send/receive from/into invalid data buffers, resulting in corrupt communication results.
    • The top level README has been updated to include more information about linking applications and porting ESMF to other platforms.
    • Added an optional environment variable, ESMF_ARRAY_LITE, to restrict the number of array-level interfaces automatically generated. This is required on the NEC build since single byte integer arrays are not supported on that platform, but is available as a general feature to reduce the number of overloaded interfaces and code size.
    • Added additional environment variables, ESMF_CXX_LIBRARY_PATH and ESMF_F90_LIBRARY_PATH, to support explicit library search paths. If defined, these will be used at link time to locate the correct F90 and C++ libraries. Otherwise, the standard LD_LIBRARY_PATH will be used.
    • Added "Appendix B: ESMF Error Return Codes" to the ESMF Reference Manual.
    Same as release ESMF_2_2_1 with the exception that ESMF_VMSendNbVMRecvNbUTest.F90 unit test no longer fails on the Linux Lahey compiler. Also see the SourceForge bugs list.
    ESMF_2_2_1

    Reference Manual
    (html) (pdf)

    User's Guide
    (html) (pdf)

    Supported Platforms
    2/14/06
    • List of ESMF interface changes since ESMF_2_2_0rp1
    • Changed the following in the ESMF_GridCompCreate() and ESMF_CplCompCreate() interfaces:
      • Added an interface with all optional arguments. In subsequent releases this may become the only supported interface.
      • We recommend that users call ESMF_xxxCompCreate() without specifying the parent VM. All tests, examples and demo codes have been modified to match the new convention.
      • If the parent VM is not specified the current VM is used as the parent VM (previously the global VM was used in this case).
    • Added support for high level MPMD. This makes xxxCompCreate() collective across all parent PETs and xxxCompSetServices() collective only across participating parent PETs.
    • Added a new blockingFlag for xxxCompInitialze(), xxxCompRun() and xxxCompFinalize() that eliminates some unnecessary synchronization points. This flag (ESMF_VASBLOCKING) is now the default.
    • The ESMF_StateReconcile() method no longer dumps core when the buffer size is exceeded. It returns an error message instead. (Please see the first item in the Known Bugs column.)
    • DELayout now issues warning if non 1-to-1 DELayouts or non logically rectangular DELayouts are created since most of the rest of the system does not handle these yet.
    • The Field and Bundle level redistribution communication calls have been rewritten. They now include 3+D data support and optimized communication calls for Bundles which contain Fields where the data arrays are congruent (same data type, same rank, etc).
    • Validate routines have been added for the Field and Bundle levels of the regrid, halo, and redistribution routines. If a hang or core dump appears to be associated with a call to the run routine for these operations, the route handle can be passed to a Validate routine which looks for inconsistencies and other problems.
    • Added full Fortran interface access to the non-blocking VM communications calls:
      • ESMF_VMSend()
      • ESMF_VMRecv()
      • ESMF_VMSendRecv()
      • ESMF_VMScatter()
      • ESMF_VMGather()
      • ESMF_VMReduce()
      • ESMF_VMAllGather()
      • ESMF_VMBroadcast()
    • Configuration attributes can be validated.
    • Configuration attributes can be checked for uniqueness. The "unique" argument in ESMF_ConfigLoadFile() now works.
    • Configuration files may contain logical attribute values.
    • LogErr no longer opens/closes logfile upon each write, enhancing performance.
    • The Log message length increased to 256 characters.
    • All Log messages (originating in either C++ or Fortran) go through Fortran for consistent, ordered output.
    • Time Manager defaults for any year, month, day calendar:
      • year = 0
      • month = 1
      • day =1
    • Improved regrid unit test structure to facilitate increasing test coverage.
    • Added system tests for MPMD (these are not automatically run by test scripts).
    • Added a unit test VMUserMpiInitUTest which checks that the user can explicitly call MPI_Init() before ESMF_Initialize().
    • Added unit tests for non-blocking VM communications.
    • Added support for the LSF batch queueing system on the AIX platform. (For NCAR computer users, this supports 'bluevista'.)
    • The files 'esmfuser.mk' and 'makefile.sample' in $ESMF_DIR/build replace the outdated 'link_rules.mk' files, showing how to link a user application with the ESMF library for any platform.
    • Added MPICH2 and IntelMPI support for Linux.intel.
    • The default linker front end on IRIX64 has been changed to CC, i.e. MIPSPro C++, for all ESMF applications. CC is now being used during link time regardless of the language in which the main program was written (Fortran90 or C++). In previous ESMF releases the f90 linker front end had been used to link applications on IRIX64 that provided the main program in Fortran. However, this led to problems during link time for applications that also contained C++ user code.
    • More documentation was added to the implementation notes part of the communication routines section.
    • Documented non-blocking features and limitations.
    • Added extensive new documentation on Grids and regridding.
    • Added reference documentation explaining that ESMF_CAL_NOCALENDAR does not track days, since the unit of a day is planet-specific.
    • The ESMF_StateReconcile() call, generally used by couplers dealing with concurrently running Components, has an internal size limit (which will be addressed in a subsequent release). There is no hard-coded limit to how many elements, i.e. States, Bundles, Fields, Arrays can be added to the State. However, there is a limit, roughly 20, on the maximum number of sub-elements contained by any element in the State passed into ESMF_StateReconcile().
    • ESMF_BundleGather(), ESMF_FieldGather() and ESMF_ArrayGather() are only expected to work on 2D grids with 2D data.
    • The conservative regrid algorithm for full spheres, which includes special treatment of the poles, is largely untested and expected to produce errors.
    • When ESMF_Array objects are created with a non-zero halo width, that width is added to all dimensions of the array, not only ones which correspond to the decomposed grid axes.
    • The C/C++ Component interface is broken, so the C++ examples have been removed from the makefiles for this release.
    • On AIX systems there is a problem in the XL Fortran compiler version 8.1.1.6, when compiling with the combination ESMF_PREC=64 and ESMF_BOPT=g. (Compiler version 8.1.1.4 works fine with all compile options, and ESMF_PREC=32 or ESMF_BOPT=O also work with either compiler version.)
    • On AIX the "ESMF_FlowWithCoupling" system test and the "ESMF_COUPLED_FLOW" demo fail intermittently.
    • On the Linux PGI 5.0-2 compiler the "ESMF_FlowWithCoupling" system test and the "ESMF_COUPLED_FLOW" demo fail to compile with -g, resulting in an internal compiler error. (PGI 5.2-4 works fine)
    • When compiling with Intel 8.1 and using MPICH, make sure the Intel/MPICH libraries are configured with: #define MPI2CPP_HAVE_BOOL 1 which may not be the default output from configure.
    • Main programs in C/C++ do not work with the PGI compiler.
    • On the Linux PGI compiler the followingunit tests fail:
      • ESMF_ArrayUTest.F90
      • ESMF_VMBarrierUTest.F90
    • On the Linux Lahey compiler the ESMF_VMSendNbVMRecvNbUTest.F90 unit test fails.
    • Support for the LSF batch queueing system on the AIX platform needs script not included in the release.
    • All NAG compiler combinations we tested (on Linux as well as on Darwin) show random run-time failures when ESMF is compiled in optimized mode and also whe n user code that links with ESMF is compiled in optimized mode. We advise NAG users to set environment variable ESMF_OPTLEVEL to 0 (zero) when compiling ESMF a nd to not use optimization on user code that links against the ESMF library.
    • Also see SourceForge bugs list.
    ESMF_2_2_0rp1

    Reference Manual
    (html) (pdf)

    User's Guide
    (html) (pdf)

    Supported Platforms
    11/04/05
    • Fixed a bug in ESMF_StateReconcile() thatcaused memory corruption for objects containing attributes.
    • Modified an ESMF_StateReconcile() unit test to catch the bug described above.
    • Increased the size of the buffer used by ESMF_StateReconcile(), which caused a core dump if the buffer size was exceeded (this needs a better solution, but the patch will work for many users).
    • Removed all references to netcdf calls when ESMF_NO_IOCODE is set to true (due to conflicts between ESMF netcdf calls and user versions of the netcdf library)./li>
    • Fixed and documented an issue with MPICH that caused the working directory to be changed during the ESMF_Initialize() call. Now the API description of ESMF_Initialize() contains a paragraph discussing its dependency on MPI_Init() and the consequences thereof.
    • Added support for Intel 9.0 compilers on IA32 and IA64.
    • Same as release ESMF_2_2_0r with the exception of the ESMF_StateReconcile() bug.
    • Also see SourceForge bugs list.
    ESMF_2_2_0r

    Reference Manual
    (html) (pdf)

    User's Guide
    (html) (pdf)

    Supported Platforms
    07/13/05
    • List of ESMF interface changes since ESMF_2_1_1.
    • Fixed the following known bugs from previous releases:
      • Calling the redistribution and halo functions from a parent, coupler, or higher level Component fails on ESMF_Fields created in a child Component if that child is running on only a subset of the caller Component's PETs. This is the case, for example, when child Components are running concurrently.
      • Fixed the issue in the first-order conservative regrid sweep algorithm that calculated correct results but was inefficient.
    • Renamed dePetList argument in ESMF_DELayoutCreate() to petList to be more consistent with rest of the framework. Added more helpful comments describing the function of the petList argument.
    • Eliminated a bug in the VM setup which caused the sequence of PETs in the petList argument to ESMF_XXXCompCreate() to be ignored. Now the PETs of the child Component map to the parent PETs as is expected according to the petList sequence.
    • The initial phases of new code needed to optimize ESMF_Bundle level communication code have been added. An ESMF_Bundle which contains identically structured ESMF_Fields will only compute a single Route table and apply it to all ESMF_Fields. Later releases will include more optimizations inside the communication code itself.
    • The Log HALT_ON_ERROR function is now implemented.
    • In the Time Manager, ESMF_Clocks and ESMF_Alarms can now be run in reverse via a new optional ESMF_Direction type argument, which can have the values ESMF_MODE_REVERSE or ESMF_MODE_FORWARD in ESMF_ClockSet()/Get(). Forward mode is the default.
    • In the source tree, the Base directory was split into Base and Util, with utility routines which are not directly part of the Base class moved into the Util directory.
    • The default pointer size on Linux systems is now automatically determined using the output of the 'uname' command. 64-bit hardware defaults to ESMF_PREC 64, 32-bit hardware defaults to ESMF_PREC 32.
    • The ESMF_StateReconcile() call, generally used by couplers dealing with concurrently running Components, has an internal size limit (which will be addressed in a subsequent release). There is no hard-coded limit to how many elements, i.e. States, Bundles, Fields, Arrays can be added to the State. However, there is a limit, roughly 2, on the maximum number of sub-elements contained by any element in the State passed into ESMF_StateReconcile(). ESMF_StateReconcile() causes a core dump inthe buffer size is exceeded.
    • Calling ESMF_StateReconcile() for objects with attributes causes memory corruption.
    • The conservative regrid algorithm for full spheres, which includes special treatment of the poles, is largely untested and expected to produce errors.
    • When ESMF_Array objects are created with a non-zero halo width, that width is added to all dimensions of the array, not only ones which correspond to the decomposed grid axes.
    • The following Time Manager calendar conversions do not work: Gregorian <--> NoLeap <--> 360  Day.
    • Main programs in C/C++ do not work with the PGI compiler.
    • The ESMF_Bundle halo and redist functions must be called with ESMF_Bundles which contain identical ESMF_Field data types, rank, shape, and relative locations. Any other type of ESMF_Bundle will return with an ESMF_RC_NOT_IMPL error.
    • ESMF_BundleGather(), ESMF_FieldGather() and ESMF_ArrayGather() are only expected to work on 2D grids with 2D data.
    • On the Cray X1:
      • The __LINE__ macro does not work in F90; "line 8" is always printed from methods such as ESMF_Test();
      • The ESMF_RowReduce system test, ESMF_LogErrEx.F90 and ESMFVMAllFullReduceEx.F90 examples and several unit tests fail.
    • On IRIX64:
      • The ESMF_LogErrUTest.F90 unit tests fail due to a difference in log file format as described in Sourceforge bug 1172026;
      • The ESMF_LogErrUTest.F90 crashes when attempting to read a log file in Fortran that was created in C as described in Sourceforge bug 1212893.
    • All NAG compiler combinations we tested (on Linux as well as on Darwin) show random run-time failures when ESMF is compiled in optimized mode and also when user code that links with ESMF is compiled in optimized mode. We advise NAG users to set environment variable ESMF_OPTLEVEL to 0 (zero) when compiling ESMF and to not use optimization on user code that links against the ESMF library.
    • On Darwin XLF compiler:
      • ESMF_FieldRegridMulti system test fails with ESMF_COMM=mpiuni;
      • Several examples fail with ESMF_COMM=lam and ESMF_BOPT=g;
      • ESMF_BundleCommUTest.F90 unit test fails with ESMF_BOPT=g and ESMF_COMM=mpiuni.
    • On AIX systems there is a problem in the XL Fortran compiler version 8.1.1.6, when compiling with the combination ESMF_PREC=64 and ESMF_BOPT=g. (Compiler version 8.1.1.4 works fine with all compile options, and ESMF_PREC=32 or ESMF_BOPT=O also work with either compiler version.)
    • When compiling with Intel 8.1 and using MPICH, make sure the Intel/MPICH libraries are configured with:
      #define MPI2CPP_HAVE_BOOL 1
      which may not be the default output from configure.
    • Also see SourceForge bugs list.
    ESMF_2_1_1
    Reference Manual
    (html) (pdf)

    User's Guide
    (html) (pdf)

    Supported Platforms
    05/27/05
    • List of ESMF interface changes since ESMF_2_1_0rp3
    • More error checking added to the State routines.
    • Added the Abort option on ESMF_Finalize().
    • Fixed two regrid bugs for globally-indexed Fields:
      • Appeared in the bilinear regridding around domain edges on any processor except PE 0.
      • Appeared for 3D fields where the non-grid related index did not have a lower bound of one, for any regridding algorithm.
    • Additional regrid unit tests added.
    • ESMF_BundleRedistStore(), ESMF_FieldRedistStore() and ESMF_ArrayRedistStore() routines take a VMinstead of a DELayout.
    • Optional argument to select internal route data-packing and asychronous settings added to the ESMF_BundleRedistStore(), ESMF_FieldRedistStore() and ESMF_ArrayRedistStore() routines.
    • Halowidth removed from the ESMF_FieldCreate() calls where an already-created array is specified, which already contains halo information.
    • ESMF_VMAllGather() routine added.
    • Added option to create child Components which share the parent's VM.
    • Julian calendar added to Time Manager.
    • Added ESMF_CalendarIsLeapYear() and ESMF_TimeIsLeapYear() methods.
    • Additional internal packing options added to support different memory/ communication overhead ratios on different hardware platforms.
    • Extensive makefile and build system updates, including 64-bit PGI support, user-contributed g95 support, support for mpich on OSF (HP), updated make targets, and updated README's and User's Guide content.
    • It is now possible to compile ESMF withoutPthread dependency. If the environment variable ESMF_PTHREADS is set to OFF all Pthread dependent features of ESMF will be disabled. This can be used to simplify debugging and on systems that don't provide a functioning Pthread library. (This is different from the pthread-less default VM that was introduced in 2.1.0rp2!)
    • Compile-time options in system tests to run with threading enabled.
    • Output of unit and systems tests now go through the ESMF_Log facility, allowing multiprocessor test output to be cleanly collated.
    • Updated output scripts for results of unit and system tests.
    • More complete compiler and flag information printed for the output of 'gmake info'.
    • Site files need now only contain differences, not entire copies of the default makefiles.
    • Makefile target changes:
      • The previous default target was named 'all' but it only built libesmf. The current default target is named 'lib' and does the same thing. The new target 'all' builds the library and all the examples, tests, and demos.
      • The previous targets which were just 'tests' (e.g. build_tests) are now more specific: build_unit_tests, etc. There are new targets which are 'all_tests' to build everything.
      • Complete makefile target documentation may be found in the ESMF top level README file and in the User's Guide (subsection 6.2.3).
    • Added reference documentation about negative timesteps (as currently implemented, which is different from the upcoming reverse mode).
    • Expanded and updated the reference document bibliography, with citations.
    • The ESMF_StateReconcile() call, generally used by couplers dealing with concurrently running Components, has an internal size limit (which will be addressed in a subsequent release). There is no hard-coded limit to how many elements, i.e. States, Bundles, Fields, Arrays can be added to the State. However, there is a limit, roughly 2, on the maximum number of sub-elements contained by any element in the State passed into ESMF_StateReconcile(). ESMF_StateReconcile() causes a core dump if the buffer size is exceeded.
    • Calling ESMF_StateReconcile() for objects with attributes causes memory corruption.
    • Calling the redistribution and halo functions from a parent, coupler, or higher level Component fails on ESMF_Fields created in a child Component if that child is running on only a subset of the caller Component's PETs. This is the case, for example, when child Components are running concurrently. ESMF_FieldRegrid() does work correctly in this situation. Redistribution and halo functions work correctly when called from a child Component on ESMF_Fields created in that child Component.
    • The Log function "Halt on Error" has not been implemented, so we expect an error from the unit test ESMF_LogErrHaltUTest.F90 on all platforms. Log message is: FAILED, ESMF_LogErrHaltUTest.F90, line 96. The Halt on Error test failed.
    • The conservative regrid algorithm for full spheres, which includes special treatment of the poles, is largely untested and expected to produce errors. There is also some logic that is not correctly handling the situation where the physical domains of two Grids are slightly offset.  It does not result in errors or incorrect results but can be very inefficient.
    • ESMF_BundleGather(), ESMF_FieldGather() and ESMF_ArrayGather() are only expected to work on 2D grids with 2D data.
    • When ESMF_Array objects are created with a non-zero halo width, that width is added to all dimensions of the array, not only ones which correspond to the decomposed grid axes.
    • The following Time Manager calendar conversions do not work: Gregorian <--> NoLeap <--> 360  Day.
    • On several platforms examples with a main in C/C++ do not link.These include Linux/Lahey, Linux/Absoft, Linux/PGI with mpiuni, and OSF1 (halem).
    • The operating system and all compilers on the Cray X1 at Ames were updated this week. There are problems running with the updated system; to run on the X1 follow the instructions for reverting back to the original environment, or contact us for help in running with the latest system.
    • The default compiler on the Compaq (halem), Fortran  551J, is compatible with this release. Fortran 551L core dumps when running system tests.
    • On AIX systems there is a problem in the XL Fortran compiler version 8.1.1.6, when compiling with the combination ESMF_PREC=64 and ESMF_BOPT=g. (Compiler version 8.1.1.4 works fine with all compile options, and ESMF_PREC=32 or ESMF_BOPT=O also work with either compiler version.)
    • When compiling with Intel 8.1 and using MPICH, make sure the Intel/MPICH libraries are configured with:
      #define MPI2CPP_HAVE_BOOL 1
      which may not be the default output from configure.
    • On the Cray X1, the __LINE__ macro does not work in F90; "line 8" is always printed from methods such as ESMF_Test().
    • The ESMF_VMAllFullReduceEx.F90 example fails on the Cray X1.
    • On the SGI IRIX64 with ESMF_PREC=32, all system tests pass but hang up with ESMF_PREC=64.
    • The ESMF_LogErrUTest.F90 unit tests fail on IRIX64 due to a difference in log file format as described in Sourceforge bug 1172026.
    • The ESMF_VMBarrierUTest.F90 unit test fails on Linux/NAG, Linux/Intel, Linux/PGI and Linux/Lahey platforms.
    • Also see SourceForge bugs list.
    ESMF_2_1_0rp3
    Reference Manual
    (html) (pdf)

    User's Guide
    (html) (pdf)

    Supported Platforms
    05/19/05
    • Fixed the known bug listed in release ESMF_2_1_0rp2:
      • Banding problem in bilinear regridding, at the bottom or left boundaries of some destination DEs (those bounding other DEs and not at domain edges in those directions), in regridding decompositions except R-to-C or C-to-R and under certain coordinate overlay circumstances.
    • Added unit tests to test more Regrid combinations and to verify that the above bug has been fixed.
    • Same as release ESMF_2_1_0rp2 with the exception of the bilinear regridding banding problem.
    • Also see SourceForge bugs list.
    ESMF_2_1_0rp2

    Reference Manual
    (html (pdf)

    User's Guide
    (html) (pdf)
    03/11/05
    • Pthread-less default VM, i.e. the VMs of newly created Components will by default run without the use of threads.
    • Extended the command line argument code to correctly set options for ps(1) of BSD derived OSs, e.g. Darwin.
    • Fixed a problem with ESMF_StateReconcile() that would overwrite memory depending on the DELayout type. Sometimes no symptoms were evident and other times it would core dump. When the problem occurred, system test ESMF_FieldExcl would fail intermittently on the Linux platforms.
    • The performance for the redistribution function with arbitrarily distributed grids has been optimized.
    • The makefiles have been updated to support PGI 5.x, Absoft 9.x and NAG 5.x on Linux clusters.
    • Modified the command line Perl scripts so that now the number of passes and failures are printed as the last line of the script output instead of the first line.
    • Fixed bug in unit tests that falsely indicated that they were failing by relabeling tests that were mislabeled as non-exhaustive but were actually exhaustive. This problem occurred on all platforms. The following unit tests failed when run as non-exhaustive multi-processor:
      • ESMF_FRoute4UTest.F90
      • ESMF_FRoute8UTest.F90
      • ESMF_RegridUTest.F90
      • ESMF_TimeUTest.F90
    • The ESMF_StateReconcile() call, generally used by couplers dealing with concurrently running Components, has an internal size limit (which will be addressed in a subsequent release). There is no hard-coded limit to how many elements, i.e. States, Bundles, Fields, Arrays can be added to the State. However, there is a limit, roughly 2, on the maximum number of sub-elements contained by any element in the State passed into ESMF_StateReconcile(). ESMF_StateReconcile() causes a core dump if the buffer size is exceeded.
    • Calling ESMF_StateReconcile() for objects with attributes causes memory corruption.
    • Calling the redistribution and halo functions from a parent, coupler, or higher level Component fails on ESMF_Fields created in a child Component if that child is running on only a subset of the caller Component's PETs. This is the case, for example, when child Components are running concurrently. ESMF_FieldRegrid() does work correctly in this situation. Redistribution and halo functions work correctly when called from a child Component on ESMF_Fields created in that child Component.
    • Banding problem in bilinear regridding, at the bottom or left boundaries of some destination D Es (those bounding other DEs and not at domain edges in those directions), in regridding decompositions except R-to-C or C-to-R and under certain coordinate overlay circumstances.
    • ESMF_HaltType settings ESMF_LOG_HALTWARNING and ESMF_LOG_HALTERROR do not actually halt the code.
    • The default compiler on the Compaq (halem), Fortran 551J, is compatible with this release. Fortran 551L core dumps when running system tests.
    • ESMF_BundleGather(), ESMF_FieldGather() and ESMF_ArrayGather() are only expected to work on 2D grids with 2D data.
    • When ESMF_Array objects are created with a non-zero halo width, that width is added to all dimensions of the array, not only ones which correspond to the decomposed grid axes.
    • On AIX systems there is a problem in the XL Fortran compiler version 8.1.1.6, when compiling with the combination ESMF_PREC=64 and ESMF_BOPT=g. (Compiler version 8.1.1.4 works fine with all compile options, and ESMF_PREC=32 or ESMF_BOPT=O also work with either compiler version.)
    • When compiling with Intel 8.1 and using MPICH, make sure the Intel/MPICH libraries are configured with:
      #define MPI2CPP_HAVE_BOOL 1
      which may not be the default output from configure.
    • Main programs in C/C++ do not work with the PGI compiler.
    • On the Cray X1, the __LINE__ macro does not work in F90; "line 8" is always printed from methods such as ESMF_Test().
    • The following Time Manager calendar conversions do not work: Gregorian <--> NoLeap <--> 360 Day.
    • On the Darwin XLF compiler:
      • Example ESMF_LogErrEx.F90 fails.
      • System test, ESMF_FieldRegridMulti uni fails with ESMF_BOPT=O and ESMF_COMM=mpiuni.
      • Several examples fail with ESMF_BOPT=g and ESMF_COMM=lam.
    • On the Linux PGI Compiler, 9 examples fail with ESMF_BOPT=g.
    • Also see SourceForge bugs list.
    ESMF_2_1_0rp1
    Reference Manual
    (html) (pdf)

    User's Guide
    (html) (pdf)
    02/11/05
    • Concurrent Components run on the Altix and most Linux platforms now. Check the specific compiler release notes for any restrictions on supported compiler versions.
    • The Intel makefiles have been updated to support more compile-time configuration options.
    • A makefile target for running the demo uniprocessor was added (useful, for example, when ESMF is compiled with the mpiuni bypass libs.)  It is: ESMF_COUPLED_FLOW_uni
    • The output when building and running the unit tests, the examples, and the system tests has been improved.
    • Better defaults have been added to several Log interfaces, so the operations happen on the default Log if no user Log is specified.
    • Fixed problem that existed on halem linking applications with C main.
    • Added better diagnostic for when pthread_create fails because of limit settings.
    • Fixed an issue creating a Field from an allocated Fortran array.
    • Fixed the following Time Manager bugs:
      • Bug #1101904, "Disable stopTime to make clock run forever."
      • Bug #1113438, "getting dayOfYear should impose day boundary for getting hrs."
      • Bug #1118178, "ESMF_TimeGet returns string with %lld not %04lld."
    • The ESMF_StateReconcile() call, generally used by couplers dealing with concurrently running Components, has an internal size limit (which will be addressed in a subsequent release). There is no hard-coded limit to how many elements, i.e. States, Bundles, Fields, Arrays can be added to the State. However, there is a limit, roughly 2, on the maximum number of sub-elements contained by any element in the State passed into ESMF_StateReconcile(). ESMF_StateReconcile() causes a core dump if the buffer size is exceeded.
    • Calling ESMF_StateReconcile() for objects with attributes causes memory corruption.
    • Calling the redistribution and halo functions from a parent, coupler, or higher level Component fails on ESMF_Fields created in a child Component if that child is running on only a subset of the caller Component's PETs.  This is the case, for example, when child Components are running concurrently.  ESMF_FieldRegrid() does work correctly in this situation.  Redistribution and halo functions work correctly when called from a child Component on ESMF_Fields created in that child Component.
    • The default compiler on the Compaq (halem), Fortran  551J, is compatible with this release. Fortran 551L core dumps when running system tests.
    • ESMF_BundleGather(), ESMF_FieldGather() and ESMF_ArrayGather() are only expected to work on 2D grids with 2D data.
    • When ESMF_Array objects are created with a non-zero halo width, that width is added to all dimensions of the array, not only ones which correspond to the decomposed grid axes.
    • On AIX systems there is a problem in the XL Fortran compiler version 8.1.1.6, when compiling with the combination ESMF_PREC=64 and ESMF_BOPT=g. (Compiler version 8.1.1.4 works fine with all compile options, and ESMF_PREC=32 or ESMF_BOPT=O also work with either compiler version.)
    • When compiling with Intel 8.1 and using MPICH, make sure the Intel/MPICH libraries are configured with:
       #define MPI2CPP_HAVE_BOOL 1
       which may not be the default output from configure.
    • Main programs in C/C++ do not work with the PGI compiler.
    • The ESMF_StateReconcile() call does not work with the Lahey compiler.  The ESMF_FieldExcl system test uses this call, so it will not work either.
    • On the Cray X1, the __LINE__ macro does not work in F90; "line 8" is always printed from methods such as ESMF_Test().
    • The ESMF_VMGetMPICommunicatorEx example fails on the Linux/intel 64-bit build.
    • The ESMF_StateUTest test fails on the Linux/PGI build.
    • The ESMF_StateEx example fails on the Linux/PGI build.
    • Several of the examples fail on the Mac OS X/xlf build.
    • The ESMF_StateUTest fails 2 tests on the Linux/absoft build
    • The following Time Manager calendar conversions do not work: Gregorian <--> NoLeap <--> 360  Day
    • Also see SourceForge bugs list.
    ESMF_2_1_0r

    Reference Manual
    (html) (pdf)

    User's Guide
    (html) (pdf)
    01/14/05
    • Concurrent Component support has been added.  Component initialize, run, and finalize routines can be called with a flag which causes control to return immediately to the calling Component which can then start other Components in parallel.  Concurrency features are not yet operating correctly on Linux and Altix platforms (see Known Bugs).
    • Components can be created and run on a subset of the parent's Persistent Execution Threads, PETs. An ESMF_StateReconcile() call was added so that a parent Component can ensure that child Components who will be communicating each have all the State information that they need, even if Fields were not originally declared on both children.
    • Support for arbitrarily distributed grids and their redistribution to and from grids with block decompositions has been added.  This is currently very slow.
    • The issue with regrid that occasionally caused banding has been fixed.
    • The issue with regrid that occasionally returned NaNs on spherical coordinate systems has been fixed.
    • The issue with halo for periodic boundaries on non-decomposed axes has been fixed.
    • Problems with Route on PET counts that are not powers of 2 were fixed, and Route now works correctly for any PET count.
    • Some Field level communication routines now take a VM, not a DELayout.
    • Redist and regrid use asynchronous communications internally.
    • Support for concurrent Components has been activated in the VM.
    • Overhead when entering/exiting VM contexts has been reduced.
    • Conflicts between VM and MPICH's devices due to POSIX signal overlaps have been resolved.
    • New VM now works reliably with old, non-posix compliant (pre-NPTL) LinuxThreads, as well as POSIX compliant threads.
    • Internally added non-blocking communications to the VM to enhance RouteRun performance.
    • Several of the communication calls in the VM API have been overloaded to accept data of type 'ESMF_Logical' and 'Character'.
    • A new method ESMF_VMGetCurrent() has been added which can be used anywhere from within an ESMF application to obtain the current VM context without the need to provide any input information.
    • Component wait can now be called even on not currently running Components, in which case it will return immediately.
    • Buffering support has been added to LogErr. Writes to a Log now go through an adjustable buffer to improve performance.
    • Every entry in a Log now has the PET number prepended to the entry.
    • Multilog support has been added to LogErr. When the multilog option is set during Log initialization, separate Log files are created for the output from each PET.
    • Units of fractional seconds (specified with integers) are now supported within ESMF_Time and ESMF_TimeInterval.  The shortcut arguments ms, us, and ns (millisecond, microsecond, nanosecond) for Set()/Get() methods can be used as well as the more general arguments sN and sD (numerator, denominator).  Fractional seconds in Times and Time Intervals may fully participate in Clock, Alarm and Calendar objects and methods.  They also are fully functional within arithmetic (integer based) and comparison operations.
    • The following Time Manager bugs/enhancements have been fixed/done:
      • The Alarm limit per Clock is now dynamic in increments of 200.
      • ESMF_ClockPrint()/ESMF_TimePrint() now correctly adjusts for leap years.
      • The implicit day boundary in ESMF_TimeGet() has changed to a user-supplied boundary (fixed bug 1099731).
      • An Alarm will now ring immediately if its ring time is set equal to the Clock start time.
      • Can now drive No-Leap and 360-Day calendar-based times with seconds only.
      • Direct errors are now generated when dd < 1 or mm < 1 in ESMF_TimeSet().
      • Can now call ESMF_TimeSet() with only the dd argument; yy defaults to 0 and mm defaults to 1.
      • ESMF_ClockValidate() now works correctly after the end of an ESMF_ClockAdvance() loop.
    • The ESMF_StateReconcile() call, generally used by couplers dealing with concurrently running Components, has an internal size limit (which will be addressed in a subsequent release). There is no hard-coded limit to how many elements, i.e. States, Bundles, Fields, Arrays can be added to the State. However, there is a limit, roughly 2, on the maximum number of sub-elements contained by any element in the State passed into ESMF_StateReconcile(). ESMF_StateReconcile() causes a core dump if the buffer size is exceeded.
    • Calling ESMF_StateReconcile() for objects with attributes causes memory corruption.
    • Calling the redistribution and halo functions from a parent, coupler, or higher level Component fails on ESMF_Fields created in a child Component if that child is running on only a subset of the caller Component's PETs.  This is the case, for example, when child Components are running concurrently.  ESMF_FieldRegrid() does work correctly in this situation.  Redistribution and halo functions work correctly when called from a child Component on ESMF_Fields created in that child Component.
    • The default compiler on the Compaq (halem), Fortran  551J, is compatible with this release. Fortran 551L core dumps when running system tests.
    • ESMF_BundleGather(), ESMF_FieldGather() and ESMF_ArrayGather() are only expected to work on 2D grids with 2D data.
    • When ESMF_Array objects are created with a non-zero halo width, that width is added to all dimensions of the array, not only ones which correspond to the decomposed grid axes.
    • On AIX systems there is a problem in the XL Fortran compiler version 8.1.1.6, when compiling with the combination ESMF_PREC=64 and ESMF_BOPT=g. (Compiler version 8.1.1.4 works fine with all compile options, and ESMF_PREC=32 or ESMF_BOPT=O also work with either compiler version.)
    • When compiling with Intel 8.1 and using MPICH, make sure the Intel/MPICH libraries are configured with:
       #define MPI2CPP_HAVE_BOOL 1
       which may not be the default output from configure.
    • Main programs in C/C++ do not work with the PGI compiler.
    • The ESMF_StateReconcile() call does not work on the Linux and Altix builds of ESMF.  The ESMF_FieldExcl system test uses this call, so it will not work on these platforms either.
    • On the Cray X1, the __LINE__ macro does not work in F90; "line 8" is always printed from methods such as ESMF_Test().
    • The ESMF_VMGetMPICommunicatorEx example fails on the Linux/intel 64-bit build.
    • The ESMF_StateUTest test fails on the Linux/PGI build.
    • The ESMF_StateEx example fails on the Linux/PGI build.
    • Several of the examples fail on the Mac OS X/xlf build.
    • The ESMF_StateUTest fails 2 tests on the Linux/absoft build.
    • Also see SourceForge bugs list.


    ESMF_2_0_1

    ReferenceManual
    (html) (pdf)

    User's Guide
    (html) (pdf)

    9/24/04
    • ESMF_ArrayGather() now works even when the individual Arrays aren't identical sizes. This also fixes problems with ESMF_ArrayWrite()/ESMF_FieldWrite().
    • Array object now sets halo width correctly (bug 995309).
    • ESMF_FieldSetDataPointer() routine fixed to correctly create an Array.
    • ESMF_FieldWrite() supports R8 data.
    • Bundle level communication routines added (loops over Fields).
    • Bundles now validate that the Grids match, and contain more error checking.
    • The src/Infrastructure/IO/src files no longer require preprocessing.
    • Regrid supports R4 data.
    • Regrid code now correctly handles Fields with non-zero halo widths.
    • LogError now buffers messages internally before flushing them to a file, and has updated documentation.
    • Config no longer triggers errors from Fortran OPEN() when the file has read-only file permissions.
    • Time manager bugs fixed:
      • Bug 1014766 fixed; "dayOfYear always returns 1."  (also fixed similar bug in dayOfWeek).
      • The maximum number of alarms per clock has been increased to 200, from 100 previously.
      • Bug 996229 fixed; "Non-sticky alarms not functioning properly".
    • Build and test fixes:
      • ESMF_BUILD fixed for building files in a different directory from the source.
      • Automated nightly unit tests now run multiprocessor on some platforms.
    • Documents now build without having to compile the library first.
    • More documentation added for Grid and Regrid functions.
    • The default compiler on the Compaq (halem), Fortran  551J, is compatible with this release. Fortran 551L core dumps when running system tests.
    • ESMF_BundleGather(), ESMF_FieldGather() and ESMF_ArrayGather() are only expected to work on 2D grids with 2D data.
    • When ESMF_Array objects are created with a non-zero halo width, that width is added to all dimensions of the array, not only ones which correspond to the decomposed grid axes.
    • This release returns an internal compile error with the PGI/Linux compiler and ESMF_BOPT=g. It builds successfully with ESMF_BOPT=O.
    • There are known problems running with -O on Mac OS X compiling with absoft and running with lam mpi.  The workaround is to compile with ESMF_BOPT=g.
    • On AIX systems there is a problem in the XL Fortran compiler version 8.1.1.6, when compiling with the exact combination ESMF_PREC=64 and ESMF_BOPT=g. (Compiler version 8.1.1.4 works fine with all compile options, and ESMF_PREC=32 or ESMF_BOPT=O also work with either compiler version.)
    •  When compiling with Intel 8.1 and using MPICH, make sure the Intel/MPICH libraries are configured with:
      #define MPI2CPP_HAVE_BOOL 1
      which may not be the default output from configure.
    • Main programs in C/C++ do not work with the PGI compiler.
    • Routes will only work on PET counts that are powers of 2.
    • Some regrid results may show "banding" when regridding between decompositions where processor domains overlap only slightly.
    • Halo does not work correctly for periodic boundaries on non-decomposed axes.
    • Also see SourceForge bugs list.
    ESMF_2_0_0rp3

    Reference Manual
    (html) (pdf)

    User's Guide
    (html) (pdf)
    5/6/05
    • The following Regrid known bugs from the previous release have been fixed:
      • ESMF regrid functions operate only on R8 data. (It now operates on R4 and R8 data.)
      • ESMF regrid functions do not operate correctly on data with non-zero halo widths. 
      • Some regrid results may show some "banding" when regridding between decompositions where processor domains overlap only slightly.
    • Same as known bugs for ESMF_2_0_0rp2 excluding those listed in the Release Notes column.
    • Also see SourceForge bugs list.
    ESMF_2_0_0rp2

    Reference Manual
    (html) (pdf)

    User's Guide
    (html) (pdf)
    7/26/04
    • All AddAttribute() routines have been renamed SetAttribute() to address problems with method semantics.
    • Support has been added for attribute types of I4, I8 and R4, R8, both single values and lists.
    • Support has been added for setting and getting attributes on Grid objects.
    • Two bugs have been fixed in the VM code:
      • On some MPICH environments VM ran with only one PET, regardless of the number specified on the mpirun command line.
      • Segmentation faults when using MPICH's P4 device.
    • The following TimeMgr bugs have been fixed:
      • The 20 TimeInterval unit test failures under the PGI compile caused by PGI's peculiarity on passing missing F90 optional args, as well as inconsistent F90 local variable initializer behavior.
      • Erroneous LogErr messages caused by a lack of F95 derived-type initializers on all platforms.
    • Main programs in C/C++ do not work with the PGI compiler.
    • All the system tests pass running single processor except ESMF_FieldRedist() with the lahey compiler and ESMF_BOPT=O.
    • When ESMF_Array objects are created with a non-zero halo width, that width is added to all dimensions of the array, not only ones which correspond to the decomposed grid axes.
    • ESMF regrid functions operate only on R8 data. 
    • ESMF regrid functions do not operate correctly on data with non-zero halo widths.
    • ESMF_BundleGather(), ESMF_FieldGather() and ESMF_ArrayGather() are only expected to work on 2D grids with 2D data.
    • ESMF_FieldCreate() does not do sufficient error checking when given an already-existing ESMF_Array or allocated Fortran array pointer. Please see the examples in the Field directory.
    • Non-sticky alarms in Time Manager are not functioning properly.
    • ESMF_FieldGather() and ESMF_ArrayGather() only work correctly for evenly decomposed Grids.
    • When compiling with Intel 8.1 and using MPICH, make sure the Intel/MPICH libraries are configured with:
      #define MPI2CPP_HAVE_BOOL 1
      which may not be the default output from configure.
    • Routes will only work on processor counts that are powers of 2.
    • Some regrid results may show some "banding" when regridding between decompositions where processor domains overlap only slightly.
    • Halo does not work correctly for periodic boundaries on non-decomposed axes.
    • Also see SourceForge bugs list.
    ESMF_2_0_0rp1

    Reference Manual
    (html) (pdf)

    User's Guide
    (html) (pdf)
    6/30/04
    • Added Mac xlf build.
    ESMF_2_0_0r

    Reference Manual
    (html) (pdf)

    User's Guide
    (html) (pdf)

    6/23/04
    • Reference Manual and User's Guide reviewed and cleaned up.
    • ESMF_FieldWrite() working.
    • Main programs in C do not work with the PGI compiler.
    • All the system tests pass running single processor except ESMF_FieldRedist() with the lahey compiler and ESMF_BOPT=O.
    • When ESMF_Array objects are created with a non-zero haloWidth, that width is added to all dimensions of the array, not only ones which correspond to the decomposed grid axes.
    • ESMF regrid functions operate only on R8 data. 
    • ESMF regrid functions do not operate correctly on data with non-zero halo widths.
    • ESMF_BundleGather(), ESMF_FieldGather() and ESMF_ArrayGather() are only expected to work on 2D grids with 2D data.
    • ESMF_FieldCreate() does not do sufficient error checking when given an already-existing ESMF_Array or allocated Fortran array pointer. Please see the examples in the Field directory.
    • Problems with the VM:
      • On some MPICH environments VM ran with only one PET, regardless of the number specified on the mpirun command line.
      • Segmentation faults when using MPICH's P4 device.
    • Time Manager bugs:
      • Non-sticky alarms in Time Manager are not functioning properly.
    • ESMF_FieldGather() and ESMF_ArrayGather() only work correctly for evenly decomposed Grids.
    • Also see SourceForge bugs list.