Installation

Requirements

The following packages are required to work with ESMPy:

The following packages are optional:

  • ESMF installation with NetCDF - required to create Grids, Meshes and Fields from file, and to write regridding weights to file

    • NetCDF must be built as a shared library for ESMPy installation to succeed

  • ESMF installation with PIO (the Parallel IO library) - required to create Meshes and Fields from file, and to write regridding weights to file

  • mpi4py- python bindings to MPI, needed to run some of the parallel regridding examples

  • pytest - for testing

  • scipy - for testing (for reading NetCDF files in Python tests)

Getting the code

The ESMPy source code can be downloaded from the ESMF git repository.

The ESMF User’s Guide contains information on building and installing esmpy.

The ESMF Reference Manual contains information on the architecture of ESMF, example code, and details of the API (Application Programming Interface).

Anaconda Packages

ESMPy conda packages are available through the conda-forge channel:

conda create -n esmpy -c conda-forge esmpy

Specific versions of the conda package can be installed like this:

conda create -n esmpy -c conda-forge esmpy=8.1.0

Development versions can be found in the esmpy_dev channel:

conda create -n esmpy -c conda-forge -c esmpy_dev esmpy

Installing ESMPy from Source

When installing from source, ESMPy uses pip to build and install the package. An installation of ESMPy in the default location for Python packages can be done with the following command issued from the top level ESMPy directory (src/addon/esmpy):

python3 -m pip install .

Please contact esmf_support@ucar.edu with any questions.

Note

With some python versions, there is a problem with the automatic installation of the required package setuptools-git-versioning. If the output from the pip install command ends with Successfully installed esmpy-0.0.0, this indicates that the setuptools-git-versioning package was not invoked and the build thinks you are using ESMF version 0.0.0. This will cause problems when you try to use ESMPy. To solve this problem, simply rerun the above installation command (python3 -m pip install .). You should then see a reasonable version number at the end, such as, Successfully installed esmpy-8.4.0.

Importing ESMPy

ESMPy relies on a build of the underlying ESMF library. When using a conda package, this is made available automatically. However, when installing ESMPy from source, the corresponding ESMF library must also be built, per the ESMF documentation.

Before importing ESMPy, the environment variable ESMFMKFILE must be set, pointing to the location of the esmf.mk file from the ESMF installation. (If this environment variable is not found, the package will try to guess a few common locations, but we recommend correctly setting the variable nonetheless.) The location of this file is:

<ESMF_INSTALL_DIR>/lib/lib<g<or>O>/<platform>/esmf.mk

For example, with the bash shell on a Mac, where an optimized version of ESMF has been built with the gfortran+clang compilers and the OpenMPI MPI library, ESMFMKFILE can be set with the following (replacing the <ESMF_INSTALL_DIR> placeholder with the appropriate path):

export ESMFMKFILE=<ESMF_INSTALL_DIR>/lib/libO/Darwin.gfortranclang.64.openmpi.default/esmf.mk

After ESMFMKFILE has been set appropriately, to use ESMPy in an external program, import it with:

import esmpy

Note

The Python module name for ESMPy was changed in v8.4.0 from “ESMF” to “esmpy”. If you are using a version older than v8.4.0, the import command is import ESMF. See the ESMF Release Notes for more details and links to previous versions of the ESMPy documentation.

Validation

The ESMPy testing is done with the pytest package, both in serial and parallel. Basic unit tests can be run with the following command, from the top level ESMPy directory (src/addon/esmpy):

python3 -m pytest

There are a few other pytest targets available for a wider range of testing if greater test coverage is desired:

make test_unit

make test_examples

make test_unit_parallel

make test_examples_parallel

make test_regrid_from_file

Note

The regrid_from_file and example tests can take up a lot of memory and bandwidth. The download_regrid_from_file and download_examples commands will simply download the test files without actually running them (allowing the stress on the machine to be applied to bandwidth first, and then memory).

Note

By default, test data will be downloaded to a data subdirectory of the ESMPy installation directory. This location can be changed by setting one of the following environment variables:

  • If ESMPY_DATA_DIR is set, this should point to a directory that has already been populated with the necessary data; the pre-existing data will be read from this directory and no automatic downloads will be attempted.

  • Otherwise, if ESMPY_DATA_NEW_DIR is set, data will be downloaded to the path set by this variable instead of using the data subdirectory of the ESMPy installation directory.

Limitations

ESMPy doesn’t include many aspects of ESMF, including components, field bundles, time management, etc. The limitations listed here are relative to ESMF offline and integrated regridding capabilities.

  • ESMPy cannot use an ESMF installation that is built with external LAPACK support.

  • Coordinates cannot be retrieved from the elements of a Mesh. This can affect the ability to set Field values on a source Mesh created from file when using conservative regridding.

  • Multi-tile Grid support is limited to cubed-sphere grids. A Field created from a cubed-sphere Grid cannot be written to file in parallel.

  • There is no FieldBundle class, only single Fields.

Testing related:

  • Nightly regression testing is limited to a small subset of the ESMF test platforms, including Darwin and Linux with the gfortran and intel compilers.