Subsections


9 Building and Installing the ESMF

This section goes into more detail about how to build and install the ESMF software.

9.1 ESMF Download Options

Major releases of the ESMF software can be downloaded by following the instructions on the the Download link on the ESMF website, http://www.earthsystemmodeling.org.

The ESMF is distributed as a full source code tree. Follow the instructions in the following sections to build the library and link it with your application.

9.2 Acquiring Development Snapshots

Occasionally, it is helpful to acquire a development snapshot of ESMF in order to test emerging capabilities, optimizations, and bug fixes before they are available in a formal release. Development snaphots are “use at your own risk.” Efforts are made to ensure that most unit and system tests are passing on typical platforms, but there are no guarantees of the stability of development snapshots. New APIs available in development snapshots may change before the next release.

Users aware of these risks may check out development snapshots using the appropriate git tag.

Starting with ESMF 8.3.0 beta snapshot 07, the naming convention for development tags has the form:

v<VERSION>b<NUMBER>

For example:

v8.3.0b07

Prior to this version, the tag naming convention for development tags is:

ESMF_<VERSION>_beta_snapshot_<NUMBER>

For example:

ESMF_8_2_0_beta_snapshot_23

Use the following example command as a guide to check out a specific development tag:

  git clone https://github.com/esmf-org/esmf.git --branch v8.3.0b13 --depth 1

Once downloaded, development snapshots are built in the same way as releases.


9.3 System Requirements

The following compilers and utilities are required for compiling, linking and testing the ESMF software:

Internal packages that can optionally reference external libraries:

Optional external packages that must be specified for certain functions:

ESMF can be built using a single-processor MPI-bypass library that comes with ESMF by setting ESMF_COMM=mpiuni. This allows ESMF applications to be linked and run in single-process mode.

In order to build html and pdf versions of the ESMF documentation, LATEX, the latex2html conversion utility, and the Unix/Linux dvipdf utility must be installed. The csh shell is also required to complete the documentation build.


9.4 Third Party Libraries

Some portions of the ESMF library can offer enhanced capabilities when certain third party libraries are available. This section describes these dependencies and the associated environment settings that allow the user to control them.

On many platforms, the ESMF library is also created as a shared library. When third party libraries are called from ESMF, it is recommended that they are also available as shared libraries. In cases where they are not, they should at least be compiled with the position independent code option enabled (e.g., -fPIC on Linux with gfortran/gcc) where necessary, so that the ESMF shared library build can successfully incorporate them.


9.4.1 LAPACK

The patch recovery regridding method of the ESMF Mesh class requires solving local least squares problems. It uses the LAPACK DGELSY solver to carry out this task.

The following environment variables control whether a minimal set of LAPACK code that comes with ESMF is used, or whether ESMF should link against an externally available LAPACK installation. Alternatively, ESMF's LAPACK-dependent features can be turned off altogether.

ESMF_LAPACK
Possible value: "internal" (default), "OFF", "system", "mkl", "netlib", "scsl", openblas, <userstring>.

"internal" (default)
ESMF will be compiled with LAPACK-dependent features. A minimal set of LAPACK/BLAS code included in ESMF will be used to satisfy the dependencies.

"OFF"
Disables LAPACK-dependent code.

"system"
A system-dependent external LAPACK/BLAS installation is used to satisfy the external dependencies of the LAPACK-dependent ESMF code. Sets ESMF_LAPACK_LIBS appropriately.

"mkl"
The Intel MKL library is used to satisfy the external dependencies of the LAPACK-dependent ESMF code. When ESMF_COMPILER is set to "intel", ESMF_LAPACK_LIBS is set to "-mkl". Otherwise ESMF_LAPACK_LIBS is set to "-lmkl_lapack -lmkl", unless it is already defined in the user environment.

"netlib"
The NETLIB library is used to satisfy the external dependencies of the LAPACK-dependent ESMF code. Sets ESMF_LAPACK_LIBS to "-llapack -lblas", unless it is already defined in the user environment.

"scsl"
The SCSL library is used to satisfy the external dependencies of the LAPACK-dependent ESMF code. Sets ESMF_LAPACK_LIBS to "-lscs", unless it is already defined in the user environment.

"openblas"
The OpenBLAS library is used to satisfy the external dependencies of the LAPACK-dependent ESMF code. Sets ESMF_LAPACK_LIBS to "-openblas", unless it is already defined in the user environment.

<userstring>
Enables ESMF's LAPACK-dependent code, but does not set a default for ESMF_LAPACK_LIBS. ESMF_LAPACK_LIBS, and if required, ESMF_LAPACK_LIBPATH, must be set explicitly in the user environment.

ESMF_LAPACK_LIBPATH
Typical value: /usr/local/lib (no default).

Specifies the path where the LAPACK library is located.

ESMF_LAPACK_LIBS
Typical value: "-llapack -lblas" (default is dependent on ESMF_LAPACK).

Specifies the linker directive needed to link the LAPACK library to the application. On some systems, the BLAS library must also be included.


9.4.2 NetCDF

ESMF provides the ability to read Grid and Mesh data in NetCDF format.

Beginning with NetCDF 4.2, the C and Fortran API libraries are released as separate packages. To compile ESMF with NetCDF 4.2 and newer releases, the ESMF_NETCDF environment variable can be set to "split". The "split" option requires the NetCDF C library, and the NetCDF Fortran API library be installed in the same directory. As an alternative, the "nc-config" option may be used to automatically determine the include and lib directory locations. The "nc-config" option supports separate C and Fortran directories.

The following environment variables enable, and specify the name and location of the desired NetCDF library and associated header files:

ESMF_NETCDF
Possible value: not set (default), "nc-config", "split", "standard", <userstring>.

not set (default)
NetCDF-dependent features will be disabled. The ESMF_NETCDF_INCLUDE, ESMF_NETCDF_LIBPATH, and ESMF_NETCDF_LIBS environment variables will be ignored.

"nc-config"
The NetCDF nc-config and if available, nf-config, tools will be used to determine the proper settings of ESMF_NETCDF_INCLUDE, ESMF_NETCDF_LIBPATH, and ESMF_NETCDF_LIBS. The shell PATH environment variable must include the NetCDF bin directories where nc-config and nf-config reside. This option supports having the main NetCDF library and the Fortran API library reside in separate directories.

"split"
ESMF_NETCDF_LIBS will be set to "-lnetcdff -lnetcdf". This option is useful for systems which have the Fortran and C bindings archived in separate library files. The ESMF_NETCDF_INCLUDE and ESMF_NETCDF_LIBPATH environment variables will also be used, if defined.

"standard"
ESMF_NETCDF_LIBS will be set to "-lnetcdf". This option is useful when the Fortran and C bindings are archived together in the same library file. The ESMF_NETCDF_INCLUDE and ESMF_NETCDF_LIBPATH environment variables will also be used, if defined.

<userstring>
If set, ESMF_NETCDF_INCLUDE, ESMF_NETCDF_LIBPATH, and ESMF_NETCDF_LIBS environment variables will be used, if defined.

ESMF_NETCDF_INCLUDE
Typical value: /usr/local/include (no default).

Specifies the path where the NetCDF header files are located.

ESMF_NETCDF_LIBPATH
Typical value: /usr/local/lib (no default).

Specifies the path where the NetCDF library file is located.

ESMF_NETCDF_LIBS
Typical value: "-lnetcdf"

Specifies the linker directives needed to link the NetCDF library to the application.

The default value depends on the setting of ESMF_NETCDF. For the typical case where ESMF_NETCDF is set to "standard", ESMF_NETCDF_LIBS is set to "-lnetcdf". When ESMF_NETCDF is set to "split", ESMF_NETCDF_LIBS is set to "-lnetcdff -lnetcdf".

If the hdf5 library is required, append "-lhdf5_hl -lhdf5" to the desired setting. E.g. "-lnetcdff -lnetcdf -lhdf5_hl -lhdf5"


9.4.3 Parallel-NetCDF

ESMF provides the ability to write data using Parallel-NetCDF.

Some file systems, for example Lustre, may need to have locking attributes enabled when the file system is mounted.

The following environment variables enable and specify the name and location of the desired Parallel-NetCDF library and associated header files:

ESMF_PNETCDF
Possible value: not set (default), "pnetcdf-config", "standard", <userstring>.

When defined, enables the use of Parallel-NetCDF.

not set (default)
PNETCDF-dependent features will be disabled. The ESMF_PNETCDF_INCLUDE, ESMF_PNETCDF_LIBPATH, and ESMF_PNETCDF_LIBS environment variables will be ignored.

"pnetcdf-config"
The PNetCDF pnetcdf-config tool will be used to determine the proper settings of ESMF_PNETCDF_INCLUDE, ESMF_PNETCDF_LIBPATH, and ESMF_PNETCDF_LIBS. The shell PATH environment variable must include the PNetCDF bin directory where pnetcdf-config resides.

"standard"
ESMF_PNETCDF_LIBS will be set to "-lpnetcdf". The ESMF_PNETCDF_INCLUDE and ESMF_PNETCDF_LIBPATH environment variables will also be used, if defined.

<userstring>
If set, ESMF_PNETCDF_INCLUDE, ESMF_PNETCDF_LIBPATH, and ESMF_PNETCDF_LIBS environment variables will be used.

ESMF_PNETCDF_INCLUDE
Typical value: /usr/local/include (no default).

Specifies the path where the Parallel-NetCDF header files are located.

ESMF_PNETCDF_LIBPATH
Typical value: /usr/local/lib (no default).

Specifies the path where the Parallel-NetCDF library file is located.

ESMF_PNETCDF_LIBS
Typical value: "-lpnetcdf".

Specifies the linker directives needed to link the Parallel-NetCDF library to the application.


9.4.4 PIO

ESMF provides the ability to read and write data in NetCDF format through ParallelIO (PIO), a third-party I/O software library that is integrated into the ESMF library. The following environment variable enables PIO functionality inside of ESMF.

The PIO code depends on MPI I/O support by the underlying MPI implementation for parallel I/O. Almost all current MPI implementations support MPI I/O to the required degree. For NetCDF format support the integrated PIO code depends on ESMF_NETCDF (see 9.4.2) being enabled and optionally ESMF_PNETCDF (see 9.4.3) being enabled.

ESMF_PIO
Possible value: not set (default), "internal", "external", "OFF".

not set (default)
PIO-dependent features will be enabled on supported platforms, as determined by the ESMF build configuration.

"internal"
PIO-dependent features will be enabled and will use the PIO library that is included and built with ESMF. Internal builds of PIO require CMake version 2.8.12 or newer be available in the path.

"external"
PIO-dependent features will be enabled and will use an external PIO library. The additional parameters ESMF_PIO_INCLUDE (path to PIO include files) and ESMF_PIO_LIBPATH (path to PIO library files) should also be set when using this option. The minimum version of PIO for this option is 2.5.7.

"OFF"
Disables PIO-dependent code.

ESMF_PIO_INCLUDE
(no default)

Specifies the path where the PIO header files are located.

ESMF_PIO_LIBPATH
(no default)

Specifies the path where the PIO library is located.


9.4.5 Accelerator Software Stacks

ESMF provides the ability to query various third party accelerator software stacks and gather information about the accelerator devices available in a system. The users can query the number of accelerator devices accessible from a PET using the OpenCL, OpenACC, Intel MIC and OpenMP software stacks.

The following environment variables enable, and specify the name and location of the desired accelerator software stacks and associated header files:

ESMF_ACC_SOFTWARE_STACK
Possible value: not set (default), "opencl", "openacc", "intelmic", "openmp4".

not set (default)
All accelerator software stack related features will be disabled. The ESMF_ACC_SOFTWARE_STACK_INCLUDE, ESMF_ACC_SOFTWARE_STACK_LIBPATH, and ESMF_ACC_SOFTWARE_STACK_LIBS environment variables will be ignored.

"opencl"
The ESMF library will use the OpenCL framework to query information about accelerator devices in the system. The ESMF_ACC_SOFTWARE_STACK_INCLUDE, ESMF_ACC_SOFTWARE_STACK_LIBPATH and ESMF_ACC_SOFTWARE_STACK_LIBS environment variables will be used to build and link the library.

"openacc"
The ESMF library will use the interfaces defined in the OpenACC standard to query information about accelerator devices in the system. The ESMF_ACC_SOFTWARE_STACK_INCLUDE, ESMF_ACC_SOFTWARE_STACK_LIBPATH and ESMF_ACC_SOFTWARE_STACK_LIBS environment variables are not typically defined since the standard is supported inherently by a OpenACC standard compliant compiler.

"intelmic"
The ESMF library will use the interfaces defined by the Intel MIC software stack to query information about accelerator devices in the system. The ESMF_ACC_SOFTWARE_STACK_INCLUDE, ESMF_ACC_SOFTWARE_STACK_LIBPATH and ESMF_ACC_SOFTWARE_STACK_LIBS environment variables are not typically defined since the standard is supported inherently by the Intel compiler.

"openmp4"
The ESMF library will use the interfaces defined in the OpenMP v4.0 standard to query information about accelerator devices in the system. The ESMF_ACC_SOFTWARE_STACK_INCLUDE, ESMF_ACC_SOFTWARE_STACK_LIBPATH and ESMF_ACC_SOFTWARE_STACK_LIBS environment variables are not typically defined since the standard is supported inherently by a standard compliant compiler.

ESMF_ACC_SOFTWARE_STACK_INCLUDE
(no default)

Specifies the path where the header files for the accelerator software stack is located. If not set, this environment variable is ignored.

ESMF_ACC_SOFTWARE_STACK_LIBPATH
(no default)

Specifies the path where the libraries for the accelerator software stack is located. If not set, this environment variable is ignored.

ESMF_ACC_SOFTWARE_STACK_LIBS
(no default)

Specifies the linker directives required to link the library with the accelerator software stack. If not set, this environment variable is ignored.


9.4.6 XERCES

ESMF provides the ability to read Attribute data in XML file format via the XERCES C++ library. (Writing Attribute XML files is performed with the standard C++ output file stream facility, rather than with Xerces). The following environment variables enable, and specify the name and location of the desired XERCES C++ library and associated header files:

ESMF_XERCES
Possible value: not set (default), "standard", <userstring>.

not set (default)
XERCES-dependent features will be disabled. The ESMF_XERCES_INCLUDE, ESMF_XERCES_LIBPATH, and ESMF_XERCES_LIBS environment variables will be ignored.

"standard"
ESMF_XERCES_LIBS will be set to "-lxerces-c". The ESMF_XERCES_INCLUDE and ESMF_XERCES_LIBPATH environment variables will also be used, if defined.

<userstring>
If set, ESMF_XERCES_INCLUDE, ESMF_XERCES_LIBPATH, and ESMF_XERCES_LIBS environment variables will be used, if defined.

ESMF_XERCES_INCLUDE
Typical value: /usr/local/include (no default).

Specifies the path where the XERCES C++ header files are located.

ESMF_XERCES_LIBPATH
Typical value: /usr/local/lib (no default).

Specifies the path where the XERCES C++ library file is located.

ESMF_XERCES_LIBS
Typical value: "-lxerces-c".

Specifies the linker directives needed to link the XERCES C++ library to the application.

The default value depends on the setting of ESMF_XERCES. For the typical case where ESMF_XERCES is set to "standard", ESMF_XERCES_LIBS is set to "-lxerces-c".


9.4.7 yaml-cpp

Support for I/O in YAML Ain't Markup Language (YAMLTM) may be added to ESMF through the open-source yaml-cpp library, a YAML parser and emitter written in C++ that implements YAML Version 1.2 specifications.

ESMF includes the option to build the yaml-cpp from sources kept inside the ESMF source tree, or to link against an external build of the yaml-cpp library. The following environment variables control the details of how ESMF interacts with yaml-cpp:

ESMF_YAMLCPP
Possible values: "internal" (default), "OFF", "standard", <userstring>.

"internal" (default)
The YAML-dependent code inside of ESMF will be enabled. The yaml-cpp sources included with ESMF will be used to provide YAML support. The ESMF_YAMLCPP_INCLUDE, ESMF_YAMLCPP_LIBPATH, and ESMF_YAMLCPP_LIBS environment variables will be ignored.

"OFF"
Disables YAML-dependent code.

"standard"
The YAML-dependent code inside of ESMF will be enabled. ESMF_YAMLCPP_LIBS will be set to "-lyaml-cpp" if not set. The ESMF_YAMLCPP_INCLUDE and ESMF_YAMLCPP_LIBPATH environment variables will also be used, if defined.

<userstring>
The YAML-dependent code inside of ESMF will be enabled. If set, ESMF_YAMLCPP_INCLUDE, ESMF_YAMLCPP_LIBPATH, and ESMF_YAMLCPP_LIBS environment variables will be used.

ESMF_YAMLCPP_INCLUDE
Typical value: /usr/local/include (no default).

Specifies the path where the yaml-cpp C++ header files are located.

ESMF_YAMLCPP_LIBPATH
Typical value: /usr/local/lib (no default).

Specifies the path where the yaml-cpp C++ library file is located.

ESMF_YAMLCPP_LIBS
Typical value: "-lyaml-cpp".

Specifies the linker directives needed to link the yaml-cpp C++ library to the application.

The default value depends on the setting of ESMF_YAMLCPP. For the typical case where ESMF_YAMLCPP is set to "standard", ESMF_YAMLCPP_LIBS is set to "-lyaml-cpp".


9.4.8 MOAB

The Mesh Oriented datABase (MOAB) can be used to build an ESMF unstructured Mesh as an alternative to the "native" ESMF Mesh implementation. The decision to use either MOAB or the native ESMF Mesh implementation is made at run time. This aspect is described in the Reference Manual, section ESMF_MeshSetMOAB() .The default is to use the native ESMF Mesh. ESMF will build an internal version of MOAB by default, but an external MOAB installation can be used if desired. The build parameters covered in this section are used to determine which version of MOAB is available to ESMF.

ESMF_MOAB
Possible values: "internal" (default), "OFF", "external".

"internal" (default)
The MOAB dependent code inside of ESMF will be enabled. The MOAB sources included with ESMF will be used to provide MOAB support. The ESMF_MOAB_INCLUDE, ESMF_MOAB_LIBPATH, and ESMF_MOAB_LIBS environment variables will be ignored.

"OFF"
Disables MOAB dependent code.

"external"
The MOAB dependent code inside of ESMF will be enabled. The ESMF_MOAB_INCLUDE, ESMF_MOAB_LIBPATH and ESMF_MOAB_LIBS environment variables must also be specified.

ESMF_MOAB_INCLUDE
Typical value: /usr/local/include (no default).

Specifies the path where the MOAB C++ header files are located.

ESMF_MOAB_LIBPATH
Typical value: /usr/local/lib (no default).

Specifies the path where the MOAB C++ library file is located.

ESMF_MOAB_LIBS
Typical value: "-lMOAB" (no default).

Specifies the linker directives needed to link the MOAB C++ library to the application.


9.5 ESMF Environment Variables

The following is a full alphabetical list of all environment variables which are used by the ESMF build system. In many cases only ESMF_DIR must be set. On Linux and Darwin systems ESMF_COMPILER and ESMF_COMM must also be set to select the appropriate Fortran and C++ compilers and MPI implementation. The other variables have default values which work for most systems.

ESMF_ABI
Possible value: 32, 64, x86_64_32, x86_64_small, x86_64_medium

If a system supports 32-bit and 64-bit (pointer wordsize) application binary interfaces (ABIs), this variable can be set to select which ABI to use. Valid values are 32 or 64. By default the most common ABI is chosen. On x86_64 architectures three additional, more specific ABI settings are available, x86_64_32, x86_64_small and x86_64_medium.

ESMF_ARRAY_LITE
Possible value: TRUE, FALSE (default)

Not normally set by user. ESMF auto-generates subroutine interfaces for a wide variety of data arrays of different ranks, shapes, and types. Setting this variable to TRUE instructs ESMF to not generating interfaces for 5D, 6D, and 7D arrays. This shrinks the amount of autogenerated code as well as the number of overloaded interfaces.

ESMF_BOPT
Possible value: g, O (default)

This environment variable controls the build option. To make a debuggable version of the library set ESMF_BOPT to g before building. The default is O (capital oh) which builds an optimized version of the library. If ESMF_BOPT is O, ESMF_OPTLEVEL can also be set to a numeric value between 0 and 4 to select a specific optimization level.

ESMF_COMM
Possible value: system-dependent

On systems with a vendor-supplied MPI communications library, the vendor library is chosen by default for communications. On these systems ESMF_COMM is set to mpi, signaling to the ESMF build system to use the vendor MPI implementation. For other systems (e.g. Linux or Darwin) where a multitude of MPI implementations are available, ESMF_COMM must be set to indicate which implementation is used to build the ESMF library. Set ESMF_COMM according to your situation to: mpt, mpich, mpich1, mpich2, mpich3, mvapich2, lam, openmpi or intelmpi. ESMF_COMM may also be set to user indicating that the user will set all the required flags using advanced ESMF environment variables. Some individual MPI builds may create additional libraries that need to be linked in, such as the legacy C++ bindings. These may be specified via the ESMF_CXXLINKLIBS and ESMF_F90LINKLIBS environment variables.

Alternatively, ESMF comes with a single-processor MPI-bypass library which is the default for Linux and Darwin systems. To force the use of this bypass library set ESMF_COMM equal to mpiuni.

ESMF_COMPILER
Possible value: system-dependent

The ESMF library build requires a working Fortran90 and C++ compiler. On platforms that don't come with a single vendor supplied compiler suite (e.g. Linux or Darwin) ESMF_COMPILER must be set to select which Fortran and C++ compilers are being used to build the ESMF library. Notice that setting the ESMF_COMPILER variable does not affect how the compiler executables are located on the system. ESMF_COMPILER (together with ESMF_COMM) affect the name that is expected for the compiler executables. Furthermore, the ESMF_COMPILER setting is used to select compiler and linker flags consistent with the compilers indicated.

By default Fortran and C++ compiler executables are expected to be located in a location contained in the user's PATH environment variable. This means that if you cannot locate the correct compiler executable via the which command on the shell prompt the ESMF build system won't find it either!

There are advanced ESMF environment variables that can be used to select specific compiler executables by specifying the full path. This can be used to pick specific compiler executables without having to modify the PATH environment variable.

Use 'gmake info' to see which compiler executables the ESMF build system will be using according to your environment variable settings.

To see possible values for ESMF_COMPILER, cd to $ESMF_DIR/build_config and list the directories there. The first part of each directory name corresponds to the output of 'uname -s' for this platform. The second part contains possible values for ESMF_COMPILER. In some cases multiple combinations of Fortran and C++ compilers are possible, e.g. there is intel and intelgcc available for Linux. Setting ESMF_COMPILER to intel indicates that both Intel Fortran and C++ compilers are used, whereas intelgcc indicates that the Intel Fortran compiler is used in combination with GCC's C++ compiler.

If you do not find a configuration that matches your situation you will need to port ESMF.

ESMF_CXX
Possible value: executable

This variable can be used to override the default C++ compiler and linker front-end executables. The executable may be specified with absolute path overriding the location determined by default from the user's PATH variable.

ESMF_CXXCOMPILEOPTS
Possible value: list of flags

Prepend compiler flags to the list of flags the ESMF build system determines.

ESMF_CXXCOMPILEPATHS
Possible value: list of paths, each prepended with -I

Prepend compiler search paths to the list of search paths the ESMF build system determines.

ESMF_CXXCOMPILER
Possible value: executable

This variable can be used to override the default C++ compiler front-end executables. The executable may be specified with absolute path overriding the location determined by default from the user's PATH variable.

ESMF_CXXLINKER
Possible value: executable

This variable can be used to override the default C++ linker front-end executables. The executable may be specified with absolute path overriding the location determined by default from the user's PATH variable.

ESMF_CXXLINKLIBS
Possible value: list of libraries, each prepended with -l

Prepend libraries to the list of libraries the ESMF build system determines.

ESMF_CXXLINKOPTS
Possible value: list of flags

Prepend linker flags to the list of flags the ESMF build system determines.

ESMF_CXXLINKPATHS
Possible value: list of paths, each prepended with -L

Prepend linker search paths to the list of search paths the ESMF build system determines.

ESMF_CXXLINKRPATHS
Possible value: list of paths, each prepended with the correct rpath option

Prepend linker rpaths to the list of rpaths the ESMF build system determines.

ESMF_CXXOPTFLAG
Possible value: flag

This variable can be used to override the default C++ optimization flag.

ESMF_CXXSTD
Possible value: integer or default or sysdefault

Used to set the C++ language standard. If unset or default, the ESMF default C++ language standard is used: C++11. If set to an integer, the integer is used to indicate the respective C++ language standard to the compiler. ESMF does not check whether the integer corresponds to an existing language standard. Setting sysdefault results in usage of the compiler specific default C++ language standard. This can lead to build issue if the compiler default is below the level required by ESMF.

ESMF_DEFER_LIB_BUILD
Possible value: ON (default), OFF

This variable can be used to override the deferring of the build of the ESMF library. By default, the library is built after all of the source files have been compiled. This speeds up the build process. It also allows parallel compilation of source code when the -j flag is used with gmake. Setting this environment variable to OFF forces the library to be updated after each individual compilation, thus disabling the ability to use parallel compilation.

ESMF_DIR
Possible value: absolute path

The environment variable ESMF_DIR must be set to the full pathname of the top level ESMF directory before building the framework. This is the only environment variable which is required to be set on all platforms under all conditions.

ESMF_F90
Possible value: executable

This variable can be used to override the default Fortran90 compiler and linker front-end executables. The executable may be specified with absolute path overriding the location determined by default from the user's PATH variable.

ESMF_F90COMPILEOPTS
Possible value: list of flags

Prepend compiler flags to the list of flags the ESMF build system determines.

ESMF_F90COMPILEPATHS
Possible value: list of paths, each prepended with -I

Prepend compiler search paths to the list of search paths the ESMF build system determines.

ESMF_F90COMPILER
Possible value: executable

This variable can be used to override the default Fortran90 compiler front-end executables. The executable may be specified with absolute path overriding the location determined by default from the user's PATH variable.

ESMF_F90IMOD
Possible value: flag

This variable can be used to override the default flag (-I) used to specify a Fortran module directory.

ESMF_F90LINKER
Possible value: executable

This variable can be used to override the default Fortran90 linker front-end executables. The executable may be specified with absolute path overriding the location determined by default from the user's PATH variable.

ESMF_F90LINKLIBS
Possible value: list of libraries, each prepended with -l

Prepend libraries to the list of libraries the ESMF build system determines.

ESMF_F90LINKOPTS
Possible value: list of flags

Prepend linker flags to the list of flags the ESMF build system determines.

ESMF_F90LINKPATHS
Possible value: list of paths, each prepended with -L

Prepend linker search paths to the list of search paths the ESMF build system determines.

ESMF_F90LINKRPATHS
Possible value: list of paths, each prepended with the correct rpath option

Prepend linker rpaths to the list of rpaths the ESMF build system determines.

ESMF_F90OPTFLAG
Possible value: flag

This variable can be used to override the default Fortran90 optimization flag.

ESMF_INSTALL_BINDIR
Possible value: relative or absolute path

Location into which to install the ESMF apps during installation. This location can be specified as absolute path (starting with "/") or relative to ESMF_INSTALL_PREFIX.

ESMF_INSTALL_DOCDIR
Possible value: relative or absolute path

Location into which to install the documentation during installation. This location can be specified as absolute path (starting with "/") or relative to ESMF_INSTALL_PREFIX.

ESMF_INSTALL_HEADERDIR
Possible value: relative or absolute path

Location into which to install the header files during installation. This location can be specified as absolute path (starting with "/") or relative to ESMF_INSTALL_PREFIX.

ESMF_INSTALL_LIBDIR
Possible value: relative or absolute path

Location into which to install the library files during installation. This location can be specified as absolute path (starting with "/") or relative to ESMF_INSTALL_PREFIX.

ESMF_INSTALL_MODDIR
Possible value: relative or absolute path

Location into which to install the F90 module files during installation. This location can be specified as absolute path (starting with "/") or relative to ESMF_INSTALL_PREFIX.

ESMF_INSTALL_PREFIX
Possible value: relative or absolute path

This variable specifies the prefix of the installation path used during the installation process accessible thought the install target. Libraries, F90 module files, header files and documentation all are installed relative to ESMF_INSTALL_PREFIX by default. The ESMF_INSTALL_PREFIX may be provided as absolute path (starting with "/") or relative to ESMF_DIR.

ESMF_LAPACK
See 9.4.1

ESMF_LAPACK_LIBPATH
See 9.4.1

ESMF_LAPACK_LIBS
See 9.4.1

ESMF_MACHINE
Possible value: output of uname -m where available.

Not normally set by user. This variable indicates architectural details about the machine on which the ESMF library is being built. The value of this variable will affect which ABI settings are available and what they mean. ESMF_MACHINE is set automatically.

ESMF_MPIBATCHOPTIONS
Possible value: system-dependent

Variable used to pass system-specific queue options to the batch system. Typically the queue, project and limits are set. See section 11.1.1 for a discussion of this option.

ESMF_MPILAUNCHOPTIONS
Possible value: system-dependent

Variable used to pass system-specific options to the MPI launch facility. See section 11.1.1 for a discussion of this option.

ESMF_MPIMPMDRUN
Possible value: executable

This variable can be used to override the default utility used to launch parallel execution of ESMF test applications in MPMD mode. The executable in ESMF_MPIMPMDRUN may be specified with path.

ESMF_MPIRUN
Possible value: executable

This variable can be used to override the default utility used to launch parallel ESMF test or example applications. The executable in ESMF_MPIRUN may be specified with path. See section 11.1.1 for a discussion of this option.

ESMF_MPISCRIPTOPTIONS
Possible value: system-dependent

Variable used to pass system-specific options to the first level MPI script accessed by ESMF. See section 11.1.1 for a discussion of this option.

ESMF_NETCDF
See 9.4.2

ESMF_NETCDF_INCLUDE
See 9.4.2

ESMF_NETCDF_LIBPATH
See 9.4.2

ESMF_NETCDF_LIBS
See 9.4.2

ESMF_NO_INTEGER_1_BYTE
Possible value: TRUE (default), FALSE

Not normally set by user. Setting this variable to FALSE instructs ESMF to generating data array interfaces for data types of 1-byte integers.

ESMF_NO_INTEGER_2_BYTE
Possible value: TRUE (default), FALSE

Not normally set by user. Setting this variable to FALSE instructs ESMF to generating data array interfaces for data types of 2-byte integers.

ESMF_OPENACC
Possible value: ON, OFF (default)

Compiles and links the ESMF library with OpenACC compiler flags.

ESMF_OPENMP
Possible value: ON, OFF (default is system dependent)

Compiles and links the ESMF library with OpenMP compiler flags.

ESMF_OPTLEVEL
Possible value: numerical value

See ESMF_BOPT for details.

ESMF_OS
Possible value: output of uname -s except when cross-compiling or for UNICOS/mp where ESMF_OS is Unicos.

Not normally set by user unless cross-compiling. This variable indicates the target system for which the ESMF library is being built. Under normal circumstances, i.e. ESMF is being build on the target system, ESMF_OS is set automatically. However, when cross-compiling for a different target system ESMF_OS must be set to the respective target OS. For example, when compiling for the Cray X1 on an interactive X1 node ESMF_OS will be set automatically. However, when ESMF is being cross-compiled for the X1 on a Linux host the user must set ESMF_OS to Unicos manually in order to indicate the intended target platform.

ESMF_PNETCDF
See 9.4.3

ESMF_PNETCDF_INCLUDE
See 9.4.3

ESMF_PNETCDF_LIBPATH
See 9.4.3

ESMF_PNETCDF_LIBS
See 9.4.3

ESMF_PTHREADS
Possible value: ON (default on most platforms), OFF

This compile-time option controls ESMF's dependency on a functioning Pthreads library. The default option is set to ON with the exception of IRIX64 and platforms that don't provide Pthreads. On IRIX64 the use of Pthreads in ESMF is disabled by default because the Pthreads library conflicts with the use of OpenMP on this platform.

The user can override the default setting of ESMF_PTHREADS on all platforms that provide Pthread support. Setting the ESMF_PTHREADS environment variable to OFF will disable ESMF's Pthreads feature set. On platforms that don't support Pthreads, e.g. IBM BlueGene/L or Cray XT3, the default OFF setting cannot be overridden!

ESMF_SITE
Possible value: site string, default

Build configure file site name or the value default. If not set, then the value of default is assumed. When including platform-specific files, this value is used as the third part of the directory name (parts 1 and 2 are the ESMF_OS value and ESMF_COMPILER value, respectively.)

The Sourceforge esmfcontrib repository contains makefiles which have already been customized for certain machines. If one exists for your site and you wish to use it, download the corresponding files into the build_contrib directory and set ESMF_SITE to your location (which corresponds to the last part of the directory name). See the Sourceforge site http://sourceforge.net/projects/esmfcontrib for more information.

ESMF_TESTESMFMKFILE
Possible value: ON, OFF (default)

Variable specifying whether the ESMFMKFILE variable is evaluated to determine which ESMF installation is being tested against. If set to the value ON, all tests and examples are build against the ESMF installation referenced by the ESMFMKFILE variable. For OFF, the ESMFMKFILE variable is ignored and the tests and examples are build against the ESMF under ESMF_DIR. This is the default.

ESMF_TESTEXHAUSTIVE
Possible value: ON, OFF (default)

Variable specifying how to compile the unit tests. If set to the value ON, then all unit tests will be compiled and will be executed when the test is run. If unset or set to any other value, only a subset of the unit tests will be included to verify basic functions. Note that this is a compile-time selection, not a run-time option.

ESMF_TESTFORCEOPENACC
Possible value: ON, OFF (default)

The ON setting enforces usage of OpenACC compiler flags when building ESMF test applications. This allows testing of user-level OpenACC usage even with ESMF_OPENACC set to OFF.

ESMF_TESTFORCEOPENMP
Possible value: ON, OFF (default)

The ON setting enforces usage of OpenMP compiler flags when building ESMF test applications. This allows testing of user-level OpenMP usage even with ESMF_OPENMP set to OFF.

ESMF_TESTHARNESS_ARRAY
Possible value: test harness make target (default not set)

Variable specifying the test harness makefile target for the array class. If this variable is not specified, a default test scenario will be run for the array class. See the ESMF Software Developer's Guide for instructions for selecting other test harness scenarios.

ESMF_TESTHARNESS_FIELD
Possible value: test harness make target (default not set)

Variable specifying the test harness makefile target for the field class. If this variable is not specified, a default test scenario will be run for the field class. See the ESMF Software Developer's Guide for instructions for selecting other test harness scenarios.

ESMF_TESTMPMD
Possible value: ON, OFF (default)

Variable specifying whether to run MPMD-style tests, i.e. test applications that start up as multiple separate executables.

ESMF_TESTSHAREDOBJ
Possible value: ON, OFF (default)

Variable specifying whether to run shared object tests. This requires that the compute environment supports shared objects, and that the ESMF library is available in form of a shared library.

ESMF_TESTWITHTHREADS
Possible value: ON, OFF (default)

If this environment variable is set to ON before the ESMF system tests are build they will activate ESMF threading in their code. Specifically each component will be executed using ESMF single threading instead of the default non-threaded mode. The difference between non-threaded and ESMF single threaded execution should be completely transparent. Notice that the setting of ESMF_TESTWITHTHREADS does not alter ESMF's dependency on Pthreads but tests ESMF threading features during the system tests. An ESMF library that was compiled with disabled Pthread features (via the ESMF_PTHREADS variable) will produce ESMF error messages during system test execution if the system tests were compiled with ESMF_TESTWITHTHREADS set to ON.

ESMF_TRACE_LIB_BUILD
Possible value: ON (default), OFF

This variables determines whether extra libraries are built that are used to add additional symbols to the ESMF tracing and profiling capability, such as MPI communication functions. If set to ON the libraries are built and placed into the ESMF_INSTALL_LIBDIR alongside the ESMF library itself.

ESMF_XERCES
See 9.4.6

ESMF_XERCES_INCLUDE
See 9.4.6

ESMF_XERCES_LIBPATH
See 9.4.6

ESMF_XERCES_LIBS
See 9.4.6

ESMF_YAMLCPP
See 9.4.7

ESMF_YAMLCPP_INCLUDE
See 9.4.7

ESMF_YAMLCPP_LIBPATH
See 9.4.7

ESMF_YAMLCPP_LIBS
See 9.4.7

Environment variables must be set in the user's shell or when calling gmake. It is not necessary to edit ESMF makefiles or other build system files to set these variables. Here is an example of setting an environment variable in the csh/tcsh shell:

  setenv ESMF_ABI 32

In bash/ksh shell environment variables are set this way:

  export ESMF_ABI=32

Environment variables can also be set from the gmake command line:

  gmake ESMF_ABI=32

9.6 Supported Platforms

The platforms that are tested and supported depend on the release of ESMF. To see the specific list of supported platforms, click the Supported Platforms link under one of the releases on the ESMF releases page.



Building the library for multiple architectures or options at the same time is supported; building or running the tests or examples is restricted to one platform/architecture at a time. The output from the test cases will be stored in a separate directories so the results will be kept separate for different architectures or options.


9.7 Building the ESMF Library

GNU make is required to build the library. On some systems this will be just the command make. On others it might be installed as gmake or even gnumake. In any event, use the -version option with the make command to determine if it is GNU make.

Build the library with the command:

  gmake

Makefiles throughout the framework are configured to allow users to compile files only in the directory where gmake is entered. Shared libraries are rebuilt only if necessary. In addition the entire ESMF framework may be built from any directory by entering gmake all, assuming that all the environmental variables are set correctly as described in Section 9.5.

The makefiles are also configured to allow multiple make targets to be compiled in parallel, via the gmake -j flag. For example, to use eight parallel processes to build the library, use -j8:

  gmake -j8 lib

The parallel compilation feature depends on ESMF_DEFER_LIB_BUILD=ON (the default) so that the library build will be deferred until all files have been compiled.

The -j option should only be used during the creation of the library. The test base and examples will not work correctly with -j set larger than 1.

Users may also run examples or execute unit tests of specific classes by changing directories to the desired class examples or tests directories and entering gmake run_examples or gmake run_unit_tests, respectively. For non-multiprocessor machines, uni-processor targets are available as gmake run_examples_uni or gmake run_unit_tests_uni.


9.8 Building the ESMF Documentation

The ESMF source documentation consists of an ESMF User's Guide and an ESMF Reference Manual for Fortran.

The tarballs on the ESMF website for ESMF versions 3.0.1 and later do not contain the ESMF documentation files. The documentation is available on the ESMF website in html or pdf form and most users should not need to build it from the source.

If a user does want to build the documentation, they will need to download the ESMF code repository (see section 5.1.1). Latex, latex2html, perl and csh must also be installed. For example, dependencies may be installed on Ubuntu Linux using:

  [sudo] apt-get install texlive latex2html perl csh

To build documentation:

  gmake doc              ! Builds the manuals, including pdf and html.

The resulting documentation files will be located in the top level directory $ESMF_DIR/doc


9.9 Installing the ESMF

The ESMF build system offers the standard install target to install all necessary files created during the build process into user specified locations. The installation procedure will also install the ESMF documentation if it has been built successfully following the procedure outlined above.

The installation location can be customized using six ESMF_ environment variables:

Section 9.5 describes what each of these environment variables does and how to set them.

Install ESMF with the command:

  gmake install

Check the ESMF installation with the command:

  gmake installcheck

Advice to installers. To complete the installation of ESMF, a single ESMF specific environment variable should be set. The variable is named ESMFMKFILE, and it must point to the esmf.mk file that was generated during the installation process. Systems that support multiple ESMF installations via management software (e.g. modules, softenv, ...) should set/reset the ESMFMKFILE environment variable as part of the configuration.

Additionally, it is typically convenient to append the user's PATH environment variable to provide access to the ESMF applications that were built during the installation process. The application binaries are located in the directory that was specified as ESMF_INSTALL_BINDIR during the ESMF installation. The location is also stored in variable ESMF_APPSDIR, defined in file esmf.mk. Systems that make ESMF installations available through management software (e.g. modules, softenv, ...) should modify the user's PATH environment variable as part of the configuration.

Hint. By default, file esmf.mk is located next to the ESMF library file in directory ESMF_INSTALL_LIBDIR. Consequently, unless esmf.mk has been moved to a different location after the installation, the correct setting for ESMFMKFILE is $(ESMF_INSTALL_LIBDIR)/esmf.mk.

Rationale. The only piece of information that is needed to use an ESMF installation is the exact location of the associated esmf.mk file. This file contains all of the relevant settings and flags that allow a user to build their application against the ESMF installation. Standardizing the mechanism by which the location of esmf.mk is made available to the user by the system will help users in the design of portable application build systems. (See sections 6 and 8 for details about the usage of esmf.mk.) Further, modifying the user's PATH environment variable is optional, since the location of the ESMF application binaries is available through the esmf.mk file. However, setting the user's PATH variable so that the ESMF applications are directly and conveniently accessible from the command line is recommended, especially if management software (e.g. modules, softenv, ...) is used on the system.

esmf_support@ucar.edu