ESMF superstructure classes define an architecture for assembling Earth system applications from modeling components. A component may be defined in terms of the physical domain that it represents, such as an atmosphere or sea ice model. It may also be defined in terms of a computational function, such as a data assimilation system. Earth system research often requires that such components be coupled together to create an application. By coupling we mean the data transformations and, on parallel computing systems, data transfers, that are necessary to allow data from one component to be utilized by another. ESMF offers regridding methods and other tools to simplify the organization and execution of inter-component data exchanges.
In addition to components defined at the level of major physical domains and computational functions, components may be defined that represent smaller computational functions within larger components, such as the transformation of data between the physics and dynamics in a spectral atmosphere model, or the creation of nested higher resolution regions within a coarser grid. The objective is to couple components at varying scales both flexibly and efficiently. ESMF encourages a hierarchical application structure, in which large components branch into smaller sub-components (see Figure 2). ESMF also makes it easier for the same component to be used in multiple contexts without changes to its source code.
Key Features |
Modular, component-based architecture. |
Hierarchical assembly of components into applications. |
Use of components in multiple contexts without modification. |
Sequential or concurrent component execution. |
Single program, multiple datastream (SPMD) applications for maximum portability and reconfigurability. |
Multiple program, multiple datastream (MPMD) option for flexibility. |
There are a small number of classes in the ESMF superstructure:
The second part of an ESMF Component is user code, such as a model or data assimilation system. Users set entry points within their code so that it is callable by the framework. In practice, setting entry points means that within user code there are calls to ESMF methods that associate the name of a Fortran subroutine with a corresponding standard ESMF operation. For example, a user-written initialization routine called myOceanInit might be associated with the standard initialize routine of an ESMF Gridded Component named “myOcean” that represents an ocean model.
An ESMF coupled application typically involves a parent Gridded Component, two or more child Gridded Components and one or more Coupler Components.
The parent Gridded Component is responsible for creating the child Gridded Components that are exchanging data, for creating the Coupler, for creating the necessary Import and Export States, and for setting up the desired sequencing. The application's “main” routine calls the parent Gridded Component's initialize, run, and finalize methods in order to execute the application. For each of these standard methods, the parent Gridded Component in turn calls the corresponding methods in the child Gridded Components and the Coupler Component. For example, consider a simple coupled ocean/atmosphere simulation. When the initialize method of the parent Gridded Component is called by the application, it in turn calls the initialize methods of its child atmosphere and ocean Gridded Components, and the initialize method of an ocean-to-atmosphere Coupler Component. Figure 3 shows this schematically.
Components are allocated computational resources in the form of Persistent Execution Threads, or PETs. A list of a Component's PETs is contained in a structure called a Virtual Machine, or VM. The VM also contains information about the topology and characteristics of the underlying computer. Components are created hierarchically, with parent Components creating child Components and allocating some or all of their PETs to each one. By default ESMF creates a new VM for each child Component, which allows Components to tailor their VM resources to match their needs. In some cases, a child may want to share its parent's VM - ESMF supports this, too.
A Gridded Component may exist across all the PETs in an application. A Gridded Component may also reside on a subset of PETs in an application. These PETs may wholly coincide with, be wholly contained within, or wholly contain another Component.
When a set of Gridded Components and a Coupler runs in sequence on the same set of PETs the application is executing in a sequential mode. When Gridded Components are created and run on mutually exclusive sets of PETs, and are coupled by a Coupler Component that extends over the union of these sets, the mode of execution is concurrent.
Figure 4 illustrates a typical configuration for a simple coupled sequential application, and Figure 5 shows a possible configuration for the same application running in a concurrent mode.
Parent Components can select if and when to wait for concurrently executing child Components, synchronizing only when required.
It is possible for ESMF applications to contain some Component sets that are executing sequentially and others that are executing concurrently. We might have, for example, atmosphere and land Components created on the same subset of PETs, ocean and sea ice Components created on the remainder of PETs, and a Coupler created across all the PETs in the application.
All data transfers within an ESMF application occur within a component. For example, a Gridded Component may contain halo updates. Another example is that a Coupler Component may redistribute data between two Gridded Components. As a result, the architecture of ESMF does not depend on any particular data communication mechanism, and new communication schemes can be introduced without affecting the overall structure of the application.
Since all data communication happens within a component, a Coupler Component must be created on the union of the PETs of all the Gridded Components that it couples.
The scope of distributed objects is the VM of the currently executing Component. For this reason, all PETs in the current VM must make the same distributed object creation calls. When a Coupler Component running on a superset of a Gridded Component's PETs needs to make communication calls involving objects created by the Gridded Component, an ESMF-supplied function called ESMF_StateReconcile() creates proxy objects for those PETs that had no previous information about the distributed objects. Proxy objects contain no local data but can be used in communication calls (such as regrid or redistribute) to describe the remote source for data being moved to the current PET, or to describe the remote destination for data being moved from the local PET. Figure 6 is a simple schematic that shows the sequence of events in a reconcile call.
The ESMF design enables the user to configure ESMF applications so that data is transferred directly from one component to another, without requiring that it be copied or sent to a different data buffer as an interim step. This is likely to be the most efficient way of performing inter-component coupling. However, if desired, an application can also be configured so that data from a source component is sent to a distinct set of Coupler Component PETs for processing before being sent to its destination.
The ability to overlap computation with communication is essential for performance. When running with ESMF the user can initiate data sends during Gridded Component execution, as soon as the data is ready. Computations can then proceed simultaneously with the data transfer.
The following is a simplified Unified Modeling Language (UML) diagram showing the relationships among ESMF superstructure classes. See Appendix A, A Brief Introduction to UML, for a translation table that lists the symbols in the diagram and their meaning.
Every ESMF application needs a driver code. Typically the driver layer is implemented as the "main" of the application, although this is not strictly an ESMF requirement. For most ESMF applications the task of the application driver will be very generic: Initialize ESMF, create a top-level Component and call its Initialize, Run and Finalize methods, before destroying the top-level Component again and calling ESMF Finalize.
ESMF provides a number of different application driver templates in the $ESMF_DIR/src/Superstructure/AppDriver directory. An appropriate one can be chosen depending on how the application is to be structured:
In a sequential execution model, every Component executes on all PETs, with each Component completing execution before the next Component begins. This has the appeal of simplicity of data consumption and production: when a Gridded Component starts, all required data is available for use, and when a Gridded Component finishes, all data produced is ready for consumption by the next Gridded Component. This approach also has the possibility of less data movement if the grid and data decomposition is done such that each processor's memory contains the data needed by the next Component.
In a concurrent execution model, subgroups of PETs run Gridded Components and multiple Gridded Components are active at the same time. Data exchange must be coordinated between Gridded Components so that data deadlock does not occur. This strategy has the advantage of allowing coupling to other Gridded Components at any time during the computational process, including not having to return to the calling level of code before making data available.
Coupler Components are responsible for taking data from one Gridded Component and putting it into the form expected by another Gridded Component. This might include regridding, change of units, averaging, or binning.
Coupler Components can be written for pairwise data exchange: the Coupler Component takes data from a single Component and transforms it for use by another single Gridded Component. This simplifies the structure of the Coupler Component code.
Couplers can also be written using a hub and spoke model where a single Coupler accepts data from all other Components, can do data merging or splitting, and formats data for all other Components.
Multiple Couplers, using either of the above two models or some mixture of these approaches, are also possible.
The ESMF framework currently has Fortran interfaces for all public functions. Some functions also have C interfaces, and the number of these is expected to increase over time.
The simplest way to run an application is to run the same executable program on all PETs. Different Components can still be run on mutually exclusive PETs by using branching (e.g., if this is PET 1, 2, or 3, run Component A, if it is PET 4, 5, or 6 run Component B). This is a SPMD model, Single Program Multiple Data.
The alternative is to start a different executable program on different PETs. This is a MPMD model, Multiple Program Multiple Data. There are complications with many job control systems on multiprocessor machines in getting the different executables started, and getting inter-process communications established. ESMF currently has some support for MPMD: different Components can run as separate executables, but the Coupler that transfers data between the Components must still run on the union of their PETs. This means that the Coupler Component must be linked into all of the executables.
DESCRIPTION:
The ESMF_End_Flag determines how an ESMF application is shut down.
The type of this flag is:
type(ESMF_End_Flag)
The valid values are:
ESMF encourages application organization in which there is a single top-level Gridded Component. This provides a simple, clear sequence of operations at the highest level, and also enables the entire application to be treated as a sub-Component of another, larger application if desired. When a simple application is organized in this fashion the standard AppDriver can probably be used without much modification.
Examples of program organization using the AppDriver can be found in the src/Superstructure/AppDriver directory. A set of subdirectories within the AppDriver directory follows the naming convention:
<seq|concur>_<pairwise|hub>_<f|c>driver_<spmd|mpmd>
The example that is currently implemented is seq_pairwise_fdriver_spmd, which has sequential component execution, a pairwise coupler, a main program in Fortran, and all processors launching the same executable. It is also copied automatically into a top-level quick_start directory at compilation time.
The user can copy the AppDriver files into their own local directory. Some of the files can be used unchanged. Others are template files which have the rough outline of the code but need additional application-specific code added in order to perform a meaningful function. The README file in the AppDriver subdirectory or quick_start directory contains instructions about which files to change.
Examples of concurrent component execution can be found in the system tests that are bundled with the ESMF distribution.
--------------------------------------------------------------------------- --------------------------------------------------------------------------- EXAMPLE: This is an AppDriver.F90 file for a sequential ESMF application. --------------------------------------------------------------------------- --------------------------------------------------------------------------- The ChangeMe.F90 file that's included below contains a number of definitions that are used by the AppDriver, such as the name of the application's main configuration file and the name of the application's SetServices routine. This file is in the same directory as the AppDriver.F90 file. --------------------------------------------------------------------------- #include "ChangeMe.F90" program ESMF_AppDriver #define ESMF_METHOD "program ESMF_AppDriver" #include "ESMF.h" ! ESMF module, defines all ESMF data types and procedures use ESMF ! Gridded Component registration routines. Defined in "ChangeMe.F90" use USER_APP_Mod, only : SetServices => USER_APP_SetServices implicit none --------------------------------------------------------------------------- Define local variables --------------------------------------------------------------------------- ! Components and States type(ESMF_GridComp) :: compGridded type(ESMF_State) :: defaultstate ! Configuration information type(ESMF_Config) :: config ! A common Grid type(ESMF_Grid) :: grid ! A Clock, a Calendar, and timesteps type(ESMF_Clock) :: clock type(ESMF_TimeInterval) :: timeStep type(ESMF_Time) :: startTime type(ESMF_Time) :: stopTime ! Variables related to the Grid integer :: i_max, j_max ! Return codes for error checks integer :: rc, localrc --------------------------------------------------------------------------- Initialize ESMF. Note that an output Log is created by default. --------------------------------------------------------------------------- call ESMF_Initialize(defaultCalKind=ESMF_CALKIND_GREGORIAN, rc=localrc) if (ESMF_LogFoundError(localrc, ESMF_ERR_PASSTHRU, & ESMF_CONTEXT, rcToReturn=rc)) & call ESMF_Finalize(rc=localrc, endflag=ESMF_END_ABORT) call ESMF_LogWrite("ESMF AppDriver start", ESMF_LOGMSG_INFO) --------------------------------------------------------------------------- Create and load a configuration file. The USER_CONFIG_FILE is set to sample.rc in the ChangeMe.F90 file. The sample.rc file is also included in the directory with the AppDriver.F90 file. --------------------------------------------------------------------------- config = ESMF_ConfigCreate(rc=localrc) if (ESMF_LogFoundError(localrc, ESMF_ERR_PASSTHRU, & ESMF_CONTEXT, rcToReturn=rc)) & call ESMF_Finalize(rc=localrc, endflag=ESMF_END_ABORT) call ESMF_ConfigLoadFile(config, USER_CONFIG_FILE, rc = localrc) if (ESMF_LogFoundError(localrc, ESMF_ERR_PASSTHRU, & ESMF_CONTEXT, rcToReturn=rc)) & call ESMF_Finalize(rc=localrc, endflag=ESMF_END_ABORT) --------------------------------------------------------------------------- Get configuration information. A configuration file like sample.rc might include: - size and coordinate information needed to create the default Grid. - the default start time, stop time, and running intervals for the main time loop. --------------------------------------------------------------------------- call ESMF_ConfigGetAttribute(config, i_max, label='I Counts:', & default=10, rc=localrc) if (ESMF_LogFoundError(localrc, ESMF_ERR_PASSTHRU, & ESMF_CONTEXT, rcToReturn=rc)) & call ESMF_Finalize(rc=localrc, endflag=ESMF_END_ABORT) call ESMF_ConfigGetAttribute(config, j_max, label='J Counts:', & default=40, rc=localrc) if (ESMF_LogFoundError(localrc, ESMF_ERR_PASSTHRU, & ESMF_CONTEXT, rcToReturn=rc)) & call ESMF_Finalize(rc=localrc, endflag=ESMF_END_ABORT) --------------------------------------------------------------------------- Create the top Gridded Component. --------------------------------------------------------------------------- compGridded = ESMF_GridCompCreate(name="ESMF Gridded Component", & rc=localrc) if (ESMF_LogFoundError(localrc, ESMF_ERR_PASSTHRU, & ESMF_CONTEXT, rcToReturn=rc)) & call ESMF_Finalize(rc=localrc, endflag=ESMF_END_ABORT) call ESMF_LogWrite("Component Create finished", ESMF_LOGMSG_INFO) ---------------------------------------------------------------------------- Register the set services method for the top Gridded Component. ---------------------------------------------------------------------------- call ESMF_GridCompSetServices(compGridded, userRoutine=SetServices, rc=rc) if (ESMF_LogFoundError(rc, msg="Registration failed", rcToReturn=rc)) & call ESMF_Finalize(rc=localrc, endflag=ESMF_END_ABORT) ---------------------------------------------------------------------------- Create and initialize a Clock. ---------------------------------------------------------------------------- call ESMF_TimeIntervalSet(timeStep, s=2, rc=localrc) if (ESMF_LogFoundError(localrc, ESMF_ERR_PASSTHRU, & ESMF_CONTEXT, rcToReturn=rc)) & call ESMF_Finalize(rc=localrc, endflag=ESMF_END_ABORT) call ESMF_TimeSet(startTime, yy=2004, mm=9, dd=25, rc=localrc) if (ESMF_LogFoundError(localrc, ESMF_ERR_PASSTHRU, & ESMF_CONTEXT, rcToReturn=rc)) & call ESMF_Finalize(rc=localrc, endflag=ESMF_END_ABORT) call ESMF_TimeSet(stopTime, yy=2004, mm=9, dd=26, rc=localrc) if (ESMF_LogFoundError(localrc, ESMF_ERR_PASSTHRU, & ESMF_CONTEXT, rcToReturn=rc)) & call ESMF_Finalize(rc=localrc, endflag=ESMF_END_ABORT) clock = ESMF_ClockCreate(timeStep, startTime, stopTime=stopTime, & name="Application Clock", rc=localrc) if (ESMF_LogFoundError(localrc, ESMF_ERR_PASSTHRU, & ESMF_CONTEXT, rcToReturn=rc)) & call ESMF_Finalize(rc=localrc, endflag=ESMF_END_ABORT) ---------------------------------------------------------------------------- Create and initialize a Grid. The default lower indices for the Grid are (/1,1/). The upper indices for the Grid are read in from the sample.rc file, where they are set to (/10,40/). This means a Grid will be created with 10 grid cells in the x direction and 40 grid cells in the y direction. The Grid section in the Reference Manual shows how to set coordinates. ---------------------------------------------------------------------------- grid = ESMF_GridCreateNoPeriDim(maxIndex=(/i_max, j_max/), & name="source grid", rc=localrc) if (ESMF_LogFoundError(localrc, ESMF_ERR_PASSTHRU, & ESMF_CONTEXT, rcToReturn=rc)) & call ESMF_Finalize(rc=localrc, endflag=ESMF_END_ABORT) ! Attach the grid to the Component call ESMF_GridCompSet(compGridded, grid=grid, rc=localrc) if (ESMF_LogFoundError(localrc, ESMF_ERR_PASSTHRU, & ESMF_CONTEXT, rcToReturn=rc)) & call ESMF_Finalize(rc=localrc, endflag=ESMF_END_ABORT) ---------------------------------------------------------------------------- Create and initialize a State to use for both import and export. In a real code, separate import and export States would normally be created. ---------------------------------------------------------------------------- defaultstate = ESMF_StateCreate(name="Default State", rc=localrc) if (ESMF_LogFoundError(localrc, ESMF_ERR_PASSTHRU, & ESMF_CONTEXT, rcToReturn=rc)) & call ESMF_Finalize(rc=localrc, endflag=ESMF_END_ABORT) ---------------------------------------------------------------------------- Call the initialize, run, and finalize methods of the top component. When the initialize method of the top component is called, it will in turn call the initialize methods of all its child components, they will initialize their children, and so on. The same is true of the run and finalize methods. ---------------------------------------------------------------------------- call ESMF_GridCompInitialize(compGridded, importState=defaultstate, & exportState=defaultstate, clock=clock, rc=localrc) if (ESMF_LogFoundError(rc, msg="Initialize failed", rcToReturn=rc)) & call ESMF_Finalize(rc=localrc, endflag=ESMF_END_ABORT) call ESMF_GridCompRun(compGridded, importState=defaultstate, & exportState=defaultstate, clock=clock, rc=localrc) if (ESMF_LogFoundError(rc, msg="Run failed", rcToReturn=rc)) & call ESMF_Finalize(rc=localrc, endflag=ESMF_END_ABORT) call ESMF_GridCompFinalize(compGridded, importState=defaultstate, & exportState=defaultstate, clock=clock, rc=localrc) if (ESMF_LogFoundError(rc, msg="Finalize failed", rcToReturn=rc)) & call ESMF_Finalize(rc=localrc, endflag=ESMF_END_ABORT) ---------------------------------------------------------------------------- Destroy objects. ---------------------------------------------------------------------------- call ESMF_ClockDestroy(clock, rc=localrc) if (ESMF_LogFoundError(localrc, ESMF_ERR_PASSTHRU, & ESMF_CONTEXT, rcToReturn=rc)) & call ESMF_Finalize(rc=localrc, endflag=ESMF_END_ABORT) call ESMF_StateDestroy(defaultstate, rc=localrc) if (ESMF_LogFoundError(localrc, ESMF_ERR_PASSTHRU, & ESMF_CONTEXT, rcToReturn=rc)) & call ESMF_Finalize(rc=localrc, endflag=ESMF_END_ABORT) call ESMF_GridCompDestroy(compGridded, rc=localrc) if (ESMF_LogFoundError(localrc, ESMF_ERR_PASSTHRU, & ESMF_CONTEXT, rcToReturn=rc)) & call ESMF_Finalize(rc=localrc, endflag=ESMF_END_ABORT) ---------------------------------------------------------------------------- Finalize and clean up. ---------------------------------------------------------------------------- call ESMF_Finalize() end program ESMF_AppDriver
There are a few methods that every ESMF application must contain. First, ESMF_Initialize() and ESMF_Finalize() are in complete analogy to MPI_Init() and MPI_Finalize() known from MPI. All ESMF programs, serial or parallel, must initialize the ESMF system at the beginning, and finalize it at the end of execution. The behavior of calling any ESMF method before ESMF_Initialize(), or after ESMF_Finalize() is undefined.
Second, every ESMF Component that is accessed by an ESMF application requires that its set services routine is called through ESMF_<Grid/Cpl>CompSetServices(). The Component must implement one public entry point, its set services routine, that can be called through the ESMF_<Grid/Cpl>CompSetServices() library routine. The Component set services routine is responsible for setting entry points for the standard ESMF Component methods Initialize, Run, and Finalize.
Finally, the Component can optionally call ESMF_<Grid/Cpl>CompSetVM() before calling ESMF_<Grid/Cpl>CompSetServices(). Similar to ESMF_<Grid/Cpl>CompSetServices(), the ESMF_<Grid/Cpl>CompSetVM() call requires a public entry point into the Component. It allows the Component to adjust certain aspects of its execution environment, i.e. its own VM, before it is started up.
The following sections discuss the above mentioned aspects in more detail.
INTERFACE:
subroutine ESMF_Initialize(configFilename, & defaultCalKind, defaultDefaultLogFilename, defaultLogFilename, & defaultLogAppendFlag, logAppendFlag, defaultLogKindFlag, logKindFlag, & mpiCommunicator, ioUnitLBound, ioUnitUBound, & defaultGlobalResourceControl, globalResourceControl, config, vm, rc)ARGUMENTS:
-- The following arguments require argument keyword syntax (e.g. rc=rc). -- character(len=*), intent(in), optional :: configFilename type(ESMF_CalKind_Flag), intent(in), optional :: defaultCalKind character(len=*), intent(in), optional :: defaultDefaultLogFilename character(len=*), intent(in), optional :: defaultLogFilename logical, intent(in), optional :: defaultLogAppendFlag logical, intent(in), optional :: logAppendFlag type(ESMF_LogKind_Flag), intent(in), optional :: defaultLogKindFlag type(ESMF_LogKind_Flag), intent(in), optional :: logKindFlag integer, intent(in), optional :: mpiCommunicator integer, intent(in), optional :: ioUnitLBound integer, intent(in), optional :: ioUnitUBound logical, intent(in), optional :: defaultGlobalResourceControl logical, intent(in), optional :: globalResourceControl type(ESMF_Config), intent(out), optional :: config type(ESMF_VM), intent(out), optional :: vm integer, intent(out), optional :: rcSTATUS:
DESCRIPTION:
This method must be called once on each PET before any other ESMF methods are used. The method contains a barrier before returning, ensuring that all processes made it successfully through initialization.
Typically ESMF_Initialize() will call MPI_Init() internally unless MPI has been initialized by the user code before initializing the framework. If the MPI initialization is left to ESMF_Initialize() it inherits all of the MPI implementation dependent limitations of what may or may not be done before MPI_Init(). For instance, it is unsafe for some MPI implementations, such as MPICH1, to do I/O before the MPI environment is initialized. Please consult the documentation of your MPI implementation for details.
Note that when using MPICH1 as the MPI library, ESMF needs to use the application command line arguments for MPI_Init(). However, ESMF acquires these arguments internally and the user does not need to worry about providing them. Also, note that ESMF does not alter the command line arguments, so that if the user obtains them they will be as specified on the command line (including those which MPICH1 would normally strip out).
ESMF_Initialize() supports running ESMF inside a user MPI program. Details of this feature are discussed under the VM example 49.3.4. It is not necessary that all MPI ranks are handed to ESMF. Section 49.3.5 shows how an MPI communicator can be used to execute ESMF on a subset of MPI ranks. ESMF_Initialize() supports running multiple concurrent instances of ESMF under the same user MPI program. This feature is discussed under 49.3.6.
In order to use any of the advanced resource management functions that ESMF provides via the ESMF_*CompSetVM*() methods, the MPI environment must be thread-safe. ESMF_Initialize() handles this automatically if it is in charge of initializing MPI. However, if the user code initializes MPI before calling into ESMF_Initialize(), it must do so via MPI_Init_thread(), specifying MPI_THREAD_SERIALIZED or above for the required level of thread support.
In cases where ESMF_*CompSetVM*() methods are used to move processing elements (PEs), i.e. CPU cores, between persistent execution threads (PETs), ESMF uses POSIX signals between PETs. In order to do so safely, the proper signal handlers must be installed before MPI is initialized. ESMF_Initialize() handles this automatically if it is in charge of initializing MPI. If, however, MPI is explicitly initialized by user code, then to ensure correct signal handling it is necessary to call ESMF_InitializePreMPI() from the user code prior to the MPI initialization.
By default, ESMF_Initialize() will open multiple error log files, one per processor. This is very useful for debugging purpose. However, when running the application on a large number of processors, opening a large number of log files and writing log messages from all the processors could become a performance bottleneck. Therefore, it is recommended to turn the Error Log feature off in these situations by setting logKindFlag to ESMF_LOGKIND_NONE.
When integrating ESMF with applications where Fortran unit number conflicts exist, the optional ioUnitLBound and ioUnitUBound arguments may be used to specify an alternate unit number range. See section 51.2.1 for more information on how ESMF uses Fortran unit numbers.
Before exiting the application the user must call ESMF_Finalize() to release resources and clean up ESMF gracefully. See the ESMF_Finalize() documentation about details relating to the MPI environment.
The arguments are:
INTERFACE:
subroutine ESMF_InitializePreMPI(rc)ARGUMENTS:
-- The following arguments require argument keyword syntax (e.g. rc=rc). -- integer, intent(out), optional :: rcDESCRIPTION:
This method is only needed for cases where MPI is initialized explicitly by user code. In most typical cases ESMF_Initialize() is called before MPI is initialized, and takes care of all the internal initialization, including MPI.
There are circumstances where it is necessary or convenient to initialize MPI before calling into ESMF_Initialize(). This option is supported by ESMF, and for most cases no special action is required on the user side. However, for cases where ESMF_*CompSetVM*() methods are used to move processing elements (PEs), i.e. CPU cores, between persistent execution threads (PETs), ESMF uses POSIX signals between PETs. In order to do so safely, the proper signal handlers must be installed before MPI is initialized. This is accomplished by calling ESMF_InitializePreMPI() from the user code prior to the MPI initialization.
Note also that in order to use any of the advanced resource management functions that ESMF provides via the ESMF_*CompSetVM*() methods, the MPI environment must be thread-safe. ESMF_Initialize() handles this automatically if it is in charge of initializing MPI. However, if the user code initializes MPI before calling into ESMF_Initialize(), it must do so via MPI_Init_thread(), specifying MPI_THREAD_SERIALIZED or above for the required level of thread support.
The arguments are:
INTERFACE:
function ESMF_IsInitialized(rc)RETURN VALUE:
logical :: ESMF_IsInitializedARGUMENTS:
-- The following arguments require argument keyword syntax (e.g. rc=rc). -- integer, intent(out), optional :: rcDESCRIPTION:
Returns .true. if the framework has been initialized. This means that ESMF_Initialize() has been called. Otherwise returns .false.. If an error occurs, i.e. rc /= ESMF_SUCCESS is returned, the return value of the function will also be .false..
The arguments are:
INTERFACE:
function ESMF_IsFinalized(rc)RETURN VALUE:
logical :: ESMF_IsFinalizedARGUMENTS:
-- The following arguments require argument keyword syntax (e.g. rc=rc). -- integer, intent(out), optional :: rcDESCRIPTION:
Returns .true. if the framework has been finalized. This means that ESMF_Finalize() has been called. Otherwise returns .false.. If an error occurs, i.e. rc /= ESMF_SUCCESS is returned, the return value of the function will also be .false..
The arguments are:
INTERFACE:
subroutine ESMF_Finalize(endflag, rc)ARGUMENTS:
-- The following arguments require argument keyword syntax (e.g. rc=rc). -- type(ESMF_End_Flag), intent(in), optional :: endflag integer, intent(out), optional :: rcSTATUS:
DESCRIPTION:
This must be called once on each PET before the application exits to allow ESMF to flush buffers, close open connections, and release internal resources cleanly. The optional argument endflag may be used to indicate the mode of termination. Note that this call must be issued only once per PET with endflag=ESMF_END_NORMAL, and that this call may not be followed by ESMF_Initialize(). This last restriction means that it is not possible to restart ESMF within the same execution.
The arguments are:
Many programs call some library routines. The library documentation must explain what the routine name is, what arguments are required and what are optional, and what the code does.
In contrast, all ESMF components must be written to be called by another part of the program; in effect, an ESMF component takes the place of a library. The interface is prescribed by the framework, and the component writer must provide specific subroutines which have standard argument lists and perform specific operations. For technical reasons none of the arguments in user-provided subroutines must be declared as optional.
The only required public interface of a Component is its SetServices method. This subroutine must have an externally accessible name (be a public symbol), take a component as the first argument, and an integer return code as the second. Both arguments are required and must not be declared as optional. If an intent is specified in the interface it must be intent(inout) for the first and intent(out) for the second argument. The subroutine name is not predefined, it is set by the component writer, but must be provided as part of the component documentation.
The required function that the SetServices subroutine must provide is to specify the user-code entry points for the standard ESMF Component methods. To this end the user-written SetServices routine calls the
ESMF_<Grid/Cpl>CompSetEntryPoint() method to set each Component entry point.
See sections 17.2.1 and 18.2.1 for examples of how to write a user-code SetServices routine.
Note that a component does not call its own SetServices routine; the AppDriver or parent component code, which is creating a component, will first call ESMF_<Grid/Cpl>CompCreate() to create a component object, and then must call into ESMF_<Grid/Cpl>CompSetServices(), supplying the user-code SetServices routine as an argument. The framework then calls into the user-code SetServices, after the Component's VM has been started up.
It is good practice to package the user-code implementing a component into a Fortran module, with the user-code SetService routine being the only public module method. ESMF supports three mechanisms for accessing the user-code SetServices routine from the calling AppDriver or parent component.
Pros: Standard Fortran module use: name mangling and interface checking is handled by the Fortran compiler.
Cons: Fortran 90/95 has no mechanism to implement a "smart" dependency scheme through USE association. Any change in a lower level component module (even just adding or changing a comment!) will trigger a complete recompilation of all of the higher level components throughout the component hierarchy. This situation is particularly annoying for ESMF componentized code, where the prescribed ESMF component interfaces, in principle, remove all interdependencies between components that would require recompilation.
Fortran submodules, introduced as an extension to Fortran 2003, and now part for the Fortran 2008 standard, are designed to avoid this "false" dependency issue. A code change to an ESMF component that keeps the actual implementation within a submodule, will not trigger a recompilation of the components further up in the component hierarchy. Unfortunately, as of mid-2015, only two compiler vendors support submodules.
Pros: Avoids Fortran USE dependencies: a change to lower level component code will not trigger a complete recompilation of all of the higher level components throughout the component hierarchy. Name mangling is handled by the Fortran compiler.
Cons: The user-code SetServices interface is not checked by the compiler. The user must ensure uniqueness of the external routine name across the entire application.
Pros: Avoids Fortran USE dependencies: a change to lower level component code will not trigger a complete recompilation of all of the higher level components throughout the component hierarchy. The component code does not have to be accessible until runtime and may be located in a shared object, thus avoiding relinking of the application.
Cons: The user-code SetServices interface is not checked by the compiler. The user must explicitly deal with all of the Fortran name mangling issues: 1) Accessing a module routine requires precise knowledge of the name mangling rules of the specific compiler. Alternatively, the user-code SetServices routine may be implemented as an external routine, avoiding the module name mangling. 2) Even then, Fortran compilers typically append one or two underscores on a symbol name. This must be considered when passing the name into the ESMF_<Grid/Cpl>CompSetServices() method.
The required standard ESMF Component methods, for which user-code entry points must be set, are Initialize, Run, and Finalize. Currently optional, a Component may also set entry points for the WriteRestart and ReadRestart methods.
Sections 17.2.1 and 18.2.1 provide examples of how the entry points for Initialize, Run, and Finalize are set during the user-code SetServices routine, using the ESMF_<Grid/Cpl>CompSetEntryPoint() library call.
All standard user-code methods must abide exactly to the prescribed interfaces. None of the arguments must be declared as optional.
The names of the Initialize, Run, and Finalize user-code subroutines do not need to be public; in fact it is far better for them to be private to lower the chances of public symbol clashes between different components.
See sections 17.2.2, 17.2.3, 17.2.4, and 18.2.2, 18.2.3, 18.2.4 for examples of how to write entry points for the standard ESMF Component methods.
When the AppDriver or parent component code calls ESMF_<Grid/Cpl>CompCreate() it has the option to specify a petList argument. All of the parent PETs contained in this list become resources of the child component. By default, without the petList argument, all of the parent PETs are provided to the child component.
Typically each component has its own virtual machine (VM) object. However, using the optional contextflag argument during ESMF_<Grid/Cpl>CompCreate() a child component can inherit its parent component's VM. Unless a child component inherits the parent VM, it has the option to set certain aspects of how its VM utilizes the provided resources. The resources provided via the parent PETs are the associated processing elements (PEs) and virtual address spaces (VASs).
The optional user-written SetVM routine is called from the parent for the child through the ESMF_<Grid/Cpl>CompSetVM() method. This is the only place where the child component can set aspects of its own VM before it is started up. The child component's VM must be running before the SetServices routine can be called, and thus the parent must call the optional ESMF_<Grid/Cpl>CompSetVM() method before ESMF_<Grid/Cpl>CompSetServices().
Inside the user-code called by the SetVM routine, the component has the option to specify how the PETs share the provided parent PEs. Further, PETs on the same single system image (SSI) can be set to run multi-threaded within a reduced number of virtual address spaces (VAS), allowing a component to leverage shared memory concepts.
Sections 17.2.5 and 18.2.5 provide examples for simple user-written SetVM routines.
One common use of the SetVM approach is to implement hybrid parallelism based on MPI+OpenMP. Under ESMF, each component can use its own hybrid parallelism implementation. Different components, even if running on the same PE resources, do not have to agree on the number of MPI processes (i.e. PETs), or the number of OpenMP threads launched under each PET. Hybrid and non-hybrid components can be mixed within the same application. Coupling between components of any flavor is supported under ESMF.
In order to obtain best performance when using SetVM based resource control for hybrid parallelism, it is strongly recommended to set OMP_WAIT_POLICY=PASSIVE in the environment. This is one of the standard OpenMP environment variables. The PASSIVE setting ensures that OpenMP threads relinquish the PEs as soon as they have completed their work. Without that setting ESMF resource control threads can be delayed, and context switching between components becomes more expensive.
Internal procedures are nested within a surrounding procedure, and only local to the surrounding procedure. They are specified by using the CONTAINS statement.
Prior to Fortran-2008 an internal procedure could not be used as a user-provided callback procedure. In Fortran-2008 this restriction was lifted. It is important to note that if ESMF is passed an internal procedure, that the surrounding procedure be active whenever ESMF calls it. This helps ensure that local variables at the surrounding procedures scope are properly initialized.
When internal procedures contained within a main program unit are used for callbacks, there is no problem. This is because the main program unit is always active. However when internal procedures are used within other program units, initialization could become a problem. The following outlines the issue:
module my_procs_mod use ESMF implicit none contains subroutine my_procs (...) integer :: my_setting : call ESMF_GridCompSetEntryPoint(gridcomp, methodflag=ESMF_METHOD_INITIALIZE, & userRoutine=my_grid_proc_init, rc=localrc) : my_setting = 42 contains subroutine my_grid_proc_init (gridcomp, importState, exportState, clock, rc) : ! my_setting is possibly uninitialized when my_grid_proc_init is used as a call-back something = my_setting : end subroutine my_grid_proc_init end subroutine my_procs end module my_procs_mod
The Fortran standard does not specify whether variable my_setting is statically or automatically allocated, unless it is explicitly given the SAVE attribute. Thus there is no guarantee that its value will persist after my_procs has finished. The SAVE attribute is usually given to a variable via specifying a SAVE attribute in its delaration. However it can also be inferred by initializing the variable in its declaration:
: integer, save : my_setting :
or,
: integer :: my_setting = 42 :
Because of the potential initialization issues, it is recommended that internal procedures only be used as ESMF callbacks when the surrounding procedure is also active.
In Earth system modeling, the most natural way to think about an ESMF Gridded Component, or ESMF_GridComp, is as a piece of code representing a particular physical domain, such as an atmospheric model or an ocean model. Gridded Components may also represent individual processes, such as radiation or chemistry. It's up to the application writer to decide how deeply to “componentize.”
Earth system software components tend to share a number of basic features. Most ingest and produce a variety of physical fields, refer to a (possibly noncontiguous) spatial region and a grid that is partitioned across a set of computational resources, and require a clock for things like stepping a governing set of PDEs forward in time. Most can also be divided into distinct initialize, run, and finalize computational phases. These common characteristics are used within ESMF to define a Gridded Component data structure that is tailored for Earth system modeling and yet is still flexible enough to represent a variety of domains.
A well designed Gridded Component does not store information internally about how it couples to other Gridded Components. That allows it to be used in different contexts without changes to source code. The idea here is to avoid situations in which slightly different versions of the same model source are maintained for use in different contexts - standalone vs. coupled versions, for example. Data is passed in and out of Gridded Components using an ESMF State, this is described in Section 21.1.
An ESMF Gridded Component has two parts, one which is user-written and another which is part of the framework. The user-written part is software that represents a physical domain or performs some other computational function. It forms the body of the Gridded Component. It may be a piece of legacy code, or it may be developed expressly for use with ESMF. It must contain routines with standard ESMF interfaces that can be called to initialize, run, and finalize the Gridded Component. These routines can have separate callable phases, such as distinct first and second initialization steps.
ESMF provides the Gridded Component derived type, ESMF_GridComp. An ESMF_GridComp must be created for every portion of the application that will be represented as a separate component. For example, in a climate model, there may be Gridded Components representing the land, ocean, sea ice, and atmosphere. If the application contains an ensemble of identical Gridded Components, every one has its own associated ESMF_GridComp. Each Gridded Component has its own name and is allocated a set of computational resources, in the form of an ESMF Virtual Machine, or VM.
The user-written part of a Gridded Component is associated with an ESMF_GridComp derived type through a routine called ESMF_SetServices(). This is a routine that the user must write, and declare public. Inside the SetServices routine the user must call ESMF_SetEntryPoint() methods that associate a standard ESMF operation with the name of the corresponding Fortran subroutine in their user code.
A Gridded Component is a computational entity which consumes and produces data. It uses a State object to exchange data between itself and other Components. It uses a Clock object to manage time, and a VM to describe its own and its child components' computational resources.
This section shows how to create Gridded Components. For demonstrations of the use of Gridded Components, see the system tests that are bundled with the ESMF software distribution. These can be found in the directory esmf/src/system_tests.
Every ESMF_GridComp is required to provide and document a public set services routine. It can have any name, but must follow the declaration below: a subroutine which takes an ESMF_GridComp as the first argument, and an integer return code as the second. Both arguments are required and must not be declared as optional. If an intent is specified in the interface it must be intent(inout) for the first and intent(out) for the second argument.
The set services routine must call the ESMF method ESMF_GridCompSetEntryPoint() to register with the framework what user-code subroutines should be called to initialize, run, and finalize the component. There are additional routines which can be registered as well, for checkpoint and restart functions.
Note that the actual subroutines being registered do not have to be public to this module; only the set services routine itself must be available to be used by other code.
! Example Gridded Component module ESMF_GriddedCompEx ! ESMF Framework module use ESMF implicit none public GComp_SetServices public GComp_SetVM contains subroutine GComp_SetServices(comp, rc) type(ESMF_GridComp) :: comp ! must not be optional integer, intent(out) :: rc ! must not be optional ! Set the entry points for standard ESMF Component methods call ESMF_GridCompSetEntryPoint(comp, ESMF_METHOD_INITIALIZE, & userRoutine=GComp_Init, rc=rc) call ESMF_GridCompSetEntryPoint(comp, ESMF_METHOD_RUN, & userRoutine=GComp_Run, rc=rc) call ESMF_GridCompSetEntryPoint(comp, ESMF_METHOD_FINALIZE, & userRoutine=GComp_Final, rc=rc) rc = ESMF_SUCCESS end subroutine
When a higher level component is ready to begin using an ESMF_GridComp, it will call its initialize routine.
The component writer must supply a subroutine with the exact interface shown below. Arguments must not be declared as optional, and the types and order must match.
At initialization time the component can allocate data space, open data files, set up initial conditions; anything it needs to do to prepare to run.
The rc return code should be set if an error occurs, otherwise the value ESMF_SUCCESS should be returned.
subroutine GComp_Init(comp, importState, exportState, clock, rc) type(ESMF_GridComp) :: comp ! must not be optional type(ESMF_State) :: importState ! must not be optional type(ESMF_State) :: exportState ! must not be optional type(ESMF_Clock) :: clock ! must not be optional integer, intent(out) :: rc ! must not be optional print *, "Gridded Comp Init starting" ! This is where the model specific setup code goes. ! If the initial Export state needs to be filled, do it here. !call ESMF_StateAdd(exportState, field, rc) !call ESMF_StateAdd(exportState, bundle, rc) print *, "Gridded Comp Init returning" rc = ESMF_SUCCESS end subroutine GComp_Init
During the execution loop, the run routine may be called many times. Each time it should read data from the importState, use the clock to determine what the current time is in the calling component, compute new values or process the data, and produce any output and place it in the exportState.
When a higher level component is ready to use the ESMF_GridComp it will call its run routine.
The component writer must supply a subroutine with the exact interface shown below. Arguments must not be declared as optional, and the types and order must match.
It is expected that this is where the bulk of the model computation or data analysis will occur.
The rc return code should be set if an error occurs, otherwise the value ESMF_SUCCESS should be returned.
subroutine GComp_Run(comp, importState, exportState, clock, rc) type(ESMF_GridComp) :: comp ! must not be optional type(ESMF_State) :: importState ! must not be optional type(ESMF_State) :: exportState ! must not be optional type(ESMF_Clock) :: clock ! must not be optional integer, intent(out) :: rc ! must not be optional print *, "Gridded Comp Run starting" ! call ESMF_StateGet(), etc to get fields, bundles, arrays ! from import state. ! This is where the model specific computation goes. ! Fill export state here using ESMF_StateAdd(), etc print *, "Gridded Comp Run returning" rc = ESMF_SUCCESS end subroutine GComp_Run
At the end of application execution, each ESMF_GridComp should deallocate data space, close open files, and flush final results. These functions should be placed in a finalize routine.
The component writer must supply a subroutine with the exact interface shown below. Arguments must not be declared as optional, and the types and order must match.
The rc return code should be set if an error occurs, otherwise the value ESMF_SUCCESS should be returned.
subroutine GComp_Final(comp, importState, exportState, clock, rc) type(ESMF_GridComp) :: comp ! must not be optional type(ESMF_State) :: importState ! must not be optional type(ESMF_State) :: exportState ! must not be optional type(ESMF_Clock) :: clock ! must not be optional integer, intent(out) :: rc ! must not be optional print *, "Gridded Comp Final starting" ! Add whatever code here needed print *, "Gridded Comp Final returning" rc = ESMF_SUCCESS end subroutine GComp_Final
Every ESMF_GridComp can optionally provide and document a public set vm routine. It can have any name, but must follow the declaration below: a subroutine which takes an ESMF_GridComp as the first argument, and an integer return code as the second. Both arguments are required and must not be declared as optional. If an intent is specified in the interface it must be intent(inout) for the first and intent(out) for the second argument.
The set vm routine is the only place where the child component can use the ESMF_GridCompSetVMMaxPEs(), or ESMF_GridCompSetVMMaxThreads(), or ESMF_GridCompSetVMMinThreads() call to modify aspects of its own VM.
A component's VM is started up right before its set services routine is entered. ESMF_GridCompSetVM() is executing in the parent VM, and must be called before ESMF_GridCompSetServices().
subroutine GComp_SetVM(comp, rc) type(ESMF_GridComp) :: comp ! must not be optional integer, intent(out) :: rc ! must not be optional type(ESMF_VM) :: vm logical :: pthreadsEnabled ! Test for Pthread support, all SetVM calls require it call ESMF_VMGetGlobal(vm, rc=rc) call ESMF_VMGet(vm, pthreadsEnabledFlag=pthreadsEnabled, rc=rc) if (pthreadsEnabled) then ! run PETs single-threaded call ESMF_GridCompSetVMMinThreads(comp, rc=rc) endif rc = ESMF_SUCCESS end subroutine end module ESMF_GriddedCompEx
ESMF provides the concept of an Internal State that is associated with a Component. Through the Internal State API a user can attach a private data block to a Component, and later retrieve a pointer to this memory allocation. Setting and getting of Internal State information are supported from anywhere in the Component's SetServices, Initialize, Run, or Finalize code.
The code below demonstrates the basic Internal State API of ESMF_<Grid|Cpl>SetInternalState() and ESMF_<Grid|Cpl>GetInternalState(). Notice that an extra level of indirection to the user data is necessary!
! ESMF Framework module use ESMF use ESMF_TestMod implicit none type(ESMF_GridComp) :: comp integer :: rc, finalrc ! Internal State Variables type testData sequence integer :: testValue real :: testScaling end type type dataWrapper sequence type(testData), pointer :: p end type type(dataWrapper) :: wrap1, wrap2 type(testData), target :: data type(testData), pointer :: datap ! extra level of indirection
!------------------------------------------------------------------------- call ESMF_Initialize(defaultlogfilename="InternalStateEx.Log", & logkindflag=ESMF_LOGKIND_MULTI, rc=rc) if (rc /= ESMF_SUCCESS) call ESMF_Finalize(endflag=ESMF_END_ABORT) !------------------------------------------------------------------------- ! Creation of a Component comp = ESMF_GridCompCreate(name="test", rc=rc) if (rc .ne. ESMF_SUCCESS) finalrc = ESMF_FAILURE !------------------------------------------------------------------------- ! This could be called, for example, during a Component's initialize phase. ! Initialize private data block data%testValue = 4567 data%testScaling = 0.5 ! Set Internal State wrap1%p => data call ESMF_GridCompSetInternalState(comp, wrap1, rc) if (rc .ne. ESMF_SUCCESS) finalrc = ESMF_FAILURE !------------------------------------------------------------------------- ! This could be called, for example, during a Component's run phase. ! Get Internal State call ESMF_GridCompGetInternalState(comp, wrap2, rc) if (rc .ne. ESMF_SUCCESS) finalrc = ESMF_FAILURE ! Access private data block and verify data datap => wrap2%p if ((datap%testValue .ne. 4567) .or. (datap%testScaling .ne. 0.5)) then print *, "did not get same values back" finalrc = ESMF_FAILURE else print *, "got same values back from GetInternalState as original" endif
When working with ESMF Internal States it is important to consider the applying scoping rules. The user must ensure that the private data block that is being referenced persists for the entire access period. This is not an issue in the previous example, where the private data block was defined on the scope of the main program. However, the Internal State construct is often useful inside of Component modules to hold Component specific data between calls. One option to ensure persisting private data blocks is to use the Fortran SAVE attribute either on local or module variables. A second option, illustrated in the following example, is to use Fortran pointers and user controlled memory management via allocate() and deallocate() calls.
One situation where the Internal State is useful is in the creation of ensembles of the same Component. In this case it can be tricky to distinguish which data, held in saved module variables, belongs to which ensemble member - especially if the ensemble members are executing on the same set of PETs. The Internal State solves this problem by providing a handle to instance specific data allocations.
module user_mod use ESMF implicit none ! module variables private ! Internal State Variables type testData sequence integer :: testValue ! scalar data real :: testScaling ! scalar data real, pointer :: testArray(:) ! array data end type type dataWrapper sequence type(testData), pointer :: p end type
contains !--------------------------------------------------------------
subroutine mygcomp_init(gcomp, istate, estate, clock, rc) type(ESMF_GridComp):: gcomp type(ESMF_State):: istate, estate type(ESMF_Clock):: clock integer, intent(out):: rc ! Local variables type(dataWrapper) :: wrap type(testData), pointer :: data integer :: i rc = ESMF_SUCCESS ! Allocate private data block allocate(data) ! Initialize private data block data%testValue = 4567 ! initialize scalar data data%testScaling = 0.5 ! initialize scalar data allocate(data%testArray(10)) ! allocate array data do i=1, 10 data%testArray(i) = real(i) ! initialize array data enddo ! In a real ensemble application the initial data would be set to ! something unique for this ensemble member. This could be ! accomplished for example by reading a member specific config file ! that was specified by the driver code. Alternatively, Attributes, ! set by the driver, could be used to label the Component instances ! as specific ensemble members. ! Set Internal State wrap%p => data call ESMF_GridCompSetInternalState(gcomp, wrap, rc) end subroutine !------------------------------------------------------- subroutine mygcomp_run(gcomp, istate, estate, clock, rc) type(ESMF_GridComp):: gcomp type(ESMF_State):: istate, estate type(ESMF_Clock):: clock integer, intent(out):: rc ! Local variables type(dataWrapper) :: wrap type(testData), pointer :: data logical :: match = .true. integer :: i rc = ESMF_SUCCESS ! Get Internal State call ESMF_GridCompGetInternalState(gcomp, wrap, rc) if (rc/=ESMF_SUCCESS) return ! Access private data block and verify data data => wrap%p if (data%testValue .ne. 4567) match = .false. ! test scalar data if (data%testScaling .ne. 0.5) match = .false. ! test scalar data do i=1, 10 if (data%testArray(i) .ne. real(i)) match = .false. ! test array data enddo if (match) then print *, "got same values back from GetInternalState as original" else print *, "did not get same values back" rc = ESMF_FAILURE endif end subroutine !------------------------------------------------------- subroutine mygcomp_final(gcomp, istate, estate, clock, rc) type(ESMF_GridComp):: gcomp type(ESMF_State):: istate, estate type(ESMF_Clock):: clock integer, intent(out):: rc ! Local variables type(dataWrapper) :: wrap type(testData), pointer :: data rc = ESMF_SUCCESS ! Get Internal State call ESMF_GridCompGetInternalState(gcomp, wrap, rc) if (rc/=ESMF_SUCCESS) return ! Deallocate private data block data => wrap%p deallocate(data%testArray) ! deallocate array data deallocate(data) end subroutine !-------------------------------------------------------------- end module
INTERFACE:
interface assignment(=) gridcomp1 = gridcomp2ARGUMENTS:
type(ESMF_GridComp) :: gridcomp1 type(ESMF_GridComp) :: gridcomp2STATUS:
DESCRIPTION:
Assign gridcomp1 as an alias to the same ESMF GridComp object in memory as gridcomp2. If gridcomp2 is invalid, then gridcomp1 will be equally invalid after the assignment.
The arguments are:
INTERFACE:
interface operator(==) if (gridcomp1 == gridcomp2) then ... endif OR result = (gridcomp1 == gridcomp2)RETURN VALUE:
logical :: resultARGUMENTS:
type(ESMF_GridComp), intent(in) :: gridcomp1 type(ESMF_GridComp), intent(in) :: gridcomp2STATUS:
DESCRIPTION:
Test whether gridcomp1 and gridcomp2 are valid aliases to the same ESMF GridComp object in memory. For a more general comparison of two ESMF GridComps, going beyond the simple alias test, the ESMF_GridCompMatch() function (not yet implemented) must be used.
The arguments are:
INTERFACE:
interface operator(/=) if (gridcomp1 /= gridcomp2) then ... endif OR result = (gridcomp1 /= gridcomp2)RETURN VALUE:
logical :: resultARGUMENTS:
type(ESMF_GridComp), intent(in) :: gridcomp1 type(ESMF_GridComp), intent(in) :: gridcomp2STATUS:
DESCRIPTION:
Test whether gridcomp1 and gridcomp2 are not valid aliases to the same ESMF GridComp object in memory. For a more general comparison of two ESMF GridComps, going beyond the simple alias test, the ESMF_GridCompMatch() function (not yet implemented) must be used.
The arguments are:
INTERFACE:
recursive function ESMF_GridCompCreate(grid, gridList, & mesh, meshList, locstream, locstreamList, xgrid, xgridList, & config, configFile, clock, petList, contextflag, name, rc)RETURN VALUE:
type(ESMF_GridComp) :: ESMF_GridCompCreateARGUMENTS:
-- The following arguments require argument keyword syntax (e.g. rc=rc). -- type(ESMF_Grid), intent(in), optional :: grid type(ESMF_Grid), intent(in), optional :: gridList(:) type(ESMF_Mesh), intent(in), optional :: mesh type(ESMF_Mesh), intent(in), optional :: meshList(:) type(ESMF_LocStream), intent(in), optional :: locstream type(ESMF_LocStream), intent(in), optional :: locstreamList(:) type(ESMF_XGrid), intent(in), optional :: xgrid type(ESMF_XGrid), intent(in), optional :: xgridList(:) type(ESMF_Config), intent(in), optional :: config character(len=*), intent(in), optional :: configFile type(ESMF_Clock), intent(in), optional :: clock integer, intent(in), optional :: petList(:) type(ESMF_Context_Flag), intent(in), optional :: contextflag character(len=*), intent(in), optional :: name integer, intent(out), optional :: rcSTATUS:
DESCRIPTION:
This interface creates an ESMF_GridComp object. By default, a separate VM context will be created for each component. This implies creating a new MPI communicator and allocating additional memory to manage the VM resources. When running on a large number of processors, creating a separate VM for each component could be both time and memory inefficient. If the application is sequential, i.e., each component is running on all the PETs of the global VM, it will be more efficient to use the global VM instead of creating a new one. This can be done by setting contextflag to ESMF_CONTEXT_PARENT_VM.
The return value is the new ESMF_GridComp.
The arguments are:
INTERFACE:
recursive subroutine ESMF_GridCompDestroy(gridcomp, & timeout, timeoutFlag, rc)ARGUMENTS:
type(ESMF_GridComp), intent(inout) :: gridcomp -- The following arguments require argument keyword syntax (e.g. rc=rc). -- integer, intent(in), optional :: timeout logical, intent(out), optional :: timeoutFlag integer, intent(out), optional :: rcSTATUS:
DESCRIPTION:
Destroys an ESMF_GridComp, releasing the resources associated with the object.
The arguments are:
INTERFACE:
recursive subroutine ESMF_GridCompFinalize(gridcomp, & importState, exportState, clock, syncflag, phase, timeout, timeoutFlag, & userRc, rc)ARGUMENTS:
type(ESMF_GridComp), intent(inout) :: gridcomp -- The following arguments require argument keyword syntax (e.g. rc=rc). -- type(ESMF_State), intent(inout), optional :: importState type(ESMF_State), intent(inout), optional :: exportState type(ESMF_Clock), intent(inout), optional :: clock type(ESMF_Sync_Flag), intent(in), optional :: syncflag integer, intent(in), optional :: phase integer, intent(in), optional :: timeout logical, intent(out), optional :: timeoutFlag integer, intent(out), optional :: userRc integer, intent(out), optional :: rcSTATUS:
DESCRIPTION:
Call the associated user-supplied finalization routine for an ESMF_GridComp.
The arguments are:
INTERFACE:
recursive subroutine ESMF_GridCompGet(gridcomp, & gridIsPresent, grid, gridList, meshIsPresent, mesh, meshList, & locstreamIsPresent, locstream, locstreamList, xgridIsPresent, & xgrid, xgridList, importStateIsPresent, importState, & exportStateIsPresent, exportState, configIsPresent, config, & configFileIsPresent, configFile, clockIsPresent, clock, localPet, & petCount, contextflag, currentMethod, currentPhase, comptype, & vmIsPresent, vm, name, rc)ARGUMENTS:
type(ESMF_GridComp), intent(in) :: gridcomp -- The following arguments require argument keyword syntax (e.g. rc=rc). -- logical, intent(out), optional :: gridIsPresent type(ESMF_Grid), intent(out), optional :: grid type(ESMF_Grid), allocatable, intent(out), optional :: gridList(:) logical, intent(out), optional :: meshIsPresent type(ESMF_Mesh), intent(out), optional :: mesh type(ESMF_Mesh), allocatable, intent(out), optional :: meshList(:) logical, intent(out), optional :: locstreamIsPresent type(ESMF_LocStream), intent(out), optional :: locstream type(ESMF_LocStream), allocatable, intent(out), optional :: locstreamList(:) logical, intent(out), optional :: xgridIsPresent type(ESMF_XGrid), intent(out), optional :: xgrid type(ESMF_XGrid), allocatable, intent(out), optional :: xgridList(:) logical, intent(out), optional :: importStateIsPresent type(ESMF_State), intent(out), optional :: importState logical, intent(out), optional :: exportStateIsPresent type(ESMF_State), intent(out), optional :: exportState logical, intent(out), optional :: configIsPresent type(ESMF_Config), intent(out), optional :: config logical, intent(out), optional :: configFileIsPresent character(len=*), intent(out), optional :: configFile logical, intent(out), optional :: clockIsPresent type(ESMF_Clock), intent(out), optional :: clock integer, intent(out), optional :: localPet integer, intent(out), optional :: petCount type(ESMF_Context_Flag), intent(out), optional :: contextflag type(ESMF_Method_Flag), intent(out), optional :: currentMethod integer, intent(out), optional :: currentPhase type(ESMF_CompType_Flag), intent(out), optional :: comptype logical, intent(out), optional :: vmIsPresent type(ESMF_VM), intent(out), optional :: vm character(len=*), intent(out), optional :: name integer, intent(out), optional :: rcSTATUS:
DESCRIPTION:
Get information about an ESMF_GridComp object.
The arguments are:
INTERFACE:
subroutine ESMF_GridCompGetInternalState(gridcomp, wrappedDataPointer, rc)ARGUMENTS:
type(ESMF_GridComp) :: gridcomp type(wrapper) :: wrappedDataPointer integer, intent(out) :: rcSTATUS:
DESCRIPTION:
Available to be called by an ESMF_GridComp at any time after ESMF_GridCompSetInternalState has been called. Since init, run, and finalize must be separate subroutines, data that they need to share in common can either be module global data, or can be allocated in a private data block and the address of that block can be registered with the framework and retrieved by this call. When running multiple instantiations of an ESMF_GridComp, for example during ensemble runs, it may be simpler to maintain private data specific to each run with private data blocks. A corresponding ESMF_GridCompSetInternalState call sets the data pointer to this block, and this call retrieves the data pointer. Note that the wrappedDataPointer argument needs to be a derived type which contains only a pointer of the type of the data block defined by the user. When making this call the pointer needs to be unassociated. When the call returns, the pointer will now reference the original data block which was set during the previous call to ESMF_GridCompSetInternalState.
Only the last data block set via ESMF_GridCompSetInternalState will be accessible.
CAUTION: If you are working with a compiler that does not support Fortran 2018 assumed-type dummy arguments, then this method does not have an explicit Fortran interface. In this case do not specify argument keywords when calling this method!
The arguments are:
INTERFACE:
recursive subroutine ESMF_GridCompInitialize(gridcomp, & importState, exportState, clock, syncflag, phase, timeout, timeoutFlag, & userRc, rc)ARGUMENTS:
type(ESMF_GridComp), intent(inout) :: gridcomp -- The following arguments require argument keyword syntax (e.g. rc=rc). -- type(ESMF_State), intent(inout), optional :: importState type(ESMF_State), intent(inout), optional :: exportState type(ESMF_Clock), intent(inout), optional :: clock type(ESMF_Sync_Flag), intent(in), optional :: syncflag integer, intent(in), optional :: phase integer, intent(in), optional :: timeout logical, intent(out), optional :: timeoutFlag integer, intent(out), optional :: userRc integer, intent(out), optional :: rcSTATUS:
DESCRIPTION:
Call the associated user initialization routine for an ESMF_GridComp.
The arguments are:
INTERFACE:
function ESMF_GridCompIsCreated(gridcomp, rc)RETURN VALUE:
logical :: ESMF_GridCompIsCreatedARGUMENTS:
type(ESMF_GridComp), intent(in) :: gridcomp -- The following arguments require argument keyword syntax (e.g. rc=rc). -- integer, intent(out), optional :: rcDESCRIPTION:
Return .true. if the gridcomp has been created. Otherwise return .false.. If an error occurs, i.e. rc /= ESMF_SUCCESS is returned, the return value of the function will also be .false..
The arguments are:
INTERFACE:
recursive function ESMF_GridCompIsPetLocal(gridcomp, rc)RETURN VALUE:
logical :: ESMF_GridCompIsPetLocalARGUMENTS:
type(ESMF_GridComp), intent(in) :: gridcomp -- The following arguments require argument keyword syntax (e.g. rc=rc). -- integer, intent(out), optional :: rcSTATUS:
DESCRIPTION:
Inquire if this ESMF_GridComp object is to execute on the calling PET.
The return value is .true. if the component is to execute on the calling PET, .false. otherwise.
The arguments are:
INTERFACE:
subroutine ESMF_GridCompPrint(gridcomp, rc)ARGUMENTS:
type(ESMF_GridComp), intent(in) :: gridcomp -- The following arguments require argument keyword syntax (e.g. rc=rc). -- integer, intent(out), optional :: rcSTATUS:
DESCRIPTION:
Prints information about an ESMF_GridComp to stdout.
The arguments are:
INTERFACE:
recursive subroutine ESMF_GridCompReadRestart(gridcomp, & importState, exportState, clock, syncflag, phase, timeout, timeoutFlag, & userRc, rc)ARGUMENTS:
type(ESMF_GridComp), intent(inout) :: gridcomp -- The following arguments require argument keyword syntax (e.g. rc=rc). -- type(ESMF_State), intent(inout), optional :: importState type(ESMF_State), intent(inout), optional :: exportState type(ESMF_Clock), intent(inout), optional :: clock type(ESMF_Sync_Flag), intent(in), optional :: syncflag integer, intent(in), optional :: phase integer, intent(in), optional :: timeout logical, intent(out), optional :: timeoutFlag integer, intent(out), optional :: userRc integer, intent(out), optional :: rcSTATUS:
DESCRIPTION:
Call the associated user read restart routine for an ESMF_GridComp.
The arguments are:
INTERFACE:
recursive subroutine ESMF_GridCompRun(gridcomp, & importState, exportState, clock, syncflag, phase, timeout, timeoutFlag, & userRc, rc)ARGUMENTS:
type(ESMF_GridComp), intent(inout) :: gridcomp -- The following arguments require argument keyword syntax (e.g. rc=rc). -- type(ESMF_State), intent(inout), optional :: importState type(ESMF_State), intent(inout), optional :: exportState type(ESMF_Clock), intent(inout), optional :: clock type(ESMF_Sync_Flag), intent(in), optional :: syncflag integer, intent(in), optional :: phase integer, intent(in), optional :: timeout logical, intent(out), optional :: timeoutFlag integer, intent(out), optional :: userRc integer, intent(out), optional :: rcSTATUS:
DESCRIPTION:
Call the associated user run routine for an ESMF_GridComp.
The arguments are:
INTERFACE:
recursive subroutine ESMF_GridCompServiceLoop(gridcomp, & importState, exportState, clock, syncflag, port, timeout, timeoutFlag, rc)ARGUMENTS:
type(ESMF_GridComp), intent(inout) :: gridcomp -- The following arguments require argument keyword syntax (e.g. rc=rc). -- type(ESMF_State), intent(inout), optional :: importState type(ESMF_State), intent(inout), optional :: exportState type(ESMF_Clock), intent(inout), optional :: clock type(ESMF_Sync_Flag), intent(in), optional :: syncflag integer, intent(in), optional :: port integer, intent(in), optional :: timeout logical, intent(out), optional :: timeoutFlag integer, intent(out), optional :: rcDESCRIPTION:
Call the ServiceLoop routine for an ESMF_GridComp. This tries to establish a "component tunnel" between the actual Component (calling this routine) and a dual Component connecting to it through a matching SetServices call.
The arguments are:
INTERFACE:
subroutine ESMF_GridCompSet(gridcomp, grid, gridList, & mesh, meshList, locstream, locstreamList, xgrid, xgridList, & config, configFile, clock, name, rc)ARGUMENTS:
type(ESMF_GridComp), intent(inout) :: gridcomp -- The following arguments require argument keyword syntax (e.g. rc=rc). -- type(ESMF_Grid), intent(in), optional :: grid type(ESMF_Grid), intent(in), optional :: gridList(:) type(ESMF_Mesh), intent(in), optional :: mesh type(ESMF_Mesh), intent(in), optional :: meshList(:) type(ESMF_LocStream), intent(in), optional :: locstream type(ESMF_LocStream), intent(in), optional :: locstreamList(:) type(ESMF_XGrid), intent(in), optional :: xgrid type(ESMF_XGrid), intent(in), optional :: xgridList(:) type(ESMF_Config), intent(in), optional :: config character(len=*), intent(in), optional :: configFile type(ESMF_Clock), intent(in), optional :: clock character(len=*), intent(in), optional :: name integer, intent(out), optional :: rcSTATUS:
DESCRIPTION:
Sets or resets information about an ESMF_GridComp.
The arguments are:
INTERFACE:
recursive subroutine ESMF_GridCompSetEntryPoint(gridcomp, methodflag, & userRoutine, phase, rc)ARGUMENTS:
type(ESMF_GridComp), intent(inout) :: gridcomp type(ESMF_Method_Flag), intent(in) :: methodflag interface subroutine userRoutine(gridcomp, importState, exportState, clock, rc) use ESMF_CompMod use ESMF_StateMod use ESMF_ClockMod implicit none type(ESMF_GridComp) :: gridcomp ! must not be optional type(ESMF_State) :: importState ! must not be optional type(ESMF_State) :: exportState ! must not be optional type(ESMF_Clock) :: clock ! must not be optional integer, intent(out) :: rc ! must not be optional end subroutine end interface -- The following arguments require argument keyword syntax (e.g. rc=rc). -- integer, intent(in), optional :: phase integer, intent(out), optional :: rcSTATUS:
DESCRIPTION:
Registers a user-supplied userRoutine as the entry point for one of the predefined Component methodflags. After this call the userRoutine becomes accessible via the standard Component method API.
The arguments are:
INTERFACE:
subroutine ESMF_GridCompSetInternalState(gridcomp, wrappedDataPointer, rc)ARGUMENTS:
type(ESMF_GridComp) :: gridcomp type(wrapper) :: wrappedDataPointer integer, intent(out) :: rcSTATUS:
DESCRIPTION:
Available to be called by an ESMF_GridComp at any time, but expected to be most useful when called during the registration process, or initialization. Since init, run, and finalize must be separate subroutines, data that they need to share in common can either be module global data, or can be allocated in a private data block and the address of that block can be registered with the framework and retrieved by subsequent calls. When running multiple instantiations of an ESMF_GridComp, for example during ensemble runs, it may be simpler to maintain private data specific to each run with private data blocks. A corresponding ESMF_GridCompGetInternalState call retrieves the data pointer.
Only the last data block set via ESMF_GridCompSetInternalState will be accessible.
CAUTION: If you are working with a compiler that does not support Fortran 2018 assumed-type dummy arguments, then this method does not have an explicit Fortran interface. In this case do not specify argument keywords when calling this method!
The arguments are:
INTERFACE:
recursive subroutine ESMF_GridCompSetServices(gridcomp, & userRoutine, userRc, rc)ARGUMENTS:
type(ESMF_GridComp), intent(inout) :: gridcomp interface subroutine userRoutine(gridcomp, rc) use ESMF_CompMod implicit none type(ESMF_GridComp) :: gridcomp ! must not be optional integer, intent(out) :: rc ! must not be optional end subroutine end interface -- The following arguments require argument keyword syntax (e.g. rc=rc). -- integer, intent(out), optional :: userRc integer, intent(out), optional :: rcSTATUS:
DESCRIPTION:
Call into user provided userRoutine which is responsible for setting Component's Initialize(), Run(), and Finalize() services.
The arguments are:
The userRoutine, when called by the framework, must make successive calls to ESMF_GridCompSetEntryPoint() to preset callback routines for standard Component Initialize(), Run(), and Finalize() methods.
INTERFACE:
! Private name; call using ESMF_GridCompSetServices() recursive subroutine ESMF_GridCompSetServicesShObj(gridcomp, userRoutine, & sharedObj, userRoutineFound, userRc, rc)ARGUMENTS:
type(ESMF_GridComp), intent(inout) :: gridcomp character(len=*), intent(in) :: userRoutine -- The following arguments require argument keyword syntax (e.g. rc=rc). -- character(len=*), intent(in), optional :: sharedObj logical, intent(out), optional :: userRoutineFound integer, intent(out), optional :: userRc integer, intent(out), optional :: rcSTATUS:
DESCRIPTION:
Call into a user provided routine which is responsible for setting Component's Initialize(), Run(), and Finalize() services. The named userRoutine must exist in the executable, or in the shared object specified by sharedObj. In the latter case all of the platform specific details about dynamic linking and loading apply.
The arguments are:
INTERFACE:
interface subroutine userRoutine(gridcomp, rc) type(ESMF_GridComp) :: gridcomp ! must not be optional integer, intent(out) :: rc ! must not be optional end subroutine end interfaceDESCRIPTION:
The userRoutine, when called by the framework, must make successive calls to ESMF_GridCompSetEntryPoint() to preset callback routines for standard Component Initialize(), Run(), and Finalize() methods.
INTERFACE:
! Private name; call using ESMF_GridCompSetServices() recursive subroutine ESMF_GridCompSetServicesComp(gridcomp, & actualGridcomp, rc)ARGUMENTS:
type(ESMF_GridComp), intent(inout) :: gridcomp type(ESMF_GridComp), intent(in) :: actualGridcomp -- The following arguments require argument keyword syntax (e.g. rc=rc). -- integer, intent(out), optional :: rcDESCRIPTION:
Set the services of a Gridded Component to serve a "dual" Component for an "actual" Component. The component tunnel is VM based.
The arguments are:
INTERFACE:
! Private name; call using ESMF_GridCompSetServices() recursive subroutine ESMF_GridCompSetServicesSock(gridcomp, port, & server, timeout, timeoutFlag, rc)ARGUMENTS:
type(ESMF_GridComp), intent(inout) :: gridcomp integer, intent(in) :: port -- The following arguments require argument keyword syntax (e.g. rc=rc). -- character(len=*), intent(in), optional :: server integer, intent(in), optional :: timeout logical, intent(out), optional :: timeoutFlag integer, intent(out), optional :: rcDESCRIPTION:
Set the services of a Gridded Component to serve a "dual" Component for an "actual" Component. The component tunnel is socket based.
The arguments are:
INTERFACE:
recursive subroutine ESMF_GridCompSetVM(gridcomp, userRoutine, & userRc, rc)ARGUMENTS:
type(ESMF_GridComp), intent(inout) :: gridcomp interface subroutine userRoutine(gridcomp, rc) use ESMF_CompMod implicit none type(ESMF_GridComp) :: gridcomp ! must not be optional integer, intent(out) :: rc ! must not be optional end subroutine end interface -- The following arguments require argument keyword syntax (e.g. rc=rc). -- integer, intent(out), optional :: userRc integer, intent(out), optional :: rcSTATUS:
DESCRIPTION:
Optionally call into user provided userRoutine which is responsible for setting Component's VM properties.
The arguments are:
The subroutine, when called by the framework, is expected to use any of the ESMF_GridCompSetVMxxx() methods to set the properties of the VM associated with the Gridded Component.
INTERFACE:
! Private name; call using ESMF_GridCompSetVM() recursive subroutine ESMF_GridCompSetVMShObj(gridcomp, userRoutine, & sharedObj, userRoutineFound, userRc, rc)ARGUMENTS:
type(ESMF_GridComp), intent(inout) :: gridcomp character(len=*), intent(in) :: userRoutine -- The following arguments require argument keyword syntax (e.g. rc=rc). -- character(len=*), intent(in), optional :: sharedObj logical, intent(out), optional :: userRoutineFound integer, intent(out), optional :: userRc integer, intent(out), optional :: rcSTATUS:
DESCRIPTION:
Optionally call into user provided userRoutine which is responsible for setting Component's VM properties. The named userRoutine must exist in the executable, or in the shared object specified by sharedObj. In the latter case all of the platform specific details about dynamic linking and loading apply.
The arguments are:
INTERFACE:
interface subroutine userRoutine(gridcomp, rc) type(ESMF_GridComp) :: gridcomp ! must not be optional integer, intent(out) :: rc ! must not be optional end subroutine end interfaceDESCRIPTION:
The subroutine, when called by the framework, is expected to use any of the ESMF_GridCompSetVMxxx() methods to set the properties of the VM associated with the Gridded Component.
INTERFACE:
subroutine ESMF_GridCompSetVMMaxPEs(gridcomp, & maxPeCountPerPet, prefIntraProcess, prefIntraSsi, prefInterSsi, & pthreadMinStackSize, openMpHandling, openMpNumThreads, & forceChildPthreads, rc)ARGUMENTS:
type(ESMF_GridComp), intent(inout) :: gridcomp -- The following arguments require argument keyword syntax (e.g. rc=rc). -- integer, intent(in), optional :: maxPeCountPerPet integer, intent(in), optional :: prefIntraProcess integer, intent(in), optional :: prefIntraSsi integer, intent(in), optional :: prefInterSsi integer, intent(in), optional :: pthreadMinStackSize character(*), intent(in), optional :: openMpHandling integer, intent(in), optional :: openMpNumThreads logical, intent(in), optional :: forceChildPthreads integer, intent(out), optional :: rcDESCRIPTION:
Set characteristics of the ESMF_VM for this ESMF_GridComp. Attempts to associate up to maxPeCountPerPet PEs with each PET. Only PEs that are located on the same single system image (SSI) can be associated with the same PET. Within this constraint the call tries to get as close as possible to the number specified by maxPeCountPerPet.
The other constraint to this call is that the number of PEs is preserved. This means that the child Component in the end is associated with as many PEs as the parent Component provided to the child. The number of child PETs however is adjusted according to the above rule.
The typical use of ESMF_GridCompSetVMMaxPEs() is to allocate multiple PEs per PET in a Component for user-level threading, e.g. OpenMP.
The arguments are:
For cases where OpenMP threads are used by the user code, each thread allocates its own private stack. For all threads other than the master, the stack size is set via the typical OMP_STACKSIZE environment variable mechanism. The PET itself, however, becomes the master of the OpenMP thread team, and is not affected by OMP_STACKSIZE. It is the master's stack that can be sized via the pthreadMinStackSize argument, and a large enough size is often critical.
When pthreadMinStackSize is absent, the default is to use the system default set by the limit or ulimit command. However, the stack of a Pthread cannot be unlimited, and a shell stacksize setting of unlimited, or any setting below the ESMF implemented minimum, will result in setting the stack size to 20MiB (the ESMF minimum). Depending on how much private data is used by the user code under the master thread, the default might be too small, and pthreadMinStackSize must be used to allocate sufficient stack space.
INTERFACE:
subroutine ESMF_GridCompSetVMMaxThreads(gridcomp, & maxPetCountPerVas, prefIntraProcess, prefIntraSsi, prefInterSsi, & pthreadMinStackSize, forceChildPthreads, rc)ARGUMENTS:
type(ESMF_GridComp), intent(inout) :: gridcomp -- The following arguments require argument keyword syntax (e.g. rc=rc). -- integer, intent(in), optional :: maxPetCountPerVas integer, intent(in), optional :: prefIntraProcess integer, intent(in), optional :: prefIntraSsi integer, intent(in), optional :: prefInterSsi integer, intent(in), optional :: pthreadMinStackSize logical, intent(in), optional :: forceChildPthreads integer, intent(out), optional :: rcDESCRIPTION:
Set characteristics of the ESMF_VM for this ESMF_GridComp. Attempts to provide maxPetCountPerVas threaded PETs in each virtual address space (VAS). Only as many threaded PETs as there are PEs located on the single system image (SSI) can be associated with the VAS. Within this constraint the call tries to get as close as possible to the number specified by maxPetCountPerVas.
The other constraint to this call is that the number of PETs is preserved. This means that the child Component in the end is associated with as many PETs as the parent Component provided to the child. The threading level of the child PETs however is adjusted according to the above rule.
The typical use of ESMF_GridCompSetVMMaxThreads() is to run a Component multi-threaded with groups of PETs executing within a common virtual address space.
The arguments are:
For cases where OpenMP threads are used by the user code, each thread allocates its own private stack. For all threads other than the master, the stack size is set via the typical OMP_STACKSIZE environment variable mechanism. The PET itself, however, becomes the master of the OpenMP thread team, and is not affected by OMP_STACKSIZE. It is the master's stack that can be sized via the pthreadMinStackSize argument, and a large enough size is often critical.
When pthreadMinStackSize is absent, the default is to use the system default set by the limit or ulimit command. However, the stack of a Pthread cannot be unlimited, and a shell stacksize setting of unlimited, or any setting below the ESMF implemented minimum, will result in setting the stack size to 20MiB (the ESMF minimum). Depending on how much private data is used by the user code under the master thread, the default might be too small, and pthreadMinStackSize must be used to allocate sufficient stack space.
INTERFACE:
subroutine ESMF_GridCompSetVMMinThreads(gridcomp, & maxPeCountPerPet, prefIntraProcess, prefIntraSsi, prefInterSsi, & pthreadMinStackSize, forceChildPthreads, rc)ARGUMENTS:
type(ESMF_GridComp), intent(inout) :: gridcomp -- The following arguments require argument keyword syntax (e.g. rc=rc). -- integer, intent(in), optional :: maxPeCountPerPet integer, intent(in), optional :: prefIntraProcess integer, intent(in), optional :: prefIntraSsi integer, intent(in), optional :: prefInterSsi integer, intent(in), optional :: pthreadMinStackSize logical, intent(in), optional :: forceChildPthreads integer, intent(out), optional :: rcDESCRIPTION:
Set characteristics of the ESMF_VM for this ESMF_GridComp. Reduces the number of threaded PETs in each VAS. The max argument may be specified to limit the maximum number of PEs that a single PET can be associated with.
Several constraints apply: 1) the number of PEs cannot change, 2) PEs cannot migrate between single system images (SSIs), 3) the number of PETs cannot increase, only decrease, 4) PETs cannot migrate between virtual address spaces (VASs), nor can VASs migrate between SSIs.
The typical use of ESMF_GridCompSetVMMinThreads() is to run a Component across a set of single-threaded PETs.
The arguments are:
For cases where OpenMP threads are used by the user code, each thread allocates its own private stack. For all threads other than the master, the stack size is set via the typical OMP_STACKSIZE environment variable mechanism. The PET itself, however, becomes the master of the OpenMP thread team, and is not affected by OMP_STACKSIZE. It is the master's stack that can be sized via the pthreadMinStackSize argument, and a large enough size is often critical.
When pthreadMinStackSize is absent, the default is to use the system default set by the limit or ulimit command. However, the stack of a Pthread cannot be unlimited, and a shell stacksize setting of unlimited, or any setting below the ESMF implemented minimum, will result in setting the stack size to 20MiB (the ESMF minimum). Depending on how much private data is used by the user code under the master thread, the default might be too small, and pthreadMinStackSize must be used to allocate sufficient stack space.
INTERFACE:
subroutine ESMF_GridCompValidate(gridcomp, rc)ARGUMENTS:
type(ESMF_GridComp), intent(in) :: gridcomp -- The following arguments require argument keyword syntax (e.g. rc=rc). -- integer, intent(out), optional :: rcSTATUS:
DESCRIPTION:
Currently all this method does is to check that the gridcomp was created.
The arguments are:
INTERFACE:
subroutine ESMF_GridCompWait(gridcomp, syncflag, & timeout, timeoutFlag, userRc, rc)ARGUMENTS:
type(ESMF_GridComp), intent(inout) :: gridcomp -- The following arguments require argument keyword syntax (e.g. rc=rc). -- type(ESMF_Sync_Flag), intent(in), optional :: syncflag integer, intent(in), optional :: timeout logical, intent(out), optional :: timeoutFlag integer, intent(out), optional :: userRc integer, intent(out), optional :: rcSTATUS:
DESCRIPTION:
When executing asynchronously, wait for an ESMF_GridComp to return.
The arguments are:
INTERFACE:
recursive subroutine ESMF_GridCompWriteRestart(gridcomp, & importState, exportState, clock, syncflag, phase, timeout, timeoutFlag, & userRc, rc)ARGUMENTS:
type(ESMF_GridComp), intent(inout) :: gridcomp -- The following arguments require argument keyword syntax (e.g. rc=rc). -- type(ESMF_State), intent(inout), optional :: importState type(ESMF_State), intent(inout), optional :: exportState type(ESMF_Clock), intent(inout), optional :: clock type(ESMF_Sync_Flag), intent(in), optional :: syncflag integer, intent(in), optional :: phase integer, intent(in), optional :: timeout logical, intent(out), optional :: timeoutFlag integer, intent(out), optional :: userRc integer, intent(out), optional :: rcSTATUS:
DESCRIPTION:
Call the associated user write restart routine for an ESMF_GridComp.
The arguments are:
In a large, multi-component application such as a weather forecasting or climate prediction system running within ESMF, physical domains and major system functions are represented as Gridded Components (see Section 17.1). A Coupler Component, or ESMF_CplComp, arranges and executes the data transformations between the Gridded Components. Ideally, Coupler Components should contain all the information about inter-component communication for an application. This enables the Gridded Components in the application to be used in multiple contexts; that is, used in different coupled configurations without changes to their source code. For example, the same atmosphere might in one case be coupled to an ocean in a hurricane prediction model, and to a data assimilation system for numerical weather prediction in another. A single Coupler Component can couple two or more Gridded Components.
Like Gridded Components, Coupler Components have two parts, one that is provided by the user and another that is part of the framework. The user-written portion of the software is the coupling code necessary for a particular exchange between Gridded Components. This portion of the Coupler Component code must be divided into separately callable initialize, run, and finalize methods. The interfaces for these methods are prescribed by ESMF.
The term “user-written” is somewhat misleading here, since within a Coupler Component the user can leverage ESMF infrastructure software for regridding, redistribution, lower-level communications, calendar management, and other functions. However, ESMF is unlikely to offer all the software necessary to customize a data transfer between Gridded Components. For instance, ESMF does not currently offer tools for unit tranformations or time averaging operations, so users must manage those operations themselves.
The second part of a Coupler Component is the ESMF_CplComp derived type within ESMF. The user must create one of these types to represent a specific coupling function, such as the regular transfer of data between a data assimilation system and an atmospheric model. 2
The user-written part of a Coupler Component is associated with an ESMF_CplComp derived type through a routine called ESMF_SetServices(). This is a routine that the user must write and declare public. Inside the ESMF_SetServices() routine the user must call ESMF_SetEntryPoint() methods that associate a standard ESMF operation with the name of the corresponding Fortran subroutine in their user code. For example, a user routine called “couplerInit” might be associated with the standard initialize routine in a Coupler Component.
A Coupler Component manages the transformation of data between Components. It contains a list of State objects and the operations needed to make them compatible, including such things as regridding and unit conversion. Coupler Components are user-written, following prescribed ESMF interfaces and, wherever desired, using ESMF infrastructure tools.
Every ESMF_CplComp is required to provide and document a public set services routine. It can have any name, but must follow the declaration below: a subroutine which takes an ESMF_CplComp as the first argument, and an integer return code as the second. Both arguments are required and must not be declared as optional. If an intent is specified in the interface it must be intent(inout) for the first and intent(out) for the second argument.
The set services routine must call the ESMF method ESMF_CplCompSetEntryPoint() to register with the framework what user-code subroutines should be called to initialize, run, and finalize the component. There are additional routines which can be registered as well, for checkpoint and restart functions.
Note that the actual subroutines being registered do not have to be public to this module; only the set services routine itself must be available to be used by other code.
! Example Coupler Component module ESMF_CouplerEx ! ESMF Framework module use ESMF implicit none public CPL_SetServices contains subroutine CPL_SetServices(comp, rc) type(ESMF_CplComp) :: comp ! must not be optional integer, intent(out) :: rc ! must not be optional ! Set the entry points for standard ESMF Component methods call ESMF_CplCompSetEntryPoint(comp, ESMF_METHOD_INITIALIZE, & userRoutine=CPL_Init, rc=rc) call ESMF_CplCompSetEntryPoint(comp, ESMF_METHOD_RUN, & userRoutine=CPL_Run, rc=rc) call ESMF_CplCompSetEntryPoint(comp, ESMF_METHOD_FINALIZE, & userRoutine=CPL_Final, rc=rc) rc = ESMF_SUCCESS end subroutine
When a higher level component is ready to begin using an ESMF_CplComp, it will call its initialize routine.
The component writer must supply a subroutine with the exact interface shown below. Arguments must not be declared as optional, and the types and order must match.
At initialization time the component can allocate data space, open data files, set up initial conditions; anything it needs to do to prepare to run.
The rc return code should be set if an error occurs, otherwise the value ESMF_SUCCESS should be returned.
subroutine CPL_Init(comp, importState, exportState, clock, rc) type(ESMF_CplComp) :: comp ! must not be optional type(ESMF_State) :: importState ! must not be optional type(ESMF_State) :: exportState ! must not be optional type(ESMF_Clock) :: clock ! must not be optional integer, intent(out) :: rc ! must not be optional print *, "Coupler Init starting" ! Add whatever code here needed ! Precompute any needed values, fill in any inital values ! needed in Import States rc = ESMF_SUCCESS print *, "Coupler Init returning" end subroutine CPL_Init
During the execution loop, the run routine may be called many times. Each time it should read data from the importState, use the clock to determine what the current time is in the calling component, compute new values or process the data, and produce any output and place it in the exportState.
When a higher level component is ready to use the ESMF_CplComp it will call its run routine.
The component writer must supply a subroutine with the exact interface shown below. Arguments must not be declared as optional, and the types and order must match.
It is expected that this is where the bulk of the model computation or data analysis will occur.
The rc return code should be set if an error occurs, otherwise the value ESMF_SUCCESS should be returned.
subroutine CPL_Run(comp, importState, exportState, clock, rc) type(ESMF_CplComp) :: comp ! must not be optional type(ESMF_State) :: importState ! must not be optional type(ESMF_State) :: exportState ! must not be optional type(ESMF_Clock) :: clock ! must not be optional integer, intent(out) :: rc ! must not be optional print *, "Coupler Run starting" ! Add whatever code needed here to transform Export state data ! into Import states for the next timestep. rc = ESMF_SUCCESS print *, "Coupler Run returning" end subroutine CPL_Run
At the end of application execution, each ESMF_CplComp should deallocate data space, close open files, and flush final results. These functions should be placed in a finalize routine.
The component writer must supply a subroutine with the exact interface shown below. Arguments must not be declared as optional, and the types and order must match.
The rc return code should be set if an error occurs, otherwise the value ESMF_SUCCESS should be returned.
subroutine CPL_Final(comp, importState, exportState, clock, rc) type(ESMF_CplComp) :: comp ! must not be optional type(ESMF_State) :: importState ! must not be optional type(ESMF_State) :: exportState ! must not be optional type(ESMF_Clock) :: clock ! must not be optional integer, intent(out) :: rc ! must not be optional print *, "Coupler Final starting" ! Add whatever code needed here to compute final values and ! finish the computation. rc = ESMF_SUCCESS print *, "Coupler Final returning" end subroutine CPL_Final
Every ESMF_CplComp can optionally provide and document a public set vm routine. It can have any name, but must follow the declaration below: a subroutine which takes an ESMF_CplComp as the first argument, and an integer return code as the second. Both arguments are required and must not be declared as optional. If an intent is specified in the interface it must be intent(inout) for the first and intent(out) for the second argument.
The set vm routine is the only place where the child component can use the ESMF_CplCompSetVMMaxPEs(), or ESMF_CplCompSetVMMaxThreads(), or ESMF_CplCompSetVMMinThreads() call to modify aspects of its own VM.
A component's VM is started up right before its set services routine is entered. ESMF_CplCompSetVM() is executing in the parent VM, and must be called before ESMF_CplCompSetServices().
subroutine GComp_SetVM(comp, rc) type(ESMF_CplComp) :: comp ! must not be optional integer, intent(out) :: rc ! must not be optional type(ESMF_VM) :: vm logical :: pthreadsEnabled ! Test for Pthread support, all SetVM calls require it call ESMF_VMGetGlobal(vm, rc=rc) call ESMF_VMGet(vm, pthreadsEnabledFlag=pthreadsEnabled, rc=rc) if (pthreadsEnabled) then ! run PETs single-threaded call ESMF_CplCompSetVMMinThreads(comp, rc=rc) endif rc = ESMF_SUCCESS end subroutine end module ESMF_CouplerEx
INTERFACE:
interface assignment(=) cplcomp1 = cplcomp2ARGUMENTS:
type(ESMF_CplComp) :: cplcomp1 type(ESMF_CplComp) :: cplcomp2STATUS:
DESCRIPTION:
Assign cplcomp1 as an alias to the same ESMF CplComp object in memory as cplcomp2. If cplcomp2 is invalid, then cplcomp1 will be equally invalid after the assignment.
The arguments are:
INTERFACE:
interface operator(==) if (cplcomp1 == cplcomp2) then ... endif OR result = (cplcomp1 == cplcomp2)RETURN VALUE:
logical :: resultARGUMENTS:
type(ESMF_CplComp), intent(in) :: cplcomp1 type(ESMF_CplComp), intent(in) :: cplcomp2STATUS:
DESCRIPTION:
Test whether cplcomp1 and cplcomp2 are valid aliases to the same ESMF CplComp object in memory. For a more general comparison of two ESMF CplComps, going beyond the simple alias test, the ESMF_CplCompMatch() function (not yet implemented) must be used.
The arguments are:
INTERFACE:
interface operator(/=) if (cplcomp1 /= cplcomp2) then ... endif OR result = (cplcomp1 /= cplcomp2)RETURN VALUE:
logical :: resultARGUMENTS:
type(ESMF_CplComp), intent(in) :: cplcomp1 type(ESMF_CplComp), intent(in) :: cplcomp2STATUS:
DESCRIPTION:
Test whether cplcomp1 and cplcomp2 are not valid aliases to the same ESMF CplComp object in memory. For a more general comparison of two ESMF CplComps, going beyond the simple alias test, the ESMF_CplCompMatch() function (not yet implemented) must be used.
The arguments are:
INTERFACE:
recursive function ESMF_CplCompCreate(config, configFile, & clock, petList, contextflag, name, rc)RETURN VALUE:
type(ESMF_CplComp) :: ESMF_CplCompCreateARGUMENTS:
-- The following arguments require argument keyword syntax (e.g. rc=rc). -- type(ESMF_Config), intent(in), optional :: config character(len=*), intent(in), optional :: configFile type(ESMF_Clock), intent(in), optional :: clock integer, intent(in), optional :: petList(:) type(ESMF_Context_Flag), intent(in), optional :: contextflag character(len=*), intent(in), optional :: name integer, intent(out), optional :: rcSTATUS:
DESCRIPTION:
This interface creates an ESMF_CplComp object. By default, a separate VM context will be created for each component. This implies creating a new MPI communicator and allocating additional memory to manage the VM resources. When running on a large number of processors, creating a separate VM for each component could be both time and memory inefficient. If the application is sequential, i.e., each component is running on all the PETs of the global VM, it will be more efficient to use the global VM instead of creating a new one. This can be done by setting contextflag to ESMF_CONTEXT_PARENT_VM.
The return value is the new ESMF_CplComp.
The arguments are:
INTERFACE:
recursive subroutine ESMF_CplCompDestroy(cplcomp, & timeout, timeoutFlag, rc)ARGUMENTS:
type(ESMF_CplComp), intent(inout) :: cplcomp -- The following arguments require argument keyword syntax (e.g. rc=rc). -- integer, intent(in), optional :: timeout logical, intent(out), optional :: timeoutFlag integer, intent(out), optional :: rcSTATUS:
DESCRIPTION:
Destroys an ESMF_CplComp, releasing the resources associated with the object.
The arguments are:
INTERFACE:
recursive subroutine ESMF_CplCompFinalize(cplcomp, & importState, exportState, clock, syncflag, phase, timeout, timeoutFlag, & userRc, rc)ARGUMENTS:
type(ESMF_CplComp), intent(inout) :: cplcomp -- The following arguments require argument keyword syntax (e.g. rc=rc). -- type(ESMF_State), intent(inout), optional :: importState type(ESMF_State), intent(inout), optional :: exportState type(ESMF_Clock), intent(inout), optional :: clock type(ESMF_Sync_Flag), intent(in), optional :: syncflag integer, intent(in), optional :: phase integer, intent(in), optional :: timeout logical, intent(out), optional :: timeoutFlag integer, intent(out), optional :: userRc integer, intent(out), optional :: rcSTATUS:
DESCRIPTION:
Call the associated user-supplied finalization routine for an ESMF_CplComp.
The arguments are:
INTERFACE:
subroutine ESMF_CplCompGet(cplcomp, configIsPresent, config, & configFileIsPresent, configFile, clockIsPresent, clock, localPet, & petCount, contextflag, currentMethod, currentPhase, vmIsPresent, & vm, name, rc)ARGUMENTS:
type(ESMF_CplComp), intent(in) :: cplcomp -- The following arguments require argument keyword syntax (e.g. rc=rc). -- logical, intent(out), optional :: configIsPresent type(ESMF_Config), intent(out), optional :: config logical, intent(out), optional :: configFileIsPresent character(len=*), intent(out), optional :: configFile logical, intent(out), optional :: clockIsPresent type(ESMF_Clock), intent(out), optional :: clock integer, intent(out), optional :: localPet integer, intent(out), optional :: petCount type(ESMF_Context_Flag), intent(out), optional :: contextflag type(ESMF_Method_Flag), intent(out), optional :: currentMethod integer, intent(out), optional :: currentPhase logical, intent(out), optional :: vmIsPresent type(ESMF_VM), intent(out), optional :: vm character(len=*), intent(out), optional :: name integer, intent(out), optional :: rcSTATUS:
DESCRIPTION:
Get information about an ESMF_CplComp object.
The arguments are:
INTERFACE:
subroutine ESMF_CplCompGetInternalState(cplcomp, wrappedDataPointer, rc)ARGUMENTS:
type(ESMF_CplComp) :: cplcomp type(wrapper) :: wrappedDataPointer integer, intent(out) :: rcSTATUS:
DESCRIPTION:
Available to be called by an ESMF_CplComp at any time after ESMF_CplCompSetInternalState has been called. Since init, run, and finalize must be separate subroutines, data that they need to share in common can either be module global data, or can be allocated in a private data block and the address of that block can be registered with the framework and retrieved by this call. When running multiple instantiations of an ESMF_CplComp, for example during ensemble runs, it may be simpler to maintain private data specific to each run with private data blocks. A corresponding ESMF_CplCompSetInternalState call sets the data pointer to this block, and this call retrieves the data pointer. Note that the wrappedDataPointer argument needs to be a derived type which contains only a pointer of the type of the data block defined by the user. When making this call the pointer needs to be unassociated. When the call returns, the pointer will now reference the original data block which was set during the previous call to ESMF_CplCompSetInternalState.
Only the last data block set via ESMF_CplCompSetInternalState will be accessible.
CAUTION: If you are working with a compiler that does not support Fortran 2018 assumed-type dummy arguments, then this method does not have an explicit Fortran interface. In this case do not specify argument keywords when calling this method!
The arguments are:
INTERFACE:
recursive subroutine ESMF_CplCompInitialize(cplcomp, & importState, exportState, clock, syncflag, phase, timeout, timeoutFlag, & userRc, rc)ARGUMENTS:
type(ESMF_CplComp), intent(inout) :: cplcomp -- The following arguments require argument keyword syntax (e.g. rc=rc). -- type(ESMF_State), intent(inout), optional :: importState type(ESMF_State), intent(inout), optional :: exportState type(ESMF_Clock), intent(inout), optional :: clock type(ESMF_Sync_Flag), intent(in), optional :: syncflag integer, intent(in), optional :: phase integer, intent(in), optional :: timeout logical, intent(out), optional :: timeoutFlag integer, intent(out), optional :: userRc integer, intent(out), optional :: rcSTATUS:
DESCRIPTION:
Call the associated user initialization routine for an ESMF_CplComp.
The arguments are:
INTERFACE:
function ESMF_CplCompIsCreated(cplcomp, rc)RETURN VALUE:
logical :: ESMF_CplCompIsCreatedARGUMENTS:
type(ESMF_CplComp), intent(in) :: cplcomp -- The following arguments require argument keyword syntax (e.g. rc=rc). -- integer, intent(out), optional :: rcDESCRIPTION:
Return .true. if the cplcomp has been created. Otherwise return .false.. If an error occurs, i.e. rc /= ESMF_SUCCESS is returned, the return value of the function will also be .false..
The arguments are:
INTERFACE:
recursive function ESMF_CplCompIsPetLocal(cplcomp, rc)RETURN VALUE:
logical :: ESMF_CplCompIsPetLocalARGUMENTS:
type(ESMF_CplComp), intent(in) :: cplcomp -- The following arguments require argument keyword syntax (e.g. rc=rc). -- integer, intent(out), optional :: rcSTATUS:
DESCRIPTION:
Inquire if this ESMF_CplComp object is to execute on the calling PET.
The return value is .true. if the component is to execute on the calling PET, .false. otherwise.
The arguments are:
INTERFACE:
subroutine ESMF_CplCompPrint(cplcomp, rc)ARGUMENTS:
type(ESMF_CplComp), intent(in) :: cplcomp -- The following arguments require argument keyword syntax (e.g. rc=rc). -- integer, intent(out), optional :: rcSTATUS:
DESCRIPTION:
Prints information about an ESMF_CplComp to stdout.
The arguments are:
INTERFACE:
recursive subroutine ESMF_CplCompReadRestart(cplcomp, & importState, exportState, clock, syncflag, phase, timeout, timeoutFlag, & userRc, rc)ARGUMENTS:
type(ESMF_CplComp), intent(inout) :: cplcomp -- The following arguments require argument keyword syntax (e.g. rc=rc). -- type(ESMF_State), intent(inout), optional :: importState type(ESMF_State), intent(inout), optional :: exportState type(ESMF_Clock), intent(inout), optional :: clock type(ESMF_Sync_Flag), intent(in), optional :: syncflag integer, intent(in), optional :: phase integer, intent(in), optional :: timeout logical, intent(out), optional :: timeoutFlag integer, intent(out), optional :: userRc integer, intent(out), optional :: rcSTATUS:
DESCRIPTION:
Call the associated user read restart routine for an ESMF_CplComp.
The arguments are:
INTERFACE:
recursive subroutine ESMF_CplCompRun(cplcomp, & importState, exportState, clock, syncflag, phase, timeout, timeoutFlag, & userRc, rc)ARGUMENTS:
type(ESMF_CplComp), intent(inout) :: cplcomp -- The following arguments require argument keyword syntax (e.g. rc=rc). -- type(ESMF_State), intent(inout), optional :: importState type(ESMF_State), intent(inout), optional :: exportState type(ESMF_Clock), intent(inout), optional :: clock type(ESMF_Sync_Flag), intent(in), optional :: syncflag integer, intent(in), optional :: phase integer, intent(in), optional :: timeout logical, intent(out), optional :: timeoutFlag integer, intent(out), optional :: userRc integer, intent(out), optional :: rcSTATUS:
DESCRIPTION:
Call the associated user run routine for an ESMF_CplComp.
The arguments are:
INTERFACE:
recursive subroutine ESMF_CplCompServiceLoop(cplcomp, & importState, exportState, clock, syncflag, port, timeout, timeoutFlag, rc)ARGUMENTS:
type(ESMF_CplComp), intent(inout) :: cplcomp -- The following arguments require argument keyword syntax (e.g. rc=rc). -- type(ESMF_State), intent(inout), optional :: importState type(ESMF_State), intent(inout), optional :: exportState type(ESMF_Clock), intent(inout), optional :: clock type(ESMF_Sync_Flag), intent(in), optional :: syncflag integer, intent(in), optional :: port integer, intent(in), optional :: timeout logical, intent(out), optional :: timeoutFlag integer, intent(out), optional :: rcDESCRIPTION:
Call the ServiceLoop routine for an ESMF_CplComp. This tries to establish a "component tunnel" between the actual Component (calling this routine) and a dual Component connecting to it through a matching SetServices call.
The arguments are:
INTERFACE:
subroutine ESMF_CplCompSet(cplcomp, config, configFile, & clock, name, rc)ARGUMENTS:
type(ESMF_CplComp), intent(inout) :: cplcomp -- The following arguments require argument keyword syntax (e.g. rc=rc). -- type(ESMF_Config), intent(in), optional :: config character(len=*), intent(in), optional :: configFile type(ESMF_Clock), intent(in), optional :: clock character(len=*), intent(in), optional :: name integer, intent(out), optional :: rcSTATUS:
DESCRIPTION:
Sets or resets information about an ESMF_CplComp.
The arguments are:
INTERFACE:
recursive subroutine ESMF_CplCompSetEntryPoint(cplcomp, methodflag, & userRoutine, phase, rc)ARGUMENTS:
type(ESMF_CplComp), intent(inout) :: cplcomp type(ESMF_Method_Flag), intent(in) :: methodflag interface subroutine userRoutine(cplcomp, importState, exportState, clock, rc) use ESMF_CompMod use ESMF_StateMod use ESMF_ClockMod implicit none type(ESMF_CplComp) :: cplcomp ! must not be optional type(ESMF_State) :: importState ! must not be optional type(ESMF_State) :: exportState ! must not be optional type(ESMF_Clock) :: clock ! must not be optional integer, intent(out) :: rc ! must not be optional end subroutine end interface -- The following arguments require argument keyword syntax (e.g. rc=rc). -- integer, intent(in), optional :: phase integer, intent(out), optional :: rcSTATUS:
DESCRIPTION:
Registers a user-supplied userRoutine as the entry point for one of the predefined Component methodflags. After this call the userRoutine becomes accessible via the standard Component method API.
The arguments are:
INTERFACE:
subroutine ESMF_CplCompSetInternalState(cplcomp, wrappedDataPointer, rc)ARGUMENTS:
type(ESMF_CplComp) :: cplcomp type(wrapper) :: wrappedDataPointer integer, intent(out) :: rcSTATUS:
DESCRIPTION:
Available to be called by an ESMF_CplComp at any time, but expected to be most useful when called during the registration process, or initialization. Since init, run, and finalize must be separate subroutines data that they need to share in common can either be module global data, or can be allocated in a private data block and the address of that block can be registered with the framework and retrieved by subsequent calls. When running multiple instantiations of an ESMF_CplComp, for example during ensemble runs, it may be simpler to maintain private data specific to each run with private data blocks. A corresponding ESMF_CplCompGetInternalState call retrieves the data pointer.
Only the last data block set via ESMF_CplCompSetInternalState will be accessible.
CAUTION: If you are working with a compiler that does not support Fortran 2018 assumed-type dummy arguments, then this method does not have an explicit Fortran interface. In this case do not specify argument keywords when calling this method!
The arguments are:
INTERFACE:
recursive subroutine ESMF_CplCompSetServices(cplcomp, userRoutine, & userRc, rc)ARGUMENTS:
type(ESMF_CplComp), intent(inout) :: cplcomp interface subroutine userRoutine(cplcomp, rc) use ESMF_CompMod implicit none type(ESMF_CplComp) :: cplcomp ! must not be optional integer, intent(out) :: rc ! must not be optional end subroutine end interface -- The following arguments require argument keyword syntax (e.g. rc=rc). -- integer, intent(out), optional :: userRc integer, intent(out), optional :: rcSTATUS:
DESCRIPTION:
Call into user provided userRoutine which is responsible for setting Component's Initialize(), Run(), and Finalize() services.
The arguments are:
The userRoutine, when called by the framework, must make successive calls to ESMF_CplCompSetEntryPoint() to preset callback routines for standard Component Initialize(), Run(), and Finalize() methods.
INTERFACE:
! Private name; call using ESMF_CplCompSetServices() recursive subroutine ESMF_CplCompSetServicesShObj(cplcomp, userRoutine, & sharedObj, userRoutineFound, userRc, rc)ARGUMENTS:
type(ESMF_CplComp), intent(inout) :: cplcomp character(len=*), intent(in) :: userRoutine -- The following arguments require argument keyword syntax (e.g. rc=rc). -- character(len=*), intent(in), optional :: sharedObj logical, intent(out), optional :: userRoutineFound integer, intent(out), optional :: userRc integer, intent(out), optional :: rcSTATUS:
DESCRIPTION:
Call into a user provided routine which is responsible for setting Component's Initialize(), Run(), and Finalize() services. The named userRoutine must exist in the executable, or in the shared object specified by sharedObj. In the latter case all of the platform specific details about dynamic linking and loading apply.
The arguments are:
INTERFACE:
interface subroutine userRoutine(cplcomp, rc) type(ESMF_CplComp) :: cplcomp ! must not be optional integer, intent(out) :: rc ! must not be optional end subroutine end interfaceDESCRIPTION:
The userRoutine, when called by the framework, must make successive calls to ESMF_CplCompSetEntryPoint() to preset callback routines for standard Component Initialize(), Run(), and Finalize() methods.
INTERFACE:
! Private name; call using ESMF_CplCompSetServices() recursive subroutine ESMF_CplCompSetServicesComp(cplcomp, & actualCplcomp, rc)ARGUMENTS:
type(ESMF_CplComp), intent(inout) :: cplcomp type(ESMF_CplComp), intent(in) :: actualCplcomp -- The following arguments require argument keyword syntax (e.g. rc=rc). -- integer, intent(out), optional :: rcDESCRIPTION:
Set the services of a Coupler Component to serve a "dual" Component for an "actual" Component. The component tunnel is VM based.
The arguments are:
INTERFACE:
! Private name; call using ESMF_CplCompSetServices() recursive subroutine ESMF_CplCompSetServicesSock(cplcomp, port, & server, timeout, timeoutFlag, rc)ARGUMENTS:
type(ESMF_CplComp), intent(inout) :: cplcomp integer, intent(in) :: port -- The following arguments require argument keyword syntax (e.g. rc=rc). -- character(len=*), intent(in), optional :: server integer, intent(in), optional :: timeout logical, intent(out), optional :: timeoutFlag integer, intent(out), optional :: rcDESCRIPTION:
Set the services of a Coupler Component to serve a "dual" Component for an "actual" Component. The component tunnel is socket based.
The arguments are:
INTERFACE:
recursive subroutine ESMF_CplCompSetVM(cplcomp, userRoutine, & userRc, rc)ARGUMENTS:
type(ESMF_CplComp), intent(inout) :: cplcomp interface subroutine userRoutine(cplcomp, rc) use ESMF_CompMod implicit none type(ESMF_CplComp) :: cplcomp ! must not be optional integer, intent(out) :: rc ! must not be optional end subroutine end interface -- The following arguments require argument keyword syntax (e.g. rc=rc). -- integer, intent(out), optional :: userRc integer, intent(out), optional :: rcSTATUS:
DESCRIPTION:
Optionally call into user provided userRoutine which is responsible for setting Component's VM properties.
The arguments are:
The subroutine, when called by the framework, is expected to use any of the ESMF_CplCompSetVMxxx() methods to set the properties of the VM associated with the Coupler Component.
INTERFACE:
! Private name; call using ESMF_CplCompSetVM() recursive subroutine ESMF_CplCompSetVMShObj(cplcomp, userRoutine, & sharedObj, userRoutineFound, userRc, rc)ARGUMENTS:
type(ESMF_CplComp), intent(inout) :: cplcomp character(len=*), intent(in) :: userRoutine -- The following arguments require argument keyword syntax (e.g. rc=rc). -- character(len=*), intent(in), optional :: sharedObj logical, intent(out), optional :: userRoutineFound integer, intent(out), optional :: userRc integer, intent(out), optional :: rcSTATUS:
DESCRIPTION:
Optionally call into user provided userRoutine which is responsible for setting Component's VM properties. The named userRoutine must exist in the executable, or in the shared object specified by sharedObj. In the latter case all of the platform specific details about dynamic linking and loading apply.
The arguments are:
INTERFACE:
interface subroutine userRoutine(cplcomp, rc) type(ESMF_CplComp) :: cplcomp ! must not be optional integer, intent(out) :: rc ! must not be optional end subroutine end interfaceDESCRIPTION:
The subroutine, when called by the framework, is expected to use any of the ESMF_CplCompSetVMxxx() methods to set the properties of the VM associated with the Coupler Component.
INTERFACE:
subroutine ESMF_CplCompSetVMMaxPEs(cplcomp, & maxPeCountPerPet, prefIntraProcess, prefIntraSsi, prefInterSsi, & pthreadMinStackSize, forceChildPthreads, rc)ARGUMENTS:
type(ESMF_CplComp), intent(inout) :: cplcomp -- The following arguments require argument keyword syntax (e.g. rc=rc). -- integer, intent(in), optional :: maxPeCountPerPet integer, intent(in), optional :: prefIntraProcess integer, intent(in), optional :: prefIntraSsi integer, intent(in), optional :: prefInterSsi integer, intent(in), optional :: pthreadMinStackSize logical, intent(in), optional :: forceChildPthreads integer, intent(out), optional :: rcDESCRIPTION:
Set characteristics of the ESMF_VM for this ESMF_CplComp. Attempts to associate up to maxPeCountPerPet PEs with each PET. Only PEs that are located on the same single system image (SSI) can be associated with the same PET. Within this constraint the call tries to get as close as possible to the number specified by maxPeCountPerPet.
The other constraint to this call is that the number of PEs is preserved. This means that the child Component in the end is associated with as many PEs as the parent Component provided to the child. The number of child PETs however is adjusted according to the above rule.
The typical use of ESMF_CplCompSetVMMaxPEs() is to allocate multiple PEs per PET in a Component for user-level threading, e.g. OpenMP.
The arguments are:
For cases where OpenMP threads are used by the user code, each thread allocates its own private stack. For all threads other than the master, the stack size is set via the typical OMP_STACKSIZE environment variable mechanism. The PET itself, however, becomes the master of the OpenMP thread team, and is not affected by OMP_STACKSIZE. It is the master's stack that can be sized via the pthreadMinStackSize argument, and a large enough size is often critical.
When pthreadMinStackSize is absent, the default is to use the system default set by the limit or ulimit command. However, the stack of a Pthread cannot be unlimited, and a shell stacksize setting of unlimited, or any setting below the ESMF implemented minimum, will result in setting the stack size to 20MiB (the ESMF minimum). Depending on how much private data is used by the user code under the master thread, the default might be too small, and pthreadMinStackSize must be used to allocate sufficient stack space.
INTERFACE:
subroutine ESMF_CplCompSetVMMaxThreads(cplcomp, & maxPetCountPerVas, prefIntraProcess, prefIntraSsi, prefInterSsi, & pthreadMinStackSize, forceChildPthreads, rc)ARGUMENTS:
type(ESMF_CplComp), intent(inout) :: cplcomp -- The following arguments require argument keyword syntax (e.g. rc=rc). -- integer, intent(in), optional :: maxPetCountPerVas integer, intent(in), optional :: prefIntraProcess integer, intent(in), optional :: prefIntraSsi integer, intent(in), optional :: prefInterSsi integer, intent(in), optional :: pthreadMinStackSize logical, intent(in), optional :: forceChildPthreads integer, intent(out), optional :: rcDESCRIPTION:
Set characteristics of the ESMF_VM for this ESMF_CplComp. Attempts to provide maxPetCountPerVas threaded PETs in each virtual address space (VAS). Only as many threaded PETs as there are PEs located on the single system image (SSI) can be associated with the VAS. Within this constraint the call tries to get as close as possible to the number specified by maxPetCountPerVas.
The other constraint to this call is that the number of PETs is preserved. This means that the child Component in the end is associated with as many PETs as the parent Component provided to the child. The threading level of the child PETs however is adjusted according to the above rule.
The typical use of ESMF_CplCompSetVMMaxThreads() is to run a Component multi-threaded with groups of PETs executing within a common virtual address space.
The arguments are:
For cases where OpenMP threads are used by the user code, each thread allocates its own private stack. For all threads other than the master, the stack size is set via the typical OMP_STACKSIZE environment variable mechanism. The PET itself, however, becomes the master of the OpenMP thread team, and is not affected by OMP_STACKSIZE. It is the master's stack that can be sized via the pthreadMinStackSize argument, and a large enough size is often critical.
When pthreadMinStackSize is absent, the default is to use the system default set by the limit or ulimit command. However, the stack of a Pthread cannot be unlimited, and a shell stacksize setting of unlimited, or any setting below the ESMF implemented minimum, will result in setting the stack size to 20MiB (the ESMF minimum). Depending on how much private data is used by the user code under the master thread, the default might be too small, and pthreadMinStackSize must be used to allocate sufficient stack space.
INTERFACE:
subroutine ESMF_CplCompSetVMMinThreads(cplcomp, & maxPeCountPerPet, prefIntraProcess, prefIntraSsi, prefInterSsi, & pthreadMinStackSize, forceChildPthreads, rc)ARGUMENTS:
type(ESMF_CplComp), intent(inout) :: cplcomp -- The following arguments require argument keyword syntax (e.g. rc=rc). -- integer, intent(in), optional :: maxPeCountPerPet integer, intent(in), optional :: prefIntraProcess integer, intent(in), optional :: prefIntraSsi integer, intent(in), optional :: prefInterSsi integer, intent(in), optional :: pthreadMinStackSize logical, intent(in), optional :: forceChildPthreads integer, intent(out), optional :: rcDESCRIPTION:
Set characteristics of the ESMF_VM for this ESMF_CplComp. Reduces the number of threaded PETs in each VAS. The max argument may be specified to limit the maximum number of PEs that a single PET can be associated with.
Several constraints apply: 1) the number of PEs cannot change, 2) PEs cannot migrate between single system images (SSIs), 3) the number of PETs cannot increase, only decrease, 4) PETs cannot migrate between virtual address spaces (VASs), nor can VASs migrate between SSIs.
The typical use of ESMF_CplCompSetVMMinThreads() is to run a Component across a set of single-threaded PETs.
The arguments are:
For cases where OpenMP threads are used by the user code, each thread allocates its own private stack. For all threads other than the master, the stack size is set via the typical OMP_STACKSIZE environment variable mechanism. The PET itself, however, becomes the master of the OpenMP thread team, and is not affected by OMP_STACKSIZE. It is the master's stack that can be sized via the pthreadMinStackSize argument, and a large enough size is often critical.
When pthreadMinStackSize is absent, the default is to use the system default set by the limit or ulimit command. However, the stack of a Pthread cannot be unlimited, and a shell stacksize setting of unlimited, or any setting below the ESMF implemented minimum, will result in setting the stack size to 20MiB (the ESMF minimum). Depending on how much private data is used by the user code under the master thread, the default might be too small, and pthreadMinStackSize must be used to allocate sufficient stack space.
INTERFACE:
subroutine ESMF_CplCompValidate(cplcomp, rc)ARGUMENTS:
type(ESMF_CplComp), intent(in) :: cplcomp -- The following arguments require argument keyword syntax (e.g. rc=rc). -- integer, intent(out), optional :: rcSTATUS:
DESCRIPTION:
Currently all this method does is to check that the cplcomp was created.
The arguments are:
INTERFACE:
subroutine ESMF_CplCompWait(cplcomp, syncflag, & timeout, timeoutFlag, userRc, rc)ARGUMENTS:
type(ESMF_CplComp), intent(inout) :: cplcomp -- The following arguments require argument keyword syntax (e.g. rc=rc). -- type(ESMF_Sync_Flag), intent(in), optional :: syncflag integer, intent(in), optional :: timeout logical, intent(out), optional :: timeoutFlag integer, intent(out), optional :: userRc integer, intent(out), optional :: rcSTATUS:
DESCRIPTION:
When executing asynchronously, wait for an ESMF_CplComp to return.
The arguments are:
INTERFACE:
recursive subroutine ESMF_CplCompWriteRestart(cplcomp, & importState, exportState, clock, syncflag, phase, timeout, timeoutFlag, & userRc, rc)ARGUMENTS:
type(ESMF_CplComp), intent(inout) :: cplcomp -- The following arguments require argument keyword syntax (e.g. rc=rc). -- type(ESMF_State), intent(inout), optional :: importState type(ESMF_State), intent(inout), optional :: exportState type(ESMF_Clock), intent(inout), optional :: clock type(ESMF_Sync_Flag), intent(in), optional :: syncflag integer, intent(in), optional :: phase integer, intent(in), optional :: timeout logical, intent(out), optional :: timeoutFlag integer, intent(out), optional :: userRc integer, intent(out), optional :: rcSTATUS:
DESCRIPTION:
Call the associated user write restart routine for an ESMF_CplComp.
The arguments are:
In Earth system modeling, a particular piece of code representing a physical domain, such as an atmospheric model or an ocean model, is typically implemented as an ESMF Gridded Component, or ESMC_GridComp. However, there are times when physical domains, or realms, need to be represented, but aren't actual pieces of code, or software. These domains can be implemented as ESMF Science Components, or ESMC_SciComp.
Unlike Gridded and Coupler Components, Science Components are not associated with software; they don't include execution routines such as initialize, run and finalize. The main purpose of a Science Component is to provide a container for Attributes within a Component hierarchy.
A Science Component is a container object intended to represent scientific domains, or realms, in an Earth Science Model. It's primary purpose is to provide a means for representing Component metadata within a hierarchy of Components, and it does this by being a container for Attributes as well as other Components.
This example illustrates the use of the ESMF_SciComp to attach Attributes within a Component hierarchy. The hierarchy includes Coupler, Gridded, and Science Components and Attributes are attached to the Science Components. For demonstrable purposes, we'll add some CIM Component attributes to the Gridded Component.
Create the top 2 levels of the Component hierarchy. This example creates a parent Coupler Component and 2 Gridded Components as children.
! Create top-level Coupler Component cplcomp = ESMF_CplCompCreate(name="coupler_component", rc=rc) ! Create Gridded Component for Atmosphere atmcomp = ESMF_GridCompCreate(name="Atmosphere", rc=rc) ! Create Gridded Component for Ocean ocncomp = ESMF_GridCompCreate(name="Ocean", rc=rc)
Now add CIM Attribute packages to the Component. Also, add a CIM Component Properties package, to contain two custom attributes.
convCIM = 'CIM 1.5' purpComp = 'ModelComp' purpProp = 'CompProp' purpField = 'Inputs' purpPlatform = 'Platform' convISO = 'ISO 19115' purpRP = 'RespParty' purpCitation = 'Citation' ! Add CIM Attribute package to the Science Component call ESMF_AttributeAdd(atmcomp, convention=convCIM, & purpose=purpComp, attpack=attpack, rc=rc)
The Attribute package can also be retrieved in a multi-Component setting like this:
call ESMF_AttributeGetAttPack(atmcomp, convCIM, purpComp, & attpack=attpack, rc=rc)
Now, add some CIM Component attributes to the Atmosphere Grid Component.
! ! Top-level model component attributes, set on gridded component ! call ESMF_AttributeSet(atmcomp, 'ShortName', 'EarthSys_Atmos', & attpack=attpack, rc=rc)
call ESMF_AttributeSet(atmcomp, 'LongName', & 'Earth System High Resolution Global Atmosphere Model', & attpack=attpack, rc=rc)
call ESMF_AttributeSet(atmcomp, 'Description', & 'EarthSys brings together expertise from the global ' // & 'community in a concerted effort to develop coupled ' // & 'climate models with increased horizontal resolutions. ' // & 'Increasing the horizontal resolution of coupled climate ' // & 'models will allow us to capture climate processes and ' // & 'weather systems in much greater detail.', & attpack=attpack, rc=rc)
call ESMF_AttributeSet(atmcomp, 'Version', '2.0', & attpack=attpack, rc=rc)
call ESMF_AttributeSet(atmcomp, 'ReleaseDate', '2009-01-01T00:00:00Z', & attpack=attpack, rc=rc)
call ESMF_AttributeSet(atmcomp, 'ModelType', 'aerosol', & attpack=attpack, rc=rc)
call ESMF_AttributeSet(atmcomp, 'URL', & 'www.earthsys.org', attpack=attpack, rc=rc)
Now create a set of Science Components as a children of the Atmosphere Gridded Component. The hierarchy is as follows:
! ! Atmosphere Dynamical Core Science Component ! dc_scicomp = ESMF_SciCompCreate(name="AtmosDynamicalCore", rc=rc)
call ESMF_AttributeAdd(dc_scicomp, & convention=convCIM, purpose=purpComp, & attpack=attpack, rc=rc) call ESMF_AttributeSet(dc_scicomp, "ShortName", "AtmosDynamicalCore", & attpack=attpack, rc=rc) call ESMF_AttributeSet(dc_scicomp, "LongName", & "Atmosphere Dynamical Core", & attpack=attpack, rc=rc)
purpSci = 'SciProp' dc_sciPropAtt(1) = 'TopBoundaryCondition' dc_sciPropAtt(2) = 'HeatTreatmentAtTop' dc_sciPropAtt(3) = 'WindTreatmentAtTop' call ESMF_AttributeAdd(dc_scicomp, & convention=convCIM, purpose=purpSci, & attrList=dc_sciPropAtt, & attpack=attpack, rc=rc) call ESMF_AttributeSet(dc_scicomp, 'TopBoundaryCondition', & 'radiation boundary condition', & attpack=attpack, rc=rc) call ESMF_AttributeSet(dc_scicomp, 'HeatTreatmentAtTop', & 'some heat treatment', & attpack=attpack, rc=rc) call ESMF_AttributeSet(dc_scicomp, 'WindTreatmentAtTop', & 'some wind treatment', & attpack=attpack, rc=rc)
! ! Atmosphere Advection Science Component ! adv_scicomp = ESMF_SciCompCreate(name="AtmosAdvection", rc=rc)
call ESMF_AttributeAdd(adv_scicomp, & convention=convCIM, purpose=purpComp, & attpack=attpack, rc=rc) call ESMF_AttributeSet(adv_scicomp, "ShortName", "AtmosAdvection", & attpack=attpack, rc=rc) call ESMF_AttributeSet(adv_scicomp, "LongName", "Atmosphere Advection", & attpack=attpack, rc=rc)
adv_sciPropAtt(1) = 'TracersSchemeName' adv_sciPropAtt(2) = 'TracersSchemeCharacteristics' adv_sciPropAtt(3) = 'MomentumSchemeName' call ESMF_AttributeAdd(adv_scicomp, & convention=convCIM, purpose=purpSci, & attrList=adv_sciPropAtt, & attpack=attpack, rc=rc) call ESMF_AttributeSet(adv_scicomp, 'TracersSchemeName', 'Prather', & attpack=attpack, rc=rc) call ESMF_AttributeSet(adv_scicomp, 'TracersSchemeCharacteristics', & 'modified Euler', & attpack=attpack, rc=rc) call ESMF_AttributeSet(adv_scicomp, 'MomentumSchemeName', 'Van Leer', & attpack=attpack, rc=rc)
! ! Atmosphere Radiation Science Component ! rad_scicomp = ESMF_SciCompCreate(name="AtmosRadiation", rc=rc)
call ESMF_AttributeAdd(rad_scicomp, & convention=convCIM, purpose=purpComp, & attpack=attpack, rc=rc) call ESMF_AttributeSet(rad_scicomp, "ShortName", "AtmosRadiation", & attpack=attpack, rc=rc) call ESMF_AttributeSet(rad_scicomp, "LongName", & "Atmosphere Radiation", & attpack=attpack, rc=rc)
rad_sciPropAtt(1) = 'LongwaveSchemeType' rad_sciPropAtt(2) = 'LongwaveSchemeMethod' call ESMF_AttributeAdd(rad_scicomp, & convention=convCIM, purpose=purpSci, & attrList=rad_sciPropAtt, & attpack=attpack, rc=rc) call ESMF_AttributeSet(rad_scicomp, & 'LongwaveSchemeType', & 'wide-band model', & attpack=attpack, rc=rc) call ESMF_AttributeSet(rad_scicomp, & 'LongwaveSchemeMethod', & 'two-stream', & attpack=attpack, rc=rc)
Finally, destroy all of the Components.
call ESMF_SciCompDestroy(rad_scicomp, rc=rc) call ESMF_SciCompDestroy(adv_scicomp, rc=rc) call ESMF_SciCompDestroy(dc_scicomp, rc=rc) call ESMF_GridCompDestroy(atmcomp, rc=rc) call ESMF_GridCompDestroy(ocncomp, rc=rc) call ESMF_CplCompDestroy(cplcomp, rc=rc)
INTERFACE:
interface assignment(=) scicomp1 = scicomp2ARGUMENTS:
type(ESMF_SciComp) :: scicomp1 type(ESMF_SciComp) :: scicomp2DESCRIPTION:
Assign scicomp1 as an alias to the same ESMF SciComp object in memory as scicomp2. If scicomp2 is invalid, then scicomp1 will be equally invalid after the assignment.
The arguments are:
INTERFACE:
interface operator(==) if (scicomp1 == scicomp2) then ... endif OR result = (scicomp1 == scicomp2)RETURN VALUE:
logical :: resultARGUMENTS:
type(ESMF_SciComp), intent(in) :: scicomp1 type(ESMF_SciComp), intent(in) :: scicomp2DESCRIPTION:
Test whether scicomp1 and scicomp2 are valid aliases to the same ESMF SciComp object in memory. For a more general comparison of two ESMF SciComps, going beyond the simple alias test, the ESMF_SciCompMatch() function (not yet implemented) must be used.
The arguments are:
INTERFACE:
interface operator(/=) if (scicomp1 /= scicomp2) then ... endif OR result = (scicomp1 /= scicomp2)RETURN VALUE:
logical :: resultARGUMENTS:
type(ESMF_SciComp), intent(in) :: scicomp1 type(ESMF_SciComp), intent(in) :: scicomp2DESCRIPTION:
Test whether scicomp1 and scicomp2 are not valid aliases to the same ESMF SciComp object in memory. For a more general comparison of two ESMF SciComps, going beyond the simple alias test, the ESMF_SciCompMatch() function (not yet implemented) must be used.
The arguments are:
INTERFACE:
recursive function ESMF_SciCompCreate(name, rc)RETURN VALUE:
type(ESMF_SciComp) :: ESMF_SciCompCreateARGUMENTS:
-- The following arguments require argument keyword syntax (e.g. rc=rc). -- character(len=*), intent(in), optional :: name integer, intent(out), optional :: rcDESCRIPTION:
This interface creates an ESMF_SciComp object. The return value is the new ESMF_SciComp.
The arguments are:
INTERFACE:
subroutine ESMF_SciCompDestroy(scicomp, rc)ARGUMENTS:
type(ESMF_SciComp), intent(inout) :: scicomp -- The following arguments require argument keyword syntax (e.g. rc=rc). -- integer, intent(out), optional :: rcDESCRIPTION:
Destroys an ESMF_SciComp, releasing the resources associated with the object.
The arguments are:
INTERFACE:
subroutine ESMF_SciCompGet(scicomp, name, rc)ARGUMENTS:
type(ESMF_SciComp), intent(in) :: scicomp -- The following arguments require argument keyword syntax (e.g. rc=rc). -- character(len=*), intent(out), optional :: name integer, intent(out), optional :: rcDESCRIPTION:
Get information about an ESMF_SciComp object.
The arguments are:
INTERFACE:
function ESMF_SciCompIsCreated(scicomp, rc)RETURN VALUE:
logical :: ESMF_SciCompIsCreatedARGUMENTS:
type(ESMF_SciComp), intent(in) :: scicomp -- The following arguments require argument keyword syntax (e.g. rc=rc). -- integer, intent(out), optional :: rcDESCRIPTION:
Return .true. if the scicomp has been created. Otherwise return .false.. If an error occurs, i.e. rc /= ESMF_SUCCESS is returned, the return value of the function will also be .false..
The arguments are:
INTERFACE:
subroutine ESMF_SciCompPrint(scicomp, rc)ARGUMENTS:
type(ESMF_SciComp), intent(in) :: scicomp -- The following arguments require argument keyword syntax (e.g. rc=rc). -- integer, intent(out), optional :: rcDESCRIPTION:
Prints information about an ESMF_SciComp to stdout.
The arguments are:
INTERFACE:
subroutine ESMF_SciCompSet(scicomp, name, rc)ARGUMENTS:
type(ESMF_SciComp), intent(inout) :: scicomp -- The following arguments require argument keyword syntax (e.g. rc=rc). -- character(len=*), intent(in), optional :: name integer, intent(out), optional :: rcDESCRIPTION:
Sets or resets information about an ESMF_SciComp.
The arguments are:
INTERFACE:
subroutine ESMF_SciCompValidate(scicomp, rc)ARGUMENTS:
type(ESMF_SciComp), intent(in) :: scicomp -- The following arguments require argument keyword syntax (e.g. rc=rc). -- integer, intent(out), optional :: rcDESCRIPTION:
Currently all this method does is to check that the scicomp was created.
The arguments are:
For ensemble runs with many ensemble members, fault-tolerance becomes an issue of very critical practical impact. The meaning of fault-tolerance in this context refers to the ability of an ensemble application to continue with normal execution after one or more ensemble members have experienced catastrophic conditions, from which they cannot recover. ESMF implements this type of fault-tolerance on the Component level via a timeout paradigm: A timeout parameter is specified for all interactions that need to be fault-tolerant. When a connection to a component times out, maybe because it has become inaccessible due to some catastrophic condition, the driver application can react to this condition, for example by not further interacting with the component during the otherwise normal continuation of the model execution.
The fault-tolerant connection between a driver application and a Component is established through a Component Tunnel. There are two sides to a Component Tunnel: the "actual" side is where the component is actually executing, and the "dual" side is the portal through which the Component becomes accessible on the driver side. Both the actual and the dual side of a Component Tunnel are implemented in form of a regular ESMF Gridded or Coupler Component.
Component Tunnels between Components can be based on a number of low level implementations. The only implementation that currently provides fault-tolerance is socket based. In this case an actual Component typically runs as a separate executable, listening to a specific port for connections from the driver application. The dual Component is created on the driver side. It connects to the actual Component during the SetServices() call.
A Component Tunnel connects a dual Component to an actual Component. This connection can be based on a number of different low level implementations, e.g. VM-based or socket-based. VM-based Component Tunnels require that both dual and actual Components run within the same application (i.e. execute under the same MPI_COMM_WORLD). Fault-tolerant Component Tunnels require that dual and actual Components run in separate applications, under different MPI_COMM_WORLD communicators. This mode is implemented in the socket-based Component Tunnels.
The creation process of an actual Gridded Component, which will become one of the two end points of a Component Tunnel, is identical to the creation of a regular Gridded Component. On the actual side, an actual Component is very similar to a regular Component. Here the actual Component is created with a custom petList.
petList = (/0,1,2/) actualComp = ESMF_GridCompCreate(petList=petList, name="actual", rc=rc)
The same way an actual Component appears as a regular Component in the context of the actual side application, a dual Component is created as a regular Component on the dual side. A dual Gridded Component with custom petList is created using the regular create call.
petList = (/4,3,5/) dualComp = ESMF_GridCompCreate(petList=petList, name="dual", rc=rc)
After creation, the regular procedure for registering the standard Component methods is followed for the actual Gridded Component.
call ESMF_GridCompSetServices(actualComp, userRoutine=setservices, & userRc=userRc, rc=rc)
So far the actualComp object is no different from a regular Gridded Component. In order to turn it into the actual end point of a Component Tunnel the ServiceLoop() method is called. Here the socket-based implementation is chosen.
call ESMF_GridCompServiceLoop(actualComp, port=61010, timeout=20, rc=rc)
This call opens the actual side of the Component Tunnel in form of a socket-based server, listening on port 61010. The timeout argument specifies how long the actual side will wait for the dual side to connect, before the actual side returns with a time out condition. The time out is set to 20 seconds.
At this point, before a dual Component connects to the other side of the Component Tunnel, it is possible to manually connect to the waiting actual Component. This can be useful when debugging connection issues. A convenient tool for this is the standard telnet application. Below is a transcript of such a connection. The manually typed commands are separate from the previous responses by a blank line.
$ telnet localhost 61010 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. Hello from ESMF Actual Component server! date Tue Apr 3 21:53:03 2012 version ESMF_VERSION_STRING: 5.3.0
If at any point the telnet session is manually shut down, the ServiceLoop() on the actual side will return with an error condition. The clean way to disconnect the telnet session, and to have the ServiceLoop() wait for a new connection, e.g. from a dual Component, is to send the reconnect command. This will automatically shut down the telnet connection.
reconnect Actual Component server will reconnect now! Connection closed by foreign host. $
At this point the actual Component is back in listening mode, with a time out of 20 seconds, as specified during the ServiceLoop() call.
Before moving on to the dual side of the GridComp based Component Tunnel example, it should be pointed out that the exact same procedure is used to set up the actual side of a CplComp based Component Tunnel. Assuming that actualCplComp is a CplComp object for which SetServices has already been called, the actual side uses ESMF_CplCompServiceLoop() to start listening for connections from the dual side.
call ESMF_CplCompServiceLoop(actualCplComp, port=61011, timeout=2, & timeoutFlag=timeoutFlag, rc=rc)
Here the timeoutFlag is specified in order to prevent the expected time-out condition to be indicated through the return code. Instead, when timeoutFlag is present, the return code is still ESMF_SUCCESS, but timeoutFlag is set to .true. when a time-out occurs.
On the dual side, the dualComp object needs to be connected to the actual Component in order to complete the Component Tunnel. Instead of registering standard Component methods locally, a special variant of the SetServices() call is used to connect to the actual Component.
call ESMF_GridCompSetServices(dualComp, port=61010, server="localhost", & timeout=10, timeoutFlag=timeoutFlag, rc=rc)
The port and server arguments are used to connect to the desired actual Component. The time out of 10 seconds ensures that if the actual Component is not available, a time out condition is returned instead of resulting in a hang. The timeoutFlag argument further absorbs the time out condition, either returning as .true. or .false.. In this mode the standard rc will indicate success even when a time out condition was reached.
Once a Component Tunnel is established, the actual Component is fully under the control of the dual Component. A standard Component method invoked on the dual Component is not executed by the dual Component itself, but by the actual Component instead. In fact, it is the entry points registered with the actual Component that are executed when standard methods are invoked on the dual Component. The connected dualComp object serves as a portal through which the connected actualComp becomes accessible on the dual side.
Typically the first standard method called is the CompInitialize() routine.
call ESMF_GridCompInitialize(dualComp, timeout=10, timeoutFlag=timeoutFlag, & userRc=userRc, rc=rc)
Again, the timeout argument serves to prevent the dual side from hanging if the actual Component application has experienced a catastrophic condition and is no longer available, or takes longer than expected. The presence of the timeoutFlag allows time out conditions to be caught gracefully, so the dual side can deal with it in an orderly fashion, instead of triggering an application abort due to an error condition.
The CompRun() and CompFinalize() methods follow the same format.
call ESMF_GridCompRun(dualComp, timeout=10, timeoutFlag=timeoutFlag, & userRc=userRc, rc=rc)
call ESMF_GridCompFinalize(dualComp, timeout=10, timeoutFlag=timeoutFlag, & userRc=userRc, rc=rc)
Standard Component methods called on a connected dual Component are executed on the actual side, across the PETs of the actual Component. By default the dual Component PETs are blocked until the actual Component has finished executing the invoked Component method, or until a time out condition has been reached. In many practical applications a more loose synchronization between dual and actual Components is useful. Having the PETs of a dual Component return immediately from a standard Component method allows multiple dual Component, on the same PETs, to control multiple actual Components. If the actual Components are executing in separate executables, or the same executable but on exclusive sets of PETs, they can execute concurrently, even with the controlling dual Components all running on the same PETs. The non-blocking dual side regains control over the actual Component by synchronizing through the CompWait() call.
Any of the standard Component methods can be called in non-blocking mode by setting the optional syncflag argument to ESMF_SYNC_NONBLOCKING.
call ESMF_GridCompInitialize(dualComp, syncflag=ESMF_SYNC_NONBLOCKING, rc=rc)
If communication between the dual and the actual Component was successful, this call will return immediately on all of the dual Component PETs, while the actual Component continues to execute the invoked Component method. However, if the dual Component has difficulties reaching the actual Component, the call will block on all dual PETs until successful contact was made, or the default time out (3600 seconds, i.e. 1 hour) has been reached. In most cases a shorter time out condition is desired with the non-blocking option, as shown below.
First the dual Component must wait for the outstanding method.
call ESMF_GridCompWait(dualComp, rc=rc)
Now the same non-blocking CompInitialize() call is issued again, but this time with an explicit 10 second time out.
call ESMF_GridCompInitialize(dualComp, syncflag=ESMF_SYNC_NONBLOCKING, & timeout=10, timeoutFlag=timeoutFlag, rc=rc)
This call is guaranteed to return within 10 seconds, or less, on the dual Component PETs, either without time out condition, indicating that the actual Component has been contacted successfully, or with time out condition, indicating that the actual Component was unreachable at the time. Either way, the dual Component PETs are back under user control quickly.
Calling the CompWait() method on the dual Component causes the dual Component PETs to block until the actual Component method has returned, or a time out condition has been reached.
call ESMF_GridCompWait(dualComp, userRc=userRc, rc=rc)
The default time out for CompWait() is 3600 seconds, i.e. 1 hour, just like for the other Component methods. However, the semantics of a time out condition under CompWait() is different from the other Component methods. Typically the timeout is simply the maximum time that any communication between dual and actual Component is allowed to take before a time out condition is raised. For CompWait(), the timeout is the maximum time that an actual Component is allowed to execute before reporting back to the dual Component. Here, even with the default time out, the dual Component would return from CompWait() immediately with a time out condition if the actual Component has already been executing for over 1 hour, and is not already waiting to report back when the dual Component calls CompWait(). On the other hand, if it has only been 30 minutes since CompInitialize() was called on the dual Component, then the actual Component still has 30 minutes before CompWait() returns with a time out condition. During this time (or until the actual Component returns) the dual Component PETs are blocked.
A standard Component method is invoked in non-blocking mode.
call ESMF_GridCompRun(dualComp, syncflag=ESMF_SYNC_NONBLOCKING, & timeout=10, timeoutFlag=timeoutFlag, rc=rc)
Once the user code on the dual side is ready to regain control over the actual Component it calls CompWait() on the dual Component. Here a timeout of 60s is specified, meaning that the total execution time the actual Component spends in the registered Run() routine may not exceed 60s before CompWait() returns with a time out condition.
call ESMF_GridCompWait(dualComp, timeout=60, userRc=userRc, rc=rc)
A dual Component that is connected to an actual Component through a Component Tunnel is destroyed the same way a regular Component is. The only difference is that a connected dual Component may specify a timeout argument to the CompDestroy() call.
call ESMF_GridCompDestroy(dualComp, timeout=10, rc=rc)
The timeout argument again ensures that the dual side does not hang indefinitely in case the actual Component has become unavailable. If the actual Component is available, the destroy call will indicate to the actual Component that it should break out of the ServiceLoop(). Either way, the local dual Component is destroyed.
An actual Component that is in a ServiceLoop() must first return from that call before it can be destroyed. This can either happen when a connected dual Component calls its CompDestroy() method, or if the ServiceLoop() reaches the specified time out condition. Either way, once control has been returned to the user code, the actual Component is destroyed in the same way a regular Component is, by calling the destroy method.
call ESMF_GridCompDestroy(actualComp, rc=rc)
A State contains the data and metadata to be transferred between ESMF Components. It is an important class, because it defines a standard for how data is represented in data transfers between Earth science components. The State construct is a rational compromise between a fully prescribed interface - one that would dictate what specific fields should be transferred between components - and an interface in which data structures are completely ad hoc.
There are two types of States, import and export. An import State contains data that is necessary for a Gridded Component or Coupler Component to execute, and an export State contains the data that a Gridded Component or Coupler Component can make available.
States can contain Arrays, ArrayBundles, Fields, FieldBundles, and other States. They cannot directly contain native language arrays (i.e. Fortran or C style arrays). Objects in a State must span the VM on which they are running. For sequentially executing components which run on the same set of PETs this happens by calling the object create methods on each PET, creating the object in unison. For concurrently executing components which are running on subsets of PETs, an additional method, called ESMF_StateReconcile(), is provided by ESMF to broadcast information about objects which were created in sub-components.
State methods include creation and deletion, adding and retrieving data items, adding and retrieving attributes, and performing queries.
DESCRIPTION:
Specifies whether a ESMF_State contains data to be imported
into a component or exported from a component.
The type of this flag is:
type(ESMF_StateIntent_Flag)
The valid values are:
DESCRIPTION:
Specifies the type of object being added to or retrieved from an
ESMF_State.
The type of this flag is:
type(ESMF_StateItem_Flag)
The valid values are:
A Gridded Component generally has one associated import State and one export State. Generally the States associated with a Gridded Component will be created by the Gridded Component's parent component. In many cases, the States will be created containing no data. Both the empty States and the newly created Gridded Component are passed by the parent component into the Gridded Component's initialize method. This is where the States get prepared for use and the import State is first filled with data.
States can be filled with data items that do not yet have data allocated. Fields, FieldBundles, Arrays, and ArrayBundles each have methods that support their creation without actual data allocation - the Grid and Attributes are set up but no Fortran array of data values is allocated. In this approach, when a State is passed into its associated Gridded Component's initialize method, the incomplete Arrays, Fields, FieldBundles, and ArrayBundles within the State can allocate or reference data inside the initialize method.
States are passed through the interfaces of the Gridded and Coupler Components' run methods in order to carry data between the components. While we expect a Gridded Component's import State to be filled with data during initialization, its export State will typically be filled over the course of its run method. At the end of a Gridded Component's run method, the filled export State is passed out through the argument list into a Coupler Component's run method. We recommend the convention that it enters the Coupler Component as the Coupler Component's import State. Here is it transformed into a form that another Gridded Component requires, and passed out of the Coupler Component as its export State. It can then be passed into the run method of a recipient Gridded Component as that component's import State.
While the above sounds complicated, the rule is simple: a State going into a component is an import State, and a State leaving a component is an export State.
Objects inside States are normally created in unison where each PET executing a component makes the same object create call. If the object contains data, like a Field, each PET may have a different local chunk of the entire dataset but each Field has the same name and is logically one part of a single distributed object. As States are passed between components, if any object in a State was not created in unison on all the current PETs then some PETs have no object to pass into a communication method (e.g. regrid or data redistribution). The ESMF_StateReconcile() method must be called to broadcast information about these objects to all PETs in a component; after which all PETs have a single uniform view of all objects and metadata.
If components are running in sequential mode on all available PETs and States are being passed between them there is no need to call ESMF_StateReconcile since all PETs have a uniform view of the objects. However, if components are running on a subset of the PETs, as is usually the case when running in concurrent mode, then when States are passed into components which contain a superset of those PETs, for example, a Coupler Component, all PETs must call ESMF_StateReconcile on the States before using them in any ESMF communication methods. The reconciliation process broadcasts information about objects which exist only on a subset of the PETs. On PETs missing those objects it creates a proxy object which contains any qualities of the original object plus enough information for it to be a data source or destination for a regrid or data redistribution operation.
States can be created and destroyed at any time during application execution. The ESMF_StateCreate() routine can take many different combinations of optional arguments. Refer to the API description for all possible methods of creating a State. An empty State can be created by providing only a name and type for the intended State:
state = ESMF_StateCreate(name, stateintent=ESMF_STATEINTENT_IMPORT, rc=rc)
When finished with an ESMF_State, the ESMF_StateDestroy method removes it. However, the objects inside the ESMF_State created externally should be destroyed separately, since objects can be added to more than one ESMF_State.
Creation of an empty ESMF_State, and adding an ESMF_FieldBundle to it. Note that the ESMF_FieldBundle does not get destroyed when the ESMF_State is destroyed; the ESMF_State only contains a reference to the objects it contains. It also does not make a copy; the original objects can be updated and code accessing them by using the ESMF_State will see the updated version.
statename = "Ocean" state2 = ESMF_StateCreate(name=statename, & stateintent=ESMF_STATEINTENT_EXPORT, rc=rc)
bundlename = "Temperature" bundle1 = ESMF_FieldBundleCreate(name=bundlename, rc=rc) print *, "FieldBundle Create returned", rc
call ESMF_StateAdd(state2, (/bundle1/), rc=rc) print *, "StateAdd returned", rc
call ESMF_StateDestroy(state2, rc=rc)
call ESMF_FieldBundleDestroy(bundle1, rc=rc)
If a component could potentially produce a large number of optional items, one strategy is to add the names only of those objects to the ESMF_State. Other components can call framework routines to set the ESMF_NEEDED flag to indicate they require that data. The original component can query this flag and then produce only the data that is required by another component.
statename = "Ocean" state3 = ESMF_StateCreate(name=statename, & stateintent=ESMF_STATEINTENT_EXPORT, rc=rc)
dataname = "Downward wind:needed" call ESMF_AttributeSet (state3, dataname, .false., rc=rc)
dataname = "Humidity:needed" call ESMF_AttributeSet (state3, dataname, .false., rc=rc)
How to set the NEEDED state of an item.
dataname = "Downward wind:needed" call ESMF_AttributeSet (state3, name=dataname, value=.true., rc=rc)
Query an item for the NEEDED status, and creating an item on demand. Similar flags exist for "Ready", "Valid", and "Required for Restart", to mark each data item as ready, having been validated, or needed if the application is to be checkpointed and restarted. The flags are supported to help coordinate the data exchange between components.
dataname = "Downward wind:needed" call ESMF_AttributeGet (state3, dataname, value=neededFlag, rc=rc)
if (rc == ESMF_SUCCESS .and. neededFlag) then bundlename = dataname bundle2 = ESMF_FieldBundleCreate(name=bundlename, rc=rc)
call ESMF_StateAdd(state3, (/bundle2/), rc=rc)
else print *, "Data not marked as needed", trim(dataname) endif
The set services routines are used to tell ESMF which routine hold the user code for the initialize, run, and finalize blocks of user level Components. These are the separate subroutines called by the code below.
! Initialize routine which creates "field1" on PETs 0 and 1 subroutine comp1_init(gcomp, istate, ostate, clock, rc) type(ESMF_GridComp) :: gcomp type(ESMF_State) :: istate, ostate type(ESMF_Clock) :: clock integer, intent(out) :: rc type(ESMF_Field) :: field1 integer :: localrc print *, "i am comp1_init" field1 = ESMF_FieldEmptyCreate(name="Comp1 Field", rc=localrc) call ESMF_StateAdd(istate, (/field1/), rc=localrc) rc = localrc end subroutine comp1_init ! Initialize routine which creates "field2" on PETs 2 and 3 subroutine comp2_init(gcomp, istate, ostate, clock, rc) type(ESMF_GridComp) :: gcomp type(ESMF_State) :: istate, ostate type(ESMF_Clock) :: clock integer, intent(out) :: rc type(ESMF_Field) :: field2 integer :: localrc print *, "i am comp2_init" field2 = ESMF_FieldEmptyCreate(name="Comp2 Field", rc=localrc) call ESMF_StateAdd(istate, (/field2/), rc=localrc) rc = localrc end subroutine comp2_init subroutine comp_dummy(gcomp, rc) type(ESMF_GridComp) :: gcomp integer, intent(out) :: rc rc = ESMF_SUCCESS end subroutine comp_dummy
! !PROGRAM: ESMF_StateReconcileEx - State reconciliation ! ! !DESCRIPTION: ! ! This program shows examples of using the State Reconcile function !----------------------------------------------------------------------------- #include "ESMF.h" ! ESMF Framework module use ESMF use ESMF_TestMod use ESMF_StateReconcileEx_Mod implicit none ! Local variables integer :: rc, petCount type(ESMF_State) :: state1 type(ESMF_GridComp) :: comp1, comp2 type(ESMF_VM) :: vm character(len=ESMF_MAXSTR) :: comp1name, comp2name, statename
A Component can be created which will run only on a subset of the current PET list.
! Get the global VM for this job. call ESMF_VMGetGlobal(vm=vm, rc=rc) comp1name = "Atmosphere" comp1 = ESMF_GridCompCreate(name=comp1name, petList=(/ 0, 1 /), rc=rc) print *, "GridComp Create returned, name = ", trim(comp1name)
comp2name = "Ocean" comp2 = ESMF_GridCompCreate(name=comp2name, petList=(/ 2, 3 /), rc=rc) print *, "GridComp Create returned, name = ", trim(comp2name)
statename = "Ocn2Atm" state1 = ESMF_StateCreate(name=statename, rc=rc)
Here we register the subroutines which should be called for initialization. Then we call ESMF_GridCompInitialize() on all PETs, but the code runs only on the PETs given in the petList when the Component was created.
Because this example is so short, we call the entry point code directly instead of the normal procedure of nesting it in a separate SetServices() subroutine.
! This is where the VM for each component is initialized. ! Normally you would call SetEntryPoint inside set services, ! but to make this example very short, they are called inline below. ! This is o.k. because the SetServices routine must execute from within ! the parent component VM. call ESMF_GridCompSetVM(comp1, comp_dummy, rc=rc)
call ESMF_GridCompSetVM(comp2, comp_dummy, rc=rc)
call ESMF_GridCompSetServices(comp1, userRoutine=comp_dummy, rc=rc)
call ESMF_GridCompSetServices(comp2, userRoutine=comp_dummy, rc=rc)
print *, "ready to set entry point 1" call ESMF_GridCompSetEntryPoint(comp1, ESMF_METHOD_INITIALIZE, & comp1_init, rc=rc)
print *, "ready to set entry point 2" call ESMF_GridCompSetEntryPoint(comp2, ESMF_METHOD_INITIALIZE, & comp2_init, rc=rc)
print *, "ready to call init for comp 1" call ESMF_GridCompInitialize(comp1, exportState=state1, rc=rc)
print *, "ready to call init for comp 2" call ESMF_GridCompInitialize(comp2, exportState=state1, rc=rc)
Now we have state1 containing field1 on PETs 0 and 1, and state1 containing field2 on PETs 2 and 3. For the code to have a rational view of the data, we call ESMF_StateReconcile which determines which objects are missing from any PET, and communicates information about the object. After the call to reconcile, all ESMF_State objects now have a consistent view of the data.
print *, "State before calling StateReconcile()" call ESMF_StatePrint(state1, rc=rc)
call ESMF_StateReconcile(state1, vm=vm, rc=rc)
print *, "State after calling StateReconcile()" call ESMF_StatePrint(state1, rc=rc)
end program ESMF_StateReconcileEx
! ESMF Framework module use ESMF use ESMF_TestMod implicit none ! Local variables type(ESMF_State) :: state type(ESMF_Array) :: latArray, lonArray, timeArray, humidArray, & tempArray, pArray, rhArray type(ESMF_VM) :: vm integer :: localPet, rc
The following line of code will read all Array data contained in a NetCDF file, place them in ESMF_Arrays and add them to an ESMF_State. Only PET 0 reads the file; the States in the other PETs remain empty. Currently, the data is not decomposed or distributed; each PET has only 1 DE and only PET 0 contains data after reading the file. Future versions of ESMF will support data decomposition and distribution upon reading a file.
Note that the third party NetCDF library must be installed. For more details, see the "ESMF Users Guide", "Building and Installing the ESMF, Third Party Libraries, NetCDF" and the website http://www.unidata.ucar.edu/software/netcdf.
! Read the NetCDF data file into Array objects in the State on PET 0 call ESMF_StateRead(state, "io_netcdf_testdata.nc", rc=rc) ! If the NetCDF library is not present (on PET 0), cleanup and exit if (rc == ESMF_RC_LIB_NOT_PRESENT) then call ESMF_StateDestroy(state, rc=rc) goto 10 endif
Only reading data into ESMF_Arrays is supported at this time; ESMF_ArrayBundles, ESMF_Fields, and ESMF_FieldBundles will be supported in future releases of ESMF.
To see that the State now contains the same data as in the file, the following shows how to print out what Arrays are contained within the State and to print the data contained within each Array. The NetCDF utility "ncdump" can be used to view the contents of the NetCDF file. In this example, only PET 0 will contain data.
if (localPet == 0) then ! Print the names and attributes of Array objects contained in the State call ESMF_StatePrint(state, rc=rc) ! Get each Array by name from the State call ESMF_StateGet(state, "lat", latArray, rc=rc) call ESMF_StateGet(state, "lon", lonArray, rc=rc) call ESMF_StateGet(state, "time", timeArray, rc=rc) call ESMF_StateGet(state, "Q", humidArray, rc=rc) call ESMF_StateGet(state, "TEMP", tempArray, rc=rc) call ESMF_StateGet(state, "p", pArray, rc=rc) call ESMF_StateGet(state, "rh", rhArray, rc=rc) ! Print out the Array data call ESMF_ArrayPrint(latArray, rc=rc) call ESMF_ArrayPrint(lonArray, rc=rc) call ESMF_ArrayPrint(timeArray, rc=rc) call ESMF_ArrayPrint(humidArray, rc=rc) call ESMF_ArrayPrint(tempArray, rc=rc) call ESMF_ArrayPrint(pArray, rc=rc) call ESMF_ArrayPrint(rhArray, rc=rc) endif
Note that the Arrays "lat", "lon", and "time" hold spatial and temporal coordinate data for the dimensions latitude, longitude and time, respectively. These will be used in future releases of ESMF to create ESMF_Grids.
All the Array data within the State on PET 0 can be written out to a NetCDF file as follows:
! Write Arrays within the State on PET 0 to a NetCDF file call ESMF_StateWrite(state, "io_netcdf_testdata_out.nc", rc=rc)
Currently writing is limited to PET 0; future versions of ESMF will allow parallel writing, as well as parallel reading.
One important request by the user community during the ESMF object design was that there be no communication overhead or synchronization when creating distributed ESMF objects. As a consequence it is required to create these objects in unison across all PETs in order to keep the ESMF object identification in sync.
When running components on subsets of the original VM all the PETs can create consistent objects but then when they are put into a State and passed to a component with a different VM and a different set of PETs, a communication call (reconcile) must be made to communicate the missing information to the PETs which were not involved in the original object creation. The reconcile call broadcasts object lists; those PETs which are missing any objects in the total list can receive enough information to reconstruct a proxy object which contains all necessary information about that object, with no local data, on that PET. These proxy objects can be queried by ESMF routines to determine the amount of data and what PETs contain data which is destined to be moved to the local PET (for receiving data) and conversely, can determine which other PETs are going to receive data and how much (for sending data).
For example, the FieldExcl system test creates 2 Gridded Components on separate subsets of PETs. They use the option of mapping particular, non-monotonic PETs to DEs. The following figures illustrate how the DEs are mapped in each of the Gridded Components in that test:
In the coupler code, all PETs must make the reconcile call before accessing data in the State. On PETs which already contain data, the objects are unchanged. On PETs which were not involved during the creation of the FieldBundles or Fields, the reconcile call adds an object to the State which contains all the same metadata associated with the object, but creates a slightly different Grid object, called a Proxy Grid. These PETs contain no local data, so the Array object is empty, and the DELayout for the Grid is like this:
The following is a simplified UML diagram showing the structure of the State class. States can contain FieldBundles, Fields, Arrays, or nested States. See Appendix A, A Brief Introduction to UML, for a translation table that lists the symbols in the diagram and their meaning.
INTERFACE:
interface assignment(=) state1 = state2ARGUMENTS:
type(ESMF_State) :: state1 type(ESMF_State) :: state2STATUS:
DESCRIPTION:
Assign state1 as an alias to the same ESMF State object in memory as state2. If state2 is invalid, then state1 will be equally invalid after the assignment.
The arguments are:
INTERFACE:
interface operator(==) if (state1 == state2) then ... endif OR result = (state1 == state2)RETURN VALUE:
logical :: resultARGUMENTS:
type(ESMF_State), intent(in) :: state1 type(ESMF_State), intent(in) :: state2STATUS:
DESCRIPTION:
Test whether state1 and state2 are valid aliases to the same ESMF State object in memory. For a more general comparison of two ESMF States, going beyond the simple alias test, the ESMF_StateMatch() function (not yet implemented) must be used.
The arguments are:
INTERFACE:
interface operator(/=) if (state1 /= state2) then ... endif OR result = (state1 /= state2)RETURN VALUE:
logical :: resultARGUMENTS:
type(ESMF_State), intent(in) :: state1 type(ESMF_State), intent(in) :: state2STATUS:
DESCRIPTION:
Test whether state1 and state2 are not valid aliases to the same ESMF State object in memory. For a more general comparison of two ESMF States, going beyond the simple alias test, the ESMF_StateMatch() function (not yet implemented) must be used.
The arguments are:
INTERFACE:
subroutine ESMF_StateAdd(state, <itemList>, relaxedFlag, rc)ARGUMENTS:
type(ESMF_State), intent(inout) :: state <itemList>, see below for supported values -- The following arguments require argument keyword syntax (e.g. rc=rc). -- logical, intent(in), optional :: relaxedFlag integer, intent(out), optional :: rcSTATUS:
DESCRIPTION:
Add a list of items to a ESMF_State. It is an error if any item in <itemlist> already matches, by name, an item already contained in state.
Supported values for <itemList> are:
The arguments are:
INTERFACE:
subroutine ESMF_StateAddReplace(state, <itemList>, rc)ARGUMENTS:
type(ESMF_State), intent(inout) :: state <itemList>, see below for supported values -- The following arguments require argument keyword syntax (e.g. rc=rc). -- integer, intent(out), optional :: rcSTATUS:
DESCRIPTION:
Add or replace a list of items to an ESMF_State. If an item in <itemList> does not match any items already present in state, it is added. Items with names already present in the state replace the existing item.
Supported values for <itemList> are:
The arguments are:
INTERFACE:
function ESMF_StateCreate(stateintent, & arrayList, arraybundleList, & fieldList, fieldbundleList, & nestedStateList, & routehandleList, name, vm, rc)RETURN VALUE:
type(ESMF_State) :: ESMF_StateCreateARGUMENTS:
-- The following arguments require argument keyword syntax (e.g. rc=rc). -- type(ESMF_StateIntent_Flag), intent(in), optional :: stateintent type(ESMF_Array), intent(in), optional :: arrayList(:) type(ESMF_ArrayBundle), intent(in), optional :: arraybundleList(:) type(ESMF_Field), intent(in), optional :: fieldList(:) type(ESMF_FieldBundle), intent(in), optional :: fieldbundleList(:) type(ESMF_State), intent(in), optional :: nestedStateList(:) type(ESMF_RouteHandle), intent(in), optional :: routehandleList(:) character(len=*), intent(in), optional :: name type(ESMF_VM), intent(in), optional :: vm integer, intent(out), optional :: rcSTATUS:
DESCRIPTION:
Create a new ESMF_State, set default characteristics for objects added to it, and optionally add initial objects to it.
The arguments are:
INTERFACE:
recursive subroutine ESMF_StateDestroy(state, noGarbage, rc)ARGUMENTS:
type(ESMF_State), intent(inout) :: state -- The following arguments require argument keyword syntax (e.g. rc=rc). -- logical, intent(in), optional :: noGarbage integer, intent(out), optional :: rcSTATUS:
DESCRIPTION:
Releases resources associated with this ESMF_State. Actual objects added to ESMF_States will not be destroyed, it remains the responsibility of the user to destroy these objects in the correct context.
The arguments are:
It is generally recommended to leave the noGarbage argument set to .FALSE. (the default), and to take advantage of the ESMF garbage collection system which will prevent problems with dangling aliases or incorrect sequences of destroy calls. However this level of support requires that a small remnant of the object is kept in memory past the destroy call. This can lead to an unexpected increase in memory consumption over the course of execution in applications that use temporary ESMF objects. For situations where the repeated creation and destruction of temporary objects leads to memory issues, it is recommended to call with noGarbage set to .TRUE., fully removing the entire temporary object from memory.
INTERFACE:
! Private name; call using ESMF_StateGet() subroutine ESMF_StateGetInfo(state, & itemSearch, itemorderflag, nestedFlag, & stateintent, itemCount, itemNameList, itemTypeList, name, rc)ARGUMENTS:
type(ESMF_State), intent(in) :: state -- The following arguments require argument keyword syntax (e.g. rc=rc). -- character (len=*), intent(in), optional :: itemSearch type(ESMF_ItemOrder_Flag), intent(in), optional :: itemorderflag logical, intent(in), optional :: nestedFlag type(ESMF_StateIntent_Flag), intent(out), optional :: stateintent integer, intent(out), optional :: itemCount character (len=*), intent(out), optional :: itemNameList(:) type(ESMF_StateItem_Flag), intent(out), optional :: itemTypeList(:) character (len=*), intent(out), optional :: name integer, intent(out), optional :: rcSTATUS:
DESCRIPTION:
Returns the requested information about this ESMF_State. The optional itemSearch argument may specify the name of an individual item to search for. When used in conjunction with the nestedFlag, nested States will also be searched.
Typically, an ESMF_StateGet() information request will be performed twice. The first time, the itemCount argument will be used to query the size of arrays that are needed. Arrays can then be allocated to the correct size for itemNameList and itemtypeList as needed. A second call to ESMF_StateGet() will then fill in the values.
The arguments are:
INTERFACE:
! Private name; call using ESMF_StateGet() subroutine ESMF_StateGetItemInfo(state, itemName, itemType, rc)ARGUMENTS:
type(ESMF_State), intent(in) :: state character (len=*), intent(in) :: itemName type(ESMF_StateItem_Flag), intent(out) :: itemType -- The following arguments require argument keyword syntax (e.g. rc=rc). -- integer, intent(out), optional :: rcSTATUS:
DESCRIPTION:
Returns the type for the item named name in this ESMF_State. If no item with this name exists, the value ESMF_STATEITEM_NOTFOUND will be returned and the error code will not be set to an error. Thus this routine can be used to safely query for the existence of items by name whether or not they are expected to be there. The error code will be set in case of other errors, for example if the ESMF_State itself is invalid.
The arguments are:
INTERFACE:
subroutine ESMF_StateGet(state, itemName, <item>, rc)ARGUMENTS:
type(ESMF_State), intent(in) :: state character (len=*), intent(in) :: itemName <item>, see below for supported values -- The following arguments require argument keyword syntax (e.g. rc=rc). -- integer, intent(out), optional :: rcSTATUS:
DESCRIPTION:
Returns an <item> from an ESMF_State by item name. If the ESMF_State contains the <item> directly, only itemName is required.
If the state contains nested ESMF_States, the itemName argument may specify a fully qualified name to access the desired item with a single call. This is performed using the '/' character to separate the names of the intermediate State names leading to the desired item. (E.g., itemName='state1/state12/item').
Supported values for <item> are:
The arguments are:
INTERFACE:
function ESMF_StateIsCreated(state, rc)RETURN VALUE:
logical :: ESMF_StateIsCreatedARGUMENTS:
type(ESMF_State), intent(in) :: state -- The following arguments require argument keyword syntax (e.g. rc=rc). -- integer, intent(out), optional :: rcDESCRIPTION:
Return .true. if the state has been created. Otherwise return .false.. If an error occurs, i.e. rc /= ESMF_SUCCESS is returned, the return value of the function will also be .false..
The arguments are:
INTERFACE:
subroutine ESMF_StatePrint(state, options, nestedFlag, rc)ARGUMENTS:
type(ESMF_State), intent(in) :: state character(len=*), intent(in), optional :: options logical, intent(in), optional :: nestedFlag integer, intent(out), optional :: rcDESCRIPTION:
Prints information about the state to stdout.
The arguments are:
INTERFACE:
subroutine ESMF_StateRead(state, fileName, rc)ARGUMENTS:
type(ESMF_State), intent(inout) :: state character (len=*), intent(in) :: fileName integer, intent(out), optional :: rcDESCRIPTION:
Currently limited to read in all Arrays from a NetCDF file and add them to a State object. Future releases will enable more items of a State to be read from a file of various formats.
Only PET 0 reads the file; the States in other PETs remain empty. Currently, the data is not decomposed or distributed; each PET has only 1 DE and only PET 0 contains data after reading the file. Future versions of ESMF will support data decomposition and distribution upon reading a file. See Section 21.3.7 for an example.
Note that the third party NetCDF library must be installed. For more details, see the "ESMF Users Guide", "Building and Installing the ESMF, Third Party Libraries, NetCDF" and the website http://www.unidata.ucar.edu/software/netcdf.
The arguments are:
INTERFACE:
subroutine ESMF_StateReconcile(state, vm, rc)ARGUMENTS:
type(ESMF_State), intent(inout) :: state type(ESMF_VM), intent(in), optional :: vm integer, intent(out), optional :: rcDESCRIPTION:
Must be called for any ESMF_State which contains ESMF objects that have not been created on all the PETs of the currently running ESMF_Component. For example, if a coupler is operating on data which was created by another component that ran on only a subset of the couplers PETs, the coupler must make this call first before operating on any data inside that ESMF_State. After calling ESMF_StateReconcile all PETs will have a common view of all objects contained in this ESMF_State.
This call is collective across the specified VM.
The arguments are:
INTERFACE:
! Private name; call using ESMF_StateRemove () subroutine ESMF_StateRemoveOneItem (state, itemName, & relaxedFlag, rc)ARGUMENTS:
type(ESMF_State), intent(inout) :: state character(*), intent(in) :: itemName -- The following arguments require argument keyword syntax (e.g. rc=rc). -- logical, intent(in), optional :: relaxedFlag integer, intent(out), optional :: rcSTATUS:
DESCRIPTION:
Remove an existing reference to an item from a State.
The arguments are:
If the state contains nested ESMF_States, the itemName argument may specify a fully qualified name to remove the desired item with a single call. This is performed using the "/" character to separate the names of the intermediate State names leading to the desired item. (E.g., itemName="state1/state12/item".
Since an item could potentially be referenced by multiple containers, it remains the responsibility of the user to manage its destruction when it is no longer in use.
INTERFACE:
! Private name; call using ESMF_StateRemove () subroutine ESMF_StateRemoveList (state, itemNameList, relaxedFlag, rc)ARGUMENTS:
type(ESMF_State), intent(inout) :: state character(*), intent(in) :: itemNameList(:) -- The following arguments require argument keyword syntax (e.g. rc=rc). -- logical, intent(in), optional :: relaxedFlag integer, intent(out), optional :: rcSTATUS:
DESCRIPTION:
Remove existing references to items from a State.
The arguments are:
If the state contains nested ESMF_States, the itemName arguments may specify fully qualified names to remove the desired items with a single call. This is performed using the "/" character to separate the names of the intermediate State names leading to the desired items. (E.g., itemName="state1/state12/item".
Since items could potentially be referenced by multiple containers, it remains the responsibility of the user to manage their destruction when they are no longer in use.
INTERFACE:
subroutine ESMF_StateReplace(state, <itemList>, relaxedflag, rc)ARGUMENTS:
type(ESMF_State), intent(inout) :: state <itemList>, see below for supported values -- The following arguments require argument keyword syntax (e.g. rc=rc). -- logical, intent(in), optional :: relaxedflag integer, intent(out), optional :: rcSTATUS:
DESCRIPTION:
Replace a list of items with a ESMF_State. If an item in <itemList> does not match any items already present in state, an error is returned.
Supported values for <itemList> are:
The arguments are:
INTERFACE:
subroutine ESMF_StateSet(state, stateIntent, name, rc)ARGUMENTS:
type(ESMF_State), intent(inout) :: state -- The following arguments require argument keyword syntax (e.g. rc=rc). -- type(ESMF_StateIntent_Flag), intent(in), optional :: stateIntent character(len = *), intent(in), optional :: name integer, intent(out), optional :: rcDESCRIPTION:
Set the info in the state object.
The arguments are:
INTERFACE:
subroutine ESMF_StateValidate(state, nestedFlag, rc)ARGUMENTS:
type(ESMF_State), intent(in) :: state -- The following arguments require argument keyword syntax (e.g. rc=rc). -- logical, intent(in), optional :: nestedFlag integer, intent(out), optional :: rcSTATUS:
DESCRIPTION:
Validates that the state is internally consistent. Currently this method determines if the State is uninitialized or already destroyed. The method returns an error code if problems are found.
The arguments are:
INTERFACE:
subroutine ESMF_StateWrite(state, fileName, rc)ARGUMENTS:
type(ESMF_State), intent(in) :: state character (len=*), intent(in) :: fileName integer, intent(out), optional :: rcDESCRIPTION:
Currently limited to write out all Arrays of a State object to a netCDF file. Future releases will enable more item types of a State to be written to files of various formats.
Writing is currently limited to PET 0; future versions of ESMF will allow parallel writing, as well as parallel reading.
See Section 21.3.7 for an example.
Note that the third party NetCDF library must be installed. For more details, see the "ESMF Users Guide", "Building and Installing the ESMF, Third Party Libraries, NetCDF" and the website http://www.unidata.ucar.edu/software/netcdf.
The arguments are:
ESMF allows user methods to be attached to Components and States. Providing this capability supports a more object oriented way of model design.
Attachable methods on Components can be used to implement the concept of generic Components where the specialization requires attaching methods with well defined names. This methods are then called by the generic Component code.
Attaching methods to States can be used to supply data operations along with the data objects inside of a State object. This can be useful where a producer Component not only supplies a data set, but also the associated processing functionality. This can be more efficient than providing all of the possible sets of derived data.
The following examples demonstrate how a producer Component attaches a user defined method to a State, and how it implements the method. The attached method is then executed by the consumer Component.
The producer Component attaches a user defined method to exportState during the Component's initialize method. The user defined method is attached with label finalCalculation by which it will become accessible to the consumer Component.
subroutine init(gcomp, importState, exportState, clock, rc) ! arguments type(ESMF_GridComp):: gcomp type(ESMF_State):: importState, exportState type(ESMF_Clock):: clock integer, intent(out):: rc rc = ESMF_SUCCESS call ESMF_MethodAdd(exportState, label="finalCalculation", & userRoutine=finalCalc, rc=rc) if (rc /= ESMF_SUCCESS) return ! just for testing purposes add the same method with a crazy string label call ESMF_MethodAdd(exportState, label="Somewhat of a SILLY @$^@_ label", & userRoutine=finalCalc, rc=rc) if (rc /= ESMF_SUCCESS) return end subroutine !--------------------------------------------------------------
The producer Component implements the attached, user defined method finalCalc. Strict interface rules apply for the user defined method.
subroutine finalCalc(state, rc) ! arguments type(ESMF_State):: state integer, intent(out):: rc rc = ESMF_SUCCESS ! access data objects in state and perform calculation print *, "dummy output from attached method " end subroutine !--------------------------------------------------------------
The consumer Component executes the user defined method on the importState.
subroutine init(gcomp, importState, exportState, clock, rc) ! arguments type(ESMF_GridComp):: gcomp type(ESMF_State):: importState, exportState type(ESMF_Clock):: clock integer, intent(out):: rc integer:: userRc, i logical:: isPresent character(len=:), allocatable :: labelList(:) rc = ESMF_SUCCESS
The importState can be queried for a list of all the attached methods.
call ESMF_MethodGet(importState, labelList=labelList, rc=rc) if (rc /= ESMF_SUCCESS) return ! print the labels do i=1, size(labelList) print *, labelList(i) enddo
It is also possible to check the importState whether a specific method is attached. This allows the consumer code to implement alternatives in case the method is not available.
call ESMF_MethodGet(importState, label="finalCalculation", & isPresent=isPresent, rc=rc) if (rc /= ESMF_SUCCESS) return
Finally call into the attached method from the consumer side.
call ESMF_MethodExecute(importState, label="finalCalculation", & userRc=userRc, rc=rc) if (rc /= ESMF_SUCCESS) return rc = userRc if (rc /= ESMF_SUCCESS) return end subroutine !--------------------------------------------------------------
INTERFACE:
! Private name; call using ESMF_MethodAdd() subroutine ESMF_MethodCplCompAdd(cplcomp, label, index, userRoutine, rc)ARGUMENTS:
type(ESMF_CplComp) :: cplcomp character(len=*), intent(in) :: label integer, intent(in), optional :: index interface subroutine userRoutine(cplcomp, rc) use ESMF_CompMod implicit none type(ESMF_CplComp) :: cplcomp ! must not be optional integer, intent(out) :: rc ! must not be optional end subroutine end interface integer, intent(out), optional :: rcDESCRIPTION:
Attach userRoutine. Error out if there is a previous attached method under the same label and index.
The arguments are:
The subroutine must have the exact interface shown above for the userRoutine argument. Arguments in userRoutine must not be declared as optional, and the types, intent and order must match. Prior to Fortran-2008, the subroutine must be either a module scope procedure, or an external procedure that has a matching interface block specified for it. An internal procedure which is contained within another procedure must not be used. From Fortran-2008 onwards, an internal procedure contained within either a main program or a module procedure may be used. If the internal procedure is contained within a module procedure, it is subject to initialization requirements. See: 16.4.9
INTERFACE:
! Private name; call using ESMF_MethodAdd() subroutine ESMF_MethodCplCompAddShObj(cplcomp, label, index, userRoutine, & sharedObj, rc)ARGUMENTS:
type(ESMF_CplComp) :: cplcomp character(len=*), intent(in) :: label integer, intent(in), optional :: index character(len=*), intent(in) :: userRoutine character(len=*), intent(in), optional :: sharedObj integer, intent(out), optional :: rcDESCRIPTION:
Attach userRoutine. Error out if there is a previous attached method under the same label and index.
The arguments are:
The subroutine must have the exact interface shown in ESMF_MethodCplCompAdd for the userRoutine argument. Arguments in userRoutine must not be declared as optional, and the types, intent and order must match. Prior to Fortran-2008, the subroutine must be either a module scope procedure, or an external procedure that has a matching interface block specified for it. An internal procedure which is contained within another procedure must not be used. From Fortran-2008 onwards, an internal procedure contained within either a main program or a module procedure may be used. If the internal procedure is contained within a module procedure, it is subject to initialization requirements. See: 16.4.9
INTERFACE:
! Private name; call using ESMF_MethodAdd() subroutine ESMF_MethodGridCompAdd(gcomp, label, index, userRoutine, rc)ARGUMENTS:
type(ESMF_GridComp) :: gcomp character(len=*), intent(in) :: label integer, intent(in), optional :: index interface subroutine userRoutine(gcomp, rc) use ESMF_CompMod implicit none type(ESMF_GridComp) :: gcomp ! must not be optional integer, intent(out) :: rc ! must not be optional end subroutine end interface integer, intent(out), optional :: rcDESCRIPTION:
Attach userRoutine. Error out if there is a previous attached method under the same label and index.
The arguments are:
The subroutine must have the exact interface shown above for the userRoutine argument. Arguments in userRoutine must not be declared as optional, and the types, intent and order must match. Prior to Fortran-2008, the subroutine must be either a module scope procedure, or an external procedure that has a matching interface block specified for it. An internal procedure which is contained within another procedure must not be used. From Fortran-2008 onwards, an internal procedure contained within either a main program or a module procedure may be used. If the internal procedure is contained within a module procedure, it is subject to initialization requirements. See: 16.4.9
INTERFACE:
! Private name; call using ESMF_MethodAdd() subroutine ESMF_MethodGridCompAddShObj(gcomp, label, index, userRoutine, & sharedObj, rc)ARGUMENTS:
type(ESMF_GridComp) :: gcomp character(len=*), intent(in) :: label integer, intent(in), optional :: index character(len=*), intent(in) :: userRoutine character(len=*), intent(in), optional :: sharedObj integer, intent(out), optional :: rcDESCRIPTION:
Attach userRoutine. Error out if there is a previous attached method under the same label and index.
The arguments are:
The subroutine must have the exact interface shown in ESMF_MethodGridCompAdd for the userRoutine argument. Arguments in userRoutine must not be declared as optional, and the types, intent and order must match. Prior to Fortran-2008, the subroutine must be either a module scope procedure, or an external procedure that has a matching interface block specified for it. An internal procedure which is contained within another procedure must not be used. From Fortran-2008 onwards, an internal procedure contained within either a main program or a module procedure may be used. If the internal procedure is contained within a module procedure, it is subject to initialization requirements. See: 16.4.9
INTERFACE:
! Private name; call using ESMF_MethodAdd() subroutine ESMF_MethodStateAdd(state, label, index, userRoutine, rc)ARGUMENTS:
type(ESMF_State) :: state character(len=*), intent(in) :: label integer, intent(in), optional :: index interface subroutine userRoutine(state, rc) use ESMF_StateMod implicit none type(ESMF_State) :: state ! must not be optional integer, intent(out) :: rc ! must not be optional end subroutine end interface integer, intent(out), optional :: rcDESCRIPTION:
Attach userRoutine. Error out if there is a previous attached method under the same label and index.
The arguments are:
The subroutine must have the exact interface shown above for the userRoutine argument. Arguments in userRoutine must not be declared as optional, and the types, intent and order must match. Prior to Fortran-2008, the subroutine must be either a module scope procedure, or an external procedure that has a matching interface block specified for it. An internal procedure which is contained within another procedure must not be used. From Fortran-2008 onwards, an internal procedure contained within either a main program or a module procedure may be used. If the internal procedure is contained within a module procedure, it is subject to initialization requirements. See: 16.4.9
INTERFACE:
! Private name; call using ESMF_MethodAdd() subroutine ESMF_MethodStateAddShObj(state, label, index, userRoutine, & sharedObj, rc)ARGUMENTS:
type(ESMF_State) :: state character(len=*), intent(in) :: label integer, intent(in), optional :: index character(len=*), intent(in) :: userRoutine character(len=*), intent(in), optional :: sharedObj integer, intent(out), optional :: rcDESCRIPTION:
Attach userRoutine. Error out if there is a previous attached method under the same label and index.
The arguments are:
The subroutine must have the exact interface shown in ESMF_MethodStateAdd for the userRoutine argument. Arguments in userRoutine must not be declared as optional, and the types, intent and order must match. Prior to Fortran-2008, the subroutine must be either a module scope procedure, or an external procedure that has a matching interface block specified for it. An internal procedure which is contained within another procedure must not be used. From Fortran-2008 onwards, an internal procedure contained within either a main program or a module procedure may be used. If the internal procedure is contained within a module procedure, it is subject to initialization requirements. See: 16.4.9
INTERFACE:
! Private name; call using ESMF_MethodAddReplace() subroutine ESMF_MethodCplCompAddRep(cplcomp, label, index, userRoutine, rc)ARGUMENTS:
type(ESMF_CplComp) :: cplcomp character(len=*), intent(in) :: label integer, intent(in), optional :: index interface subroutine userRoutine(cplcomp, rc) use ESMF_CompMod implicit none type(ESMF_CplComp) :: cplcomp ! must not be optional integer, intent(out) :: rc ! must not be optional end subroutine end interface integer, intent(out), optional :: rcDESCRIPTION:
Attach userRoutine. Replacing potential previous attached method under the same label and index.
The arguments are:
The subroutine must have the exact interface shown above for the userRoutine argument. Arguments in userRoutine must not be declared as optional, and the types, intent and order must match. Prior to Fortran-2008, the subroutine must be either a module scope procedure, or an external procedure that has a matching interface block specified for it. An internal procedure which is contained within another procedure must not be used. From Fortran-2008 onwards, an internal procedure contained within either a main program or a module procedure may be used. If the internal procedure is contained within a module procedure, it is subject to initialization requirements. See: 16.4.9
INTERFACE:
! Private name; call using ESMF_MethodAddReplace() subroutine ESMF_MethodCplCompAddRepShObj(cplcomp, label, index, userRoutine, & sharedObj, rc)ARGUMENTS:
type(ESMF_CplComp) :: cplcomp character(len=*), intent(in) :: label integer, intent(in), optional :: index character(len=*), intent(in) :: userRoutine character(len=*), intent(in), optional :: sharedObj integer, intent(out), optional :: rcDESCRIPTION:
Attach userRoutine. Replacing potential previous attached method under the same label and index.
The arguments are:
The subroutine must have the exact interface shown in ESMF_MethodCplCompAdd for the userRoutine argument. Arguments in userRoutine must not be declared as optional, and the types, intent and order must match. Prior to Fortran-2008, the subroutine must be either a module scope procedure, or an external procedure that has a matching interface block specified for it. An internal procedure which is contained within another procedure must not be used. From Fortran-2008 onwards, an internal procedure contained within either a main program or a module procedure may be used. If the internal procedure is contained within a module procedure, it is subject to initialization requirements. See: 16.4.9
INTERFACE:
! Private name; call using ESMF_MethodAddReplace() subroutine ESMF_MethodGridCompAddRep(gcomp, label, index, userRoutine, rc)ARGUMENTS:
type(ESMF_GridComp) :: gcomp character(len=*), intent(in) :: label integer, intent(in), optional :: index interface subroutine userRoutine(gcomp, rc) use ESMF_CompMod implicit none type(ESMF_GridComp) :: gcomp ! must not be optional integer, intent(out) :: rc ! must not be optional end subroutine end interface integer, intent(out), optional :: rcDESCRIPTION:
Attach userRoutine. Replacing potential previous attached method under the same label and index.
The arguments are:
The subroutine must have the exact interface shown above for the userRoutine argument. Arguments in userRoutine must not be declared as optional, and the types, intent and order must match. Prior to Fortran-2008, the subroutine must be either a module scope procedure, or an external procedure that has a matching interface block specified for it. An internal procedure which is contained within another procedure must not be used. From Fortran-2008 onwards, an internal procedure contained within either a main program or a module procedure may be used. If the internal procedure is contained within a module procedure, it is subject to initialization requirements. See: 16.4.9
INTERFACE:
! Private name; call using ESMF_MethodAddReplace() subroutine ESMF_MethodGridCompAddRepShObj(gcomp, label, index, userRoutine, & sharedObj, rc)ARGUMENTS:
type(ESMF_GridComp) :: gcomp character(len=*), intent(in) :: label integer, intent(in), optional :: index character(len=*), intent(in) :: userRoutine character(len=*), intent(in), optional :: sharedObj integer, intent(out), optional :: rcDESCRIPTION:
Attach userRoutine. Replacing potential previous attached method under the same label and index.
The arguments are:
The subroutine must have the exact interface shown in ESMF_MethodGridCompAdd for the userRoutine argument. Arguments in userRoutine must not be declared as optional, and the types, intent and order must match. Prior to Fortran-2008, the subroutine must be either a module scope procedure, or an external procedure that has a matching interface block specified for it. An internal procedure which is contained within another procedure must not be used. From Fortran-2008 onwards, an internal procedure contained within either a main program or a module procedure may be used. If the internal procedure is contained within a module procedure, it is subject to initialization requirements. See: 16.4.9
INTERFACE:
! Private name; call using ESMF_MethodAddReplace() subroutine ESMF_MethodStateAddRep(state, label, index, userRoutine, rc)ARGUMENTS:
type(ESMF_State) :: state character(len=*), intent(in) :: label integer, intent(in), optional :: index interface subroutine userRoutine(state, rc) use ESMF_StateMod implicit none type(ESMF_State) :: state ! must not be optional integer, intent(out) :: rc ! must not be optional end subroutine end interface integer, intent(out), optional :: rcDESCRIPTION:
Attach userRoutine. Replacing potential previous attached method under the same label and index.
The arguments are:
The subroutine must have the exact interface shown above for the userRoutine argument. Arguments in userRoutine must not be declared as optional, and the types, intent and order must match. Prior to Fortran-2008, the subroutine must be either a module scope procedure, or an external procedure that has a matching interface block specified for it. An internal procedure which is contained within another procedure must not be used. From Fortran-2008 onwards, an internal procedure contained within either a main program or a module procedure may be used. If the internal procedure is contained within a module procedure, it is subject to initialization requirements. See: 16.4.9
INTERFACE:
! Private name; call using ESMF_MethodAddReplace() subroutine ESMF_MethodStateAddRepShObj(state, label, index, userRoutine, & sharedObj, rc)ARGUMENTS:
type(ESMF_State) :: state character(len=*), intent(in) :: label integer, intent(in), optional :: index character(len=*), intent(in) :: userRoutine character(len=*), intent(in), optional :: sharedObj integer, intent(out), optional :: rcDESCRIPTION:
Attach userRoutine. Replacing potential previous attached method under the same label and index.
The arguments are:
The subroutine must have the exact interface shown in ESMF_MethodStateAdd for the userRoutine argument. Arguments in userRoutine must not be declared as optional, and the types, intent and order must match. Prior to Fortran-2008, the subroutine must be either a module scope procedure, or an external procedure that has a matching interface block specified for it. An internal procedure which is contained within another procedure must not be used. From Fortran-2008 onwards, an internal procedure contained within either a main program or a module procedure may be used. If the internal procedure is contained within a module procedure, it is subject to initialization requirements. See: 16.4.9
INTERFACE:
! Private name; call using ESMF_MethodExecute() recursive subroutine ESMF_MethodCplCompExecute(cplcomp, label, index, existflag, & userRc, rc)ARGUMENTS:
type(ESMF_CplComp) :: cplcomp character(len=*), intent(in) :: label integer, intent(in), optional :: index logical, intent(out), optional :: existflag integer, intent(out), optional :: userRc integer, intent(out), optional :: rcDESCRIPTION:
Execute attached method.
The arguments are:
INTERFACE:
! Private name; call using ESMF_MethodExecute() recursive subroutine ESMF_MethodGridCompExecute(gcomp, label, index, existflag, & userRc, rc)ARGUMENTS:
type(ESMF_GridComp) :: gcomp character(len=*), intent(in) :: label integer, intent(in), optional :: index logical, intent(out), optional :: existflag integer, intent(out), optional :: userRc integer, intent(out), optional :: rcDESCRIPTION:
Execute attached method.
The arguments are:
INTERFACE:
! Private name; call using ESMF_MethodExecute() recursive subroutine ESMF_MethodStateExecute(state, label, index, existflag, & userRc, rc)ARGUMENTS:
type(ESMF_State) :: state character(len=*), intent(in) :: label integer, intent(in), optional :: index logical, intent(out), optional :: existflag integer, intent(out), optional :: userRc integer, intent(out), optional :: rcDESCRIPTION:
Execute attached method.
The arguments are:
INTERFACE:
! Private name; call using ESMF_MethodGet() subroutine ESMF_MethodCplCompGet(cplcomp, label, index, isPresent, rc)ARGUMENTS:
type(ESMF_CplComp) :: cplcomp character(len=*), intent(in) :: label integer, intent(in), optional :: index logical, intent(out), optional :: isPresent integer, intent(out), optional :: rcDESCRIPTION:
Access information about attached method.
The arguments are:
INTERFACE:
! Private name; call using ESMF_MethodGet() subroutine ESMF_MethodCplCompGetList(cplcomp, labelList, rc)ARGUMENTS:
type(ESMF_CplComp) :: cplcomp character(len=:), allocatable, intent(out) :: labelList(:) integer, intent(out), optional :: rcDESCRIPTION:
Access labels of all attached methods.
The arguments are:
INTERFACE:
! Private name; call using ESMF_MethodGet() subroutine ESMF_MethodGridCompGet(gcomp, label, index, isPresent, rc)ARGUMENTS:
type(ESMF_GridComp) :: gcomp character(len=*), intent(in) :: label integer, intent(in), optional :: index logical, intent(out), optional :: isPresent integer, intent(out), optional :: rcDESCRIPTION:
Access information about attached method.
The arguments are:
INTERFACE:
! Private name; call using ESMF_MethodGet() subroutine ESMF_MethodGridCompGetList(gcomp, labelList, rc)ARGUMENTS:
type(ESMF_GridComp) :: gcomp character(len=:), allocatable, intent(out) :: labelList(:) integer, intent(out), optional :: rcDESCRIPTION:
Access labels of all attached methods.
The arguments are:
INTERFACE:
! Private name; call using ESMF_MethodGet() subroutine ESMF_MethodStateGet(state, label, index, isPresent, rc)ARGUMENTS:
type(ESMF_State) :: state character(len=*), intent(in) :: label integer, intent(in), optional :: index logical, intent(out), optional :: isPresent integer, intent(out), optional :: rcDESCRIPTION:
Access information about attached method.
The arguments are:
INTERFACE:
! Private name; call using ESMF_MethodGet() subroutine ESMF_MethodStateGetList(state, labelList, rc)ARGUMENTS:
type(ESMF_State) :: state character(len=:), allocatable, intent(out) :: labelList(:) integer, intent(out), optional :: rcDESCRIPTION:
Access labels of all attached methods.
The arguments are:
INTERFACE:
! Private name; call using ESMF_MethodRemove() subroutine ESMF_MethodCplCompRemove(cplcomp, label, index, rc)ARGUMENTS:
type(ESMF_CplComp) :: cplcomp character(len=*), intent(in) :: label integer, intent(in), optional :: index integer, intent(out), optional :: rcDESCRIPTION:
Remove attached method.
The arguments are:
INTERFACE:
! Private name; call using ESMF_MethodRemove() subroutine ESMF_MethodGridCompRemove(gcomp, label, index, rc)ARGUMENTS:
type(ESMF_GridComp) :: gcomp character(len=*), intent(in) :: label integer, intent(in), optional :: index integer, intent(out), optional :: rcDESCRIPTION:
Remove attached method.
The arguments are:
INTERFACE:
! Private name; call using ESMF_MethodRemove() subroutine ESMF_MethodStateRemove(state, label, index, rc)ARGUMENTS:
type(ESMF_State) :: state character(len=*), intent(in) :: label integer, intent(in), optional :: index integer, intent(out), optional :: rcDESCRIPTION:
Remove attached method.
The arguments are:
The goal of the ESMF Web Services is to provide the tools to allow ESMF Users to make their Components available via a web service. The first step is to make the Component a service, and then make it accessible via the Web.
At the heart of this architecture is the Component Service; this is the application that does the model work. The ESMF Web Services part provides a way to make the model accessible via a network API (Application Programming Interface). ESMF provides the tools to turn a model component into a service as well as the tools to access the service from the network.
The Process Controller is a stand-alone application that provides a control mechanism between the end user and the Component Service. The Process Controller is responsible for managing client information as well as restricting client access to a Component Service. (The role of the Process Controller is expected to expand in the future.)
The tomcat/axis2 application provides the access via the Web using standard SOAP protocols. Part of this application includes the SOAP interface definition (using a WSDL file) as well as some java code that provides the access to the Process Controller application.
Finally, the Registrar maintains a list of Component Services that are currently available; Component Services register themselves with the Registrar when they startup, and unregister themselves when they shutdown. The list of available services is maintained in an XML file and is accessible from the Registrar using its network API.
The primary function in ESMF Web Services is the ESMF_WebServicesLoop routine. This function registers the Component Service with the Registrar and then sets up a network socket service that listens for requests from a client. It starts a loop that waits for incoming requests and manages the routing of these requests to all PETs. It is also responsible for making sure the appropriate ESMF routine (ESMF_Initialize, ESMF_Run or ESMF_Finalize) is called based on the incoming request. When the client has completed its interaction with the Component Service, the loop will be terminated and it will unregister the Component Service from the Registrar.
To make all of this happen, the Application Driver just needs to replace its calls to ESMF_Initialize, ESMF_Run, and ESMF_Finalize with a single call to ESMF_WebServicesLoop.
use ESMF_WebServMod .... call ESMF_WebServicesLoop(gridComponent, portNumber, returnCode)
That's all there is to turning an ESMF Component into a network-accessible ESMF Component Service. For a detailed example of an ESMF Component turned into an ESMF Component Service, see the Examples in the Web Services section of the Developer' Guide.
However, the goal of ESMF Web Services is to make an ESMF Component accessible through a standard web service, which is accomplished through the Process Controller and the Tomcat/Axis2 applications
Interfacing to a Component service is fairly simple using the ESMF library. The following code is a simple example of how to interface to a Component Service in C++ and request the initialize operation (the entire sample client can be found in the Web Services examples section of the ESMF Distribution):
#include "ESMCI_WebServCompSvrClient.h" int main(int argc, char* argv[]) { int portNum = 27060; int clientId = 101; int rc = ESMF_SUCCESS; ESMCI::ESMCI_WebServCompSvrClient client("localhost", portNum, clientId); rc = client.init(); printf("Initialize return code: %d\n", rc); }
To see a complete description of the NetEsmfClient class, refer to the netesmf library section of the Web Services Reference Manual.
The Process Controller is basically just a instance of a C++ client application. It manages client access to the Component Service (only 1 client can access the service at a time), and will eventually be responsible for starting up and shutting down instances of Component Services (planned for a future release). The Process Controller application is built with the ESMF library and is included in the apps section of the distribution.
The Tomcat/Axis2 "application" is essentially the Apache Tomcat server using the Apache Axis2 servlet to implement web services using SOAP protocols. The web interface is defined by a WSDL file, and its implementation is handled by the Component Connector java code. Tomcat and Axis2 are both open source projects that should be downloaded from the Apache web site, but the WSDL file, the Component Connector java code, and all required software for supporting the interface can be found next to the ESMF distribution in the web_services_server directory. This code is not included with the ESMF distribution because they can be distributed and installed independent of each other.
The following examples demonstrate how to use ESMF Web Services.
In this example, a standard ESMF Component is made available through the Web Services interface.
The first step is to make sure your callback routines for initialize, run and finalize are setup. This is done by creating a register routine that sets the entry points for each of these callbacks. In this example, we've packaged it all up into a separate module.
module ESMF_WebServUserModel ! ESMF Framework module use ESMF implicit none public ESMF_WebServUserModelRegister contains !------------------------------------------------------------------------- ! The Registration routine ! subroutine ESMF_WebServUserModelRegister(comp, rc) type(ESMF_GridComp) :: comp integer, intent(out) :: rc ! Initialize return code rc = ESMF_SUCCESS print *, "User Comp1 Register starting" ! Register the callback routines. call ESMF_GridCompSetEntryPoint(comp, ESMF_METHOD_INITIALIZE, & userRoutine=user_init, rc=rc) if (rc/=ESMF_SUCCESS) return ! bail out call ESMF_GridCompSetEntryPoint(comp, ESMF_METHOD_RUN, & userRoutine=user_run, rc=rc) if (rc/=ESMF_SUCCESS) return ! bail out call ESMF_GridCompSetEntryPoint(comp, ESMF_METHOD_FINALIZE, & userRoutine=user_final, rc=rc) if (rc/=ESMF_SUCCESS) return ! bail out print *, "Registered Initialize, Run, and Finalize routines" print *, "User Comp1 Register returning" end subroutine !------------------------------------------------------------------------- ! The Initialization routine ! subroutine user_init(comp, importState, exportState, clock, rc) type(ESMF_GridComp) :: comp type(ESMF_State) :: importState, exportState type(ESMF_Clock) :: clock integer, intent(out) :: rc ! Initialize return code rc = ESMF_SUCCESS print *, "User Comp1 Init" end subroutine user_init !------------------------------------------------------------------------- ! The Run routine ! subroutine user_run(comp, importState, exportState, clock, rc) type(ESMF_GridComp) :: comp type(ESMF_State) :: importState, exportState type(ESMF_Clock) :: clock integer, intent(out) :: rc ! Initialize return code rc = ESMF_SUCCESS print *, "User Comp1 Run" end subroutine user_run !------------------------------------------------------------------------- ! The Finalization routine ! subroutine user_final(comp, importState, exportState, clock, rc) type(ESMF_GridComp) :: comp type(ESMF_State) :: importState, exportState type(ESMF_Clock) :: clock integer, intent(out) :: rc ! Initialize return code rc = ESMF_SUCCESS print *, "User Comp1 Final" end subroutine user_final end module ESMF_WebServUserModel
The actual driver code then becomes very simple; ESMF is initialized, the component is created, the callback functions for the component are registered, and the Web Service loop is started.
program WebServicesEx #include "ESMF.h" ! ESMF Framework module use ESMF use ESMF_TestMod use ESMF_WebServMod use ESMF_WebServUserModel implicit none ! Local variables type(ESMF_GridComp) :: comp1 !! Grid Component integer :: rc !! Return Code integer :: finalrc !! Final return code integer :: portNum !! The port number for the listening socket
A listening socket will be created on the local machine with the specified port number. This socket is used by the service to wait for and receive requests from the client. Check with your system administrator to determine an appropriate port to use for your service.
finalrc = ESMF_SUCCESS call ESMF_Initialize(defaultlogfilename="WebServicesEx.Log", & logkindflag=ESMF_LOGKIND_MULTI, rc=rc)
! create the grid component comp1 = ESMF_GridCompCreate(name="My Component", rc=rc)
! Set up the register routine call ESMF_GridCompSetServices(comp1, & userRoutine=ESMF_WebServUserModelRegister, rc=rc)
portNum = 27060 ! Call the Web Services Loop and wait for requests to come in !call ESMF_WebServicesLoop(comp1, portNum, rc=rc)
The call to ESMF_WebServicesLoop will setup the listening socket for your service and will wait for requests from a client. As requests are received, the Web Services software will process the requests and then return to the loop to continue to wait.
The 3 main requests processed are INIT, RUN, and FINAL. These requests will then call the appropriate callback routine as specified in your register routine (as specified in the ESMF_GridCompSetServices call). In this example, when the INIT request is received, the user_init routine found in the ESMF_WebServUserModel module is called.
One other request is also processed by the Component Service, and that is the EXIT request. When this request is received, the Web Services loop is terminated and the remainder of the code after the ESMF_WebServicesLoop call is executed.
call ESMF_Finalize(rc=rc)
end program WebServicesEx
INTERFACE:
subroutine ESMF_WebServicesLoop(comp, portNum, clientId, registrarHost, rc)ARGUMENTS:
type(ESMF_GridComp) :: comp integer, intent(inout), optional :: portNum character(len=*), intent(in), optional, target :: clientId character(len=*), intent(in), optional, target :: registrarHost integer, intent(out), optional :: rcDESCRIPTION:
Encapsulates all of the functionality necessary to setup a component as a component service. On the root PET, it registers the component service and then enters into a loop that waits for requests on a socket. The loop continues until an "exit" request is received, at which point it exits the loop and unregisters the service. On any PET other than the root PET, it sets up a process block that waits for instructions from the root PET. Instructions will come as requests are received from the socket.
The arguments are:
INTERFACE:
subroutine ESMF_WebServicesCplCompLoop(comp, portNum, clientId, registrarHost, rc)ARGUMENTS:
type(ESMF_CplComp) :: comp integer, intent(inout), optional :: portNum character(len=*), intent(in), optional, target :: clientId character(len=*), intent(in), optional, target :: registrarHost integer, intent(out), optional :: rcDESCRIPTION:
Encapsulates all of the functionality necessary to setup a component as a component service. On the root PET, it registers the component service and then enters into a loop that waits for requests on a socket. The loop continues until an "exit" request is received, at which point it exits the loop and unregisters the service. On any PET other than the root PET, it sets up a process block that waits for instructions from the root PET. Instructions will come as requests are received from the socket.
The arguments are:
esmf_support@ucar.edu