2 Design and Implementation Notes

The NUOPC Layer is implemented in Fortran on top of the public ESMF Fortran API.

The NUOPC utility routines form a very straight forward Fortran API, accessible through the NUOPC Fortran module. The interfaces only use native Fortran types and public ESMF derived types. In order to access the utility API of the NUOPC Layer, user code must include the following two use lines:

  use ESMF
  use NUOPC

2.1 Generic Components

The NUOPC generic components are implemented as a collection of Fortran modules. Each module implements a single, well specified set of standard ESMF_GridComp or ESMF_CplComp methods. The nomenclature of the generic component modules starts with the NUOPC_ prefix and continues with the flavor: Driver, Model, Mediator, or Connector. This is optionally followed by a string of additional descriptive terms. The four flavors of generic components implemented by the NUOPC Layer are:

• NUOPC_Driver - A generic driver component. It implements a child component harness, made of State and Component objects, that follows the NUOPC Common Model Architecture. It is specialized by plugging Model, Mediator, and Connector components into the harness. Driver components can be plugged into the harness to construct component hierarchies. The generic Driver initializes its child components according to a standard Initialization Phase Definition, and drives their Run() methods according a customizable run sequence.

• NUOPC_Model - A generic model component that wraps a model code so it is suitable to be plugged into a generic Driver component.

• NUOPC_Mediator - A generic mediator component that wraps custom coupling code (flux calculations, averaging, etc.) so it is suitable to be plugged into a generic Driver component.

• NUOPC_Connector - A generic component that implements Field matching based on metadata and executes simple transforms (Regrid and Redist). It can be plugged into a generic Driver component.

The user code accesses the desired generic component(s) by including a use line for each one. Each generic component defines a small set of public names that are made available to the user code through the use statement. At a minimum the SetServices method is made public. Some of the generic components define additional public routines and labels as part of their user interface. It is recommended to rename the entries of an imported generic component module in the local scope as part of the use association. This prevents name clashes.

  use NUOPC_<GenericComp>, only: &
    <GenericComp>_SS      => SetServices, &
    <GenericComp>_labelA  => labelA

A generic component is used by user code to implement a specialized version of the generic component. The user component derives from the generic component code by implementing its own public SetServices routine that calls into the generic SetServices routine before doing anything else. It is through this mechanism that the deriving component inherits functionality that is implemented in the generic component. The example shows how a specific model component is implemented to derive from the generic NUOPC_Model:

  use NUOPC_Model, only: &
    model_SS => SetServices

  subroutine SetServices(model, rc)
    type(ESMF_GridComp)  :: model
    integer, intent(out) :: rc

    ! derive this specific "model" component from generic NUOPC_Model
    call NUOPC_CompDerive(model, model_SS, rc=rc)

    ! specializing code for "model" to follow
    
  end subroutine

There are three mechanisms through which user code specializes generic components. The first two methods specialize through call-back mechanisms into user implemented routines. The third method specializes by setting the values of parameters implemented by the generic component.

1. The specializing user code sets entry points for standard component methods not implemented by the generic component by calling NUOPC_CompSetEntryPoint(). Methods (and phases) that need to be implemented are clearly documented in the generic component description. The user code may further overwrite standard methods already implemented by the generic component code. However, this should rarely be necessary, and may indicate that there is a better fitting generic component available. Finally, some generic components come with generic routines that are suitable candidates for the standard component methods, yet require that the specializing code registers them as appropriate. Setting entry points for standard component methods should be done in the SetServices() routine right after NUOPC_CompDerive() is called.

2. Some generic components require that specific methods are attached to the component by calling NUOPC_CompSpecialize(). If a generic component uses specialization through attachable methods, the specific method labels (i.e. the names by which these methods are registered) and the purpose of the method are clearly documented. In some cases attachable methods are optional. This is clearly documented. Further, some generic components attach a default method to a label, which then is used for all phases. This default can be overwritten with a phase specific attachable method. Attaching methods to the component should be done in the SetServices routine right after setting entry points for the standard component methods.

3. Some generic components offer methods that allow parameter specialization. The options of this specialization type are documented as part of the Set() methods.

Components that inherit from a generic component may choose to only specialize certain aspects, leaving other aspects unspecified. This allows a hierarchy of generic components to be implemented with a high degree of code re-use. The variable level of specialization supports the very differing user needs. Figure 1 depicts the inheritance structure of the standard generic components implemented by the NUOPC Layer. There are two trees, one is rooted in ESMF_GridComp, while the other is rooted in ESMF_CplComp.

Figure 1: The NUOPC Generic Component inheritance structure. The tree on the left is rooted in ESMF_GridComp, while the tree on the right is rooted in ESMF_CplComp. The ESMF data types are shown in green. The four main NUOPC Generic Component flavors are shown in dark blue boxes. The yellow box shows a partial specialization in the inheritance tree.

2.2 Field Dictionary

The NUOPC Layer uses standard metadata on Fields to guide the decision making that is implemented in generic code. The generic NUOPC_Connector component, for instance, uses the StandardName Attribute to construct a list of matching Fields between the import and export States. The NUOPC Field Dictionary provides a software implementation of a controlled vocabulary for the StandardName Field Attribute. It also associates each registered StandardName with CanonicalUnits.

The NUOPC Layer provides a number of default entries in the Field Dictionary, shown in the table below. The StandardName Attribute of these entries complies with the Climate and Forecast (CF) conventions as documented at http://cfconventions.org/Data/cf-standard-names/docs/guidelines.html.

Currently it is common practice that a user of the NUOPC Layer extends the Field Dictionary by calling the NUOPC_FieldDictionaryAddEntry() interface to add additional entries. It is the intention to increase the number of default entries over time, and to more strongly leverage the NUOPC Field Dictionary as a means to increase the interoperability between codes that use the NUOPC Layer.

Currently the NUOPC Layer uses the CanonicalUnits entry to verify that Fields are provided in their canonical units. The plan is to extend support to include unit conversion in the future.

StandardName	CanonicalUnits
air_pressure_at_sea_level	Pa
magnitude_of_surface_downward_stress	Pa
precipitation_flux	kg m-2 s-1
sea_surface_height_above_sea_level	m
sea_surface_salinity	1e-3
sea_surface_temperature	K
surface_eastward_sea_water_velocity	m s-1
surface_downward_eastward_stress	Pa
surface_downward_heat_flux_in_air	W m-2
surface_downward_water_flux	kg m-2 s-1
surface_downward_northward_stress	Pa
surface_net_downward_shortwave_flux	W m-2
surface_net_downward_longwave_flux	W m-2
surface_northward_sea_water_velocity	m s-1

2.3 Metadata

The NUOPC Layer makes extensive use of the ESMF Attribute class to implement metadata on Components, States, and Fields. ESMF Attribute Packages (or AttPacks for short) are used to build an Attribute hierarchy for each object.

In some cases the lowest level NUOPC AttPack contains a nested AttPack defined by ESMF. For all objects, the highest level of the NUOPC AttPack hierarchy is implemented with convention="NUOPC", purpose="Instance". The public NUOPC Layer API allows a user to add Attributes to the highest AttPack hierarchy level.

2.3.1 Driver Component Metadata

The Driver component metadata is implemented as an ESMF Attribute Package:

• Convention: NUOPC
• Purpose: Instance
• Includes:
  • CIM Model Component Simulation Description (see for example the Component Attribute packages section in the ESMF v5.2.0rp2 documentation)

Attribute name	Definition	Controlled vocabulary
Kind	String value indicating component kind.	Driver
Verbosity	String value controlling the verbosity of the component. This attribute is used by the generic code layer but should also be considered by any specializing code that implements verbose diagnostic output.	0, 1, ... max
Profiling	String value controlling the profiling level. This attribute is used by the generic code layer but should also be considered by any specializing code that implements profiling.	0, 1, ... max
CompLabel	String value holding the label under which the component was added to its parent driver.	no restriction
InitializePhaseMap	List of string values, mapping the logical NUOPC initialize phases, of a specific Initialize Phase Definition (IPD) version, to the actual ESMF initialize phase number under which the entry point is registered.	IPDvXXpY=Z, where XX = two-digit revision number, e.g. 01, Y = logical NUOPC phase number, Z = actual ESMF phase number, with Y, Z > 0 and Y, Z < 10
RunPhaseMap	List of string values, mapping the logical NUOPC run phases to the actual ESMF run phase number under which the entry point is registered.	label-string=Z, where label-string can be chosen freely, and Z = actual ESMF phase number.
FinalizePhaseMap	List of string values, mapping the logical NUOPC finalize phases to the actual ESMF finalize phase number under which the entry point is registered.	label-string=Z, where label-string can be chosen freely, and Z = actual ESMF phase number.
InternalInitializePhaseMap	List of string values, mapping the logical NUOPC initialize phases, of a specific Initialize Phase Definition (IPD) version, to the actual ESMF initialize phase number under which the entry point is registered.	IPDvXXpY=Z, where XX = two-digit revision number, e.g. 01, Y = logical NUOPC phase number, Z = actual ESMF phase number, with Y, Z > 0 and Y, Z < 10
NestingGeneration	Integer value enumerating nesting level.	0, 1, 2, ...
Nestling	Integer value enumerating siblings within the same generation.	0, 1, 2, ...
InitializeDataComplete	String value indicating whether all initialize data dependencies have been satisfied.	false, true
InitializeDataProgress	String value indicating whether progress is being made resolving initialize data dependencies.	false, true

2.3.2 Model Component Metadata

The Model component metadata is implemented as an ESMF Attribute Package:

• Convention: NUOPC
• Purpose: Instance
• Includes:
  • CIM Model Component Simulation Description (see for example the Component Attribute packages section in the ESMF v5.2.0rp2 documentation)

Attribute name	Definition	Controlled vocabulary
Kind	String value indicating component kind.	Model
Verbosity	String value controlling the verbosity of the component. This attribute is used by the generic code layer but should also be considered by any specializing code that implements verbose diagnostic output.	0, 1, ... max
Profiling	String value controlling the profiling level. This attribute is used by the generic code layer but should also be considered by any specializing code that implements profiling.	0, 1, ... max
CompLabel	String value holding the label under which the component was added to its parent driver.	no restriction
InitializePhaseMap	List of string values, mapping the logical NUOPC initialize phases, of a specific Initialize Phase Definition (IPD) version, to the actual ESMF initialize phase number under which the entry point is registered.	IPDvXXpY=Z, where XX = two-digit revision number, e.g. 01, Y = logical NUOPC phase number, Z = actual ESMF phase number, with Y, Z > 0 and Y, Z < 10
RunPhaseMap	List of string values, mapping the logical NUOPC run phases to the actual ESMF run phase number under which the entry point is registered.	label-string=Z, where label-string can be chosen freely, and Z = actual ESMF phase number.
FinalizePhaseMap	List of string values, mapping the logical NUOPC finalize phases to the actual ESMF finalize phase number under which the entry point is registered.	label-string=Z, where label-string can be chosen freely, and Z = actual ESMF phase number.
InternalInitializePhaseMap	List of string values, mapping the logical NUOPC initialize phases, of a specific Initialize Phase Definition (IPD) version, to the actual ESMF initialize phase number under which the entry point is registered.	IPDvXXpY=Z, where XX = two-digit revision number, e.g. 01, Y = logical NUOPC phase number, Z = actual ESMF phase number, with Y, Z > 0 and Y, Z < 10
NestingGeneration	Integer value enumerating nesting level.	0, 1, 2, ...
Nestling	Integer value enumerating siblings within the same generation.	0, 1, 2, ...
InitializeDataComplete	String value indicating whether all initialize data dependencies have been satisfied.	false, true
InitializeDataProgress	String value indicating whether progress is being made resolving initialize data dependencies.	false, true

2.3.3 Mediator Component Metadata

The Mediator component metadata is implemented as an ESMF Attribute Package:

• Convention: NUOPC
• Purpose: Instance
• Includes:
  • CIM Model Component Simulation Description (see for example the Component Attribute packages section in the ESMF v5.2.0rp2 documentation)

Attribute name	Definition	Controlled vocabulary
Kind	String value indicating component kind.	Mediator
Verbosity	String value controlling the verbosity of the component. This attribute is used by the generic code layer but should also be considered by any specializing code that implements verbose diagnostic output.	0, 1, ... max
Profiling	String value controlling the profiling level. This attribute is used by the generic code layer but should also be considered by any specializing code that implements profiling.	0, 1, ... max
CompLabel	String value holding the label under which the component was added to its parent driver.	no restriction
InitializePhaseMap	List of string values, mapping the logical NUOPC initialize phases, of a specific Initialize Phase Definition (IPD) version, to the actual ESMF initialize phase number under which the entry point is registered.	IPDvXXpY=Z, where XX = two-digit revision number, e.g. 01, Y = logical NUOPC phase number, Z = actual ESMF phase number, with Y, Z > 0 and Y, Z < 10
RunPhaseMap	List of string values, mapping the logical NUOPC run phases to the actual ESMF run phase number under which the entry point is registered.	label-string=Z, where label-string can be chosen freely, and Z = actual ESMF phase number.
FinalizePhaseMap	List of string values, mapping the logical NUOPC finalize phases to the actual ESMF finalize phase number under which the entry point is registered.	label-string=Z, where label-string can be chosen freely, and Z = actual ESMF phase number.
InternalInitializePhaseMap	List of string values, mapping the logical NUOPC initialize phases, of a specific Initialize Phase Definition (IPD) version, to the actual ESMF initialize phase number under which the entry point is registered.	IPDvXXpY=Z, where XX = two-digit revision number, e.g. 01, Y = logical NUOPC phase number, Z = actual ESMF phase number, with Y, Z > 0 and Y, Z < 10
NestingGeneration	Integer value enumerating nesting level.	0, 1, 2, ...
Nestling	Integer value enumerating siblings within the same generation.	0, 1, 2, ...
InitializeDataComplete	String value indicating whether all initialize data dependencies have been satisfied.	false, true
InitializeDataProgress	String value indicating whether progress is being made resolving initialize data dependencies.	false, true

2.3.4 Connector Component Metadata

The Connector Component metadata is implemented as an ESMF Attribute Package:

• Convention: NUOPC
• Purpose: General
• Includes:
  • ESG General (see for example the Component Attribute packages section in the ESMF v5.2.0rp2 documentation)

Attribute name	Definition	Controlled vocabulary
Kind	String value indicating component kind.	Connector
Verbosity	String value controlling the verbosity of the component. This attribute is used by the generic code layer but should also be considered by any specializing code that implements verbose diagnostic output.	0, 1, ... max
Profiling	String value controlling the profiling level. This attribute is used by the generic code layer but should also be considered by any specializing code that implements profiling.	0, 1, ... max
CompLabel	String value holding the label under which the component was added to its parent driver.	no restriction
InitializePhaseMap	List of string values, mapping the logical NUOPC initialize phases, of a specific Initialize Phase Definition (IPD) version, to the actual ESMF initialize phase number under which the entry point is registered.	IPDvXXpY=Z, where XX = two-digit revision number, e.g. 01, Y = logical NUOPC phase number, Z = actual ESMF phase number, with Y, Z > 0 and Y, Z < 10
RunPhaseMap	List of string values, mapping the logical NUOPC run phases to the actual ESMF run phase number under which the entry point is registered.	label-string=Z, where label-string can be chosen freely, and Z = actual ESMF phase number.
FinalizePhaseMap	List of string values, mapping the logical NUOPC finalize phases to the actual ESMF finalize phase number under which the entry point is registered.	label-string=Z, where label-string can be chosen freely, and Z = actual ESMF phase number.
CplList	List of StandardNames of the connected Fields. Each StandardName entry may be followed by a colon separated list of connection options. The details are discussed in section 2.4.5	Standard names as per field dictionary, followed by connection options defined in section 2.4.5.

2.3.5 State Metadata

The State metadata is implemented as an ESMF Attribute Package:

• Convention: NUOPC
• Purpose: Instance
• Includes:
  • no other packages at the moment
• Description: Namespace implementation for import and export.

Attribute name	Definition	Controlled vocabulary
Namespace	String value holding the namespace of all the objects contained in the State.	no restriction

2.3.6 Field Metadata

The Field metadata is implemented as an ESMF Attribute Package:

• Convention: NUOPC
• Purpose: Instance
• Includes:
  • ESG General
• Description: Basic Field description with connection and time stamp metadata.

Attribute name	Definition	Controlled vocabulary
Connected	Connected status.	false, true
TimeStamp	Nine integer values representing ESMF Time object.	N/A
ProducerConnection	String value indicating connection details.	open, targeted, connected
ConsumerConnection	String value indicating connection details.	open, targeted, connected
Updated	String value indicating updated status during initialization.	false, true
TransferOfferGeomObject	String value indicating a component's intention to transfer the underlying Grid or Mesh on which an advertised Field object is defined.	will provide, can provide, cannot provide
TransferActionGeomObject	String value indicating the action a component is supposed to take with respect to transferring the underlying Grid or Mesh on which an advertised Field object is defined.	provide, accept

2.4 Initialization

2.4.1 Phase Maps and Component Labels

ESMF introduces the concept of standard component methods: Initialize, Run, and Finalize. ESMF further recognizes the need for being able to split each of the standard methods into multiple phases. On the ESMF level, phases are implemented by a simple integer phase index. The NUOPC layer adds an abstraction layer that allows phases to be referenced by label.

For complex scenarios, e.g. multiple versions of multi-stage initialize sequences, the use of an integer based phase index quickly becomes confusing. The NUOPC Layer addresses this issue by introducing three component level attributes: InitializePhaseMap, RunPhaseMap, and FinalizePhaseMap. These attributes map logical NUOPC phase labels to integer ESMF phase indices. A NUOPC compliant component fully documents its available phases through the phase maps.

Currently the NUOPC Layer leverages the InitializePhaseMap during the initialization loop that is implemented by the generic NUOPC_Driver. It looks for phase map entries according to the initialize phase definition outlined in section 2.4.2. The RunPhaseMap is used when setting up run sequences in the Driver. The NUOPC_DriverAddRunElement() takes the phaseLabel argument, and uses the RunPhaseMap attribute internally to translates the label into the corresponding ESMF phase index. The FinalizePhaseMap is currently not used within the NUOPC Layer.

Within NUOPC, components under a driver are also referenced by label. Every component is associated with a label when it is added to a driver through the NUOPC_DriverAddComp() call. Multiple instances of the same component can be added to a driver, provided each instance is given a unique label. Connectors between components are identified by providing the label of the source component and destination component.

2.4.2 Initialize Phase Definitions

The interaction between NUOPC compliant components during the initialization process is regulated by the Initialize Phase Definition or IPD. The IPDs are versioned, with a higher version number indicating backward compatibility with all previous versions.

There are two perspectives of looking at the IPD. From the driver perspective the IPD regulates the sequence in which it must call the different phases of the Initialize() routines of its child components. To this end the generic NUOPC_Driver component implements support for IPDs up to a version specified in the API documentation.

The other angle of looking at the IPD is from the driver's child components. From this perspective the IPD assigns specific meaning to each initialize phase. The child components of a driver can be divided into two groups with respect to the meaning the IPD assigns to each initialize phase. In one group are the model, mediator, and driver components, and in the other group are the connector components. Child components publish their available initialize phases through the InitializePhaseMap attribute.

The driver also calls into its own internal initialize methods. This allows the driver to participate in the initialization of its children in a structured fashion. The internal initialization phases of a driver are published via the InternalInitializePhaseMap attribute.

The following tables document the meaning of each initialization phase of the available IPD versions for the child components and for the driver component itself. The phases are listed in the same sequence in which the driver calls them.

IPDv00 label	Component	Meaning
IPDv00p1	driver-self	unspecified by NUOPC
IPDv00p1	models, mediators, drivers	Advertise their import and export Fields.
IPDv00p1	connectors	Construct their CplList Attribute.
IPDv00p2	driver-self	unspecified by NUOPC
IPDv00p2	models, mediators, drivers	Realize their import and export Fields.
IPDv00p2a	connectors	Set the Connected Attribute on each import and export Field according to the CplList Attribute. Reconcile the import and export States.
IPDv00p2b	connectors	Precompute the RouteHandle.
IPDv00p3	driver-self	unspecified by NUOPC
IPDv00p3	models, mediators, drivers	Check for compatibility of their Fields' Connected status.
IPDv00p4	driver-self	unspecified by NUOPC
IPDv00p4	models, mediators, drivers	Handle Field data initialization. Timestamp their export Fields.

IPDv01 label	Component	Meaning
IPDv01p1	driver-self	unspecified by NUOPC
IPDv01p1	models, mediators, drivers	Advertise their import and export Fields.
IPDv01p1	connectors	Construct their CplList Attribute.
IPDv01p2	driver-self	Modify the CplList Attributes on the Connectors.
IPDv01p2	models, mediators, drivers	unspecified/unused by NUOPC
IPDv01p2	connectors	Set the Connected Attribute on each import and export Field according to the CplList Attribute.
IPDv01p3	driver-self	unspecified by NUOPC
IPDv01p3	models, mediators, drivers	Realize their "connected" import and export Fields.
IPDv01p3a	connectors	Reconcile the import and export States.
IPDv01p3b	connectors	Precompute the RouteHandle according to the CplList Attribute.
IPDv01p4	driver-self	unspecified by NUOPC
IPDv01p4	models, mediators, drivers	Check for compatibility of their Fields' Connected status.
IPDv01p5	driver-self	unspecified by NUOPC
IPDv01p5	models, mediators, drivers	Handle Field data initialization. Timestamp their export Fields.

IPDv02 label	Component	Meaning
IPDv02p1	driver-self	unspecified by NUOPC
IPDv02p1	models, mediators, drivers	Advertise their import and export Fields.
IPDv02p1	connectors	Construct their CplList Attribute.
IPDv02p2	driver-self	Modify the CplList Attributes on the Connectors.
IPDv02p2	models, mediators, drivers	unspecified/unused by NUOPC
IPDv02p2	connectors	Set the Connected Attribute on each import and export Field according to the CplList Attribute.
IPDv02p3	driver-self	unspecified by NUOPC
IPDv02p3	models, mediators, drivers	Realize their "connected" import and export Fields.
IPDv02p3a	connectors	Reconcile the import and export States.
IPDv02p3b	connectors	Precompute the RouteHandle according to the CplList Attribute.
IPDv02p4	driver-self	unspecified by NUOPC
IPDv02p4	models, mediators, drivers	Check for compatibility of their Fields' Connected status.
IPDv02p5	driver-self	unspecified by NUOPC
IPDv02p5	models, mediators, drivers	Handle Field data initialization. Timestamp their export Fields.
A loop is entered over all those model, mediator, driver Components that use IPDv02 and have unsatisfied data dependencies, repeating the following two steps:
Run()	connectors	Loop over all Connectors that connect to the Component that is currently indexed by the outer loop.
IPDv02p5	models, mediators, drivers	Handle Field data initialization. Timestamp their export Fields and set the Updated and InitializeDataComplete Attributes accordingly.
Repeat these two steps until all data dependencies have been statisfied, or a dead-lock situation is detected.

IPDv03 label	Component	Meaning
IPDv03p1	driver-self	unspecified by NUOPC
IPDv03p1	models, mediators, drivers	Advertise their import and export Fields and set the TransferOfferGeomObject Attribute.
IPDv03p1	connectors	Construct their CplList Attribute.
IPDv03p2	driver-self	Modify the CplList Attributes on the Connectors.
IPDv03p2	models, mediators, drivers	unspecified/unused by NUOPC
IPDv03p2	connectors	Set the Connected Attribute on each import and export Field according to the CplList Attribute. Set the TransferActionGeomObject Attribute.
IPDv03p3	driver-self	unspecified by NUOPC
IPDv03p3	models, mediators, drivers	Realize their "connected" import and export Fields that have TransferActionGeomObject equal to "provide".
IPDv03p3	connectors	Transfer the Grid/Mesh/LocStream objects (only DistGrid) for Field pairs that have a provider and an acceptor side.
IPDv03p4	driver-self	unspecified by NUOPC
IPDv03p4	models, mediators, drivers	Optionally modify the decomposition and distribution information of the accepted Grid/Mesh/LocStream by replacing the DistGrid.
IPDv03p4	connectors	Transfer the full Grid/Mesh/LocStream objects (with coordinates) for Field pairs that have a provider and an acceptor side.
IPDv03p5	driver-self	unspecified by NUOPC
IPDv03p5	models, mediators, drivers	Realize all Fields that have TransferActionGeomObject equal to "accept" on the transferred Grid/Mesh/LocStream objects.
IPDv03p5a	connectors	Reconcile the import and export States.
IPDv03p5b	connectors	Precompute the RouteHandle according to the CplList Attribute.
IPDv03p6	driver-self	unspecified by NUOPC
IPDv03p6	models, mediators, drivers	Check compatibility of their Fields' Connected status.
IPDv03p7	driver-self	unspecified by NUOPC
IPDv03p7	models, mediators, drivers	Handle Field data initialization. Timestamp the export Fields.
A loop is entered over all those model, mediator, driver Components that use IPDv02 and have unsatisfied data dependencies, repeating the following two steps:
Run()	connectors	Loop over all Connectors that connect to the Component that is currently indexed by the outer loop.
IPDv03p7	models, mediators, drivers	Handle Field data initialization. Time stamp the export Fields and set the Updated and InitializeDataComplete Attributes accordingly.
Repeat these two steps until all data dependencies have been statisfied, or a dead-lock situation is detected.

IPDv04 label	Component	Meaning
IPDv04p1	driver-self	unspecified by NUOPC
IPDv04p1	models, mediators, drivers	Advertise their import and export Fields and set the TransferOfferGeomObject Attribute.
IPDv04p1a	connectors	Consider all connection possibilities for their CplList Attribute.
IPDv04p1b	connectors	Unambiguous construction of their CplList Attribute.
IPDv04p2	driver-self	Modify the CplList Attributes on the Connectors.
IPDv04p2	models, mediators, drivers	unspecified/unused by NUOPC
IPDv04p2	connectors	Set the Connected Attribute on each import and export Field according to the CplList Attribute. Set the TransferActionGeomObject Attribute.
IPDv04p3	driver-self	unspecified by NUOPC
IPDv04p3	models, mediators, drivers	Realize their "connected" import and export Fields that have TransferActionGeomObject equal to "provide".
IPDv04p3	connectors	Transfer the Grid/Mesh/LocStream objects (only DistGrid) for Field pairs that have a provider and an acceptor side.
IPDv04p4	driver-self	unspecified by NUOPC
IPDv04p4	models, mediators, drivers	Optionally modify the decomposition and distribution information of the accepted Grid/Mesh/LocStream by replacing the DistGrid.
IPDv04p4	connectors	Transfer the full Grid/Mesh/LocStream objects (with coordinates) for Field pairs that have a provider and an acceptor side.
IPDv04p5	driver-self	unspecified by NUOPC
IPDv04p5	models, mediators, drivers	Realize all Fields that have TransferActionGeomObject equal to "accept" on the transferred Grid/Mesh/LocStream objects.
IPDv04p5a	connectors	Reconcile the import and export States.
IPDv04p5b	connectors	Precompute the RouteHandle according to the CplList Attribute.
IPDv04p6	driver-self	unspecified by NUOPC
IPDv04p6	models, mediators, drivers	Check compatibility of their Fields' Connected status.
IPDv04p7	driver-self	unspecified by NUOPC
IPDv04p7	models, mediators, drivers	Handle Field data initialization. Timestamp the export Fields.
A loop is entered over all those model, mediator, driver Components that use IPDv02 and have unsatisfied data dependencies, repeating the following two steps:
Run()	connectors	Loop over all Connectors that connect to the Component that is currently indexed by the outer loop.
IPDv04p7	models, mediators, drivers	Handle Field data initialization. Time stamp the export Fields and set the Updated and InitializeDataComplete Attributes accordingly.
Repeat these two steps until all data dependencies have been statisfied, or a dead-lock situation is detected.

IPDv05 label	Component	Meaning
IPDv05p1	driver-self	Advertise import and export Fields and set the TransferOfferGeomObject Attribute. Optionally set FieldTransferPolicy Attribute on States.
IPDv05p1	models, mediators, drivers	Advertise their import and export Fields and set the TransferOfferGeomObject Attribute. Optionally set FieldTransferPolicy Attribute on States.
IPDv05p1	connectors	Consider FieldTransferPolicy Attribute on import and export States. Advertise Fields to be transferred.
IPDv05p2	driver-self	Optionally modify import and export States before connectors construct CplList Attribute.
IPDv05p2	models, mediators, drivers	Optionally modify import and export States before connectors construct CplList Attribute.
IPDv05p2a	connectors	Consider all connection possibilities for their CplList Attribute.
IPDv05p2b	connectors	Unambiguous construction of their CplList Attribute.
IPDv05p3	driver-self	Modify the CplList Attributes on the Connectors.
IPDv05p3	models, mediators, drivers	unspecified/unused by NUOPC
IPDv05p3	connectors	Set the Connected Attribute on each import and export Field according to the CplList Attribute. Set the TransferActionGeomObject Attribute.
IPDv05p4	driver-self	Realize "connected" import and export Fields that have TransferActionGeomObject equal to "provide".
IPDv05p4	models, mediators, drivers	Realize their "connected" import and export Fields that have TransferActionGeomObject equal to "provide".
IPDv05p4	connectors	Transfer the Grid/Mesh/LocStream objects (only DistGrid) for Field pairs that have a provider and an acceptor side.
IPDv05p5	driver-self	Optionally modify the decomposition and distribution information of the accepted Grid/Mesh/LocStream by replacing the DistGrid.
IPDv05p5	models, mediators, drivers	Optionally modify the decomposition and distribution information of the accepted Grid/Mesh/LocStream by replacing the DistGrid.
IPDv05p5	connectors	Transfer the full Grid/Mesh/LocStream objects (with coordinates) for Field pairs that have a provider and an acceptor side.
IPDv05p6	driver-self	Realize all Fields that have TransferActionGeomObject equal to "accept" on the transferred Grid/Mesh/LocStream objects.
IPDv05p6	models, mediators, drivers	Realize all Fields that have TransferActionGeomObject equal to "accept" on the transferred Grid/Mesh/LocStream objects.
IPDv05p6a	connectors	Reconcile the import and export States.
IPDv05p6b	connectors	Precompute the RouteHandle according to the CplList Attribute.
IPDv05p7	driver-self	unspecified by NUOPC
IPDv05p7	models, mediators, drivers	Check compatibility of their Fields' Connected status.
IPDv05p8	driver-self	unspecified by NUOPC
IPDv05p8	models, mediators, drivers	Handle Field data initialization. Timestamp the export Fields.
A loop is entered over all those model, mediator, driver Components that use IPDv02 and have unsatisfied data dependencies, repeating the following two steps:
Run()	connectors	Loop over all Connectors that connect to the Component that is currently indexed by the outer loop.
IPDv05p8	models, mediators, drivers	Handle Field data initialization. Time stamp the export Fields and set the Updated and InitializeDataComplete Attributes accordingly.
Repeat these two steps until all data dependencies have been statisfied, or a dead-lock situation is detected.

2.4.3 Field Pairing

The NUOPC Model and Mediator components are required to advertise their import and export Fields with a standard set of Field metadata. This set includes the StandardName attribute. The NUOPC Layer implements a strategy of pairing advertised Fields that is based primarily on the StandardName of the Fields, and in more complex situations further utilizes the Namespace attribute on States.

Field pairing is accomplished as part of the initialization procedure and is a collective effort of the Driver and its child components: Models, Mediator, Connectors. The exact handshakes between these components is outlined as part of the Initialize Phase Definition in section 2.4.2.

The Connectors are the most active players when it comes to Field pairing. The end result of the process is where each Connector has a list of Fields that it connects between its importState and its exportState. Each connector keeps this list in its component level metadata as CplList attribute.

During the first stage of Field pairing, each Connector matches all of the Fields in its importState to all of the Fields in its exportState by looking at their StandardName attribute. For every match a bondLevel is calculated and stored in the Field on the export side, i.e. on the consumer side of the connection, in the Field's ConsumerConnection attribute. The larges found bondLevel is kept for each Field on the export side.

The bondLevel is a measure of how strong the pairing is considering the namespace rules explained in section 2.4.4. Without the use of namespaces the bondLevel for all Field pairs that match by their StandardName is equal to 1.

After the first stage, there may be umbiguous Field pairs present. Ambiguous Field pairs are those that map different producer Fields (i.e. Fields in the importState of a Connector) to the same consumer Field (i.e. a Field in the exportState of a Connector). While the NUOPC Layer support having multiple consumer Fields connected to a single producer Field, it does not support the opposite condition. The second stage of Field pairing is responsible for disambiguating Field pairs with the same consumer Field.

Field pair disambiguation is based on the bondLevel that was calculated and stored on the consumer side Field for each pair during the first stage. The disambiguation rule simply selects the connection with the highest bondLevel and discards all lesser connection to the same consumer side Field. However, if the highest bondLevel is not unique, i.e. there are multiple pairs with the same bondLevel, disambiguation is not possible and an error is returned to the Driver by the Connector that finds the ambiguity first.

Assuming that the disambiguation step was successful, each Connector holds a valid CplList attribute with entries that correspond to the Field pairs that it is responsible for. At this stage the Driver can still overwrite this attribute and implement custom pairs if that is desired.

2.4.4 Namespaces

Namespaces are used to control and fine-tune the disambiguation of Field pairs during the initialization. The general procedure of Field pairing and disambiguation is outlined in section 2.4.3, here the use of namespaces is described.

The NUOPC Layer implements namespaces through the Namespace attribute on ESMF_State objects. The value of this attribute is a simple character string. The NUOPC Layer automatically creates the import and export States of every Model and Mediator component that is added to a Driver. The Namespace attribute of these States is automatically set to the compLabel string that was provided during NUOPC_DriverAdd(). Doing this places every Field that is advertised through these States inside the component's unique namespace.

A secondary namespace can be added to a State using the NUOPC_StateNamespaceAdd() method. This creates a new State that is nested inside of an existing State, and sets the Namespace attribute of the new State. Fields that are advertised inside of such a nested State are in a namespace with two parts: NS1:NS2. Here NS1 is the preset namespace of the import or export State (equal to the compLabel), and NS2 is a freely chosen namespace string.

During Field pairing the namespace on each side of the connection is considered in the two part format NS1:NS2. The first part is equal to the compLabel of the corresponding component, and NS2 is either the namespace of a nested State, or empty if the Field is not inside a nested State. Using this format, the calculation of the bondLevel during Field pairing is governed by the following rules:

• Namespace matching is done in a cross wise fashion, meaning NS1 from one side is compared to NS2 of the other side, and vice versa.
• The bondLevel is incremented by one counter for each cross-wise match between namespaces. (Considering that the bondLevel starts out as 1 for any Field pair with matching standard names, the maximum bondLevel that can be reached is 3.)
• Finding one side of the cross-wise comparison being an empty string is neither counted as a match nor a mis-match. The bondLevel remains unchanged.
• A Field pair for which a mis-match in either of the two cross-wise namespace comparisons is detected is discarded from the possible pairs. It is not further considered.

In practice then, a component that targets a specific other component with its advertised Fields would add a secondary namespace to its import or export State, and set that namespace to the compLabel of the targeted component. This increases the bondLevel for each pair from 1 to 2. An even higher bondLevel of 3 is achieved when both sides target each other by specifying the other component's compLabel through a secondary namespace.

In conclusion, namespaces can affect the bondLevel calculation for each pair, but they do not affect how pairs are constructed and disambiguated. In particular, the requirement for unambiguous Field pairs for each consumer Field remains unchanged, and it is an error condition if the highest bondLevel for a consumer Field does not correspond to a unique Field pair.

2.4.5 Connection Options

Once the field pairing discussed in the previous sections is completed, each Connector component holds an attribute by the name of CplList. The CplList is a list type attribute with as many entries as there are fields for which the Connector component is responsible for connecting. The first part of each of these entries is always the StandardName of the associated field. See section 2.2 for a discussion of the NUOPC field dictionary and standard names.

After the StandardName part, each CplList entry may optionally contain a string of connection options. Each Driver component has the chance as part of the internal IPDv04p2 phase (see 2.4.2) to modify the CplList attribute of all the Connectors that it drives.

The individual connection options are colon separated, leading to the following format for each CplList entry:

StandardName[:option1[:option2[: ...]]

The format of the options is:

OptionName=value1[=spec1][,value2[=spec2][, ...]]

OptionName and the value strings are case insensitive. There are single and multi-valued options as indicated in the table below. For single valued options only value1 is relevant. If the same option is listed multiple times, only the first occurrence will be used. If an option has a default value, it is indicated in the table. If a value requires additional specification via =spec then the specifications are listed in the table.

OptionName	Definition	Type	Values
srcMaskValues	List of integer values that defines the mask values.	multi	List of integers.
dstMaskValues	List of integer values that defines the mask values.	multi	List of integers.
remapmethod	Redistribution or interpolation to compute the regridding weights.	single	redist, bilinear(default), patch, nearest_stod, nearest_dtos, conserve
polemethod	Extrapolation method around the pole(s).	single	none(default), allavg, npntavg="integer indicating number of points"
unmappedaction	The action to take when unmapped destination elements are encountered.	single	ignore(default), error
srcTermProcessing	Number of terms in each partial sum of the interpolation to process on the source side. This setting impacts the bit-for-bit reproducibility of the parallel interpolation results between runs. The strictest bit-for-bit setting is achieved by setting the value to 1.	single	integer
termOrder	Order of the terms in each partial sum of the interpolation. This setting impacts the bit-for-bit reproducibility of the parallel interpolation results between runs. The strictest bit-for-bit setting is achieved by setting the value to srcseq.	single	free(default), srcseq, srcpet
pipelineDepth	Maximum number of outstanding non-blocking communication calls during the parallel interpolation. Only relevant for cases where the automatic tuning procedure fails to find a setting that works well on a given hardware.	single	integer
dumpWeightsFlag	Enable or disable dumping of the interpolation weights into a file.	single	true, false(default)

2.4.6 Data-Dependencies during Initialize

For multi-model applications it is not uncommon that during start-up one or more components depends on data from one or more other components. These types of data-dependencies during initialize can become very complex very quickly. Finding the "correct" sequence to initialize all components for a complex dependency graph is not trivial. The NUOPC Layer deals with this issue by repeatedly looping over all components that indicate that their initialization has data dependencies on other components. The loop is finally exited when either all components have indicated completion of their initialization, or a dead-lock situation is being detected by the NUOPC Layer.

The data-dependency resolution loop is implemented as part of Initialize Phase Definition version 2 (IPDv02) as defined in section 2.4.2. Participating components communicate their current status to the NUOPC Layer via Field and Component metadata. Participants are those components that contain an IPDv02p5 assignment in their InitializePhaseMap Attribute according to sections 2.3.1, 2.3.2, 2.3.3, and 2.3.4.

Every time a component's IPDv02p5 initialization phase is called it is responsible for setting the InitializeDataComplete and InitializeDataProgress Attributes according to its current status before returning. For convenience, the NUOPC Layer provides a generic implementation of an IPDv02p5 phase initialize method for Models and Mediators (available as ESMF Initialize phase 5). This generic implementation takes care of setting the InitializeDataProgress Attribute automatically. It does so by inspecting the Updated Field Attribute (see section 2.3.6) on all the Fields in the component's exportState. The generic IPDv02p5 implementation must be specialized by attaching a method for specialization point label_DataInitialize. This specialization method is responsible for checking the Fields in the importState and for initializing any internal data structures and Fields in the exportState. Fields that are fully initialized in the exportState must be indicated by setting their Updated Attribute to "true". Once the component is fully initialized it must further set its InitializeDataComplete Attribute to "true" before returning.

During the execution of the data-dependency resolution loop the NUOPC Layer calls all of the Connectors to a Model/Mediator component before calling the component's IPDv02p5 method. Doing so ensures that all the currently available Fields are passed to the component before it tries to access them during IPDv02p5. Once a component has set its InitializeDataComplete Attribute to "true" it, and the Connectors to it, will no longer be called during the remainder of the resolution loop.

When all of the components with an IPDv02p5 initialization phase have set their InitializeDataComplete Attribute to "true", the NUOPC Layer successfully exits the data-dependency resolution loop. The loop is also interrupted before all InitializeDataComplete Attributes are set to "true" if a full cycle completes without any indicated progress. The NUOPC Layer flags this situation as a potential dead-lock and returns with error.

2.4.7 Transfer of Grid/Mesh/LocStream Objects between Components

There are modeling scenarios where the need arises to transfer physical grid information from one component to another. One common situation is that of modeling systems that utilize Mediator components to implement the interactions between Model components. In these cases the Mediator often carries out computations on a Model's native grid and performs regridding to the grid of other Model components. It is both cumbersome and error prone to recreate the Model grid in the Mediator. The Initialize Phase Definition version 3 (IPDv03) and above, defined in section 2.4.2, support the transfer of ESMF_Grid, and ESMF_Mesh, and ESMF_LocStream objects between Model and/or Mediator components during initialization.

The NUOPC Layer transfer protocol for GeomObjects (i.e. ESMF Grids, Meshes, or LocStreams) is based on two Field attributes: TransferOfferGeomObject and TransferActionGeomObject. The TransferOfferGeomObject attribute is used by the Model and/or Mediator components to indicate for each Field their intent for the associated GeomObject. The predefined values of this attribute are: "will provide", "can provide", and "cannot provide". The TransferOfferGeomObject attribute must be set during IPDv03p1.

The generic Connector uses the intents from both sides and constructs a response according to the table below. The response is provided by the Connector during IPDv03p2 by setting the value of the TransferActionGeomObject attribute to either "provide" or "accept" on each Field. Fields indicating TransferActionGeomObject equal to "provide" must be realized on a Grid, Mesh, or LocStream object in the Model/Mediator initialize method for phase IPDv03p3.

Fields that hold "accept" for the value of the TransferActionGeomObject attribute require two additional negotiation steps. By IPDv03p4 the Model/Mediator component can access the transferred Grid/Mesh/LocStream on the Fields that have the "accept" value. However, only the DistGrid, i.e. the decomposition and distribution information of the Grid/Mesh/LocStream is available at this stage, not the full physical grid information such as the coordinates. At this stage the Model/Mediator may modify this information by replacing the DistGrid object in the Grid/Mesh/LocStream. The DistGrid that is set on the Grid/Mesh/LocStream objects when leaving the Model/Mediator phase IPDv03p4 will consequently be used by the generic Connector to fully transfer the Grid/Mesh/LocStream object. The fully transferred objects are available on the Fields with "accept" during Model/Mediator phase IPDv03p5, where they can be used to realize the respective Field objects. Realizing typically just requires the ESMF_FieldEmptyComplete() call to be made. At this point all Field objects are fully realized and the initialization process can proceed as usual.

The following table shows how the generic Connector sets the TransferActionGeomObject attribute on the Fields according to the incoming value of TransferOfferGeomObject.

TransferOfferGeomObject Incoming side A	TransferOfferGeomObject Incoming side B	Outgoing setting by generic Connector
"will provide"	"will provide"	A:TransferActionGeomObject="provide" B:TransferActionGeomObject="provide"
"will provide"	"can provide"	A:TransferActionGeomObject="provide" B:TransferActionGeomObject="accept"
"will provide"	"cannot provide"	A:TransferActionGeomObject="provide" B:TransferActionGeomObject="accept"
"can provide"	"will provide"	A:TransferActionGeomObject="accept" B:TransferActionGeomObject="provide"
"can provide"	"can provide"	if (A is import side) then A:TransferActionGeomObject="provide" B:TransferActionGeomObject="accept" if (B is import side) then A:TransferActionGeomObject="accept" B:TransferActionGeomObject="provide"
"can provide"	"cannot provide"	A:TransferActionGeomObject="provide" B:TransferActionGeomObject="accept"
"cannot provide"	"will provide"	A:TransferActionGeomObject="accept" B:TransferActionGeomObject="provide"
"cannot provide"	"can provide"	A:TransferActionGeomObject="accept" B:TransferActionGeomObject="provide"
"cannot provide"	"cannot provide"	Flagged as error!

2.4.8 Field Mirroring

In some cases it is helpful for a NUOPC component to automatically mirror or match the set of fields advertised in another component. One purpose of this is to automatically resolve the import data dependencies of a component, by setting up a component that exactly provides all of the needed fields. This is currently used in the NUOPC Component Explorer: when driving a child NUOPC Model with required import fields, the Component Explorer uses the field mirroring capability to advertise in the driver-self export State the exact set of fields advertised in the child NUOPC Model. This ensures that the entire Initialize Phase Sequence will complete (because all dependencies are satisfied) and all phases can be exercised by the Component Explorer.

The field mirror capability is also useful with NUOPC Mediators since these components often exactly reflect, in separate States, the sets of fields of each of the connected components. The field mirroring capability, therefore, can be used to ensure that a Mediator is always capable of accepting fields from connected components, and removes the need to specify field lists in multiple places, i.e., both within a set of Model components connected to a Mediator and within the Mediator itself.

The field mirror capability is supported in the Initialize Phase Definition version 5 (IPDv05) and higher, defined in section 2.4.2. During IPDv05p1, driver-self, models, mediators, and drivers advertise fields in their import and export States. At this point, these components can also optionally set the FieldTransferPolicy Attribute in their import and export States. The default value "TransferNone" indicates that no fields should be mirrored. The other option, "TransferAll", indicates that fields should be mirrored in the State of a connected component.

During IPDv05p1, each Connector consider the FieldTransferPolicy Attribute on both its import and export States. If both States have a FieldTransferPolicy of "TransferAll", then fields are transferred between the States in both directions (i.e., import to export and export to import). The transfer process works as follows: First, the TransferOfferGoemObject attribute is reversed between the providing side and accepting side. Intuitively, if a field from the providing component is to be mirrored and it can provide its own geometric object, then the mirrored field on the accepting side should be set to accept a geometric object. Then, the field to be mirrored is advertised in the accepting State using a call to NUOPC_Advertise() such that the mirrored field shares the same Standard Name.

At this point the Initialization Sequence continues as usual. Since fields to be mirrored have been advertised with matching Standard Names, the field pairing algorithm will now match them in the usual way thereby establishing a connection between the original and mirrored fields.