The NUOPC Layer introduces a modeling system architecture based on Models, Mediators, Connectors, and Drivers. The Layer defines the rules of engagement between these components. Many of these rules are formulated on the basis of metadata. This metadata can be expected for compliance.
One of the challenges when inspecting a component for NUOPC Layer compliance is that many of the rules of engagement are run-time rules. This means that they address the dynamical behavior of a component during run-time. For this reason, comprehensive compliance testing cannot be done statically but requires the execution of code.
Currently there are two sets of tools available to address the issue of NUOPC Layer compliance testing. The Compliance Checker is a runtime analysis tool that can be enabled by setting an ESMF environment variable at runtime. When active, the Compliance Checker intercepts all interactions between components that go throught the ESMF component interface, and analyzes them with respect to the NUOPC Layer rules of engagement. Warnings are printed to the log files when issues or non-compliances are detected.
The Component Explorer is another compliance testing tool. It focuses on interacting with a single component, and analyzing it during the early initialization phases. The Component Explorer and Compliance Checker are compatible with each other and it is often useful to use them both at the same time.
The NUOPC Compliance Checker is a run-time analysis tool that can be turned on for any ESMF application. The Compliance Checker is turned off by default, as to not negatively affect performance critical runs. The Compliance Checker is enabled by setting the following ESMF runtime environment variable:
ESMF_RUNTIME_COMPLIANCECHECK=ONAs a run-time variable, setting it does not require recompilation of the ESMF library or the user application. The same executable and library will start to generate Compliance Checker output when the above variable is found set during execution.
The function of the Compliance Checker is to intercept all interactions between the components of an ESMF application, and to analyze them according to the NUOPC Layer rules of engagement. The following aspects are currently reported on:
Besides the above aspects, the output of the Compliance Checker also provides a means to easily get an idea of the exact dynamical control flow between the components of an application.
The Compliance Checker uses the ESMF Log facility to produce the compliance report during the execution of an ESMF application. The output is located in the default ESMF Log files. There are advantages of using the existing Log facility to generate the compliance report. First, the ESMF Log facility offers time stamping of messages, and deals with all of the file access and multi-PET issues. Second, going through the ESMF Log guarantees that all the output appears in the correct chronological order. This applies to all of the output, including entries from other ESMF system levels or from the user level.
A sample output of the Compliance Checker output in action:
20131108 172844.458 INFO PET0 COMPLIANCECHECKER:|->|->|->:ATM:>START register compliance check. 20131108 172844.458 INFO PET0 COMPLIANCECHECKER:|->|->|->:ATM: phase Zero for Initialize registered. 20131108 172844.458 INFO PET0 COMPLIANCECHECKER:|->|->|->:ATM: 5 phase(s) of Initialize registered. 20131108 172844.458 INFO PET0 COMPLIANCECHECKER:|->|->|->:ATM: 1 phase(s) of Run registered. 20131108 172844.458 INFO PET0 COMPLIANCECHECKER:|->|->|->:ATM: 1 phase(s) of Finalize registered. 20131108 172844.458 INFO PET0 COMPLIANCECHECKER:|->|->|->:ATM:>STOP register compliance check. 20131108 172844.458 INFO PET0 COMPLIANCECHECKER:|->|->|->:ATM2MED:>START register compliance check. 20131108 172844.458 INFO PET0 COMPLIANCECHECKER:|->|->|->:ATM2MED: phase Zero for Initialize registered. 20131108 172844.458 INFO PET0 COMPLIANCECHECKER:|->|->|->:ATM2MED: 3 phase(s) of Initialize registered. 20131108 172844.458 INFO PET0 COMPLIANCECHECKER:|->|->|->:ATM2MED: 1 phase(s) of Run registered. 20131108 172844.458 INFO PET0 COMPLIANCECHECKER:|->|->|->:ATM2MED: 1 phase(s) of Finalize registered. 20131108 172844.458 INFO PET0 COMPLIANCECHECKER:|->|->|->:ATM2MED:>STOP register compliance check. 20131108 172844.458 INFO PET0 COMPLIANCECHECKER:|->|->|->:MED2ATM:>START register compliance check. 20131108 172844.458 INFO PET0 COMPLIANCECHECKER:|->|->|->:MED2ATM: phase Zero for Initialize registered. 20131108 172844.458 INFO PET0 COMPLIANCECHECKER:|->|->|->:MED2ATM: 3 phase(s) of Initialize registered. 20131108 172844.458 INFO PET0 COMPLIANCECHECKER:|->|->|->:MED2ATM: 1 phase(s) of Run registered. 20131108 172844.458 INFO PET0 COMPLIANCECHECKER:|->|->|->:MED2ATM: 1 phase(s) of Finalize registered. 20131108 172844.458 INFO PET0 COMPLIANCECHECKER:|->|->|->:MED2ATM:>STOP register compliance check. 20131108 172844.459 INFO PET0 COMPLIANCECHECKER:|->|->|->:ATM: >START InitializePrologue for phase= 0 20131108 172844.459 INFO PET0 COMPLIANCECHECKER:|->|->|->:ATM: importState name: modelComp 1 Import State 20131108 172844.459 INFO PET0 COMPLIANCECHECKER:|->|->|->:ATM: importState stateintent: ESMF_STATEINTENT_IMPORT 20131108 172844.459 INFO PET0 COMPLIANCECHECKER:|->|->|->:ATM: importState itemCount: 0 20131108 172844.459 INFO PET0 COMPLIANCECHECKER:|->|->|->:ATM: exportState name: modelComp 1 Export State 20131108 172844.459 INFO PET0 COMPLIANCECHECKER:|->|->|->:ATM: exportState stateintent: ESMF_STATEINTENT_EXPORT 20131108 172844.459 INFO PET0 COMPLIANCECHECKER:|->|->|->:ATM: exportState itemCount: 0 20131108 172844.459 INFO PET0 COMPLIANCECHECKER:|->|->|->:ATM:ESMF Stats: the virtual memory used by this PET (in KB): 974868 20131108 172844.459 INFO PET0 COMPLIANCECHECKER:|->|->|->:ATM:ESMF Stats: the physical memory used by this PET (in KB): 49440 20131108 172844.459 INFO PET0 COMPLIANCECHECKER:|->|->|->:ATM:ESMF Stats: ESMF Fortran objects referenced by the ESMF garbage collection: 0 20131108 172844.459 INFO PET0 COMPLIANCECHECKER:|->|->|->:ATM:ESMF Stats: ESMF objects (F & C++) referenced by the ESMF garbage collection: 0 20131108 172844.459 INFO PET0 COMPLIANCECHECKER:|->|->|->:ATM: >STOP InitializePrologue for phase= 0 20131108 172844.459 INFO PET0 COMPLIANCECHECKER:|<-|<-|<-:ATM: >START InitializeEpilogue for phase= 0 20131108 172844.459 INFO PET0 COMPLIANCECHECKER:|<-|<-|<-:ATM:ESMF Stats: the virtual memory used by this PET (in KB): 974868 20131108 172844.459 INFO PET0 COMPLIANCECHECKER:|<-|<-|<-:ATM:ESMF Stats: the physical memory used by this PET (in KB): 49448 20131108 172844.459 INFO PET0 COMPLIANCECHECKER:|<-|<-|<-:ATM:ESMF Stats: ESMF Fortran objects referenced by the ESMF garbage collection: 0 20131108 172844.459 INFO PET0 COMPLIANCECHECKER:|<-|<-|<-:ATM:ESMF Stats: ESMF objects (F & C++) referenced by the ESMF garbage collection: 0 20131108 172844.459 INFO PET0 COMPLIANCECHECKER:|<-|<-|<-:ATM: GridComp level attribute check: convention: 'NUOPC', purpose: 'General'. 20131108 172844.459 WARNING PET0 COMPLIANCECHECKER:|<-|<-|<-:ATM: ==> Component level attribute: <ShortName> present but NOT set! 20131108 172844.459 WARNING PET0 COMPLIANCECHECKER:|<-|<-|<-:ATM: ==> Component level attribute: <LongName> present but NOT set! 20131108 172844.459 WARNING PET0 COMPLIANCECHECKER:|<-|<-|<-:ATM: ==> Component level attribute: <Description> present but NOT set! 20131108 172844.459 WARNING PET0 COMPLIANCECHECKER:|<-|<-|<-:ATM: ==> Component level attribute: <ModelType> present but NOT set! 20131108 172844.459 WARNING PET0 COMPLIANCECHECKER:|<-|<-|<-:ATM: ==> Component level attribute: <ReleaseDate> present but NOT set! 20131108 172844.459 WARNING PET0 COMPLIANCECHECKER:|<-|<-|<-:ATM: ==> Component level attribute: <PreviousVersion> present but NOT set! 20131108 172844.459 WARNING PET0 COMPLIANCECHECKER:|<-|<-|<-:ATM: ==> Component level attribute: <ResponsiblePartyRole> present but NOT set! 20131108 172844.459 WARNING PET0 COMPLIANCECHECKER:|<-|<-|<-:ATM: ==> Component level attribute: <Name> present but NOT set! 20131108 172844.459 WARNING PET0 COMPLIANCECHECKER:|<-|<-|<-:ATM: ==> Component level attribute: <EmailAddress> present but NOT set! 20131108 172844.459 WARNING PET0 COMPLIANCECHECKER:|<-|<-|<-:ATM: ==> Component level attribute: <PhysicalAddress> present but NOT set! 20131108 172844.459 WARNING PET0 COMPLIANCECHECKER:|<-|<-|<-:ATM: ==> Component level attribute: <URL> present but NOT set! 20131108 172844.459 INFO PET0 COMPLIANCECHECKER:|<-|<-|<-:ATM: Component level attribute: <Verbosity> present and set: high 20131108 172844.459 INFO PET0 COMPLIANCECHECKER:|<-|<-|<-:ATM: Component level attribute: <InitializePhaseMap>[1] present and set: IPDv02p1=1 20131108 172844.460 INFO PET0 COMPLIANCECHECKER:|<-|<-|<-:ATM: Component level attribute: <InitializePhaseMap>[2] present and set: IPDv02p3=2 20131108 172844.460 INFO PET0 COMPLIANCECHECKER:|<-|<-|<-:ATM: Component level attribute: <InitializePhaseMap>[3] present and set: IPDv02p4=3 20131108 172844.460 INFO PET0 COMPLIANCECHECKER:|<-|<-|<-:ATM: Component level attribute: <InitializePhaseMap>[4] present and set: IPDv02p5=5 20131108 172844.460 INFO PET0 COMPLIANCECHECKER:|<-|<-|<-:ATM: Component level attribute: <NestingGeneration> present and set: 0 20131108 172844.460 INFO PET0 COMPLIANCECHECKER:|<-|<-|<-:ATM: Component level attribute: <Nestling> present and set: 0 20131108 172844.460 INFO PET0 COMPLIANCECHECKER:|<-|<-|<-:ATM: importState name: modelComp 1 Import State 20131108 172844.460 INFO PET0 COMPLIANCECHECKER:|<-|<-|<-:ATM: importState stateintent: ESMF_STATEINTENT_IMPORT 20131108 172844.460 INFO PET0 COMPLIANCECHECKER:|<-|<-|<-:ATM: importState itemCount: 0 20131108 172844.460 INFO PET0 COMPLIANCECHECKER:|<-|<-|<-:ATM: exportState name: modelComp 1 Export State 20131108 172844.460 INFO PET0 COMPLIANCECHECKER:|<-|<-|<-:ATM: exportState stateintent: ESMF_STATEINTENT_EXPORT 20131108 172844.460 INFO PET0 COMPLIANCECHECKER:|<-|<-|<-:ATM: exportState itemCount: 0 20131108 172844.460 INFO PET0 COMPLIANCECHECKER:|<-|<-|<-:ATM: The incoming Clock was not modified. 20131108 172844.460 WARNING PET0 COMPLIANCECHECKER:|<-|<-|<-:ATM: ==> The internal Clock is not present! 20131108 172844.460 INFO PET0 COMPLIANCECHECKER:|<-|<-|<-:ATM: >STOP InitializeEpilogue for phase= 0
All of the output generated by the Compliance Checker contains the string COMPLIANCECHECK, which can be used to grep on. The checker currently generates two types of messages, INFO for general analysis output, and WARNING for when issues with respect to the NUOPC Layer rules are detected.
In practice, when dealing with applications that have been componentized down to a very low level of the model, the output generated by the Compliance Checker can become overwhelming. For this reason a depth parameter is available that can be specified for the Compliance Checker environment variable:
ESMF_RUNTIME_COMPLIANCECHECK=ON:depth=4This will limit the number of component levels that the Compliance Checker parses (here 4 levels), starting from the top level application.
The NUOPC Component Explorer is a run-time tool that can be used to gain insight into a NUOPC Layer compliant component, or to test a component's compliance. The Component Explorer is currently available as a separate download from the prototype repository:
https://sourceforge.net/p/esmfcontrib/svn/HEAD/tree/NUOPC/trunk/ComponentExplorer/
There are two parts to the Component Explorer. First the script nuopcExplorerScript is used to compile and link the explorer application specifically against a specified component. This part of the explorer leverages and tests the standardized component dependencies discussed in section 4. This step is initiated by calling the explorer script with the component's mk-file as an argument:
./nuopcExplorerScript <component-mk-file>
Any issues found during this step are reported. The successful completion of this step will produce an executable called nuopcExplorerApp. Success is indicated by
SUCCESS: nuopcExplorerApp successfully built ...exiting nuopcExplorerScript.
and failure by
FAILURE: nuopcExplorerApp failed to build ...exiting nuopcExplorerScript.
The second part of the Component Explorer is the explorer application itself. It can either be built using the explorer script as outlined above (recommended when a makefile fragment for the component is available) or by using the makefile directly:
make nuopcExplorerApp
In the second case the resulting nuopcExplorerApp is not tied to a specific component, instead the executable expects a component in form of a shared object to be specified as a command line argument when executing nuopcExplorerApp. In either case the explorer application needs to be started according to the execution requirements of the component it attempts to explore. This may mean that input files must be present, and that the executable be launched on a sufficient number of processes. In terms of the common mpirun tool, launching of nuopcExplorerApp may look like this
mpirun -np X ./nuopcExplorerAppfor an executable that was built against a specific component. Or like this
mpirun -np X ./nuopcExplorerApp <component-shared-object-file>for an executable that expects a the component in form of a shared object.
The nuopcExplorerApp expects to find a configuration file by the name of explorer.config in the run directory. The configuration file contains several basic model parameter used to explore the component. An example configuration file is shown here:
### NUOPC Component Explorer configuration file ### start_year: 2009 start_month: 12 start_day: 01 start_hour: 00 start_minute: 0 start_second: 0 stop_year: 2009 stop_month: 12 stop_day: 03 stop_hour: 00 stop_minute: 0 stop_second: 0 step_seconds: 21600 filter_initialize_phases: no enable_run: yes enable_finalize: yes
The nuopcExplorerApp starts to interact with the specified component, using the information read in from the configuration file. During the interaction the finding are reported to stdout, with output that will look similar to this:
NUOPC Component Explorer App
----------------------------
Exploring a component with a Fortran module front...
Model component # 1 InitializePhaseMap:
IPDv00p1=1
IPDv00p2=2
IPDv00p3=3
IPDv00p4=4
Model component # 1 // name = ocnA
ocnA: <LongName> : Attribute is present but NOT set!
ocnA: <ShortName> : Attribute is present but NOT set!
ocnA: <Description> : Attribute is present but NOT set!
--------
ocnA: importState // itemCount = 2
ocnA: importState // item # 001 // [FIELD] name = pmsl
<StandardName> = air_pressure_at_sea_level
<Units> = Pa
<LongName> = Air Pressure at Sea Level
<ShortName> = pmsl
ocnA: importState // item # 002 // [FIELD] name = rsns
<StandardName> = surface_net_downward_shortwave_flux
<Units> = W m-2
<LongName> = Surface Net Downward Shortwave Flux
<ShortName> = rsns
--------
ocnA: exportState // itemCount = 1
ocnA: exportState // item # 001 // [FIELD] name = sst
<StandardName> = sea_surface_temperature
<Units> = K
<LongName> = Sea Surface Temperature
<ShortName> = sst
Turning on the Compliance Checker (see section 5.1) will result in additional information in the log files.