While there is no one right way to write the NUOPC Model cap code, the following recommended steps represent an incremental approach to developing the cap.
The primary prerequisite software is the NUOPC library, which is included with the ESMF distribution, and your model, including any of its dependencies.
Acquire ESMF version 7 from SourceForge:
$ git archive --remote=git://git.code.sf.net/p/esmf/esmf \
--format=tar --prefix=esmf/ ESMF\_7\_0\_0 | tar xf -
Compile and install ESMF. Full installation details can be found in the ESMF User Guide. An example of the basic procedure for one particular system is outlined below.
# set environment variables for build # the actual settings depend on your platform # and the compilation options you select $ export ESMF_DIR=/path/to/esmf $ export ESMF_COMPILER=gfortran $ export ESMF_COMM=openmpi $ export ESMF_PIO=internal $ export ESMF_NETCDF=split $ export ESMF_NETCDF_INCLUDE=/usr/include $ export ESMF_NETCDF_LIBS="-lnetcdff -lnetcdf" $ export ESMF_NETCDF_LIBPATH=/usr/lib $ export ESMF_INSTALL_PREFIX=/path/to/install # build $ cd /path/to/esmf $ gmake $ gmake check $ gmake install
The model also needs to be roughly divided into several execution methods: initialize, run, and finalize. Each of these methods may contain several phases. The run method should allow the model to execute a single timestep, or accept a parameter defining the number of timesteps or a ``run until'' time.
Your NUOPC cap code will be cleanest if your model exposes data structures for input and output variables with clear, well-documented naming conventions. This will simplify the process of hooking up fields in the NUOPC cap to your model's data structures. The NUOPC Field Dictionary uses the Climate and Forecast conventions for defining field standard names, but can support field name aliases.
Finally, the model should not use the global MPI_WORLD_COMM communicator explicitly, but should accept a communicator at some point during startup. A global search and replace can be used to replace all uses of MPI_WORLD_COMM with a different communicator defined as a global variable in your model.
You should choose a configuration of your model that is simple and stable. Many models have regression test configurations that can be run quickly and have small output files. These configurations are typically low resolution, have short execution times, and sometimes have idealized initial conditions. Some models can also be configured with some of the physics options turned off to reduce the total amount of computation. More scientifically interesting or higher resolution configurations can be used after ensuring that the NUOPC cap is working for the basic case.
Compile your model on the target system and generate baseline output for the selected configuration. This will typically be a small set of history or restart files. We'll use these files later to ensure that your model is reproducing the expected output when executed through the NUOPC cap. In most cases, when your model is executed through its NUOPC cap, the output should be bit-for-bit identical with non-NUOPC runs. (The one caveat to this is that when your model is used in a coupled system, roundoff error may occur due to slight differences introduced when grid interpolation is used between models.)
If your model is already using ESMF, you will need to update your build to link against ESMF version 7 or later. Instructions for checking out this version of ESMF appear in section 3.1.
Including the cap code in your model's codebase does not imply that your model must always be run in NUOPC mode. Instead, when the cap is complete, the NUOPC mode can be viewed as a configuration option of your model.
You need not start from scratch. Instead start with a NUOPC cap template. To acquire a cap template you can:
Put the initial cap code into your model source tree. Then, modify your Makefile or build scripts so that the cap is compiled with the rest of your model code. Unless your model is already using ESMF, you'll need to add ESMF compile and linking flags in order to build the cap. When ESMF is installed, a Makefile fragment named esmf.mk is generated that contains variables that can be appended to your compile and link flags. The ESMF User Guide explains how to use these variables in your Makefile.
In either case, to simplify the process of compiling and linking against your model, your model's build process should produce a Makefile fragment file that defines the following six variables:
An example makefile fragment useful for statically linking against your model looks like this:
#file: abc.mk ESMF\_DEP\_FRONT = ABC ESMF\_DEP\_INCPATH = <absolute path to associated ABC module file> ESMF\_DEP\_CMPL\_OBJS = <absolute path>/abc.o ESMF\_DEP\_LINK\_OBJS = <absolute path>/abc.o <absolute path>/xyz.o ESMF\_DEP\_SHRD\_PATH = ESMF\_DEP\_SHRD\_LIBS =
The variables in the makefile fragment expose a set of dependencies that the higher-level build system can use to compile and link against your model. An easy way to generate the makefile fragment is to modify your model's Makefile to include a new target:
.PRECIOUS: %.o
%.mk: %.o
@echo "# ESMF self-describing build dependency makefile fragment" > $@
@echo >> $@
@echo "ESMF\_DEP\_FRONT = ABC" >> $@
@echo "ESMF\_DEP\_INCPATH = `pwd`" >> $@
@echo "ESMF\_DEP\_CMPL\_OBJS = `pwd`/"$< >> $@
@echo "ESMF\_DEP\_LINK\_OBJS = "$(addprefix `pwd`/, $(OBJS)) >> $@
@echo "ESMF\_DEP\_SHRD\_PATH = " >> $@
@echo "ESMF\_DEP\_SHRD\_LIBS = " >> $@
abc.mk: $(OBJS)
The Standardized Component Dependencies section of the NUOPC Reference Manual contains more details on setting up NUOPC makefile fragments.
Finally, if your build procedure typically produces an executable, it is recommended that you add a Makefile target (or similar build option) that produces a library instead of an executable. When used in a NUOPC system, your model's main program will not be used-instead, a NUOPC_Driver will be linked to your cap and it will be the locus of control (i.e., the main program).
Makefile Target Conventions
If your model is built using Make, a common convention is to add two special targets that build your model and also compile the NUOPC code you will write.
# this target builds your model and your NUOPC cap $ make nuopc # this target installs your NUOPC-compliant model to a particular directory $ make nuopcinstall DESTDIR=/path/to/install
NUOPC defines a precise initialization sequence-i.e., a series of steps that all NUOPC components are expected to take when starting up. Some of the steps are optional and some are required. This initialization sequence is encoded in the Initialize Phase Definition (IPD), which includes several different versions in order to allow for extension of the initialization sequence for future releases of NUOPC and to support backward compatibility.
Instead of tackling the full NUOPC initialization sequence at this point in developing your cap, we recommend that you start by adding calls in your cap's first initialization phase to your model's existing initialization subroutine(s). A good place to do this is within the Advertise Fields initialization phase. This is the phase where each component ``advertises'' the fields it requires and can potentially provide.
You will need to add use statements at the top if your cap to import the relevant initialization subroutines from your model into the NUOPC cap module. The example code in section 3.11 shows where to add the call to your model's initialization subroutine(s).
In the next section you will add another call into your model code before attempting to execute your NUOPC cap.
This call should only move the model forward a single timestep, not the full run length. If the subroutine requires a parameter such as the timestep length or the time to stop, then these parameters can be retrieved from the cap's ESMF_Clock object.
If your model does not have a subroutine that takes a single timestep, you will need to create one now.
One option for testing the cap is to run it using the NUOPC Component Explorer, a specialized NUOPC_Driver designed to execute any NUOPC_Model. Complete instructions for acquiring the Component Explorer and linking it to your NUOPC cap are available.
The instructions above also describe how to turn on the NUOPC Compliance Checker while running the Component Explorer. The Compliance Checker produces additional output in the ESMF log files that is useful for debugging. It also produces WARNINGS in the logs if a compliance issue is identified. When running with the basic cap, you should not necessarily expect to have all compliance issues resolved.
To validate that the NUOPC cap is faithfully reproducing your model's behavior when run in non-NUOPC mode, you should compare your model's output when run with the NUOPC cap against a baseline run. This is the best test to ensure that the cap is working correctly. If the NUOPC cap reproduces your baseline run, you are ready to integrate your NUOPC Model cap into a coupled system with other NUOPC components.
The following code is a starting point for creating a basic NUOPC Model cap.
module MYMODEL
!-----------------------------------------------------------------------------
! Basic NUOPC Model cap
!-----------------------------------------------------------------------------
use ESMF
use NUOPC
use NUOPC_Model, &
model_routine_SS => SetServices, &
model_label_Advance => label_Advance
! add use statements for your model's initialization
! and run subroutines
implicit none
private
public :: SetServices
!-----------------------------------------------------------------------------
contains
!-----------------------------------------------------------------------------
subroutine SetServices(model, rc)
type(ESMF_GridComp) :: model
integer, intent(out) :: rc
rc = ESMF_SUCCESS
! the NUOPC model component will register the generic methods
call NUOPC_CompDerive(model, model_routine_SS, rc=rc)
if (ESMF_LogFoundError(rcToCheck=rc, msg=ESMF_LOGERR_PASSTHRU, &
line=__LINE__, &
file=__FILE__)) &
return ! bail out
! set entry point for methods that require specific implementation
call NUOPC_CompSetEntryPoint(model, ESMF_METHOD_INITIALIZE, &
phaseLabelList=(/"IPDv04p1"/), userRoutine=AdvertiseFields, rc=rc)
if (ESMF_LogFoundError(rcToCheck=rc, msg=ESMF_LOGERR_PASSTHRU, &
line=__LINE__, &
file=__FILE__)) &
return ! bail out
call NUOPC_CompSetEntryPoint(model, ESMF_METHOD_INITIALIZE, &
phaseLabelList=(/"IPDv04p3"/), userRoutine=RealizeFields, rc=rc)
if (ESMF_LogFoundError(rcToCheck=rc, msg=ESMF_LOGERR_PASSTHRU, &
line=__LINE__, &
file=__FILE__)) &
return ! bail out
! attach specializing method(s)
call NUOPC_CompSpecialize(model, specLabel=model_label_Advance, &
specRoutine=ModelAdvance, rc=rc)
if (ESMF_LogFoundError(rcToCheck=rc, msg=ESMF_LOGERR_PASSTHRU, &
line=__LINE__, &
file=__FILE__)) &
return ! bail out
end subroutine
!-----------------------------------------------------------------------------
subroutine AdvertiseFields(model, importState, exportState, clock, rc)
type(ESMF_GridComp) :: model
type(ESMF_State) :: importState, exportState
type(ESMF_Clock) :: clock
integer, intent(out) :: rc
rc = ESMF_SUCCESS
! Eventually, you will advertise your model's import and
! export fields in this phase. For now, however, call
! your model's initialization routine(s).
! call my_model_init()
end subroutine
!-----------------------------------------------------------------------------
subroutine RealizeFields(model, importState, exportState, clock, rc)
type(ESMF_GridComp) :: model
type(ESMF_State) :: importState, exportState
type(ESMF_Clock) :: clock
integer, intent(out) :: rc
rc = ESMF_SUCCESS
! Eventually, you will realize your model's fields here,
! but leave empty for now.
end subroutine
!-----------------------------------------------------------------------------
subroutine ModelAdvance(model, rc)
type(ESMF_GridComp) :: model
integer, intent(out) :: rc
! local variables
type(ESMF_Clock) :: clock
type(ESMF_State) :: importState, exportState
rc = ESMF_SUCCESS
! query the Component for its clock, importState and exportState
call NUOPC_ModelGet(model, modelClock=clock, importState=importState, &
exportState=exportState, rc=rc)
if (ESMF_LogFoundError(rcToCheck=rc, msg=ESMF_LOGERR_PASSTHRU, &
line=__LINE__, &
file=__FILE__)) &
return ! bail out
! HERE THE MODEL ADVANCES: currTime -> currTime + timeStep
! Because of the way that the internal Clock was set by default,
! its timeStep is equal to the parent timeStep. As a consequence the
! currTime + timeStep is equal to the stopTime of the internal Clock
! for this call of the ModelAdvance() routine.
call ESMF_ClockPrint(clock, options="currTime", &
preString="------>Advancing MODEL from: ", rc=rc)
if (ESMF_LogFoundError(rcToCheck=rc, msg=ESMF_LOGERR_PASSTHRU, &
line=__LINE__, &
file=__FILE__)) &
return ! bail out
call ESMF_ClockPrint(clock, options="stopTime", &
preString="--------------------------------> to: ", rc=rc)
if (ESMF_LogFoundError(rcToCheck=rc, msg=ESMF_LOGERR_PASSTHRU, &
line=__LINE__, &
file=__FILE__)) &
return ! bail out
! Call your model's timestep routine here
! call my_model_update()
end subroutine
end module