8 Tracking and Metrics

8.1 Release Schedule

The ESMF Release Schedule is generated by the Change Review Board (CRB) (see section 2.3) as the main outcome of its quarterly meetings. The release schedule is posted on the ESMF website home page under Quick Links.

8.2 Task List

The Task List is used to archive past, current, and future development tasks. It's updated by the Core Team Manager during and after quarterly CRB meetings. The CRB bases the Release Schedule on the Task List. The Task List is browsable by anyone and is located at (click on Tasks).

8.3 Trackers

SourceForge provides the development team with a set of trackers. Current trackers are Support Requests, Bugs, Feature Requests, Application Issues, and Operations. These are described in more detail in section [*].

An item entered into a tracker is called a ticket. Tickets are assigned a unique number and can move from tracker to tracker. Tickets may be opened in response to a customer request or as a result of some finding or need of the development team.

The process for handling customer requests is located in Section [*].

All trackers categorize requests similarly. Pull down menus can assign various criteria including Assignee, Status (Open, Closed, etc.), Category (Array, Regrid, etc.), and Group (Bug, Feature Request, etc.) Tickets can be sorted and viewed based on these criteria and can be moved from one tracker to another. For example a ticket may start out as a support request and be moved to the bug tracker for resolution. To move a ticket, simply change the type using the type pull down menu.

8.3.1 Setting Ticket Priorities

The SourceForge trackers allow the user to set the priority of a ticket. Priority nine is used to label tickets that are associated with a task scheduled for release on the Task List. These are the only tickets that should be labeled at this level. Priority levels one through five can be used by individual developers to prioritize tickets that are not associated with any upcoming release.

8.3.2 Labeling Tickets Longer than 2 Weeks

The CRB only manages tickets that will take longer than two calendar weeks. All tickets that are estimated to be two calendar weeks or longer should be labeled with LONG: prepended to the title of the ticket.


8.3.3 Estimating Ticket Completion Time

The following guidelines should be used when determining ticket completion times:

8.3.4 Labeling Tickets with Time Estimates

An estimated time to completion should be placed at the end of any ticket title labeled with the LONG: key. If the time includes a design phase, this should be included in the total estimate e.g. (7) equals 3 weeks design plus 4 additional weeks for completion.

8.3.5 Cross-Referencing the Task List and Trackers

The tasks in the Task List are usually associated with one or more items in the Feature Request, Bug, and Support Request trackers. In order to cross-reference items, developers should label both the item on the Task List and tracker tickets with a unique key. The key should describe the task, be unique, and be followed by a colon. Careful placement of keys will allow the CRB to quickly visualize through the search mechanism all tasks necessary for a new release. The key is highlighted in the example below:

8.3.6 Summary

There are multiple steps to properly identifying tickets. The preceding sections cover these steps, which are summarized here:
  1. Use key words to link tracker tickets with items in the Task List
  2. Label all tickets with LONG: that will take longer than two calendar weeks
  3. Put the completion time estimate after the title (DESIGN + COMPLETION)
  4. Label tickets scheduled for release on the Task List with a priority nine.
  5. Prioritize other tickets using levels one through five.

In Task List: STDIZE_INIT: Standardize Initialization Behavior

In Bug Tracker:

8.3.7 Types of Trackers

8.4 Metrics

The Integrator is responsible for tracking various aspects of the ESMF project to measure the implementation and testing progress.

8.4.1 Unit and System Tests Coverage

Throughout the ESMF software development process, the software is constantly being unit and system tested. It is essential to know the percentage of the code that has been fully or partially tested. A script has been written that lists all the public interface subprograms (functions and subroutines) and determines which of these have been unit and/or system tested. From this script output, the tested percentage can be calculated.

8.4.2 ESMF Requirements Coverage

An Excel spreadsheet is maintained listing the requirements as described in the ESMF Requirements Document. A hard copy of the spreadsheet has been posted in a convenient place for the core team. Using color coding the core team indicates which requirements have been implemented and of these which have been tested.

8.4.3 Source Lines of Code, SLOC

The ESMF SLOC information is provided in an internal ESMF web page. The SLOC data is presented in graph form with the following columns:

SLOC Total
Lines of text

The graphs are a monthly breakdown from January 1, 2002 to the present.