NCAR Annual Report > RAL Annual Report Contents > Strategic Priority > 4. Coping with Weather/Climate Hazards

Highlight:  Developmental Testbed Center (DTC)

The Developmental Testbed Center (DTC) is a national facility created in 2003 to facilitate the interaction of the operational and research communities in accelerating the improvement of numerical weather prediction (NWP) for the U.S.  The DTC is part of the national interagency Weather Research Forecast (WRF) program that is governed by the WRF Executive Oversight Board.  The DTC directly supports NCAR's Strategic Goal 2, “Increase societal resilience to weather, climate and other atmospheric hazards”, by facilitating the transfer of new numerical weather prediction technology from research to operations.  The DTC effort at NCAR includes:

The DTC is funded by NSF, NOAA, AFWA, and FAA.

FY06 Accomplishments:


WRF Core Test:  The four panels above show the difference between the RMSE for the ARW and the NMM dynamical cores (left panels) and the bias (right panels) for the 12 hour forecast.  The red curves correspond to the retrospective tests that used the NCEP physics suite and the blue curves correspond to the retrospective tests that used a RUC-like physics suite.  The horizontal lines indicate the 95% confidence interval obtained using a pairwise comparison with a auto-correlation correction.

WRF Core Test

The current WRF Software Framework (WSF) supports two dynamical solvers: the Advanced Research WRF (ARW) developed by the MMM division of NCAR, and the Nonhydrostatic Mesoscale Model (NMM) developed by NCEP.  WRF also offers a variety of physics packages.  The DTC and the Global Systems Division (GSD) of NOAA’s Earth System Research Laboratory (ESRL) addressed the need for a controlled comparison of these two dynamical solvers through intensive retrospective testing with two separate physics suites (i.e., parallel runs of the two dynamical solvers using initial and lateral boundary conditions based on the same input data, as well as the same suite of physics parameterizations).  The goal of the WRF Core Test was to determine the impact of the dynamical solvers on the forecast.  This intensive testing provided much needed information for the dynamical core recommendation made by ESRL-GSD for the WRF Rapid Refresh to be run at NCEP.

The parallel runs produced for the WRF Core Test underwent three basic types of evaluation: objective verification based on standard verification measures (i.e., bias, root-mean-square error, and equitable threat score); subjective case studies investigating systematic differences; and aviation-specific evaluation (both case studies and statistics) by the FAA's Aviation Weather Research Program's Convective Weather, Icing, Ceiling and Visibility, and Turbulence Product Development Teams.  The differences between objective verification measures for the two dynamical cores also underwent an assessment of statistical significance, an approach that is not currently common in the NWP community.

The objective verification measures indicate the two dynamical solvers produce forecasts with comparable skill (i.e., the verification measures for the two dynamical solvers exhibit the same general distribution with height and the differences between these measures are generally small when the two dynamical solvers are run with the same physics suite).  The dynamical core with the smallest error depends on both the field under consideration (i.e., winds, temperature, relative humidity and precipitation) and the level in the atmosphere or the threshold for precipitation accumulation.

DTC Visitor Program


A highlight of the program was Caren Marzban's (University of Washington/University of Oklahoma) investigation of the application of a cluster analysis technique to verify high-resolution precipitation forecasts.  The left panels illustrate the method’s identification of clusters in the observed (top panel) and forecast (bottom panel) precipitation fields and the matching of these clusters (color coding).  The right panel shows an example of Critical Success Indexes obtained through the application of this method to actual forecasts generated for the WRF Spring Program

The DTC Visitor Program funded ten projects in FY06 to study physics parameterizations, data assimilation, and new verification techniques   The culmination of this program was a DTC Visitor Reunion held in Boulder, CO, 17-18 August 2006.  Prior to the reunion, each visitor submitted a report on his/her research project; these reports have been compiled into a volume that the DTC is making available to the community.  In addition, each visitor presented a short summary of results to the group; these presentations are posted on the DTC website.

WRF-NMM Tutorial

The DTC conducted two WRF-NMM Tutorials during FY06 (7-10 February and 8-11 August 2006).  These tutorials included lectures on the pre-processor, model, and post-processing tools, as well as practical sessions that allow the participants to gain hands-on experience building and running each component of the end-to-end system.  The tutorial participants represented a broad cross-section of both the national and international community.

FY07 Plans:

Development of the DTC Verification System

With a recent increase in funding from AFWA, the DTC will begin to assemble a state-of-the-art verification system using components already present in systems developed at NCEP, GSD, and AFWA.  New components, including the addition of routine calculation of confidence intervals and significance statistics and object-based verification methods, will then be added, significantly extending the capability of any current verification system.  Members of the verification community will join DTC staff in a workshop to be held in late winter to guide the development of the new system.  The initial outcome of this effort is expected to be a basic verification capability which includes a unified package of the existing systems and significance tests.  To achieve this goal, two new, full-time DTC positions will be added in early FY07.

WRF Tutorials

Bi-annual WRF Tutorials are planned for FY07 (i.e., winter and summer offerings).    Now that the material for the NMM component of the WRF tutorials is well-established, the ARW and NMM tutorials will be merged into a single WRF tutorial. The goal is to establish a combined tutorial for the summer offering.  The DTC will run one more NMM-specific tutorial in January or February 2007. 

WRF Reference Code Management

One of the main tasks of the DTC is maintenance of the WRF Reference Code, including a series of careful tests to establish baseline verification statistics for all Reference Code components and interoperability between all components.  Code management will be a new task for the DTC, and thus plans are being made to add a software engineer to the DTC staff.

NAM Operational System

The DTC will set up the full NCEP NAM (North American Model) system (minus the data assimilation component) on the DTC computing platforms in order to make the operational system available to the community for testing and evaluation.  The DTC currently has the necessary components of the end-to-end system codes running on DTC computing platforms.  This task will require making updates to the versions of the code running at the DTC, as well as the model and post-processsing configurations.  The DTC system will be tested using sample input and output files from the operational NAM to assure the system has been properly configured.

DTC Visitor Program


Based on the positive response of the 2006 Visitor Program, the DTC is planning another "Announcement of Opportunity" in February 2007. The number of visitor projects the DTC will be able to support under this announcement will depend on the level of funding received for the program.