Forecast Verification Reading Group

The purpose of this reading group is to discuss papers relating to forecast verification. Currently, we will be discussing papers related to the issue of verifying Tropical Cyclones. However, other topics that may come up either in conjunction with this overall topic, or in addition to this topic include:

• Ensemble Verification Methods

• Confidence Intervals and Hypothesis Testing -- Including such issues as spatial/temporal dependence, multiple testing, Non-Gaussian Random Variables, bootstrap intervals, Bayesian precision intervals, Observational Error.

• Verification of Extreme or Rare Events

• Spatial Forecast Verification

• User-specific verification

• Other

Group organizer: Eric Gilleland

Schedule

Our next meeting will be TBD, and will cover the paper: Elmore, K. L., M. E. Baldwin, and D. M. Schultz, 2006. Field significance revisited: Spatial bias errors in forecasts as applied to the Eta model. Mon. Wea. Rev., 134, 519--531.

Paper/Seminar	Time/Place	Presenter	Presentation slides (if any)
Elmore, K. L., M. E. Baldwin, and D. M. Schultz, 2006. Field significance revisited: Spatial bias errors in forecasts as applied to the Eta model. Mon. Wea. Rev., 134, 519--531.	TBD	Randy Bullock	-
Seminar Presentation: Some insights into cloud forecast biases and skill: Putting data and methods under the spotlight (pdf announcement/abstract)	Tuesday, 23 August 2011 at 3:00pm/FL2-1001	Marion Mittermeier	-
2011 Series on Tropical Cyclone (and similar) Forecast Verification
Discussion of methods used officially by the National Hurricane Center	Thursday, 6 October 2011 at 10:00am/FL3-1029	Mrinal Biswas	-
Dupont, T., M. Plu, P. Caroff, and G. Faure, 2011. Verification of ensemble-based uncertainty circles around tropical cyclone track forecasts. Accepted to Wea. Forecasting, Available at Early Online Release	Thursday, 18 August 2011 at 3:00pm/FL3-1029	Eric Gilleland	-
Davis, C. A., W. Wang, J. Dudhia and R. Torn, 2010: Does increased horizontal resolution improve hurricane wind forecasts? Wea. Forecasting, 25, (6), 1826--1841.	Thursday, 7 July 2011 at 10:00am/FL3-1021	Ligia Bernadet	-
Colle, B.A. and Charles, M.E., 2011: Spatial Distribution and Evolution of Extratropical Cyclone Errors over North America and Adjacent Oceans in the NCEP Global Forecast System Model. Wea. Forecasting, 26, 129--149. Available at Early Online Release from: Early Online Release	Thursday, 26 May 2011 at 2:00pm/FL3-1029	Eric Gilleland	-

Suggested Papers

If there are any other papers you would like us to discuss or add to the list, please email the organizer (Eric Gilleland).

A beer glass () by a paper indicates papers for upcoming sessions.

Tropical Cyclone (and similar) Forecast Verification

Aberson, S. D., 2008. An alternative tropical cyclone intensity forecast verification technique Wea. Forecasting, 23, 1304--1310.

Done, J., G. J. Holland, C. Bruyère, and A. Suzuki-Parker, 2011. Effects of climate variability and change on Gulf of Mexico tropical cyclone activity. Proc. Offshore Technology Conference, 22190 (http://www.onepetro.org/mslib/servlet/onepetropreview?id=OTC-22190-MS&soc=OTC)

Gneiting, T., Stanberry, L. I., Grimit, E. P., Held, L. and Johnson, N. A. (2008). Assessing probabilistic forecasts of multivariate quantities, with applications to ensemble predictions of surface winds. Test, 17, 211-235. (Also has comment papers and a rejoinder)

Holland, G. J., J. Done, C. Bruyère, C. Cooper, and A. Suzuki-Parker, 2010. Model investigations of the effects of climate variability and change on future Gulf of Mexico tropical cyclone activity. Proc. 2010 Offshore Technology Conference, 20690. (http://www.onepetro.org/mslib/servlet/onepetropreview?id=OTC-20690-MS&soc=OTC).

Moskaitis, J. R., 2008. A Case study of deterministic forecast verification: tropical cyclone intensity. Wea. Forecasting, 23, 1195--1220.

Extreme and Rare Events

Atger F, 2001. Verification of intense precipitation forecasts from single models and ensemble prediction systems. Nonlin. Proc. Geophys., 8:401--417

Brooks HE, M Kay and JA Hart, 1998. Objective limits on forecasting skill of rare events. 19th Conf. Severe Local Storms, Amer. Met. Soc., 552--555

Ferro, C.A.T., 2007. A Probability Model for Verifying Deterministic Forecasts of Extreme Events. Wea. Forecasting, 22(5):1089--1100, DOI: 10.1175/WAF1036.1

Ferro, C.A.T. and D.B. Stephenson, 2011: Extremal dependence indices: improved verification measures for deterministic forecasts of rare binary events. Accepted to Wea. Forecasting. Available at Early Online Release from: http://journals.ametsoc.org/doi/pdf/10.1175/WAF-D-10-05030.1

Friederichs, P., M. Göber, S. Bentzien, A. Lenz, and R. Krampitz, 2009. A probabilistic analysis of wind gusts using extreme value statistics. Meteorol. Z., 18 (6), 615--629, DOI 10.1127/0941-2948/2009/0413.

Stephenson DB, B Casati, CAT Ferro and CA Wilson, 2008. The extreme dependency score: a non-vanishing measure for forecasts of rare events. Meteorol. Appl., 15:41--50, DOI: 10.1002/met.53

Spatial Forecast Verification

Elmore, K. L., M. E. Baldwin, and D. M. Schultz, 2006. Field significance revisited: Spatial bias errors in forecasts as applied to the Eta model. Mon. Wea. Rev., 134, 519--531.

A list of spatial verification papers that is relatively up-to-date can be found at the ICP web site http://ww.ral.ucar.edu/projects/icp

Email Eric Gilleland to be added to, or removed from, the reading group mailing list.*

^{*Email addresses are retained only for announcing pertinent information about the reading group (e.g., meeting times, papers to be read, etc.). If you are on the mailing list and want to be removed, please email Eric Gilleland and request to be removed from the list.}