Forecast Verification Reading Group
- About
- Schedule
- Suggested Papers
- Join/Leave the Group
The purpose of this reading group is to discuss papers relating to forecast verification. Currently, we will be discussing papers related to the issue of verifying high-resolution gridded forecasts with gridded observations/analyses, sometimes referred to as Spatial Forecast Verification. However, other topics that may come up either in conjunction with this overall topic, or in addition to this topic include:
• Confidence Intervals and Hypothesis Testing -- Including such issues as spatial/temporal dependence, multiple testing, Non-Gaussian Random Variables, bootstrap intervals, Bayesian precision intervals, Observational Error.
• Verification of Extreme or Rare Events
• User-specific verification
• Other
Group organizer: Eric Gilleland
Our next meeting will be a seminar talk by Johan Lindström on "Image Warping for Forecast Verification" to be held on Tuesday, June 3 at 10:45am/FL-MSR (FL2-1022). This talk will be followed by a lunchtime meeting (12:15/EOL-Atrium (FL1-2198)) to discuss the Alexander et al. paper (see below for further details and abstract).
| Paper/Seminar | Time/Place | Presenter | Presentation slides (if any) |
| Alexander GD, JA Weinman, VM Karyampudi, WS Olson, and ACL Lee, 1999. The effect of assimilating rain rates derived from satellites and lightning on forecasts of the 1993 superstorm. Mon. Wea. Rev., 127:1433--1457. | Tuesday, June 3 at 12:15/EOL-Atrium (FL1-2198) | TBD | - |
| Seminar Presentation: Image Warping for Forecast Verification (abstract, txt) | Tuesday, June 3 at 10:45am/FL-MSR (FL2-1022) | Johan Lindström | - |
| Seminar Presentation: High-resolution time lagged ensembles: walking the resolution-predictability tightrope (abstract, txt) | Wednesday, April 23 at 3:00pm/FL-MSR (FL2-1022) | Marion Mittermaier | - |
| Seminar Presentation: A wavelet-based verification approach to account for the variation in scale representativeness of observation networks (abstracts, txt) | Tuesday, April 8 at 3:00pm/FL-MSR (FL2-1022) | Barbara Casati | - |
| Venugopal, V.; Basu, S.; Foufoula-Georgiou, E. 2005. A new metric for comparing precipitation patterns with an application to ensemble forecasts. J. Geophys. Res. 110: D8, D08111 10.1029/2004JD005395. | Friday, April 4 at 10:00am/FL1-2133 | David Ahijevych | (ppt, pdf) |
| Roberts NM and HW Lean, 2008. Scale-selective verification of rainfall accumulations from high-resolution forecasts of convective events. Mon. Wea. Rev., 136:78--96. | Thursday, March 13 at 3:00pm/FL3-2072 | Eric Gilleland |
If there are any other papers you would like us to discuss or add to the list, please email the organizer (Eric Gilleland).
A beer glass (
) by a paper indicates papers for upcoming sessions.
Extreme and Rare Events
Ferro, C.A.T., 2007. A Probability Model for Verifying Deterministic Forecasts of Extreme Events. Wea. Forecasting, 22(5):1089--1100, DOI: 10.1175/WAF1036.1
Stephenson DB, B Casati, CAT Ferro and CA Wilson, 2008. The extreme dependency score: a non-vanishing measure for forecasts of rare events. Meteorol. Appl., 15:41--50, DOI: 10.1002/met.53
Spatial Forecast Verification
Alexander GD, JA Weinman, VM Karyampudi, WS Olson, and ACL Lee, 1999. The effect of assimilating rain rates derived from satellites and lightning on forecasts of the 1993 superstorm. Mon. Wea. Rev., 127:1433--1457.
Alexander GD, JA Weinman, and JL Schols, 1998. The use of digital image warping of microwave integrated water vapor imagery to improve forecasts of marine extratropical cyclones. Mon. Wea. Rev., 126:1469--1496.
Atger F, 2001. Verification of intense precipitation forecasts from single models and ensemble prediction systems. Nonlin. Proc. Geophys., 8:401--417
Baldwin ME and JS Kain, 2006. Sensitivity of several performance measures to displacement error, bias, and event frequency. Wea. Forecasting, 21:636--648
Basu S, B Dodov, and E Foufoula-Georgiou, 2003. A novel measure for QPF verification and its usefulness in multimodel ensemble forecasting, Geophys. Res. Abstracts, 5, 04323, European Geophysical Society.
Brooks HE, M Kay and JA Hart, 1998. Ob jective limits on forecasting skill of rare events. 19th Conf. Severe Local Storms, Amer. Met. Soc., 552--555
Casati B, G Ross, and DB Stephenson, 2004. A new intensity-scale approach for the verification of spatial precipitation forecasts. Meteorol. Appl. 11:141--154.
Damrath U, 2004. Verification against precipitation observations of a high den- sity networkwhat did we learn? Intl. Verification Methods Workshop, 15-17 September 2004, Montreal, Canada.
Davis CA, BG Brown, and RG Bullock, 2006a. Object-based verification of precipitation forecasts, Part I: Methodology and application to mesoscale rain areas. Mon. Wea. Rev. 134:1772--1784.
Davis CA, BG Brown, and RG Bullock, 2006b. Object-based verification of precipitation forecasts, Part II: Application to convective rain systems. Mon. Wea. Rev. 134:1785--1795.
Ebert EE, 2006. Fuzzy verification of high resolution gridded forecasts: A review and proposed framework. Submitted to Meteorol. Appl.
Ebert EE and JL McBride, 2000. Verification of precipitation in weather systems: determination of systematic errors. J. Hydrology, 239:179--202.
Germann U and I Zawadzki, 2004. Scale dependence of the predictability of precipitation from continental radar images. Part II: Probability forecasts. J. Appl. Meteorol., 43:74--89
Gilleland E, TCM Lee, J Halley Gotway, RG Bullock, and BG Brown, 2007. Computationally efficient spatial forecast verification using Baddeley's Δ image metric. Accepted to Mon. Wea. Rev.
Grams JS, WA Gallus Jr., SE Koch, LS Wharton, A Loughe, and EE Ebert, 2006. The Use of a Modified Ebert-McBride Technique to Evaluate Mesoscale Model QPF as a Function of Convective System Morphology during IHOP 2002. Wea. Forecasting, 21:288--306.
Harris D, E Foufoula-Georgiou, KK Droegemeier, JJ Levit, 2001. Multiscale statistical properties of a high-resolution precipitation forecast. J. Hydrometeorology 2:406--418.
Hoffman RN and C Grassotti, 1996. A technique for assimilating SSM/I observations of marine atmospheric storms: Tests with ECMWF analyses. J. Appl. Meteorol., 35:1177--1188.
Hoffman RN, Z Liu, J-F Louis, and C Grassotti. 1995. Distortion representation of forecast errors. Mon. Wea. Rev.123:2758--2770.
Kain JS, SJ Weiss, ME Baldwin, GW Carbin, DR Bright, JJ Levit, and JA Hart, 2005: Evaluating high-resolution configurations of the WRF model that are used to forecast severe convective weather: The 2005 SPC/NSSL Spring Program. Preprints, 21th Conference on Weather Analysis and Forecasting/17th Conference on Numerical Weather Prediction, Washington, D. C., Amer. Meteor. Soc., CD-ROM, 2A.5.
Keil C and GC Craig, 2006. A displacement-based error measure applied in a regional ensemble forecasting system. Submitted to Mon. Wea. Rev.
Lin Y and KE Mitchell, 2005. The NCEP Stage II/IV hourly precipitation analyses: Development and applications. 19th Conf on Hydrology, Amer. Meteorol. Soc. 1.2 Available at http://www.emc.ncep.noaa.gov/mmb/ylin/pcpanl/refs/stage2-4.19hydro.pdf
Marsigli, Boccanera, A Montani, and T Paccagnella, 2005. The COSMO-LEPS ensemble system: validation of the methodology and verification, Nonlinear Processes in Geophysics, 12:527--536. Available at http://www.nonlin-processes-geophys.net/12/issue4.html.
Marsigli C, A Montani, and T Paccagnella, 2006. Verification of the COSMO-LEPS new suite in terms of precipitation distribution, COSMO Newsletter No. 6, available at http://www.cosmo-model.org/public/newsLetters.htm
Marzban, C., S. Sandgathe, and H. Lyons 2007: An Object-oriented Verification of Three NWP Model Formulations via Cluster Analysis: An objective and a subjective analysis. Conditionally accepted to Mon. Wea. Rev. (Available at: http://faculty.washington.edu/marzban/)
Marzban, C., S. Sandgathe, and H. Lyons 2007: Assessment of an automatic, object-oriented approach to the verification of spatial fields. Paper presented at 7th Euopean Meteorological Society Annual Meeting, El Escorial, Spain, October. (Available at: http://faculty.washington.edu/marzban/)
Marzban, C., S. Sandgathe, 2006a: Cluster Analysis for Ob ject-Oriented Verification of Fields: A Variation. Submitted to Monthly Weather Review.
Marzban, C., S. Sandgathe, 2006b: Cluster analysis for verification of precipitation fields. Wea. Forecasting, 21(5):824-838.
Mesinger, F., 2007: Bias adjusted precipitation threat scores. Submitted to Wea. Forecasting. (pdf)
Micheas AC, NI Fox, SA Lack, and CK Wikle. 2007. Cell identification and verification of QPF ensembles using shape analysis techniques. J. of Hydrology, 343:105--116.
Mittermaier, MP, 2006. Using an intensity-scale technique to assess the added benefit of high-resolution model precipitation forecasts. Atmos. Sci. Lett., 7(2):35--42
Nachamkin, J. E., S. Chen, and J. S.Schmidt 2005: Evaluation of heavy precipitation forecasts using composite-based methods: A distributions-oriented approach. Mon. Wea. Rev., 133:2163-2177.
Nachamkin, J. E., 2004: Mesoscale verification using meteorological composites. Mon. Wea. Rev., 132:941-955.
Nehrkorn T, RN Hoffman, C Grassotti, and J-F Louis. 2003. Feature calibration and alignment to represent forecast errors: Empirical regularization. Q.J.R. Meteorol. Soc., 129:195--218. doi: 10.1256/qj.02.18.
Rezacova D, Z Sokol, and P Pesice (2005) A radar-based verification of precipitation forecast for local convective storms. Atmos. Res., in press.
Roberts, 2005. An investigation of the ability of a storm-scale configuration of the Met Office NWP model to predict flood-producing rainfall. Forecasting Research Tech. Rept. 455, Met Office, 80 pp.
Tustison B, D Harris and E Foufoula-Georgiou, 2001. Scale issues in verification of precipitation forecasts. J. of Geophysical Res. 106(D11):11,775--11,784.
Venugopal, V.; Basu, S.; Foufoula-Georgiou, E. 2005. A new metric for comparing precipitation patterns with an application to ensemble forecasts. J. Geophys. Res. 110: D8, D08111 10.1029/2004JD005395.
Wernli H, et al., 2007 Structure Amplitude Location (SAL). Manuscript in preparation for MWR.
Weygandt SS, AF Loughe, SG Benjamin, and JL Mahoney, 2004. Scale sensitivities in model precipitation skill scores during IHOP. 22nd Conf. Severe Local Storms, Amer. Met. Soc., 4-8 October 2004, Hyannis, M.A.
Yates ES, S Anquetin, V Ducrocq, J-D Creutin, D Richard, and K Chancibault, 2006. Point and areal validation of forecast precipitation fields. Meteorol. Appl., 13:1--20
Zepeda-Arce J, E Foufoula-Georgiou, and KK Droegemeier, 2000. Space-time rainfall organization and its role in validating quantitative precipitation forecasts. J. Geophys. Res., 105(D8):10,12910,146
*Email addresses are retained only for announcing pertinent information about the reading group (e.g., meeting times, papers to be read, etc.). If you are on the mailing list and want to be removed, please email Eric Gilleland and request to be removed from the list.