29. References
Aberson, S.D., 1998: Five-day Tropical cyclone track forecasts in the North
Atlantic Basin. Weather & Forecasting, 13, 1005-1015.
Ahijevych, D., E. Gilleland, B.G. Brown, and E.E. Ebert, 2009. Application of
spatial verification methods to idealized and NWP-gridded precipitation forecasts.
Weather Forecast., 24 (6), 1485 - 1497, doi: 10.1175/2009WAF2222298.1.
Barker, T. W., 1991: The relationship between spread and forecast error in
extended-range forecasts. Journal of Climate, 4, 733-742.
Bradley, A.A., S.S. Schwartz, and T. Hashino, 2008: Sampling Uncertainty
and Confidence Intervals for the Brier Score and Brier Skill Score.
Weather and Forecasting, 23, 992-1006.
Brill, K. F., and F. Mesinger, 2009: Applying a general analytic method
for assessing bias sensitivity to bias-adjusted threat and equitable
threat scores. Weather and Forecasting, 24, 1748-1754.
Brown, B.G., R. Bullock, J. Halley Gotway, D. Ahijevych, C. Davis,
E. Gilleland, and L. Holland, 2007: Application of the MODE object-based
verification tool for the evaluation of model precipitation fields.
AMS 22nd Conference on Weather Analysis and Forecasting and 18th
Conference on Numerical Weather Prediction, 25-29 June, Park City, Utah,
American Meteorological Society (Boston), Available at
Buizza, R., 1997: Potential forecast skill of ensemble prediction and spread
and skill distributions of the ECMWF ensemble prediction system. Monthly
Weather Review,125, 99-119.
Bullock, R., T. Fowler, and B. Brown, 2016: Method for Object-Based
Diagnostic Evaluation. NCAR Technical Note NCAR/TN-532+STR, 66 pp.
Candille, G., and O. Talagrand, 2008: Impact of observational error on the
validation of ensemble prediction systems. Quarterly Journal of the Royal
Meteorological Society 134: 959-971.
Casati, B., G. Ross, and D. Stephenson, 2004: A new intensity-scale approach
for the verification of spatial precipitation forecasts. Meteorological
Applications 11, 141-154.
Davis, C.A., B.G. Brown, and R.G. Bullock, 2006a: Object-based verification
of precipitation forecasts, Part I: Methodology and application to
mesoscale rain areas. Monthly Weather Review, 134, 1772-1784.
Davis, C.A., B.G. Brown, and R.G. Bullock, 2006b: Object-based verification
of precipitation forecasts, Part II: Application to convective rain systems.
Monthly Weather Review, 134, 1785-1795.
Dawid, A.P., 1984: Statistical theory: The prequential approach. Journal of
the Royal Statistical Society A147, 278-292.
Ebert, E.E., 2008: Fuzzy verification of high-resolution gridded forecasts:
a review and proposed framework. Meteorological Applications, 15, 51-64.
Eckel, F. A., M.S. Allen, M. C. Sittel, 2012: Estimation of Ambiguity in
Efron, B. 2007: Correlation and large-scale significance testing. Journal
of the American Statistical Association,* 102(477), 93-103.
Epstein, E. S., 1969: A scoring system for probability forecasts of ranked categories.
J. Appl. Meteor., 8, 985-987, 10.1175/1520-0450(1969)008<0985:ASSFPF>2.0.CO;2.
Gilleland, E., 2010: Confidence intervals for forecast verification.
NCAR Technical Note NCAR/TN-479+STR, 71pp.
Gilleland, E., 2017. A new characterization in the spatial verification
framework for false alarms, misses, and overall patterns.
Weather Forecast., 32 (1), 187 - 198, doi: 10.1175/WAF-D-16-0134.1.
Gilleland, E., 2020. Bootstrap methods for statistical inference.
Part I: Comparative forecast verification for continuous variables.
Journal of Atmospheric and Oceanic Technology, 37 (11), 2117 - 2134,
doi: 10.1175/JTECH-D-20-0069.1.
Gilleland, E., 2020. Bootstrap methods for statistical inference.
Part II: Extreme-value analysis. Journal of Atmospheric and Oceanic
Technology, 37 (11), 2135 - 2144, doi: 10.1175/JTECH-D-20-0070.1.
Gilleland, E., 2021. Novel measures for summarizing high-resolution forecast
performance. Advances in Statistical Climatology, Meteorology and Oceanography,
7 (1), 13 - 34, doi: 10.5194/ascmo-7-13-2021.
Gneiting, T., A. Westveld, A. Raferty, and T. Goldman, 2004: Calibrated
Probabilistic Forecasting Using Ensemble Model Output Statistics and
Minimum CRPS Estimation. Technical Report no. 449, Department of
Statistics, University of Washington. Available at
Hamill, T. M., 2001: Interpretation of rank histograms for verifying ensemble
forecasts. Monthly Weather Review, 129, 550-560.
Hersbach, H., 2000: Decomposition of the Continuous Ranked Probability Score
for Ensemble Prediction Systems. Weather and Forecasting, 15, 559-570.
Jolliffe, I.T., and D.B. Stephenson, 2012: Forecast verification. A
practitioner’s guide in atmospheric science. Wiley and Sons Ltd, 240 pp.
Knaff, J.A., M. DeMaria, C.R. Sampson, and J.M. Gross, 2003: Statistical,
Five-Day Tropical Cyclone Intensity Forecasts Derived from Climatology
and Persistence. Weather & Forecasting, Vol. 18 Issue 2, p. 80-92.
Mason, S. J., 2004: On Using “Climatology” as a Reference Strategy
in the Brier and Ranked Probability Skill Scores. Monthly Weather Review,
132, 1891-1895.
Mason, S. J., 2008: Understanding forecast verification statistics.
Meteor. Appl., 15, 31-40, doi: 10.1002/met.51.
Mittermaier, M., 2014: A strategy for verifying near-convection-resolving
model forecasts at observing sites. Weather Forecasting, 29, 185-204.
Mood, A. M., F. A. Graybill and D. C. Boes, 1974: Introduction to the
Theory of Statistics, McGraw-Hill, 299-338.
Murphy, A.H., 1969: On the ranked probability score. Journal of Applied
Meteorology and Climatology, 8 (6), 988 - 989,
doi: 10.1175/1520-0450(1969)008<0988:OTPS>2.0.CO;2.
Murphy, A.H., and R.L. Winkler, 1987: A general framework for forecast
verification. Monthly Weather Review, 115, 1330-1338.
Ou, M. H., Charles, M., & Collins, D. C. 2016: Sensitivity of calibrated week-2
probabilistic forecast skill to reforecast sampling of the NCEP global
ensemble forecast system. Weather and Forecasting, 31(4), 1093-1107.
Roberts, N.M., and H.W. Lean, 2008: Scale-selective verification of rainfall
accumulations from high-resolution forecasts of convective events.
Monthly Weather Review, 136, 78-97.
Saetra O., H. Hersbach, J-R Bidlot, D. Richardson, 2004: Effects of
observation errors on the statistics for ensemble spread and
reliability. Monthly Weather Review 132: 1487-1501.
Santos C. and A. Ghelli, 2012: Observational probability method to assess
ensemble precipitation forecasts. Quarterly Journal of the Royal
Meteorological Society 138: 209-221.
Schwartz C. and Sobash R., 2017: Generating Probabilistic Forecasts from
Convection-Allowing Ensembles Using Neighborhood Approaches: A Review
and Recommendations. Monthly Weather Review, 145, 3397-3418.
Stephenson, D.B., 2000: Use of the “Odds Ratio” for diagnosing
forecast skill. Weather and Forecasting, 15, 221-232.
Stephenson, D.B., B. Casati, C.A.T. Ferro, and C.A. Wilson, 2008: The extreme
dependency score: A non-vanishing measure for forecasts of rare events.
Meteorological Applications 15, 41-50.
Tödter, J. and B. Ahrens, 2012: Generalization of the Ignorance Score:
Continuous ranked version and its decomposition. Mon. Wea. Rev.,
140 (6), 2005 - 2017, doi: 10.1175/MWR-D-11-00266.1.
Weniger, M., F. Kapp, and P. Friederichs, 2016: Spatial Verification Using
Wavelet Transforms: A Review. Quarterly Journal of the Royal
Meteorological Society, 143, 120-136.
Wilks, D.S. 2010: Sampling distributions of the Brier score and Brier skill
score under serial dependence. Quarterly Journal of the Royal
Meteorological Society,, 136, 2109-2118. doi:10.1002/qj.709
Wilks, D., 2011: Statistical methods in the atmospheric sciences.
Elsevier, San Diego.