31. References

Aberson, S.D., 1998: Five-day Tropical cyclone track forecasts in the North
Atlantic Basin. Weather and Forecasting, 13, 1005-1015.

Adams, S. V., R. W. Ford, M. Hambley, J.M. Hobson, I. Kavčič, C. M. Maynard,
T. Melvin, E. H. Müller, S. Mullerworth, A. R. Porter, M. Rezny, B. J. Shipway,
and R. Wong, 2019: LFRic: Meeting the challenges of scalability and performance
portability in Weather and Climate models. Journal of Parallel and Distributed Computing,
Ahijevych, D., E. Gilleland, B.G. Brown, and E.E. Ebert, 2009: Application of
spatial verification methods to idealized and NWP-gridded precipitation forecasts.
Weather and Forecasting, 24 (6), 1485 - 1497.

Anderson JL., 1996: A method for producing and evaluating probabilistic forecasts
from ensemble model integrations. J. Clim. 9: 1518-1530.

Barker, T. W., 1991: The relationship between spread and forecast error in
extended-range forecasts. Journal of Climate, 4, 733-742.

Bradley, A.A., S.S. Schwartz, and T. Hashino, 2008: Sampling Uncertainty
and Confidence Intervals for the Brier Score and Brier Skill Score.
Weather and Forecasting, 23, 992-1006.

Brill, K. F., and F. Mesinger, 2009: Applying a general analytic method
for assessing bias sensitivity to bias-adjusted threat and equitable
threat scores. Weather and Forecasting, 24, 1748-1754.

Brown, B.G., R. Bullock, J. Halley Gotway, D. Ahijevych, C. Davis,
E. Gilleland, and L. Holland, 2007: Application of the MODE object-based
verification tool for the evaluation of model precipitation fields.
AMS 22nd Conference on Weather Analysis and Forecasting and 18th
Conference on Numerical Weather Prediction, 25-29 June, Park City, Utah,
American Meteorological Society (Boston), Available at

Bröcker J, Smith LA., 2007: Scoring probabilistic forecasts: The importance
of being proper. Weather Forecasting, 22, 382-388.

Buizza, R., 1997: Potential forecast skill of ensemble prediction and spread
and skill distributions of the ECMWF ensemble prediction system. Monthly
Weather Review, 125, 99-119.

Bullock, R., T. Fowler, and B. Brown, 2016: Method for Object-Based
Diagnostic Evaluation. NCAR Technical Note NCAR/TN-532+STR, 66 pp.

Candille G, Côté C, Houtekamer PL, Pellerin G, 2007: Verification of an
ensemble prediction system against observations. Mon. Weather Rev.
135: 2688-2699.

Candille, G., and O. Talagrand, 2008: Impact of observational error on the
validation of ensemble prediction systems. Quarterly Journal of the Royal
Meteorological Society 134: 959-971.

Casati, B., G. Ross, and D. Stephenson, 2004: A new intensity-scale approach
for the verification of spatial precipitation forecasts. Meteorological
Applications 11, 141-154.

Davis, C.A., B.G. Brown, and R.G. Bullock, 2006a: Object-based verification
of precipitation forecasts, Part I: Methodology and application to
mesoscale rain areas. Monthly Weather Review, 134, 1772-1784.

Davis, C.A., B.G. Brown, and R.G. Bullock, 2006b: Object-based verification
of precipitation forecasts, Part II: Application to convective rain systems.
Monthly Weather Review, 134, 1785-1795.

Dawid, A.P., 1984: Statistical theory: The prequential approach. Journal of
the Royal Statistical Society A147, 278-292.

Ebert, E.E., 2008: Fuzzy verification of high-resolution gridded forecasts:
a review and proposed framework. Meteorological Applications, 15, 51-64.

Eckel, F. A., M.S. Allen, M. C. Sittel, 2012: Estimation of Ambiguity in
Ensemble Forecasts. Weather Forecasting, 27, 50-69.

Efron, B. 2007: Correlation and large-scale significance testing. Journal
of the American Statistical Association*, 102(477), 93-103.

Epstein, E. S., 1969: A scoring system for probability forecasts of ranked categories.
Ferro C. A. T., 2017: Measuring forecast performance in the presence of observation error.
Q. J. R. Meteorol. Soc., 143 (708), 2665-2676.

Gilleland, E., 2010: Confidence intervals for forecast verification.
NCAR Technical Note NCAR/TN-479+STR, 71pp.

Gilleland, E., 2017: A new characterization in the spatial verification
framework for false alarms, misses, and overall patterns.
Weather and Forecasting, 32 (1), 187 - 198.

Gilleland, E., 2020: Bootstrap methods for statistical inference.
Part I: Comparative forecast verification for continuous variables.
Journal of Atmospheric and Oceanic Technology, 37 (11), 2117 - 2134.

Gilleland, E., 2020: Bootstrap methods for statistical inference.
Part II: Extreme-value analysis. Journal of Atmospheric and Oceanic
Technology, 37 (11), 2135 - 2144.
Gilleland, E., 2021: Novel measures for summarizing high-resolution forecast
performance. Advances in Statistical Climatology, Meteorology and Oceanography,
7 (1), 13 - 34.

Gneiting, T., A. Westveld, A. Raferty, and T. Goldman, 2004: Calibrated
Probabilistic Forecasting Using Ensemble Model Output Statistics and
Minimum CRPS Estimation. Technical Report no. 449, Department of
Statistics, University of Washington. Available at

Haiden, T., M.J. Rodwell, D.S. Richardson, A. Okagaki, T. Robinson, T. Hewson, 2012:
Intercomparison of Global Model Precipitation Forecast Skill in 2010/11
Using the SEEPS Score. Monthly Weather Review, 140, 2720-2733.

Hamill, T. M., 2001: Interpretation of rank histograms for verifying ensemble
forecasts. Monthly Weather Review, 129, 550-560.

Hersbach, H., 2000: Decomposition of the Continuous Ranked Probability Score
for Ensemble Prediction Systems. Weather and Forecasting, 15, 559-570.

Jolliffe, I.T., and D.B. Stephenson, 2012: Forecast verification. A
practitioner’s guide in atmospheric science. Wiley and Sons Ltd, 240 pp.

Knaff, J.A., M. DeMaria, C.R. Sampson, and J.M. Gross, 2003: Statistical,
Five-Day Tropical Cyclone Intensity Forecasts Derived from Climatology
and Persistence. Weather and Forecasting, Vol. 18 Issue 2, p. 80-92.

Mason, S. J., 2004: On Using “Climatology” as a Reference Strategy
in the Brier and Ranked Probability Skill Scores. Monthly Weather Review,
132, 1891-1895.

Mason, S. J., 2008: Understanding forecast verification statistics.
Meteor. Appl., 15, 31-40.

Mittermaier, M., 2014: A strategy for verifying near-convection-resolving
model forecasts at observing sites. Weather Forecasting, 29, 185-204.

Mood, A. M., F. A. Graybill and D. C. Boes, 1974: Introduction to the
Theory of Statistics, McGraw-Hill, 299-338.

Murphy, A.H., 1969: On the ranked probability score. Journal of Applied
Meteorology and Climatology, 8 (6), 988 - 989,

Murphy, A.H., and R.L. Winkler, 1987: A general framework for forecast
verification. Monthly Weather Review, 115, 1330-1338.

North, R.C., M.P. Mittermaier, S.F. Milton, 2022. Using SEEPS with a
TRMM-derived Climatology to Assess Global NWP Precipitation Forecast Skill.
Monthly Weather Review, 150, 135-155.

Ou, M. H., Charles, M., & Collins, D. C. 2016: Sensitivity of calibrated week-2
probabilistic forecast skill to reforecast sampling of the NCEP global
ensemble forecast system. Weather and Forecasting, 31(4), 1093-1107.

Roberts, N.M., and H.W. Lean, 2008: Scale-selective verification of rainfall
accumulations from high-resolution forecasts of convective events.
Monthly Weather Review, 136, 78-97.

Rodwell, M.J., D.S. Richardson, T.D. Hewson and T. Haiden, 2010: A new equitable
score suitable for verifying precipitation in numerical weather prediction.
Quarterly Journal of the Royal Meteorological Society, 136: 1344-1463.

Rodwell, M.J., T. Haiden, D.S. Richardson, 2011: Developments in Precipitation
Verification. ECMWF Newsletter Number 128.

Röpnack A, Hense A, Gebhardt C, Majewski D., 2013: Bayesian model verification
of NWP ensemble forecasts. Mon. Weather Rev. 141: 375–387.

Saetra Ø., H. Hersbach, J-R Bidlot, D. Richardson, 2004: Effects of
observation errors on the statistics for ensemble spread and
reliability. Monthly Weather Review, 132: 1487-1501.

Santos C. and A. Ghelli, 2012: Observational probability method to assess
ensemble precipitation forecasts. Quarterly Journal of the Royal
Meteorological Society 138: 209-221.

Schwartz C. and Sobash R., 2017: Generating Probabilistic Forecasts from
Convection-Allowing Ensembles Using Neighborhood Approaches: A Review
and Recommendations. Monthly Weather Review, 145, 3397-3418.

Skamarock, W. C., J. B. Klemp, M. G. Duda, L. D. Fowler, S. Park, and
T. Ringler, 2012: A Multiscale Nonhydrostatic Atmospheric Model Using
Centroidal Voronoi Tesselations and C-Grid Staggering. Mon. Wea. Rev.,

Stephenson, D.B., 2000: Use of the “Odds Ratio” for diagnosing
forecast skill. Weather and Forecasting, 15, 221-232.

Stephenson, D.B., B. Casati, C.A.T. Ferro, and C.A. Wilson, 2008: The extreme
dependency score: A non-vanishing measure for forecasts of rare events.
Meteorological Applications 15, 41-50.

Tödter, J. and B. Ahrens, 2012: Generalization of the Ignorance Score:
Continuous ranked version and its decomposition. Monthly Weather Review,
140 (6), 2005 - 2017.

Weniger, M., F. Kapp, and P. Friederichs, 2016: Spatial Verification Using
Wavelet Transforms: A Review. Quarterly Journal of the Royal
Meteorological Society, 143, 120-136.

Wilks, D.S. 2010: Sampling distributions of the Brier score and Brier skill
score under serial dependence. Quarterly Journal of the Royal
Meteorological Society, 136, 2109-2118.

Wilks, D., 2011: Statistical methods in the atmospheric sciences.
Elsevier, San Diego.