Ensemble Consistency Testing: a new form of quality assurance

Date and Time: 
Tuesday, April 5th, 2016
Center Green
Dorit Hammerling

Climate simulation codes, such as the Community Earth System Model (CESM), are especially complex and continually evolving. Their ongoing state of development requires frequent software verification in the form of quality assurance to both preserve the quality of the code and instill model confidence. To formalize and simplify this previously subjective and computationally expensive aspect of the verification process, we have developed a new tool for evaluating climate consistency, the CESM ensemble consistency test, referred to as CESM-ECT. An ensemble of CESM climate runs is created, from which we obtain a statistical distribution that can be used to determine whether a new climate run is statistically distinguishable from the original ensemble. The tool has so far been developed for the atmosphere and ocean components, and while both rely on ensembles, the underlying testing algorithms are surprisingly different due to the number and characteristics of the respective variables involved. The tool has already proven its utility in detecting errors in software and hardware environments and providing rapid feedback to model developers. We will describe the tool and present results from theoretical and practical applications. We will also describe our latest efforts to develop an ultrafast version, which only requires nine model steps (equivalent to 4.5 hours in model time) and which has shown promising preliminary results.

Speaker Description: 

Dorit Hammerling is a researcher at the Institute for Mathematics Applied to Geosciences, in the National Center for Atmospheric Research. She is interested in spatio-temporal statistical methods applied to the geosciences, with a focus on massive data from satellites and climate models. To that end, she is interested in ways to optimally use the high performance computing facilities available at NCAR.

PDF icon ECT_SEA2016_Hammerling.pdf7.27 MB

Event Category: