Bias-Correcting NA-CORDEX: A Case Study in Parallelizing Data Analysis

Date and Time: 
2017 June 29th @ 3pm
ML main seminar
Seth McGinnis

Although many data analysis tasks are highly or even embarrassingly parallel in nature, climate scientists don't typically make much use of parallelism when they're analyzing the output from climate models. I certainly never did, until I was asked to bias-correct a large dataset with a short deadline, and the only way to get it done in time was to do it in parallel. In this talk, I will tell the story of what the process of shifting from serial to parallel data analysis using NCAR cyberinfrastructure looks like from a user perspective: the problem that motivated me to make the switch, hurdles I encountered along the way, what worked and what didn't, how I had to change my thinking and the way I structured the problem, how much effort it took, and what I was able to get out of it in the end.

Speaker Description: 

Seth McGinnis is an Associate Scientist IV in the Institute for Mathematics Applied to Geosciences (IMAGe) at NCAR.  As the Data Manager for the NARCCAP and NA-CORDEX data collections, he makes the output from regional climate models usable by and available to people who need information about climate change in North America.  His research focuses on bias correction, interpolation, and other issues affecting the practical use of model output by non-specialists.

File Slides4.36 MB

Event Category: