Using open source tools to reduce large data sets in a distributed environment.

Date and Time: 
Monday, April 4th, 2016
Center Green
Scott Collis

Radar data is complex and with the archive of NOAA NEXRAD data approaching the petabyte range, managing and extracting geophysical insight from this data is truly challenging. This presentation will discuss an approach using Python, specifically using resources available through project Jupyter, to solve pleasantly parallel problems. This is especially pertinent given the increasing prevalence of distributed computing resources. The example we will showcase is a decadal study of precipitation near the city of Portland, Maine where we radar data was used to understand the seasonal and diurnal cycle of precipitation. This single radar study uses 10’s of terabytes of radar data and 100’s of cores to reduce the radar data to domain mean parameters easily comparable with climate and regionally modeled climatologies.

Speaker Description: 

Scott Collis is radar metereologist at Argonne National Lab.

PDF icon SEA2016_Collis.pdf3.74 MB

Event Category: