Authors: Mike Daniels, William Cooper, Janine Aquino, Teresa Campos, William Brown (All from NCAR/EOL)

New tools now enable new ways of achieving reproducibility in data analysis, especially as scientific workflows are more integrated with papers and reports. Reproducibility is growing in visibility because skeptics among colleagues, funding agencies, politicians, and professional societies can damage professional reputations and can be hard to combat without good records. Reproducibility is also beneficial to the original authors when it becomes desirable to extend the work and memories have grown stale.

A reproducible project should at least have these characteristics:

1. Data sources should be available in public repositories, with assigned DOI numbers, and there should be sufficient provenance to trace their origins and those responsible for their collection and curation.

2. Analysis software should similarly be preserved and where possible assigned identifiers that ensure their preservation and public access.

3. Workflow and/or provenance descriptions should accompany the work. It is not sufficient to preserve the software, which depends on a reproducer being able to understand the program and run it in a suitable environment. A workflow description can include the requisite conditions (software packages, version numbers of commercial packages, the computing environment) and can also describe parts of the investigation pursued and found unproductive, how choices were made and figures constructed, etc. These descriptions need to be archived with the analysis programs so that others might duplicate or extend the work.