Reproducibility in Large-Scale Computing: A Dissident View from the Trenches Doug James Texas Advanced Computing Center An XSEDE14 Workshop July 14, 2014 Ver 1.0 as of 2 Jul 2014

Where Do I Stand?* “No, thank you...” Run your code Duplicate your environment Bitwise reproducibility “Yes, please...” Implement your algorithm Portable results Understand the differences reproducibility > re-enactment *your mileage may vary

What Do I Want to See Next?* Promote the value of fundamentals a la "Ten Simple Rules" – Especially automation, change management, traceability – No magic bullets imposed from above; let the marketplace decide Expect clarity – Readable, well-documented source code – Datasets: provenance, formats, meaning of fields Encourage discussion and reflection – e.g. NSF Grant Proposal Guide Ch II.C.2.j (Data Management Plan): "Plans for data management...may include..." [How 'bout a sixth item that says something like...] 6. mechanisms and processes that enhance opportunities for reproducibility." *your mileage may vary