Nick Barnes at NCDC, climatecode.org1 Better Science Through Software Copyright Climate Code Foundation, license CC-BY

Nick Barnes at NCDC, climatecode.org2 What is the CCF? A non-profit founded in 2010, based in the UK; Continuing projects started in 2008; A few software consultants, currently unpaid part-time; Advisory committee of a dozen experts; A growing network of climate scientists and others; Several projects and publications; and big plans.

Nick Barnes at NCDC, climatecode.org3 What is the problem? Scientists have to write code, but: They aren’t well-trained; They aren’t properly rewarded; There is no incentive to publish it. The public need to know about climate science, but: The science isn’t accessible; The practices aren’t always transparent; They are lied to about ‘tricks’ and secrecy.

Nick Barnes at NCDC, climatecode.org4 Foundation goals "to promote public understanding of climate science, by increasing the visibility and clarity of the software used in climate science, and by encouraging climate scientists to do the same; by encouraging good software development and management practices among climate scientists; by encouraging the publication of climate science software as open source.”

Nick Barnes at NCDC, climatecode.org5 Advisory Committee Climate Scientists Kate Willett James Annan V. Balaji Stefan Brönnimann John Christy Reto Ruedy Peter Thorne Other Scientists Steve Easterbrook Peter Murray-Rust Cameron Neylon Andrew Woolf Non-scientists Paul Edwards Glyn Moody

Nick Barnes at NCDC, climatecode.org6 Clear Climate Code Over-riding goal is clarity: code which interested members of the public can download, run, read and understand. Open-source, of course. First target NASA GISTEMP: ccc-gistemp.googlecode.com 12 KLOC of Fortran (etc). became 3678 lines of Python (including 1500 of docstrings) fixed minor bugs. fosters new science: one paper out now, more draft.

Nick Barnes at NCDC, climatecode.org7 What is clarity? def step1(record_source): """An iterator for step 1. Produces a stream of `giss_data.Series` instances. :Param record_source: An iterable source of `giss_data.Series` instances (which it will assume are station records). """ records = comb_records(record_source) helena_adjusted = adjust_helena(records) combined_pieces = comb_pieces(helena_adjusted) without_strange = drop_strange(combined_pieces) for record in alter_discont(without_strange): yield record

Nick Barnes at NCDC, climatecode.org8 Why clarity? Original motivation was to answer critics: Not the real code; Can’t be run; Contains “obvious bugs”; “divinci code written by the shortbus crew.” But also a key message of software engineering: Your target audience is people, not compilers Those people are often yourselves.

Nick Barnes at NCDC, climatecode.org9 Clarity yields new science It enables “computational thinking” (Wing, NSF): Clear code raises new questions… … and lets you answer them. Airport-only trends? Effect of US data? Effect of restricting to long-record stations? Use of land data for ocean cells? Adding more data scraped from met sites?

Nick Barnes at NCDC, climatecode.org10 Homogenization project GHCN 3.0 dataset (Menne & Williams 2009); Re-implemented by Dan Rothenberg (Cornell, now MIT); Working with Menne and Williams at NCDC; Algorithm improved, bugs fixed; Revised dataset – GHCN-M – see M&W tech note; Funded by Google (Summer of Code 2011); Praised by NCDC: letter from Scott Hausman. To be presented at AMS New Orleans, Many extensions possible: Peter Thorne has a dream….

Nick Barnes at NCDC, climatecode.org11 Common Climate Project Web framework for visualizing climate datasets; Late Holocene paleoclimatology: Emile-Geay (USC), Smerdon & Anchukaitis (LDEO); Open-source, open datasets; Prototype online at commonclimate.net; Implemented by Hannah Aizenman (grad student at CUNY); Funded by Google (Summer of Code 2011). To be presented at AMS New Orleans,

Nick Barnes at NCDC, climatecode.org12 Google Summer of Code Google pays students to work on open-source projects; Annual since months work, $5000 (if you pass); Any student, anywhere, can apply by proposing a project; Mentoring organisations select projects, supervise, and grade. 2011: 175 organisations, 1115 students; Mostly mainstream open-source software: GNU, *BSD, Minix, Linux distros, NTP, KDE; Apache, Mono, Python, GCC, OpenMRS; Firefox, GIMP, haskell, Inkscape, AbiWord, LibreOffice.

Nick Barnes at NCDC, climatecode.org13 Google Summer of Code About 20 science-oriented projects: Informatics: Neuro-, Geno-, Bio-, Evo-, ML. CernVM, Scilab, SymPy, R, VTK; SETI, Astrometry.net, NetGen; and CCF. CCF as an “umbrella” org: Our own projects (e.g. Filipe Fernandes) but also: External projects (e.g. Hannah Aizenman), with external mentors; Or hybrids (e.g. Dan Rothenberg).

Nick Barnes at NCDC, climatecode.org14 Google Summer of Code Timetable 2011: Jan 24:program announced; Feb 28–Mar 11:brief window for orgs to apply; Mar 18:orgs announced; Mar 28–Apr 8: brief window for students to apply; Apr 22:deadline for orgs to rate proposals; Apr 25:projects announced; May 23–Aug 22:Coding! Jul 15:mid-term evaluations; Aug 29:final results; Oct 22/23:mentor summit may be different.

Nick Barnes at NCDC, climatecode.org15 Open Science Accelerating trend towards more openness in science. Redefining publication: Open Access; Open Data; Open Knowledge; Open Notebooks; Data-driven intelligence; Workshops, conferences, summits; There’s a war on: PRISM, RWA; Royal Society policy study: Science as a Public Enterprise; But no coherent message about open software in science. Michael Nielsen: Reinventing Discovery

Science Code Manifesto Code:All source code written specifically to process data for a published paper must be available to the reviewers and readers of the paper. Copyright: The copyright ownership and license of any released source code must be clearly stated. Citation:Researchers who use or adapt science source code in their research must credit the code's creators in resulting publications. Credit:Software contributions must be included in systems of scientific assessment, credit, and recognition. Curation:Source code must remain available, linked to related materials, for the useful lifetime of the publication. Nick Barnes at NCDC, sciencecodemanifesto.org16

Nick Barnes at NCDC, climatecode.org17 Future Plans Changing policies: Transparency; Rewards for all research products. Training scientists: Basic techniques (testing, version control, agile, etc); Code publication and reuse. Providing resources: White papers, blog posts; Directories. Building networks, partnering with institutions; Leading by example: ccc-gistemp; ccf-homogenization; etc….

Nick Barnes at NCDC, climatecode.org18 Questions?

Nick Barnes at NCDC, climatecode.org19 Funding I say "non-profit". Approximately “non-revenue". All accounts open. Total revenue to date£ (+ GSoC students). Total costs to date£ (as of ). All work unpaid (not counting GSoC students). Personal lost income to date probably £30-40K. Funding model seeks £150K-£500K annually from corporate or NGO sponsorship (plus some project money from academic collaborations). Too much? Not enough? Depends who you ask. Open to suggestions!