ISIS: towards a 21st century facility computing environment (slowly) Robert McGreevy Science and Technology Facilities Council ISIS, Rutherford Appleton Laboratory, Harwell Science and Innovation Campus, Didcot, OX11 0QX, UK 3 rd November 2008

Source Instruments Sample environment Data treatment Scientific results

Source Instruments Sample environment Data treatment Scientific results

30 years ago …

ISIS Target Station 1

ISIS Target Station 2

Structures NIMROD Intermediate range order in liquids WISH High-resolution magnetic structure SANS2D Large molecule structure in multi- component systems Reflectometry INTER Air/ liquid/ solid interface interactions OFFSPEC Structures of membrane, protein and liquid interfaces POLREF Interface measurements in magnetic sensor devices Dynamics LET High-resolution measurement of material energy scales TS2 Instruments Phase 1

Structures NIMROD Intermediate range order in liquids LMX Large molecule single crystal diffraction ZOOM Focusing small angle scattering IMAT Imaging / diffraction Structure/Dynamics LARMOR SESANS, High resolution diffraction, MIEZE, MISANS TS2 Instruments Phase2

Understanding Data analysis Data visualisation Data treatment Data diagnostics Data acquisition Measurement The data chain Data curation Data (scientific) Data (raw) Data (publication) Data (metadata) Software Hardware

Understanding Data analysis Data visualisation Data treatment Data diagnostics Data acquisition Measurement Data curation Data (scientific) Data (raw) Data (publication) Data (metadata) Facility User Software Hardware The data chain

Ideas Knowledge and Understanding Exploitation (Economic Impact) GRID HPC ESS XFEL ILL ESRF IRUVX HPC DATA Time to Market

Proposal system From single instrument, single experiment to multi-instrument + support facilities + computing Damian Flannery, Tom Griffin, Hannah Guest

Experiment (controls, electronics, data formats) Freddie Akeroyd (NeXus) User Interface, DAE-III

Data catalogue, Data curation, Data Policy Damian Flannery (ICAT) Single sign-on

Data reduction/analysis – MANTID Nick Draper

Data analysis – software ‘instrument scientists’ EPSR, Single crystal inelastic, RMC …

Data analysis – modelling/simulation Simulation – source and instruments Tom Griffin (GRID-MP) SCARF (UK GRID) - CASTEP, Tobyfit McStas

Structures NIMROD Intermediate range order in liquids LMX Large molecule single crystal diffraction ZOOM Focusing small angle scattering IMAT Imaging / diffraction Structure/Dynamics LARMOR SESANS, High resolution diffraction, MIEZE, MISANS TS2 Instruments Phase2 Software infrastructure MANTID Bio-SANS Single crystal Imaging/enginering

Gateway Centres Imaging Solutions Centre, Hartree Centre

Collaboration European Data Infrastructure for Neutron and Photon Sources STFC, ESRF, ILL, DLS, PSI, DESY, ELETTRA, SOLEIL, ALBA, BESSY

What will Mantid do? Mantid –Manipulation and Analysis Toolkit for ISIS Data Aims –To provide a framework for Data Analysis that is not instrument or technique/dependent. –To support multiple target platforms (Windows, Linux). –The framework must be easily extensible by Instruments Scientists/Users. –The framework must be freely redistributable to visiting scientists. –The framework should provide low-level functionalities for Scripting, Visualization, Data transformation, Implementing Algorithms, Virtual Instrument Geometry.

Top Requirements Easily extensible. –Support all current and future analysis. –Support current and future file formats. –Provide a simple but powerful objects and services to support user created algorithm code. No user license costs. Supportable. Portable. –Operating System (Windows, Linux). –Computing Power (Laptop, Server).

Architectural Design - Overview Mantid Framework Command line & Scripting interface QTIPlot Graphing tool RAW data files NEXUS data files Future Data analysis GUI Instrument log files API Standard Algorithms User Defined Algorithms Matlab Interface DAE direct access

Development Process Iterative development – Why? –Adaptability - the ability to rapidly respond to changes in strategy, priorities, and plans –Value - continuous delivery of more useful functionality –Visibility - stakeholder collaboration and validation throughout the development life-cycle –Risk - the reduction in overall project risk as a result of #1-3 above

Software led by Scientists This is project is intended to support the needs of all of the beamlines. Scientific Steering Committee –To ensure input from each group of instruments To ensure what we build fits what you need. To ensure good communication between the Mantid project and the scientists. To allow each group to have equal input into the project. –Prioritise the list of tasks to be done over the next iterations. –Review the progress of each iteration.

Milestones –Architecture design completed October 2007 Completed on time –Prototype framework released: March 2008 Released to a limited audience in May 2008 Purposefully delayed to allow for additional functionality –Functional framework complete applications released March 2009 Timeline

What Mantid can do now Data Input/Export –Read Raw files and associated log files –Read Instrument definition files (Mantid XML format) –Nexus files (limited support – data only) Internal Data Structures –1D and 2D workspaces –Managed 2D workspaces for large volumes of data Instrument Definition –Sufficient definition to allow conversion of units –Loading instrument definition from XML files or RAW files

What Mantid can do now Algorithms –Simple binary mathematical operations +, -, *, / –Integration –Rebin –ConvertUnits TOF,, energy, d-Spacing, q, Q 2, Energy transfer (meV and 1/cm) –Grouping of detectors –Marking detectors as dead

User Interfaces Early Prototypes –QTIPlot-Mantid A graphical plotting tool which interfaces directly to Mantid for data interaction and analysis. –Python Command line access and scripting Proof of concept –Matlab Interface

QTIPLOT-Mantid Loading a Raw File

QTIPLOT-Mantid Contour and 3D Plots

QTIPLOT-Mantid Line graphs of individual detectors

Matlab Interface