Mantid Stakeholder Review Nick Draper 01/11/2007

Agenda Why do we need Mantid? What will Mantid do? Progress so far Plan for the future Questions

Why do we need Mantid? Previously different tools were used on the different beamlines. –PC Collette, Ariel, SXD2000, OpenGenie, LibISIS. –Confusing for users. –Replicated effort across the tools. –Difficult to support. Mantid –To standardise data reduction tools between instruments. –To improve the support and documentation for the tools. –To handle the expansion of data volume. –While having the flexibility to be extended by scientists as required.

What will Mantid do? Mantid –Manipulation and Analysis Toolkit for ISIS Data Aims –To provide a framework for Data Analysis that is not instrument or technique/dependent. –To support multiple target platforms (Windows, Linux). –The framework must be easily extensible by Instruments Scientists/Users. –The framework must be freely redistributable to visiting scientists. –The framework should provide low-level functionalities for Scripting, Visualization, Data transformation, Implementing Algorithms, Virtual Instrument Geometry.

What will Mantid do? Scope –Data reduction and analysis, not Instrument Control. –Creation of an extensible framework. –Creation of a command line and scripting interface. –Creation of a Visualization tool. –Creation of a repeatable automated test environment. –Provision of user and support documentation. –Providing user support for usage and extension of the framework. –Creation of specific beamline interfaces as required. (Later)

Milestones –Architecture design completed October 2007 –Prototype framework released: March 2008 –Functional framework complete applications released March 2009 Timeline

Meet the team Project Manager –Nick Draper, Tessella Developers –Russell Taylor, Tessella –Laurent Chapon –Matt Clarke –Dickon Champion –Freddie Akeroyd –Anders Markvarsden –Stuart Ansell Sponsors –Paolo Radaelli –Toby Perring

Progress so far Requirements Gathering and Analysis –Dunstan Thomas – Large meetings –Myself – one to one discussions Architecture Prototypes –GAUDI –LOQ – PC Collette Architectural Design Development of the core framework functionality

Top Requirements Easily extensible. –Support all current and future analysis. –Support current and future file formats. –Provide a simple but powerful objects and services to support user created algorithm code. No user license costs. Supportable. Portable. –Operating System (Windows, Linux). –Computing Power (Laptop, Server).

Architecture Prototypes - GAUDI A data analysis framework for event based data. Created at CERN for LHCb experiment. Built to be generic enough to be used by other institutions. Good points –Careful design based on good ideas Separating algorithms from data. Encapsualting “user code” in specific places. Ensuring the framework is extensible.

Architecture Prototypes - GAUDI Bad Points –Control of the analysis was complex and rigid. –File input and output was complex and confusing. –No memory management was included. All down to the design concept –Event Based data 1,000s of files in a single run Small individual data files Repeated analysis for each file, and generally the same across runs.

Architecture Prototypes - LOQ Collette software –currently undergoing reimplementation on PC/Linux. Good points –Neutron specific framework. –Aimed at handling similar file sizes to what we need. Bad Points –A bit too specific for the LOQ instrumentation. –Being developed by a single developer. –Not complete.

Architecture Prototypes - Conclusion Options 1.Convert Gaudi To handle Histogram data and a more flexible control system. 2.Extend LOQ To better support the various instruments across ISIS. 3.Take the best ideas and approaches from both. Build a bespoke framework to support Neutron data and practices. We chose option 3.

Architectural Design - Overview Mantid Framework Command line & Scripting interface Visualization tool RAW data files NEXUS data files Future Data analysis GUI Instrument log files API Standard Algorithms User Defined Algorithms

Development Process Iterative development – Why? –Adaptability - the ability to rapidly respond to changes in strategy, priorities, and plans –Value - continuous delivery of more useful functionality –Visibility - stakeholder collaboration and validation throughout the development life-cycle –Risk - the reduction in overall project risk as a result of #1-3 above

Development Process Iterative development. –Top level aim defined. –Tasks identified and allocated to fulfill the aim. –4 week of development and testing. –Automated tests built alongside all functionality. –Review, and plan the next iteration.

Development Process Iterations so far –1 Aim: Create enough of the framework structure to support the same test as performed in the Pilots. –Load a raw file -> integrate the counts for all time bins -> output Result: Success but with a few compromises. –2 Aim: –Add sufficient geometry support to allow conversion of tof bins to wavelength. –To add a python interface to the framework –To include the ability to add new algorithms through user created libraries, loaded at run time. This iteration completes at the end of this week.

Software led by Scientists This is project is intended to support the needs of all of the beamlines. We need input from each group of instruments –To ensure what we build fits what you need. –To ensure good communication between the Mantid project and the scientists. –To allow each group to have equal input into the project. To do this we will set up a scientific steering committee.

Scientific Steering Committee Who? –The Mantid project manager and one representative from each group of instruments. What will they do? –Raise new enhancement requests and report issues. –Prioritise the list of tasks to be done over the next iterations. –Review the progress of each iteration.

Scientific Steering Committee How often? –Once every 4 weeks just before an iteration starts. When will this start? –We plan to start at the end of January. Where do I sign up? –Voluteers (with the support of their group) should contact me (Nick Draper).

Further Information Project Web Page – Project Introduction Document – troduction%20Document.dochttp://svn.mantidproject.org/mantid/trunk/Documents/Requirements/Project%20In troduction%20Document.doc User Requirements Document – Architectural Design Document – ign%20Document.dochttp://svn.mantidproject.org/mantid/trunk/Documents/Design/Architecture%20Des ign%20Document.doc