Introduction to DANSE Brent Fultz Prof. Materials Science and Applied Physics California Institute of Technology Distributed Data Analysis Architecture for Neutron Scattering Experiments Technical Issues, Scientific Organization From a Concept to a National Project

$ 1,411,000,000 hardware 10,000,000 software

ARCS

Analysis Software for Neutron Scattering Enables science Helps new users get up to speed Better science with better software optimize beamtime usage experimental procedures tuned on-the-fly sample knowledge can constrain data interpretations Some experiments are impossible with today’s software New opportunities to connect to materials theory

Logo for ARCS Spectrometer Headstock Neck Frets Pickups Volume/Tone Solid Body Tailpiece (Whammy Optional) Logo for DANSE??

Software with Power and Beauty $?

Software with Coherence DANSE architecture is General but All developers must know it Hence this Workshop

Data Analysis as a Web Service Data analysis is a service controlled by the user User’s laptop issues commands and receives results Computation is arranged by the web server

Present Example from inelastic scattering: Born - von Kármán Lattice Dynamics Simplicity: Complexity: 1.Undergrowth of indices for tensor quantities: 2.Crystal structure: lattice, l, basis,  allowed elements depend on symmetry

The Bigger Concept Underneath Components Pre-compiled Python objects called and re-arranged by the Python Interpreter Data Streams Standard communication protocol between components (standard streams can connect components located anywhere)

DANSE Project: Develop Tools for Programmers Component Templates Standard Data Streams

Levels of User Expertise For using existing scripts, entry barrier nearly zero. For altering existing Python scripts, entry barrier is very low. For writing new Python code, entry barrier is modest. Performance may be comparable to IDL or Matlab. Transition to high performance compiled code ARCS: Writing Python bindings for C++ DANSE: Component templates for C++, FORTRAN, Java? Focus of this Workshop

Importance of remote control of neutron instruments Remote collaboratory capability Remote data analysis capability No strong support for remote control -- too risky (sample temperature and saving data okay) Strong support for remote data analysis capability Overwhelming support for standardization of data analysis (across different instruments and different facilities) NSSA Neutron Software Poll

Making a Software Project Plan Define and justify scientific needs Define and justify the software architecture Define the scope of the project Accurately estimate the effort Proposal -- what, why, who, where, when, how?

A DANSE Project Would Have 2 Parts Build Tools (hardware and software) for Developers Extend Software Beyond ARCS Diffraction Engineering diffraction Reflectometry SANS Inelastic scattering (ARCS+chemistry)

Money Software tools development Hardware for multiple users Science-based subprojects Four-Year Budget (salaries mostly) 10 M$ 4-5 subprojects 10 M$ Central Resources 7 FTE + hardware

Software Development

ARCS Software Project

Recent Developments DANSE is “the SNS standard” architecture NIST reflectometry sub-proposal TeraGrid Extension to ORNL (DANSE is core) SNS software developers at HFIR(?) SNS to write specifications for data analysis software censored NSF intermediate-scale instrumentation program (December 2003) NSF ITR (Spring 2004)

Reasons for Optimism about DANSE Upsurge in interest from DOE and beyond Alternative architectures are not full solutions Technically possible within the scale discussed No better group to do it

DANSE Software Workshop Wednesday Sept. 3, 2003 IntroductionsLunch Python and OOP DinnerEvening tutorials Thursday Sept. 4, 2003 The Python/C API, NeXusLunch Subgroup Talks/FundingEvening tutorials Friday Sept. 5, 2003 Architectural FrameworkLunch Programmers' Tools DinnerEvening tutorials Saturday Sept. 6, 2003 Software design for DANSELunch"Edward Weston: A Legacy" Sunday Monday Sept. 8, 2003 Python Extensions in C++ Lunch FORTRAN, Operating systems. DANSE Project integration. Dinner

Welcome to California

End of Presentation Michael is Next