The Role of DANSE at SNS Steve Miller Scientific Computing Group Leader January 22, 2007

O AK R IDGE N ATIONAL L ABORATORY U. S. D EPARTMENT OF E NERGY 2 DANSE Workshop 1/22-23/07 SNS Progress Update  NeXus – producing NeXus files and using NeXus routinely  Data Reduction – operational for first three instruments, initially via command line, now via IDL GUI tools  Commissioning Support – providing utilities on an as needed (regular) basis  Data Management – Adapted ICAT for accommodating individual users and groups, created a “check-in” utility as a proposal system proxy  Data Portal  Utilizes user and group permissions  Currently supports data browsing and metadata search  ISAW visualization tools adapted for use with the portal  Prototype version of the ‘Remote Display Conduit’ operational  Working with HFIR and LANSCE to make their data available via the portal  Simulation  McStas integrated with SNS portal and can demonstrate running jobs via MPI on the TeraGrid – works in progress

O AK R IDGE N ATIONAL L ABORATORY U. S. D EPARTMENT OF E NERGY 3 DANSE Workshop 1/22-23/07 Upcoming SNS Talks:  John Ankner – Software Commissioning and Development for the SNS Liquids Reflectometer  ReflPak integration with SNS  Pete Peterson – Status of the SNS/DANSE Reduction Effort  SNS Common Library  Jim Kohl – The SNS Portal: Web-Based Visualization, Analysis and Simulation  These will give more insight into SNS status, direction, and needs

O AK R IDGE N ATIONAL L ABORATORY U. S. D EPARTMENT OF E NERGY 4 DANSE Workshop 1/22-23/07

O AK R IDGE N ATIONAL L ABORATORY U. S. D EPARTMENT OF E NERGY 5 DANSE Workshop 1/22-23/07 SNS Computing Hardware in Place Analysis Computer Portal Server Data Server Data Storage Application Server Development Server Database Server Metadata Catalog Compute Server Repository and Web Server Software Repository Analysis Computer Dev Portal Instruments & Remote Instruments Support SW Development Support User Support Dev Data Server Data Storage Dev Database Server Metadata Catalog

O AK R IDGE N ATIONAL L ABORATORY U. S. D EPARTMENT OF E NERGY 6 DANSE Workshop 1/22-23/07 SNS Science Goal for FY07  Demonstrate new science on first three instruments!  Currently able to reduce data from these instruments  Using the following analysis software:  DAVE for the Backscattering Spectrometer  Reflectometers – planning to use ReflPak SNS)  Various legacy software starting to surface (mostly FORTRAN)  Needs are arising for new, advanced analysis software, or for extending existing software – this presents opportunities for DANSE

O AK R IDGE N ATIONAL L ABORATORY U. S. D EPARTMENT OF E NERGY 7 DANSE Workshop 1/22-23/07 Facility Schedule

O AK R IDGE N ATIONAL L ABORATORY U. S. D EPARTMENT OF E NERGY 8 DANSE Workshop 1/22-23/07 Instrument Commissioning Software Support Event data histogramming tools Calibration Tools Data exploration Data reduction Create NeXus files

O AK R IDGE N ATIONAL L ABORATORY U. S. D EPARTMENT OF E NERGY 9 DANSE Workshop 1/22-23/07 Instrument Support Pattern Emerging…  Transform Initial Data to NeXus data  Perform data reduction  Discover that which needs to be automated  Produce support tools – currently using IDL  Make community or legacy code available for analysis  Currently very little time for SNS to produce analysis software…

O AK R IDGE N ATIONAL L ABORATORY U. S. D EPARTMENT OF E NERGY 10 DANSE Workshop 1/22-23/07 Pathway for Transforming Raw Data Event Histogramming Detector to Pixel mapping Instrument Geometry Metadata extraction Create NeXus file  Catalog and Store  Reduce Data All subsystems functional to some degree

O AK R IDGE N ATIONAL L ABORATORY U. S. D EPARTMENT OF E NERGY 11 DANSE Workshop 1/22-23/07 Future Work for FY 07  NeXus  Instrument Definitions for next instruments  Complete automated “Live Cataloging”  Serve SNS, HFIR, and LANSCE (NPDF) initially  Data Reduction  Support for next instruments  Data Management  Long term storage utilizing HPSS  ICAT-3 development with ISIS  Testing  Live Cataloging  Data browse and search  “check-in” utility

O AK R IDGE N ATIONAL L ABORATORY U. S. D EPARTMENT OF E NERGY 12 DANSE Workshop 1/22-23/07 Future Work Continued  Portal Development  Continue Application Manager development  Continue “Weeerd-X” integration and test  Continue TeraGrid integration for running McStas  Add more data visualization and inspection tools  Incorporate data reduction within the portal  Support more data formats for visualization

O AK R IDGE N ATIONAL L ABORATORY U. S. D EPARTMENT OF E NERGY 13 DANSE Workshop 1/22-23/07  The Scientific Data Management Center for Enabling Technologies (SDM) – Arie Shoshani (~$3.3M/yr)  Kepler for data workflows, “Dashboard”, Data Mining, SRM  Center for Technology for Advanced Scientific Component Software (TASCS) – David Bernholdt (~$3.0M/yr)  CCA  Center for Enabling Distributed Petascale Science (CEDPS) – Ian Foster (~2.4M/yr)  Pyglobus, web services, application hosting service – K. Jackson  Interactions with SNS actively being defined

O AK R IDGE N ATIONAL L ABORATORY U. S. D EPARTMENT OF E NERGY 14 DANSE Workshop 1/22-23/07 Grid Computing  Continue to work closely with Neutron Science TeraGrid Gateway (NSTG) located at ORNL  Running distributed version of McStas  Utilizing MPI with event mode data being produced  Enabling McStas via SNS portal and will utilize a community account  Open Science Grid  Discussions with Ruth Pordes regarding collaborations and SBIR topics  Future – to put OSG software stack on NSTG  Providing grid computing seen as key to supporting a large computing user base – we’re still learning how to do this

O AK R IDGE N ATIONAL L ABORATORY U. S. D EPARTMENT OF E NERGY 15 DANSE Workshop 1/22-23/07 McStas Simulation Work  Meeting last week with the McStas development team at SNS  Attending: Peter Willendrup, Emmanuel Farhi, Kim Lefmann, Vickie Lynch, Meili Chen, Pete Peterson, Mark Hagen, Jiao Lin  Topics discussed:  NeXus Support  Event Mode data and file merging  Portal access to McStas  Sample kernels  Parallel and Distributed computing

O AK R IDGE N ATIONAL L ABORATORY U. S. D EPARTMENT OF E NERGY 16 DANSE Workshop 1/22-23/07 Summarizing the Role of DANSE at SNS  Providing analysis code to support instruments as they begin commissioning – getting “first science” from instruments is crucial!  Code re-use and sharing – taking first steps with the SNS common library, NeXus compatibility, and data reduction  Integration with SNS portal – how to add DANSE apps to this portal  Compatibility with grid computing – TeraGrid and OSG in particular  Compatibility with SciDAC efforts which SNS will be involved