DMSC services Stig Skelboe

Slides:



Advertisements
Similar presentations
University of Chicago Department of Energy The Parallel and Grid I/O Perspective MPI, MPI-IO, NetCDF, and HDF5 are in common use Multi TB datasets also.
Advertisements

ESS Timing System Plans and Requirements Timo Korhonen Chief Engineer, Integrated Control System Division May 19, 2014.
Yat Li Department of Chemistry & Biochemistry University of California, Santa Cruz CHEM 146C_Experiment #3 Identification of Crystal Structures by Powder.
Data Management and Software Centre Mark Hagen Head of DMSC Institutional Export Conference, Tallinn,
The Data Management Requirements at SNS Shelly Ren & Steve Miller Scientific Computing Group, SNS-ORNL December 11, 2006.
Experimental Facilities Division ANL-ORNL SNS Experimental Data Standards (Status) Richard Riedel SNS Data Acquisition Group Leader.
1 Basics of Digital Imaging Digital Image Capture and Display Kevin L. Lorick, Ph.D. FDA, CDRH, OIVD, DIHD.
Hungarian Contribution to the ESS - NMX Workshop 17 Marc 2015 Péter HARMAT Control electronics and DAQ for neutron spectrometers
Dott. Dario Tresoldi CNR IPCF ME
Data Acquisition at the NSLS II Leo Dalesio, (NSLS II control group) Oct 22, 2014 (not 2010)
A. Homs, BLISS Day Out – 15 Jan 2007 CCD detectors: spying with the Espia D. Fernandez A. Homs M. Perez C. Guilloud M. Papillon V. Rey V. A. Sole.
2/10/2000 CHEP2000 Padova Italy The BaBar Online Databases George Zioulas SLAC For the BaBar Computing Group.
EPICS Archiving Appliance Test at ESS
Experimental Facilities DivisionOak Ridge SNS INSTRUMENTS OVERVIEW R. K. Crawford Instrument Systems Senior Team Leader September 10, 2004 HYSPEC IDT Meeting.
Nick Draper Teswww.mantidproject.orgwww.mantidproject.org Instrument Independent Reduction and Analysis at ISIS and SNS.
AMICA Antarctic Multiband Infrared CAmera Favio Bortoletto INAF – Osservatorio Astronomico di Padova on behalf of the AMICA collaboration Telescope and.
ISIS: towards a 21st century facility computing environment (slowly) Robert McGreevy Science and Technology Facilities Council ISIS, Rutherford Appleton.
D0 SAM – status and needs Plagarized from: D0 Experiment SAM Project Fermilab Computing Division.
Igor Gaponenko ( On behalf of LCLS / PCDS ).  An integral part of the LCLS Computing System  Provides:  Mid-term (1 year) storage for experimental.
Mantid Scientific Steering Committee Nick Draper 10/11/2010.
Software breakout session: FPA related OPUS issues Michael J. Nasse, Carol Hirschmugl.
Instrumentation of the SAM-Grid Gabriele Garzoglio CSC 426 Research Proposal.
Detectors for Light Sources Contribution to the eXtreme Data Workshop of Nicola Tartoni Diamond Light Source.
Aquatic Spectrometer & Turbidity Meter ECE 4007 L1, Group 8 Paul Johnson Daniel Lundy John Reese Asad Hashim.
Fast Crash Recovery in RAMCloud. Motivation The role of DRAM has been increasing – Facebook used 150TB of DRAM For 200TB of disk storage However, there.
Real-time Acquisition and Processing of Data from the GMRT Pulsar Back- ends Ramchandra M. Dabade (VNIT, Nagpur) Guided By, Yashwant Gupta.
Introduction CMS database workshop 23 rd to 25 th of February 2004 Frank Glege.
Databases in CMS Conditions DB workshop 8 th /9 th December 2003 Frank Glege.
ESFRI & e-Infrastructure Collaborations, EGEE’09 Krzysztof Wrona September 21 st, 2009 European XFEL.
Mantid Current Development and Future Plans Nicholas Draper ICNS 2013.
X-ray powder diffractometer X-ray powder diffractometer.
Jefferson Lab IT in the 12GeV Era - Review Accelerator Controls Matt Bickley 05/20/11.
Predrag Buncic Future IT challenges for ALICE Technical Workshop November 6, 2015.
Past and Future Insights from Neutron Scattering Collin Broholm * Johns Hopkins University and NIST Center for Neutron Research  Virtues and Limitations.
ESS Timing System Prototype 2012 Miha Reščič, ICS
Data Management and Software Centre Mark Hagen Head of DMSC IKON6, February 27 th 2014.
Virtual experiments for teaching and training Kim Lefmann MCNSI, ILL, 26/5-05.
Jian Gui WANG Bragg Institute Meeting Java Algorithm Library Dec Java DRA Algorithm Library For Opal Neutron Scattering Data Analysis Team Jian.
Presentations for this session Anton – Detector concept and rate estimations Scott – Readout Jonathan – Data Management.
Hands-On Microsoft Windows Server 2008 Chapter 7 Configuring and Managing Data Storage.
Data Management and Software Centre Mark Hagen Head of DMSC IKON7, September 15th 2014.
ESS TDR Internal Review Chapter and 12. October 2012 Oliver Kirstein.
Stéphanie Hustache-Ottini S LEIL SYNCHROTRON iWoRID 2011 – 140 – July 5 th Towards ps and fs diffraction with the XPAD detector S. Hustache-Ottini 1, J.-C.
Nick Draper Tessella The Mantid Project: Notes from an international software collaboration.
ORNL is managed by UT-Battelle for the US Department of Energy Status Report: Data Acquisition and Instrument Controls for the Spallation Neutron Source.
Mantid Scientific Steering Committee Nick Draper 18/06/2010.
SWCam Instrument software DraftV2 10June2014
Tom Van Steenkiste Supervisor: Predrag Buncic
CHARACTERIZATION OF THE STRUCTURE OF SOLIDS
SuperB and its computing requirements
reduction data treatment for ARCS
LHC experiments Requirements and Concepts ALICE
Memory Parts of a computer
ProtoDUNE SP DAQ assumptions, interfaces & constraints
WP5 Report: Virtual Laboratories
Toward a costing model What next? Technology decision n Schedule
A First Look J. Pilcher 12-Mar-2004
VNX Storage Report Project: Sample VNX Report Project ID:
Cover page.
BlueGene/L Supercomputer
X-ray powder diffractometer.
X-ray powder diffractometer.
Tomography at Advanced Photon Source
ICS update 7th Experiment Control Workshop
Afonso Mukai Scientific Software Engineer
Diamond is all about data…
Screaming Udder: data streaming at ISIS
Computed Tomography (C.T)
Instrument control – lessons learned from V20 and from personal experience at engineering beamlines General NICOS deployed at V20: well perceived so far.
Inelastic Neutron Scattering Correction
Presentation transcript:

DMSC services Stig Skelboe Project Leader ESS Data Management and Software Center

DMSC services - measurements

Experiment control software Control of instrument Configuration of instrument Sequence of setting Control of sample environment Sequencing of instrument and sample environment settings Logging of settings and detectors Presentation of experiment configuration Authentification of user Remote control of experiments

Data acquisition, event mode High data rates Accelerator/ neutron pulses numbered and characterized Detector electronics, depends on detector type, includes calibration logic Hardware buffers Global instrument clock Separate data network (Gigabit, Infiniband...) Separate data acquisition computers

Neutron detectors 105 – 106 pixels Deadtime ≈ 10 μs 103 – 106 neutrons per pulse (2 ms) 1 neutron event gives 8 Bytes 8 kB – 8 MB in 2 ms 112 kB/s – 112 MB/s

Data rates and volumes Peak per instrument: 100 MB/sec -> 360 GB/h -> 1 TB/measurement Fiber: 10 Gb/sec ~ 1 GB/sec Transfer of 1 TB ~ 1000 sec ~ 17 min Max data per year: 1 PB -> 3 TB/day -> 140 GB/day/instrument

Imaging and tomography Resolution: 4k x 4k -> 16M pixels -> 32 MB 360 angle views: 11 GB 10 sec per view -> 11 GB per hour 3D reconstruction: 4k x 4k x 4k -> 128 GB

Storage system Large data volumes, 1 PB/year Standard format, e.g. Nexus Measurement data and corresponding meta data Controlled access from ”everywhere” Provide all data for analysis steps Seemless interface with analysis software Safe storage, archiving Database with meta data and links to data files ICAT used by ISIS, Diamond, SNS, ANSTO

Bragg’s law nλ=2d sin θ

Neutron Powder Diffractometer at LANSCE

Powder diffraction pattern from spallation source nλ=2d sin θ

Data analysis General tools used with most instruments Instrument specific analysis software Instruments have yet to be determined ESS challenges Exploit event based data Exploit repetition rate multiplication Current development: Mantid ISIS, SNS, HFIR, ILL recently High-performance computing on neutron data Provides a set of common services, algorithms and data objects Instrument or technique independent

Repetition rate multiplication Inelastic scattering: λin ≠λout 100m neutron guide 6-7 choppers 8-10 narrow band pulses for each 2.8 ms pulse λ = 3.9 - 6.2 Å, Δλ = 0.25 Å Inelastic scattering requires narrow band pulses

DMSC services - simulations

Data analysis and simulation Simulation software, e.g. McSTAS To be integrated in analysis framework Used for Training Experiment planning Instrument design Analysis method development Data correction, e.g. for multiple reflections

Theoretical materials modeling Interpretation of measurements using: Classical molecular dynamics Ab-initio molecular dynamics Quantum mechanical methods Density functional theory based Hartree-Fock methods All very compute intensive methods

Thank you ! Stig Skelboe Project Leader ESS Data Management and Software Center