2. Diamond is all about data…

3. … some people love huge amounts of data…
… new users can be less keen…

4. … but everyone really wants this.
Science!
Some variable / Unit
Some metric / Unit

5. OPERATES A BEAMLINE: GDA DOESN’T: DAWN
Client server
Technology,
Communication with
EPICS and hardware,
Scan mechanism,
Requires Beamline.
Java Maths,
File Loading,
Visualisation,
Analysis
Tools,
Jython
and Python.
Calibration,
Processing,
Data Conversion,
You Can Take It Home.
Acquisition
Analysis

6. Powder Diffraction / Small Angle Scattering

7. Calibration

8. Masking, Correction, Reduction, Fitting…

9. [61, 76, 1679, 1475]

10. Requirements Integrate tightly with GDA
Near infinite number of processing steps
Process series images or XY data – linear chain
NeXus (n-dimensional datasets – maintain rank)
Data provenance
Simple.

11. (Part of Pipeline Runner)
Architecture
41k peak fits
Highly Modular (Plugins + Services)
Pipeline Runner, Processing Steps, File Writing
All independent and swappable
Data Iterator
(Part of Pipeline Runner)
Processing
Step 1
Processing
Step 2
Processing
Step 3
File Writer
Pipeline Runner

12. (Part of Pipeline Runner)
Runner – Iterates through frames
Series, Parallel, Passarelle Workflow, Dynamic (SWMR)
Processing steps – Maths + Parameters separate
Store id and parameters – Provenance
Can be written in DAWN!
File Writing – NeXus, when in UI, writes to Screen
41k peak fits
Data Iterator
(Part of Pipeline Runner)
Processing
Step 1
Processing
Step 2
Processing
Step 3
File Writer
Pipeline Runner

15. GDA/Cluster Plot tool for quick processing live (not saved)
Headless runner for auto-processing

16. HDF5-SWMR Read HDF5 files while they are being written!
Simultaneously better and looser GDA integration
Only need to write SWMR Pipeline Runner
Processing steps don’t need to change
Raw
Processed

17. Generic Live Processing
DAWN processing runs on cluster during data collection
“We have our best people working on it...”

18. Run Processing was pressed > 5000 times in Diamond the last year

19. Feedback Button
“Treat every bug report like a love letter.”

20. For more information: scientificsoftware@diamond.ac.uk
Acknowledgements
Beamlines (and scientists therein): I15, I12, I22, I11-1, I15-1, I18, I07
Data Analysis Diamond
Data Acquisition Diamond
For more information:
J. Synchrotron Rad. (2015). 22,