Autoprocessing updates at the MX beamlines Jun Aishima Postdoc – Monash University Advanced Molecular Imaging Centre of Excellence and Australian Synchrotron

Feedback to users Processing software Analysis software Introduction to MX Feedback to users Processing software Analysis software

MX processing system outline Expt. type Indexing screening dataset Availability MX1 only MX2 only Both Images collected 1-2 images 100 images 1800+ images Info obtained Unit cell parameters Dataset quality Dataset Example output I23 78, 78, 78, 90, 90, 90

Software used to process data at the MX beamlines Custom pipeline software – Nathan Mudie - https://github.com/AustralianSynchrotron/mx-auto-dataset Custom web application software – Nathan Mudie - https://github.com/AustralianSynchrotron/mx-sample-manager xdsme – my upgrades to add handling inverted rotation axis, improve integration speed, etc.- (BSD license following from original) https://github.com/JunAishima/xdsme Pierre Legrand (Synchrotron Soleil) (BSD license) https://github.com/legrandp/xdsme XDS http://xds.mpimf-heidelberg.mpg.de/ Pointless, Aimless (CCP4) Sadabs and Xprep for CX data

What additional features beyond just processing the data? Merging datasets Dataset plots Retriggering (reprocessing) Results spreadsheet Whole goal is to help users evaluate their data, get the best out of it, and then keep track of the results

The Eiger 16M detector Rastering plots New feature s Put in rastering tab pic – show diffraction image, move to

Rastering plots

Eiger is a pixel array detector CCD PAD

How is the Eiger different from previous detectors? Data size: Eiger test crystal dataset, 180 degrees, 0.1 degrees/image – 9.5 GB ADSC test crystal dataset, 180 degrees, 1.0 degree/image – 800 MB Collection speed: Eiger test crystal dataset – 18 sec – continuous collection (3 microsecond readout) ADSC test crystal dataset – 7.2 minutes – ~1.4 sec readout per 1 second image Data rate: hundreds times faster! HDF5 files – master and multiple data files in NXMX format (full metadata necessary for processing) Typically 200 images/data file and experiments have 1800 images (9 files)

What is the real effect of the Eiger on data produced on the beamlines? 1 TB typical on a single visit

XDS also had support for the Eiger How much did we need to change the processing software to integrate the Eiger? The good news When we installed the Eiger 16M on MX2, xdsme already had been upgraded to support Eigers XDS also had support for the Eiger In terms of processing, not much code had to be rewritten Testing infrastructure also helped to rapidly develop and test new code while the beamlines were both still running CCD detectors => We get for free all of the features we have developed for the past few years with the new detector!! (merging, retriggering, web application, results spreadsheet) But the large amount of data -> slow!

Data Acquisition System Current Eiger processing system outline – MX2 Data Acquisition System Web app http://processing/retrigger/submit autodataset autodataset-cpu1,2,3 host:EPU autodataset host:EPU02 autodataset host:EPU03 autodataset host:ASCI05 autodataset host:ASCI06 Some processing (integration in XDS) is split between multiple nodes

Data Acquisition System In progress… Australian Synchrotron Computing Infrastructure (ASCI) processing cluster Data Acquisition System Web app http://processing/retrigger/submit autodataset autodataset host: node 1 autodataset host: node 2 autodataset host: node 3 autodataset host: node 4 autodataset host: node 5 autodataset host: node 6 autodataset host: node 7 autodataset host: node 8 autodataset host: node 9 autodataset host: node 10 autodataset host: node 11 autodataset host: node 12 autodataset host: node 13 autodataset host: node 14

What’s coming up in the next year? Collection strategies Compare with ADSC Sample services Collection Strategies Sample tracking - these mean better automatic processing, combining datasets that should be combined - scientists think about the science, not each individual dataset

Main dataset processing Retriggering Merging Results spreadsheet Summary Eiger Main dataset processing Retriggering Merging Results spreadsheet Rastering plots Plots Show diffraction image Move to image location In progress ASCI processing cluster Future Collection strategies Sample tracking LCP injector

Thanks to… The MX beamline – Tom Caradoc-Davies, David Aragao, Daniel Eriksson, Santosh Panjikar, Jason Price, Alan Riboldi-Tunnicliffe, Rachel Williamson Scientific Computing group – Andreas Moll, John Marcou, Robbie Clarken, Nathan Mudie, Ron Bosworth Australian Cancer Research Foundation for helping fund the Eiger 16M detector Users – for being patient, requesting features, providing feedback to our systems High Data Rate MX meeting – great advice and collaboration opportunities with other beamline scientists and Dectris to help deal with the Eiger detectors