1. Automated collection and processing of macromolecular diffraction data with DNA Project started in 2001 following an ESRF user meeting Currently involves scientists at Diamond, ESRF, EMBL Grenoble, EMBL Hamburg, SRS and SOLEIL. Funded (indirectly) by AUTOSTRUCT, BIOXHIT, eHTPX, CCP4 Aim is to automate data collection and processing at synchrotron beamlines. Data Not Analysed Definately Not Attractive Does Not at All DogPatch Neighbourhood Association Distributed Network AttackDeadly Nocturnal Assassins Deer Natural Aroma Dark Native ApostleDrink No Alcohol Disco Night Action Dawn of the New AgeDesire and Needs Assessment Drugz N Alchohol Douglas Noel Adams (Thanks to acronym.com)

2. Participants DNA developers –Karen Ackroyd (a) –Alun Ashton (b) –Gleb Bourenkov (c) –Sandor Brockhauser (d) –Marie-Francoise Incardona (f) –Steve Kinder (a) –Pierre Legrand (e) –Karl Levik (b) –Romeu Pieritz (f) –Sasha Popov –Harry Powell (g) –Darren Spruce (f) –Olof Svensson (f) –Graeme Winter (a) DNA Exec. Committee: –Gérard Bricogne (h) –Andrew Leslie (g) –Sean McSweeney (f) –Colin Nave (a) –Alexander Popov (c) –Raimond Ravelli (d) –Andrew Thompson (e) Collaborating sites –(a) CLRC Daresbury, UK –(b) Diamond light source, UK –(c) EMBL Hamburg, Germany –(d) EMBL Grenoble, France –(e) Synchrotron Soleil, France –(f) ESRF, France –(g) MRC LMB, Cambridge, UK –(h) Global Phasing, Cambridge, UK –NSLS Brookhaven Funding from: EU (AUTOSTRUCT, BIOXHIT), BBSRC (EHTPX), CCP4

3. Current Structure Exec - policy decisions, conflict resolution. One representative from each institution that hosts a developer. Project Coordinator - (Alun Ashton). Arranges VCs, ensures actions are carried out. Developers - Do the work Full DNA meetings twice a year. Additional developers meetings two- three times a year. Video conferencing ~ every two weeks. Bugzilla for bug tracking. CVS at D/L (moving to Diamond)

4. LABELIT POINTLESS Current structure of DNA 1.0 ISPyB

5. Characterising a single sample: collect reference images Assumes that the sample is already centred in the beam.

6. Project Status DNA 1.0 released December 2004. Basically it didn’t work. DNA 1.1 planned release April 2007, although something very close to 1.1 is already installed on 7 MX beamlines at ESRF. Improvements: Robustness improved significantly Datasets integrated and scaled “on the fly” Allows a “sample ranking” mode for multiple crystals of the same type Significant improvement in autoindexing success Includes an interface to the EMBL/ESRF mini kappa goniostat. Now at a critical phase.. People are starting to use it !

7. Future plans DNA 2.0 involves a complete rewrite of the code. Project manager Olof Svensson. Currently a “spike” development is underway to establish which tools to use. A “scientific case” is being prepared which will define the scientific (user) objectives. Objectives: More sophisticated data collection strategies (multiple wavelength, multiple crystals, multiple sweeps) Feedback from “downstream” processing to improve data quality More sophisticated treatment of radiation damage Facilitate incorporation of other new/replacement modules, eg alternative data processing programs, absorption corrections etc Improved summary of experimental results in database (ISPyB) Automated selection of wavelength for anomalous data (peak).

8. The End

9. Protein Production Crystallisation Data Collection Phasing Protein Structure Deposition Structure analysis Target Selection Automating the data collection step of MX - the DNA project

10. Objective of DNA Given some basic information about the project and the available crystals, to determine and carry out the best possible diffraction experiment from the available crystals in the available time, by a procedure that requires little or no intervention from the user during the experiment. Make it easier for inexperienced users to collect good data. Should facilitate both “Fedex” crystallography and remote operation.

11. Input to DNA The diffraction plan (supplied by user) The type of experiment (MAD/SAD phasing, sulphur phasing, high resolution data for refinement). Resolution: ideal, minimum acceptable, maximum required. Type of anomalous scatterers for phasing experiments Unit cell and space group (if known) Crystal lifetime (if known) X-ray source properties (supplied by beamline scientists) Typical crystal lifetime (in seconds or photons) Maximum rotation rate for spindle Minimum safe exposure time (depends on shutter) Accessible wavelength range

12. Output from DNA A summary of the diffraction characteristics for every crystal tested diffraction limit (based on analysis of spot finding/ BEST)) unit cell, probable Laue group/lattice type (autoindexing) mosaicity (MOSFLM) When defined criteria have been met (resolution, best sample of those tested), one or more (MAD) data sets (h, k, l, F, sig(F)) of scaled and (optionally) merged structure factor amplitudes. The data processing may be preliminary (to save time) but will give a realistic estimate of data quality. assessment of crystal perfection (based on presence of multiple lattices) assessment of spot shape (single, streaky, split, multiple)

13. Simplified Architecture of DNA Replace the user with an “expert system”, a program with built-in decision taking. The system needs to be modular in order to accommodate existing site- specific beamline and sample control software. Different data processing software and databases should also be possible.

14. Basis of operation The expert system is responsible for issuing requests and controlling the sequence of operations. Load the next sample and centre it in the beam (BCM) Collect two initial reference images separated by 90 o in phi at resolution stated in diffraction plan (BCM) Pre-screen images for strength of diffraction (if any), presence of ice rings (DPM) If images OK, auto-index singly and together (DPM - MOSFLM) Apply acceptance criteria to indexing (rms error in spot positions, %age spots rejected from indexing, shift in direct beam position) (ES) If indexing successful, integrate both images and estimate mosaicity (DPM - MOSFLM) Based on results of integration, determine effective resolution limit of data and a data collection strategy (DPM - BEST). Write results of characterisation back to database (ISPyB). Repeat for all samples of same type, determine ranking (ES). Collect data set from the best crystal, integrate data as it is collected, run “quick scaling” (SCALA).

15. Implementation of DNA at ESRF DNA is installed on all MX beamlines except ID13. Started with a desktop icon (after starting MxCube) Currently two modes of operation: Characterising a single sample Sample screening (pipeline mode)

16. Characterising a single sample: index reference images

17. Characterising a single sample: calculate strategy

18. Characterising a single sample: modify strategy

19. Characterising a single sample: examine results I

20. Characterising a single sample: examine results II

21. Characterising a single sample: examine results III

22. Characterising a single sample: examine results IV

23. Characterising a single sample: collect data

24. Characterising a single sample: refining cell prior to integration

25. Characterising a single sample: integrating and scaling data

26. Characterising a single sample: examining log files (SCALA)

27. Sample screening: get list of samples

28. Sample screening: select samples

29. Sample screening: ranking

30. Sample screening: ranking criteria

31. Sample screening: ranking by resolution

32. “View Rank Result” allows a quick comparison of samples

33. Select one or several samples for data collection If several samples are chosen, only automatic collection is allowed (no opportunity to modify data collection parameters).

34. Collect post ref images Post refinement Data collection Integration 1-10 Data collection scenario Integration 11-20 Integration ….. Quick scale Process Time Change sample Collect two images 90° apart Index 1 st image Index 2 nd image Index both images Screening/ ranking scenario Integration, strategy and ranking Pre- screen DNA screening / data collection scenarios ~3 mins

35. Issues Conceptual How ambitious to be in terms of handling “less than perfect” crystals. How much user control should be accommodated ? Technical Depends on success of crystal centring Dependency of success of autoindexing Choice of criteria for ranking How to deal with user requested resolution ** How to deal with radiation damage ** How to make processing keep up with data collection

36. Future Plans Scrap it all and start again Extend to include collection and analysis of edge scans for design of anomalous diffraction data collection Optimise beam size wrt crystal size Allow more sophisticated data collection strategies (about more than one axis, variable exposure time/oscillation angle per image, using more than one position on a crystal or several different crystals) Allow strategy to be updated if initial estimate of point group was incorrect Automatic determination of crystal lifetime from a “sacrificial” crystal. Improve level of information stored in the ISPyB database Improve robustness

37. Diffraction plan for ISPyB