1. Solange Delageniere ESRF/TID/MIS
TID Day
June 8th, 2012
ISPyB
MX beamlines upgrade and ISPyB new features
Solange Delageniere ESRF/TID/MIS
ISPyB – TID Day – 8 June 2012

2. Contents What is ISPyB? Tracking, experiments, samples, data flows…
Features
How does it help?
Ongoing evolutions linked to the beamlines upgrade
UPBL10 - MASSIF (Massively Automated Sample Selection Integrated Facility)
BM 29 BioSaxs
ISPyB – TID Day – 8 June 2012

3. What is ISPyB?
ISPyB: Information System for Protein CrystallographY Beamlines
Laboratory Information Management System (LIMS) for protein crystallography experiments on synchrotron beamlines
Web based interface
The EYE of HORUS has a very specific meaning
Helps to see the invisible
Consists of 6 parts
The 6 senses
6 doors to collect data
Egyptian fraction system to measure the world
(1/2, 1/4, 1/8, 1/16, 1/32, 1/64)
I spy on Beamlines
The EYE OF HORUS has a very specific meaning.
Helps to see the invisible : painted on the ship prow to allow to see and to hold the course at sea.
The eye is represented as a figure with 6 parts. These 6 parts correspond to the six senses - Touch, Taste, Hearing, Thought, Sight, Smell. These are the 6 parts of the *eye*. The eye is the receptor of *input*. It has these six doors, to receive data
ISPyB – TID Day – 8 June 2012

4. Where is ISPyB? Access Where is it used at the ESRF?
To log-in: ESRF experiment number and password
Where is it used at the ESRF?
ID14-1, ID14-4
BM29 very soon ! (BioSAXS)
ID23-1, ID23-2, ID29
BM14
ISPyB – TID Day – 8 June 2012

5. History 2001-2005: Pxweb 2005-2011: ISPyB Python/Zope, MySql
JSBG Project (ESRF/EMBL)
Experiment logging for MXPress users (FX users)
: ISPyB
Java/Struts/Jboss, Mysql
ESRF/Spine/Bioxhit and BM14/MRC/eHTPX collaboration
Pxweb features + new features + Dewar Tracking
2009: Collaboration with Diamond (code sharing)
The project has existed for over 10 years
ISPyB – TID Day – 8 June 2012

6. ISPyB in 2012 Important upgrade Very near future Development forge
ISPyB data model extensions (Autoprocessing, EDNA)
Jboss server 6.0, EJBs3
Web services to communicate with BLC (MXCube)
Very near future
ISPyB for MASSIF :
ISPyB for BioSaxs
Development forge
ISPyB – TID Day – 8 June 2012

7. Sample tracking and Data Flow
ISPyB – TID Day – 8 June 2012

8. ISPyB Features & Modules
Shipment, dewars, containers, samples
Dewar tracking
ISPyB Features & Modules
Experiments management
Samples management
Experiments logging
Search and reporting
ISPyB – TID Day – 8 June 2012

9. ISPyB Features & Modules
Experiments management
Link with the SMIS
Proposals
Sessions (date, beamline, local contact)
Lab Contacts ( , address,..)
Proteins (approved sample sheets)
ISPyB Features & Modules
Experiments management
Samples management
Experiments logging
Search and reporting
ISPyB – TID Day – 8 June 2012

10. ISPyB Features & Modules
Experiments logging
Image thumbnails
Parameters & Results
Crystal Snapshots
XRF Spectrum
Crystal ranking
Experiments management
Samples management
Experiments logging
Search and reporting
ISPyB – TID Day – 8 June 2012

11. ISPyB Features & Modules
Reporting
Experiments management
Samples management
Experiments logging
Search and reporting
ISPyB – TID Day – 8 June 2012

12. How does it help? Users Communication between Users and ESRF Staff
Prepare Experiment at home (and not waste time at the beamline)
Remote tracking by colleagues at home
Process experiment results back into the Home LIMS
Long term tracking of experiments (stored in the database)
Communication between Users and ESRF Staff
Dewar description (beamline, local contact, user address)
Sample description (description, bar-code, location, known unit cell dimensions, space group, diffraction plans, comments)
Operator comments on data collections
Nice crystal
Low resolution
ISPyB – TID Day – 8 June 2012

13. How does it help? Management and Operations Safety
Dashboards for monitoring operations
Statistics on how experiments are carried out
Screenings/Full data collections ratio,…
Figures on feature usage (training, interface improvement…)
Figures on data throughput (Data Collection Pipeline optimization)
Safety
Green/yellow/red samples depending on the level of risk
Sample location thanks to the Dewar Tracking
ISPyB – TID Day – 8 June 2012

14. Ongoing Evolutions
Playback Centring and Grid Scanning integration
Data presentation (by project or acronym instead of sessions)
MASSIF project (Massively Automated Sample Selection Integrated Facility)
ISPyB for BioSaxs
ISPyB – TID Day – 8 June 2012

15. UPBL10 / MASSIF and ISPyB
Current Workflow: Sequential Data Collection Pipeline
Receiving dewars
[ [ Screening / Online Data Analysis ]* / Ranking / Data Collection ]*
Returning dewars
ISPyB - Bio Struct BioSAXS meeting 22 May 2012

16. UPBL10 / MASSIF and ISPyB Sample Sorting (*1)
Depends on sample description (size, mounting,…)
Depends on beamline workload
Concurrent screening Automation and Data Analysis (*2)
More reliant on information from the user (composition…)
Grid Scanning and Sample Centering results
Data Storage (images, characterisation and strategy)
ISPyB – TID Day – 8 June 2012

17. ISPyB For BioSAXS Features: Developments:
Sample tracking, experiment management and recording, search and reporting and web interface as for MX beamlines
Developments:
Experiment recording: data model and GUI extensions for BioSAXS
Other modules: as it is in ISPyB or with small customisations
Link with MX experiments and SANS experiments…
ISPyB – TID Day – 8 June 2012

18. Thanks to… Questions… Elspeth Gordon ESRF Stéphanie Monaco
Sean McSweeney
Gordon Leonard
Gianluca Cioci
Pernot Petra
Adam Round
EMBL
Matthew Bowler
Max Nanao
Eric Fertin
Thierry Caruana
Markus Oscarsson
Olof Svensson
Darren Spruce
Matias Guijarro
Bernard Lavault
Alun Ashton
DLS
Martin Walsh
DLS/BM14
Karl Levik
Graeme Winter
Patrice Brenchereau
ESRF
Marjolaine Bodin
Alejandro De Maria Antolinos
ESRF - BIOStruct
Ludovic Launer
MRC/BM14
Emmanuel Eyer
Christian Rolland
Samuel Jones
Luke Kirkpatrick
Stephanie Veyrier
BioXhit
… people I forgot to mention (sorry)
… TID/MIS group for their technical support and kindness
… and all users for their constructive feedbacks
Questions…
ISPyB – TID Day – 8 June 2012

19. Creating Shipments Shipment Creation Shipment Description (Excel)
Dewar Labels (sending, return)
ISPyB – TID Day – 8 June 2012

20. To be customized
ISPyB Data Model
ISPyB – TID Day – 8 June 2012

21. Experiment Recording Items: To be customized To be developed
Session info (date, beamline name, operator,…)
Experiment parameters (wavelength, energy, detector distance,…)
Beamline parameters (name, beam transmission, beam size,…)
Crystal snapshots
Images (thumbnails, link to real images)
Data Analysis / EDNA results (characterisation & strategy, mosflm.log, pointless.log,…)
To be customized
To be developed
ISPyB – TID Day – 8 June 2012

22. Data Collections Data Collections of a Session
ISPyB – TID Day – 8 June 2012

23. Search and Reporting To be developed Search Reports
Data Collection (protein, sample, experiment date,…)
Proteins
Samples
Reports
Pdf, Word, Csv
ISPyB – TID Day – 8 June 2012

24. ISPyB Technology Pure Web Interface Java
Jboss Application Server (Ejb)
Struts (MVC Web Application Framework)
MySql / Oracle Database
Communication with User Portal (SMIS) via Web Services
Communication with BCM and EDNA
Through web Services
ISPyB – TID Day – 8 June 2012

25. Dewar Tracking
ISPyB – TID Day – 8 June 2012

26. ISPyB Features & Modules
Experiment Management
Sample Management
Experiment Recording
Search and Reporting
ISPyB – TID Day – 8 June 2012

27. Experiment Management
Items:
Proposals
Sessions (date, beamline, local contact)
Lab Contacts ( , address,..)
Proteins (approved sample sheets)
Features:
Data retrieved from the User Portal (SMIS)
Edit Lab Contact Card
ISPyB – TID Day – 8 June 2012

28. Sample Management Items: Features: Shipments (set of dewars)
Dewars (barcode, courier tracking number)
Containers (barcode)
Samples (barcode, protein acronym, crystal form,…)
Features:
Online shipment description
Shipment description upload (Excel file)
Dewar Tracking
ISPyB – TID Day – 8 June 2012

29. Data Collections Parameters & Results Crystal Snapshots
Image thumbnails (image wall)
Data Analysis reports
Edge Scan
XRF Spectrum
ISPyB – TID Day – 8 June 2012

30. Ranking samples Ranked Samples Sample Screening DNA/EDNA
Characterisation
Sample Ranking
& Selection
Data Collect …
ISPyB - Bio Struct BioSAXS meeting 22 May 2012

31. Reporting for Managers Dashboard
ISPyB – TID Day – 8 June 2012

32. Sample tracking and Data Flow
Synchrotron
User Home laboratory
Beamline
Detector
Sample
Changer C o n t r l S f w a e s
BCM
ISPyB
database
NICE
disk storage
User Portal
SMIS ( 2 )
EDNA
Experiment
Request
Information
Reports 1 3 4 5
Files
images , …
User Information System
Home
Samples
ISPyB – TID Day – 8 June 2012

33. The data flow and application environment of ISPyB
The data flow and application environment of ISPyB. ISPyB (database, web application, dbserver) is highlighted in green, the User Home Laboratory information system in pink and experiment software external to ISPyB in blue. The means of communication between each component is described in the main text.