2. ISPyB
December 4th, 2013
From sample to data analysis: how to track every step of an experiment in the ISPyB database.
Marjolaine Bodin, ESRF/EXP/Structural Biology Group
ISPyB – 04/12/2013

3. Contents What is ISPyB? History Project management Technology Users
From sample to data analysis
Dewar Tracking
Before the experiment
Ongoing experiment
Data analysis result
Data mining and reporting
ISPyB – 04/12/2013

4. What is ISPyB?
ISPyB: Information System for Protein CrystallographY Beamlines
Laboratory Information Management System (LIMS) for protein crystallography experiments on synchrotron beamlines
Web based interface
Access
From a web browser
To log in: ESRF experiment number (ESRF login) and password (LDAP authentication)
Where is it used at the ESRF?
ID14-4
ID23-1, ID23-2
ID29
BM14
MASSIF (ID30)
BM29 (BioSAXS)
ISPyB – 04/12/2013

5. History 2001 - 2005: Pxweb 2005 - now: ISPyB
joint development between the ESRF Joint Structural Biology Group (JSBG), BM14 (e-HTPX), the EU funded SPINE project
2009: Collaboration with Diamond (code sharing)
2012: BioSAXS
Year
Total collects
2002
1598
2005
33,861
2011
155,684
2013
106,373
ISPyB – 04/12/2013

6. Project management Software project leader: Solange Delageniere
Scientist project leader: Stephanie Malbet Monaco
Mailing lists:
to give general news about ISPyB project (new releases, major achievements, ...) , the mail shall include which technique/institute is concerned by the news.
to give feedbacks on ISPyB in general on the ESRF site, the feedback mail may contain the technique/proposal concerned
to discuss all the technical and development subjects (code sharing, data model, etc...) concerning MX internally and with other institutes.
to discuss all the technical and development subjects (code sharing, data model, etc...) concerning BioSaxs internally and with other institutes.
Development forge:
Bug and New Feature tracking
ISPyB – 04/12/2013

7. ISPyB Technology Java technology,
Struts (MVC Web Application Framework)
running on a JBoss 6 application server,
supports both MySQL and Oracle databases (15-20 GB)
Sencha Ext JS 4: Javascript framework
Web services API (SOAP)
Free software: LGPL licence
ISPyB – 04/12/2013

8. ISPyB Technology New server installed in early 2013 pydevserv
pyproserv
pydevserv
Prod. JBOSS server
Spare JBOSS server
(Mirrored deployments & config)
Prod. MySQL DB
Test JBOSS server
Replicated MySQL DB
Test MySQL DB
Dev MySQL DB
File system: /data/pyarch/
Dell R720xd, running on Debian 6
Main characteristics: Dual Intel Xeon processors (e.g. E GHz 6 cores) 16 TB high-speed storage 64 GB RAM 10 Gbit networks… Ability to add external disk trays for more storage Ability to configure more internal storage
File system copy /data/pyarch/
(asynchronous – rsynch based)
ISPyB – 04/12/2013

9. ISPyB users End Users: users on site or remote.
Type of proposals: Industrials (IX or FX) or MX users
IX: Industrial Users who come to the ESRF
FX: Industrial Users who don’t come to the ESRF (MXPress Service)
MX: Academic Users (mainly BAG / Block Allocation Group)
Local contacts (dewar tracking + monitoring of the collected data)
Managers (monitoring of the collected data + statistics)
Stores (dewar tracking)
Blom (dewar tracking + statistics on a beamline) U LC M S B
ISPyB – 04/12/2013

10. From sample to data analysis
ISPyB – 04/12/2013

11. Dewar tracking Dewar location at whole timeU LC M S B
Dewar location at whole time
Communication between users / ESRF staff
Dewar description (beamline, local contact, user address)
Labels automatically filled
Automatic on arrival / departure at / from the ESRF store
Search options for Stores
Dewar Location
Dewar Labels (sending, return)
ISPyB – 04/12/2013

12. Dewar tracking U LC M S B
ISPyB – 04/12/2013

13. Before the experiment U LC
Prepare Experiment at home (and not waste time at the beamline)
Communication between users / Local contacts
Sample description (description, bar-code, location, known unit cell dimensions, space group, diffraction plans, comments
Sample information loaded into the BCM (MxCuBE)
Shipment Description (Excel)
ISPyB – 04/12/2013

14. Before the experiment Shipment Description (web application)U LC
Shipment Description (web application)
Shipment Description (CSV file)
ISPyB – 04/12/2013

15. Ongoing experiment U LC M B
Remote tracking by colleagues at home while the experiment is performed
Communication between users / Local contacts
Operator comments on data collections
Facilitates real-time online monitoring of the collected data LC U M
ISPyB – 04/12/2013

16. Experiment metadata / Electronic logbook
Session info: date, beamline name, operator…
Experiment parameters: wavelength, energy, detector distance…
Beamline parameters: name, beam transmission, beam size,…
Crystal snapshots
Images (thumbnails, link to real images)
EDNA results
Autoprocessing results
Parameters and results
Crystal Snapshots
Image thumbnails
ISPyB – 04/12/2013

17. Characterisation parameters Characterisation results
Data analysis results U LC M B
Results from the EDNA MX characterisation pipeline can be accessed via the web application as a HTML file detailing one or more proposed data collection strategies.
Characterisation parameters
Characterisation results
ISPyB – 04/12/2013

18. For suitable data collections, automatic data processing is carried out and results are accessible from the web application.
Autoprocessing report
Autoprocessing results
ISPyB – 04/12/2013

19. Data mining and reportingU LC M B
Long term tracking of experiments
Process experiment results back into the Home LIMS
Reports
Statistics on how experiments are carried out
Screenings/Full data collections ratio,…
Figures on feature usage (training, interface improvement…) report
Data collection report
ISPyB – 04/12/2013

20. ISPyB data model ISPyB – 04/12/2013 Protein / Crystal / Sample
Autoprocessing
Shipment / Dewar / container
Experiment
EDNA
Data Collection
ISPyB – 04/12/2013

21. References and Acknowledgements
Delageniere,S. et al. (2011) ISPyB: an information management system for synchrotron macromolecular crystallography. Bioinformatics, Vol. 27 no. 22, 3186– 3192.
Thanks to
ISPyB – 04/12/2013