Solange Delageniere ESRF/TID/MIS Bio Struct BioSAXS meeting May 22, 23 2012 ISPyB From your sample to your data analysis: how to track every step of your experiment in a database. An example with ISPyB for MX experiments Solange Delageniere ESRF/TID/MIS ISPyB - Bio Struct BioSAXS meeting 22 May 2012

Contents What is ISPyB? Tracking, experiments, samples, data flows… Features : how can it be adapted to BioSaxs ? Architecture How does it help? Ongoing Evolutions ISPyB - Bio Struct BioSAXS meeting 22 May 2012

What is ISPyB? ISPyB: Information System for Protein CrystallographY Beamlines Laboratory Information Management System (LIMS) for protein crystallography experiments on synchrotron beamlines Web based interface Access http://ispyb.esrf.fr To log-in: ESRF experiment number and password Where is it used at the ESRF? ID14-1, ID14-4 BM29 very soon ! (BioSAXS) ID23-1, ID23-2, ID29 BM14 I SPY on Beamlines ISPyB - Bio Struct BioSAXS meeting 22 May 2012

Sample tracking and Data Flow ISPyB - Bio Struct BioSAXS meeting 22 May 2012

History 2001-2005: Pxweb 2005-2011: ISPyB Python/Zope, MySql JSBG Project (ESRF/EMBL) Experiment logging for MXPress users (FX users) 2005-2011: ISPyB Java/Struts/Jboss, Mysql ESRF/Spine/Bioxhit and BM14/MRC/eHTPX collaboration Pxweb features + new features + Dewar Tracking 2009: Collaboration with Diamond (code sharing) ISPyB - Bio Struct BioSAXS meeting 22 May 2012

ISPyB in 2012 Important upgrade Very near future Project management ISPyB data model extensions (Autoprocessing, EDNA) Jboss server 6.0, EJBs3 Web services to communicate with BLC (MXCube) Very near future ISPyB for MASSIF : ISPyB for BioSaxs Project management Scientist project leader : Elspeth Gordon Software project leader : Solange Delageniere Meeting and video conference with Diamond Light Source Development forge http://forge.epn-campus.eu/projects/ispyb ISPyB - Bio Struct BioSAXS meeting 22 May 2012

ISPyB Features & Modules Experiment Management Sample Management Experiment Recording Search and Reporting Ready for BioSaxs ? To be customized for BioSaxs ? To be developed for BioSaxs ISPyB - Bio Struct BioSAXS meeting 22 May 2012

Experiment Management Ready Experiment Management Items: Proposals Sessions (date, beamline, local contact) Lab Contacts (email, address,..) Proteins (approved sample sheets) Features: Data retrieved from the User Portal (SMIS) Edit Lab Contact Card ISPyB - Bio Struct BioSAXS meeting 22 May 2012

Lab Contacts Ready Search Edit Select ISPyB - Bio Struct BioSAXS meeting 22 May 2012

Sample Management To be customized Items: Features: Shipments (set of dewars) Dewars (barcode, courier tracking number) Containers (barcode) Samples (barcode, protein acronym, crystal form,…) Features: Online shipment description Shipment description upload (Excel file) Dewar Tracking ISPyB - Bio Struct BioSAXS meeting 22 May 2012

Creating Shipments Ready Shipment Creation Shipment Description (Excel) Dewar Labels (sending, return) ISPyB - Bio Struct BioSAXS meeting 22 May 2012

Ready Dewar Tracking ISPyB - Bio Struct BioSAXS meeting 22 May 2012

Experiment Recording Items: To be customized To be developed Session info (date, beamline name, operator,…) Experiment parameters (wavelength, energy, detector distance,…) Beamline parameters (name, beam transmission, beam size,…) Crystal snapshots Images (thumbnails, link to real images) Data Analysis / EDNA results (characterisation & strategy, mosflm.log, pointless.log,…) To be customized To be developed ISPyB - Bio Struct BioSAXS meeting 22 May 2012

Data Collections To be developed Data Collections of a Session ISPyB - Bio Struct BioSAXS meeting 22 May 2012

Data Collections To be customized Parameters & Results Crystal Snapshots Image thumbnails (image wall) Data Analysis reports Edge Scan XRF Spectrum ISPyB - Bio Struct BioSAXS meeting 22 May 2012

Search and Reporting To be developed Search Reports Data Collection (protein, sample, experiment date,…) Proteins Samples Reports Pdf, Word, Csv ISPyB - Bio Struct BioSAXS meeting 22 May 2012

Ranking samples To be developed Ranked Samples Sample Screening DNA/EDNA Characterisation Sample Ranking & Selection Data Collect … ISPyB - Bio Struct BioSAXS meeting 22 May 2012

Reporting for Managers Dashboard Ready Reporting for Managers Dashboard ISPyB - Bio Struct BioSAXS meeting 22 May 2012

ISPyB Data Model To be customized ISPyB - Bio Struct BioSAXS meeting 22 May 2012

ISPyB Technology Pure Web Interface Java Jboss Application Server (Ejb) Struts (MVC Web Application Framework) MySql / Oracle Database Communication with User Portal (SMIS) via Web Services Communication with BCM and EDNA Through web Services ISPyB - Bio Struct BioSAXS meeting 22 May 2012

How does it help? Users Communication between Users and ESRF Staff Prepare Experiment at home (and not waste time at the beamline) Remote tracking by colleagues at home while the experiment is performed Process experiment results back into the Home LIMS Long term tracking of experiments (stored in the database) Communication between Users and ESRF Staff Dewar description (beamline, local contact, user address) Sample description (description, bar-code, location, known unit cell dimensions, space group, diffraction plans, comments) Operator comments on data collections ISPyB - Bio Struct BioSAXS meeting 22 May 2012

How does it help? Management and Operations Safety Dashboards for monitoring operations Statistics on how experiments are carried out Screenings/Full data collections ratio,… Figures on feature usage (training, interface improvement…) Figures on data throughput (Data Collection Pipeline optimization) Safety Green/yellow/red samples depending on the level of risk Sample location thanks to the Dewar Tracking ISPyB - Bio Struct BioSAXS meeting 22 May 2012

Ongoing Evolutions MASSIF project ISPyB for BioSaxs Playback Centring and Grid Scanning integration Data presentation (by project or acronym instead of sessions) MASSIF project ISPyB for BioSaxs ISPyB - Bio Struct BioSAXS meeting 22 May 2012

UPBL10 / MASSIF and ISPyB Current Workflow: Sequential Data Collection Pipeline Receiving dewars [ [ Screening / Online Data Analysis ]* / Ranking / Data Collection ]* Returning dewars ISPyB - Bio Struct BioSAXS meeting 22 May 2012

UPBL10 / MASSIF and ISPyB Sample Sorting (*1) Depends on sample description (size, mounting,…) Depends on beamline workload Screening Automation and Data Analysis (*2) More reliant on information from the user (composition…) Grid Scanning and Sample Centering results Data Storage (images, characterisation and strategy) ISPyB - Bio Struct BioSAXS meeting 22 May 2012

ISPyB For BioSAXS Features: Developments: Sample tracking, experiment management and recording, search and reporting and web interface as for MX beamlines Developments: Experiment recording: data model and GUI extensions for BioSAXS Other modules: as it is in ISPyB or with small customisations Link with MX experiments and SANS experiments… ISPyB - Bio Struct BioSAXS meeting 22 May 2012

Thanks to… Questions… Elspeth Gordon ESRF Stéphanie Monaco Sean McSweeney Gordon Leonard Gianluca Cioci Pernot Petra Adam Round EMBL Matthew Bowler Max Nanao Martine Moroni Serges Ronseaux Thierry Caruana Eric Fertin Markus Oscarsson Olof Svensson Darren Spruce Matias Guijarro Bernard Lavault Alun Ashton DLS Martin Walsh DLS/BM14 Karl Levik Graeme Winter Patrice Brenchereau ESRF Marjolaine Bodin Alejandro De Maria Antolinos ESRF - BIOStruct Ludovic Launer MRC/BM14 Emmanuel Eyer Christian Rolland Samuel Jones Luke Kirkpatrick … people I forgot to mention (sorry) … TID/MIS group for their technical support and kindness … and all users for their constructive feedbacks Questions… ISPyB - Bio Struct BioSAXS meeting 22 May 2012