Solange Delageniere ESRF/TID/MIS TID Day June 8th, 2012 ISPyB MX beamlines upgrade and ISPyB new features Solange Delageniere ESRF/TID/MIS ISPyB – TID Day – 8 June 2012

Contents What is ISPyB? Tracking, experiments, samples, data flows… Features How does it help? Ongoing evolutions linked to the beamlines upgrade UPBL10 - MASSIF (Massively Automated Sample Selection Integrated Facility) BM 29 BioSaxs ISPyB – TID Day – 8 June 2012

What is ISPyB? ISPyB: Information System for Protein CrystallographY Beamlines Laboratory Information Management System (LIMS) for protein crystallography experiments on synchrotron beamlines Web based interface The EYE of HORUS has a very specific meaning Helps to see the invisible Consists of 6 parts The 6 senses 6 doors to collect data Egyptian fraction system to measure the world (1/2, 1/4, 1/8, 1/16, 1/32, 1/64) I spy on Beamlines The EYE OF HORUS has a very specific meaning. Helps to see the invisible : painted on the ship prow to allow to see and to hold the course at sea. The eye is represented as a figure with 6 parts. These 6 parts correspond to the six senses - Touch, Taste, Hearing, Thought, Sight, Smell. These are the 6 parts of the *eye*. The eye is the receptor of *input*. It has these six doors, to receive data ISPyB – TID Day – 8 June 2012

Where is ISPyB? Access Where is it used at the ESRF? http://ispyb.esrf.fr To log-in: ESRF experiment number and password Where is it used at the ESRF? ID14-1, ID14-4 BM29 very soon ! (BioSAXS) ID23-1, ID23-2, ID29 BM14 ISPyB – TID Day – 8 June 2012

History 2001-2005: Pxweb 2005-2011: ISPyB Python/Zope, MySql JSBG Project (ESRF/EMBL) Experiment logging for MXPress users (FX users) 2005-2011: ISPyB Java/Struts/Jboss, Mysql ESRF/Spine/Bioxhit and BM14/MRC/eHTPX collaboration Pxweb features + new features + Dewar Tracking 2009: Collaboration with Diamond (code sharing) The project has existed for over 10 years ISPyB – TID Day – 8 June 2012

ISPyB in 2012 Important upgrade Very near future Development forge ISPyB data model extensions (Autoprocessing, EDNA) Jboss server 6.0, EJBs3 Web services to communicate with BLC (MXCube) Very near future ISPyB for MASSIF : ISPyB for BioSaxs Development forge http://forge.epn-campus.eu/projects/ispyb ISPyB – TID Day – 8 June 2012

Sample tracking and Data Flow ISPyB – TID Day – 8 June 2012

ISPyB Features & Modules Shipment, dewars, containers, samples Dewar tracking ISPyB Features & Modules Experiments management Samples management Experiments logging Search and reporting ISPyB – TID Day – 8 June 2012

ISPyB Features & Modules Experiments management Link with the SMIS Proposals Sessions (date, beamline, local contact) Lab Contacts (email, address,..) Proteins (approved sample sheets) ISPyB Features & Modules Experiments management Samples management Experiments logging Search and reporting ISPyB – TID Day – 8 June 2012

ISPyB Features & Modules Experiments logging Image thumbnails Parameters & Results Crystal Snapshots XRF Spectrum Crystal ranking Experiments management Samples management Experiments logging Search and reporting ISPyB – TID Day – 8 June 2012

ISPyB Features & Modules Reporting Experiments management Samples management Experiments logging Search and reporting ISPyB – TID Day – 8 June 2012

How does it help? Users Communication between Users and ESRF Staff Prepare Experiment at home (and not waste time at the beamline) Remote tracking by colleagues at home Process experiment results back into the Home LIMS Long term tracking of experiments (stored in the database) Communication between Users and ESRF Staff Dewar description (beamline, local contact, user address) Sample description (description, bar-code, location, known unit cell dimensions, space group, diffraction plans, comments) Operator comments on data collections Nice crystal Low resolution ISPyB – TID Day – 8 June 2012

How does it help? Management and Operations Safety Dashboards for monitoring operations Statistics on how experiments are carried out Screenings/Full data collections ratio,… Figures on feature usage (training, interface improvement…) Figures on data throughput (Data Collection Pipeline optimization) Safety Green/yellow/red samples depending on the level of risk Sample location thanks to the Dewar Tracking ISPyB – TID Day – 8 June 2012

Ongoing Evolutions Playback Centring and Grid Scanning integration Data presentation (by project or acronym instead of sessions) MASSIF project (Massively Automated Sample Selection Integrated Facility) ISPyB for BioSaxs ISPyB – TID Day – 8 June 2012

UPBL10 / MASSIF and ISPyB Current Workflow: Sequential Data Collection Pipeline Receiving dewars [ [ Screening / Online Data Analysis ]* / Ranking / Data Collection ]* Returning dewars ISPyB - Bio Struct BioSAXS meeting 22 May 2012

UPBL10 / MASSIF and ISPyB Sample Sorting (*1) Depends on sample description (size, mounting,…) Depends on beamline workload Concurrent screening Automation and Data Analysis (*2) More reliant on information from the user (composition…) Grid Scanning and Sample Centering results Data Storage (images, characterisation and strategy) ISPyB – TID Day – 8 June 2012

ISPyB For BioSAXS Features: Developments: Sample tracking, experiment management and recording, search and reporting and web interface as for MX beamlines Developments: Experiment recording: data model and GUI extensions for BioSAXS Other modules: as it is in ISPyB or with small customisations Link with MX experiments and SANS experiments… ISPyB – TID Day – 8 June 2012

Thanks to… Questions… Elspeth Gordon ESRF Stéphanie Monaco Sean McSweeney Gordon Leonard Gianluca Cioci Pernot Petra Adam Round EMBL Matthew Bowler Max Nanao Eric Fertin Thierry Caruana Markus Oscarsson Olof Svensson Darren Spruce Matias Guijarro Bernard Lavault Alun Ashton DLS Martin Walsh DLS/BM14 Karl Levik Graeme Winter Patrice Brenchereau ESRF Marjolaine Bodin Alejandro De Maria Antolinos ESRF - BIOStruct Ludovic Launer MRC/BM14 Emmanuel Eyer Christian Rolland Samuel Jones Luke Kirkpatrick Stephanie Veyrier BioXhit … people I forgot to mention (sorry) … TID/MIS group for their technical support and kindness … and all users for their constructive feedbacks Questions… ISPyB – TID Day – 8 June 2012

Creating Shipments Shipment Creation Shipment Description (Excel) Dewar Labels (sending, return) ISPyB – TID Day – 8 June 2012

To be customized ISPyB Data Model ISPyB – TID Day – 8 June 2012

Experiment Recording Items: To be customized To be developed Session info (date, beamline name, operator,…) Experiment parameters (wavelength, energy, detector distance,…) Beamline parameters (name, beam transmission, beam size,…) Crystal snapshots Images (thumbnails, link to real images) Data Analysis / EDNA results (characterisation & strategy, mosflm.log, pointless.log,…) To be customized To be developed ISPyB – TID Day – 8 June 2012

Data Collections Data Collections of a Session ISPyB – TID Day – 8 June 2012

Search and Reporting To be developed Search Reports Data Collection (protein, sample, experiment date,…) Proteins Samples Reports Pdf, Word, Csv ISPyB – TID Day – 8 June 2012

ISPyB Technology Pure Web Interface Java Jboss Application Server (Ejb) Struts (MVC Web Application Framework) MySql / Oracle Database Communication with User Portal (SMIS) via Web Services Communication with BCM and EDNA Through web Services ISPyB – TID Day – 8 June 2012

Dewar Tracking ISPyB – TID Day – 8 June 2012

ISPyB Features & Modules Experiment Management Sample Management Experiment Recording Search and Reporting ISPyB – TID Day – 8 June 2012

Experiment Management Items: Proposals Sessions (date, beamline, local contact) Lab Contacts (email, address,..) Proteins (approved sample sheets) Features: Data retrieved from the User Portal (SMIS) Edit Lab Contact Card ISPyB – TID Day – 8 June 2012

Sample Management Items: Features: Shipments (set of dewars) Dewars (barcode, courier tracking number) Containers (barcode) Samples (barcode, protein acronym, crystal form,…) Features: Online shipment description Shipment description upload (Excel file) Dewar Tracking ISPyB – TID Day – 8 June 2012

Data Collections Parameters & Results Crystal Snapshots Image thumbnails (image wall) Data Analysis reports Edge Scan XRF Spectrum ISPyB – TID Day – 8 June 2012

Ranking samples Ranked Samples Sample Screening DNA/EDNA Characterisation Sample Ranking & Selection Data Collect … ISPyB - Bio Struct BioSAXS meeting 22 May 2012

Reporting for Managers Dashboard ISPyB – TID Day – 8 June 2012

Sample tracking and Data Flow Synchrotron User Home laboratory Beamline Detector Sample Changer C o n t r l S f w a e s BCM ISPyB database NICE disk storage User Portal SMIS ( 2 ) EDNA Experiment Request Information Reports 1 3 4 5 Files images , … User Information System Home Samples ISPyB – TID Day – 8 June 2012

The data flow and application environment of ISPyB The data flow and application environment of ISPyB. ISPyB (database, web application, dbserver) is highlighted in green, the User Home Laboratory information system in pink and experiment software external to ISPyB in blue. The means of communication between each component is described in the main text.