1. Developments with CCP4i & the Database Handler Peter Briggs

2. Outline CCP4i developments DbCCP4i & database developments Where we are now Still to do & future plans

3. CCP4i developments: summary New tasks: buccaneer, pointless, rapper, pisa, xia2, mrbump, getax, phaser (experimental phasing) New utilities: idiffdisp, baubles Integration of iMosflm and Ctruncate Very many other bug fixes and improvements see newsletter article from January, “New Developments in CCP4i” (#47): http://www.ccp4.ac.uk/newsletters/newsletter47/articles/ccp4i.html

4. Tasklist Presentation Based on suggestions from Charlie Bond 2 years ago Intended to provide framework for reorganising presentation of tasklists now implemented & released in 6.0.99 series Ongoing work doing the actual reorganisation received input from Eleanor, Phil, others…

5. Plugins Enables you to launch external applications from the results of a job initially applied to launching Coot and CCP4mg after Refmac 5 jobs can be extended programmatically to other tasks and applications

6. Other usability additions Improvements to interactions with job list right-click menu, double-click to see logfile, … Colourisation feature currently available in 6.0.2 but now more easily accessible Loggraph supports XMGR files from SCALA useful for Windows

7. Let’s see some of these in action…

8. DbCCP4i & the Database DbCCP4i is a server process that takes care of managing the job database Allows many programs to communicate with the job database at once (not just CCP4i) Has an associated visualiser (“DbViewer”) Project includes applications to interact with job history (starKey, reaper…)

9. Background to DbCCP4i work Funded by EU Framework 6 project “BIOXHIT” – due to end June 2008 Employed 1 full time developer (Wendy Yang) plus 1 at 50% effort (me) Described in various CCP4 newsletter articles: # 46: http://www.ccp4.ac.uk/newsletters/newsletter46/articles/project-tracking.html # 45: http://www.ccp4.ac.uk/newsletters/newsletter45/articles/ccp4_bioxhit.html

10. 10 System Architecture dbccp4i - database handler dbviewer - visualiser XIA StarKey - XML generator CCP4i projects - database.def files CCP4i db CCP4i 1.4 current version Tcl Client API CCP4i 2.0 SQLite SQL knowledge database Python Client API …other client applications (e.g. consoles,…)

11. Significant activities since last March Integration into MrBUMP and public release (July 2007) Expansion of the job database to support “subjobs” (August 2007) Initial implementation of demonstration SQLite crystallographic knowledge base (August 2007) Wendy left (September 2007) Integration into CCP4i for 6.1 (ongoing, initial phase completed March 2008 & released with 6.0.99c)

12. Integration with MrBUMP Work done by Ronan and Wendy Uses a customised version of DbCCP4i in parallel with user’s existing projects Creates new CCP4i project database for each run Steps in MrBUMP are stored as jobs DbViewer allows progress to be monitored and reviewed

13. Integration with CCP4i CCP4i 2.0 (in 6.0.99c) uses DbCCP4i as database backend by default seems to work but still some issues to resolve update of job list is slow, especially for large projects & using colourisation not tested fully on Windows (expect some problems) compatibility issues for MrBUMP old “direct access” mode still available as a workaround Fully expect to resolve issues before official 6.1 release

14. Where we are now – to-do for 6.1 Complete work on tasklist reorganisation – including updated documentation Complete testing and integration of database handler into CCP4i fix know issues with speed of job display properly integrate DbViewer into CCP4i address compatibility issues for MrBUMP Various bug fixes required and minor features still to be added ~50 issues currently logged in bug tracker

15. Looking slightly further ahead Bioxhit deliverables (for June 2008): “reaper” program to analyse job database and extract harvesting information (deliverable 5.2.17) Make demonstration knowledge base accessible from DbViewer (deliverable 5.2.16) MrBUMP and database: update MrBUMP to use subjobs mechanism to record individual steps in each run

16. Looking ahead even further Wider adoption of database by other applications or pipelines? Expansion of database content? Store more data for each job, better treatment of history links, application-specific data/tags … Work on SQLite database to store project-based data? Improve the visualisation options Data-centric rather than process-centric view, improve richness of view …

17. Acknowledgements CCP4i Originally developed by Liz Potterton Current development by DL staff (Peter Briggs, Charles Ballard, Ronan Keegan, Francois Remacle, Norman Stein & Martyn Winn) Contributions also from Kevin Cowtan, Phil Evans, Airlie McCoy & Randy Read, Nick Furnham DbCCP4i Wanjuan Yang, Peter Briggs and Ronan Keegan And Martyn for lending me his laptop for the demos…