1. Center for Bioimaging Informatics UCSB Bio-imaging Infrastructure December 2006 Center for Bioimaging Informatics www.bioimage.ucsb.edu Supported by NSF ITR #0331697 www.bioimage.ucsb.edu Supported by NSF ITR #0331697

2. Center for Bioimaging Informatics December 20062 Projects Bisque/OME –In use at UCSB Bisquik – Next generation system (in construction)

3. Center for Bioimaging Informatics December 20063 Motivation Analysis creates knowledge –Image analysis –Querying –Mining Available Storage systems are growing Available Processing increasing

4. Center for Bioimaging Informatics December 20064 Challenges Diverse users –UCSB Neurosciences institute Retinal detachment : Fischer et al Microtubule dynamics : Feinstein and Wilson –CMU Murphy Lab –Flybase –Computable Plant Dataset challenges –Multiple types and collection techniques –Complex images (5D) and metadata –Acquisition and management of large sets –Security of private data –Sharing of experimental data

5. Center for Bioimaging Informatics December 20065 Challenges Metadata –Collection –Organization –Sharing/interpretation Dataset management –Personal collections –Multiple organizations –Privacy and security Analysis design Analysis integration

6. Center for Bioimaging Informatics December 20066 Block System diagram Image and metadata capture DB Storage Image Analysis Collection Analysis Semantic Analysis Browse Search Content + metadata Interactive Enhancement Knowledge Discovery Ground truth Collection

7. Center for Bioimaging Informatics December 20067 BISQUE Bio-Image Semantic Query User Environment Current dataset collection BISQUE functionality –Data management –Ground truth acquisition –Analysis Architecture

8. Center for Bioimaging Informatics December 20068 Current collections UCSB Retinal Objects (confocal, EM) UCSB Microtubule Objects(light, AFM) TypeCurrentBacklogRate/yExpected 4Yrs2D Images Retinal EM5002200050023,00023K Retinal confocal P3000600240010,00010K Retinal confocal Z600140001200010,00050K Microtubule light30002500 13,000400K Microtubule AFM05001200500030K Flybase0125K0 Total estimated size in TBs and growing for 1 Lab Complexity and analysis are the main issues

9. Center for Bioimaging Informatics December 20069 Data management capabilities Digital notebook –Direct import process –Reconfigurable (confocal, 4 x microtubule, etc) Web –Web access and browsing –Organize images and metadata –Data sharing environment –Search by metadata or content –Integrated analysis

10. Center for Bioimaging Informatics December 200610 Screenshots (Digital Notebook)

11. Center for Bioimaging Informatics December 200611 Data management capabilities Digital notebook –Capture experimental/image parameters –Direct Import process –Reconfigurable (confocal, 4 x microtubule, etc) Web –Web access and browsing –Organize images and metadata –Data sharing environment –Search by metadata or content –Integrated analysis

12. Center for Bioimaging Informatics December 200612 Screenshots (browsing) Dataset Browsing

13. Center for Bioimaging Informatics December 200613 Screenshots (search)

14. Center for Bioimaging Informatics December 200614 Screenshots (search)

15. Center for Bioimaging Informatics December 200615 Screenshots (search similar)

16. Center for Bioimaging Informatics December 200616 Screenshots (search similar)

17. Center for Bioimaging Informatics December 200617 Screenshots (personal collection) Data sharing

18. Center for Bioimaging Informatics December 200618 Screenshots (5D Viewer)

19. Center for Bioimaging Informatics December 200619 Screenshot (5D with tracks)

20. Center for Bioimaging Informatics December 200620 Image analysis Integrated image analysis –Image enhancement –Cell counter (ImageJ) –Quantify microtubule dynamics –Microtubule tracker (manual, automatic) –Track identification Integration in progress –Segmentation and classification –Modeling microtubule dynamics –Relevance feedback improving search

21. Center for Bioimaging Informatics December 200621 Screenshots (Image Analysis) Image J Cell counter

22. Center for Bioimaging Informatics December 200622 System description overview Current dataset collection Current functionality –Data management –Ground truth acquisition –Analysis Architecture

23. Center for Bioimaging Informatics December 200623 Hardware/software infrastructure Hardware –16 node Cluster dual Intel Xeon 3GHZ Gigabit network switch 2 TB Storage Software –Bisque –OME (Apache, Postgresql) –Linux

24. Center for Bioimaging Informatics December 200624 Bisque Architecture Bio-Image Semantic Query User Environment Image/ metadata server WEB Digital Notebook image search external internal BISQUE XML Research in image processing and indexing Cell counter lug-in for ImageJ Research in biology features analysis Distributed computing cluster Cell counter lug-in for ImageJ Cell counter lug-in for ImageJ Cell counter plug-in for ImageJ metadata XML

25. Center for Bioimaging Informatics December 200625 OME Base Open Microscopy Environment –“OME is an open source software project to develop a database-driven system for the quantitative analysis of biological images. OME is a collaborative effort among academic labs and a number of commercial entities”. Provides base for image/metadata storage and analysis integration Boston: Sorger Lab Baltimore: Goldberg Lab Dundee: Swedlow Lab Madison: LOCI

26. Center for Bioimaging Informatics December 200626 Bisque extensions Ongoing extensions with OME –Content based search –Analysis integration with OME Segmentation Cell Counting MT identification and Analysis Etc. –Front end dataset and analysis support –Schema additions for image types and analysis –(Uncertainty modeling and queries)

27. Center for Bioimaging Informatics December 200627 Bisque Built 1 st generation w/ >5000 5-D images –Integrated several useful analyses –Being used internally –External Interest Immediate future –Continued development analysis integration Dataset integration –Remote deployments –Integration with other projects Flybase : Integration of schema/datasets in progress Computableplant.org

28. Center for Bioimaging Informatics December 200628 Projects Bisque/OME –In use at UCSB –http://hammer.ece.ucsb.edu/bisquehttp://hammer.ece.ucsb.edu/bisque guest/bioimage Bisquik – Next generation system (in construction)

29. Center for Bioimaging Informatics December 200629 Bisque/OME lessons learned Getting the “correct” metadata is hard –Correct may need to change or be reinterpreted –Sql schema difficult to change Getting the “right” name is difficult –Different terms used in different labs make data integration painful Analysis integration needs to be easy –Researcher balk at learning complex software Users expect a rich experience –Google, flickr, etc have raised the bar

30. Center for Bioimaging Informatics December 200630 New project: Bisquik DoughDB: Flexible metadata Ontology support Rich user experience –Web 2.0 : Ajax, SVG –Web based tools: DN, Graphical Annotator Programming toolkit –Smaller is better Distributed data store Support for large scale computation Expected early 2007

31. Center for Bioimaging Informatics December 200631 Block Components Blob/Image Server Metadata Renderer DoughDB Metadata Annotation Analysis Engine Permission Ontology Remote Access Web UI

32. Center for Bioimaging Informatics December 200632 Tagging Screen Shot

33. Center for Bioimaging Informatics December 200633 Motivation: Current metadata model is inflexible Adding new experimental images requires: –Changes to Digital notebook –Changes to Bisque interface –Changes to OME/postgres Shouldn’t this be easier? –“Find images tagged with rod-opsin” –“Create a region and specify the object” –“Add experimental metadata to this dataset”

34. Center for Bioimaging Informatics December 200634 Bisquik requirements Add new tag/value pairs to any db object –(Foo,2) –(visible-cell, rod) Allow multiple tags with same value –(visible-cell, rod) –(visible-cell, muller) Support fine-grained tag permission/visibility –Tags have creators and access control Support update semantics & preserve history –Timestamp tags –No deletes (except under certain conditions)

35. Center for Bioimaging Informatics December 200635 Bisquik: DoughDB imageOID1 Feature f1[ … ] NameGH1020 Foo2 pixels[OID2 OID3] imageOID1 dataServer://… Pixel-typeraw OID1 OID4 OID2

36. Center for Bioimaging Informatics December 200636 Bisquik queries Find objects (images) with tag “foo” –Select a.foo Find images with name “GV100” –Select a.name = “gv100” Find images with cellcount < 100 –Select a.cellcount < 100 Find images with a region similar to r1 based on feature f1 –Select r.image = i and l2(r.f1, r1[ … ]) < 10

37. Center for Bioimaging Informatics December 200637 DoughDB key features No classes or types only tag/value pairs –Open ended data model Pair values have owner and acl Preserves history of annotations SQL like query language Simple keyword queries

38. Center for Bioimaging Informatics December 200638 Bisquik ontology support “ontology is a data model that represents a domain and is used to reason about the objects in that domain and the relations between them”data model domainreason Unstructured tag/value –Great for taggers –Unhappy searchers Different labs use different terms for the same object.

39. Center for Bioimaging Informatics December 200639 Bisquik ontology support Dictionary of terms and relations Require (or strongly suggest) that tags and value are defined before use Drop into ontology editor when new values and tags are encountered. Integrated into search system –Permit (or offer) ‘alias’ ‘part-of’ ‘related-to’ searches

40. Center for Bioimaging Informatics December 200640 Analysis Engine Analysis Components Analysis Applications Glue language: python Component connection library –Memory for single node –MPI for cluster based execution Execution engine –Automatic component placement –Resources and efficient communication

41. Center for Bioimaging Informatics December 200641 Analysis Engine Interfaced with DoughDB All input/output are Pairs or objects Application executions recorded for data provenance

42. Center for Bioimaging Informatics December 200642 Bisquik interface http://oib.ece.ucsb.edu/bisquik Some bisque functionality Flickr-like interface for region tagging Pair tagging and keyword tagging Metadata renderers –Textual (tag) –Graphical (regions, geometry, graph data) –Analysis oriented (histogram)

43. Center for Bioimaging Informatics December 200643 Region Tagging screenshot

44. Center for Bioimaging Informatics December 200644 Bisquik Metadata Annotation Unified offline (Digital Notebook) and online manipulation. Easy to build annotation forms Allow “schema” modification “in field” Permit annotation templates to be shared between DN and Bisquik Graphical geometry annotator

45. Center for Bioimaging Informatics December 200645 Blob Image server Extensible server for write-once objects –Pixels, Features Pluggable transforms/operations –Thumbnails, slices –pixel transforms (watermarks) –Graphical metadata renderers

46. Center for Bioimaging Informatics December 200646 Remote Access All basic services are web accessible: –Soap and WSDL –DoughDB, Image server, DoughDB pairs have unique 64 bit IDs –Split between machine ID/Pair ID –All pairs are addressable –Query engine processes foreign pairs

47. Center for Bioimaging Informatics December 200647 Bisquik new hardware 10 TB disk mirrored array (20TB) Image Server Database (backup) server 8-16 Query/Analysis nodes Image Server DB/Backup Server 10TB Query Index 1TB Query Index 1TB Query Index 1TB Query Index 1TB Query Index 1TB Query Index 1TB Query Index 1TB Query Index 1TB

48. Center for Bioimaging Informatics December 200648 Status Web UI –Uploading, Tagging, simple searches DoughDB –Single node storage and queries Analysis Engine –In design

49. Center for Bioimaging Informatics December 200649 Conclusion Bisque Team: August, Jiejun, Melissa,Kris Bisquik Team: –Interface: August Jiejun Jaechok –Analysis Engine: Kris, Mellisa, Dmitry –Ontology: David(summer), Verleen(summer), Kris –DoughDB: Kris,Vebjorn

50. Center for Bioimaging Informatics December 200650 Biowall

51. Center for Bioimaging Informatics December 200651 Biowall 5x4 1600x1200 displays –(8000x4800 pixels) –38 MegaPixels 1 Head node + 5 slaves Distributed multihead X (X proxy) New simplified viewer –OpenEV image server –Local client –Web/Database client 3D visualizations Working with very large screen areas