towards a zoomable cell abstract cell natural coordinate system Data > D Protein Structures from PDB ? A IHGFBCDE > Images from scientific publications 1

Computer Graphics and Visualization TECHNISCHE UNIVERSITÄT DRESDEN Zoomable Cell Stefan Gumhold Michael Schröder Norbert Blenn Anne Tuukkanen Marcel Spehr Matthias Reimann

Computer Graphics and Visualization Zoomable Cell, SPP 1335, Kickoff Meeting, , Dagstuhl3 Goals  Data analysis  Natural coordinate system (NCS)  Mapping of images from literature to NCS  3D models of complexes in NCS  Visualization  aggregation of images, volumes and 3D models  Rendering across scale from 10  m to 1Å  Natural adjustment of visualization parameters with dynamic labeling  HCI  support for Virtual Reality environments  speech control and input device development  flexible navigation  community support through web integration  Impact  Interface life scientists „from different scales“  data aggregation and analysis platform  production of illustrative materials

Computer Graphics and Visualization Zoomable Cell, SPP 1335, Kickoff Meeting, , Dagstuhl4 Human Cells New Problems Several different instances of the same type each instance is flexible cells are treated badly before imaging very different imaging modalities are used Deformation Framework

Computer Graphics and Visualization Zoomable Cell, SPP 1335, Kickoff Meeting, , Dagstuhl5 Various Data Types cell nucleus pore complexes proteins primitives, smooth surfaces implicit surfaces height fields images: 2D, 3D, perspective

Computer Graphics and Visualization Zoomable Cell, SPP 1335, Kickoff Meeting, , Dagstuhl6 Data Augmentation  define reference models  for each dataset  scale  imaging modality  features  points  curves  regions  labeling of features  for pairs of datasets  feature mapping  additional alignment information nucleolus envelop pore

Computer Graphics and Visualization Zoomable Cell, SPP 1335, Kickoff Meeting, , Dagstuhl7 Integration of Datasets Segmentation Feature Detection Labeling non-rigid Registration

Computer Graphics and Visualization Zoomable Cell, SPP 1335, Kickoff Meeting, , Dagstuhl8 Deformation reference model

Computer Graphics and Visualization Zoomable Cell, SPP 1335, Kickoff Meeting, , Dagstuhl9 Plan to a Solution start with fully interactive tools add automation step by step with full interactivity for corrections find features that persist over different scales develop learning based segmentation approaches exploit mutual information to register datasets of different dimension and modality

Computer Graphics and Visualization Zoomable Cell, SPP 1335, Kickoff Meeting, , Dagstuhl10 Visualization Engine  protein structures  primitive splatting  tubes, surfaces  deferred shading  sorting based transparency  3d surface models  LOD based rendering  depth peeling based transparency  Images & Volumes  volume rendering  compression  transfer functions

Computer Graphics and Visualization Example Images

Computer Graphics and Visualization Query Based Exploration of Images

Available image information Expert labeled text (categorical) Unstructured information of related text (textual) Inherent image features (abstract description of image appearance) More reliable and structured Less reliable and structured

Navigation/Exploration Around images currently available to us Even with automatic analysis one needs supporting browsing techniques If we have features that measure appropriate image similarities: – Hierarchical Browsing – Fish-Eye View

Hierarchical Browsing

Fish-Eye View

Methods to structure image data set By hand Automatic analysis (off-the-shelf methods) – Unsupervised (Clustering) – Supervised (Multiclass Support Vector machines) Need for appropriate problem oriented feature set

Image Feature Definition Vast numbers of image descriptors are available Need for general purpose image descriptors because of wide variety of image origins Standardized Multimedia content description (MPEG-7)

Class information from Image Features 1.Definition of semantic classes (assisted and manually, Gene Ontology labels) 2.Relation of abstract image descriptors to semantic classes (training, learning) 3.Evaluation of generalization ability

GoImage – Semantic Image Search Comprehensive protein-interaction mapping projects underway What is the cost of completing an interactome map and what is the best strategy for minimizing the cost? How can quality and coverage of interaction data be maximized?

GoImage – Semantic Image Search

Refinement of a search for membranes through selecting nuclear envelope p.a.