1. 3D Reconstruction of Anatomical Structures from Serial EM images

2. Biological Motivation Reconstruct 3D structures from serial stack of EM images to understand –Distribution of cell type and organelles. –Connectivity and vasculature. Requires –Tracing of cell membranes, organelles, blood vessels, etc. –Identification of structures for serial correspondence. Currently, reconstruct only a small fraction of volume (very few objects). –Time consuming (~20hours per specimen). –Wealth of information in surround structures not utilized. Synapse Web, Kristen M. Harris, PI http://synapses.clm.utexas.edu/

3. Serial TEM Dataset typical volume: –20-50 slices –8 x 5 x 0.05  m per slice –40 to 100  m 3 volume –fly brain volume: 0.1mm 3 resolution: –xy: 2.6 nm/pixel (350~400 pixels/  m) –z: 0.05  m ~20 pixels apart storage size: –small volume: ~100MB –fly brain: 3.2x10 14 pixels –compression of 4 would result in 73 terabytes. –(source: Fiala, BU) serial direction Data from Synapse Web, Kristen M. Harris, PI http://synapses.clm.utexas.edu/

4. Challenges for Computer Vision Segmenting objects –EM images are inherently noisy. –Gaps in membrane. –Adjacent structures share weak membrane boundary. –Organelles too small to use common descriptors such as texture. Identification and correspondence –Structures can merge, split, appear, or disappear (yellow arrow). –z-axis structures (red) are easier to maintain correspondences than lateral structures (green). –z resolution much lower than xy resolution (large changes serially). –Automatic registration difficult (no ground truth) 3D reconstruction –good software available, but getting to this step is the challenge.

5. Preliminary 2D Segmentation Parametric snakes Red-initial contour Green-final contour Highly sensitive to initialization (bottom) Automatic initialization is a big challenge.

6. Preliminary 2D Segmentation Geometric active contours. Provides topological flexibility. Less sensitive to initialization. Adjacent objects often merge (bottom).

7. Preliminary 2D Segmentation Level set with elastic edge interaction* Zero level contour of v provides “good” initialization. Still many problems. *Xiang et al. J. Comp. Phys. 2006

8. Preliminary 2D Segmentation Previous method produces binary masks of cross sections. Correspondences can be made based on distance and area of overlap. Inconsistencies occur often (green)

9. Preliminary 3D Reconstruction Reconstructing “everything” at the same time produces confusing volume. Inconsistencies in segmentation and correspondence produce artifacts.

10. Open Issues 2D Segmentation challenges –Automatic initialization. –Segmenting adjacent objects sharing weak edges. –Noise. Cross section correspondence –Identifying objects (synapse, mitochondria, etc.) –Tracking contours serially and detecting merging/splitting events. –Automatic registration. Current Work: simultaneous segmentation and correspondence.