1. 1 CVIP Laboratory 1 Image/Volume Registration: A validation using the Finite Element Method In collaboration with A. Abdel-Hakim and A. Elbaz R. Fahmi, A. Abdel Hakim, A. Elbaz and A. A. Farag, “ New Deformable Registration Technique Using Scale Space and Curve Evolution Theory and A Finite Element Based Validation Framework ”, EMBS’06. A. El-Baz, R. Fahmi, S. Yuksel, A. A. Farag, W. Miller, M. A. El-Ghar, and T. Eldiasty, “ A New CAD System for the Evaluation of Kidney Diseases Using DCE- MRI ”, MICCAI’06.

2. 2 CVIP Laboratory 2 Image/Volume Registration  The process of spatially aligning two or more images so point- by-point correspondences can be established between them  Points corresponding to the same anatomical point are mapped to each other.  Two main families: 1.Feature-based: Extract and match salient features: edges, corners, line intersections,… 2.Area-based: directly match intensity maps: sensitive to noise, hard to solve anatomical correspondences pb, …  Other techniques: active contour-based approaches (Vemuri’03)…  New approach combining feature and level sets is proposed.

3. 3 CVIP Laboratory 3  This stage involves three main steps: 1.Interest Points Detection: Scale Space Theory is used to detect the most stable features w.r.t. scale changes (Abdel Hakim and farag’07)  First, we build invariant feature descriptors which will be matched to find the correspondent pairs of control points Step1: Feature extraction and global alignment Finer Scales The locations of extrema in the DoG pyramid correspond to the most stable features with respect to scale changes (Mikolajczyk’02, Lowe’04) compared to gradient, Hessian, Harris corner detector, …

4. 4 CVIP Laboratory 4 Feature descriptor building (cont.) 2. To achieve invariance to scales, descriptors are built using the histograms of gradient orientations on neighborhood of interest points (Mikolajczyk & Shmidth’05, Abdel Hakim & Farag’07). 3. Feature matching using the Euclidean distance.  The matched features are then used to estimate the global alignment transformation: 5-parameter matrix in 2D cases and 9-parameter transformation matrix in 3D cases.

5. 5 CVIP Laboratory 5 Local Alignment  After global alignment, iso-surfaces are evolved in one image to match those of the other image in four steps  Generate the distance map inside of the imaged organ (object of interest).  Use this distance map to generate iso-surfaces  Number to be set by user (trade-off between accuracy and speed).  Find correspondences between iso-surfaces using NCC and extracted features.  Evolve iso-surfaces in image A to match those in image B.

6. 6 CVIP Laboratory 6 : Iso-surface on source image : Iso-surface on target image : Euclidean Distance between two corresponding iso-surface points : Euclidean Distance between and Our Evolution Approach  Let’s first define the followings

7. 7 CVIP Laboratory 7 Volume B Volume A Interface Evolution Scenario

8. 8 CVIP Laboratory 8 Speed Function  We define the evolution function V such that Otherwise  We then chose  We update the iso-contours as

9. 9 CVIP Laboratory 9 Local Alignment Before Alignment Global Alignment 3D Example

10. 10 CVIP Laboratory 10 Validation using Finite Element Method F.E. Solving (Abaqus) Brain Tissue Segmentation (GM, WM, CSF) Mesh Generation (TetSplit) Mechanical Parameters Assignment B.C.’s and Loads Definitions Three Deformations are Simulated: 2x Gravity Induced Deformations (L.E. & Ogden H.E.) Ventricles Contraction  3D Validation on Brain MRI’s: 1 234 5 6

11. 11 CVIP Laboratory 11 1 A MRF-based approach was used to segment the images into different tissue classes (GM, WM, CSF, …). 2 We used “TetSplit” (SBIA/UPENN) to produce linear tetrahedral meshes conforming to quality measure required by Abaqus. 3 Each voxel is assigned Mech. Par. of its underlying tissue class. Ventricles are modeled with hyperfoam material to anticipate contact between their walls (Miller & Chinzei’02). 4 Points where falx meets skull are pinned and remaining points on outer surface are free to slide only in the plane tangent to brain surface (Miga’99 and Wasserman’99,07). 5 3 deformations are simulated:  Two gravity-induced deformations: L.E. and H.E. models are resp. used.  Ventricle contraction using H.E. model. Let’s look at each component

12. 12 CVIP Laboratory 12 Non deformed MeshDeformed Mesh Z-plane Cut of Overlay Examples from case#3

13. 13 CVIP Laboratory 13 Generation of Deformed Images  Deformed images are generated by means of F.E. interpolations.  For each simulation case, compute the dense displacement within each finite element el : simulated nodal displacements shape functions  are piece-wise linear polynomial functions.  A program is written that takes the original gray level image, the mesh definition files, deformed nodes, and outputs a deformed gray level image.

14. 14 CVIP Laboratory 14 Initial Positions Rigid Alignment Elastic Registration Example

15. 15 CVIP Laboratory 15 Quantitative assessment of the registration accuracy. Error statistics for the three simulated cases. Displacements correspond to the simulated ones. Quantitative assessment

16. 16 CVIP Laboratory 16 Red: Abaqus Green: Ours 100 randomly selected F.E. node positions and their corresponding recovered positions using our technique for Case#3

17. 17 CVIP Laboratory 17 Medical Application: Study of autism and Dyslexia  Autism is neuro-developmental disorder  Impairments in social interaction, communication.  Unusual behaviors and interests.  According to CDC, 1/150 American kids are autistic (4:1 ratio of boys to girls).  Challenges:  No definitive medical test for diagnosis.  No reliable cause is identified.  No cure. BUT: Therapies for specific symptoms.

18. 18 CVIP Laboratory 18  Neuropathological and Neuroimaging studies  Most studies revealed macroencyphaly in autism.  Increased volume in cerebellar white matter.  Reduced size of the corpus callosum.  Inconsistencies between findings.  Increased WM volume is attributed to the large number of minicolumns in autistic brain (Casanova et al.’02,04,06).  Ongoing research: A structural MRI-based correlate to autism which relates to neural connectivity. Studies of Autism

19. 19 CVIP Laboratory 19 Current Achievements  Developed neuroimaging framework to classify autism  Gyrification window.  Used distance map inside of the WM as a discriminatory feature  Registration of the CC’s within a class to a chosen reference and building average deformation fields to classify a given subject.  Tested on postmortem and in-vivo brain MRI’s.  Several publications: ISBI’06, CARS’07, ISBI’07, J. Spec. Educ. Rehab.

20. 20 CVIP Laboratory 20 Ongoing work in dyslexia research  Main Idea: Given a sample set of brain MRI’s from each group, select a reference volume and register the remaining volumes to it. For each group, create an average anatomical atlas in the space of the reference volume. Given a test subject, register its data set with each atlas and compare the CDF of the generated disp. field with the appropriate average CDF representing each group. Dyslexic Control Average the deformations field obtained during the atlas creation processes and compute the cumulative distribution of their magnitude.

21. 21 CVIP Laboratory 21 Illustration and Results Confidence RateDyslexicControl 85%16/1614/14 90%15/1614/14 95%13/1612/14 Groups Control Reference Space Average Space Register Dyslexic

22. 22 CVIP Laboratory 22 Conclusions  A new dissimilarity criterion is proposed for global shape registration which can deal efficiently with scale variations.  Comparison with existing models and its application to statistical shape modeling and shape-based segmentation was highlighted.  A new energy formulation for elastic shape registration is introduced.  Potential of the proposed framework to solve the 3D face recognition problem in presence of facial expression showed promising results. 1. Shape Registration (ICIP’07, ECCV’08)  A new image/volume registration framework is proposed.  Scale space theory is employed to extract robust feature descriptors and use them for global alignment.  Curve evolution theory is employed to handle local deformations  A novel validation framework based on finite element method is introduced. 2. Image/Volume Registration (EMBS’06, MICCAI’06, ICPR’06)

23. 23 CVIP Laboratory 23  FEM-related work Soft tissue deformation (MICCAI’05, IEEE-TBE’08, 1 book chap.)  Autism and dyslexia work (ISBI’07, 2book chapters, J. Spec. Educ. & Rehab.’06, CARS’08)  Segmentation Work (SPIE’07, ICIP’08)  And more … Other achievements

24. 24 CVIP Laboratory 24 Future directions (Cont.)  Autism: use the proposed shape registration framework to capture the morphological abnormalities of the CC and identify which specific callosal segment (s) contribute the most in the size deficit of the CC.  Extend the fast implementation of the shape-based segmentation algorithm to the case of statistically learned model using for ex. PCA.  More to be done to speed up the shape registration in 3D.  Relying on intensity map only is not very accurate in solving the anatomical correspondences problem. Other techniques are to be tested for the matching step prior to the iso-contour deformations.

25. 25 CVIP Laboratory 25 Thank You Questions?

26. 26 CVIP Laboratory 26 Segmentation With Shape Prior and Pose Invariance Input Image with noise & partial occlusion Segmentation of the Corpus Callosum

27. 27 CVIP Laboratory 27  2-Phase CV Model: One LSF  n-Phase CV Model: m LSF s and  Evolving Curve  Two constants Chan and Vese Segmentation Models  For n=4: 2 level set functions  Evolving curves:  Constant Vector:

28. 28 CVIP Laboratory 28 C-V Segmentation Energies  2-Phase Model  4-Phase Model  Minimize using gradient descent to solve the corresponding Euler-Lagrange equations in a narrow band.  Energy functional has to be differentiable.  Computationally expensive (Non-linear Parabolic PDE’s)  Slow because of CFL condition.  Sensitive to initial condition.

29. 29 CVIP Laboratory 29 C-V Segmentation with selective shape priors  For each LSF, a shape prior is implicitly represented by its SD, and a shape energy is added to the segmentation functional.  A labeling function,, is added so that the segmentation of other objects is not affected (Cremers et al.’03). Ex.

30. 30 CVIP Laboratory 30 Proposed Algorithm  Idea: only the sign of LSF s is needed for segmentation not their values (B. Song and T. Chan, SIAM’03 and Gibou & Fedkiew, CVPR’02).  Direct manipulations of energy functional do not require any differentiability condition. Rachid Fahmi and Aly A. Farag, “ A Fast Algorithm for Multi-Phase Level Set Segmentation With Selective Shape Priors ”, submitted to ICIP’08.

31. 31 CVIP Laboratory 31  Initialize: partition image domain into and  Sweep: move current point,, from its region to the other and compute E, if E decreases then update  Repeat second step until E remains unchanged 2-Phase Case: If is moved from A to B and vice-versa

32. 32 CVIP Laboratory 32  Initialize: partition image domain into  Sweep: move current point,, from its region to the other 3. Assign to the region corresponding to largest decrease.  Repeat second step until E remains unchanged. Ex. If is moved from B to the other 3 regions 4-Phase Case:

33. 33 CVIP Laboratory 33 Results on Synthetic Images Initial LSF and Labeling Function Segmentation W/O Shape Prior Segmentation With Shape Prior but no labeling Segmentation With Shape Prior and labeling  2-Phase Case & One Prior Images of size 150x150. Convergence in 1sweep < 0.1sec

34. 34 CVIP Laboratory 34 Comparison with Standard CV algorithm Energy vs. iteration Noisy Image Our segmentation Standard CV model 4sweeps=0.125sec 200iters=6.36sec

35. 35 CVIP Laboratory 35 One more comparison Energy vs. iteration Noisy Image Our segmentationStandard CV model 6sweeps=0.21sec140iters=25.85sec

36. 36 CVIP Laboratory 36 Results on Synthetic Images Segmentation w/o Shape Priors Labeling functions Segmentation With Shape Priors but no labeling fncts Segmentation With Shape Prior and labeling fncts  4-Phase Case & Two Prior 4sweeps=0.25sec 1sweeps=0.125sec

37. 37 CVIP Laboratory 37 Results on Real Images Pure CV 0.984sec Using Shape Priors 0.234sec Partial Occlusion Initial LSFs

38. 38 CVIP Laboratory 38 Robustness to Noise & Partial Occlusion

39. 39 CVIP Laboratory 39 Pose Invariance Formulation  The pose and orientation of familiar object (s) are no longer supposed known.  Assume the existence of an affine transformation between each known object and its corresponding prior.  Use our new SSD criterion to recover ‘s  The parameters of are SIMULTANEOUSLY updated during the course of evolution of the segmenting LSF s.  Application to the 2-phase and 4-phase CV models, using the actual values of LSF s not only their sign.

40. 40 CVIP Laboratory 40 2-Phase Case: 4-Phase Case: Pose Invariance: Evolution Equations of LSF’s

41. 41 CVIP Laboratory 41 Pose Invariance Formulation: 2-Phase Case W/O pose invariance With pose invariance

42. 42 CVIP Laboratory 42 Pose Invariance Formulation: 4-Phase Case

43. 43 CVIP Laboratory 43

44. 44 CVIP Laboratory 44

45. 45 CVIP Laboratory 45 Euler-Lagrange Equations  Gradient flow w.r.t. u  Gradient flow w.r.t.  Initial conditions and Smoothing Operator