Dr. Yu(Jade) Qian y.qian@mdx.ac.uk MIRAGE I & II Dr. Yu(Jade) Qian y.qian@mdx.ac.uk

Content Introduction of MIRAGE project Content-based 3D brain images retrieval and visualization Conclusion and future work Demonstration

PART I Introduction of MIRAGE Project

MIRAGE (Middlesex medical Image Repository with a CBIR ArchivinG Environment) Phase 1: MIRAGE- from Creation to Archiving Strand : Start-up Repository funded by JISC Between Apr. 2009- Sep. 2010. Aim: To develop a repository of medical images benefiting MSc and research students in the immediate term and serve a wider community in the long term in providing a rich supply of medical images for data mining, to complement MU current online e-learning system OASIS+. Phase 2: MIRAGE 2011 – from Archiving to Creation Strand : Take-up and Embedding funded by JISC Between Feb. 2011- Oct. 2011. To enrich the current repository MIRAGE with two necessities of ‘3D Viewer’ and ‘Uploading’ to meet users’ needs, leading to a sustainable, usable and flexible model of data management.

Framework for MRIAGE I &II

Phase 1: MIRAGE – Online System(1) Server side: Image collection: Accommodating 100,000 2D images and 100 3D images Visual feature extraction: Pre-processing off-line using C++ and Perl. Indexing file creation Client side: Interface based on PHP generate dynamic web pages. Client – Server communication protocol: MRML: a XML based protocol

Phase 1: MIRAGE – Online System(2) Interface (1) Home page (2) Query and retrieval results

Phase 2: MIRAGE2011 – Online System(1) Image Uploading

Phase 2: MIRAGE2011 – Online System(2) 3D Viewer Montage MIRAGE 2011

PART II Content-based 3D Brain Images Retrieval and Visualization Y PART II Content-based 3D Brain Images Retrieval and Visualization Y. Qian, X. Gao , M. Loomes, R. Comley, B. Barn, R. Hui, Z. Tian, Content-based Retrieval of 3D Medical Images, eTELEMED 2011, February, 2011. (Best paper award, has been invited to be extended to a journal paper).

CBIR for 3D Brain Image ------Introduction 2D brain images ----- 3D Brain Shape-based Surface of a 3D object(e.g. tumor) Texture-based Inside of a 3D object( e.g.textures representing tissue structure properties) Aim: To develop a fast texture-based 3D brain retrieval method

CBIR for 3D Brain Image ---Methodology(1) Proposed Framework

CBIR for 3D Brain Image ---Methodology(2) Pre-processing 1) Spatial Normalization---Statistical Parametric Mapping (SPM5) Transform each individual brain into a standard brain template 2) Divide 3D brain into 64 non-overlapping equally sized blocks

CBIR for 3D Brain Image ---Methodology(3) Extraction of Volumetric Texture Features 3D Grey Level Co-occurrence Matrices (3D GLCM) 3D Wavelet Transform (3D WT) 3D Gabor Transform (3D GT) 3D Local Binary Pattern (3D LBP)

Extraction of Volumetric Textures (1) ------ 3D Grey Level Co-occurrence Matrices (3D GLCM) 3D GLCM is two dimensional matrices of the joint probability p(i,j) of occurrence of a pair of gray values (i,j) separated by a displacement d = (dx,dy,dz). Formula: Feature: 52 Displacement vectors: 4 distance * 13 direction = 52 4 Haralick texture features: energy, entropy, contrast and homogeneity Feature vector: 208 components (=4 (features) * 52 (matrices)). 13 directions

Extraction of Volumetric Textures (2) ------ 3D Wavelet Transform (3D WT) 3D WT provides a spatial and frequency representation of a volumetric image. Two scale 3D Wavelet Transform: Feature: Mean and Standard deviation Feature vector: 30 components (2 (features) +15 (sub-bands))

Extraction of Volumetric Textures (3) ------ 3D Gabor Transform (3D GT) A set of 3D Gabor filters: Gabor Transform: Feature: 144 Gabor filters 4 (F) *6(θ)*6(Φ) =144 Mean and Standard deviation Feature vector: 288 components (2 (features) +144(filters))

Extraction of Volumetric Textures (4) ------ 3D Local Binary Pattern (3D LBP) Local binary pattern(LBP) is a set of binary code Ci to define texture in a local neighborhood (p,r). A histogram Hi is then generated to calculate the occurrences of different binary patterns. LBP on three orthogonal planes (LBP-TOP), i.e., XY, XZ, and YZ planes, expressed as Feature: 59 binary patterns Feature vector: 177 components (=59(patterns)*3(planes)

CBIR for 3D Brain Image ---Methodology(4) Retrieval ---Similarity Measurement Histogram Intersection(3D LBP) Normalized Euclidean distance (3D GLCM,3D WT,3D GT)

CBIR for 3D Brain Image ---Methodology(5) Lesion Detection Assume bilateral symmetry of a normal brain along its mid-plane

Evaluation ---- Test Dataset 100 MR brain images Size: 256  256  44 DICOM (Digital Imaging and Communications in Medicine) format Collected from Neuro-imaging Centre at Beijing General Navy Hospital, China

Experimental Results(1) ------ Lesion Detection

Experimental Results(2) -------Retrieval Comparative results demonstrate that LBP outperforms four 3D texture methods in terms of retrieval precision and processing speed.

Experimental Results(3) -------Query time The query time with VOI selection offers 4 times faster operation than that without.

3D Brain Visualization(1)

3D Brain Visualization(2)

CBIR for 3D Brain Image------ On-line system(1):

CBIR for 3D Brain Image------ On-line system(2): Server side: 100 3D brain images(DICOM format to JPG format) 3D visual feature extraction(4 methods): Off-line pre-processing using Matlab. 3D visualization: using Matlab Client side: Interface based on PHP generate dynamic web pages.

PART III Conclusion and Future Work

Conclusion for MIRAGE (Middlesex medical Image Repository with a CBIR ArchivinG Environment) Create Middlesex medical Image repository ( ~100000 2D images and 100 3D brain images) Create CBIR archiving environment for 2D and 3D medical images.

Future Work(1)------ Continue working on 3D Brain Image Test on the larger dataset and enrich our repository Research on clinical purpose (EC FP7) ------ Collaborate with Neuro-imaging Centre at Beijing General Navy Hospital, China.

Future Work(2)------ Echocardiogram Video Clip Enrich our repository Research for clinical purpose(EC FP7) ------ Collaborate with First Hospital of Tsinghua University, China. B-mode 2D Video Clip B-mode and M-mode Video clip UltrasonixTABLET Ultrasound scanner Colour Doppler Video Clip

Future Work(3) ------- Grid Computing MDX Grid Machine

PART VI Demonstration