1. Dr. Yu(Jade) Qian y.qian@mdx.ac.uk
MIRAGE I & II
Dr. Yu(Jade) Qian

2. Content Introduction of MIRAGE project
Content-based 3D brain images retrieval and visualization
Conclusion and future work
Demonstration

3. PART I Introduction of MIRAGE Project

4. MIRAGE (Middlesex medical Image Repository with a CBIR ArchivinG Environment)
Phase 1: MIRAGE- from Creation to Archiving
Strand : Start-up Repository funded by JISC
Between Apr Sep
Aim:
To develop a repository of medical images benefiting MSc and research students in the immediate term and serve a wider community in the long term in providing a rich supply of medical images for data mining, to complement MU current online e-learning system OASIS+.
Phase 2: MIRAGE 2011 – from Archiving to Creation
Strand : Take-up and Embedding funded by JISC
Between Feb Oct
To enrich the current repository MIRAGE with two necessities of ‘3D Viewer’ and ‘Uploading’ to meet users’ needs, leading to a sustainable, usable and flexible model of data management.

5. Framework for MRIAGE I &II

6. Phase 1: MIRAGE – Online System(1)
Server side:
Image collection: Accommodating 100,000 2D images and 100 3D images
Visual feature extraction: Pre-processing off-line using C++ and Perl.
Indexing file creation
Client side:
Interface based on PHP generate dynamic web pages.
Client – Server communication protocol:
MRML: a XML based protocol

7. Phase 1: MIRAGE – Online System(2) Interface
(1) Home page (2) Query and retrieval results

8. Phase 2: MIRAGE2011 – Online System(1) Image Uploading

9. Phase 2: MIRAGE2011 – Online System(2) 3D Viewer
Montage
MIRAGE 2011

10. PART II Content-based 3D Brain Images Retrieval and Visualization Y
PART II Content-based 3D Brain Images Retrieval and Visualization Y. Qian, X. Gao , M. Loomes, R. Comley, B. Barn, R. Hui, Z. Tian, Content-based Retrieval of 3D Medical Images, eTELEMED 2011, February, (Best paper award, has been invited to be extended to a journal paper).

11. CBIR for 3D Brain Image ------Introduction
2D brain images D Brain
Shape-based
Surface of a 3D object(e.g. tumor)
Texture-based
Inside of a 3D object( e.g.textures representing tissue structure properties)
Aim: To develop a fast texture-based 3D brain retrieval method

12. CBIR for 3D Brain Image ---Methodology(1) Proposed Framework

13. CBIR for 3D Brain Image ---Methodology(2) Pre-processing
1) Spatial Normalization---Statistical Parametric Mapping (SPM5)
Transform each individual brain into a standard brain template
2) Divide 3D brain into 64 non-overlapping equally sized blocks

14. CBIR for 3D Brain Image ---Methodology(3) Extraction of Volumetric Texture Features
3D Grey Level Co-occurrence Matrices (3D GLCM)
3D Wavelet Transform (3D WT)
3D Gabor Transform (3D GT)
3D Local Binary Pattern (3D LBP)

15. Extraction of Volumetric Textures (1) ------ 3D Grey Level Co-occurrence Matrices (3D GLCM)
3D GLCM is two dimensional matrices of the joint probability p(i,j) of occurrence of a pair of gray values (i,j) separated by a displacement d = (dx,dy,dz).
Formula:
Feature:
52 Displacement vectors:
4 distance * 13 direction = 52
4 Haralick texture features:
energy, entropy, contrast and homogeneity
Feature vector:
208 components (=4 (features) * 52 (matrices)).
13 directions

16. Extraction of Volumetric Textures (2) ------ 3D Wavelet Transform (3D WT)
3D WT provides a spatial and frequency representation of a volumetric image.
Two scale 3D Wavelet Transform:
Feature:
Mean and Standard deviation
Feature vector:
30 components (2 (features) +15 (sub-bands))

17. Extraction of Volumetric Textures (3) ------ 3D Gabor Transform (3D GT)
A set of 3D Gabor filters:
Gabor Transform:
Feature:
144 Gabor filters
4 (F) *6(θ)*6(Φ) =144
Mean and Standard deviation
Feature vector:
288 components (2 (features) +144(filters))

18. Extraction of Volumetric Textures (4) ------ 3D Local Binary Pattern (3D LBP)
Local binary pattern(LBP) is a set of binary code Ci to define texture in a local neighborhood (p,r). A histogram Hi is then generated to calculate the occurrences of different binary patterns.
LBP on three orthogonal planes (LBP-TOP), i.e., XY, XZ, and YZ planes, expressed as
Feature:
59 binary patterns
Feature vector:
177 components (=59(patterns)*3(planes)

19. CBIR for 3D Brain Image ---Methodology(4) Retrieval ---Similarity Measurement
Histogram Intersection(3D LBP)
Normalized Euclidean distance (3D GLCM,3D WT,3D GT)

20. CBIR for 3D Brain Image ---Methodology(5) Lesion Detection
Assume bilateral symmetry of a normal brain along its mid-plane

21. Evaluation ---- Test Dataset
100 MR brain images
Size: 256  256  44
DICOM (Digital Imaging and Communications in Medicine) format
Collected from Neuro-imaging Centre at Beijing General Navy Hospital, China

22. Experimental Results(1) ------ Lesion Detection

23. Experimental Results(2) -------Retrieval
Comparative results demonstrate that LBP outperforms four 3D texture methods in terms of retrieval precision and processing speed.

24. Experimental Results(3) -------Query time
The query time with VOI selection offers 4 times faster operation than that without.

25. 3D Brain Visualization(1)

26. 3D Brain Visualization(2)

27. CBIR for 3D Brain Image------ On-line system(1):

28. CBIR for 3D Brain Image------ On-line system(2):
Server side:
100 3D brain images(DICOM format to JPG format)
3D visual feature extraction(4 methods): Off-line pre-processing using Matlab.
3D visualization: using Matlab
Client side:
Interface based on PHP generate dynamic web pages.

29. PART III Conclusion and Future Work

30. Conclusion for MIRAGE (Middlesex medical Image Repository with a CBIR ArchivinG Environment)
Create Middlesex medical Image repository ( ~ D images and 100 3D brain images)
Create CBIR archiving environment for 2D and 3D medical images.

31. Future Work(1)------ Continue working on 3D Brain Image
Test on the larger dataset and enrich our repository
Research on clinical purpose (EC FP7)
Collaborate with Neuro-imaging Centre at Beijing General Navy Hospital, China.

32. Future Work(2)------ Echocardiogram Video Clip
Enrich our repository
Research for clinical purpose(EC FP7)
Collaborate with First Hospital
of Tsinghua University, China.
B-mode 2D Video Clip
B-mode and M-mode Video clip
UltrasonixTABLET
Ultrasound scanner
Colour Doppler Video Clip

33. Future Work(3) ------- Grid Computing
MDX Grid Machine

34. PART VI Demonstration