2. Prague Institute of Chemical Technology - Department of Computing and Control Engineering Digital Signal & Image Processing Research Group Brunel University, London - Department of Electronics and Computer Engineering Communications & Multimedia Signal Processing Research Group ¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯ ENHANCEMENT OF BIOMEDICAL IMAGES Jiří Ptáček 8 th April 2002 Supervisors:Prof. Aleš Procházka Prof. Saeed Vaseghi

3. Jiri Ptacek, Department of Computing and Control Engineering, Prague Institute of Chemical Technology, DSP Research Group Department of Electronics and Computer Engineering, Brunel University, London, C&MSP Research Group 1. INTRODUCTION ¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯ Image Restoration – Removes or minimizes some known degradations in images – Special kind of Image Enhancement Image Enhancement– The improvement of digital image quality 1.INTRODUCTION Image Processing – Most signals are converted into a form tractable by digital hardware, and can then be treated by Digital Signal Processing (for one-dimensional signals) or by Digital Image Processing (in two dimensions). Image Reconstruction– Completion of missing or corrupted parts (artifacts) of images with unknown model of degradations – Special kind of Image Enhancement

4. Jiri Ptacek, Department of Computing and Control Engineering, Prague Institute of Chemical Technology, DSP Research Group Department of Electronics and Computer Engineering, Brunel University, London, C&MSP Research Group 1. INTRODUCTION 2. RECONSTRUCTION OF MR IMAGES ¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯ Main aims of Magnetic Resonance (MR) Images Enhancement: Reconstruction of missing or corrupted parts of MR Images Image De-noising Image Resolution Enhancement Criteria of Image Reconstruction:  objective –sum of squared errors between pixels of an original image and a reconstructed image (It is necessary to have undamaged image)  subjective– approximate knowledge of the image – aestetical notion (suppression of jamming defects of the image) 2. RECONSTRUCTION OF MR IMAGES Reasons of the Reconstruction: Distortion or damaged parts or whole image (for example caused by lens) Missing parts of an image (shining points in MR images, missing value one or more measuring stations of air pollution in air pollution images etc.)

5. Designed and tested methods of image reconstruction: Bilinear Interpolation – BLI Predictive Image Modelling – PIM Triangular Surface Interpolation – TSI Matrix Moving Average – MMA Used 2D signals: – simulated 2D signal (112x112 pixels) –real MR image (512x452 pixels) reconstructed part: 20 x 20 pixels Jiri Ptacek, Department of Computing and Control Engineering, Prague Institute of Chemical Technology, DSP Research Group Department of Electronics and Computer Engineering, Brunel University, London, C&MSP Research Group 2. RECONSTRUCTION OF MR IMAGES ¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯

6. 2.1 Bilinear Interpolation Linear interpolation applied row by row by the following relation: between known boundary values {z(m,n)} for row m and p=1,2,…,l-k-1 followed by a similar procedure columns by columns Simulated signal (missing part: 20x20 pixels), SSE = 41.61 Real MR Image (missing part: 20x20 pixels), SSE = 0.6704 Jiri Ptacek, Department of Computing and Control Engineering, Prague Institute of Chemical Technology, DSP Research Group Department of Electronics and Computer Engineering, Brunel University, London, C&MSP Research Group 2. RECONSTRUCTION OF MR IMAGES ¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯

7. 2.2 Predictive image modelling using AR model 2.2.1 Prediction of an 1D signal 2.2.1 Prediction of an 1D signal AR model: for i=l, l+1, l+2,...., l+M-1 matrix notation: (1) where The result is possible to write using the Toeplitz matrix A:, let us define the vector Matrix contains known samples as well as estimated samples:  in case that the matrix and the vector in eq. (1) are separated into two matrices and... contains only known values... contains only estimated values  (2) Jiri Ptacek, Department of Computing and Control Engineering, Prague Institute of Chemical Technology, DSP Research Group Department of Electronics and Computer Engineering, Brunel University, London, C&MSP Research Group 2. RECONSTRUCTION OF MR IMAGES ¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯

8. Then from eq. (2) we obtain the system of linear equations for the forward (west) prediction: (3) Similar to the forward prediction we can construct the AR model for the backward (east) prediction using following samples after the part with the missing samples: for i=l+M-1, l+M-2,..., l in matrix notation: Then (4) For obtaining solution of the eqs. (3) and (4) for forward and backward prediction we require: The estimation has to minimize the error of the east-west model:, where and In order to minimize the error, we set the derivative of  equal to zero: (5) Jiri Ptacek, Department of Computing and Control Engineering, Prague Institute of Chemical Technology, DSP Research Group Department of Electronics and Computer Engineering, Brunel University, London, C&MSP Research Group 2. RECONSTRUCTION OF MR IMAGES ¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯

9. After the substitution to eq. (5), calculation, and many treatments we obtain the system of linear equations for the vector of the estimated values:, where Sum of squared errors between the reconstructed and the original samples: Calculation of the vector of the estimated values is applied row by row for the whole missing part of an image  we obtain the matrix This algorithm is then applied for the north–south prediction  we obtain the matrix The result matrix of the interpolated values is given as the arithmetic mean of, 2.2.2 Prediction of an 2D signal 2.2.2 Prediction of an 2D signal Jiri Ptacek, Department of Computing and Control Engineering, Prague Institute of Chemical Technology, DSP Research Group Department of Electronics and Computer Engineering, Brunel University, London, C&MSP Research Group 2. RECONSTRUCTION OF MR IMAGES ¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯

10. Simulated signal – 2D sinwave (missing part: 20x20 pixels), SSE = 0 Real signal – magnetic resonance image (missing part: 20x20 pixels) Part 1 SSE1=0.4547 Order of the AR model: 4 Part 2 SSE2=0.6313 Order of the AR model: 3 Jiri Ptacek, Department of Computing and Control Engineering, Prague Institute of Chemical Technology, DSP Research Group Department of Electronics and Computer Engineering, Brunel University, London, C&MSP Research Group 2. RECONSTRUCTION OF MR IMAGES ¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯

11. 2.3 Triangular Surface Interpolation Design of a linear or another surface through 3 known points Calculation of points’ 3rd coordinate situated on the designed surface Simulated signal – 2D sin (missing part: 20x20 pixels)... (1) Real signal – MR image (missing part: 20x20 pixels)... (2) (1) SSE1=32.5 (2) SSE2=0.68 Jiri Ptacek, Department of Computing and Control Engineering, Prague Institute of Chemical Technology, DSP Research Group Department of Electronics and Computer Engineering, Brunel University, London, C&MSP Research Group 2. RECONSTRUCTION OF MR IMAGES ¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯

12. 2.4 Matrix Moving Average Calculation of the pixel value in the missing part as arithmetic mean of 5 neighbouring pixels by the relation: The algorithm is applied row by row for the whole missing part Simulated signal (missing part: 20x20 pixels), SSE = 24.14 Real signal (missing part: 20x20 pixels), SSE = 0.4844 Jiri Ptacek, Department of Computing and Control Engineering, Prague Institute of Chemical Technology, DSP Research Group Department of Electronics and Computer Engineering, Brunel University, London, C&MSP Research Group 2. RECONSTRUCTION OF MR IMAGES ¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯

13. 2.5 Conclusion Comparison of the designed methods using the aesthetical notion (subjective valuation) and the value of SSE between the original and the reconstructed part 1. BLI2. PIM3. TSI4. MMA Simulated signal41.61032.5224.14 Part 1 (MR Image)0.46760.45470.56100.3518 Part 2 (MR Image)0.67040.63130.68480.4844 Sum of squared errors SSE for each method applied to the simulated signal and 2 various missing areas of the MR image:  MMA – low SSE, not edge sensitive, the structure in reconstructed part is erased  PIM – more difficult, more edge sensitive, it saves structure and period. components 2.6 The next development in Image Reconstruction – visions Development of nonlinear methods using wavelet transform Utilize of the Bayesian models 3D interpolation in models of a human brain, purpose: acquisition of the 3D model out of a finite number of 2D horizontal brain MR scans Jiri Ptacek, Department of Computing and Control Engineering, Prague Institute of Chemical Technology, DSP Research Group Department of Electronics and Computer Engineering, Brunel University, London, C&MSP Research Group 2. RECONSTRUCTION OF MR IMAGES ¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯

14. 2.7 Interests Shielding of the whole coin Comparison of the Matrix Moving Average method (left picture) and the Predictive Image Modelling using the AR Model (right picture) Jiri Ptacek, Department of Computing and Control Engineering, Prague Institute of Chemical Technology, DSP Research Group Department of Electronics and Computer Engineering, Brunel University, London, C&MSP Research Group 2. RECONSTRUCTION OF MR IMAGES ¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯

15. Jiri Ptacek, Department of Computing and Control Engineering, Prague Institute of Chemical Technology, DSP Research Group Department of Electronics and Computer Engineering, Brunel University, London, C&MSP Research Group 3. IMAGE DE-NOISING ¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯ Signal de-noising using wavelet transform: 1.Signal decomposition using a selected wavelet function up to the given level and evaluation of wavelet transform coefficients 2.The choice of threshold limits for each decomposition level and modification of its coefficients 3.Signal reconstruction from modified wavelet transform coefficients

16. Jiri Ptacek, Department of Computing and Control Engineering, Prague Institute of Chemical Technology, DSP Research Group Department of Electronics and Computer Engineering, Brunel University, London, C&MSP Research Group 3. IMAGE DE-NOISING ¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯ Image de-noising using wavelet transform: Utilize the same principles as for signal decomposition and de-noising. Each column of an image matrix is convolved with high-pass and low-pass filter followed by downsampling. The same process is applied to image matrix rows. The choice of threshold limits  for each decomposition level and modification of its coefficients for k=0, 1, … N-1 Backward image reconstruction out of modified wavelet transform coefficients

17. Jiri Ptacek, Department of Computing and Control Engineering, Prague Institute of Chemical Technology, DSP Research Group Department of Electronics and Computer Engineering, Brunel University, London, C&MSP Research Group 4. IMAGE RESOLUTION ENHANCEMENT ¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯ Using interpolation property of Fourier transform and zero padded method

18. Jiri Ptacek, Department of Computing and Control Engineering, Prague Institute of Chemical Technology, DSP Research Group Department of Electronics and Computer Engineering, Brunel University, London, C&MSP Research Group 5. FOLLOWING WORK 6. REFERENCES ¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯ 5. FOLLOWING WORK Bayesian methods in image reconstruction Utilize the probabilistic models after wavelet decomposition Image resolution enhancement using wavelet filters Image resolution enhancement (interpolation) and edge detection 6. REFERENCES D. E. Newland : An Introduction to Random Vibrations, Spectral and Wavelet Analysis, Longman Scientific & Technical, Essex, U.K., third edition, 1994 G. Strang : Wavelets and Dilation Equations: A brief introduction, SIAM Review, 31(4):614-627, December 1998 G. Strang and T. Nguyen : Wavelets and Filter Banks, Wellesley-Cambridge Press, 1996 ELECTRONIC SOURCES: IEEE : http://www.ieee.orghttp://www.ieee.org WAVELET DIGEST : http://www.wavelet.orghttp://www.wavelet.org DSP PUBLICATIONS : http://www.dsp.rice.edu/publicationshttp://www.dsp.rice.edu/publications

19. THANK YOU FOR YOURATTENTION THANK YOU FOR YOUR ATTENTION