Chair for Computer Aided Medical Procedures & Augmented Reality Department of Computer Science | Technische Universität München Chair for Computer Aided Medical Procedures & Augmented Reality | wwwnavab.cs.tum.edu Automatic Feature Generation for Endoscopic Image Classification Ulrich Klank Nicolas Padoy Prof. Nassir Navab Supervisor: Advisor: 18 January 2007
CAMP | Department of Computer Science | Technische Universität München | 12 August Chair for Computer Aided Medical Procedures & Augmented Reality | wwwnavab.cs.tum.edu Overview Endoscopic images Differences Similarities Image feature generation using Genetic Programming A low level approach A high level approach An example
CAMP | Department of Computer Science | Technische Universität München | 12 August Chair for Computer Aided Medical Procedures & Augmented Reality | wwwnavab.cs.tum.edu Endoscopic Images of two Surgical Phases: OP3 Images from the cutting and clipping phase (OP3) Images from the bag retraction phase (OP3)
CAMP | Department of Computer Science | Technische Universität München | 12 August Chair for Computer Aided Medical Procedures & Augmented Reality | wwwnavab.cs.tum.edu …: OP1 Images from the bag retraction phase (OP1)Images from the cutting and clipping phase (OP1)
CAMP | Department of Computer Science | Technische Universität München | 12 August Chair for Computer Aided Medical Procedures & Augmented Reality | wwwnavab.cs.tum.edu Low-level Approach - Short Reminder Genetic Programming: combination of low-level operators PIXEL (With Parameters) PUSH… FOR (With Parameters) MUL LOAD (With Parameters) ADD Mutation Evaluation Code
CAMP | Department of Computer Science | Technische Universität München | 12 August Chair for Computer Aided Medical Procedures & Augmented Reality | wwwnavab.cs.tum.edu Low-level Approach - Results Distributed evaluation of programs on several computers (up to 7) Nearly programs evaluated (~300 generations) First results: Characteristics of the best programs: returning a short vector in a short time Classification rate with a linear classifier is 62% (64 images of 2 phases of 4 videos)
CAMP | Department of Computer Science | Technische Universität München | 12 August Chair for Computer Aided Medical Procedures & Augmented Reality | wwwnavab.cs.tum.edu Problems with the low-level Approach ~40% of the programs contain major errors like Infinite running time, stack overflow No reference to the input image Resulting programs still has structural similarity to the initial program. More generations needed Evaluation of a programs is very slow due to the simulation of basic instructions on images How to improve this method?
CAMP | Department of Computer Science | Technische Universität München | 12 August Chair for Computer Aided Medical Procedures & Augmented Reality | wwwnavab.cs.tum.edu Existing Software: GENIE GENIE Software published by: Los Alamos National Laboratories First publication ’97, Commercial version in development Genetic Programming for segmentation of images Application example: Segmentation of Medical Images Using a Genetic Algorithm by Payel Ghosh, Melanie Mitchell (’06)
CAMP | Department of Computer Science | Technische Universität München | 12 August Chair for Computer Aided Medical Procedures & Augmented Reality | wwwnavab.cs.tum.edu The Step to a higher level approach Erode (With Parameters) Histogram (With Parameters) MinLocOriginal CannyEdge (With Parameters) Gradient x (With Parameters) Dilate (With Parameters) PIXEL (With Parameters) PUSH… FOR (With Parameters) MUL LOAD (With Parameters) ADD Replace the basic commands in a program by higher level operators: Examples for low-level operator : Examples for high-level operator :
CAMP | Department of Computer Science | Technische Universität München | 12 August Chair for Computer Aided Medical Procedures & Augmented Reality | wwwnavab.cs.tum.edu Evaluation of a Program Semantic checks Input referred? No infinite loops? Execution with several inputs 16 images per phase 2 phases per video at the moment 4 videos used for evaluation A fitness function with 2 components: A classification of the phases by the output vectors The average execution time per input
CAMP | Department of Computer Science | Technische Universität München | 12 August Chair for Computer Aided Medical Procedures & Augmented Reality | wwwnavab.cs.tum.edu Genetic Concept – Cross Over Erode (With Parameters) Histogram (With Parameters) MinLocOriginal CannyEdge (With Parameters) Gradient x (With Parameters) Dilate (With Parameters) Gauss (With Parameters) OriginalMaxOriginal PushImage (With Parameters) Gradient y (With Parameters) Histogram (With Parameters) Program 1 Program 2 New Program
CAMP | Department of Computer Science | Technische Universität München | 12 August Chair for Computer Aided Medical Procedures & Augmented Reality | wwwnavab.cs.tum.edu High-level Approach Benefits Faster evaluation Reduced number of commands Optimized basic image operations (OpenCV) Resulting programs easier to understand
CAMP | Department of Computer Science | Technische Universität München | 12 August Chair for Computer Aided Medical Procedures & Augmented Reality | wwwnavab.cs.tum.edu Current Results: Running time: 480 ms (in simulation) Output length: a vector of 48 signed integer Classification rate: 67% Rate based on 512 testing images out of 4 videos and 2 phases Number of generations needed: ~80
CAMP | Department of Computer Science | Technische Universität München | 12 August Chair for Computer Aided Medical Procedures & Augmented Reality | wwwnavab.cs.tum.edu Future Work Completion of the available image operators Extension to multi-phases classification Comparison of the fitness function with a standard classifier Comparison with several standard features Features evaluation within the workflow segmentation system
CAMP | Department of Computer Science | Technische Universität München | 12 August Chair for Computer Aided Medical Procedures & Augmented Reality | wwwnavab.cs.tum.edu Thank you for your attention!