1. Orion Image Understanding for Object Recognition Monique Thonnat INRIA Sophia Antipolis

2. 2 Orion research project A multi-disciplinary team at the frontier of computer vision, knowledge-based systems and software engineering Research on Automatic Image Understanding Video Interpretation : subway and bank monitoring, airport activity monitoring... Natural Object Recognition : galaxy classification, pollen categorization...

3. 3 INRA/INRIA Cooperation Main goals: Automatic recognition of biological organisms in their natural environment In particular early rose disease diagnosis by image understanding for Integrated Pest Management Partners: INRIA Orion Team Monique Thonnat and Celine Hudelot (PhD) INRA Sophia Antipolis, URIH Paul Boissard

4. 4 INRA/INRIA Cooperation Why Image Understanding ? Plant disease diagnosis = visual observation which aims at inferring disease presence by the observation of signs and symptoms TO BE ABLE TO REASON : signs and symptoms interpretation in terms of diseases TO BE ABLE TO SEE : Focusing on relevant criteria Star shape network of white and thin filaments (5-10 μ) Presence of elliptical white blobs in the centre of the network Climatic Context: High humidity, Temperature : 25 °C Early powdery mildew infection in propitious conditions Early diagnosis: Microscopic image (x64) of rose leaf part

5. 5 Natural Object Recognition : main issues Powdery mildew : State of infection : early Vegetal support : red leaf Powdery mildew : State of infection : very early Vegetal support : green leaf Two white flies close to their eggs Need of domain knowledge Intelligent management of image processing programs Complexity and variability of object appearance Variability of contexts Scene knowledge and spatial reasoning Multiple objects and various object types

6. 6 Natural Object Recognition : proposed solution A platform with 3 dedicated tasks Semantic data interpretation pathologist knowledge (domain taxonomy and terminology) Ontological engineering to facilitate knowledge acquisition Data management Matching between numerical image data and symbols Scene analysis using spatial reasoning Image processing numerical object description program supervision techniques : to automate the management of an image processing library

7. 7 Input : User Request Fungi infection? Image + Context Variety : Leonidas Leaf : young Season: summer Temp: 24° C Humidity: 80... Example Interpretation Domain concept tree traversal to build visual object hypotheses Leaf Scene VegetalPart Disease Insects Virus Fungi White fly Penicillium Powdery mildew Dispersed Clump Aphid Subpart Subclass Acarid Very Early Pellet 1 Data Management Symbolic request to image processing request 3 Goal: segmentation Contraints:Image entity = ridge Object.width = [1..3] Object.intensity > 150 Input Data: Image : input image Mask : area of interest Image Processing Request 4 Image Processing: request processing by program supervision techniques 5 Image Data Ridge 1 numerical descriptors Ridge 3 + Numerical descriptors Ridge 2 + Numerical Descriptors 6 Interpretation Domain concept tree traversal to build visual object hypotheses Visual Object Hypothesis 2 Group of : Geometry: star shape network of { Geometry: line Thickness : thin width [7..10  m] very thin width [5..7  m] Straightness : almost straight Lightness: bright} Spatial Relation: Connected} 1

8. 8 Example Image Data Ridge 1 numerical descriptors Ridge 3 + Numerical descriptors Ridge 2 + Numerical Descriptors Data Management Visual object hypothesis verification and instantiation 7 Interpretation : classification Matching between visual object instances and domain concepts 9 Interpretation : diagnosis Post classification rules activation 11 Visual Object Instance Network of lines Line 1 Line 2 Line 3 Line 5Line 4 EC Line line1 Thickness:=thin (0.8) Straightness:= straight (0.5) Lightness:=bright (0.7) Connected (line2) Connected (line4) + link to image data 8 Diagnosis Early powdery mildew infection on young leaf 12 Recognised domain concept 10 Freely dispersed mycelium

9. 9 Conclusion A generic platform for automatic recognition of natural objects a formalism and an ontology for knowledge base building 3 dedicated reusable engines semantic interpretation image/symbol matching and spatial reasoning management of a generic image processing library Evaluation and validation (on going) with microscopic images of greenhouse rose leaf diseases Future works machine learning for image/symbol matching (Nicolas Maillot) and for image segmentation (Vincent Martin) biological long term objective : continuous disease monitoring in greenhouse

10. 10 Object Recognition Platform Visual Concept Ontology Image Data Ontology Visual Object Instances Image Processing Data Management Data Management and Scene Analysis Knowledge Base Data Management Engine Interpretation Application Domain Knowledge Base Interpretation Engine Program Utilization Knowledge Base Library of Image Processing Programs Program Supervision Engine Visual Object Hypotheses Image Processing Requests Image Numerical data User Request Interpretation results