Towards a Video Camera Network for Early Pest Detection in Greenhouses Vincent Martin1, Sabine Moisan1 Bruno Paris2, Olivier Nicolas2 1. I N R I A Sophia Antipolis Méditerranée, Pulsar project-team, France 2.CREAT, Chambre d'Agriculture des Alpes Maritimes, France

Motivations Temperature and hygrometric conditions inside a greenhouse favor frequent and rapid attacks of bioagressors (insects, spider mites, fungi). Difficult to know starting time and location of such attacks. Need to automatically identify and count populations to allow rapid decisions Help workers in charge of greenhouse biological monitoring Improve and cumulate knowledge of greenhouse attack history Control populations after beneficial releases or chemical applications Collaborative Research Initiative BioSerre between INRIA, INRA, and Chambre d’Agriculture des Alpes Maritimes

Objectives Help producers to take protection decisions Context: Integrated Pest Management Early pest detection to reduce pesticide use Approach: Automatic vision system for in situ, non invasive, and early detection based on a video sensor network using video processing and understanding, machine learning, and a priori knowledge Help producers to take protection decisions White fly photo : Inra (Brun) Aphid photo: Inra (Brun)

DIViNe1: A Decision Support System 1Detection of Insects by a Video Network Identification and counting of pests Manual method DIViNe system Result delivery Up to 2 days Near real-time Advantages Discrimination capacity Autonomous system, temporal sampling, cost Disadvantages Need of a specialized operator (taxonomist); precision vs time Predefined insect types; video camera installation Manpower cost, cheap camera

First DIViNe Prototype Network of 5 wireless video cameras (protected against water projection and direct sun). In a 130 m2 greenhouse at CREAT planted with 3 varieties of roses. Observing sticky traps continuously during daylight. High image resolution (1600x1200 pixels) at up to 10 frames per second. Automatic data acquisition scheduled from distant computers

Processing Chain Intelligent Acquisition Detection Classification Current work Future work Detection Image sequences with moving objects Classification Pest counting results Regions of interest Tracking Pest identification Behaviour Recognition Pest trajectories Scenarios (laying, predation…)

Preliminary Results video clip Acquisition: sticky trap zoom Detection: regions of interest in white by background subraction Classification: regions labeled according to insect types based on visual features

Conclusion and Future Work A greenhouse equipped with video cameras A software prototype: Intelligent image acquisition Pest detection (few species) Future: Detect more species Observe directly on plant organs (e.g. spider mites) Behaviour recognition Integrated biological sensor See http://www-sop.inria.fr/pulsar/projects/bioserre/

Laying scenario example state: insideZone( Insect, Zone ) event: exitZone( Insect, Zone ) state: rotating( Insect ) scenario: WhiteflyPivoting( Insect whitefly, Zone z ) { A: insideZone( whitefly, z ) // B: rotating( whitefly ); constraints: duration( A ) > duration( B ); } scenario: EggAppearing( Insect whitefly, Insect egg, Zone z ) { insideZone( whitefly, z ) then insideZone( egg, z ); main scenario: Laying( Insect whitefly, Insect egg, Zone z ) { WhiteflyPivoting( whitefly, z ) // loop EggAppearing( egg, z ) until exitZone( whitefly, z ); then send(”Whitefly is laying in ” + z.name);

Add on Expert knowledge of white flies: choose features for detection and classification An ontology for the description of visual appearance of objects in images based on: Pixel colours Region texture Geometry (shape, size,…) Adaptive techniques to deal with illumination changes, moving background by means of machine learning