Spectral Weed Detection and Precise Spraying Laboratory of AgroMachinery and Processing Els Vrindts, Dimitrios Moshou, Jan Reumers Herman Ramon, Josse De Baerdemaeker Katholieke Universiteit Research sponsored by IWT and the Belgian Ministry of Small Trade and Agriculture

Katholieke Universiteit Leuven Overview Spectral measurements of crops and weeds in laboratory in field Processing of spectral data with neural networks Precise spraying

Katholieke Universiteit Leuven Optical detection of weeds Techniques red/NIR detectors (vegetation index) image processing (color, texture, shape) remote sensing of weed patches reflection in visible & NIR light different detection possibilities, different scales Requirements for on-line weed detection: fast & accurate weed detection synchronized with treatment

Katholieke Universiteit Leuven Spectral weed detection Factors affecting spectral plant signals leaf reflection, dependent on species and environment, stress, disease canopy & measurement geometry light conditions detector sensitivity

Katholieke Universiteit Leuven Spectral analysis of plant leaves in laboratory sample spectrophotometer integrating sphere computer Diffuse Reflectance Spectroscopy of Crop and Weed Leaves Laboratory measurements

Katholieke Universiteit Leuven Diffuse Reflectance of a Leaf Laboratory measurements

Katholieke Universiteit Leuven Spectral Dataset Laboratory measurements

Katholieke Universiteit Leuven Reflectance of crop and weed leaves Laboratory measurements

Katholieke Universiteit Leuven Spectral analysis stepwise selection of discriminant wavelengths multivariate discriminant analysis, based on reflectance response at selected wavelengths (dataset a) assuming multivariate normal distribution quadratic discriminant rule classes with different covariance structure testing the discriminant function: classification of spectra from dataset b Laboratory measurements

Katholieke Universiteit Leuven Spectral response of beet & weeds Laboratory measurements

Katholieke Universiteit Leuven Laboratory measurements Spectral response of maize & weeds

Katholieke Universiteit Leuven Spectral response of potato & weeds Laboratory measurements

Katholieke Universiteit Leuven Classification results Laboratory measurements

Field measurement of crop and weeds Variation in light condition Measurement geometry Detector sensitivity Processing method Signal path Field measurements

Katholieke Universiteit Leuven Equipment for field measurement spectrograph + 10-bit CCD, digital camera, computer, 12 V battery and transformer on mobile platform Field measurements

Katholieke Universiteit Leuven Equipment - Spectrograph both spatial and spectral information in images Field measurements

Katholieke Universiteit Leuven Image data maize, sugarbeet, 11 weeds 2 different days, different light conditions 755 x 484 pixels spatial axis spectral axis Field measurements

Katholieke Universiteit Leuven Spectral response of sensor Field measurements

Katholieke Universiteit Leuven Data processing spectral resolution: 0.71 nm /pixel plant/soil discrimination with ratio: NIR (745 nm) / red (682 nm) data reduction by calculating average per 2.1 nm, removing noisy ends resulting spectra: nm range, 2.1 nm step independent datasets of maize, sugarbeet and weeds Field measurements

Katholieke Universiteit Leuven Spectral datasets Field measurements

Katholieke Universiteit Leuven Mean canopy reflections Field measurements

Canonical analysis of Sugarbeet - weeds Field measurements

Canonical analysis of Maize - weeds Field measurements

Katholieke Universiteit Leuven Discriminant analysis Sugarbeet Field measurements

Katholieke Universiteit Leuven Discriminant analysis Maize Field measurements

Graphic comparison datasets Field measurements

Graphic comparison datasets Field measurements

Katholieke Universiteit Leuven Graphic comparison datasets Field measurements

Discriminant analysis ratios Sugarbeet Field measurements

Discriminant analysis ratios Maize Field measurements

Katholieke Universiteit Leuven Results only spectral info ( nm) classification based on narrow bands in discriminant functions good results in similar light and crop conditions large decrease in performance for other light conditions using ratios of narrow bands improvement, but not sufficient Field measurements

Katholieke Universiteit Leuven Improving results influence of light conditions adaption of classification rule determining light condition and applying appropriate calibration/LUT spectral inputs that are less affected by environment measuring irradiance, calculating reflectance other classification methods Field measurements

Katholieke Universiteit Leuven Neural network for classification Comparison of different NN techniques for classification Self-Organizing Map (SOM) neural network for classification used in a supervised way for classification neurons of the SOM are associated with local models achieves fast convergence and good generalisation. Crop-weed classification

Neural lattice (A) Input Space (V) SOMMLP class s3(k) first hidden layer s2(k)s4(k)s1(k) …. second hidden layer weights PNN ADVANTAGES Learns with reduced amounts of data Fast Learning Visualisation Retrainable DISADVANTAGES Discrete output ADVANTAGES Good extrapolation DISADVANTAGES Slow Learning Local minima Needs a lot of data ADVANTAGES Fast Learning Retrainable DISADVANTAGES Needs all training data during operation Needs a lot of data Crop-weed classification Neural network for classification

Crop-weed classification Comparison between methods MLP: Multi-Layer Perceptron, PNN: Probabilistic N Network, SOM: Self- Organizing Map, LVQ: Learning Vector Qantization, LLM: Local Linear Mapping Moshou et al., 1998, AgEng98, Oslo Moshou et al., 2001, Computers and Electronics in Agriculture 31 (1): 5-16

Katholieke Universiteit Leuven MLP: Multi-Layer Perceptron, PNN: Probabilistic N Network, SOM: Self- Organizing Map, LVQ: Learning Vector Qantization, LLM: Local Linear Mapping Comparison between methods Crop-weed classification

Katholieke Universiteit Leuven Crop-weed classification MLP: Multi-Layer Perceptron, PNN: Probabilistic N Network, SOM: Self- Organizing Map, LVQ: Learning Vector Qantization, LLM: Local Linear Mapping Comparison between methods

Katholieke Universiteit Leuven The strongest point is the local representation of the data accompanied by a local updating algorithm Local updating algorithms assure much faster convergence than global updating algorithms (e.g. backpropagation for MLPs) Because of the topologically preserving character of the SOM, the proposed classification method can deal with missing or noisy data, outperforming “optimal” classifiers (PNN) The proposed method has been tested and gave superior results compared to a variety statistical and neural classifiers Crop-weed classification Conclusions on LLM SOM technique

Precision spraying through controlled dose application Unwanted variations in dose caused by horizontal and vertical boom movements Precision treatment

Katholieke Universiteit Leuven Active horizontal stabilisation of spray boom Validation with ISO 5008 track movement of spray boom tip with and without controller Time (s) Distance (m) Precision treatment

Katholieke Universiteit Leuven Vertical stabilisation of spray boom Slow-active system for slopes Resulting boom movement Precision treatment

Katholieke Universiteit Leuven On-line selective weed treatment Indoor test of on-line weed detection and treatment Precision treatment

Katholieke Universiteit Leuven Sensor: Spectral line camera Classification: Probabilistic neural network Program in Labview with c-code Image acquisition frequence: 10 images/sec, travel speed: 30cm/sec, segmentation with NDVI ( > 0.3) Off-line training of NN, On-line classification Decision to spray: > 20 weed pixels and > 35% of vegetation is weed Spray boom with PWM nozzles and controller, provided by Teejet Technologies Indoor test of on-line weed detection and treatment Precision treatment

Indoor test of on-line weed detection and treatment Color image and spectral image Precision treatment

Katholieke Universiteit Leuven Indoor test - Results Comparison of nozzle activation with weed positions Precision treatment

Katholieke Universiteit Leuven Indoor test - Results camera nozzle weed Experimental set up - separate weed classes (4) did not improve crop-weed classification -Correct detection of nearly all weeds - Only 6 % redundant spraying of crop - Up to 70 % reduction of herbicide use Precision treatment