1. 29/03/2001Task 3: Semi-Automatic System for Pollen Recognition 1 Partners: –REA (Barcelona) –REA (Cordoba) –LASMEA (Clermont-Ferrand) –INRIA (Sophia-Antipolis)

2. 29/03/2001Task 3: Semi-Automatic System for Pollen Recognition 2 Plan 1) Recognition system integration (WP5330) Global measures computed Flowering information Specific characteristics (cytoplasm of Cupressaceae) Partial integration 2) System Validation (WP6300)

3. 29/03/2001Task 3: Semi-Automatic System for Pollen Recognition 3 Steps for recognition Two steps for pollen recognition 1- Compute global measures on the grain 2- Search specific characteristics Integration of all components –almost finished for the first step –in progress for the second step

4. 29/03/2001Task 3: Semi-Automatic System for Pollen Recognition 4 Compute Global Measures Size, colour (RGB), shape, convexity Flowering period (if given) These measures –give the first estimations about the type –help to select which characteristics to search Output: sorted list of possible types Examples: –Grain(Olea) : Olea (86%), Salix (35%), Quercus (34%) –Grain(Populus) : Populus (58%), Cupressaceae (47%), Plantago (46%) –Grain(Brassicaceae) : Salix (54%), Brassicaceae (41%), Plantago (27%) –Grain(Celtis) : Cupressaceae (70%), Plantago (53%), Platanus (51%)

5. 29/03/2001Task 3: Semi-Automatic System for Pollen Recognition 5 Compute Global Measures Methodology: –Segmentation of the central image –Image measures done on this image –Covariance matrices computed for each type –Classification done using Mahalanobis distance –For each grain, a list of possible types is built Improvement (in progress): –Principal component analysis on the features

6. 29/03/2001Task 3: Semi-Automatic System for Pollen Recognition 6 Compute Global Measures Intermediate results on reference images: –On the four ASTHMA types: 99 % –On 30 different types: 66 % –Separation in two classes: Cupressaceae and Brassicaceae –Some false positives for Cupressaceae and Poaceae Intermediate results on aerobiological images: –On the four ASTHMA types: N.A. –On different types:  30% Need more information for classification –for redundancies –to work on other types (other cities)

7. 29/03/2001Task 3: Semi-Automatic System for Pollen Recognition 7 Compute Global Measures Similar types using colour and size for classification: –Cupressaceae: Plantago, Platanus, Populus –Olea: Quercus, Salix, Alnus –Parietaria: Plantago, Cupressaceae, Brassicaceae (??) –Poaceae: none Types with big variances (with lot of false positives): –Coriaria, Plantago, Ambrosia, Quercus, Alnus, Brassicaceae –Solutions: Separate in different classes some types More accurate classification: discrete component analysis

8. 29/03/2001Task 3: Semi-Automatic System for Pollen Recognition 8 Flowering information

9. 29/03/2001Task 3: Semi-Automatic System for Pollen Recognition 9 Flowering information Flowering information used –mean weekly pollen concentration –tested with data of Manresa (Barcelona pilot site) Classification methodology –Input: sampling date (week) –Two possible functions: Probability(type) = Concentration (type,week)  type Concentration (type, week) Probability(type) = Concentration (type,week)  week Concentration (type, week)

10. 29/03/2001Task 3: Semi-Automatic System for Pollen Recognition 10 Classification using only flowering information Random test with 1000 samples per class Probability(type) = Concentration (type,week)  type Concentration (type, week) Flowering information Results:

11. 29/03/2001Task 3: Semi-Automatic System for Pollen Recognition 11 Classification using only flowering information Random test with 1000 samples per class Probability(type) = Concentration (type,week)  week Concentration (type, week) Flowering information Results:

12. 29/03/2001Task 3: Semi-Automatic System for Pollen Recognition 12 Flowering information Problem –Can increase the number of false positives –Examples (Manresa): Cupressaceae, Quercus, Pinus, Urticaceae (Parietaria), Chenopodiaceae-Amarathaceae Solutions –Give a smaller weight in the global evaluation –Different evaluation functions for the flowering period –Can help at the end to discriminate similar pollen types –Confused types are not the same than with image processing

13. 29/03/2001Task 3: Semi-Automatic System for Pollen Recognition 13 Specific Characteristics Second step of recognition Look for cytoplasm, reticulum, pores, … Search done for the most probable types Search uses 3D information (  10 images) Two steps: –Segmentation of several chosen 2D images –Validation of results in 3D on all segmented images

14. 29/03/2001Task 3: Semi-Automatic System for Pollen Recognition 14 Specific Characteristics (estimation on reference grains) Cupressaceae Characteristics: Cytoplasm Granules  Intine  Broken grains Global recognition Parietaria Characteristics: Pores Exine Global recognition Poaceae Characteristics: Pores Cytoplasm Intine  Global recognition Olea Characteristics: Reticulum Colpi  Exine Global recognition Ok  Maybe Difficult / Don't know  Impossible

15. 29/03/2001Task 3: Semi-Automatic System for Pollen Recognition 15 Specific Characteristics (cytoplasm of Cupressaceae) Cytoplasm is in the center of the pollen grains Easily visible for the Cupressaceae type No precise shape to look for Methodology to detect it: –Look for bright regions in images above the center –Look for dark regions in images below the center –Compare bright and dark regions (overlapping)

16. 29/03/2001Task 3: Semi-Automatic System for Pollen Recognition 16 Specific Characteristics (cytoplasm of Cupressaceae) Above central imageBelow central image Sum of bright regions Sum of dark regions

17. 29/03/2001Task 3: Semi-Automatic System for Pollen Recognition 17 Specific Characteristics (cytoplasm of Cupressaceae) Validation use the same tools than 1st step measures –covariance matrices on selected criteria For cytoplasm, 4 criteria are used –shape, colour, size and overlapping are used Classification results using only the cytoplasm detection on reference images –7 / 12 Cupressaceae grains with cytoplasm detected (58%) –5 false positives on more than 350 grains tested Similar types : Poaceae, Salix and Parietaria –different list than with measures of 1st step

18. 29/03/2001Task 3: Semi-Automatic System for Pollen Recognition 18 Partial Integration Simple test of classification on reference images with several criteria –40% global measures –30% cytoplasm detection –15% flowering information (relative on week) –15% flowering information (relative on type) Results: –On the four ASTHMA types: 97 % –On 30 different types: 73 % –Overall number of false positives has decreased No results so far on aerobiological images

19. 29/03/2001Task 3: Semi-Automatic System for Pollen Recognition 19 Partial Integration Results can be improved (on reference images) –better combination than just a weighed function –refinement of the criteria Redundancy is necessary –to improve recognition of various grains –to work with aerobiological images Several methods are combined –Each of them give a sorted list of possible types –Similar types are different between methods

20. 29/03/2001Task 3: Semi-Automatic System for Pollen Recognition 20 Next: Aerobiological Images –Good classification on reference images does not imply a good classification on aerobiological images –To do: Clean dust, pollution and bubbles from the pollen masks Work with partial pollen grain (replace dust with empty spaces)

21. 29/03/2001Task 3: Semi-Automatic System for Pollen Recognition 21 System Validation: evaluate the quality of the developed pollen recognition system. WP6300 Semi-automatic system for pollen recognition: validation phase Responsible: REA Partners: INRIA, LASMEA Start: T30 Finish: T36 The modules of the system will be validated separately Acquisition module (LASMEA). Recognition module (INRIA).

22. 29/03/2001Task 3: Semi-Automatic System for Pollen Recognition 22 System Validation (WP6300) Steps for validation of the acquisition module (LASMEA) 1 st step: Pollen slides to test the detection and localization of the main pollen types of ASTHMA. Started at the end of February in Clermont. 2 nd step: Aerobiological slides to test the detection and localization in real conditions. Projected to do in April. In both cases the sequences will be used to validate the recognition module (INRIA)

23. 29/03/2001Task 3: Semi-Automatic System for Pollen Recognition 23 Pollen Type Poaceae Cupressaceae Parietaria Olea Total pollen analysed 273 325 89 - Number of pollen located 259 291 89 - Nº pollen not located 14 34 0 - Percentage of location 94,9 % 89,5 % 100 % 0% Validation of the image acquisition module (LASMEA): Results of the 1 st step: to test the detection and localization of the main pollen types of ASTHMA.

24. 29/03/2001Task 3: Semi-Automatic System for Pollen Recognition 24 Pollen type digitised Nº Slide Pollen type Nº sequences Poaceae R121 Poaceae 20 Cupressaceae R123 Cupressaceae 25 Parietaria R135 Parietaria 20 Olea R133 Olea 21 Populus R127 Populus 5 Broussonetia R129 Mixture 4 Fraxinus R129 Mixture 6 Phillyrea R129 Mixture 6 Pinus R129 Mixture 1 Morus R129 Mixture 6 Brassicaceae R129 Mixture 5 Ligustrum R129 Mixture 5 Urtica membranaceae R129 Mixture 6 Salix R118 Salix 5 Celtis R77 Celtis 3 Coriaria R81 Coriaria 5 Quercus R73 Quercus 3 Platanus R63 Platanus 3 TOTAL 149 Total number of sequences to validate the recognition module (INRIA)

25. 29/03/2001Task 3: Semi-Automatic System for Pollen Recognition 25  The detection and localisation of Poaceae, Cupressaceae and Parietaria by the LASMEA system give us excellent results (near to 95%).  The localisation is specially accurate in the small types like Parietaria.  The fault in the localisation is due to pollen grouped in most of the cases,. Pollen grouped is a normal effect in pollen slides but it is not frequent in aerobiological slides.  At present, the system can not detect and localise the Olea pollen grains. The reason of this problem could be the different coloration of the Olea pollen type. As possible solutions to improve the detection we are considering: The use of a blue filter to minimize the yellow effect in the microscopy lamp. Some adjustments in the detection and localisation parameters. To use other magnification that minimise this problem. E.g. 10x although other problems can be encountered. Conclusions