1. Botany Image Retrieval Haibin Ling University of Maryland, College Park

2. Content 1.Problems 2.Related works 3.Our Experiments 4.Summary: difficulties and future works

3. Problems  Reduce botany image retrieval to leaf image retrieval  One-One classification relatively easy  One-Many classification More difficult Segmentation and/or detection Query Image Prototype database Query Query Image An example from specimen database Query

4. Works at other groups  Related works and publications Linkoping Univ & Swedish Museum of Natural History : Classification of Leaves from Swedish Trees. Linkoping Univ & Swedish Museum of Natural History: Classification of Leaves from Swedish Trees.  A master thesis by Oskar J O S ö derkvist Oregon State Univ: Image Retrieval from Plant Database  A master thesis by Ashit Gandhi (supervised by Lead by Thomas Dietterich) National Institute for Agricultural Botany (UK) : Chrysanthemum Leaf Classification National Institute for Agricultural Botany (UK) : Chrysanthemum Leaf Classification  A conference paper by Abbasi, Mokhtarian and Kittler  Summary All the works are contour based Their result are at beginning stages

5. Classification of Leaves from Swedish Trees (Linkoping Univ.)  15 tree classes, 50 test images per class  Simple ANN + nine features (area, eccentricity, moment, etc.)  Average correct ratio is 82%  Some classes have rather low correct ratios: ulmus carpinifolia 52% sorbus hybrida 36% from www.isy.liu.se/cvl/Projects/Sweleaf/samplepage.htmlwww.isy.liu.se/cvl/Projects/Sweleaf/samplepage.html

6. Content-Based Image Retrieval from Plant Database (OSU)  Six species, two kinds of data: isolated leaves & herbarium specimen  Method: match contours by dynamic programming  Good for isolated leaves, average correct ratio: 96.8%  Not so good for herbarium leaves, either as query image (correct ratio 59%) or as templates (61%). Image from http://web.engr.oregonstate.edu/~tgd/leaves/dataset.htmhttp://web.engr.oregonstate.edu/~tgd/leaves/dataset.htm

7. Chrysanthemum Leaf Classification - National Institute for Agricultural Botany (UK)  Farzin Mokhtarian et al.  400 leaves from 40 varieties.  Method: curvature scale space contours + eccentricity, circularity and aspect ratio  Result: the correct ratio (among the top 5 choices) is over 95%

8. Experiments at UMD (1)  D. Jacobs, H. Ling, I.J. Chu, started on March, 2003  http://www.cs.umd.edu/~hbling/Research/Botany/bota ny.htm http://www.cs.umd.edu/~hbling/Research/Botany/bota ny.htm  Image preprocessing: Contour extraction and simplification Background subtraction  Contour based classification for isolated leaves  Model based leaf detection Contour extraction

9. Experiments at UMD (2) -- Isolated Leaves  Contour based classification for isolated leaves (nearest neighbor), using the Swedish leave data Fourier descriptor, average correct ratio 90% Shape context, average correct ratio 88%  Benefit: Invariant to scaling, translation and rotation  Problems: Both are “ global ” descriptors, difficult to handle occlusion or overlay The leaves need to be segmented first.

10. Experiments at UMD (3) -- Model-Based Leaf Detection  Goal: extract leaves from specimen images, which may contain more than one leaf  Hough transform (weighted by matching cost)  Shortest path (grouping edge segments) modelspecimen leaf detection current result, no elimination yet

11. Summary - Difficulties  Difficulties Detection/Segmentation Deformed leaf shapes Fresh vs. dry leaves (loss of color information, distortion, etc.) Left: 7 branches. Center: 9 branches, missing a stem. Right: dry specimen, no color and bad shape and texture.

12. Summary - Future works  Possible future experiments includes: Composition system method and syntactic pattern recognition Appearance based model Texture analysis Eigenshape analysis

13. Suggestions and Thanks Thanks!