DARWIN - Dolphin Photo-identification Software Adaptations to Digital Camera Acquisition and Increased Matching Accuracy K. R. Debure, J. H. Stewman, S. A. Hale, Eckerd College, St. Petersburg, FL DARWIN is a computer program that automates the photo-identification of dolphins from photographs of dorsal fins. This program allows researchers to query a database of previously identified dorsal fin images with a digital image of an unidentified dolphin's fin. The system responds with a rank ordered list of database fin images most closely resembling the query image. In this way DARWIN assists researchers to prioritize their search of database images, and potentially reduces the time required for identification. As digital cameras rapidly replace traditional film cameras for the acquisition of data in the field, adaptation of the software in response to the changing needs of the user community is critical. Compatibility with appropriate image formats, efficient handling of higher resolution image data, and automation and streamlining of data entry can facilitate the processing of increasing quantities of field data. The automated generation of the outline is one example of an effort to reduce data entry requirements. In addition, to further increase the matching accuracy and the realistic utility of the DARWIN software, the most successful aspects of a quantitative dorsal fin matching approach have been incorporated within a broad hierarchical approach using qualitative distinctions between fins. This recently enhanced software (1) facilitates subset searches of the database based on damage category (missing tip, etc), (2) selects more intuitive transformations for alignment, and (3) more closely emulates the manual photo-identification process. Automatic Outline Generation The acquisition of field data with digital cameras requires new functionality from the software. The current version reads JPEG files and uncompressed TIFF format will be available in the next version. The automatic generation of the outline of the dorsal fin is one attempt to reduce the data entry effort. The outline is extracted as follows: compute the grayscale intensity of the original image perform unsupervised thresholding based on histogram analysis use morphological processing and feature recognition to select and simplify region of interest use morphological processing to generate region outlines and select largest outline as dorsal fin contour Abstract Introduction [1] M. Kass, A. Witkin, and D. Terzopoulos. Snakes: Active contour models. International Journal of Computer Vision, pages , [2] S.Mallat. Characterization of signals from multiscale edges. IEEE Trans. on Pattern Analysis and Machine Intelligence, 14(7): , July [3] N. Otsu. A threshold selection method from gray level histograms. IEEE Trans. on Systems, Man, and Cybernetics, SMC-9:62--66, January 1992.References This research was supported by the National Science Foundation under grant numbers DBI and IIS Additional funding was provided by National Marine Fisheries Services and Eckerd College. Dorsal fin images courtesy of Eckerd College Dolphin Project. Acknowledgements DARWIN is a computer program which aids marine mammalogists in the identification of dolphins and in the management of a database of observational information. DARWIN graphically presents the digitized dorsal fin images and associated textual sighting information. The database of information can be: queried for images which are similar to the dorsal fin image of an unknown individual queried to limit the search within specified damage categories queried with the name of a specific individual organized and viewed by particular sighting information such as location, date, or fin damage category Figure 1 : Following a query of the dorsal fin database, DARWIN presents a list of database images which most closely resemble the unknown fin image. Direct comparison of fin outlines is problematic since perspective distortions can cause outlines extracted from multiple photographs of the same dolphin’s dorsal fin to differ significantly. Ideally, we would like to transform the outlines so that it appears the original fins were photographed from the same angle. The transformation process is as follows: locate sets of common feature points on pairs of fin outlines compute transformation matrix which maps one set of points onto the other apply transformation to entire outline evaluate error between registered outlines Outline Registration Figure 3 : Upper and center left: Automatically generated dorsal fin outlines. Upper and center right: Common feature points (the start of the leading edge, the end of the leading edge, the tip, the most prominent notch and the end of the trailing edge) are identified along the outline. Bottom left: Original outlines of the database and unknown fin are superimposed before alignment. Bottom center: Outlines are partially registered following an affine transform based on a triplet of feature points. Bottom right: Fully registered outlines after an iterative alignment approach. 1)Map the unknown to the database fin by using a three point affine transformation. 2)Find the ratio (R) of arc lengths between database and unknown fins 3)Traverse the database fin outline (which has evenly spaced points at approximately 3 unit separation) and at each point at arc length (A), compute a corresponding point on the unknown fin which is at a position A*R along the outline, then save the location of the midpoint between this pair of points. The sequence of midpoints forms a "medial axis" or spine that is approximately midway between the two fin outlines. 4)Traverse the medial axis, and at each point, find a perpendicular, then find the points of intersection of this perpendicular with each fin outline. The length (L) of the segment between these two points is squared (L*L) to indicate the degree of mismatch between that point pair. The arithmetic mean of these errors from the beginning of the leading edge to the end of the trailing edge is the mismatch error for the entire fin pair. Computation of Error between Outlines Two methods are used to find a "best" mapping of unknown to database fin. 1.The “Quick & Dirty” approach uses three key points (beginning of the leading edge, tip, and most significant notch) on each outline. A transformation is found mapping the unknown fin triplet to the database triplet. This transformation is then applied to the entire unknown contour effectively solving for the pose differences between the fins in the two images. 2. The “Optimal” approach adjusts the beginning positions of each leading edge and the ending positions of each trailing edge using a Newton-Raphson optimization method seeking a better three point mapping that minimizes the overall error. Algorithms for Outline Mapping Preliminary testing of the two approaches utilized a database of dorsal fin images for 200 individuals. A test set of fifty different dorsal fin images of known individuals were matched against the database. On average, both methods ranked the correct fin in the top 17% of the database. Using the “Quick & Dirty” method, the median rank was in the top 5%; using the “Optimal” method the median was in the top 8%. However, we have observed that the transformations produced by the “Optimal” method produce a more accurate alignment – thus further refinement of the error metric is needed. These results suggest that the registration methods presented herein show significant promise in perspective correction of dolphin dorsal fin images. The iterative nature of the alignment algorithm makes it less sensitive to the original designation of the outline or subsequent identification of feature points. In any case, the process has the ability to improve the registration of outlines considerably. The improved registration suitably corrects for perspective distortion and makes fin outline comparisons easier and aids in the subsequent retrieval of appropriate images. Results and Conclusions In order to query the database for fins resembling the unknown dolphin: 1.The software generates a trace of the outline of the dorsal fin 2.The user may replace the automatically generated trace with a hand-traced rough outline of the dorsal fin. Dorsal Fin Identification 3. Active contours [1] are employed to accurately position the points comprising the outline onto the actual edge of the fin. 4. Feature points along the outline are identified and used to perform a registration of the two fins. 5. Error between the registered outlines is computed and used to rank order the fins in terms of similarity. Figure 2 : The tracing window allows the user to perform a sketch based query of the database of dorsal fins. This window is also used to add new dorsal fin images and associated sighting data to the database.