Robust Image Topological Feature Extraction Kārlis Freivalds, Paulis Ķikusts Theory Days at Jõulumäe October 2008 University of Latvia

Image Analysis Image analysis steps  Pixels  features  Features  objects  Objects  knowledge  Knowledge  decision Features  Edge  Ridge  Valley (ravine)  Corner

Topological Features Image is treated as 3D surface Ridge Valley Junction Peak Pit Saddle (Edge)

Ridges in Images

Ridge Image Examples TextLine drawings Natural objectsIndustrial applications

Mathematics Image – smooth function f(x, y) Derivatives Directional derivative in direction p = (px, py) Second derivative in two directions (unit vectors) p and q

Principal Orthogonal Directions Two directions play the key role Gradient direction Tangent direction h = (hx, xy) = (–gy, gx). f h = 0 Decision is based on f gg, f hh and f hg

Ridge Definition Point (x, y) is called a ridge point if and only if

Analytical example f(x, y) = y 2 – x 2 K3: xy = 0 gives x = 0 and y = 0 K1: K2: always true Ridge: x = 0

Another example f(x, y) = –y 2 – 10x 2 K3: xy = 0 gives x = 0 and y = 0 K1: always true K2: Ridge: x = 0 Theorem: Ridge of a quadratic surface is always a line.

Naive implementation f x = (f(k + 1, l) – f(k – 1, l)) / 2, f y = (f(k, l + 1) – f(k, l – 1)) / 2, g = g x = f x / g, g y = f y / g, h x = – g y, h y = g x, f xx = f(k + 1, l) – 2f(k, l) + f(k – 1, l), f yy = f(k, l + 1) – 2f(k, l – 1) + f(k, l – 1), f xy = (f(k + 1, l + 1) + f(k – 1, l – 1) – f(k + 1, l – 1) – f(k – 1, l + 1)) / 4, f gg = f xx g x g x + 2f xy g x g y + f yy g y g y, f hh = f xx h x h x + 2f xy h x h y + f yy h y h y, f gh = f xx g x h x + f xy (g x h y + g y h x ) + f yy g y h y. Ridge conditions:

Naive implementation

Equivalent Formulation Hessian matrix Eigenvalues of H Eigenvectors of H Ridge: R. Haralick, “Ridges and Valleys on Digital Images,” Computer Vision, Graphics, and Image Processing, vol. 22, no. 10, pp. 28–38,Apr

Algorithm Calculate eigenvalues and eigenvectors See if See if there is a sign change of in the direction of Problems:  Unreliable  Requires interpolation  Relatively Slow

Lee and Kim Algorithm 4 directions of discrete grid are analyzed Orthogonal directions of greatest curvature are selected Point is classified as ridge, valley, peak or pit based on sign changes of gradients in these directions. Problems:  Unreliable  No clear distinction between ridges and peaks Lee, S. and Kim, Y. J Direct Extraction of Topographic Features for Gray Scale Character Recognition. IEEE Trans. Pattern Anal. Mach. Intell. 17, 7 (Jul. 1995),

New Discrete Algorithm Consider 4 directions of discrete grid Find direction of minimum second derivative (that will be direction perpendicular to ridge), let f hh be that second derivative. Test if the current point is a local maximum in that direction Let f gg be the second derivative in the perpendicular direction. Test if abs( f hh ) >= abs( f gg ) If all tests are true return abs( f hh ) as ridge strength Else return that this is not a ridge point

Implementation // Calculation of ridge pixel strength at the given point, 0 if the point is not a ridge point // assumes that int *image holds the image values in row by row. int RidgePixelStrengthOptimized(int x, int y){ int *p = image + (x + y*width); int f = *p; int fpp[4]; int fhh,fhhI, fhh_tmp, fhhI_tmp; fpp[0] = 2*(f - p[1] - p[-1]); // calculate magnitudes of directional second derivatives fpp[1] = - p[width+1] - p[-width-1]; fpp[2] = 2*(f - p[width] - p[-width]); fpp[3] = - p[width-1] - p[-width+1]; if (fpp[1]>fpp[0]){fhh = fpp[1];fhhI = 1;} else {fhh = fpp[0];fhhI = 0;} // find the maximum one if (fpp[2]>fpp[3]){fhh_tmp = fpp[2];fhhI_tmp = 2;} else {fhh_tmp = fpp[3];fhhI_tmp = 3;} if(fhh_tmp > fhh){fhh = fhh_tmp;fhhI = fhhI_tmp;} fhh += 2*f; // diagonal if(fhh<lowerThreshold) return 0 ;// optimization: early exit int fgg = fpp[(fhhI+2)&3] + 2*f; // second derivative in the orthogonal direction int f1Ofs[4] = {1, width+1, width, width-1}; int ofs = f1Ofs[fhhI]; if(p[ofs] < f && p[-ofs] < f && abs(fgg) <= fhh) return fhh; return 0; }

Results

Sensitivity to Noise

Sensitivity to Noise (1)

Sensitivity to Noise (2)

Sensitivity to Noise (3) Note: All images were processed with the same parameters

Results Connected lines Thin lines No artifacts Low noise sensitivity High performance 45MPix/s = 108 video frames(720 × 576) per second ( AMD Athlon 2.6GHz)

Compared to Lee and Kim Algorithm Simpler Faster No spurious ridges Original imageLee & Kim Freivalds & Ķikusts

Scale Selection Gaussian blurring (Geusebroek et.al. 2002) Scale space and automatic scale selection (Lindeberg 1996) Adaptive blurring (Elder, Zucker 1998) Geusebroek, J., Smeulders, A. W., and Weijer, J. v Fast Anisotropic Gauss Filtering. In Proceedings of the 7th European Conference on Computer Vision-Part I. LNCS, vol pp T. Lindeberg: "Edge detection and ridge detection with automatic scale selection", International Journal of Computer Vision, vol 30, number 2, pp , Earlier version presented at IEEE Conference on Pattern Recognition and Computer Vision, CVPR'96, San Francisco, California, pages , june 1996 Elder, J. and Zucker, S Local scale control for edge detection and blur estimation. IEEE Pattern Anal. Machine Intell., 20(7): 699–716

Topological Edge Detection Edge conditions:  f gg = 0  f ggg < 0 Algorithm  Find the discrete direction of maximum gradient  See if the second derivative has sign change from positive to negative in that direction

Comparison to Canny Detector Similar quality Simpler Faster OriginalCanny Proposed

Results – Shooting Target

Conclusion New robust ridge, edge detection algorithms High quality High performance; suitable for real-time applications Simple implementation Straightforward hardware implementation