Augmenting the Generalized Hough Transform to Enable the Mining of Petroglyphs Qiang Zhu, Xiaoyue Wang, Eamonn Keogh, 1 Sang-Hee Lee Dept. Of Computer Science & Eng., 1 Dept. of Anthropology University of California, Riverside

Outline  Motivation  Approach  Evaluation  Conclusion

Motivation(1)-applications Petroglyphs are one of the earliest expressions of abstract thinking. Providing a rich source of information:  climate change  existence of a certain species  patterns of human’s migrations and interactions

Motivation(2)-difficulties Progress in petroglyph research has been frustratingly slow.  due to their extraordinarily diverse and complex structure  most matching algorithms can not capture the similarity of petroglyphs  for those that can, even in limited cases, do not scale to large collections

Approach How to preprocess the raw data? How to define the distance measure? How to speed up?

Preprocessing(1) With rare exceptions, petroglyphs do not lend themselves to automatic extraction with segmentation algorithms. The border of this rock may be recognized as the edge of this petroglyph

PetroAnnotator Load the raw image into our human computation tool

PetroAnnotator (cont.) Draw an approximate boundary around object, and then trace the shape

Preprocessing(2)-downsampling A (A)Two overlaid skeleton traces (340 by 250) of the same image of a Bighorn sheep. Less than 3.5% of the pixels from each image overlap. (B) The same two images after downsampling (30 by 23). 75.6% of the pixels (denoted by black) are common to both.

Distance Measure-why GHT ? essentially makes no assumption about the data  open/closed boundaries  connected/disconnected shapes correctly captures the similarity  subjective/objective similarity on unlabeled/labeled datasets tightly lower bound the distance  allowing for very efficient searches in large datasets

Classic GHT GHT is a useful method for two dimensional arbitrary shape detection. QC

(1) Find the “star-pattern” R R

(2) Superimpose & Accumulate A C

(3) Find the “peak” C Q R R’ A

A Basic Distance Measure Classic GHT doesn’t explicitly encode a similarity measure We can simply define a GHT-based distance: minimal unmatched edge points (MUE) = number of edge points in Q – maximal matched edge points = 4 – 3 = 1 (for our toy example)

A New Cell Incrementation Strategy When can we obtain the value of a particular cell in the accumulator?  In the classic GHT, until the end of all incrementation  Is it possible to obtain the value one by one?  Need to check all positions that are possible to increase the cell value QC ?

Lower Bound In this column Q needs 2 pixels in C, and has 3 In this column Q needs 2 pixels in C, and has 2 In this column Q needs 4 pixels in C, and has only 2 In this column Q needs 2 pixels in C, and has 2 In this column Q needs 2 pixels in C, and has 3 QC SigQx = SigCx = ? 0 Minimal missed points: = 2 ? ? ? ? ? ? ? ?

Time Complexity Classic GHT  O(N Q ×N C +S 2 )  superimpose all query vectors to all edge points in the candidate image Lower bound GHT  O(S 2 )  compare one-dimensional signatures  further reduced by early abandon and shifting order  one to two orders of magnitude speed-up

Variants on the Basic Distance Measure Query-by-Content: Clustering: Finding Motifs:

Evaluation We performed three sets of experiments ：  Evaluation of Utility -on unlabeled data  Evaluation of Accuracy -on labeled data  Evaluation of Scalability -on synthetic data

Evaluation of Utility (1) Atlatls Anthropomorphs Bighorn Sheep (1)Our GHT-based distance measure correctly groups all seven pairs (2)The higher level structure of the dendrogram also correctly groups similar petroglyphs A clustering of typical Southwestern USA petroglyphs

Evaluation of Utility (2) abcdef g h SC WY

Evaluation of Utility (3) Whether our distance measure can find meaningful motifs?  2,852 real petroglyphs  4,065,526 possible pairs  52 top motifs ( %) by motif cutoff Motif Cutoff

Evaluation of Accuracy-datasets NicIcon dataset  24,441 images  14 categories  33 volunteers  234×234 pixels  WD/WI tests Farsi digits dataset  From 11,942 registration forms  60,000 digits for training  20,000 digits for testing  54×64 pixels (largest MBR)

(1) Test the Downsampling Size Resolution (R×R) of Downsampled Images (NicIcon) Error Rate (%) 5 WI WD Resolution (R×R) of Downsampled Images (Farsi) Error Rate (%) In both datasets, the error rate of one-nearest- neighbor test varies little once the resolution is greater than 10×10

(2) Competitive accuracy NicIcon dataset  Error rate for WD: 4.78%  8.46% for WI  The dataset creators tested on the online data using three classifiers.  Only one of them (DTWB) is better, however, slower Farsi digits dataset  Error rate: 4.54%  Borji et al. performed extensive empirical tests on this dataset  Of the twenty reported error rates, the mean was 8.69%  Only four beat our approach, but need to set at least six parameters

Evaluation of Scalability-datasets We made 8 synthetic petroglyph datasets  Based on 22 classic petroglyphs  Duplicated by 10 volunteers on a tablet  Applied a Random Polynomial Transformation  Containing up to 1,280,000 objects

(1) Querying by Content Leave-one-out one-nearest-neighbor test. Repeated the test for 10 times on each dataset. 10K 20K40K80K160K320K640K1280K Size of Synthetic Petroglyphs Datasets Prune Rate (%) Max Prune Rate Avg Prune Rate Min Prune Rate 10K20K40K80K160K320K640K1280K Size of Synthetic Petroglyphs Datasets % to Brute Force Time

(2) Finding Motifs  A brute force algorithm requires time quadratic in the size of dataset.  By using the triangular inequality of our distance measure, we only need to calculate a tiny fraction of the exact distance.  Even for the smallest dataset: -our algorithm is 712 times faster -we can prune 99.84% of the calculations 10K20K40K80K160K320K640K1280K Size of Synthetic Petroglyphs Datasets Speed Up (times)

Conclusion In this work we considered, for the first time, the problem of mining large collections of rock art.  Introduced a novel distance measure  Found an efficiently computable tight lower bound to this measure  Enabled mining large data archives effectively

Thanks for your listening ! All datasets and the code can be downloaded from:

Preprocessing With rare exceptions, petroglyphs do not lend themselves to automatic extraction with segmentation algorithms. Cracks in the rock are more “significant” than the actual edges

Preprocessing-existing archives There are several other rich sources of rock art data to be mined, e.g.: sketches by anthropologists From a scanned book Downsampled Binarized Thinned

By HausdroffBy GHT Experiment testing the impact of noise, a single dot is randomly added