A Computational Framework for Assembling Pottery Vessels Presented by: Stuart Andrews The study of 3D shape with applications in archaeology NSF/KDI grant #BCS Advisor: David H. Laidlaw Members of the SHAPE Lab and the Department of Computer Science

Stuart Andrews, The SHAPE Lab, Brown University A Computational Framework for Assembling Pottery Vessels 2 Why should we try to automate pottery vessel assembly? Reconstructing pots is important Tedious and time consuming hours  days per pot, 50% of “on-site” time Virtual artifact database

Stuart Andrews, The SHAPE Lab, Brown University A Computational Framework for Assembling Pottery Vessels 3 Statement of Problem

Stuart Andrews, The SHAPE Lab, Brown University A Computational Framework for Assembling Pottery Vessels 4 Statement of Problem

Stuart Andrews, The SHAPE Lab, Brown University A Computational Framework for Assembling Pottery Vessels 5 Goal A computational framework for sherd feature analysis An assembly strategy To assemble pottery vessels automatically

Stuart Andrews, The SHAPE Lab, Brown University A Computational Framework for Assembling Pottery Vessels 6 Challenges Integration of evidence Efficient search Modular and extensible system design

Stuart Andrews, The SHAPE Lab, Brown University A Computational Framework for Assembling Pottery Vessels 7 Virtual Sherd Data 1.Scan physical sherds 2.Extract iso-surface 3.Segment break curves 4.Identify corners 5.Specify axis 16 sherds 120 pairs ! 560 triples !!

Stuart Andrews, The SHAPE Lab, Brown University A Computational Framework for Assembling Pottery Vessels 8 A Greedy Bottom-Up Assembly Strategy Single sherds

Stuart Andrews, The SHAPE Lab, Brown University A Computational Framework for Assembling Pottery Vessels 9 A Greedy Bottom-Up Assembly Strategy PairsSingle sherds

Stuart Andrews, The SHAPE Lab, Brown University A Computational Framework for Assembling Pottery Vessels 10 A Greedy Bottom-Up Assembly Strategy Single sherdsPairs

Stuart Andrews, The SHAPE Lab, Brown University A Computational Framework for Assembling Pottery Vessels 11 A Greedy Bottom-Up Assembly Strategy TriplesSingle sherdsPairs

Stuart Andrews, The SHAPE Lab, Brown University A Computational Framework for Assembling Pottery Vessels 12 A Greedy Bottom-Up Assembly Strategy Single sherdsPairsTriples

Stuart Andrews, The SHAPE Lab, Brown University A Computational Framework for Assembling Pottery Vessels 13 A Greedy Bottom-Up Assembly Strategy Etc. Single sherdsPairsTriples

Stuart Andrews, The SHAPE Lab, Brown University A Computational Framework for Assembling Pottery Vessels 14 Overview Generate Likely Pair-wise Matches Generate Likely 3-Way Matches … etc.

Stuart Andrews, The SHAPE Lab, Brown University A Computational Framework for Assembling Pottery Vessels 15 Likely Pairs Proposals Likelihood Evaluations Generate Likely Pair-wise Matches

Stuart Andrews, The SHAPE Lab, Brown University A Computational Framework for Assembling Pottery Vessels 16 A Match A pair of sherds A relative placement of the sherds

Stuart Andrews, The SHAPE Lab, Brown University A Computational Framework for Assembling Pottery Vessels 17 Match Proposals Corner Alignment

Stuart Andrews, The SHAPE Lab, Brown University A Computational Framework for Assembling Pottery Vessels 18 Example Corner Alignments

Stuart Andrews, The SHAPE Lab, Brown University A Computational Framework for Assembling Pottery Vessels 19 Likely Pairs Proposals Likelihood Evaluations Generate Likely Pair-wise Matches

Stuart Andrews, The SHAPE Lab, Brown University A Computational Framework for Assembling Pottery Vessels 20 Match Likelihood Evaluations An evaluation returns the likelihood of a feature alignment Based on the notion of a residual

Stuart Andrews, The SHAPE Lab, Brown University A Computational Framework for Assembling Pottery Vessels 21 Match Likelihood Evaluations Axis Divergence Feature: Axis of rotation Residual: Angle between axes

Stuart Andrews, The SHAPE Lab, Brown University A Computational Framework for Assembling Pottery Vessels 22 Match Likelihood Evaluations Axis Separation Feature: Axis of rotation Residual: Distance between axes

Stuart Andrews, The SHAPE Lab, Brown University A Computational Framework for Assembling Pottery Vessels 23 Match Likelihood Evaluations Break-Curve Separation Feature: Break-curve Residuals: Distance between closest point pairs

Stuart Andrews, The SHAPE Lab, Brown University A Computational Framework for Assembling Pottery Vessels 24 Match Likelihood Evaluations Break-Curve Divergence Feature: Break-curve Residuals: Angle between tangents at closest point pairs

Stuart Andrews, The SHAPE Lab, Brown University A Computational Framework for Assembling Pottery Vessels 25 Match Likelihood Evaluations Fact: Assuming the residuals ~ N(0,1) i.i.d., then we can form a Chi-square:  ² observed Note: Typically, residuals are ~ N(0,  2 ) i.i.d. How likely are the measured residuals?

Stuart Andrews, The SHAPE Lab, Brown University A Computational Framework for Assembling Pottery Vessels 26 Match Likelihood Evaluations We define the likelihood of the match using the probability of observing a larger  ² random Pr{  ² random >  ² observed } = Q Individual or ensemble of features Pair-wise, 3-Way or larger matches How likely are the measured residuals?

Stuart Andrews, The SHAPE Lab, Brown University A Computational Framework for Assembling Pottery Vessels 27 Example Match Likelihood Evaluation (1) ²² nQ Axis Direction Axis Overlap Closest Pt Tangent Ensemble

Stuart Andrews, The SHAPE Lab, Brown University A Computational Framework for Assembling Pottery Vessels 28 Example Match Likelihood Evaluation (2) ²² nQ Axis Direction e-7 Axis Overlap Closest Pt Tangent Ensemble e-6

Stuart Andrews, The SHAPE Lab, Brown University A Computational Framework for Assembling Pottery Vessels 29 Local Improvement of Match Likelihood beforeafter

Stuart Andrews, The SHAPE Lab, Brown University A Computational Framework for Assembling Pottery Vessels 30 Pair-wise Match Results Summary ??

Stuart Andrews, The SHAPE Lab, Brown University A Computational Framework for Assembling Pottery Vessels 31 Pair-wise Match Results Summary Correct Matches Incorrect Matches

Stuart Andrews, The SHAPE Lab, Brown University A Computational Framework for Assembling Pottery Vessels 32 Pair-wise Match Results Summary # of pairs with correct match identified: Top 19 Top 217 Top 320 Total26 Q=1  decreasing likelihood  Q=0 True Pair Proposed matches … Correct match There is no correct match for the remaining 94 pairs!!

Stuart Andrews, The SHAPE Lab, Brown University A Computational Framework for Assembling Pottery Vessels 33 Overview Generate Likely Pair-wise Matches Generate Likely 3-Way Matches … etc.

Stuart Andrews, The SHAPE Lab, Brown University A Computational Framework for Assembling Pottery Vessels 34 Likely Triples 3-Way Proposals 3-Way Likelihood Evaluations Generate Likely 3-Way Matches

Stuart Andrews, The SHAPE Lab, Brown University A Computational Framework for Assembling Pottery Vessels 35 3-Way Match Proposals Merge pairs with common sherd +=

Stuart Andrews, The SHAPE Lab, Brown University A Computational Framework for Assembling Pottery Vessels 36 3-Way Match Likelihood Evaluation Feature alignments are measured 3-way

Stuart Andrews, The SHAPE Lab, Brown University A Computational Framework for Assembling Pottery Vessels 37 3-Way Match Results Summary

Stuart Andrews, The SHAPE Lab, Brown University A Computational Framework for Assembling Pottery Vessels 38 3-Way Match Results Summary # of 3-way matches with correct match identified: Top 13 Top 511 Top 1017 Total31

Stuart Andrews, The SHAPE Lab, Brown University A Computational Framework for Assembling Pottery Vessels 39 Overview Generate Likely Pair-wise Matches Generate Likely 3-Way Matches … etc. Future work

Stuart Andrews, The SHAPE Lab, Brown University A Computational Framework for Assembling Pottery Vessels 40 Related Work Assembly systems that rely on single features [U. Fedral Fluminense / Middle East Technical U. / U. of Athens] Multiple features and parametric shape models [The SHAPE Lab – Brown U.] Distributed systems for solving AI problems [Toronto / Michigan State / Duke U.]

Stuart Andrews, The SHAPE Lab, Brown University A Computational Framework for Assembling Pottery Vessels 41 Contributions A computational framework based on match proposal and likelihood evaluation A method for combining multiple features into one match likelihood An example (greedy) assembly strategy

Stuart Andrews, The SHAPE Lab, Brown University A Computational Framework for Assembling Pottery Vessels 42 Where to go from here? Improve accuracy of features Add new features and feature comparisons Learn how to classify true and false pairs Design specialized search strategies

Stuart Andrews, The SHAPE Lab, Brown University A Computational Framework for Assembling Pottery Vessels 43 Conclusions Encouraging progress on a difficult task We are close to a working system We can get closer by following this approach A uniform statistical analysis of features defines the basis for a complete working system

Stuart Andrews, The SHAPE Lab, Brown University A Computational Framework for Assembling Pottery Vessels 44 References 1.D. Cooper et al. VAST da Gama Leito et al. Universidade Fedral Fluminense A.D. Jepson et al. ICCV G.A. Keim et al. AAAI / IAAI, S. Pankanti et al. Michigan State, G. Papaioannou et al. IEEE Computer Graphics and Applications, G. Ucoluk et al. Computers & Graphics, 1999.

Stuart Andrews, The SHAPE Lab, Brown University A Computational Framework for Assembling Pottery Vessels 45 Results For Discussion Q Q count

Stuart Andrews, The SHAPE Lab, Brown University A Computational Framework for Assembling Pottery Vessels 46 Results For Discussion

Stuart Andrews, The SHAPE Lab, Brown University A Computational Framework for Assembling Pottery Vessels 47 Results For Discussion