Download presentation
Presentation is loading. Please wait.
1
Geometric and Semantic Matching for Cultural Heritage Artefacts
Stephen C. Phillips IT Innovation
2
https://www.flickr.com/photos/lunamodule/ This is Cyprus
© Copyright University of Southampton IT Innovation Centre and other GRAVITATE partners,
3
This is a jigsaw of Cyprus – I expect we could all do it
© Copyright University of Southampton IT Innovation Centre and other GRAVITATE partners,
4
What if lots of pieces were missing?
© Copyright University of Southampton IT Innovation Centre and other GRAVITATE partners,
5
What if it was in 3D? © Copyright University of Southampton IT Innovation Centre and other GRAVITATE partners,
6
Cracked Faded Eroded Front Back
What if the pieces were 2500 years old and were all faded, cracked and with the edges worn away so that they didn’t actually fit together any more? Front Back © Copyright University of Southampton IT Innovation Centre and other GRAVITATE partners,
7
Ashmolean Museum, Oxford Fitzwilliam Museum, Cambridge
British Museum, London Cyprus Museum, Cyprus What if they were dispersed across 4 museums and private collections? Ashmolean Museum, Oxford Fitzwilliam Museum, Cambridge © Copyright University of Southampton IT Innovation Centre and other GRAVITATE partners,
8
Jigsaws in 3D most pieces missing all the edges worn off many puzzles jumbled together pieces spread across many countries So this is the problem © Copyright University of Southampton IT Innovation Centre and other GRAVITATE partners,
9
Re-Assembly Re-Unification Re-Association Similarity Search
_______________ Similarity Search This is what GRAVITATE will do © Copyright University of Southampton IT Innovation Centre and other GRAVITATE partners,
10
Re-Assembly You have some (eroded) pieces. The computer fits them together. This is hard in 2 dimensions, never mind in 3D! Localised geometry Colour descriptors Localised detail 3D descriptors © Copyright University of Southampton IT Innovation Centre and other GRAVITATE partners,
11
Re-Unification Head in the British Museum
Re-unify 3D models of the statue 3D print copies for re-unification Re-unify statue in a virtual museum Statue in the Cyprus Museum © Copyright University of Southampton IT Innovation Centre and other GRAVITATE partners,
12
Re-Association Data in another museum catalogue
Artefact with semantic description Material, decoration, size, shape, period, style, glaze, … © Copyright University of Southampton IT Innovation Centre and other GRAVITATE partners,
13
Re-Association Find similarities leading to new insights about past cultures © Copyright University of Southampton IT Innovation Centre and other GRAVITATE partners,
14
How? © Copyright University of Southampton IT Innovation Centre and other GRAVITATE partners,
15
What’s the data? © Copyright University of Southampton IT Innovation Centre and other GRAVITATE partners,
16
Fragments of Terracotta Statues from Salamis
© Copyright University of Southampton IT Innovation Centre and other GRAVITATE partners,
17
We need to Gather all the Knowledge
Catalogue data with text descriptions Chemical analysis Archaeological papers X-ray fluorescence Excavation notes 3D scanning models © Copyright University of Southampton IT Innovation Centre and other GRAVITATE partners,
18
We need to Gather all the Knowledge
Catalogue data with text descriptions Chemical analysis Archaeological papers X-ray fluorescence Excavation notes 3D scanning models © Copyright University of Southampton IT Innovation Centre and other GRAVITATE partners,
19
We need to Gather all the Knowledge
The British Museum has 2.5M objects described in CIDOC CRM Catalogue data from Ashmolean, Fitzwilliam and Cyprus museums is now mapped to the same data model Catalogue data with text descriptions Archaeological papers Object hasNote “…lots of free-form text added by the curator” Using Natural Language Processing to extract and encode meaning from this text: References to papers, to catalogue entries Parts and features Conservation condition Measurements … CIDOC CRM / CRMarchaeo / etc Excavation notes © Copyright University of Southampton IT Innovation Centre and other GRAVITATE partners,
20
We need to Gather all the Knowledge
All fragments (211) are scanned, in a variety of resolutions and with a variety of scanners Volume, area thickness, curvature, … Chemical analysis Colour, texture, distance from convex hull, … X-ray fluorescence Feature detection Part annotation 3D scanning models Faceting: front / back / fracture CH Artefact Partonomy CRMdig © Copyright University of Southampton IT Innovation Centre and other GRAVITATE partners,
21
Faceting Assume local fold model Measure local fold angle
Select locally salient angles Connect to form facet curves Determine the facet types Robust for our fragments Very few parameters 90ᵒ © Copyright University of Southampton IT Innovation Centre and other GRAVITATE partners,
22
Feature Detection Morphological and stylistic feature detection and characterization Reasoning on similarity among fragments © Copyright University of Southampton IT Innovation Centre and other GRAVITATE partners,
23
Using CH Artefact Partonomy within CIDOC-CRM
Instantiated in class structure of CIDOC-CRM ExG: Expansion by GRAVITATE inherit its functionality, maximize acceptance use semi-automatic annotation tools © Copyright University of Southampton IT Innovation Centre and other GRAVITATE partners,
24
How do I use it? © Copyright University of Southampton IT Innovation Centre and other GRAVITATE partners,
25
Six views in the dashboard
Inspection Users can inspect 3D fragments and metadata and run a search Fragment 3D assets can be processed (faceting, geometric characterisation, feature identification) and annotated ReAssembly Exploration Datasets can be explored according to specific selected properties History To preserve the list of operations performed in the session Clipboard To save all the data the user is interested in (e.g. notes, models, annotation) © Copyright University of Southampton IT Innovation Centre and other GRAVITATE partners,
26
Six views in the dashboard
Inspection Users can inspect 3D fragments and metadata and run a search Fragment 3D assets can be processed (faceting, geometric characterisation, feature identification) and annotated ReAssembly Exploration Datasets can be explored according to specific selected properties History To preserve the list of operations performed in the session Clipboard To save all the data the user is interested in (e.g. notes, models, annotation) © Copyright University of Southampton IT Innovation Centre and other GRAVITATE partners,
27
Six views in the dashboard
Inspection Users can inspect 3D fragments and metadata and run a search Fragment 3D assets can be processed (faceting, geometric characterisation, feature identification) and annotated ReAssembly Exploration Datasets can be explored according to specific selected properties History To preserve the list of operations performed in the session Clipboard To save all the data the user is interested in (e.g. notes, models, annotation) © Copyright University of Southampton IT Innovation Centre and other GRAVITATE partners,
28
Six views in the dashboard
Inspection Users can inspect 3D fragments and metadata and run a search Fragment 3D assets can be processed (faceting, geometric characterisation, feature identification) and annotated ReAssembly Exploration Datasets can be explored according to specific selected properties History To preserve the list of operations performed in the session Clipboard To save all the data the user is interested in (e.g. notes, models, annotation) © Copyright University of Southampton IT Innovation Centre and other GRAVITATE partners,
29
Six views in the dashboard
Inspection Users can inspect 3D fragments and metadata and run a search Fragment 3D assets can be processed (faceting, geometric characterisation, feature identification) and annotated ReAssembly Exploration Datasets can be explored according to specific selected properties History To preserve the list of operations performed in the session Provenance and argumentation Clipboard To save all the data the user is interested in (e.g. notes, models, annotation) © Copyright University of Southampton IT Innovation Centre and other GRAVITATE partners,
30
© Copyright University of Southampton IT Innovation Centre and other GRAVITATE partners, 2015-2016
31
Prototype © Copyright University of Southampton IT Innovation Centre and other GRAVITATE partners,
32
And Finally: ReAssembly
Jigsaws in 3D most pieces missing all the edges worn off many puzzles jumbled together pieces spread across many countries Use all the data to guide the process Selection by Similarity Positioning clues Matching Mating © Copyright University of Southampton IT Innovation Centre and other GRAVITATE partners,
33
Mannequin © Copyright University of Southampton IT Innovation Centre and other GRAVITATE partners,
34
Matching: geometry-based
© Copyright University of Southampton IT Innovation Centre and other GRAVITATE partners,
35
Mating Mating digitally mimics gluing of the fragments
We place selected fragments in their optimal relative position based on: Geometric complementarity Skin pattern continuity Semantic constraints Global alignment Mathematical morphology used Final approval of proposed assemblage is requested of the user (SotA mating from predecessor PRESIOUS) © Copyright University of Southampton IT Innovation Centre and other GRAVITATE partners,
36
Gravitate-project.eu Coordinator: Stephen C Phillips
Gravitate-project.eu
Similar presentations
