1. - 1/22 - Centre for Digital Video Processing C E N T R E F O R D I G I T A L V I D E O P R O C E S S I N G DIGIFACT: Digitisation of Museum Artefacts Facilitating Search & Retrieval Noel O’Connor & Robert Keogh C entre for D igital V ideo P rocessing Dublin City University, Ireland

2. - 2/22 - Centre for Digital Video Processing C E N T R E F O R D I G I T A L V I D E O P R O C E S S I N G Outline Background –Motivation, project objectives, funding, timeline & plan 2D artefact indexing –Image capture, object segmentation, feature extraction, database population, search & retrieval Exploring 3D indexing –Depth-based features –Image-based rendering Conclusion

3. - 3/22 - Centre for Digital Video Processing C E N T R E F O R D I G I T A L V I D E O P R O C E S S I N G Motivation NMI collection: –4-5 million objects –4 curatorial divisions –4 separate geographical locations –Only a small fraction physically on display –Similar problems are faced by other museums Requirement for collections to be documented so that they are accessible for browsing and searching. At present 1 : –“subjective written descriptions supported by often copious 2D images which are unable to deliver concise visual definitions of complete objects” –“is excessively time consuming, cumbersome and inflexible” 1 ORION research roadmap for the European archaeological museums’ sector

4. - 4/22 - Centre for Digital Video Processing C E N T R E F O R D I G I T A L V I D E O P R O C E S S I N G Project Objectives To develop a low-cost approach to image-based indexing of artefacts –Straighforward image capture –Automatic extraction of searchable image- based metadata –Can be used to complement (extend) existing cataloging processes –Explore 3D indexing and imaging For the end-user: –“Find me similar views of similar artefacts" –"Show me this artefact from a different (user defined) angle”

5. - 5/22 - Centre for Digital Video Processing C E N T R E F O R D I G I T A L V I D E O P R O C E S S I N G Funding EI Proof of Concept: –1 year project, €90k (approx.) –Competitive bid (<25% funded) –Letters of support from NMI and MERL –Aim: validate the original idea –People: Noel O’Connor (PI), Alan Smeaton, Gareth Jones, Robert Keogh, Phil Kelly (part-time), Eddie Cooke (previously) Technology Development: –Opportunity to re-apply in 2 nd phase –Still competitive … but successful PoC helps! –3 year project, €350K Need strong business case (valuable IP generation)

6. - 6/22 - Centre for Digital Video Processing C E N T R E F O R D I G I T A L V I D E O P R O C E S S I N G Project Plan

7. - 7/22 - Centre for Digital Video Processing C E N T R E F O R D I G I T A L V I D E O P R O C E S S I N G Image Capture “Off the shelf” Components Easy to source, set- up and use Entire cost < €2,000

8. - 8/22 - Centre for Digital Video Processing C E N T R E F O R D I G I T A L V I D E O P R O C E S S I N G 2D Indexing Segmentation –Extracting the object’s shape from the (green) background Feature extraction –Calculating a description of the object’s shape, colour and texture Description –Structured, searchable metadata file imported to database –Standards compliant Use existing research: –aceToolbox*: a multi- platform software tool-box for low-level MPEG-7 visual analysis –Video object segmentation and modeling* + *Funded by EU FP6 +Funded by EI Tech Dev

9. - 9/22 - Centre for Digital Video Processing C E N T R E F O R D I G I T A L V I D E O P R O C E S S I N G 2D Indexing Demo: –Data capture (saw in lab) –Admin application: Add new artefact Edit description: extract object, calculate description, add to database –Search and Retrieval Application Select existing image/object Find similar objects by matching descriptions (only uses 1 feature at the moment)

10. - 10/22 - Centre for Digital Video Processing C E N T R E F O R D I G I T A L V I D E O P R O C E S S I N G 3D Indexing Rotating turntable is like multiple cameras placed around the object –Each pair of images is like a stereo pair –Can use computer vision techniques to extract 3D information –Rudimentary depth estimation –Info can be used to generate virtual views 5˚ 0˚ 10˚

11. - 11/22 - Centre for Digital Video Processing C E N T R E F O R D I G I T A L V I D E O P R O C E S S I N G Conclusions Good initial progress 2D indexing work well in hand 3D work at much earlier stage Thanks for your attention!