The mapping process – some observations Robina Clayphan EDLF

Local schemas > ESE Data Flow

Management of the process Sheer complexity of managing the hundreds of files going through the steps in the process keeping track of the status of the files – straight-forward ones – in the right place for the next step – problem ones - refer back to provider or a developer Use of Sharepoint document libraries and rapid establishment of procedures that all must adhere to The management of the process evolved during implementation - a very steep learning curve Maintenance of authority files – getting for meta-metadata from the providers (types etc) – collection IDs

(sort of) Policy issues Inclusion criterion: must have a link giving direct access to the digital object – check if URLs in data actually resolve to the object described Often: – resolve to metadata page with e.g. pdf icon – how many clicks are acceptable – need for policy decision – granularity mismatch – link at title level only Sometimes: – 404 page not found - refer to provider – persistence of URLs – need a plug in (e.g. DjVu) – is that OK? Occasionally: a log-in required for restricted access resources Need for providers to ensure they only provide links to resources that can be accessed

Data level problems 1 Trying to understand decision-making process of the original metadata creators – What they meant by e.g. dc:date, dc:source Trying to discern the (implicit) data model of the original metadata creators – What is the dc:relation referring to Understanding data in a foreign language or foreign script – Is negyedévenként really hungarian for terminally? And, if so, why is it in dc:format?

Data level problems 2 Questions to developers that arose from examining the data – All records have two instances of dc:identifier the first a URL the second (possibly) a shelfmark. Need to map each instance to a different ESE - can it be done? – All records have two instances of dc:rights the first appropriate the second not – is it possible to just display the first and ignore the second? – Where values had been divided between multiple instances of the same element – could they be concatenated with punctuation for a better display e.g spatial1, spatial2, spatial3 used for a geographic hierarchy. Another with up to 14 instances of dc:subject.

Normalisation level At the normalisation stage you can see if your interpretation of the record actually makes sense when it has been processed against the source data. Apply the Quality Control Checklist Edit mapping and repeat !

(my) Conclusion All indicates: – that it is easier if the mapping and normalising is done as close to source as possible, ideally by the providers they are the ones who understand what the data means and can make sensible mapping decisions they understand the language and script – Tools would be nice!

Local schemas > ESE Data Flow Transform data to populate local repository #0 Export data to Europeana #5 Aggregator? EuropeanaLocal Aggregator with provider?

EuropeanaLocal Content Provider Model - to illustrate movement of metadata only Aggregator EuropeanaLocal Parallel Test Environment Aggregator Europeana C o n t e n t p r o v i d e r r e p o s i t o r i e s C o n t e n t p r o v i d e r l o c a l s y s t e m s Customised transformations to e.g. OAI-DC Mapping and transformation to ESE, including elements Harvesting of e.g. OAI-DC No metadata transformations

Currently a great deal of manual effort goes into metadata transformation. – at provider sites: local format to repository format – by the Europeana development team harvested format to ESE – normalisation by Europeana development team Where will this work happen in EuropeanaLocal? – feasibility of central Europeana staff handling hundreds more collections? Can we minimise the current manual overhead? Issues for EuropeanaLocal What are the possibilities for automating all or some of the transformation work?