WDL Technical Architecture Working Group (TAWG) June 2010 Achievements and Recommendations Co-chaired by Noha Adly, Bibliotheca Alexandrina Babak Hamidzadeh, Library of Congress

TAWG TAWG is responsible for establishing policies and guidelines needed to govern the technical aspects of the WDL. TAWG will advise the WDL Project Manager and the future Executive Committee on matters relating to – Architecture – Workflow – Standards – Site functionality – Technology transfer

Meetings- April Members UNESCO National Library of China National Library of Russia Mexico Bibliotheca Alexandrina Library of Congress National Library of the Netherlands Germany

Meetings- April 2007 Defining the objectives and the target audience of WDL Discuss the merits of a centralized vs a portal model Discuss workflow models Define the main aspects of WDL website (Connectivity, access points, content, search, etc.) Types of content and ingestion models Presentation and usage of the website Intellectual added value Access (multilingual search and display) Context (narrative and interpretive content) Participation (inclusion of social networking)

Members in December Members: Bibliotheca Alexandrina Brown University Library King Abdullah University of Science and Technology Central Library, Qatar Foundation Italian Ministry for Cultural Heritage and Activities Library of Congress National Diet Library National Library of Bulgaria National Library of China National Library of Finland National Library of France National Library of Israel National Library of the Netherlands National Library of Sweden UNESCO University of Pretoria Yale University National Library of Serbia

WDL Content Pipeline WDL Catalog Application WDL Web Application The WDL Architecture Overview TransferProcessDescribeTranslateAssembleIngestAccess

The WDL Content Pipeline TransferProcessDescribeTranslateAssembleIngestAccess Transfer Unpack Store Archive Process WDL ID Assignment Derivative Creation Directory & File Naming Assemble Metadata Components Content Components Ingest New Items Updated Items

The WDL Catalog Application TransferProcessDescribeTranslateAssembleIngestAccess Describe Original Metadata Mapping Metadata Normalization WDL Descriptions Translate Initial Translation to English if applicable Full Translation to all supported languages Continuing support for corrections

The WDL Web Application TransferProcessDescribeTranslateAssembleIngestAccess Akamai NetStorage Akamai Global Edge Network LC Server Solaris 10 MySQL Java Python File Sys Lucene DJango nginx Solr Apache memcached Squid Uncached Searches All Item Content Statically Rendered HTML

Recommendations - Technical Ensure the WDL website has the following features – Reliability – Scalability, especially with the increase amount of content – Has multiple access points – Powerful searching tools Multilingual Metadata and content – Deliver services to users with low bandwidth Provide full-text search on metadata and contents Enabling bulk transfer of metadata Automation of fetching objects and data Bandwidth and the transfer of large files Improve documentation on digitization

Recommendations - Technical Content transfer, metadata compliance, translation – Document current practice and desired improvements – Review existing tools and specifications. – Adopt tools and workflow that are scalable Improving the cataloguing tools to allow partners editing data Develop tools for quality control to ensure consistent use of terminology Discussions with Wikipedia for linking individual WDL documents with relevant Wikipedia pages

Recommendations - Content Current methods and tools used for transferring content and standardizing metadata needs to be more scalable Consistent use of terminology for translation: Cooperation with the Translation Working Group. Content selection guidance: Cooperation with the WDL Content Selection Working Group Improve documentation on metadata standards Asking for richer metadata to enrich functionality in the future

Recommendations- Policy Developing requirements and policies for data distribution Collaboration between WDL and Europeana Extending the languages of the WDL beyond the existing seven

Future Considerations Preservation – Role of WDL wrt providing preservation services – WDL could undertake a survey of existing preservation practices among partners. – WDL partners with preservation experience train other partners to build capacity for preservation and digital conversion. Sharing the WDL software through licensing, open source, APIs Website features – Mobile devices – Building a user interactive community – Allowing users to use and annotate the content – Display options for comparing items

Future Considerations TAWGs role towards new partners and partners with less experience: – Technology transfer – Training – Adopting a country/partner Establishing sub-groups to deal with particular issues Sharing relevant experiences – Social networking – Preservation – Metadata – Access

Thank You