e-IRG Data Management Task Force (DMTF) EGI-TF Wednesday 15th September 2010 Session “Data Management Requirements in EGI” On behalf of e-IRG and DMTF task force Matti Heikkurinen, e-IRGSP2 project

Contents A (very brief) introduction to e-IRG DMTF background Task force results e-IRG and Data after DMTF How to work with e-IRG?

Mission and Vision of e-IRG The e-Infrastructure Reflection Group (e-IRG) is an inter-governmental policy body comprising national delegates from more than 30 European countries The e-IRG mission is to pave the way towards a general-purpose European e- Infrastructure The vision for the future is an open e-Infrastructure enabling flexible cooperation and optimal use of all electronically available resources 3 Key publications in : e-IRG Data Management Task Force report 2009 (jointly approved by ESFRI and e-IRG); e-IRG White Paper 2009; e- IRG roadmap 2010; Forthcoming: e-IRG Blue Paper 2010 for ESFRI; e-IRG White Paper 2010

4 e-IRGSP1/2/3: Supporting e-IRG since 2005! -Support to e-IRG provided by e-IRG Support Programmes, co-financed by the 7 th FP Mission: speed up the internal processes of the e-IRG, and support e-IRG cohesion and consensus building Runs the e-IRG secretariat and supports the e-IRG events Engages in dissemination and liaison activities, Performs legal and financial foresight studies to further increase the visibility and socio-economic impact of e-IRG Supports the actions necessary for consensus-based policy formulation of the e-IRG Maintains a Knowledge Base that provides information about e-Infrastructure services, projects and users -Transition from e-IRGSP2 to e-IRGSP3 in Dec Secretariat moving from CSC-Finland to NCF in The Netherlands

e-IRG and EGI The e-IRG sustainability task force supported the launch of the EGI design study by proposing concrete steps towards the e-Infrastructures (and Grid) sustainability (including recommendations on such funding); The e-IRG white paper 2009 tackled the issue of sustainability of computing-related e-Infrastructures components EGI and PRACE The e-IRG white paper 2009 tackled the issue of Grid compared with Clouds e-IRGSP2 is currently conducting a financial study aiming to estimate the overall cost of the EGI grid (capital + operational expenditures) and compare it with the costs of using services from a commerical cloud. The e-IRG white paper 2010 will be working on Governance of major e- Infrastructures and e-Infrastructure services: hardware, software, clouds 5

DMTF background

DMTF design goals and constraints Planning started early 2008 –e-IRG board decision19th March 2008 Numerous data-related initiatives identified –Focus on surveying the landscape Goal setting needed to be realistic: –Capture key issues –Support cooperation between initiatives

Composition and management Membership –Core group from e-IRG delegates Chaired by Dany Vandromme –Large experts group gathered Contacts of the delegates Paris workshop participants –e-IRGSP2 as support organisation Subgroups –Survey of existing initiatives (Rudolf Dimper, Peter Wittenburg) 18 social sciences, 12 health sciences, 33 natural sciences –Metadata and quality (Peter Wittenburg, Peter Doorn) Requirements of metadata architecture, quality in specific contexts – Interoperability (Brian Coghlan, Peter Wittenburg) Cross-disciplinary aspects, opportunities and challenges Liaison with ESFRI –Task force supported by ESFRI from the start –Final report endorsed by e-IRG and ESFRI

Task force results

General notes “Heavyweight” document –110 pages –Numerous contributors (23 for survey alone!) The content by necessity a “snapshot” of the situation –Challenging to keep up to date –Initial input to populate a database? Review and consultation processes –Open call for experts –Public consultation version –Review by e-IRG and ESFRI Distribution –e-IRG member states, ESFRI, e-IRG website

Key findings Metadata is a key enabling tool for quality and interoperability. Goals: –Semantically interoperable –Available throughout the resource lifetime –Standardised, interdisciplinary, usable in aggregations Quality of data –Scope: discipline, universal standards are not seen as feasible –Individual researcher in a key role –Challenge interoperability! Interoperability –Programmes supporting cross-disciplinary access needed –Communities key driver, to be supported –Open standards and organisational guidelines needed

Survey of the Data initiatives Illustrate a somewhat fragmented field –Large number of domain-specific initiative and databases –Long-term sustainbility is a major issue Hardware and software infrastructure –Focus on curating for reuse rather than long-term preservation Emerging trends –Move towards distributed/federated data repositories –Open access –New, ambitious projects changing the landscape Future –More detailed information about project-specific requirements needed –Cross-disciplinary research increases demand for interoperability –Common data analysis tools emerging (e.g. GIS) –Communication and cooperation between data initiatives should be stimulated

Metadata and Quality of Data Focuses on scoping and refining the question –What is metadata? –How it is used? –What are the relevant standards? –What are the quality issues related to data (and metadata)? Highlighting difficult questions –Responsibility for creating & maintaining metadata and data This is not free! –Open access makes peer-review of data technically feasible But automatically feasible in practice (effort)- not a magic bullet Future directions –Data Seal of Approval 16 guidelines for assuring and verifying quality of data –Open access: OECD and Berlin Declaration

Interoperability issues Key issue: cross-disciplinary use of data –Case: SHARE project Focuses on aging population Relevant to health, economy, politics,... –Interoperability usually tackled within a discipline Resource-level and semantic interoperability –Resource levels: Device Communications Middleware Deployment strategies –Semantic interoperability Data integration Ontologies Several recommendations –From standardisation to organisational structures

e-IRG and Data after DMTF

DMTF follow-up activity A follow-up action is being prepared –Led by Norbert Meyer –Identify key issues for follow-up –Analyse post-DMTF developments Not a DMTF-II! –Lightweight organisation –Focus on activating community and liaise with other actors in data domain –Expedited approval process Disseminating the results –A stand-alone document or input for other e-IRG documents –Possible foundation for other data-related activities

Other data-related activities Roadmap recommendation 3.5 proposes –Prepare a blueprint for enabling data-intensive research –e-IRG committed its support Concrete steps being discussed The e-IRG Workshop (13th - 14th Oct) –Data Management session! Future White Paper documents –Data a possible topic

How to work with e-IRG?

Several channels to work with e-IRG “Update” mailing list: sign up! –Only major announcements –Workshops, public consultations, newsletters Contact your national e-IRG delegate –Listed on the e-IRG website Participate in the workshops For specific questions: contact secretariat Make sure that you are visible and heard! –e-IRG Knowledge Base –Public consultations

Thank you! Questions? Comments? To get in touch after this event: e-IRG secretariat: Matti Heikkurinen: