WP6 – Monitoring and Evaluation GA Meeting 21/22 January 2016 Partners: UK and Canada

Evidence Gathering Evaluation of the fast-track data project Analysis to understand changes in national funding Evaluation of the impact of JPI on international /worldwide collaboration Evaluation of the reach of the JPI in terms of its profile amongst stakeholders 1 st monitoring report 2 nd monitoring report 3 rd monitoring report

Evidence Gathering Evaluation of the fast-track data project Analysis to understand changes in national funding Evaluation of the impact of JPI on international /worldwide collaboration Evaluation of the reach of the JPI in terms of its profile amongst stakeholders 1 st monitoring report 2 nd monitoring report 3 rd monitoring report

First monitoring report (D6.1) Conducted approximately 6 months after the launch of the J-AGE II project –While the Implementation of the SRA still being worked through –While the JPI work programme still under development –While the first joint transnational call still in progress Electronic, anonymous online questionnaire to GA members Questionnaire open May - August 2015

Quality of Data 18 completed questionnaires were received 4/18 respondents only answered the first page of the survey 2 of these agreed with all the questions and 1 was undecided about all of the questions 2/14 respondents who gave answers to all questions ‘agreed’ with all the questions (and gave no open comments) 2/14 respondents who gave answers to all questions were ‘undecided’ about all questions (and gave no open comments) For an ‘internal’ questionnaire we would hope to get better responses

Areas of agreement Broadly it can be concluded that GA members are happy with the JPI activities such as: –The SRA –The two fast track activities –The first joint call The General Assembly members are also happy with the J-Age and J-Age II Coordination Support Action programmes More agreement than before that JPI activities reflect input of non-academics

Areas of uncertainty JPI influence on both research strategies and stakeholders in country uncertain Uncertainty amongst GA members re: satisfaction with their country’s level of involvement First call – some comments suggest development lengthy and agenda potentially diluted by inclusivity of all interests

Areas for Improvement Governance and decision making still remain areas of concern –Also highlighted in the external evaluators report –Acknowledge that some improvements have been made over the past year Further improvement needed on: –transparency of the decision making process –clarity of the roles and responsibilities of the bodies within the JPI

Evaluation of Fast-Track Activities An evaluation of the two fast-track projects - December 2015 to March 2016 – hopefully in time to inform new fast-tracks under discussion The work will include evaluation of the: –Planning and delivery of the two Fast Track activities –Impact of the Fast Track activities outputs –Contribution of the Fast Tracks to JPI aims and objectives –Identification of potential case studies of best practise –Recommendations to the JPI for future JPI activities Evaluators may be in touch with some GA members and some SAB and SOAB members

Making a difference Plan ItDo It Evaluate It

Making a difference Want to help the GA and Steering Committee manage the JPI activities For the outcomes of the monitoring and evaluation work the JPI should formally: –Consider –Document response –Identify a route for action (or not) Proposal: –Establish a working group to develop and implement actions to respond to the evaluation reports

Thank you for your attention

WP6: Tasks,deliverables and milestones Tasks –Task 6.1: Monitoring of indicators –Task 6.2: Development of evidence for external evaluation –Task 6.3: External evaluation Deliverables –D 6.1: First Monitoring Report (month 6 – August 2015) –D 6.2: Second Monitoring Report (month 18 – August 2016) –D 6.3: Third Monitoring Report (month 30 – August 2017) –D 6.4: External Evaluation of the JPI (month 36) Milestones –M30 Establishment of work programme of data gathering/tracking activities - month 3 (May 15) –M31 Review work program and refine if necessary -month 15 - –M32 Establishment of external evaluation group (month 28) June 2017

Refreshed External Advisory Committee Members : Wendy Loretto -University of Edinburgh, UK Annette Boaz - St. George's, University of London Ismael Rafols - Polytech Univ. of Valencia, Spain Peter A.A. van den Besselaar - VU University Amsterdam, Department of Organization Sciences and the Network Institute. Stefano Bonassi - Head, Area of Clinical and Molecular Epidemiology, IRCCS San Raffaele Pisana Michael Dinges - Senior Expert Advisor, Innovation Systems Department Research, Technology & Innovation Policy Unit AIT Austrian Institute of Technology GmbH Margaret Macadam, ESRC Marc Turcotte, CIHR

Reach, profile and Influence of the JPI The profile of the JPI and its reach outside of the JPI members is an area to focus on through the J-Age II period. We were unable to give a response to the Proxy indicators as information was not available in time for inclusion in the report. PI 1 - Stakeholders engaged in each of the priorities identified in the SRA. Success Criteria – The JPI is engaged with 50 stakeholders/stakeholder groups in each priority area of the SRA. PI2 - Increase in the number of stakeholders taking part in JPI MYBL activities. Success Criteria – Each JPI event involves at least 30% stakeholder attendees. PI3 - JPI visibility in communication and dissemination media, Success Criteria – Visibility of JPI MYBL activities in European, regional, national, and international media