Excellence in Research for Australia 2015 Update ARC/NHMRC Research Administrators Seminar 16 November 2015 Presented by Sarah Howard Director, Research Evaluation

Web: arc.gov.au | ERA 2015 — where are we up to?  Currently finalising evaluation  Preparing State of Australian University Research: ERA National Report 2015 – 16 Submission review  Highlights  Challenges  Review process Introduction

Web: arc.gov.au | Not yet…. Evaluation

Web: arc.gov.au | WHAT? State of Australian University Research: Volume 1 ERA National Report 2015–16 Includes additional analysis of ten (10) year longitudinal dataset (2003–2013) ERA 2010, ERA 2012, ERA 2015 comparisons PLUS—additional volumes to come…applied research, gender, open access WHEN? Early December… ERA National Report

Web: arc.gov.au | ERA 2015 Submission Review

Web: arc.gov.au | Submission Upload Stats (excluding Stage 0) InstitutionUpload Attempts Highest nd Highest rd Highest th Highest th Highest th Lowest 17 4 th Lowest 16 3 rd Lowest 15 2 nd Lowest 13 Lowest 13 Grand Total 2396

Web: arc.gov.au | ARC-RO collaboration RO staff worked very hard—thank you! ARC ERA case managers—productive relationships between ERA staff and ROs Great feedback from ROs—thank you! Stage 0—continues to be successful Your feedback—thank you! Submission Highlights

Web: arc.gov.au | Submission completed on time Quality of submissions very high We saw improvements in:  data quality  repository readiness  explanatory statements  NTRO research statements Submission Highlights

Web: arc.gov.au | Purpose is to allow universities to upload trial submissions to test XML data structure Opened 19 January 2015 to 19 February 2015 All business rules were available - except BR024 and BR117 relating to EID tagging New report—30% ERA peer review sample Units of evaluation were not created Submissions uploaded did not carry forward to Stage 1 40 of the 41 universities used Stage 0 There were a total of 918 submission uploads in Stage 0 Stage 0

Web: arc.gov.au | Some unis considered the Stage 1 deadline a ‘soft deadline’ and did not provide a complete submission that we could check in Stage 2 This caused delays and increased pressure on everyone (ARC and unis) re data checking in Stage 2 Incentives for unis to move through Stages 1 and 2 before the deadlines? Allowing staggered upload of different indicators? Easter! If only we could have moved that to July ARC gives 9 months notice. Extending timelines compromises evaluation timing Submission Stages 1–2

Web: arc.gov.au | We are looking to improve the process, including… EID tagging—better guidance on allowable article types, citation provider interactions (data corrections and timing) Repositories—citation/alt metrics on landing pages, are metadata links and landing pages required? possibly a direct link to the output (many unis did this for ERA 2015)? —better guidance on what additional information can be provided (e.g. not a theatre programme) Submission Challenges

Web: arc.gov.au | There is always room for further improvement… data quality—in general repository readiness—REC members reported some issues explanatory statements—no alt metrics etc. NTRO research statements  variable quality  REC members reported some were not at all helpful  key issue is identifying the research component Submission Challenges

Web: arc.gov.au | We have collected and collated your feedback through submission—thank you! We will be conducting a range of other review and consultation processes through 2016— we will let you know… Submission Review Process 2015–2016

REC Member and Peer Reviewer Recruitment Research Evaluation Committee (REC) Membership 8 REC Chairs announced in November—to ensure ERA evaluations are of the highest standard Full list of RECs members can be found on the ARC website Peer Reviewers The call for Peer Reviewer nominations closed on 26 November 2014 For ERA 2015 we had ~ 1300 peer reviewers—even better than 2012! We assigned material to ~ 1100 peer reviewers. ~ 1000 peer reviewers submitted reports.

ERA-HERDC Alignment Update Consultation paper on options for alignment (8 December–13 February) Sector working group (3 July–21 September) Draft specifications completed Next steps — sector consultation on draft specifications, when…?

Questions?

Thank You Web: arc.gov.au |