ERA and Outputs Presentation for School of Music Lorna Robertson Manager, Research Assessment Unit 23 November 2010

Presentation ERA outline Outputs, especially creative works Suggestions for School of Music

ERA purpose ERA is designed to inform the Government and community about the quality of the research being funded by the Australian tax-payers ERA assesses research at the discipline level, by 2-digit and 4-digit Field of Research (FoR) codes, not by organisational unit Results from ERA 2010 will significantly affect reputation and drive funding to universities from 2011 onwards Results at institutional level will be published nationally in early 2011 ERA 2012 has been announced, likely census date 31 March 2011

ERA 2010 Data required for outputs/publications, grant income, applied indicators (patents, registered designs, commercial income), specified esteem, and staff Outputs are from ERA eligible researchers, published in the period 2003 – 2008 All other indicator data for 2006 – 2008 Creative outputs were included, eg poetry, music compositions and performances, visual arts and exhibitions; where selected for peer review, research statements and evidence for these had to be submitted Background statements for each 2-digit FoR area were written to describe the context and nature of the work

ERA – Researcher Eligibility Complex definitions for ERA 2010 Essentially, eligible researchers were T&R and RO staff employed at UWA full-time or fractional full-time (regardless of publication byline) on 31 March 2009 Some non-salaried honorary, adjunct, emeritus, or seconded researchers were also eligible, provided they showed UWA affiliation in their publication/output byline (or equivalent in the case of creative works) If ERA12 definitions are similar, eligibility will be based on researchers affiliated with UWA on 31 March 2011

ERA 2010 continued All data had to be submitted according to 4-digit Field of Research codes There is no 1-1 correspondence between FoRs and schools Schools contribute to lots of FoRs; many FoRs have data from lots of schools 1904 (Performing Arts & Creative Writing) is the FoR for Music and also Creative Writing in SCS

ERA 2010 continued Outputs the core indicator Ranked C1 journal articles is a quality indicator for all disciplines, using ARC’s ranked journal list For sciences, citation data from Elsevier’s Scopus is a key measure to indicate performance against Australian and World average performance For non-sciences, key assessment is by peer review of 20% sample of outputs by FoR ERA Evaluation Committees have a ‘dashboard’ with information for each FoR by university on outputs, staff profiles, grant income, applied indicators and esteem

ERA 2010 Rating Scale RatingDescriptor 5 The Unit of Evaluation profile is characterised by evidence of outstanding performance well above world standard presented by the suite of indicators used for evaluation. 4 The Unit of Evaluation profile is characterised by evidence of performance above world standard presented by the suite of indicators used for evaluation. 3 The Unit of Evaluation profile is characterised by evidence of average performance at world standard presented by the suite of indicators used for evaluation. 2 The Unit of Evaluation profile is characterised by evidence of performance below world standard presented by the suite of indicators used for evaluation. 1 The Unit of Evaluation profile is characterised by evidence of performance well below world standard presented by the suite of indicators used for evaluation. NA Not assessed due to low volume. The number of research outputs does not meet the volume threshold standard for evaluation in ERA.

ERA 2010 – Income and Applied Indicators Eligible research income was that received by UWA in and reported by UWA for HERDC, affiliated with UWA, not with eligible researchers Eligible applied measures were specific ARC-listed measures affiliated with UWA – except for NEGs, which were affiliated with the researcher: –Plant breeders' rights –Patents –Registered designs –Research commercialisation income –NHMRC-endorsed guidelines

ERA 2010 – Esteem Esteem had specific ARC-defined categories: Editor of a prestigious work of reference (specified list) – only HCA and SBE cluster Fellowship of a Learned Academy and Membership of AIATSIS Recipient of a Nationally-Competitive Research Fellowship (specified list) Membership of a Statutory Committee – only PAH cluster Recipient of Australia Council Grant or Fellowship – only HCA FoR19

ERA 2010 – Outputs Had to include at least one eligible researcher as author Had to be published in Specific eligible output types only: –Traditional output types as for HERDC: Research books (A1) Research book chapters (B1) Refereed journal articles (C1) but only in ARC ranked journal list Refereed conference publications (E1) –Non-traditional research outputs also in some disciplines: Original creative works (J1-J5) Live performance of creative works (J6) Recorded/rendered creative works (J7) Curated/produced substantial public exhibitions/events (J8)

ERA 2010 – Creative Outputs 1 All creative works must include a research element. Original creative works (J1-J5) include novels, short stories, musical compositions, artworks and designs. Live performance of creative works (J6) includes musical or dramatic performances or productions, where the research and creativity is in the performance. The performance must be of a new work or a demonstrably new or innovative interpretation or production of an existing work. –Documentation to substantiate the performance is required, such as a recording of the performance, reviews, performance programs, etc. –At least one of the performers must be an eligible researcher affiliated with UWA. –For non-salaried researchers (eg honorary), UWA affiliation must be demonstrated in relation to the work.

ERA 2010 – Creative Outputs 2 Recorded/rendered creative works (J7) includes works where the research component is contained within the recording/rendering, not the performance. Examples include films, digital games, creative 3D models. Curated/produced substantial public exhibitions/events (J8) includes works where the eligible researcher is the curator, not the creator. Examples include: –Exhibition: The curation and/or production of a collection of creative works exhibited together for the first time, in that particular arrangement, in a recognised gallery, museum, or event. This should be accompanied by a well researched publication that includes the date and location of the exhibition. –Festival: The curation of a festival bringing together innovative work or existing works in an innovative format or through a theme that provides new perspectives and/or experiences.

ERA 2010 – Creative Outputs 3 Creative works for peer review in ERA required submission of the output and/or documentation substantiating its existence, plus a short Research Statement (approx 250 words) including: 1. Research Background Field Context Research Question 2. Research Contribution Innovation New Knowledge 3. Research Significance Evidence of Excellence A good test of whether a creative output meets the criteria is if a research statement could be written

ERA 2010 – Creative Outputs 4 Significant work done for Music outputs for ERA Trial 2009 and ERA10 by my office with School of Music: Collection of information on outputs meeting existing UWA criteria (J1-J2) – lots missing Collection of information on outputs meeting new ERA criteria (J6-J8) – some available in OPVA files but records very incomplete Audit of new outputs Preparation of research statements and evidence portfolios for 20% sample A BIG job under pressure!

ERA 2012 – Creative Outputs ARC say ERA12 will use the same methodology as ERA10. UWA will continue to collect creative outputs for audit, using the ERA criteria. Details of outputs meeting the ERA criteria should be entered in UWA’s publications database each year. Schools must retain records of all outputs for the past 7 years for ERA including: –Scores of new compositions –Recordings –Programs –Reviews –Any other evidence of the performance, exhibition or festival Full records are needed for UWA audit and ERA submission.

Suggested action now Be aware of likely ERA12 census date 31 March 2011 in relation to staff appointments, including honorary and short term appointments. Work on publishing research in high quality outlets (journals, publishers, venues) Ensure all research outputs meeting the stated criteria are entered in UWA’s publications database. Definitions for creative works categories are being updated for clarification and consistency. Ensure full records of all creative works are gathered and retained by School of Music. Work done for ERA 2009 Trial and ERA 2010 should be continued. Electronic and hard copies recommended. Looking at ERA10 research statements and evidence portfolios for outputs selected for peer review may be helpful.

Further Information ARC ERA website UWA ERA website ERA Submission Guidelines Research Assessment Unit: –Lorna –Lizzy