REGISTRATION AND EVALUATION OF ART AND DESIGN RESEARCH OUTCOMES State of the arts ECOOM-research project ‘Research in the Arts’

Overview 1.Research project: objectives 2.Research project: methods 3.Research project: results 4.Test Case 2016 (1) 5.Test Case 2016 (2) 6.Test Case 2016 (3)

1. Objectives ECOOM-VUB: ‘Research in the Arts’ 1. architecture and design 2. performing arts: dance en theatre 3. music 4. audiovisual arts 5. visual arts 6. interdisciplinary research outcomes Objectives: development of (1) an appropriate registration system (central =>FRIS) (2) an appropriate evaluation system A&DRO = output indicators => quality/impact

ECOOM-VUB projects ‘Output indicators for art and design research outcomes’ ‘Database art and design research outcomes’ - VUB-collaborators currently working on the project: Julie Debrauwere, project team member Walter Ysebaert, senior researcher Birgit Martens, project cordinator - ECOOM funding + VUB funding

2. Applied methods Stakeholder discussions: with delegation representatives Schools of Arts + University Faculties (architecture & design) Bilateral consultations Consultation & exchanges with international experts Benchmark & analysis of international evaluation systems

3. Results Design evaluation framework: state of the art(s) – Selection & definition output indicators – Evaluation framework design: peer review process Central registration system: FRIS -General agreement on the definition of A&DRO = result of research (documented!) + dissemination -Art and design research outcome: specific research type = which diverges from traditional output in FORM + COMMUNICATION of research findings -Categories: artefact/design/event -Research roles: creator/performer/contributor -Additional fields for evaluation purposes: 4 Design test case: -required for the identification of critical issues (registration + evaluation) -data gathering and data analysis: urgent need for concrete, tangible data (many unknown elements)

4. Test case (1) PHASE 1: Participating Faculties and Schools of Arts - individual researchers => registration - descriptive fields + review specific fields: (research context, impact description, impact reference) - validation - FRIS PHASE 2: PEER REVIEW 1 - peers: individual screening of results => selection of 3 RR - peer panels (#5): collective selection of RR => peer visit PHASE 3: PEER REVIEW 2: peer visit - presentation RR peer panel selection - presentation RR selection Faculties – Schools of A. - peer panel discussion Joint discussion peer panel presidents + steering committee PHASE 4: ASSESSMENT OF EVALUATION Peer panels: reporting (1) research outcomes (2) registration + PR process

4. Test case (2) (1) Key issues: registration - agreement ‘result of research’ not praxis - but research component: not ‘visible’ in case of A&DRO - have to be explicited and documented - international systems:design objects/art works + extra documents (REF)  research context: R questions, theory, methodology, …  addition of persistent link to art work/design As a consequence: important questions to be addressed = test case: essential learning experience  (how) can research component be deduced from information provided in the additional fields?  researcher’s rhetorical skill vs. research skill?  insight: case-based reasoning + argumentation peers

4. Test case (3) (2) Key issues: evaluation - no uniform and homogeneous quality standard - recognition for heterogenity:  different research policies  research cultures => solutions: research policy document + panel composition - quality and impact indicators: defined in qualitative terms - originality - rigour - significance - representation of research findings (communication form) - consistency outcome with research policy - collaborations - impact (praxis, societal, research community)  Peer interpretation of indicators? => observation instrument  Great diversity => comparisons? (weighing against formulated objectives)

4. Test case (4) (1) Assessment of registration + evaluation (2) Learning experience: - Data gathering opportunity -Data analysis -Baseline study => does evaluation design work in a way that contributes to the advancement of art and design research which is relevant for the concerned research communities, practioners, education.

Thank you for your attention!