Peer-Review of Student Centred Learning using QA procedures as an instrument for change Anthony F. Camilleri SCL Training – 19 th December 2014 Brussels. pascl.eu

Which teacher changed your life? Did you have a year of your life you hated because of bad teaching?

Elements of Quality Assurance comparison of activities with a standard monitoring of processes measurement of outcomes iterative improvement prevention of errors pascl.eu

A look at the Quality Cycle DoCheck PlanAct アク ション 計画 実行 チェッ ク CHECK should be read as STUDY in English Check implies ‚to hold back‘ (Deming, 1980) pascl.eu

How does QA help? QA is a powerful tool to accelerate and systemise change pascl.eu

Application comparison of activities with a standard monitoring of processes measurement of outcomes iterative improvement prevention of errors use these tools to improve SCL within an Institution pascl.eu

How we are Approaching It Institution Applies for QA Procedure Completes Self Assessment PASCL nominate review team Review Team Assesses Institution Recommendations for Improvement CHANGE pascl.eu

Process Requires Commitment Institutional Prerequisites pascl.eu

LEARNING FROM PEER-REVIEW pascl.eu

Who is a peer? an expert in the field an expert in what field? Student-Centred Learning What is student-centred learning? pascl.eu

Features of an Expert experience in course delivery familiar with best-practice comparative perspective sense of mission natural communicator pascl.eu

Features of a Reviewer Ethical Open-minded Diplomatic Observant Perceptive Versatile Tenacious Decisive Self-reliant Acting with Fortitude Open to improvement Culturally Sensitive Collaborative pascl.eu

A review is about communication University Agency Review Team pascl.eu

„I don't know anything about art, but I know what I like“ Gelett Burgess pascl.eu

„I don't know anything about the standards, but I know quality when I see it“ Unnamed reviewer pascl.eu

Standards of Proof Some credible evidence Preponderance of evidence Clear and convincing evidence Beyond reasonable doubt pascl.eu

How to go about a Review 1.Re-familiarise yourself with the standard. 2.‘Take a Walk’ around the materials and websites of the applicant 3.Begin Review, slowly and progressively. 4.Keep a Questions / Doubts pad Note missing evidence Note inconsistencies 5.Check your Work pascl.eu

William E. Deming pascl.eu

NEVER leave questions unanswered breach confidentiality compare directly with your institution pascl.eu

Standards of Compliance Substantial compliance Partial compliance Non-compliance Report On: Non-compliance Opportunities for Improvement Best Practice pascl.eu

In your Report Be yourself Bad: The institution showed.... Good: The review team saw / found / observed.... pascl.eu

In your Report Be specific Bad: We observed a lot of.... Better: The review team found multiple and consistent examples of pascl.eu

In your Report Say what you know Bad: The institution gave us wrong information. Better: The review team found inconsistencies between evidence (x) and interview (y) pascl.eu

In your Report Give your opinion where relevant Bad: The institution is…. Bad: No Comment Better: We suspect, It seems likely that, Given the evidence available, etc... pascl.eu

In your Report Give your reasoning Bad: There is no Quality Better: When we consider (X), (Y) and (Z), we feel it impossible to say there is a well functioning quality cycle pascl.eu

In your Report Link Effect with Cause OK: Quality systems are in place, but there is no evidence of iterative improvement Better: Quality systems are not effective, due to lack of procedures to mainstream improvement. pascl.eu

In your Report Be Clear OK: The physical conditions of classrooms are in need of improvement Better: Classrooms are in a dismal state – no heating, broken desks and no boards lead to an environment which highly disincentivises learning pascl.eu

Let’s Hear your Jokes

How to learn from self- assessment 1.Read the quality standard 2.Understand the quality standard 3.Ask Questions 4.DO NOT fill it in 5.Discuss it – with all stakeholders in the institution a.How do we meet the standard? b.What can we do better? c.Does it apply in all areas? 6.Discuss it some more – do some implementation 7.Have you internalised standard? 8.Now fill in the questionnaire

Be Honest „Most Quality Managers are Magicians“ Look Here don‘t look here!

What will self-assessment produce A time to think, communicate and strategize A better understanding of institutional processes An overview of where we are and hopefully where we need to go A report

Success Factors Single point of Contact Unit or Individual Leading the Process Institutional Mandate / Commitment Clear Support from Institutional Management Inform your stakeholders Involvement of External Stakeholders External viewpoint of your processes are important Integration with existing Quality Tools Map the criteria against your existing standards

Success Factors (2) Methods to Collect and Act on Evidence Learning Analytics QA Reports Internal Development Plans Emphasis of Quality Improvement how are you integrating PASCL Guidelines into your mainstream activities? What changes do you plan to institute? Can you show us evidence of processes in flux?

The Show- Off The Politici an The Grump y The Bored Watch out for these ppl…

How to approach a peer-review Understand the purpose of the review Understand your role in the review Your impressions Of your part in the instituion Be candid and informative Don’t try to hide things (you’ll be caught)

Main result of peer-review an external evaluation of your institutional quality (procedures) a report a certificate pascl.eu

Run the cycle again DoCheck PlanAct アク ション 計画 実行 チェッ ク pascl.eu

Remember NO process is without errors There is ALWAYS room for improvement pascl.eu

What institutions should do with the report 1.Read the report 2.Understand the report 3.Ask Questions 4.Discuss it – with all stakeholders in the institution a.What did we miss in our Self-Assessment? b.Have our assumptions been confirmed or refuted? c.How can we do it better 5.Discuss it some more – do some implementation 6.Have you internalised the recommendations? 7.Now start the cycle again. pascl.eu

Principle Only completing the circle brings about real improvement pascl.eu

Foundations of Auditing Integrity – the foundation of professionalism Fair presentation – the obligation to report truthfully and accurately Due professional care – the application of diligence and judgment in auditing Independence - the basis for the impartiality of the audit and objectivity of the audit conclusions Evidence-based approach - the rational method for reaching reliable and reproducible audit conclusions in a systematic audit process ISO 19011:2001 pascl.eu

Under the following conditions: Attribution — You must give appropriate credit, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use. Share Alike — If you remix, transform, or build upon the material, you must distribute your contributions under the same license as the original.. Released under a Creative Commons Attribution ShareAlike 4.0 International License You are free: to Share — copy and redistribute the material in any medium or format to Remix — remix, transform, and build upon the material for any purpose, even commercially. This project has been funded with support from the European Commission. This publication reflects the views only of the authors, and the Commission cannot be held responsible for any use which may be made of the information contained therein. pascl.eu THANK-YOU for your attention Feedback is welcome! Anthony F. Camilleri