1. USING THE METHODOLOGY FOR EXTERNAL LEGAL EDUCATION QUALITY ASSESSMENT Training on behalf of USAID FAIR Justice project – 27 th and 28 th May 2015

2. Participants will be able to…  Explain the basic theory behind legal education quality assessments  Utilize the relevant basic research methods  Conduct an external legal education quality assessment using the methodology  Devise an internal legal education quality assessment for their own institution  Present the results of a legal education quality assessment accurately and effectively

3. What we mean by “Quality”…  Assurance?  Control?  Enhancement?  Assessment?  Audit?  Standards?  Culture?  Accreditation?

4. Two different historical models of quality assurance…  French  External control, under religious authority  Focused on accountability  More “extrinsic” idea of education  English  Self governing by society of fellows  Focused on peer review  “Intrinsic” idea of education

5. A general model of higher education quality assurance…  An independent “meta-level” managing agent for the quality assurance system  System based on “self-evaluation”  External assessment/peer-review  Reporting focused on helping institutions improve

6. Why externally assess legal education quality?  Standards  For safeguarding national academic standards  For accreditation purposes  Market reasons  For public provision of independently-verified information (quantitative and qualitative) about programmes or institutions  To allow for comparison in the law school market  Internal self assessment  To encourage reflection and internal self-assessment  To contribute to institutional improvement and strategic planning  To give credit for good work

7. Aspects of quality…  (1) quality as exceptional (e.g. high standards);  (2) quality as consistency (e.g. zero defects);  (3) quality as fitness for purpose (fitting customer specifications);  (4) quality as value for money, (as efficiency and effectiveness); and  (5) quality as transformative (an ongoing process that includes empowerment and enhancement).

8. Understanding the Methodology  Based on the European standards, guidelines and recommendations  This is a methodology for external assessment, but also useful for institutions devising internal assessments  It should continue to be developed and improved for Ukrainian context  Participatory in nature

9. Risks  Institution not fully cognizant of methodology  Institution sees assessment as a threat instead of an opportunity  Institution is not open to genuine full assessment  Institution seeks to offer “window-dressing”

10. Elements and Criteria of Assessment  Elements  Basic categories around which the assessment is structured  Criteria  Expressed as outcomes under each element

11. Elements of the LEQ Assessment  1: Policy and Procedures for Internal Quality Assurance:  2: Approval, monitoring and periodic review of programs and awards:  3: Admission and Assessment of students  4: Quality assurance of Teaching Staff  5: Curriculum, teaching methodologies, learning resources and student Support  6: Administration and Information Systems  7: Public information

12. Purpose of research…  To get a full, accurate “3D” picture of the institution, its strengths, weaknesses  From multiple perspectives  Using multiple sources

13. Respondent groups  Administrators  Teachers  Students  Alumni  Employers

14. Research Methods  Methodology specifies various methods to gather quantitative and qualitative information about the effectiveness of the institution  Document analysis  Surveys  Key Informant interviews  Focus group discussions  Classroom observation

15. Document analysis  Important for showing “formal” reality  Wide variety  Undertaken prior to visit  Informs questions

16. Key informant interviews  Semi-structured format, with one (max two) people  Establish context for further research  Advantages  Good for establishing basic facts  People may be more honest one on one  Limitations  Time consuming.  Might succumb to social desirability effect, answer in ways they think they interviewer wants them to answer, which can affect whether the information gathered is valid.  Limited number of people you can interview, so they may not be generalizable.

17. Interview best practices…  Devise protocol of relevant core standard questions for each respondent group  Avoid asking yes/no questions.  Use caution when asking “Why” questions…  Focus on “What and “How” questions

18. Focus Groups  Normally have between seven and ten participants  Why are they useful?  participants insights tend to “trigger” the sharing of others’ personal experiences  Provides diverse opinions and ideas.  What are some limitations of focus groups?  Susceptible to facilitator bias, which can undermine the validity and reliability of findings.  Can be dominated by a few vocal individuals.  Limited generalizability to a whole population.

19. Focus group best practices  Interviewer as moderator  Take place following interviews  Use to address “What could we do” questions  Areas of inquiry/development decided in advance  Open, informal atmosphere  Establish rules at beginning of group  Establish rapport

20. Traditional teaching methods  Teacher as information giver  Teacher-centered ; students passive  Emphasizes “what” questions  Memory is vital  Leads to conformity, students learn pre-determined outcomes.

21. Modern teaching methods  Teacher as facilitator  Active learning, student-centered  Stresses “Why” and “how” questions  Encourages investigatory and explanatory work  Allows students a degree of freedom an creativity  Uses variety of methods

22. END OF DAY ONE