1. _________________________________________________________________________ A new approach to exam management _________________________________________________________________________ Copenhagen October 2014 © 2014 Ektimo ApS - All Rights Reserved.

2. Ektimo ApS Examfolio is an initiative by Ektimo Aps. Bo Wolff – Martin Nielsen – Andy Beare Jonas Ranstam (Consultant Statistician) Based in Copenhagen, Denmark we have 12 years experience working with online peer review and evaluation software for academic organizations.

3. Collaboration with EBOT

4. 3-in-1 Role-based solution Questions 1. Authoring 2. Approval 3. Selection Exams 4. Administration 5. Monitoring 6. Feedback Analysis 7. Question Statistics 8. Exam Consistency 6. Candidate scores

5. Exam activities are integrated Questions ExamsAnalysis With Examfolio, we create a better synergy between the activities. It makes your exams better over time. QuestionsExamsAnalysis Traditionally, it can be difficult to improve exams, because processes often are detached from each other.

6. AUTHORING Writing committee sign in and author questions Questions can be evaluated pre and post exam Questions

7. APPROVAL Work on questions is organised into clear cycles of Reviewing Revising Approving For every exam held, feedback helps to improve questions Questions

8. SELECTION Question selection for exams is efficient with quick filtering. Safety features ensure no question can be selected twice… … and exams can be exported to external exam centres. Questions

9. Registration Candidates register or are pre-registered from a data file. Select from any open exams Can select a location (for invigilated exams) Receive email alerts Before the exam

10. ADMINISTRATION Administrators have a comprehensive overview of All exams Locations Candidates Before the exam

11. Exam delivery Web-based Locked down browser During the exam

12. Monitor Candidates are monitored remotely during the exam to ensure exam integrity… … which enhances qualification credibility for non-invigilated exams. During the exam

13. Feedback Candidates can see their ranking compared with others in their category. The scores and results can be presented immediately after the exam. After the exam

14. Results Results from exams can be used to compare candidates at different institutions or in different countries… … allowing organisers to deploy educational resources improve overall standards. Demographic analysis

15. Item Questions are analysed using: Point Bi-Serial correlation Discrimination index, Difficulty index Distractor efficiencies Item analysis

16. Consistency KR20 - Consistency Mean and SD Question performance Candidate performance Exam analysis

17. Cut off scores M/SD-1 75%/5 Modified Angoff IRT Item response theory (based on calculation of difficulty parameters) Exam analysis Modified Angoff methods: Estimating the proportion of borderline candidates that would answer the question correctly. How much Angoff? 1. Group committee voting (November 2014) 2. Panel of Judges (April 2015) Angoff strengths: Defensible and cut off not relative to the candidate group. Angoff Weaknesses: Difficult for panel to conceptualize a pool of borderline candidates. Resource ‘hungry’.

18. Program Data Cyber-Security Certification Security Other Program design: Limited user access and information presented on a need-to- know basis. Data: Encrypted web-traffic Secured data file output? Back-up Cyber-security: Penetration testing (external audit 2015) ISO – 27001 Certification: Best practices in company-wide security

19. Costs Other Questions database and analysis: € 2500 per year Formative and self-paced tests: € 15/20 per candidate Formal invigilated exams: € 30 per candidate

20. PHONE/TABLET Web-pages optimised for Smart-phone and tablet. Mobile

21. Complete OVERVIEW So, why Examfolio? Continual IMPROVEMENT To continually improve your questions and exam processes… … and get an insight into educational standards at departmental, national or international level. To have an up-to-the- minute overview of all your exam processes… … and reduce administative costs. Easy COLLABORATION To build and validate your pool of high quality questions… … and quickly create well balanced exams.

22. _________________________________________________________________________ Thank you _________________________________________________________________________ Ektimo ApS Gammeltoftsgade 12B, kld. 1355 Copenhagen K info@examfolio.com Try us for free @ examfolio.com