_________________________________________________________________________ A new approach to exam management _________________________________________________________________________ Copenhagen October 2014 © 2014 Ektimo ApS - All Rights Reserved.

Ektimo ApS Examfolio is an initiative by Ektimo Aps. Bo Wolff – Martin Nielsen – Andy Beare Jonas Ranstam (Consultant Statistician) Based in Copenhagen, Denmark we have 12 years experience working with online peer review and evaluation software for academic organizations.

Collaboration with EBOT

3-in-1 Role-based solution Questions 1. Authoring 2. Approval 3. Selection Exams 4. Administration 5. Monitoring 6. Feedback Analysis 7. Question Statistics 8. Exam Consistency 6. Candidate scores

Exam activities are integrated Questions ExamsAnalysis With Examfolio, we create a better synergy between the activities. It makes your exams better over time. QuestionsExamsAnalysis Traditionally, it can be difficult to improve exams, because processes often are detached from each other.

AUTHORING Writing committee sign in and author questions Questions can be evaluated pre and post exam Questions

APPROVAL Work on questions is organised into clear cycles of Reviewing Revising Approving For every exam held, feedback helps to improve questions Questions

SELECTION Question selection for exams is efficient with quick filtering. Safety features ensure no question can be selected twice… … and exams can be exported to external exam centres. Questions

Registration Candidates register or are pre-registered from a data file. Select from any open exams Can select a location (for invigilated exams) Receive alerts Before the exam

ADMINISTRATION Administrators have a comprehensive overview of All exams Locations Candidates Before the exam

Exam delivery Web-based Locked down browser During the exam

Monitor Candidates are monitored remotely during the exam to ensure exam integrity… … which enhances qualification credibility for non-invigilated exams. During the exam

Feedback Candidates can see their ranking compared with others in their category. The scores and results can be presented immediately after the exam. After the exam

Results Results from exams can be used to compare candidates at different institutions or in different countries… … allowing organisers to deploy educational resources improve overall standards. Demographic analysis

Item Questions are analysed using: Point Bi-Serial correlation Discrimination index, Difficulty index Distractor efficiencies Item analysis

Consistency KR20 - Consistency Mean and SD Question performance Candidate performance Exam analysis

Cut off scores M/SD-1 75%/5 Modified Angoff IRT Item response theory (based on calculation of difficulty parameters) Exam analysis Modified Angoff methods: Estimating the proportion of borderline candidates that would answer the question correctly. How much Angoff? 1. Group committee voting (November 2014) 2. Panel of Judges (April 2015) Angoff strengths: Defensible and cut off not relative to the candidate group. Angoff Weaknesses: Difficult for panel to conceptualize a pool of borderline candidates. Resource ‘hungry’.

Program Data Cyber-Security Certification Security Other Program design: Limited user access and information presented on a need-to- know basis. Data: Encrypted web-traffic Secured data file output? Back-up Cyber-security: Penetration testing (external audit 2015) ISO – Certification: Best practices in company-wide security

Costs Other Questions database and analysis: € 2500 per year Formative and self-paced tests: € 15/20 per candidate Formal invigilated exams: € 30 per candidate

PHONE/TABLET Web-pages optimised for Smart-phone and tablet. Mobile

Complete OVERVIEW So, why Examfolio? Continual IMPROVEMENT To continually improve your questions and exam processes… … and get an insight into educational standards at departmental, national or international level. To have an up-to-the- minute overview of all your exam processes… … and reduce administative costs. Easy COLLABORATION To build and validate your pool of high quality questions… … and quickly create well balanced exams.

_________________________________________________________________________ Thank you _________________________________________________________________________ Ektimo ApS Gammeltoftsgade 12B, kld Copenhagen K Try us for examfolio.com