1. E-assessment standards in the Netherlands Bart Ratgers - Enterprise architect Nynke de Boer - Manager Computer Based Testing Henk Nijstad Domain Manager Educational Standards

2. 2

3. INTRODUCTION CvTE – College voor Toetsen en Examens – Exam board as part of the Ministry of Education in the Netherlands – Responsible for the content and implementation of all national tests and exams – Nynke de Boer – Manager Computer Based Exams – Bart Ratgers – Enterprise Architect 3

4. FACET Testing system created in house for the national tests and exams in the Netherlands Installations: 2000 schools in in 2016, 7000 schools in in 2018 Deliveries: 1,200,000 in 2016 and 2,500,000 in 2020 Specialities: – Advanced item types that can be scored automatically – Innovative ways of adaptive testing – Reporting on multiple dimensions – Online as well as offline delivery – Using QTI as the interoperability standard – Built as open source 4

5. Henk Nijstad Edustandaard / Kennisnet IMS Global LIS 2016 EDUSTANDAARD: ‘THE IMS OF THE NETHERLANDS’

6. HENK NIJSTAD Kennisnet (The Netherlands) Edustandaard H.nijstad@kennisnet.nl

7. EDUSTANDAARD: SCOPE K12  HE Public & private All ‘chain processes’ in education: – administrative / financial / accountability – learning materials (distribution, use, etc) – assessment domain – personalised learning STANDARDS & CHAIN ARCHITECTURE development & use

8. ALL STAKEHOLDERS INVOLVED School board of K12, vocational, HE) Educational publishers, distribution & content Ministery of Education Exam organisations (CVTE, CITO, …) Student administration VLE’s … … In the education ecosystem (‘chains’) there is not a single authority who decides & determines

9. ORGANISATION (EDUSTANDAARD) Standaardisation board Architecture board 12 Working groupes Consistency on architectural level, TOGAF, best-practices Development of standards; facilitating the use; promotion Executive level

10. STANDARDISATION PROCESS: TRANSPARANCY & OPENNESS Based on BOMOS: framework for open standards Openness & transparancy: all stakeholders should be heard and can have influence Standards should be tested, implemented & used

11. STANDARDISATION PROCESS Based on BOMOS: framework for open standards Openness & transparancy: all stakeholders should be heard and can have influence Standards should be tested, implemented & used Standardisation board Standardisation board Architecture board Architecture board Working groups Support office advice

12. MAIN THEMES – 2016, 2017 IAA – identities, authentication, authorisation Privacy & security Semantics: how can we speak the same language in all those processes And: Efficiency & effectivity; speed of innovation

13. ASSESSMENT: STANDARDS QTI in The Netherlands Dutch Exam Profile (DEP) Semantics in ‘toets-domein’ Exchange of learning results New standard / frameworks like…. Caliper / XAPI ?, OneRoster ?

14. ASSESSMENT: ARCHITECTURE  CONSISTENCY, OVERALL COHERENCE (chain) Processes Services (public ánd private) Semantics, information objects Student & his identity Privacy & security Building blocks like FACET Use of standards, f.e. exchange of learning information (results) New standards like OneRoster, Caliper - xAPI IST und SOLL Comply or explain

15. 15

16. THE LIFE CYCLE OF THE DEP STANDARD 16 1.Adoption QTI 2.DEP as profile of QTI 3.Advanced items in DEP 4.Harmonizing DEP with QTI 5.Our approach: Adoption of DEP extension in new QTI releases

17. ADOPTION QTI 17 QTI 2.1 public draft was a good starting point for adopting an interoperability standard for exam content – QTI gave us the opportunity to define testparts, sections and items. – QTI gave us the opportunity to define a structured way to process responses. – CP gave us the opportunity to exchange the exam content But……… that was nog enough! – We need a way to relate several tests to each other in content package (exams/tests can be performed as a re-examination of another exam/test) – We need a way to define a strict lay-out – We need a way to define advanced adaptive testing – We need advanced item types that can automatically be scored. 1

18. DEP AS PROFILE OF QTI We created a QTI profile with the name Dutch Exam Profile (DEP) The DEP is fully documented. The documentation is available via EduStandaard website https://www.edustandaard.nl/ standaarden/afspraken/afspra ak/dep/4.0/ https://www.edustandaard.nl/ standaarden/afspraken/afspra ak/dep/4.0/ The DEP validation tool is provided by IMS: http://validator.imsglobal.org/ qti/ http://validator.imsglobal.org/ qti/ 18 2

19. ADVANCED ITEMS IN DEP 19 3

20. FIRST STEPS TO STANDARDIZE THE DEP 20 4 Separate the specification of the use from the specification of the content. The use is a specific need designed for CvTE and less relevant for content providers.

21. ADOPTION OF DEP EXTENSION IN QTI Our goal for the near future (if possible): To minimize the number of extensions or eliminating the DEP by: – Develop a generic standard in QTI for adaptive testing that can fulfill the requirements of CvTE (existing workgroup on CAT) – Investigate the capabilities of aQTI to realize our strict ideas on item layout – Investigate the capabilities of the new QTI versions to adopt some custom interaction types. – Investigate the need (outside Facet) for a specification for tool handling, like: calculators, notepad, spellcheck, symbol picker, formula list, ruler, triangle or protractor – Investigate the need outside Facet for advanced math items that can be automatically scored by use of Computer Algebra Service. We gained a lot of knowledge and we like to share it with you 21 5

22. Questions? 22