Introduction to Qualifications Wales’ Good Practice Guide on Mark Schemes in Knowledge-based Qualifications

What is the purpose of the guide? Supporting qualifications developers when designing, developing and reviewing mark schemes Intended to complement awarding bodies’ own development processes It is not a regulatory document

Why did we produce it? Lots of research and literature on mark schemes available, but no practical tools Varied level of expertise in qualifications community Supportive regulation

How did we produce it? AlphaPlus Consultancy awarded contract via competitive tender process - access to a lot of assessment expertise Review of literature currently available about mark schemes – didn’t want to ‘reinvent the wheel’ Discussions with some awarding bodies and subject experts involved in qualification development and reform (GQs and VQs)

How do we intend it to be used? Practical, interactive tool Help awarding bodies consider key principles that underpin all knowledge-based mark schemes Help awarding bodies consider principles of particular types of mark schemes – points-based and levels of response Take awarding bodies through issues to consider when developing specific types of mark schemes – e.g. ‘Hurdles’

Next steps: Currently finalising translation and interactive version Will be published in early 2019 Literature review will also be published Will keep it under review – welcome feedback from Awarding Bodies on usefulness of guide

How to use the Mark Scheme Guide The content map is an alternative to the table of contents and each block is clickable to provide a direct link to the relevant content. The map shows all the content in the Guide diagrammatically, showing the hierarchy of the content and how it all relates to each other. The map shows the difference in complexity between simple low-tariff mark schemes and more complicated high tariff ones. It also shows how the issues to consider increase as the number of marks increase. At the bottom of each page there is a control panel which takes you back and forth to the main sections of the Guide.

Principles of mark scheme design This sections starts with more high-level, fundamental principles of assessment e.g. there needs to be a clear relationship between the item and the mark scheme and marking should be positive, and builds in complexity towards more detailed information about how to ensure validity and reliability. The principles don’t suggest anything radical or un-tested but are based on established research and good practice on validity, reliability and manageability. Complexity

Selecting the type of mark scheme The structure of the Guide has been designed to provide a logical and intuitive process for developing different types of mark schemes. It takes you through different options and routes depending on what you’re trying to assess. If you choose an incorrect route through, it will provide options for getting to the right sort of mark scheme for your purpose. Selecting the type of mark scheme

ANIMATED SLIDE: Starting with a simple controlled response item, the links take you to a section with guidance on developing these sorts of assessments. The importance of the close relationship between item and mark scheme can be seen in the guidance. There’s then the option of linking to an example of a controlled response item and mark scheme, followed by a checklist (CLICK) which asks key questions to ensure you’ve considered everything you need to before moving on to designing your next item.

As Constructed response mark schemes can vary by number of marks and design, this section has more options and links.

The main process for this section of the Guide is: you will be taken to a short piece of text with guidance on designing mark schemes for this particular tariff item followed by (CLICK TO NEXT SLIDE) a series of steps or questions designed to make sure you’ve selected the right one for the purposes of what you’re trying to assess.

You’ll notice that there are helpful suggestions for working out what to do if you’ve selected the wrong item and mark scheme, plus links to other useful sections and to the Glossary to explain key terms.

ANIMATED SLIDE: As the tariff increases, the level of complexity and scope of points to consider widens: do you want a points-based mark scheme (CLICK) or a levels of response mark scheme (CLICK)? If you don’t know what these are…CLICK TO NEXT SLIDE

You can go back to the map and find out more detail in the grey boxes here. CLICK TO NEXT SLIDE

PAUSE FOR THREE SECONDS AND CLICK TO NEXT SLIDE

WAIT THREE SECONDS AND CLICK TO NEXT SLIDE

ANIMATED SLIDE: You can then go through a similar series of steps to the 1 mark item to ensure you’ve selected the right item and mark scheme and then look at some examples of this mark scheme type, followed by the checklist for final confirmation. PAUSE FOR THREE SECONDS AND ASK FOR ANY QUESTIONS.

For higher tariff items, the choice of mark scheme increases For higher tariff items, the choice of mark scheme increases. CLICK TO NEXT SLIDE

As with the 1 and 2-5 mark items, you are first taken to a short piece of text guidance on items in this mark range; however, now you are presented with three options: CLICK TO NEXT SLIDE.

Assessment of knowledge-only, a mix of knowledge and qualitative skills, or assessment of purely qualitative skills.ANIMATION SLIDE. PAUSE FOR THREE SECONDS FOR EACH ONE.

ANIMATED SLIDE. As with the 1 mark and 2-5 mark items, you have the option of reviewing some examples of each mark scheme type (CLICK), followed by a checklist (CLICK).

ANIMATED SLIDE: With higher tariff items and mark schemes, you have again, a piece of text with guidance (CLICK), followed by the three options with the different types of assessed knowledge and skills. However, there’s the additional consideration of whether the levels of response mark scheme (if that’s what you’re planning to use) should be Holistic or Analytical (CLICKx2). As with the lower tariff items, there are examples and checklists to help guide you.

Other features of the Guide The grey boxes in the content map also offer additional guidance on designing mark schemes for extended writing tasks (CLICK); these are helpfully differentiated by subject-type. There are also sections on subject-specific issues (CLICK) and e-marking (CLICK). Other features of the Guide

Other features of the Guide Glossary – in case you need to find out or be reminded of what a key term means. Wherever the term is mentioned, it’s highlighted with an interactive link to the Glossary.

Other features of the Guide There are a range of example mark schemes – some showing good practice and others where there’s potential for validity, reliability and/or manageability issues to occur. PAUSE This is not there to highlight poor practice, but rather to make people aware of potential unintended implications of developing mark schemes. Where these have been used, they have been changed or anonymised (many are from legacy qualifications).

Other features of the Guide ANIMATED SLIDE: Finally, you have the section on checklists with helpful questions to consider for more general (CLICK) mark schemes, PAUSE: and more specific mark schemes (CLICK).

Any questions? Unrhyw gwestiynau?