10 Dec 2010 NSS Assessing Student Learning Series for the HMSC Curriculum (4) - Designing Internal Assessment Part 2: Basic Question Setting Skills Mr. KWOK Kai-yin, Bill

2 Tools for setting HMSC questions HMSC Curriculum and Assessment Guide Curriculum Structure and Organization Assessment Objective SRR Information Package Assessment Framework Level Descriptors Sample papers and Marking Schemes Annotated Exemplars (for illustration of performance standards in various level) HMSC Resource Kits References (Websites, News, books, Journals, Publications etc.)

3 What are measured? demonstrate comprehensive knowledge and understanding of issues, concepts and principles in health and social care across all systems and from all perspectives apply concepts of health and social care to unfamiliar situations in a wide range of contexts demonstrate excellent critical thinking and problem solving skills, making inferences, judgements and evaluations when analysing interconnected HMSC issues effectively communicate ideas in a coherent and logical manner through extensive and sophisticated use of terminology and in an appropriate form design and conduct creative and focused investigations, synthesise and evaluate data collected, and draw valid conclusions in authentic health/ social care situations Level Descriptors (L5)

4 Paper 1 Questions will be set on the compulsory part of the curriculum. Paper format and question types PartMarksRemarks Paper 1 (2 hrs) Max. marks: 80 Section A32 marksShort Questions (Compulsory) Content-based. Mainly focus on factual recall; Simple and direct answers are needed. Section B48 marksStructured Questions Data-response questions. Data analysis is required. Data may be presented in the form of figures, tables, charts etc. Focus on assessing critical thinking and reasoning skills when interpreting phenomenon/ trends/ patterns of issues. Higher cognitive level is required.

5 Paper 2 Questions will be set on both the compulsory and elective parts of the curriculum. Paper format and question types PartMarksRemarks Paper 2 (1 1 / 2 hrs) Max. marks: 60 Section A24 marksShort Questions (Compulsory) Scenario-based. Focus on assessing the students ability on the application of knowledge in authentic situation. Section B36 marksIssue-based Questions (2 out of 3) 2 Case study questions. Focus on studying HMSC related case from different perspectives and/ or across different levels. Assembling knowledge from different topics is required. Problem solving and evaluation skills are assessed. 1 Essay type question. Focus on a health/ social care issue.

6 Paper 1: Section A Question 6 Short question: Mainly focus on factual recall

7 Paper 1: Section B (1) Question 10 Data response question: Answers should make reference to the raw figures/ trends/ patterns of the data provided.

8 Paper 1: Section B (2) Question 11 Data response question: Answers should make reference to the raw figures/ trends/ patterns of the data provided.

9 Paper 1: Section B (3) Question 13 Data response question: Answers should make reference to the content of the information provided

10 Paper 2: Section A Question 4 Scenario-based : Answers are identified and interpreted from the information provided (the scenario); no mark will be given to answers purely based on factual recall. High quality distractors may be included for this type of question.

11 Paper 2: Section B (1) Question 6 Essays: limited response with suggested answers (accept alternative reasonable answers)

12 Paper 2: Section B (2) Question 8 Essays: open-end question under pre- set framework Marking scheme should include clear assessment guidelines and mark allocation for each areas.

13 Activity One: Classification Teachers in groups, classify the questions designed by your group members into various sections of the HMSC Papers.

14 Action Verbs in Question Setting (1)

15 Action Verbs in Question Setting (2)

16 Action Verbs in Question Setting (3)

17 Teachers in groups, identify the action verbs that are commonly used in question setting of the subject. Activity Two: Action verbs * Different action/ command words may have different interpretation and cognitive requirements in different contexts

18 Common action/ command words Identify … State… Describe Suggest…… Explain….. Discuss ….. Compare and contrast… Relate … to…… Illustrate Elaborate…… Analyse…. Criticize … Propose…..(a new idea) Comment on …. Evaluate.. Justify your views…..

19 ….from various perspectives As a ….., what is your opinion (authenticity) Suggest the ways With reference to …. ….. the pros and cons of ….. According to… Relate your answer to…… Identify/ describe the role(s) of … Comment on …. and justify your views….. To what extent…. Compare the relative importance… Comment on the validity of …. Common question words

20 Support viewpoints/ arguments with relevant concepts/ facts/ information. Analyse and answer the question from various perspectives/ across different levels Strive hard to prevent factual errors when quoting evidence to support viewpoints Pay attention to the focus of the question – do not miss the key word(s) of the question Avoid contradictory assertions In essays, introductory and concluding paragraphs as well as topic sentences signaling the content of each paragraph are needed. In answering HMSC questions

21 Quality Analysis of HMSC question - Case study (1)

22 Quality Analysis of HMSC question - Case (2)

23 Quality Analysis of HMSC question - Case (3)

24 Quality Analysis of HMSC question - Case (4)

25 Quality Analysis of HMSC question - Case (5)

26 Major Concerns in question setting 1.Cognitive requirement of the question 2.Skill coverage 3.Validity of the question 4.Sufficiency of the information 5.Quality of the question – question is clear and is easy to follow with proper question words 6.Open-end or restricted response 7.Misconception check 8.Avoid Errors/ ambiguities – proofreading is needed Typographical/ spelling mistakes Grammatical mistakes Ambiguity in meaning/ instructions Inconsistency with the paper Punctuations….. 9.Arrangement of distractors 10.Avoid the acceptance of layman answers

27 Try to identify… Can you ….. Do you think ….. All the …. Based on your personal experience/ own thought,… Poor questions - examples

28 Activity 4: Proofreading Identify the errors and ambiguities of the following question within 15 minutes.

29 Objectives –Quality audit of INTERNAL examination papers/ test/ quiz from school teachers (On demand) –Provision of extra HMSC questions in form of DVDs and booklets –Provision of professional development programmes on basic question setting skills for HMSC teachers –Sharing of extra HMSC questions and the development of web-based question bank Assessment Partnership Scheme

30 –Working group for setting HMSC questions will be established for this scheme. –A maximum of 10 meetings (3hrs sessions) with 4 members (or equivalent) will be held for question setting and moderation. –The deliverables, including 80 extra HMSC questions and suggested answers for HMSC, will be disseminated to teachers in form of DVDs. –Please contact Mr. Bill KWOK if you are interested. Assessment Partnership Scheme