Download presentation
Presentation is loading. Please wait.
Published byHugo Pullen Modified over 10 years ago
1
Brian Jolly Centre for Medical and Health Sciences Education Monash University Train the Trainer Workshop October 31, 2003 Hong Kong International Consortium Blueprinting
2
Hong Kong International Consortium Objectives Describe basic ideas behind blueprinting Develop 2 examples Show how this does/does not articulate with IDEAL database
3
Hong Kong International Consortium Principle The PURPOSE of the assessment will define the content, strategy, & technique used.
4
Hong Kong International Consortium Curriculum and Assessment: ‘Ideal’
5
Hong Kong International Consortium Curriculum and Assessment: Real
6
Hong Kong International Consortium H. C. Curriculum and Assessment: Real
7
Hong Kong International Consortium Curriculum and Assessment: Blueprinting
8
Hong Kong International Consortium Face validity Content Construct Criterion Effects on learning Aspects of Validity - Constructs
9
Hong Kong International Consortium Face validity Content Construct Criterion Effects on learning Aspects of Validity - Constructs
10
Hong Kong International Consortium Face validity Content Construct Criterion Effects on learning Aspects of Validity - Constructs
11
Hong Kong International Consortium Define what is to be tested Clinical problems at some resolution Tasks for each problem Blueprint Select test methods Authenticity appropriate to clinical tasks Clinical tasks determine the method Titrate against practical constraints Content Steps
12
Hong Kong International Consortium Competence Categories CVSRS…GI…..MSSEndo/ Metab Haem / Oncology …. History Taking Physical Exam Tests and Procedures ….. Management Communication and Patient Education Simple Blueprint
13
Hong Kong International Consortium The Hospital SettingThe Community Setting Competence Categories CVSRS…GI…..CVSRS…GI History Taking Physical Exam xx Tests and Procedures xx ….. Other Parameters - Hierarchical
14
Hong Kong International Consortium InpatientOutpatient/Ambulatory Competence Categories CVSRS…GI…..CVSRS…GI History Taking Physical Exam xx Tests and Procedures xx ….. Hierarchical - IDEAL ® Related
15
Hong Kong International Consortium The ChildThe Elderly Competence Categories CVSRS…GI…..CVSRS…GI History Taking xx Physical Exam Tests and Procedures ….. Other Dimensions
16
Hong Kong International Consortium The ChildThe Elderly Cognitive /Affective Categories CVSRS…GI…..CVSRS…GI Recall xx Reasoning Affect xx Skill ….. Other Dimensions - Response
17
Hong Kong International Consortium The ChildThe Elderly Cognitive /Affective Categories CVSRS…GI…..CVSRS…GI Recall KFI Reasoning Affect CISP Skill ….. Test – Response Matrix
18
Hong Kong International Consortium Steps: A plausible test of Content Validity Chart distribution of questions by level & topic Compare this distribution with weighting of topics in curriculum or course documents Note to establish content validity we need a demonstrated relationship Jolly 2000, Medical Education,
19
Hong Kong International Consortium Some researchers use judgments of clinicians or successful test-takers Dimensions of blueprint not always easy to define c.f. the items How clinical problems, tasks or other 'concrete examples' relate to content often taken for granted in blueprinting Jolly 2000, Medical Education, Steps: A plausible test of Content Validity
20
Hong Kong International Consortium Summary Before selecting evaluation tools, need to create an evaluation “blueprint” The blueprint = matrix that outlines parameters for the assessment in 3 major areas: domains to be assessed, content & assessment instruments
21
Hong Kong International Consortium Blueprint matrix ensures balance across assessment formats - conversely Multiple methods usually required to achieve blueprint objectives Different assessment instruments allow balance of strengths and weaknesses of each method Summary
22
Hong Kong International Consortium Content In Australia RACS has sole responsibility for goals, training delivery & standards See CANMEDS, Duties of a Doctor, Investigate ways of addressing these issues Get profession & public to assist Responsibility for the ‘hard to measure’ RACS blueprint – are we a surgeon first & a specialist second? Blueprinting Issues: e.g., RACS Assessment
23
Hong Kong International Consortium Judgement (80%) Literature on judgement Measures of judgement (as yet undeveloped) Manual dexterity (20%) Simulation and testing in training Reznick, Paisley vs. Beard et al RACS Assessment - Content
Similar presentations
© 2025 SlidePlayer.com. Inc.
All rights reserved.