Download presentation
Presentation is loading. Please wait.
Published byMartin Butler Modified over 9 years ago
2
You’ve Been Shown: Now It’s Your Turn to Ask! CLEAR 2004Kansas City, Missouri Responding to Your Questions About Testing
3
Credentialing and Exam Issues: Sessions at CLEAR 2004 Ten Year Review of CBT When Certification Replaces Licensure: Show me Accreditation ADA: Let Us Show You What Works Show Me What Globalization Means for Credentialing Programs Multiple Tracks to Licensure: Show me the Way Test Piracy and Security in a Wired World CLEAR 2004Kansas City, Missouri
4
Show Me the Validity Show Me How to Get Past MCQs: New Opportunities in Measurement Show me an Evidential Approach to Assessment Design The Case of the Repeating Licensure Examination Candidate: Psychometric, Legal, and Policy Issues CLEAR 2004Kansas City, Missouri Credentialing and Exam Issues: Sessions at CLEAR 2004
5
Panelists: Carol Morrison Assistant Vice President National Board of Medical Examiners Elizabeth Witt Senior Statistician American Board of Emergency Medicine Rose McCallin Director of Office of Examination Services State of Colorado CLEAR 2004Kansas City, Missouri Moderator: Chuck Friedman Assistant Vice President, ACT
6
Speakers You... the audience!!! CLEAR 2004Kansas City, Missouri
7
Validity Reliability Item Analysis Item Review Item Writing Passing ScoreScoring Job Analysis Test Blueprint Standard Administration
9
Validity Reliability Item Analysis Item Writing Passing Score Scoring Test Blueprint Standard Administration Job Analysis Item Review
10
Should state licensing boards require continuing education? CLEAR 2004Kansas City, Missouri
11
How do I find out if my test has been compromised? If so, what should I do? CLEAR 2004Kansas City, Missouri
12
CLEAR 2004Kansas City, Missouri What should I do if I do or not do if I think a candidate or candidates have cheated?
13
Test Compromise? Missing test book –When, who? –Statistical analyses Memorization of items –Memorize and reconstruct items –Study guides –Internet postings –Statistical analyses
14
Is a self assessment exam good enough for continuing competency? CLEAR 2004Kansas City, Missouri
16
Four Components of Continuous Certification Professional Standing Professional Standing Lifelong Learning and Self Assessment Lifelong Learning and Self Assessment Assessment of Cognitive Expertise Assessment of Cognitive Expertise Assessment of Practice Performance Assessment of Practice Performance
17
Considerations in Determining the Value of a Self-Assessment How is the exam developed and scored? Can you be certain who took the exam? How high are the stakes for maintaining the credential? Does the self-assessment process foster learning? Is it easy to participate as part of one’s normal professional activities?
18
Is performance testing useful in state licensure? CLEAR 2004Kansas City, Missouri
19
Performance Assessment Does performance assessment measure something different? Reliability, validity How do you score it? Cost, complexity Answer: It depends!
20
CLEAR 2004Kansas City, Missouri What is your opinion about open book exams?
21
What should I do if a candidate receives an incorrect score report? CLEAR 2004Kansas City, Missouri
22
How do you determine if a request for accommodations is appropriate? If a test is administered under special conditions, should the score report be “flagged”? CLEAR 2004Kansas City, Missouri
23
Is Request Appropriate? Use expert consultant Level the playing field, don’t tilt it too far the other way
24
To Flag or Not to Flag? Many testing companies have stopped flagging Some claim that flagging may lead to unintended consequences Standards for Educational and Psychological Testing (APA/AERA/NCME) –Flag if do not have evidence of score comparability
25
Do we have to equate our tests? Is there one best way? ONE BEST WAY? CLEAR 2004Kansas City, Missouri
26
Reasons to Consider NOT Equating Small sample size Multidimensional exam Changing content outline
27
What If We Can’t Equate? Create a detailed blueprint Write the best possible items Build forms that are closely parallel in both content and statistics Conduct frequent standard setting studies Include common items across forms; study differences qualitatively
28
CLEAR 2004Kansas City, Missouri An examinee complains that he failed the test by one point/question. What do you tell him?
29
Other Sources of Information CLEAR Exam Review Other CLEAR Publications –ADA Handbook –Development, Administration, Scoring and Reporting of Credentialing Exams CLEAR web site: Frequently Asked Questions Joint Standards: AERA, NCME, APA Professional psychometricians CLEAR 2004Kansas City, Missouri
Similar presentations
© 2025 SlidePlayer.com. Inc.
All rights reserved.