Copyright 2005 AMP Update from AMP – New Broker Candidate Testing Method Presented to the GREC 2005 Annual School Meeting December 2, 2005 Lawrence J. Fabrey, PhD Vice President, Measurement Research Applied Measurement Professionals, Inc.
Copyright 2005 AMP Overview of Presentation Some (brief) principles of assessment Results of recent Job Analysis New Examination Specifications Purpose and structure of simulations Time frame for implementation Experience a sample simulation (or two or three)
Copyright 2005 AMP Principles of Measurement Links in the Chain of Evidence Used to Support the Validity of Examination Results: Job Analysis Test Specifications Item Writing Examination Development Standard Setting Test Administration Scoring
Copyright 2005 AMP Job Analysis Survey sent to a sample of 12,337 real estate professionals –Received responses from 776 brokers and 427 salespersons 166 real estate job tasks and 16 real estate major issues rated for significance –Task ratings used to develop multiple choice examination specifications –Issue ratings used to develop the simulation examination specifications
Copyright 2005 AMP New MC Exam Specs – overview
Copyright 2005 AMP Detailed Outline –
Copyright 2005 AMP
Free Demo Test
Copyright 2005 AMP Free Sample CBT items: click on “take a demo web test now”
Copyright 2005 AMP
Simulations: Simulations: Purpose of Simulations To provide accurate assessment of higher order thinking related to a content area of interest That is, testing more than just recall Addressing important issues facing brokers (based on JA results)
Copyright 2005 AMP Structure of Simulations Opening Scenario Information Gathering (IG) Sections Decision Making (DM) Sections –Single or multiple DM All choices are weighted (+3 to –3) Passing scores relate to judgment of ‘minimal competence’
Copyright 2005 AMP Simulation Specifications
Copyright 2005 AMP Simulation Development
Copyright 2005 AMP We’ll Have Time to Complete a Simulation or Two Samples available now through the GA REP page on First, a “simulated” login, and then, we’ll want your answers
Copyright 2005 AMP
In case we would need help
Copyright 2005 AMP
Upon Completion of a Sim
Copyright 2005 AMP Several score reports
Copyright 2005 AMP
Time line for Implementation Pilot testing has begun –Please continue to encourage students to participate Live testing begins March 1, 2006 Delayed score reporting likely More information after the first of the year
Copyright 2005 AMP Let’s take a simulation problem (or two or three)
Copyright 2005 AMP Georgia Real Estate Examination Program Contact Information Lawrence J. Fabrey, PhD, VP, Measurement Research ( ) Applied Measurement Professionals, Inc Nieman Road, Lenexa KS 66214