NMLS Testing & Education SAFE Mortgage Loan Originator Test Webinar September 10th, 2009 NMLS Testing & Education

Your Hosts Anne Altman – Manager, Mortgage Testing Programs aaltman@csbs.org 202.728.5750 Pete Marks – VP, National Mortgage Testing & Education Programs pmarks@csbs.org 202.728.5723 Rich Madison – Director, Mortgage Education Programs rmadison@csbs.org 202.728.5737

Conference Call Logistics *6 to mute and un-mute your call Please do not put your phone on hold Please identify yourself and your organization when asking a question

Agenda SAFE MLO Test: Background Info & Test Specifications Pieces of the Test Development Puzzle 1. Subject Matter Experts 2. Content Outlines 3. Test Questions 4. Review Committees 5. Test Forms Test Administration Preparing to Take the Test The Test Center Computer Test Operations and Functionality Other Testing Information Q & A

SAFE MLO Test: Background and Overview

Background Information SAFE Act requires NMLS to develop tests for MLO’s NMLS contracted FINRA and its subcontractor, Pearson VUE, to develop the SAFE MLO tests. NMLS contracted FINRA to administer the SAFE MLO tests using its current dual-vendor channels - Pearson VUE and Prometric’s high-stakes testing facilities. SAFE Mortgage Loan Originator Test National Component State-Specific Test Component

National Test Component Basics All mortgage loan originators must take and pass the national test component Test questions = 100 90 scored 10 unscored/pretest Passing score = 75% National Test Component launched July 30th

State Test Components Basics Test questions = 55-65 (varies; determined by state) 45-55 scored 10 unscored/pretest Passing score = 75% 10 state tests currently available AZ, IA, ID, LA, NH, NC, PA, RI, VT, WA More state tests to be released shortly

SAFE Deadlines for Testing Both National and State Components All new licensees: July 31, 2010 All existing licensees: December 31, 2010 Several states have elected to have earlier deadlines * Please check with each particular state’s regulatory authority

Pieces of the Test Development Puzzle

Subject Matter Experts (SME’s) Part 1. Subject Matter Experts (SME’s)

SME’s are: selected by each states’ regulators and approved by NMLS well versed in laws and regulations a mix of state regulators and industry professionals able to devote their time, effort, and expertise to the development of the SAFE test

“What do SME’s really do?”

SME’s: serve on one or both of the following test development committees: National Mortgage Review Committee State Test Development Committee

SAFE National and State Test Committees Reviewed and provided input on national or specific state test component content outlines Examined, revised, and approved all test questions on national and state test components Will participate in future content outline and test question maintenance cycles

Part 2. Content Outlines

“What are Content Outlines?”

Here’s what they look like…

Here’s what they look like…

Content outlines are: The basic structure or blue print for the test The roadmap for constructing test questions A tool to help candidates prepare for the test

“Tell me more about content outlines…”

Content Outlines Each test component has its own outline national component all state specific components Outlines map out: specific test content and question distribution (weightings)

“How were the content outlines created?”

Test Development Fast Track – national and first 11 states (AZ, IA, ID, LA, MA, NH, NC, PA, RI, VT, WA) Normal Track– all remaining states

Fast Track Content Outlines National Component Drafted by regulator work group Reviewed and approved by national committee State Components Template established by regulator work group Approved by national committee Customized for each state by state specific test committee

Normal Track Content Outlines Job Analysis In April 2009, a job analysis survey was conducted Individuals from the 55 states/jurisdictions were invited to participate Invitations were sent to regulators and industry professionals Over 10,000 people responded

Normal Track Content Outlines The job analysis survey results: produced only minor revisions to state outline template weightings validated the original national outline Moving forward: all future states will use new content outline template during annual test maintenance cycle, FT states will incorporate new outline template

“Can I access these content outlines?”

Yes, Absolutely! Outlines are on the NMLS Resource Center Professional Requirements> Testing> Test Availability Schedule and Content Outlines Currently posted: National outline State outlines AZ, IA, ID, LA, MA, MI, NH, NC, PA, RI, VT, WA DC, GA, KY, MD, NJ, VA, WY (just posted)

Content Outlines content outlines

And we should also mention… Content outlines are an overview of what will be on the test. They can be utilized as a study tool for test takers. No additional study guides or study materials will be developed or endorsed by NMLS.

Coming Soon: More Outlines In early November, more content outlines will be posted (SD, NE, AR, NM, NY, MS) As new states come on board, new outlines will be developed and published Check back often

Part 3. Test Questions

“How are test questions created?”

Test questions are: Written by subject matter experts Reviewed, edited, and approved by committees Created from content outline categories and subcategories Supported by reference materials (laws, regs, etc.)

Test Questions Debunked All questions are 4-option multiple choice No short answer, fill-in-the-blank or true/false Only one correct answer and 3 incorrect options No “All of the above” or “None of the above”

Test Questions Debunked No trick questions No questions with multiple correct answers No questions without a correct answer Good wrong answers No proper names “A borrower is applying for a loan...” not “John is applying for a loan…”

“Why are some questions not scored?”

Un-Scored (Pretest) Questions Un-scored questions are being “tried out.” We only want to put onto our tests questions that meet high quality standards.

The Process for Un-Scored Questions We gather statistics on them first When we have enough “hits”, psychometricians analyze the stats They are reviewed to determine if a question is problematic Eventually, questions that “make the cut” become scored questions

Un-Scored Questions: Do not affect a candidate’s score Are not marked in any visible way to the test-taker (candidates can’t distinguish between a scored or an un-scored question) Are scattered randomly throughout the test

“What if a law changes that affects a test question?”

Test and Item Bank Maintenance All questions are reviewed annually If a new law affects a question, it will be addressed During maintenance, pretest questions can become scored and old scored questions may be retired After maintenance, new versions of the tests are released If an outline changes, it will be updated online (with new date posted)

Test and Item Bank Maintenance What if a law change goes into affect prior to the scheduled maintenance cycle? Updates to laws and rules are consistently monitored If a question is found that is no longer accurate, the following steps may occur: SMEs may be called on to access the validity and edit or revise if necessary A question may be retired and replaced with another

Part 4. Review Committees

Who is included in the State Review Committees?

State Review Committee 6 – 8 state industry representatives Experienced in mortgage industry (time and professional associations) Balance of regulators and industry professionals Requires time and dedication

“How much time and dedication does this take?”

Virtual Meetings All meetings are conducted by Webinar There are a total of 8-9 meetings The committee has one meeting a week The committee agrees on the same day and time for each weekly meeting

Why virtual meetings? Maximum flexibility in scheduling and requires only a computer and a separate phone line Pearson VUE can work with multiple states concurrently Each committee member can see test questions directly with no security risks of paper copies Ability to track progress and customize the next steps, including which items to present, depending on the state’s unique issues

What do the meetings accomplish? The first meeting is focused on developing content outline and is 2 hours The next 6-8 meetings are focused on approving test questions and are 4 hours each

Same Goal, Different Role Test development knowledge versus content knowledge Pearson VUE is expert in how to test Subject matter experts know what to test Every process step is introduced by a training presentation of about 30 minutes

Committee Process Steps Review content outline template and develop a customized, state-specific test blueprint Review test question root pool; customize content and approve questions for use on test Develop new test questions for unique state-specific content areas Classify test questions to the areas of the content outline Reference test questions to state law, regulations, and rules

Part 5. Test Forms

“What are test forms?”

Test Forms Definition: Forms are the different versions of a test Purpose: The purpose of having multiple forms in a test is to enhance test security

Test Forms For example, the national test component has 90 scored and 10 un-scored questions. The different forms (versions) of the test all have the same number of questions. To ensure fairness and reliability, different forms of the national test are all of equal difficulty. Questions present in random, computer-generated order each time for each form.

Test Administration

Preparing to Take the Test

MLO Testing Handbook Everything you need to know about testing

Enrolling: Step-by-Step Instructions

Test Enrollment Basics A candidate does not have to complete education prior to enrolling or taking a test. Before enrolling, a candidate must have a unique NMLS ID number (must either already have made a filing or created a base record). A candidate must request and pay for an enrollment in NMLS before scheduling a test session (*enrollment fees are non-refundable– listed on testing page). Each enrollment window is open for 180 days, during which the candidate must schedule to take the test.

“After candidates create enrollments, how do they schedule test appointments?”

Next Step: Scheduling a Test

Basics of Test Scheduling To schedule, must have an NMLS ID number 500 test centers for candidates to choose from (Pearson and Prometric) After selecting a state on “Find a Test Center” page, a list of all test centers in that state will appear Scheduling– online or over the phone

Basics of Test Scheduling– More Info All tests are available at all test centers (i.e. candidate may take AZ test component in PA) A company can request a group appointment (see Prometric or Pearson web sites for more info) *Special accommodations requests may also be made if necessary.

Basics of Scheduling– More Info “Can a test-taker schedule the national test component and a state component on the same day?” Yes, but only if a test center has two back-to-back appointments available. However, please encourage candidates not to overlap their testing appointments or they can run out of time! (*The test time for the national component is 3 hours and each state component is 2 hours.)

“What if a candidate needs to reschedule a test appointment?”

Rescheduling a Test Must contact Pearson VUE or Prometric directly to reschedule Must be rescheduled no later than 12 p.m. local time two business days prior to appointment Appointment scheduled for: Must be cancelled or rescheduled no later than 12:00 p.m. local Sunday, Monday Thursday of the preceding week Tuesday Friday of the preceding week Wednesday Monday of the same week Thursday Tuesday of the same week Friday Wednesday of the same week Saturday Thursday of the same week *If a holiday occurs during the normal cancellation/reschedule period, this schedule adjusts to define "timely cancellation" as one day earlier than stated above.

The Test Center

Entering the Test Center Arrive 30 minutes prior to test time Must bring a current, government-issued ID with candidate’s photo and signature. Will be photographed; finger or palm print may be taken Candidate may not bring anything into testing room (may be asked to show empty pockets) All belongings such as notes, cell phones, watches, wallets must be placed in locker Basic calculator (not financial calculator) and scratch paper are provided Must read and agree to the Rules of Conduct

Test Security-- Policies All candidates must read and agree to the “Rules of Conduct” prior to taking the test. They stress the confidentiality of all of the test questions and test materials, as they are the copyrighted, proprietary property of NMLS.  Any attempt to modify, publish, transmit, or create derivative works of the test materials, in whole or in part, is a direct violation of copyright laws, the Agreement and the Rules of Conduct and will subject you to federal and/or state sanctions.

Test Security--Policies Also, when creating a base record, NMLS users agree to use the system and its test “only for activities related to obtaining or maintaining a mortgage loan originator license”. Candidates who violate or may violate the Agreement, the Rules of Conduct, or copyright laws, who own, operate, are employed by, or are otherwise affiliated with organizations that are NMLS Approved Course Providers or organizations that intend to seek NMLS Course Provider Approval status, may cause their NMLS Course Provider applications to be denied or their Approval status to be revoked, suspended, or subject to other sanctions.

Computer Test Operations and Functionality

Test Time-- National National Component 2.5 hours test time (150 minutes) 15 minutes allotted for tutorial (before test) 15 minutes allotted for optional survey (after test) Total seat time = 3 hours 100 questions in 150 min. = 1.5 min. per question

Test Time– State Components State Test Components 1.5 hours test time (90 minutes) 15 minutes allotted for tutorial (before test) 15 minutes allotted for survey (after test) Total seat time = 2 hours Approx. 60 questions in 90 min. = 1.5 min. per question *State test lengths vary from 55 to 65 total questions

“What is the computer test functionality. Is it simple to use “What is the computer test functionality? Is it simple to use? How does it work?”

Tutorial and Test Functionality Tutorial walks candidates step-by-step through test functionality. Some key features that candidates can utilize: Toggle back and forth between test questions Mark a question for review and then come back to it later on Change an answer

Exit Survey The exit survey is completely optional. All of the data collected in the survey is monitored and shared with us. Candidates may also provide feedback to the test center administrator on the enrollment process, test center facilities, content of test questions, etc. All comments will be reported.

Score Reports At the end of the test session, candidates will: see their test scores on their computer screens receive a paper copy of their score report These scores do not become official until they are entered in NMLS. Within 48 hours of completing a test, candidates will receive an email informing them that their official scores are now ready to be viewed in NMLS.

“What happens if a candidate fails?”

If a candidate fails… Score report has a diagnostic table that informs candidates of how they performed on each outline content area.

Retake Policy

Retake Policy Must wait 30 days between each test taking attempt May create an enrollment and schedule an appointment, but can’t take the test prior to the end of prescribed wait time After every fourth failed attempt, a 180 day wait time is instituted *After passing a test component, a test taker will not be allowed to retake that particular test component.

Other Testing Information

State Test Certification Some states that had valid tests prior to the passage of the SAFE Act may elect to certify their state tests. In these cases, candidates who previously passed that state’s test may not be required to take the SAFE state test component. Candidates must check with their state regulators. All candidates, however, must take and pass the national component.

Test Enrollment Update National AZ IA ID LA MA NH NC PA RI VT WA Total Enrollments 1230 8 53 15 40 7 19 232 96 9 1 64 1774

Tests Taken Update National AZ IA ID LA MA NH NC PA RI VT WA Total Tests Taken 318 3 17 8 21 2 7 46 1 30 470

Any Questions?

Your Hosts Anne Altman – Manager, Mortgage Education Testing Programs aaltman@csbs.org 202.728.5750 Pete Marks – VP, National Mortgage Testing & Education Programs pmarks@csbs.org 202.728.5723 Rich Madison – Director, Mortgage Education Programs rmadison@csbs.org 202.728.5737