Computerizing the CPA Examination Presented to the 2003 APLG Meeting by Richard DeVore

Agenda Computerizing the CPA Exam Examination Structure, Content, and Length Transitioning and Conditioning Overview of CBT Demonstrations of Simulation Prototypes

CPA Examination Mission Statement To admit individuals to the profession only after they have demonstrated the entry-level knowledge and skills necessary to protect the public interest in a rapidly-changing business and financial environment. Continue To Protect The Public Interest

Periodic Changes Needed Maintain the confidence of various constituencies Provide a comprehensive assessment of the entry-level knowledge and skills required by CPAs in changing business and financial environment

Whats Changed? Rapidly Changing Business World Rapidly changing role of the CPA –Demands for broader - based financial services –Expansion of the attest function –Need for professional adaptability and flexibility –Rapidly - changing technology environment

Change in Entry-Level Practice

What We Need From A New Exam The Ability to Test: –Recall of foundational knowledge –Research skills –Analytical skills –Integrated knowledge and skills

What We Need From A New Exam (contd) The Availability of: –Access to authoritative literature –Current technological tools –Secure computerized test system –Secure administrative conditions –Year-round testing

Responsibility for Decisions AICPA Board of Examiners –Content Committee Preparations Subcommittees –Psychometric Committee –Education Task Force –Communication Task Force –Systems Oversight Task Force –Other task forces and working groups

Outreach to Stakeholders Mechanisms to Guide Policy –Exposure Draft on Structure and Content (Feb 2001) –Briefing Paper #2 (Mar 2001) –Exposure Draft on Transitioning & Conditioning (Nov 2001) Focus groups of students and young professionals (Summer 2001)

Outreach to Stakeholders Education Task Force –Working with AAA –Looking at curriculum and other academic-related issues –Help students and educators prepare for the transition to CBT

Outreach to Stakeholders Ongoing Communications –New Web site with practice materials and tutorials ( –Exam Alert (contact – updates to stakeholders

BOE Decisions Examination Structure, Content and Length

Examination Structure Changes New mix of content groupings New section Testing of skills Testing of knowledge integration Length of Exam

Examination Structure 4 Independent Sections –Auditing & Attestation –Financial Accounting & Reporting –Regulation –Business Environment & Concepts (NB: Full Content Specs available on website)

Auditing & Attestation Auditing Procedures Generally Accepted Auditing Standards Other Standards Related to Attest Engagements Skills Needed to Apply That Knowledge

Financial Accounting & Reporting Generally Accepted Accounting Principles for: –Business enterprises –Not-for-profit organizations –Government entities Skills Needed to Apply That Knowledge

Regulation Federal Taxation Ethics Professional and Legal Responsibilities Business Law Skills Needed to Apply That Knowledge

Business Environment & Concepts Knowledge of Underlying Reasons and Implications of Business Transactions Skills Needed to Apply That Knowledge in Audit and Attest Engagements To Be Tested In Context of Entry-Level CPA Practice

Business Environment & Concepts (2) Economics Planning and budgeting Corporate financial management Cost measurement and pricing Management Information identification, control, and analysis

Comparison to Current Exam AUDIT Audit & Attest LPR Regulation FARE Account & Report ARE – Tax Regulation ARE – Govt Account & Report ARE – Mngrl Business Environ

Examination Length 14 Hours –In Total for Four Sections Primary Considerations –Psychometric Reliability –Adequate Content Coverage

Proposed Section Lengths Auditing & Attestation –4.5 hours Accounting & Reporting –4 hours Regulation –3 hours Business Environment & Concepts –2.5 hours

Examination Format Multiple-Choice Questions Case-Based simulations –Test integrated knowledge –More closely replicate real world –Assess research, analytical skills –Various question formats –Business simulations 2005

Exam Expectations Exam success will require less memorization, more application Candidates expected to do their work on a computer Candidates expected to have technology knowledge, as well as computer skills Candidates expected to integrate across accounting areas

Conditioning and Transitioning Recommendations, Rationale, & Benefits

Conditioning Policy Candidates can sit for each section individually and in any order All 4 sections – including the first one taken – must be passed within a rolling 18 month period Conditioning

Conditioning Policy (contd) No minimum score requirement for failed sections Failed section(s) cannot be retaken within same window Credit for passed section(s) retained for 18 months

Major Benefits of Conditioning Policy Protects the Public –Tests breadth of knowledge & skills within a reasonable time period Increases fairness to candidates by eliminating minimum scores –Each section is independent Increased flexibility for candidates

Major Benefits of Conditioning Policy (2) Allows Candidates to Focus Their Study and Prepare For Each Section Overall Cost to Candidate is Reduced –Not required to sit for sections not prepared to pass –No retake of passed sections due to score on failed sections

Transitioning How do you equate the current sections of the CPA exam to the revised sections? Upon launch, how do you treat candidates who have conditioned under the paper & pencil test? Issues to Consider

Transitioning Policies Paper-Based CBT 1. Audit 1. Audit & Attest 2. Finl. Acctg.2. Finl. Acctg. and Reporting and Reporting 3. Acctg. and 3. Regulation Reporting 4. Law & Prof.4. Bus. Environ & Responsibilities Concepts

Transitioning Conditioned candidates have same number of testing opportunities within same time period remaining Candidate can establish credit for sections passed under CBT even if paper and pencil conditioning is lost Policies (contd.)

Moving Forward Getting From Here To There

Launch Date November Last Pencil and Paper Administration Early First CBT Administration Window Will Exams Run in Tandem? –No Getting from here to there

Computer-Based Testing Roles and Responsibilities of State Boards of Accountancy and AICPA Basically Unchanged AICPA Remains the Examination Provider State Boards of Accountancy Determine if a Candidate has met Requirements to Test How CBT Will Work

The Testing Process CPA Examination Delivered at Prometric and State-Operated Testing Centers Candidate Registers with Prometric (phone or web) Candidate Takes the CPA Examination at a Prometric or State-Operated Testing Center How CBT Will Work

The Testing Process The Process (continued) –Prometric Forwards Results to AICPA –AICPA Compiles and Forwards Advisory Grades and Diagnostics to Jurisdiction –State Board Advises Candidate of Grades How CBT Will Work

Examination Security In A CBT Environment Test Administration –2 IDs –Double log-on to computer –Candidate sees only one item at a time –Video camera and walk through surveillance – Exit checks

Critical Path Issues Key Milestone Areas Include: –Item Development & Pretesting –Tutorials & practice materials –Systems Development –Communications –Legislation Getting From Here To There

Simulations Definition : Assessment of knowledge and skills in a context approximating that found on the job through the use of realistic scenarios and tasks, and access to normally available and familiar resources.

Simulation Question Types MC Drop-down selection lists Check boxes Spread sheets (enter, drag & drop, formula) Forms completion Written communication Research (access to standards, codes, search engines)

Advantages of Simulations Affordable performance testing Skills assessment Integration of content Multiple correct answers Partial credit Access to resources Innovative item types Workplace realism

Simulations Design 20 – 40 minutes in length Complementary to MC portion of test Aimed at entry-level Developed by educators and practitioners Approved by practitioners Practice related

Simulations Interface Tab–style screens Consistent across simulations and tests Mouse-driven response Full-screen on 17 monitors On-line resources available Split-screen capabilities Review capabilities within a simulation

Simulations Scoring Multiple points per task/question Contingent scoring accommodated Partial credit possible Variable credit per task possible (i.e., credit related to importance)

Demonstrations of Simulation Prototypes

AND NOW...