Saturday 8:00 am- 9:30 am Overview of the Outcomes Assessment Process Presented by: Janice Stoudemire, CPA, ATA, ABA Associate Degree Board of Commissioner Copyright Protected: Material can not be use or reproduced without permission from Janice Stoudemire ACBSP

Session Goals Overview of outcomes assessment process Role of outcomes assessment in the accreditation and reaffirmation Purpose and role of quality assurance reports Use of online filing of reports via Livetext.com

X How do you react when you hear… “Quality Assurance Reports or outcomes assessment”? X

ATTITUDE Any task facing us is NOT as important as our attitude towards it, for that determines our success or failure. Norman Vincent Peale

My perspective is multi-faceted: as an instructor wanting to improve student success, as an evaluator wanting to see documentation that learning has taken place, as a developer of assessment techniques wanting to be part of the accountability era.

Overview of Outcomes Assessment Process

Why outcomes? State, Regional, and National accrediting organizations require it. State and federal governments are attaching funds to it. Students, employers, etc. expect us to be accountable for the learning process.

Because…. Courses/programs have creditability. Employers take notice of skill sets. Neighboring institutions are more acceptable to transfer courses. Students learn!

What is Outcomes Assessment?

Outcomes Assessment is a means for continuously improving the learning experience, requires identifying expected outcomes for programs, requires systematically gathering evidence on performance.

Outcomes Assessment involves analyzing and interpreting the evidence. using the resulting information to document, explain, and make changes that improve performance.

Two Types of Assessment Direct Assessment: Measures a specific competency attainment . Includes Student Learning Outcomes. Examples: Pre/post exams, case studies/projects, etc.

Two Types of Assessment Indirect Assessment: Defined as something that happens to a student as a result of one’s educational experience . Includes Behavioral Outcomes. Measured by retention rates, surveys, etc.

Characteristics of Outcomes

Outcomes… have a foundation in the institution’s mission, goals, and objectives. have support/collaboration of faculty and administration. involve systematic use of quantitative and qualitative measures. involve realistic goals and timetable, and supported by appropriate investment.

One size doesn’t fit all! No right or wrong measure or outcome! What is appropriate for your program(s)? Important: Use results to make improvements!

Outcomes versus Competency

Performance Appraisal What’s in a name? Student Outcomes Criteria for Success Learning Outcomes Performance Appraisal Performance Indicators Objectives Competencies Performance Measures

An outcome is…. is a performance statement of student learning. the qualities that will characterize students after completion of course/program of study. is dependent upon the educational experience.

An outcome is…. “what” you expect students to achieve. the knowledge and skills your students should master. is the “end”, not the means!

A competency …. demonstrates "how" students can achieve an outcome. is the specific activity used to measure a student’s mastery of the outcome. is a “means” to the end.

Example: Outcome “What” you expect students to achieve? Outcome: Students should understand the elements of, the relationships among, and the accounting concepts underlying the primary financial statements.

Example: Competency Demonstrates "how" students can achieve this outcome. Competency 1: Apply the fundamental accounting equation to prepare a balance sheet of any entity (e.g., a person, sole proprietorship, partnership, corporation, etc.).

Example: Competency Competency 2: Apply the fundamental concepts revenues and expenses to prepare an income statement of any entity (e.g., a person, sole proprietorship, partnership, corporation, etc.). Competency 3: Apply the fundamental concept changes in owner's equity to prepare a statement of owner’s equity (or Retained Earnings) of any entity (e.g., a person, sole proprietorship, partnership, corporation, etc.).

Outcomes Assessment Process

Outcomes Assessment is a Moving Target! Past: Establish outcome, measurement instrument, collect data. Present: Analyze & Interpret results to enhance student learning. Future: Trend data of results, benchmarking, and comparisons.

Six Simple Steps to Outcomes Success! Define your outcome (what you expect students to achieve). Determine the competency (how will students demonstrate achievement of outcome). Is it appropriate? It is valid? Is it reliable?

Six Simple Steps to Outcomes Success! Select or design measurement instrument and deploy the process of data collection. Report results - any areas of success? Analyze results and take action. Report results of action taken (occurs in following semester, cycle, year etc).

Important! Outcomes assessment should be a process NOT a person! X

Develop the plan, process, forms to document completion of process. Process needs to be formalized and fully deployed. Develop the plan, process, forms to document completion of process. When will measurements be made? How often? Who will collect the data? Who will interpret the data? When? Who will report the results? When? Consider annual report of progress.

ANNUAL SUMMARY REPORT ON OUTCOMES ASSESSMENT PROCESS Intended Outcome Competency Measurement Instrument and Success Criteria Respon-sible person Process Deployed Yes No Partial Expected Results Achieved Yes No Partial Analysis and Action Taken Results of Action Taken Understand appropriate application of computer technology to complete accounting functions. Demonstrate proficiency in accounting applications of computer databases, spreadsheets, and integrated accounting software. 75% will score a 70% or higher on the project in ACC 000 Jane Faculty  For the past five semesters, the standard was met for both. 90% of students scored 70% or better. Criteria will be increased to 80%. None at this time; Will continue to monitor the success rates at the 80% level and determine if any corrective is required in the future. 75% will score a 70% or higher on the project in ACC 001 Apply analytical and logical skills to use information retrieval and technology to solve accounting problems. Demonstrate proficiency in using the internet to retrieve info and research accounting problems. Internet-based research for cases in ACC 003 and 004 80% of students will score 2 out of 3 on the research rubric Joe Faculty For past 3 semesters, standard was met. 87% scored 2 or better. No suggested at changes at this time. Continue to monitor None

Close the “Loop” Use the results to make improvements! Meet to discuss results and determine appropriate action. Use results to initiate action aimed at improving program. Clearly relate back to the intended outcomes. Documentation is the key!

Identify outcomes, competencies, measurements Outcomes Assessment Identify outcomes, competencies, measurements Make Improvements Collect data Analyze Results

Sample Timeline for Assessment Set outcomes, competencies and establish measure instruments. Continue deployment, collection and analysis. Address any problems from last term Summer Fall Spring Summer Deploy assessment, collect data, start analysis Draw conclusions and make improvements, Progress Reports, Revise activities.

Outcomes Assessment from the Evaluator's View

Has the Process Come to Fruition? Is there evidence of indefinable outcome measure? Is there evidence of a systematic approach (process) for data collection? Is there evidence the approach is deployed? Is there evidence of results?

Has the Process come to Fruition? Is there evidence of analysis of results and findings? Is there evidence of improvements implemented? Is there evidence of follow-up in respect to improvements made? (Did you close the loop?)

Key is………… Evidence

Let’s see how it all fits together!

Linear Connection Institutional Goals/Objectives Departmental Goals/Objectives Program Goals/Objectives Course Goals/Objectives

Mission Statements Midlands Technical College is a comprehensive, multi-campus, public, two-year college serving the primary region of Richland, Lexington and Fairfield counties of South Carolina. College programs and services provide accessible, affordable, high quality post-secondary education that prepares a diverse student population to enter the job market, transfer to senior colleges and universities, and achieve their professional and personal goals. Through its programs and services, the college equitably provides higher education opportunities and strengthens the economic and social vitality of the community. Midlands Technical College’s Business Department provides an innovative, up-to-date learning environment that enables individuals to achieve their personal, professional, and educational goals for the purpose of contributing to the economic growth of the community. Midlands Technical College’s Accounting Program provides students with skills, knowledge, and business values to prepare them to enter the workplace, and learning experiences to encourage self-development. College Dept. ACC

Business Departmental Goals Quality Instruction: To assure consistently high-quality instruction in all modes of delivery. Student Success: To work collaboratively with other areas of the colleges to encourage students to achieve their educational goals and succeed in work and/or continued education. Program and Curricula: To develop, market and maintain quality programs that are responsive to the needs of our service area. Technology: To employ appropriate technologies effectively to enhance instruction and student learning. Professional Development: To foster continuing education and professional development for faculty and staff.

Accounting Program Goals Quality Instruction: To promote learning that develops individual potential and produces competitive graduates for the workforce. Student Success: To prepare students for entry-level accounting-related occupations or for transfer to four-year institutions. To provide individuals with accounting skills for personal use or for upgrade of job skills and knowledge. Program and Curricula: To offer a curriculum that prepares graduates for the workplace and meets the needs of area employers. Technology: To produce students that can utilize current technology in the workplace. Professional Development: To produce students that can communicate effectively. To develop the critical, logical, and analytical abilities of students.

Accounting Program Outcomes and Related Competencies Program's Goals/Mission Program’s Outcomes Competencies Performance Measures (at program/course levels) Success Criteria

Accounting Program Outcomes and Related Competencies Attainment of program’s mission and goals is based upon program outcomes. PLO’s have been developed that address “what” we expect students to achieve and describe specific skills and abilities to be mastered by students completing the accounting program. Each outcome is supported by a competency that demonstrates “how" students will achieve the outcome. Each competency is support by a performance measure that identifies how the competency will be measured. Specific course outcomes and related competencies have been developed to support the accounting program outcomes.

Performance Measurement Outcome Competency Performance Measurement 1. Understand appropriate application of computer technology to complete Accounting functions. Demonstrate proficiency in accounting applications of computer databases, spreadsheets, and integrated accounting software. Successful completion of: - specific projects in ACC 245 and ACC 246 2. Apply analytical and logical skills to use information retrieval and technology to solve accounting problems. -Demonstrate proficiency in using the internet to retrieve information and research -Demonstrate analytical ability to solve accounting cases. Successful of completion of: -internet-based research for completion of accounting cases in ACC 201 and 202 3. Communicate effectively with others utilizing appropriate forms of communication methods. listening, writing, oral, and presentation skills. -a student communication skills portfolio in BUS 130. 4. Understand the application of the conceptual framework of accounting, generally accepted accounting principles, and current laws to financial information and understand the various practices of business. -Demonstrate basic understanding of the use and interpretation of financial statements and their inter- relationships. -Demonstrate a basic understanding of the unique accounting practices. -embedded exam questions in ACC 101,102 ,111, 112, 115 - standardized final exams in ACC 101, 111 - financial statement analysis case in ACC 202. - specific projects in ACC 150, ACC 265 -written responses to articles and cases in ACC 203 and 260 - a set of tax problems in ACC 124

Define your Program Outcomes (What you expect students to achieve?) Understand the application of the conceptual framework of accounting, generally accepted accounting principles, and current laws to financial information and understand the various practices of business.

Determine the Program Competency (How will students demonstrate achievement of outcome?) Demonstrate basic understanding of the use and interpretation of financial statements and their inter-relationships. Demonstrate a basic understanding of the unique accounting practices.

Select Measurement Instruments: -Embedded exam questions in ACC 102, 112, 115. Standardized final exam in ACC 101 and 111, Financial statement analysis case in ACC 202. -Specific projects in ACC 150, ACC 265. -Written responses to articles and cases in ACC 203 and 260. - Set of tax problems in ACC 124.

Course Name: Accounting Principles I Outcome Number: One STUDENT PERFORMANCE OUTCOMES ASSESSMENT PLAN FOR BUSINESS DEPARTMENT COURSES Course Name: Accounting Principles I Outcome Number: One Lead Faculty Member: Barbara Wagers Scheduled Review of Process: Each summer semester Date: PART 1: OUTCOMES, COMPETENCIES, AND MEASUREMENT INSTRUMENT Note: The intended course outcomes must address “what” we expect students to achieve and the course competencies (Performance Measures) must demonstrate "how" students will achieve the outcome. The course learning outcomes are supported by the competences which are supported by the course learning objectives. All competencies must have an identified measurement instrument, a “criteria for success” measure and designated data collection cycle (i.e. semester, annually etc.).

Intended Course Outcome #1: Students will demonstrate overall knowledge of the learning objectives for introductory accounting. Course Competency (Performance Measure): Students will demonstrate understanding of financial transactions and financial statements, and understanding of the accounting concepts involving assets, including cash, receivables, inventory, and fixed assets. Measurement Instrument and Success Criteria: A standardized departmental final exam developed by accounting faculty that is based on the course learning objectives will be administered each semester. The goal is 75% of the students will pass the exam.

PART 2: REPORTED RESULTS Note: Under actual results include any areas of success and any problems encountered. Expected Results: At least 75% of the students will pass the standardized departmental final exam that is based on the course learning objectives. Summary of Actual Results: 84% of the 296 students enrolled in ACC 101 for spring 2007 scored a 70% or better on the standardized exam. The standard not only was met, but was exceeded by 9%.

PART 3: ANALYSIS OF RESULTS AND ACTION TAKEN Note: If expected results are being achieved, you may consider revising the competency or redefining the success criteria. Under actions taken, include the time period in which the action took place. Summary of Analysis of Results: Although the standard was met, a task force of accounting faculty are analyzing the exams to determine which areas the students were still having difficultly with. Once these areas are identified, they will be tied to specific learning objectives. By the end of summer 2007, the task force will report their findings and recommendations to the accounting faculty to see if any action is deemed necessary. Actions Taken: At this time, no action is being taken, however, it does appear that changes made in the past semesters have been positive since the passed rate has improved. These actions include the creation of an online tutorial site and the development of student practice reviews. The accounting faculty will consider increasing the standard benchmark to 80%.

PART 4: RESULTS OF ACTIONS TAKEN Note: Since the results of action taken will occur in subsequent periods, you may have to wait until sufficient time has passed before realizing the results of actions taken. In order to maintain trend data, be sure to add each reporting cycle’s result to the trend graph section. Summary Results of Actions Taken: None at this time Graph of Trend Data

Role of Outcomes Assessment in Accreditation and Reaffirmation

Have you heard? Outcomes assessment is the foundation of the accreditation and reaffirmation processes! Baldrige criteria is fully integrated into our standards.

Accreditation and Reaffirmation Processes Help indicate the effectiveness of your programs. Help to improve/enhance programs. Provide documentation and evidence. Focuses on intended results not the process.

Accreditation and Reaffirmation Processes Shift in focus to what students learn rather than what we teach. Answers the question, “Have we made a difference?” Continuous Quality Improvement - Use results to make improvements.

Accreditation and Reaffirmation Processes Accrediting bodies expect institutions to embrace the student learning outcomes process. Use evidence of student learning to make judgments about academic quality and accredited status.

Connection with Quality Assurance Reports QA reports are the “lifeline” between reaffirmations! Keep institutions focused outcomes and continuous improvement. Next reaffirmation will built on the past QA reports. Use past years of trend data.

Quality Assurance Reports

Background of QA Report What is it? Report due at ACBSP headquarters every other year. Not a self-study, less is MORE! Not the same as periodic reports-which are due every other August to report on existing notes or conditions. X

Background of QA Report Why? Required to maintain accreditation. Keeps ACBSP informed of activities at institutions. Are you still in compliance with ACBSP accreditation standards?

Background of QA Report What programs? For all accredited programs. Do not include: Programs not accredited by ACBSP. No certificates –ACBSP does NOT accredit certificates.

Contents of QA Report Reports any changes within institution or business unit that alters the status of business faculty, curriculum and resources of business unit. Include contact info of who prepared report Reports progress in area of ongoing outcomes assessments.

Possible Actions by Commission Accept QA report! Request additional information or clarification. Request annual QA. Issue note/condition. Suspension or revocation of accreditation.

Coming Attractions! Online submission using Livetext.com

Coming Attractions!

Coming Attractions!

Resources for Preparing QA We are here to help YOU! Contact any Commissioner or ACBSP home office. ACBSP website has: PowerPoint presentations, Best practices Examples of QA reports More to come!

Additional Resources Baldrige National Quality Program at http://www.quality.nist.gov Continuous Quality Improvement Network (CQIN) at http://www.cqin.org Academic Quality Improvement Project (AQIP) at http://www.aqip.org Council for Higher Education Accreditation (CHEA) at http://www.chea.org Middle States Commission on Higher Education at http://www.msache.org/mainstudents.html

Happy Assessing! Any Questions?