The Academic and Student Services Assessment Process at CALIFORNIA BAPTIST UNIVERSITY Note: This is an “overview” and many implementation details are not included What is assessment and why should I even care? Neal F. McBride, Ed.D., Ph.D. Professor and Associate Provost Phil Martinez, M.A. Director of Assessment Office of Institutional Research, Planning, and Assessment

? What is assessment within higher education (and here at CBU)

3 What is Assessment? Put simply, assessment is collecting evidence (data) to ascertain to what degree an academic or student life department is meeting its stated student learning and institutional capacity (administrative) outcomes. The collected evidence provides the basis for deciding on what improvements, if any, are warranted.

4 WHY DO ASSESSMENT? Assessment is an important tool in helping us to: 1. BETTER SERVE OUR STUDENTS BY PURSUING ACADEMIC EXCELLENCE 2. OBTAIN RELIABLE INFORMATION FOR MAKING DECISIONS (Budget, Personnel, Curriculum, etc.) 3. MEET CONSITUENCY AND ACCREDITATION EXPECTATIONS Why I Should Care

5 EDUCATIONAL EFFECTIVENESS HOW DO WE KNOW WE ARE DOING WHAT WE SAY WE ARE DOING? CLAIM: “We offer quality education.” How do we know this claim is true? HOW DO WE KNOW WE ARE DOING WHAT WE SAY WE ARE DOING? CLAIM: “We offer quality education.” How do we know this claim is true? Educational effectiveness is the common goal for all CBU colleges, schools, departments, and programs Demonstrating educational effectiveness is the goal for assessment... collecting EVIDENCE THE BASIC, KEY QUESTION:

6 ASSESSMENT IS INTEGRAL TO HIGHER EDUCATION FACT: We have always done course-level student assessment, now we apply assessment processes to academic and student services programs in order to pursue educational effectiveness throughout CBU Assessment is NOT optional... It is a professional obligation Assessment is not a passing fad... Holding your breath until it goes away won’t work

7 Assessment or Evaluation? Evaluation – The next logical step after assessment, evaluation seeks to make judgments and decisions, based on the assessment data, about the appropriate course of action (i.e., continue on as is, make indicated adjustments, add a needed element, and/or terminate all or part of the program/service) Functionally, evaluation is done as an integral step within assessment conducting by individual programs and/or departments, as well as by their respective schools, colleges, or student services leadership Evaluation – The next logical step after assessment, evaluation seeks to make judgments and decisions, based on the assessment data, about the appropriate course of action (i.e., continue on as is, make indicated adjustments, add a needed element, and/or terminate all or part of the program/service) Functionally, evaluation is done as an integral step within assessment conducting by individual programs and/or departments, as well as by their respective schools, colleges, or student services leadership

8 A Side Comment About Terms GOAL OBJECTIVE AIM GOAL OBJECTIVE AIM ALL EQUAL “OUTCOME” A statement expressing intent We use “outcome” as a generic replacement for the other terms

9 Who is Responsible for Assessment? ► The appropriate VP and/or School/College Dean ► Assessment Coordinators in each school, college, or student services ► Department Chair or Program Director ► All faculty and student services staff Assessment is a shared responsibility No one person can do it alone! Successful assessment demands a team effort

10 Doing assessment to collect data (evidence) on educational effectiveness is not a one-time event or an isolated process, but is an on-going, continuous expectation Please Keep in Mind: Assessment is a yearly activity that provides data for Program Review, which is a periodic event

11 The Assessment Process at CBU Step 1 - MISSION  Mission Statement Step 2 - OUTCOMES  Student Learning  Capacity Outcomes (Optional) Step 3 - OVERALL PLAN  Overall Plan 201X-201X Assessment Cycle  Annual Plan  Yearly Report THE BASIC SYSTEM OR “SKELETON”

12 FIRST, LET’S LOOK AT THE OVERALL ASSESSMENT PROCESS DEPICTED IN A FLOW DIAGRAM FIRST, LET’S LOOK AT THE OVERALL ASSESSMENT PROCESS DEPICTED IN A FLOW DIAGRAM

13 The Basic Assessment Process CBU Mission and Priorities and University Student Outcomes (Core 4) CBU Mission and Priorities and University Student Outcomes (Core 4) Student Learning Outcomes (including faith Integration) and Optional Institutional Capacity Outcomes “Mapping” Student Learning Outcomes (including faith Integration) and Optional Institutional Capacity Outcomes “Mapping” WRITE Given the College, School, Department and/or program Mission College, School, Department and/or program Mission WRITE Continued...

14 The Basic Assessment Process Student Learning Outcomes (including faith integration) Optional Institutional Capacity Outcomes Maps Student Learning Outcomes (including faith integration) Optional Institutional Capacity Outcomes Maps WRITE Given the “OPlan” Overall Assessment Plan “OPlan” Overall Assessment Plan WRITE CONTINUED “APlan” Annual Assessment Plan “APlan” Annual Assessment Plan Continued...

15 The Basic Assessment Process THEN WRITE CONTINUED Yearly Assessment Report Yearly Assessment Report DO ASSESSMENT Implement the Annual Assessment Plan; collect data (evidence); analyze the data; make decisions (evaluate); and make any needed changes/adjustments, etc. DO ASSESSMENT Implement the Annual Assessment Plan; collect data (evidence); analyze the data; make decisions (evaluate); and make any needed changes/adjustments, etc. Continued...

16 The Basic Assessment Process CONTINUED WRITE As Needed CBU Program Reviews or WASC Self Studies or Professional Accreditation Reports or Grant Requests Etc. CBU Program Reviews or WASC Self Studies or Professional Accreditation Reports or Grant Requests Etc. Yearly Assessment Reports Yearly Assessment Reports Given the What do you do with the evidence (data)?

17 NOW THAT YOU CAN PICTURE THE WHOLE PROCESS, LET’S QUICKLY CONSIDER EACH ELEMENT IN THE ASSESSMENT PROCESS NOW THAT YOU CAN PICTURE THE WHOLE PROCESS, LET’S QUICKLY CONSIDER EACH ELEMENT IN THE ASSESSMENT PROCESS

18 Mission Statement An adequate mission statement clearly identifies 1) what your program is, 2) what your program does (why you exist), 3) how you do it, and 4) who your program serves (not necessarily in that order)... it is not a idealistic “vision” statement, which delineates what you want to accomplish or become in the future Usually accomplished once; updated from time to time based on assessment data. The program mission must support the CBU mission, priorities, and University Student Outcomes An adequate mission statement clearly identifies 1) what your program is, 2) what your program does (why you exist), 3) how you do it, and 4) who your program serves (not necessarily in that order)... it is not a idealistic “vision” statement, which delineates what you want to accomplish or become in the future Usually accomplished once; updated from time to time based on assessment data. The program mission must support the CBU mission, priorities, and University Student Outcomes

19 1.Write student learning (including faith integration) outcomes 2.Write institutional capacity outcomes (optional) 3.Outcomes-goals are mapped 1.Write student learning (including faith integration) outcomes 2.Write institutional capacity outcomes (optional) 3.Outcomes-goals are mapped Each program may develop as many outcome statements as deemed necessary, but be realistic and pragmatic. What can you really accomplish? OUTCOMES Step Two involves three activities: OUTCOMES Outcomes are used in formulating your program’s Overall Assessment Plan (OPlan) and subsequent Yearly Assessment Plans (APlan)

20 Academic assessment plans include student learning outcomes (including faith integration) and optional institutional capacity outcomes Student Services assessment plans include mainly institutional capacity outcomes (administrative goals), but in some cases also include student learning outcomes OUTCOMESOUTCOMES

21 Overall Assessment Plan - “OPlan” The assessment “Big Picture” or “catalog” listing basic information about the program’s outcomes and related general assessment information As descriptive as necessary to identify the various elements in the overall plan, but exacting specificity is not necessary because such information is reserved for the yearly assessment plan

22 Overall Assessment Plan - “OPlan” The Program’s Overall Assessment Plan identifies: ►Where each outcome is addressed— specific courses, services, internships, service projects, work experiences, independent studies, etc. (based on the previously completed “map”) ►Proposed assessment methods (the ideal is 2 methods per outcome) ►Criteria for each assessment method— the desired, intended performance standard, the level of acceptable performance or accomplishment ►Assessment frequency— how often assessed; yearly, every other year, etc.

23 Measuring Educational Effectiveness Assessment methods must be appropriate to the academic discipline and level or student service area; select and use either or both quantitative or qualitative strategies for measuring stated outcomes There are many different assessment strategies and methods. Select at least one option for each outcome you intend to measure

24 Some Assessment Every Year The goal is to accomplish some assessment each year... accumulating evidence over time, building a weight of evidence. It is not necessary or possible to assess everything every year Assessment must be realistic and manageable So, find what works and do a little each year

25 Annual Assessment Plan - “APlan” Identify two or three (number can vary) student learning outcomes and perhaps optional institutional capacity outcomes to assess during the up-coming year Identify two or three (number can vary) student learning outcomes and perhaps optional institutional capacity outcomes to assess during the up-coming year Focusing on just a few outcomes each year adds up over time and results in all the outcomes being assessed two or three times within a ten-year period Focusing on just a few outcomes each year adds up over time and results in all the outcomes being assessed two or three times within a ten-year period Annual Assessment Plans are developed each year– for the subsequent fiscal year– after the Yearly Assessment Report is completed Annual Assessment Plans are developed each year– for the subsequent fiscal year– after the Yearly Assessment Report is completed In some limited cases, particular outcomes may require assessment each year In some limited cases, particular outcomes may require assessment each year

26 Annual Assessment Plan - “APlan” 1)WHAT assessment method(s)—identify the exact assessment method(s) to be used 2)WHEN the assessment takes place—date(s), time(s) 3)HOW the assessment method is accomplished—the specific steps or procedures, including any necessary materials 4)WHO is responsible to complete the assessment—person or persons who are tasked with actually implementing the assessment process and collecting the assessment data Identify four elements for each selected student learning or optional institutional capacity outcome:

27 During the school year or at other designated times, Time for action... During the school year or at other designated times, collect the desired assessment data (evidence) Follow your Annual Assessment Plan (APlan), but make whatever adjustments are needed along the way to insure collecting the data you need DO ASSESSMENT

28 Doing Assessment This is the “so what” process within assessment It is necessary to take action based on the resulting assessment information Once the desired data (evidence) is collected and analyzed, make decisions (evaluate) supported by data, which lead to making adjustments or changes in the program, curriculum, processes, etc.

29 Yearly Assessment Report Every year after the annual assessment cycle is completed, it is time to file a short report, via LiveText, outlining the results and any actions taken or planned as a result of the assessment data Assessment reports are due July 31, annually, and are based on the preceding fiscal year— July 1 to June 30, including summer sessions Once the report is submitted, it is time to develop next year’s Annual Assessment Plan

30 Yearly Assessment Report (1)List the specific outcome (2)Briefly describe the assessment method(s) used (3)Summarize the evidence collected—what you found out (4)Describe the actions taken or planned based on the assessment evidence (5)(Optional) Cite any lessons learned For each student learning and institutional capacity outcome assessed during the past fiscal year, the assessment report must contain four or five sections:

31  State your program’s mission  Write student learning (including faith integration) and institutional capacity outcomes; “map” your outcomes  Explore and select assessment methods  Develop an overall assessment plan (OPlan)  Each year write an annual implementation plan (APlan)  Do assessment  Collect and analyze data (evidence)  Use data to make identified adjustment, improvements, etc.  Each year report assessment actions and results (Yearly Report)  Implement improvements Summary: BASIC Steps in Assessment

32 LiveText is the administrative software Used to Facilitate the CBU Assessment Process LiveText is the administrative software Used to Facilitate the CBU Assessment Process LiveText is Web-based portfolio software All academic programs and student service departments are required to maintain a assessment portfolio in LiveText The program’s assessment portfolio requires specific assessment documents (OPlan, APlans, etc.), but faculty/staff enjoy wide latitude in writing appropriate outcomes and selecting assessment methods LiveText is Web-based portfolio software All academic programs and student service departments are required to maintain a assessment portfolio in LiveText The program’s assessment portfolio requires specific assessment documents (OPlan, APlans, etc.), but faculty/staff enjoy wide latitude in writing appropriate outcomes and selecting assessment methods

33 Using Assessment Results At the appropriate times, use the collected assessment information to write...  Yearly Assessment Reports  CBU Program Reviews  WASC Self-Study Reports  Professional Accreditation Reports  Other documents requiring assessment data (e.g., grant applications) REMEMBER: Must act upon the assessment data Must act upon the assessment data Assessment is on-going and never ends Assessment is on-going and never ends

34 Office of Institutional Research Planning, and Assessment (OIRPA) ► The OIRPA exists to help academic and student life departments succeed in their assessment activities ► The OIRPA does not do evaluation, but assists programs, as needed, to collect evidence (data) so they can evaluate their own programs ►The OIRPA does not stipulate a program’s outcomes or assessment methods, this is a programmatic prerogative ► The OIRPA depends on the programs’ assessment activities as a basis for making statements about assessment at CBU as a whole ► The OIRPA exists to help academic and student life departments succeed in their assessment activities ► The OIRPA does not do evaluation, but assists programs, as needed, to collect evidence (data) so they can evaluate their own programs ►The OIRPA does not stipulate a program’s outcomes or assessment methods, this is a programmatic prerogative ► The OIRPA depends on the programs’ assessment activities as a basis for making statements about assessment at CBU as a whole

35 Assessment Assistance Who to go to for help: Tools provided to help you: 1. The Assessment Coordinator in your School or College or Student Services 2. OIRPA Assessment training Assessment training Assessment “how-to” manuals Assessment “how-to” manuals Public Folders documents Public Folders documents LiveText software and training LiveText software and training Etc. Etc.

36 Office of Institutional Research Planning, and Assessment (OIRPA) Personnel working with assessment:  Dr. Neal McBride, Associate Provost  Phil Martinez, Director of Assessment  Office Location: James 103  Telephone: Our job is to help YOU succeed in doing assessment

37 Programss with professional accreditation find that 80-95% of the assessment activities necessary for their professional accreditation are also satisfactory for CBU and WASC assessment expectations, with only minor editing required when reporting in LiveText

38 The Assessment Process