11 Reporting Outcomes Results and Improvements Produced by Non-Instructional Subcommittee of Assessment Committee

22 Review of PET Form Past year’s PET Form instruction included developing –Purpose Statement –Goals –Objectives (preferably as Outcomes)

33

44 Review of Purpose Purpose Statement should: –fall within AC’s mission –identify THE reason for the department’s/division’s/program’s existence –encompass all the key aspects of the department/division/program

55 Review of Goals Goals should be: –based on the purpose for the division/department/program –stated as broad/general/vague expectations for a division/department/program –recommend 2 to 4 with no more than 5 (address the key areas desired for assessment and resulting improvement and/or growth)

66 Review of Objectives –the person or group expected to achieve the objective –the specific task to be accomplished –the standard/benchmark/target to be achieved –the timeline or deadline for completion Objectives should identify:

77 Review of Objective Types Types of objectives and definitions: –Inputs = resources –Outputs= types, levels and targets of services (may include satisfaction) –Processes = efficiencies or intended accomplishments –Outcomes = specific changes in student’s/client’s knowledge, skill, expertise, attitude or behavior

88 Review of Outcomes as Objectives Outcomes should be: –a statement of what someone should be able to do or develop (knowledge, skill, expertise, attitude or behavior) –written from the audience’s (e.g. student or client) perspective –the preferred type of objective NOTE: Each dept./program should develop at least one outcome and include in PET form. May have other types of objectives (e.g. output objectives, process objectives) in PET form as well.

99 Review of Assessing Outcomes All outcomes (objectives) do not have to be assessed all of the time All outcomes (objectives) do have to be assessed sometime (maybe some every semester and others annually or monthly) Adapted from James Madison University

10 New information: Results Column ANNUALLY: –Report the Results for each objective (outcomes, outputs, processes or inputs) –Reported results will be data for quantitative objectives –Reported results will be summary narrative for qualitative objectives

11 ResultsResults

12 ResultsResults Quantitative –Statistical –Time reference ? Participation %(N=H) Qualitative

13 New information: Use of Results Column ANNUALLY: —Report the Use of Results for each objective (outcomes, outputs, processes or inputs) —Reported Use of Results will be the revisions and/or improvements recommended or made as a result of analysis of the evidence in the Results column —Be sure to provide a summary of the analysis in the Use of Results column before identifying the revisions and/or improvements recommended or made

14 Use of Results

15 Use of Results Sentences describing how the department will make improvements based on the data Provide an answer of what will be done next

16 Outcomes - Example 1 Outcomes (objectives) should be worded to express what the student/client will learn, know, or do as a result of the intervention – NOT what the instructor or service will do for the student/client. –Typical Outcome: Provide student/client with knowledge about how the library works. –Better Outcome: After taking the Research Methods course, student/client will be able to demonstrate their knowledge of how the library works by finding 10 sources for a research paper in the library. Adapted from James Madison University

17 Outcomes - Example 2 Outcomes should be reasonable. They should reflect learning that the student/client can accomplish as a result of the division/department/program. –Typical Outcome: Students will demonstrate open- mindedness for all cultures by strongly agreeing with all of the items on the Open-Mindedness Inventory (OMI). –Better Outcome: Upon completion of the Study Abroad program, participants will show an increase in open-mindedness through at least a 10-point increase on the Open-Mindedness Inventory (OMI). Adapted from James Madison University

18 Outcomes - Example 3 Student/client learning should be assessed with an observable, measurable outcome. Verbs such as “know” and “understand” are not observable. –Typical Outcome: Students will understand the electoral college. –Better Outcome at intervention method: Students will be able to match at least 40 of the United States with their number of electoral votes. –Or: Students will be able to evaluate (e.g. Govt of US Courses) the effectiveness of the electoral college in modern elections. Adapted from James Madison University

19 Outcomes - Example 4 Outcome should specify the criterion of ACCEPTABLE student/client performance. Goal: Understand the concepts that contribute to career decision- making. Typical Outcome: Students will be able to select a career choice. Better Outcomes: Upon completion of the career and life planning course, students will be able to match a list of majors to appropriate career choices. OR: Upon completion of the career and life planning course, students will be able to state their “work personality” as measured by the Work Abilities, Values, and Interests (WAVI) Inventory. Adapted from James Madison University

20 Outcomes - The ABCDE Method A = Audience (Who are you assessing?) B = Behavior (What is expected of the audience?) C = Conditions (What intervention is required?) D = Degree (To what degree is the behavior to be reached?) E = Evaluation (What tool will be used to measure this?) Recipe for Writing an Outcome Statement = C + A,B + D,E Adapted from James Madison University

21 Outcomes - ABCDE Example 1 (Police Dept) A = Audience (employees) B = Behavior (will take precautions) C = Conditions (After mock drills) D = Degree (decrease by 20% annually) E = Evaluation (number of reported campus burglaries) Outcome statement = After mock drills, employees will take precautions, which will result in at least a 20% decrease annually in the number of reported campus burglaries. Adapted from James Madison University

22 Outcomes - ABCDE Example 2 (IST HelpDesk) A = Audience (employees) B = Behavior (indicate that the technology problem was resolved during the time the problem was identified) C = Conditions (After seeking assistance with a technology request) D = Degree (75%) E = Evaluation (automatically generate an with an anonymous web survey link) Outcome statement = After requesting assistance with a technology request, employees seeking assistance with a technology request will indicate that the technology problem was resolved during the time the problem was identified 75% of the time and automatically generate an with an anonymous web survey link. Adapted from James Madison University

23 Outcomes - ABCDE Example 3 (Student Services) A = Audience (employees) B = Behavior (demonstrate mastery of FERPA regulations) C = Conditions (After training session on FERPA requirements) D = Degree (80%) E = Evaluation (on post-test) Outcome statement = After attending a training session on FERPA requirements, all participating employees will demonstrate mastery of FERPA regulations at 80% on the post-test. Adapted from James Madison University

24 Outcomes ABCDE Example 4 (Testing Services) A = Audience (faculty (FT & PT) requesting testing services) B = Behavior (will follow identified procedures) C = Conditions (After reading the website on testing instructions and completing corresponding survey) D = Degree (60%) E = Evaluation (based on log of all procedure violations) Outcome statement = After reading instructions regarding testing from a website and then completing a corresponding survey, 60% of faculty (full-time & part-time) requesting testing services will follow identified procedures based on log of all procedure violations. Adapted from James Madison University

25 Outcomes - ABCDE Example 5 (Continuing Education) A = Audience (companies with 25+ employees) B = Behavior (implement the program addressing generational differences in the workplace) C = Conditions (After attending a pilot program) D = Degree (25% of companies) E = Evaluation (based upon signed Training Agreements.) Outcome statement = After attending a pilot program, 25% of companies with 25+ employees will implement the program addressing generational differences in the workplace based upon signed Training Agreements. Adapted from James Madison University

26 Outcomes - ABCDE Example 6 (Business Office) A = Audience (employees with purchasing privileges) B = Behavior (will file paperwork in a complete and timely fashion in order that vendors will be paid by due date) C = Conditions (After receiving procedures instructions) D = Degree (75% of the time) E = Evaluation (based upon a transaction log.) Outcome statement = After receiving procedural instructions, employees with purchasing privileges will file paperwork in a complete and timely fashion so that vendors will be paid by due date 75% of the time based upon a transaction log. Adapted from James Madison University

27 Outcomes Practice BRAINSTORM 1.Purpose - restate the purpose statement: (What is the primary reason that this division/department/program exists?) 2. Goals - select 1 or 2 key goals: (Based on the purpose statement and experiences of the staff, what are the biggest challenges or problems that this division/department/program desires to improve?)

28 Outcomes Practice BRAINSTORM 3. Outcomes: Fill in each of the following for one outcome of one goal: A = Audience - B = Behavior - C = Conditions - D = Degree - E = Evaluation - Write as a sentence (see recipe): Outcome Statement = Adapted from James Madison University

29 AssignmentAssignment PET form: Use the skills you’ve developed to write at least one outcome for the division’s/department’s/ program’s PET form.

30 AssignmentAssignment PET form AC Website ( Scroll to bottom of screen AC Forms & Policies Planning & Evaluation Tracking (PET) Form

31 AssignmentAssignment Recommendation: All employees in the division/department/ program should work together to brainstorm and decide on finalized PET form.

32 AssignmentAssignment Finalized PET form should be approved through appropriate President’s Cabinet member.

33 AssignmentAssignment Submit finalized PET form electronically to: Brandy Hayes in the Office of I.E. & Advancement Deadline for submission: December 8, 2006