1. Assessment 101
Elizabeth Bledsoe

2. Agenda Introduction to Assessment Components of an Assessment Plan
Mission
Outcomes
Measures
Achievement Targets
Question and Answer Session

3. SACS Comprehensive Standard 3.3.1
SACS Expectations
SACS Comprehensive Standard 3.3.1
3.3 Institutional Effectiveness
The institution identifies expected outcomes, assesses the extent to which it achieves these outcomes, and provides evidence of improvement based on analysis of the results in each of the following areas: (Institutional Effectiveness)
educational programs, to include student learning outcomes
administrative support services
educational support services
research within its educational mission, if appropriate
community/public service within its educational mission, if appropriate

4. SACS Comprehensive Standard 3.3.1
SACS Expectations
and provides evidence of improvement based on analysis of the results…
SACS Comprehensive Standard 3.3.1
3.3 Institutional Effectiveness
The institution identifies expected outcomes, assesses the extent to which it achieves these outcomes, and provides evidence of improvement based on analysis of the results in each of the following areas: (Institutional Effectiveness)
educational programs, to include student learning outcomes
administrative support services
educational support services
research within its educational mission, if appropriate
community/public service within its educational mission, if appropriate

5. The Assessment Circle Develop Program Mission & Outcomes
Design an Assessment Plan
Implement the Plan & Gather Information
Interpret/ Evaluate Information
Modify & Improve
Adapted from: Trudy Banta, IUPUI

6. Develop Mission and Outcomes
Develop Program Mission & Outcomes

7. Mission Statement
The mission statement links the functions of your unit to the overall mission of the institution.
A few questions to consider in formulating the mission of your unit:
What is the primary function of your unit?
What core activities are involved?
What should those you serve experience after interacting with your unit?

8. Characteristics of a Well-Defined Mission Statement
Brief, concise, distinctive
Clearly identifies the program’s purpose
Clearly aligns with the mission of the division and the University
Explicitly articulates the essential functions/activities of the program
Clearly identifies the primary stakeholders of the program: i.e., students, faculty, parents, etc.

9. Example of a Mission Statement
“The primary purpose of the Office of Academic Advising is to assist students in the development and implementation of their educational plans. To this end the Office of Academic Advising subscribes to the philosophy of developmental advising; advising is a cooperative effort between advisor and student that consists not only of course planning and selection, but the development of the person as a whole. This includes the selection of career and life-long goals.” (University of La Verne)

10. Outcomes Learning Outcomes Program Outcomes
There are two categories of outcomes:
Learning Outcomes
Program Outcomes

11. Learning Outcomes
When writing Learning Outcomes, the focus must be on the students and what they will think, know, do, or value as a result of participation in the educational environment.

12. Cognitive Learning Affective Learning
Knowledge - to recall or remember facts without necessarily understanding them
articulate, define, indicate, name, order, recognize, relate, recall, reproduce, list, tell, describe, identify, show, label, tabulate, quote
Comprehensive - to understand and interpret learned information
classify, describe, discuss, explain, express, interpret, contrast, associate, differentiate, extend, translate, review, suggest, restate
Application - to put ideas and concepts to work in solving problems
apply, compute, give examples, investigate, experiment, solve, choose, predict, translate, employ, operate, practice, schedule
Analysis - to break information into its components to see interrelationships
analyze, appraise, calculate, categorize, compare, contrast, criticize, differentiate, distinguish, examine, investigate, interpret
Synthesis - to use creativity to compose and design something original
arrange, assemble, collect, compose, construct, create, design, formulate, manage, organize, plan, prepare, propose, set up
Evaluation - to judge the value of information based on established criteria
appraise, assess, defend, judge, predict, rate, support, evaluate, recommend, convince, conclude, compare, summarize
Affective Learning
appreciate, accept, attempt, challenge, defend, dispute, join, judge, praise, question, share, support

13. Examples of Learning Outcomes
Students who participate in career counseling will be able to define the next step(s) in their career development process.
Students will identify various aspects of architectural diversity in their design projects.

14. Program Outcomes Process statements Satisfaction statements
Relate to what the unit intends to accomplish
Level or volume of activity
Efficiency with which you conduct the processes
Compliance with external standards of “good practice in the field” or regulations
Satisfaction statements
Describe how those you serve rate their satisfaction with your unit’s processes or services

15. Examples of Program Outcomes
Process statements
The Registrar’s office will promptly process transcript requests.
Students will utilize the University Writing Center.
Satisfaction statements
Students will report satisfaction in usefulness of the registration system.
Transfer students will report satisfaction with admissions application processing.

16. When Writing Outcomes…
Consider questions such as:
What are the most important results or impacts that should occur as a result of your unit’s activities?
What are your critical work processes and how should they function?
What does the end user experience through interaction with your unit?
What should students in your degree program know, do, and/or value?

17. When Writing Outcomes…
Outcomes should be:
linked to the unit’s mission
realistic and attainable
limited in number (manageable)
something that is under the control of the unit
measurable and/or observable
meaningful

18. When Writing Outcomes…
Outcomes should also:
target key services or change points
use action verbs

19. Student Learning Outcome Example:
Students receiving a degree from this program will be effective communicators.
Students receiving a degree from this program will be able to effectively communicate their research findings both verbally and in writing.
Student Learning Outcome A
Student Learning Outcome B

20. Program Outcome Example:
The Office of Student Financial Aid will respond to all meeting requests within two business days.
The Admissions Office will process applications within a timely manner.
Program Outcome A
Program Outcome B

21. Design an Assessment Plan

22. Assessment Measures After establishing your outcomes…
Define and identify the sources of evidence you will use to determine whether you are achieving your outcomes.
Detail what will be measured and how
Identify or create measures which can inform decisions about your program’s processes and services.

23. Characteristics of an Effective Assessment Measure
Measurable and/or observable
You can either observe it, count it, quantify, etc.
Meaningful
It captures enough of the essential components of the objective to represent it adequately
It will yield vital information about your program
Manageable
It can be measured without excessive cost or effort

24. Types of Assessment Measures (Palomba and Banta, 1999)
There are two basic types of assessment measures: Direct Measures Indirect Measures

25. Direct Measures
Direct measures are those designed to directly measure:
what a stakeholder knows or is able to do (i.e., requires a stakeholder to actually demonstrate the skill or knowledge)
The effectiveness and/or value of the program or process

26. Common Direct Measures
Participation data
Observation of behavior
Collection of work samples (student work)
Volume of activity
Level of efficiency (average response time)
Measure of quality (average errors)

27. Indirect Measures Indirect measures focus on:
stakeholders’ perception of their level of learning
stakeholders’ perception of the benefit of programming or intervention
stakeholders’ satisfaction with some aspect of the program or service

28. Common Indirect Measures
Surveys
Exit interviews
Retention/graduation data
Demographics
Focus groups

29. Choosing Assessment Measures
Some things to think about:
How would you describe the end result of the outcome?
How will you know if this outcome is being accomplished?
What will provide you with this information?
Where are you currently delivering the outcome?
Are there any naturally occurring assessment opportunities?
What measures are currently available?

30. Choosing Assessment Measures
Some more things to think about:
Will the resulting data provide information that could lead to improvement of your services?
Who will analyze the information and how easily will it fit into their regular responsibilities?
How will it fit into your budget and timeline?

31. Achievement Targets
An achievement target is the result, target, benchmark, or value that will represent success at achieving a given outcome.
Achievement targets should be specific numbers or trends.

32. Examples of Achievement Targets
95% of our users will be “very satisfied or satisfied” with our services.
90% of the transcripts will be sent within three days.
Each employee will participate in a minimum of two training/development programs per year.
Students will score a 2.5 out of 4 on the writing rubric.

33. Implement & Gather Information
Implement the Plan & Gather Information

34. Findings
In a WEAVEonline Assessment Report, “Findings” refer to a concise summary of the results you gathered from a given assessment measure.
The language of this statement should parallel the corresponding achievement target.
Results should be described in enough detail to prove you have met, partially met, or not met the achievement target.

35. Examples of Findings Statements
Achievement Target
Overall mean score of students from program will meet or exceed the state average score of 79.
Findings
The achievement target was met. The overall mean score of students from the Teaching, Learning, and Culture program exceeded that of the state average score of the state certification exam. Results: Program overall mean scaled score—91.50, State overall mean scaled score—79.13.

36. Examples of Findings Statements
Achievement Target
90% of the survey results will indicate the highest level of satisfaction with each of the five services provided by the Office of Student Success.
Findings
PM 3/5—edit--- 94% of the survey respondents indicated the highest level of satisfaction with services provided by the Office of Student Success.

37. Interpret/Evaluate Information

38. Analyzing Findings
Reflect on what has been learned during an assessment cycle:
Based on the analysis of the findings, what changes could or should be made to improve the program?
What specific findings led to this decision?

39. Analyzing Findings Three key questions at the heart of the analysis:
What did you find and learn?
So What does that mean for your academic program or support unit?
Now What will you do as a result of the first two answers?

40. Analysis Questions
In WEAVEonline, the Analysis Question responses provide an opportunity to explain the Findings analysis process.

41. Modify/Improve
Modify & Improve

42. Action Plans
After reflecting on the findings, you and your colleagues should determine appropriate action to improve the program.
Actions outlined in the Action Plan should be specific and relate to the outcome and the results of assessment.
Action Plans should not be related to the assessment process itself

43. Action Plans
Establish a plan for using evidence of student learning achievement or service quality at the program level
Establish a decision-making process for approving/implementing recommendations
Clearly identify the parties responsible for implementing the approved recommendations

44. Examples of Assessment Plans

45. Putting It All Together
Outcome: Demonstrate timeliness in processing admission applications.
Measure 1 (Direct): Random sample audit of 100 Applications for Admission received by both the Reception Desk and electronically.
Achievement Target: % of all Applications for Admission will be processed within two weeks of receipt of application.

46. Putting It All Together
Outcome: Undergraduate students who attend the "Registration Overview" during Orientation will be able to successfully utilize the University Registration system Measure 1 (Indirect): Online post-registration survey gathered, compiled and released to the Registrar's Office in late September.
Achievement Target: 80% of the undergraduate students who participate in the "Registration Overview" presentation will answer “Agree” and “Strongly Agree” to the statement: “The "Registration Overview" helped me understand how to register for classes” on an online post-registration survey.

47. Putting It All Together
Outcome: Faculty and staff members who participate in FERPA Rules and Regulations training will be able to demonstrate fundamental knowledge of FERPA rules and regulations that pertain to their roles.
Measure 1 (Direct): FERPA training posttest given to faculty and staff members who participate in the online FERPA training webcourse.
Achievement Target: 80% of the faculty and staff members will achieve a score of 90% or better on their first attempt at the FERPA training posttest.

48. Putting It All Together
Outcome: The Office of Admissions provides useful university information to high school counselors.
Measure 1 (Indirect): An Evaluation Survey is distributed to all participants who attend the Counselor Update seminar. Achievement Target: 90% of high school counselors attending Counselor Update seminars will indicate that the information provided is a “5” (excellent) on a 5 level scale.

49. Putting It All Together
Outcome: University has the optimal number of newly enrolled students to achieve our enrollment goals for the total number of students and credit hour production.
Measure 1 (Direct): The number of new freshmen and transfers enrolled at the conclusion of Late Registration will be used to calculate our success in achieving this outcome for the fall of 2013.
Achievement Target: A total of 2200 incoming freshmen and incoming transfer students are the enrollment targets set by the Committee for fall 2013 with a total enrollment goal of 28, The Enrollment Management Committee sets targets for the optimal number of incoming students. The optimal number of newly enrolled students for each term is determined by projecting the number of continuing students who will enroll, the capacity of the University to provide instructional resources and the needs of the University for tuition revenue.

50. Putting It All Together
Outcome: Improve the quality of commencement ceremony for future participants.
Measure 1 (Indirect): An online post-Commencement survey administered in September 2013 (Summer 2013 graduates), January (Fall 2013 graduates), and June 2014 (Spring 2014 graduates).
Achievement Target: 95% of the recent graduates for Summer , Fall 2013, and Spring 2014 will answer “Yes” to the questions: “Was the information accurate on these aspects of your diploma: (1) your name, (2) your degree type, (3) your major?” on an online post- Commencement survey

51. Continuous Improvement
Office of Institutional Assessment Assessment Report
Continuous Improvement
To fulfill the action plan to address the unmet target of 80% of conference respondents indicating satisfaction with the variety of poster sessions offered, the Office of Institutional Assessment (OIA) along with the Assessment Conference Committee (ACC) sought more variety in the posters for the 2012 Assessment Conference. As a result, the percentage of respondents satisfied with the variety of posters increased from 74% to 78%. Although the 85% target was still not met during the cycle, this result shows improvement towards the target. To complete the other action plan, OIA enhanced the Assessment Review Guidelines to include more practical and applicable “good practices” for assessment liaisons to pass along to their programs as formative assessment. Additionally, the Assessment Review Rubric was modified to be more exhaustive in its evaluation of assessment reports. As a result, less variance was observed in the quality of assessment reports. Lastly, the Vice Provost of Academic Affairs supplied each dean with a college-specific, personalized memo addressing the strengths and weaknesses of assessment reports in each college. This process was well received and will continue as a service to colleges from the Office of the Vice Provost.
Outcome/Objective
Measure
Target
Finding
Action Plan
O 5: Provide Excellent Concurrent and Poster Sessions Provide excellent concurrent and poster sessions for participants at the Annual Assessment Conference.
M 8: Overall Assessment Conference
Survey
85%, or more, of the Annual Assessment Conference attendees will report satisfaction with the Concurrent and Poster Sessions.
Status: Partially Met Following the end of the 13th Annual Texas A&M Assessment Conference, an on-line conference evaluation survey was sent out to all attendees. Information gained from this survey was organized into the 13th Annual Conference Survey Report, and was distributed to the Assessment Conference Committee for review. Results from the survey questions relating to Concurrent and Plenary Sessions are below: Concurrent Sessions - Question 16: "How satisfied were you with the quantity of Concurrent Sessions?" % were "Very Satisfied" or "Satisfied" Question 17: "How satisfied were you with the variety of Concurrent Sessions?" % were "Very Satisfied" or "Satisfied" Poster Sessions - Question 19: "How satisfied were you with the quantity of Poster Sessions?" % were "Very Satisfied" or "Satisfied" Question 20: "How satisfied were you with the variety of Poster Sessions?" % were "Very Satisfied" or "Satisfied"
Although we improved from the findings of 73%, based on our findings from the Assessment Cycle, 77% of respondents indicated that they were satisfied with the variety of poster sessions offered. In response, the Office of Institutional Assessment will seek posters from each track to provide a greater variety of posters during the 14th Annual Texas A&M Assessment Conference.
Use of Results
Although the satisfaction results from the conference survey related to the variety of poster sessions increased from 74% to 78%, the 85% target was still not met. In response, the Office of Institutional Assessment (OIA) and the Assessment Conference Committee (ACC) will ensure that each of the conference “tracks” has coverage in the poster session. OIA and the ACC have traditionally ensured track coverage in concurrent session offerings but has never paid close attention to track coverage in the poster session offerings. This strategy includes contacting specific authors of concurrent session proposals in underrepresented tracks and inviting them to consider a poster presentation, perhaps in addition to the concurrent session. Next, as referenced in the “Enhance Workshop Presentations” action plan for this cycle, according to our “One Minute Evaluations” results, some workshop attendees have requested to see more examples of quality assessment during the workshop. In response, OIA is enhancing our workshop presentations to include screenshots of actual assessment plans and reports from WEAVEonline to allow attendees to work through and critique assessment reports with our staff to gain a better understanding of quality assessment and reporting. One Minute Evaluations will be analyzed again next year to ensure the examples added to the workshops improve our attendees’ reported satisfaction.

52. Take-Home Messages You do not have to assess everything every year
Modify something already being done that is meaningful to the program
Be flexible—this is an iterative process

53. One-Minute Evaluation
What was the most valuable thing you learned?
What is one question that you still have?
What do you think is the next step that your program needs to take in order to implement course embedded assessment?

54. February 16-18, 2014 College Station, TX
For more information on the conference and registration, visit

55. References
The Principles of Accreditation: Foundations for Quality Enhancement. SACS COC Edition.
Banta, Trudy W., & Palomba, C. (1999). Assessment Essentials. San Francisco: Jossey- Bass.
Banta, Trudy W. (2004). Hallmarks of Effective Outcomes Assessment. San Francisco: John Wiley and Sons.
Walvoord, Barbara E. (2004). Assessment Clear and Simple: A Practical Guide for Institutions, Departments, and General Education. San Francisco: Jossey-Bass.
Assessment manuals from Western Carolina University, Texas Christian University, the University of Central Florida were very helpful in developing this presentation.
Putting It All Together examples adapted from Georgia State University, the University of North Texas, and the University of Central Florida’s Assessment Plans