1. ODU Faculty Development March 18, 2011
Academic Assessment
ODU Faculty Development
March 18, 2011

2. Plan for the day Update on changes in Higher Learning Commission
Update on Academic Assessment Webpage
Program Assessment Survey Results &

3. Plan for the day Best Practices in Program Outcomes
Course Matrix Development & Utilization
Open Forum: Posters from Focus Groups
Focus Groups Meet

4. Higher Learning Commission Update
Linda Schoen

5. Update on Academic Assessment Committee Work
Anna Parkman

6. What we’ve been doing Program Assessment Survey Analysis & Comparisons
Designing Webpage for Academic Assessment
Completing and submitting HLC AA Report for Jan 2011 Submission
Working with Focus Groups

7. Program Assessment Survey Results
&
What we’ve learned
Next Steps
Questions

8. Please evaluate the current state of your program mission statement.
Answer Options
Response
2010
Response 2009
We do not have a program level mission statement at this time.
0.0%
Our program mission statement has been developed, but it needs work.
48.0%
51%
Our program mission statement has been developed and we are satisfied with it.
52.0% 49
2010 results updated fall 2011 to include late submissions.

9. Please evaluate the current state of your student learning outcomes.
Answer Options
Response
2010
Response 2009
We don't have student learning outcomes developed at this time.
0.0% 6%
Our student learning outcomes are developed, but need significant work. 4%
Our student learning outcomes are developed, but need review by faculty & some minor revisions.
72.0%
66%
Our student learning outcomes are developed and are in good shape.
24.0%
22%
2010 results updated fall 2011 to include late submissions

10. We don't have an outcome course matrix developed at this time. 16% 24%
Please evaluate the current state of your program outcome course matrix.
Answer Options
Response 2010
Response 2009
We don't have an outcome course matrix developed at this time.
16%
24%
An outcome course matrix has been developed, but needs significant work.
12%
The outcome course matrix has been developed, but needs a review by the program faculty and perhaps some minor revisions.
58%
50%
The outcome course matrix is well developed and is in good shape.
13%
15%
2010 results updated fall 2011 to include late submissions

11. Please evaluate the current state of your student learning outcome measures.
Answer Options
Response 2010
Response 2009
We don't have student learning outcome measures developed at this time.
24%
29%
Student learning outcome measures have been developed, but never utilized to collect data.
16%
14%
Student learning outcome measures have been employed and data collected, but never utilized.
12% 9%
Student learning outcome measures have been employed, data has been collected and has been utilized.
48% 49
2010 results updated fall 2011 to include late submissions

12. Program Learning Outcomes
Best Practices

13. What are program learning outcomes?
Formal statements that articulate:
What students are able to at the completion of the program
Program Goals vs. Outcomes
Process/Fluid

14. Best Practices Simply put
Know what you are doing
Know why you are doing it
Know what students are learning as a result
Changing because of that information

15. Writing Learning Outcomes
Learning Outcomes Formula
Bloom’s Taxonomy
Characteristics of Good Learning Outcomes

16. 1. What do you want the student to be able to do?
This question asks you to develop the outcome.
For Example:
Student identifies, consults and evaluates reference books appropriate to the topic in order to locate background information and statistics.
ACRL’s IIL Immersion Summer 2005

17. Learning Outcomes Formula
Verb Or
Action Phrase
Great Learning Outcomes +
Why? =
“In Order To” OR
What students need to know?
“Student identifies, consults and evaluates reference books appropriate to the topic”
Why do they need to know this?
“locate background information and statistics.”
“In Order To”
ACRL’s IIL Immersion Summer 2005

18. Bloom’s Taxonomy

19. Characteristics of an Effective Outcome
A good outcome statement
Consists of action verbs
Uses simple language
Describes student rather than instructor behaviors
Focuses on the end result

20. Characteristics of an Effective Outcome
Is measurable
Is limited to a single outcome
Is attainable by the end of the program
Is clear to those outside the discipline
Can be validated by colleagues

21. Problematic Program Outcomes
The graduate
develops the knowledge, experience and abilities required to be successful in this fascinating world of art history.
identifies competing values in business decisions and can act with a commitment to integrity in interactions that lead to the development of trustworthy relationships with stakeholders - customers, colleagues, suppliers and other key business partners.

22. Example 1: Strong Set of Outcomes
Accurately interprets the biological basis of phenomena while recognizing the limitations of scientific analysis
Demonstrates proficiency in data gathering.
Designs, conducts and communicates biological experiments
Applies concepts of biology to societal issues

23. Review and Discussion of Your Program Outcomes at Your Table

24. Review your program outcomes
Are there “and” statements that make the statement too confusing?
Are these higher order bloom statements?
Can you get them down to 5-8 outcome statements?
Can you measure them?
What will you do next?

25. A Great Resource

26. The Program Learning Outcomes/Course Matrix
Jim Cottrill

27. Purpose of the outcomes/course matrix
Planning:
Are the components necessary for student success in place and correctly ordered?
Assessment:
Has everything that we ought to measure been measured?
Is there a coordinated effort to assess SLO across courses from introduction to mastery?

28. Two Types of Matrices for Planning and Assessment
Course level matrix maps KSDs to particular assessment items planned for a course.
Used to ensure that all KSDs are assessed
Visually depicts the “balance” of the course

29. Two Types of Matrices for Planning and Assessment (cont.)
Program level matrix maps SLOs to the courses required in the program.
Assumes a mapping between course KSDs and program SLOs
Maps progress from introduction to reinforcing to mastery of outcome (I recommend using I, R, M)
May identify lack of balance in levels, such as mastery in 100 level course or introduction in Capstone

30. Some examples:

31. Some examples:

32. S2 To develop students’ expertise in specific technology usage.
CIS 150
CIS 180
CIS 234
CIS 316
MTH 110
MTH 145
PHL 101
Knowledge
1.1 X
1.2
1.3
2.1
2.2
2.3
3.1
3.2
3.3
3.4
4.1
4.2
4.3
S2 To develop students’ expertise in specific technology usage.
2.1 Students will develop a proficiency with database management systems.
2.2Students will be able to explain various network topologies and protocols.
2.3Students will be able to analyze network graphs to determine efficient paths.

33. Outcome Matrices Division of Business Overall
I=Introduce R=Reinforce E=Emphasize
HST
106
107
201
202
231
232
330
498
Students will develop the ability to read and write critically about historical subjects R
R/E
Students will develop the ability to research, to organize material, and to present their findings clearly and cogently
Students will develop the ability to analyze and synthesize various approaches to history, and primary and secondary documents
I/R

34. Review & Discussion of Your Matrix at Your Table
Do You have one?
What do you need to update/change?
Next Steps

35. What do program faculty do in the “focus group” year Math Biology Env
With Time & Energy
What do program faculty do in the “focus group” year
Math
Biology
Env

36. FAQ What do program faculty do in the “focus group” year Math ppt
Biology ppt

37. Focus Groups Work One
Program Mission Statement
Program Outcomes Statements
Program Outcomes/Course Matrix
Outcome Measures Utilization
5-Year Assessment Plan

38. Closing 2011-2012 PAS Program Outcomes Program /Outcomes Matrix
Focus on measurement tools