1. Direct vs Indirect Assessment of Student Learning: An Introduction
Dr. Sheila Handy, Chair Business Management and Co-Chair University Assessment Committee
Sponsored by University Assessment Committee
February 24, 2014

2. Assessment
“The most important purpose of assessment should be not improvement or accountability but their common aim: Everyone wants students to get the best possible education. Everyone wants them to learn what’s most important. A college’s mission statement and goals are essentially promises that the college is making to its students, their families, employers, and society. Today’s world needs people with the attributes we promise. We need skilled writers, thinkers, problem-solvers and leaders. We need people who are prepared to act ethically, to help those in need, and to participate meaningfully in an increasingly diverse and global society. Imagine what the world would be like if every one of our graduates achieved the goals we promise them! We need people with those traits, and we need them now. Assessment is simply a vital tool to help us make sure we fulfill the crucial promises we make to our students and society. “
Linda Suskie, 2010

3. Continuous Improvement of Student Learning
Articulate Goals
Offer Programs, Services and Initiatives
Collect Assessment Information
Use Results for Improvement

4. Setting the Stage Middle States Commission on Higher Education
“…each program should employ direct measures of student learning…”
“…each stated learning outcome should be aligned to a measure, a performance goal, and a finding. Measures can be used to evaluate more than a single learning outcome.”

5. Assessment Measures - Definitions
Direct
Indirect
Direct examination or observation of student knowledge, skills, attitudes or behaviors to provide evidence of learning outcomes.
Perceived extent or value of learning experiences

6. Assessment Measures - Examples
Direct
Indirect
Exam
Standardized
Locally developed
Embedded questions
Juried review
Portfolios with rubrics
Surveys
Students
Alumni
Focus Groups
Student Records
Class Attendance

7. Short Table Activity Introductions Envelopes
Match Assessment Measure and Description
Sort into Direct and Indirect Measures

8. Discussion of Direct Measures

9. Standardized Tests Licensing Exams Major Field Test
Education - Praxis, PAPA
Nursing - NCLEX
Accounting - CPA Exam
Major Field Test
Developed by ETS
Administered in a proctored environment as paper and pencil or online

10. MFT subjects Biology Business Chemistry Computer Science
Criminal Justice
Literature in English
Math
Physics
Political Science
Psychology
Sociology

11. Standardized Tests Carefully developed Highly reliable
Advantages
Disadvantages
Carefully developed
Highly reliable
Professionally scored
Nationally normed
If tests don’t match LOs of program, scores will be low
MC questions that test facts – faculty may focus on higher order skills
May be expensive
Students may not take seriously

12. Locally Developed Exam
Advantages
Disadvantages
Can be tied-in to specific LOs
Can be administered as part of regular semester testing
Can use publisher developed questions
Reliability and generalizability not known
Time required to prepare questions
Scoring may take a long time
Norms not available

13. Embedded Questions Can be conducted as part of regular student testing
Advantages
Disadvantages
Can be conducted as part of regular student testing
Does not require collection of additional data
Can be used for both grading and assessment
Faculty may think they are being assessed
Takes time to develop and grade

14. Embedded Questions One way to assess learning without faculty “angst”
Add assessment questions as a quiz administered at the beginning of the next course in the sequence
Effective if different faculty teach multiple sections of an introductory course

15. Juried Review Suitable for assessing Performing arts projects
Written assignments
Oral presentations

16. Juried Review Provides experience and practice for student performers
Advantages
Disadvantages
Provides experience and practice for student performers
Can be formative and summative – indicating “value added”
Can also introduce students to the standards they will be expected to achieve
Achieving consensus with regard to the evaluation rubric
May not be sufficient to determine if students have achieved the LOs set by the faculty

17. Portfolios With Rubrics
Suitable for many majors
Art
English
Computer Science
Business
Education

18. Portfolios With Rubrics
Advantages
Disadvantages
Provides students with evidence to submit to potential employers
Tracks student work over time
Can assess higher levels of learning
Allows students to reflect on their learning
Faculty must agree on assessment rubric
Time consuming
Storage of evidence

19. Case Study
Your goal is to develop a plan to assess this student learning outcome for the Engineering Program for the upcoming academic year. This plan should include two direct measures.

20. Wrap Up Resources Materials in Folders University Assessment Committee
Assessment Consulting Team
Website