1. Director, Center for Teaching, Learning, & Assessment
Disseminating Results of IU Kokomo’s Assessment of Student Engagement
Voluntary System of Accountability (VSASM)
NSSE/USA Today Initiative
Sharon K. Calhoon
Director, Center for Teaching, Learning, & Assessment

2. The National Issues
2006 Report of the Spellings Commission on the Future of Higher Education—4 concerns
Access
Affordability
Quality of instruction
Accountability
US News & World Report rankings
Focuses on measures (e.g., resources, reputation) NOT shown to be related to student learning
Institutions who (rightfully) criticize these rankings are viewed as being unwilling to be accountable
Margaret Spellings is Secretary of Education
Report released 9/26/06

3. Taking Control of the Conversation
Two New National Projects
Voluntary System of Accountability (VSA)
National Survey of Student Engagement (NSSE)/USA Today initiative

4. VSA & NSSE/USA Today Projects
Institutions voluntarily publish assessment/accountability data
Institutions are NOT ranked
Changes focus from institutional “prestige” to institutional mission and student learning
IU Kokomo is participating in both projects

5. Voluntary System of Accountability (VSASM)
A response to the Spellings commission concerns
A joint project of:
American Association of State Colleges and Universities (AASCU)
National Association of State Universities and Land Grant Colleges (NASULGC)
IU Kokomo is member of AASCU

6. VSA Goals “Demonstrate accountability and stewardship to [the] public”
“Measure educational outcomes to identify effective educational practices”
“Assemble information that is accessible, understandable, and comparable”
“Improve public understanding of how public colleges and universities operate”
“Affirm the significance of the many diverse missions of U.S. higher education”

7. VSA Features a web template, the College Portrait, for reporting data
College Portrait has a section on Student Experiences and Perceptions
Institution reports data from one of four national student engagement surveys
We will report our National Survey of Student Engagement (NSSE) scores

8. NSSE/USA Today Initiative
Goals
“to educate the public about the link between student engagement and a high quality undergraduate experience”
“provide information…about aspects of collegiate quality not [readily] available”

9. Our NSSE Results in VSA Format
See “VSA College Portrait for IU Kokomo” (last page of handout)
Colleges choosing to participate in VSA may choose which survey to give, but they must report their scores on these items, using this format

10. NSSE Data in USA Today Format
Shows our scores on five “Benchmarks of Effective Educational Practice” and average scores of our peer group
Links to our NSSE page, which gives more detail and highlights our strengths:

11. More About NSSE Administered by IU Center for Postsecondary Research
More than 500 institutions nationwide participate each year
Survey of first year and senior students
Designed to measure “Student Engagement”
We’ve participated 5 times since 2000

12. Two Components of Student Engagement
More About NSSE
Two Components of Student Engagement
What students do—time and energy spent devoted to educationally purposeful activities
What institutions do—using effective educational practices to induce students to do the right things

13. Why Engagement?
Student engagement is one measure of the quality of the institution
Research shows that students who are engaged
Learn more
Move further along in their development
Are more likely to persist to complete their degree

14. NSSE at IU Kokomo Respondents Benchmarks Difference Scores
Overall Satisfaction

15. Respondent Characteristics
How they spend their time compared to peers:
Less time working for pay on campus
More time working for pay off campus
Less time in co-curricular activities (except 2006)
About the same amount of time preparing for class
About the same amount of time relaxing & socializing
More time caring for dependents living with them (except 2006 FY)
About the same amount of time commuting to class
Refer to handout with demographic data
Re commuting to class: (3 exceptions: FY in 2006 spent less time, FY & Sr in 2005 spent more time)

16. Benchmark Scores Level of Academic Challenge
Active and Collaborative Learning
Student-Faculty Interaction
Enriching Educational Experiences
Supportive Campus Environment

17. Level of Academic Challenge
First Year Students
Senior Students
Peers are Carnegie Peers (no data for 2004); Bac-Gen become Baccalaureate-Diverse Fields in 2006
2005: p < .01 for FY Bac-Gen; p < .05 for Sr Selected Peers; p < .001 for Sr Bac-Gen
No tests of significance for 2006 available (I would have to calculate them by hand)

18. Active & Collaborative Learning
First Year Students
Senior Students
Peers are Carnegie Peers (no data for 2004)
Bac-Gen became Bac-Diverse in 2006
No significance information for 2006 Bac-Gen.

19. Student-Faculty Interaction
First Year Students
Senior Students
Peers are Carnegie Peers (no data for 2004). Bac-Gen became Bac-Div in 2006.

20. Enriching Educational Experiences
First Year Students
Senior Students
Peers are Carnegie Peers (no data for 2004); would need to recalculate if use Bac-Gen

21. Supportive Campus Environment
First Year Students
Senior Students
Peers are Carnegie Peers (no data for 2004); would need to recalculate if use Bac-Gen

22. Mean Differences from Selected Peers 2005 - 06
Peers are Carnegie Peers (no data for 2004); would need to recalculate if use Bac-Gen
2004 did not have selected peer group—peer group was Bac-Gen. Error bars show standard error of the mean. (SE is much larger for difference scores, because there are several sources of error.)
* p < ** p < *** p < .001

23. Significant Mean Differences from Selected Peers 2005 - 06
Level of Academic Challenge
* p < ** p < *** p < .001

24. Overall Satisfaction Items
Overall, how would you evaluate the quality of academic advising you have received at your institution?
How would you evaluate your entire educational experience at this institution?
If you could start over again, would you go to the same institution you are now attending?

25. Academic Advising: Peer Comparison
Overall, how would you evaluate the quality of academic advising you have received at your institution?
1 = Poor
2 = Fair
3 = Good
4 = Excellent
Peers = Bac-Gen for 2004; Selected Peers for 2005 & 2006
FY 2004 p < .05
FY 2005 p < .01 (effect size -.25)
Sr 2005 p < .01 (effect size -.24)
Sr 2006 p < .05 (effect size -.29)

26. Academic Advising: Frequencies
First Year Students
Senior Students

27. Entire Educational Experience: Peer Comparison
How would you evaluate your entire educational experience at this institution?
1 = Poor
2 = Fair
3 = Good
4 = Excellent
(Peers = Selected Peers)
2004 FY p < .05
2005 FY p < .01
2005 Sr p < .05
(all effect sizes in mid 20s)

28. Entire Educational Experience: Frequencies
First Year Students
Senior Students

29. Choose Same Institution: Peer Comparison
If you could start over again, would you go to the same institution you are now attending?
1 = Definitely No
2 = Probably No
3 = Probably Yes
4 = Definitely Yes
Peers = Bac-Gen for 2004 & 2005; Selected Peers for 2006
2006 FY p < .05 (effect size = -.27)
No other significant differences

30. Choose Same Institution: Frequencies
First Year Students
Senior Students