2. Building Assessment Skills and Competencies

3. Science Education Assessment (SEA) Scholars: A local, authentic approach to teaching graduate students about assessment
Katie Kearns and Lisa Wiltbank
Representing Indiana University Bloomington

4. Assessment of information literacy skills
Undergraduate achievement
Graduate student development of assessment skills
Faculty culture around course and curriculum improvement
KDK

5. Authentic context: Assessing undergraduate information literacy skills
SEA Scholars (grad-faculty partnerships)
College-level assessment for accreditation
Biology undergraduate major program goals
Information literacy among biology majors
KDK

6. Graduate student learning outcomes: Authentic methods - performance of assessment
Wrote learning outcomes for information literacy across the curriculum
Program
Developed meaningful, assessable assignments
Wrote rubrics
Collected, analyzed, interpreted evidence of student learning from courses
Course
Program
Synthesized assessments data
Reported on assessments at curriculum level
Shared findings and recommendations with program faculty
LBW

7. Graduate student learning outcomes: Additional assessment methods
Pre- and post-program assessment
15 words you associate with “assessment”
For what reasons might an instructor find assessment of student learning valuable?
What methods might instructors use to assess student learning?
Regular reflection and feedback on SEA Scholar meetings
KDK

8. Graduate student learning outcomes: Cognitive assessments – knowing about assessment
First list was synonyms
Second list of nouns was longer, course-embedded, and multi-modal: learning outcomes, quantitative, qualitative, essay, exam, quiz
What nouns do you associate with assessment?
First list was outcome-oriented: effectiveness, evidence, failure, success
Second list was process-oriented: improvement, feedback, progress, accountability
What outcomes do you associate with assessment?
KDK

9. Reflections on learning
Graduate student learning outcomes: Open-ended assessment – reflecting on identity
“We have been exposed to the biology undergraduate program in a broader and deeper way than we could otherwise.”
“Graduate students must think about improving student learning both on the course level and at the larger scale of the entire curriculum, which is a way of thinking that may be entirely new to the graduate students.”
Reflections on learning
Course AND curriculum view of assessment
Role negotiation
Developing an expert identity with faculty
Learning how to ask for things
LBW

10. Graduate student learning outcomes: Beyond the formal assessments – authentic products
Dissertation chapters about course and curriculum assessment projects
Book chapter
Manuscript
Career choices
LBW

11. Lessons and recommendations for teaching future faculty about assessment
Instruction process
Engage in authentic work
Encourage interprofessional collaborations – faculty, graduate students, staff
Ongoing “tasting the soup”
Assessment methods
Ask open-ended questions
Look long-range for outcomes
Provide opportunities for multiple audiences for reflection
Include those being assessed in what they want to be assessed on
The flip side: the authentic assessment project ended, as did the SEA Scholar partnerships.
KDK

12. Building Assessment Skills and Competencies – Perspectives from Cornell University
Colleen McLinn
Director, CIRTL at Cornell
Cornell University Graduate School
&
Derina Samuel
Associate Director for Graduate Student Teaching Development
Center for Teaching Innovation
Speakers represent the two main collaborating units, but also worked with the Knight Institute for Writing in the Disciplines, the Office of Undergraduate Biology, the Department of Physics, and others. Dean Barbara Knuth of the Graduate School was the PI for this project.

13. Key Activities PREPARE Workshops (17) Visiting Speakers CIRTL Webinars
University-Wide Conversations
First-Year Writing Seminar Assessment Fellows (16)
Biology and Physics TAs
Disciplinary Skills Development
17 future faculty earned Prepare certificates
16 FWS assessment fellows conducted TAR projects
University-wide and beyond: Create new opportunities for grad students and postdocs to learn about defining / assessing undergraduate learning outcomes
PREPARE workshop series and certificates
Visiting speakers and symposia
CIRTL Network webinar series
Disciplinary: Develop assessment and action research skills
First-Year Writing Seminar Assessment Fellows designed, conducted, presented Teaching as Research projects
Customized Biology and Physics TA workshops
Flipping the classroom
Clickers, concept inventories, etc.
Project website -

14. Evidence of Effectiveness
Workshop session evaluations
Artifacts from fellows’ Teaching as Research projects (posters, papers)
Pre-/post-surveys of fellows
Topics: How effective was your university? Were they prepared?
At the beginning, did focus groups for needs assessment

15. Specific Competencies and Themes
Identity and interactions
Content and opportunities
Varied sources, including qualitative
Use findings to improve
In-and out-of- class time
Active and student-centered
Goals for students
Skills needed in the discipline
Think big
Use time
Consider diversity
Collect evidence
1. Thinking big about goals for students, and skills needed in the discipline
2. Thoughtful use of in- and out-of-class time
3. Active and student-centered pedagogies
4. Inclusive Teaching: Attending to learner identity and interactions and instructor identity. Varied sources of content and opportunities to demonstrate learning.
5. Identifying possible sources of evidence for student learning, including qualitative measures
6. Using your findings to improve teaching

16. Successes Student perspectives
Framing assessment as integral to future faculty preparation
Necessary skills for designing courses
Especially beneficial for humanities students
Creating learning communities focused on assessment
Benefits of Teaching as Research projects
Future faculty as project coordinators
Frame assessment as integral to future faculty preparation:
Workshops
Consultations on writing competitive job applications
Student perspectives
Explicit focus on learning outcomes and assessment is logical step in designing courses
Seek to assess own teaching effectiveness
Surveys, rubrics, formative assessment
Humanities students more directly involved with course design, already oriented to careers in the professoriate
Cohort-based programming and discipline-specific examples more coherent
Benefits of students working in groups to conduct Teaching as Research (TAR) projects
Cost-effective, shorter-term
Raised disciplinary faculty awareness of and interest in TAR
Faculty partner engagement in TAR
Identify candidate students
Generate research questions
4 – both success and challenge
Targeted outreach
Develop and facilitate workshops
Coordinate certificate programs (STEM, Humanities)
Near-peer mentoring
Realistic examples of assignments and questions

17. Challenges Future faculty coordinators time- constrained
STEM TAs early in graduate school, lots to learn
STEM faculty not initially eager to involve TA viewpoints in course overhaul discussions
Insufficient resources to develop humanities certificate

18. In Hindsight, and for the Future…
More direct observations and longitudinal studies of future faculty
Develop a day-long or multi-day institute in course design and student learning assessment
More discussions bringing together faculty and teaching assistants

19. Thank you!
Questions?

20. Building Assessment Skills and Competencies
Think/Pair/Share
In groups of two
2 minutes: Think about what is the most important skill or mindset that future faculty need for assessing the learning of undergraduates?
Share your response with your partner

21. Building Assessment Skills and Competencies
Think/Pair/Share
In groups of two
2 minutes: Think about what was the most successful strategy you used for helping graduate students assess student learning?
Share your response with your partner