1. Where We Are & What We’ve Learned So Far
FYS Assessment
Where We Are & What We’ve
Learned So Far

2. Fall 2015 FYS Instructor Survey
76% include participation as part of grade (median % = 10%)
90% using WyoCourses (59% using extensively, 31% occasionally)
# of assignments – median 10, average 11.5
Wide range of assignments in FYS courses
Research papers, reflective papers, other papers, group projects, group presentations, annotated bibliographies, library research/citation exercises/library modules, exams and quizzes, case studies, letter writing, debates, service learning/community engagement projects, interviews, speech, poster, video, blog, lab exercise, group-led discussion, resume, magazine cover, budget exercise, threaded discussion analysis, personal leadership philosophy, autobiography, etc. etc.

3. Distribution of SLOs Preliminary analysis = We’ve got them covered.
Major Project
17%
16%
Assign 2
18%
15%
Assign 3
13%
14%
Assign 4
Assign 5
Assign 6
20%

4. Fall 2015 Data Collection
Draft FYS rubric and SLOs made available to faculty on WyoCourses
18 instructor volunteers submitted example of student work
Instructors asked which SLOs were covered by assignment
Most, but not all assignments were submitted via WyoCourses
About 450 pieces of student work collected

5. Spring 2015 Examination of Student Work
Recruited group of 8 FYS instructors to examine student work
Used WyoCourses to evaluate student work (each person had 2-4 assignments)
Group discussion of process itself
Revised rubric
Read assignments and applied rubric
Met this month to discuss findings and next steps

6. Old vs. New Rubric Changed header names and assigned numerical values
Revised language (ex. 2 vs. 3)
Some inconsistent (comparing apples to oranges)
Removed aspects of outcomes that could not be reasonably evaluated by out readers after the fact

7. Quick Results SLO 2A: Avg 1.53, Med 2 SLO 2B: Avg 1.11, Med 1
SLO 3: Avg 1.36, Med 1
SLO 4: Avg 1.68, Med 2
SLO 5: Avg 1.10, Med 1
SLO 6A: Avg 1.88, Med 2
SLO 6B: Avg 1.64, Med 2
479 duplicated pieces of student work rated.
SLO 6A and 6B rated the most.
SLO 5 rated the fewest.

8. Themes We Discussed
Still have about 250 pieces of student work to rate
Rubric seems to be ok to proceed
Difficult for raters to see all of the SLOs in various assignments (e.g. instructors said they were in assignment, but readers couldn’t see it clearly)
Difficult to evaluate 4-6 SLOs per assignment
Sometimes assignment details weren’t always clear
Even though raters struggled a bit with ratings, overall ratings were fairly consistent between raters (O vs. 1 or 1 vs. 2 or 2 vs. 3). But will examine further.
Would be better that raters talk and ask questions of each other instead of working independently
Expertise is needed to develop consistency in ratings over time

9. Recommendations Continue to use same rubric
Examine inter-related reliability
Develop template for reporting results
Statistics
Qualitative themes the readers see
Areas of focus for improvement for future FYS courses
Professional development opportunities
Regular communication with FYS instructors
University-wide recommendations to “close the loop”

10. Summer Work
4 volunteers from initial group to evaluate remaining FYS student work from
Continue to discuss identified issues
Decide on a report template, analyze results, and draft first report
Working toward completion of HLC Quality Initiative Report due in Jan 2017

11. Fall 2016 Collection of Student Work
FYS instructors asked to submit example of student work for class
One example focusing on one or two SLOs
Examples of student work must be submitted on WyoCourses because we have a new LTI system built especially for assessment purposes at UW

12. Directions Assignment must be in assignment box
Student work uploaded by students
Instructor tags assignment for assessment
Instructor attaches rubric with SLO(s) – one or two
Rubric is ready made. Instructor attaches and deletes all but selected SLOS
Specific instructions will be coming out this summer to FYS instructors (sometime in June)
In order to receive USP designation, you are required to participate in assessment process