1. How Technologically Literate are EMCC Students ?

2. Technological Literacy A COMPARISON OF 2012 AND 2015 ASSESSMENTS SAAC REPORT 11/25/15

3. Purpose and Methodology  Spring 2012: Technological Literacy assessment rubric developed  Designed to assess student technological literacy across the College  Spring 2015: Reassessed with same rubric

4. Purpose and Methodology Seven common “deliverable components” were assessed:  Data management  References  Assignment content  Communication  Layout  Embedded objects  Conventions

5. What was Assessed?  Data managing : was the respondent’s name properly displayed, was the course identified, and was the file saved in the correct format?  References : were the appropriate number of references used, were all references listed in APA or MLA style, and were the references embedded into the document?  Assignment content : was there evidence of effective research, did the student achieve the purpose of the assignment, and was the correct audience targeted?  Communication : was the assignment successfully transmitted, and does the student provide evidence supporting the retrievability of the file  Layout : did headings/subheadings reflect a logical structure, was there a consistent visual theme, were appropriate font color and size used, and were animations appropriately used?  Embedded objects : did the deliverable contain the following types of objects: pictures, video, audio files, animation, hyperlinks, and other (per instructor)?  Conventions : phrases are of appropriate length for the medium, punctuation is appropriate, grammar usage is appropriate, and spelling is accurate?

6. 2012-2015 Participant Information Information20122015 Prefixes87 Sections1514 Instructors116 Students237234 Class Level0 Dev Ed, 22% 100 level, 78% 200 level 1 Dev Ed, 28% 100 level, 68% 200 level Sophomores19% Sophomores47% sophomores Freshmen42% New Freshmen20% New Freshmen Lowest AreasEmbedded Objects, References

7. Overall Proficiency (scored at 3 or 4)

8. Overall Differences in the Means Aggregate score of the Technology Literacy assessment was high in both years  2012: 3.80 (on a 4-point scale)*  2015: 3.60 (on a 4-point scale)* * statistically significant level

9. COMPARING NEW FRESHMEN to SOPHOMORES 2015 RESULTS Mean scores for Sophomores were only statistically significantly different than Freshmen in Data Management.  Freshmen: 3.83 (on a 4-point scale)*  Sophomores: 3.98 (on a 4-point scale)* * statistically significant level

10. Analysis - Limitations  Cannot extrapolate the results to represent the college population due to small sample size  Differences in scoring methodologies between 2012 and 2015.  Faculty did not identify specific deliverable components missing in 2012 (just if the total number was sufficient).  This was corrected in 2015 but may compromise the ability to accurately compare results between the two assessments  Proportional changes between new freshmen and sophomores further compromises the ability to accurately compare results between the two years  With only one instructor responsible for almost half of all 2015 assessments, the final results cannot be extrapolated to be representative of the college at large  Anecdotally three instructors (representing more than half the students sampled) indicated their grading rigor increased between 2012 and 2015

11. Improvements For Next Cycle  Feedback from Leadership Council (11/25)  Feedback from Classroom Conversations (12/2)  Need to get more faculty involved  Review the rigor of the assessment (is it too easy?)  Inform the faculty that references, embedded objects were lowest areas of proficiency for two cycles in a row.  Work with Instructional Computing faculty members for creating a tutorial video for how to embed objects into student work.  Work with Writing Center to integrate appropriate format for references across the curriculum

12. Questions and Suggestions  Thanks for your input!