1. Meeting Technology Literacy Expectations: Assessing and Assisting Incoming Students Cathy Manly Director of Distance Learning Manchester Community College Connecticut Distance Learning Consortium NERCOMP Conference, March 2005 Copyright Cathy Manly 2005. This work is the intellectual property of the author. Permission is granted for this material to be shared for non-commercial, educational purposes, provided that this copyright statement appears on the reproduced materials and notice is given that the copying is by permission of the author. To disseminate otherwise or to republish requires written permission from the author.

2. Why Technology Literacy? Students without basic skills struggle with technology instead of academic content Some students drop courses because of technology requirements Online students need technology skills, but increasingly classroom students need them also

3. Addressing Tech Lit Survey Identify what constitutes basic tech lit Assessment Identify how to assess these skills Remediation Identify possible remediation alternatives

4. Formation of CTDLC--1998 WWW.CTDLC.ORG Members: 37 Higher Ed; 10 Affiliates

5. CTDLC-FIPSE Collaborators Private 4-year institutions: Quinnipiac UniversitySacred Heart University Post University Public 4-year institutions: Charter Oak St. College Univ. of Connecticut Public 2-year institutions: Capital CCManchester CC Naugatuck Valley CCNorthwestern CT CC Three Rivers CCTunxis CC

6. CTDLC’s eSuite Funded by Grants: Fund for the Improvement of Post Secondary Education and Davis Education Foundation

7. Building eSuite Components

8. Tech Lit Assessment Directions Cover faculty-identified items Easy to administer and get results Institutional identification Accessible online Offer remediation possibilities Basic remediation (through the grant) Indicate institutional assistance

9. Tech Skill Survey Results Fall 2002 survey 541 faculty and staff at 11 institutions What tech skills should students have on the first day of class? Choices: Required, Helpful, Not Required Targeted “required” skills Identified by > 50% of faculty teaching introductory courses

10. Faculty Expectations of Student Technology Literacy Basic computer use Basic word processing Basic email Basic web use

11. Basic Computer Use Use a mouse (80, 94%) Type/keyboard (79, 96%) Use copy and paste (54, 85%) Access a CD-ROM (52, 76%) Open and exit a software application In parentheses: percent of faculty teaching introductory courses indicating the skill is: (required, required+helpful.)

12. Basic Word Processing Use a word processing program (76, 91%) to create text reports or papers (72, 87%) Create a blank line Tab text to indent a line Apply basic formatting (bold, italics, center) Save a file (71, 88%) Locate a saved file (70, 88%) Print a file (69, 88%)

13. Basic Email Send and receive email (73, 93%) Attach a file to an email (60, 88%)

14. Basic Web Use Go to a specified website (79, 93%) Use an internet browser (78, 95%) Click on a link Scroll through a webpage Use the Back button Use a search engine (64, 92%) Perform a basic search and identify search results Download and view a file from a webpage (57, 83%) Print a webpage (58, 85%)

15. Highest Expectations Use a mouse (80, 94%) Type/keyboard (79, 96%) Use a word processing program (76, 91%) Send and receive email (73, 93%) Go to a specified website (79, 93%) Use an internet browser (78, 95%)

16. 3 Assessment Methods Compared Method 1: Self-assessment CTDLC created Students answer 1-4 or n/a Method 2: Tasks CTDLC created Web form-based assignments Method 3: Simulation SAM by Course Technology Test environment captures mouse and typing

17. Piloting the Assessments First Pilot – January 2004 403 students, 8 schools - self-test and web assignment 360 students, 2 schools - SAM simulation Second Pilot – April/May 2004 643 students from 9 schools 169 students - self-test 474 students - web assignment 215 students, 1 school - SAM simulation

18. Method 1: Self-Assessment Insignificant correlation between self- assessment and simulation Most students indicated they could “do easily” things they were unable to do Our conclusions: Useful for student self-knowledge Useful to direct students to assistance Not an accurate indicator of student skill

19. Method 2: Tasks Learned from both pilots Revised web piece for 2nd pilot Now revising word processing part 20 min timeframe provides good results Average <10 min, range 3-47 min Placement testing situations benefit from enforced time limits Observation showed some students clearly struggled with skills

20. Method 3: Simulation (SAM) SAM simulation worked well 20 min timed test; students averaged 9 min Observation showed it accurately measures student ability Over 40% of students need assistance Student scores from the first pilot: 11% need significant help (F) 31% need review (C-D) 58% are okay (A-B)

21. Demonstrated Student Tech Skills and Deficiencies An unexpectedly high number of students need significant help or review Students were best at: Basic computer operation, mouse Using links on a webpage, Back button Students had the most difficulty with: Email (send/receive/attachments) Going to a specific web address Copy and paste

22. Correlation Investigation No significant detectable correlation found between simulation scores and: Age Gender Race Full-time versus part-time Length of time since high school Analysis done with Manchester CC students

23. Relationship Between Tech Skills and Placement Scores Significant relationship found between simulation scores and both: English placement test scores Math placement test scores Students with better English or math placement test scores were more likely to do well on the tech lit assessment Analysis done with Manchester CC students

24. Possible Remediation Alternatives Investigated Incorporation into regular classes Credit or non-credit tech courses Workshops specifically covering basic tech skills Online remediation modules currently under development

25. Online Remediation Modules Coming for our Spring 2005 pilot Basic Computer Skills Remediation Self-Paced Available in a variety of LMS Designed for these very basic skills

26. Future: Spring 2005 Pilot Third Pilot – April/May 2005 Focus on all incoming students taking placement testing Only using SAM Challenge simulation Goals Are our results repeatable? Expand our correlation investigation

27. To View a Demo Interested in viewing a demo of the technology literacy assessment? Go to http://www.ctdlc.org/techlittest/http://www.ctdlc.org/techlittest/ (This link is on the handout.)

28. Q&A Cathy Manly, Manchester CC cmanly@mcc.commnet.edu cmanly@mcc.commnet.edu More information about the project: http://www.ctdlc.org/Evaluation/grants- FIPSE.html http://www.ctdlc.org/Evaluation/grants- FIPSE.html Contact Diane Goldsmith, CTDLC dgoldsmith@ctdlc.org