Meeting Technology Literacy Expectations: Assessing and Assisting Incoming Students Cathy Manly Director of Distance Learning Manchester Community College Connecticut Distance Learning Consortium NERCOMP Conference, March 2005 Copyright Cathy Manly This work is the intellectual property of the author. Permission is granted for this material to be shared for non-commercial, educational purposes, provided that this copyright statement appears on the reproduced materials and notice is given that the copying is by permission of the author. To disseminate otherwise or to republish requires written permission from the author.

Why Technology Literacy? Students without basic skills struggle with technology instead of academic content Some students drop courses because of technology requirements Online students need technology skills, but increasingly classroom students need them also

Addressing Tech Lit Survey Identify what constitutes basic tech lit Assessment Identify how to assess these skills Remediation Identify possible remediation alternatives

Formation of CTDLC Members: 37 Higher Ed; 10 Affiliates

CTDLC-FIPSE Collaborators Private 4-year institutions: Quinnipiac UniversitySacred Heart University Post University Public 4-year institutions: Charter Oak St. College Univ. of Connecticut Public 2-year institutions: Capital CCManchester CC Naugatuck Valley CCNorthwestern CT CC Three Rivers CCTunxis CC

CTDLC’s eSuite Funded by Grants: Fund for the Improvement of Post Secondary Education and Davis Education Foundation

Building eSuite Components

Tech Lit Assessment Directions Cover faculty-identified items Easy to administer and get results Institutional identification Accessible online Offer remediation possibilities Basic remediation (through the grant) Indicate institutional assistance

Tech Skill Survey Results Fall 2002 survey 541 faculty and staff at 11 institutions What tech skills should students have on the first day of class? Choices: Required, Helpful, Not Required Targeted “required” skills Identified by > 50% of faculty teaching introductory courses

Faculty Expectations of Student Technology Literacy Basic computer use Basic word processing Basic Basic web use

Basic Computer Use Use a mouse (80, 94%) Type/keyboard (79, 96%) Use copy and paste (54, 85%) Access a CD-ROM (52, 76%) Open and exit a software application In parentheses: percent of faculty teaching introductory courses indicating the skill is: (required, required+helpful.)

Basic Word Processing Use a word processing program (76, 91%) to create text reports or papers (72, 87%) Create a blank line Tab text to indent a line Apply basic formatting (bold, italics, center) Save a file (71, 88%) Locate a saved file (70, 88%) Print a file (69, 88%)

Basic Send and receive (73, 93%) Attach a file to an (60, 88%)

Basic Web Use Go to a specified website (79, 93%) Use an internet browser (78, 95%) Click on a link Scroll through a webpage Use the Back button Use a search engine (64, 92%) Perform a basic search and identify search results Download and view a file from a webpage (57, 83%) Print a webpage (58, 85%)

Highest Expectations Use a mouse (80, 94%) Type/keyboard (79, 96%) Use a word processing program (76, 91%) Send and receive (73, 93%) Go to a specified website (79, 93%) Use an internet browser (78, 95%)

3 Assessment Methods Compared Method 1: Self-assessment CTDLC created Students answer 1-4 or n/a Method 2: Tasks CTDLC created Web form-based assignments Method 3: Simulation SAM by Course Technology Test environment captures mouse and typing

Piloting the Assessments First Pilot – January students, 8 schools - self-test and web assignment 360 students, 2 schools - SAM simulation Second Pilot – April/May students from 9 schools 169 students - self-test 474 students - web assignment 215 students, 1 school - SAM simulation

Method 1: Self-Assessment Insignificant correlation between self- assessment and simulation Most students indicated they could “do easily” things they were unable to do Our conclusions: Useful for student self-knowledge Useful to direct students to assistance Not an accurate indicator of student skill

Method 2: Tasks Learned from both pilots Revised web piece for 2nd pilot Now revising word processing part 20 min timeframe provides good results Average <10 min, range 3-47 min Placement testing situations benefit from enforced time limits Observation showed some students clearly struggled with skills

Method 3: Simulation (SAM) SAM simulation worked well 20 min timed test; students averaged 9 min Observation showed it accurately measures student ability Over 40% of students need assistance Student scores from the first pilot: 11% need significant help (F) 31% need review (C-D) 58% are okay (A-B)

Demonstrated Student Tech Skills and Deficiencies An unexpectedly high number of students need significant help or review Students were best at: Basic computer operation, mouse Using links on a webpage, Back button Students had the most difficulty with: (send/receive/attachments) Going to a specific web address Copy and paste

Correlation Investigation No significant detectable correlation found between simulation scores and: Age Gender Race Full-time versus part-time Length of time since high school Analysis done with Manchester CC students

Relationship Between Tech Skills and Placement Scores Significant relationship found between simulation scores and both: English placement test scores Math placement test scores Students with better English or math placement test scores were more likely to do well on the tech lit assessment Analysis done with Manchester CC students

Possible Remediation Alternatives Investigated Incorporation into regular classes Credit or non-credit tech courses Workshops specifically covering basic tech skills Online remediation modules currently under development

Online Remediation Modules Coming for our Spring 2005 pilot Basic Computer Skills Remediation Self-Paced Available in a variety of LMS Designed for these very basic skills

Future: Spring 2005 Pilot Third Pilot – April/May 2005 Focus on all incoming students taking placement testing Only using SAM Challenge simulation Goals Are our results repeatable? Expand our correlation investigation

To View a Demo Interested in viewing a demo of the technology literacy assessment? Go to (This link is on the handout.)

Q&A Cathy Manly, Manchester CC More information about the project: FIPSE.html FIPSE.html Contact Diane Goldsmith, CTDLC