Basic eSkills Foundation or Frustration Diane J. Goldsmith Dean of Planning, Research and Assessment Connecticut Distance Learning Consortium dgoldsmith@ctdlc.org

CTDLC MISSION: Promote Quality Distance Learning 9 years old 49 member institutions Higher Education K-12 State Agencies

Research Questions What Computer skills do faculty believe students need on their first day of class? Do students enter higher education with those skills? Research funded by: Fund for the Improvement of Post Secondary Education. Sloan Foundation

Authors Diane J. Goldsmith, PhD. Cathy Manly, M.B.A Project Director Cathy Manly, M.B.A Director of Distance Learning, Manchester Community College George Rezendes, PhD. Director of Institutional Research, Three Rivers Community College David Nielsen, M.A. Director of Institutional Research, Manchester Community College

Basic Skills Survey of faculty in 2002 11 institutions Associate and Bachelor degree granting Public and private Teach online and face-to-face

Statement of Minimum Skills Basic Computer/Windows Skills Type Open a software application Exit a software application Access a CD-ROM Locate a saved file Use copy and paste

Statement of Minimum Skills Basic Word Processing Skills Use the enter/return key to create a blank line Tab text to indent a line Apply basic formatting: bold, italics, and centering Save a file (name the file and locate where to save the file) Print a file

Statement of Minimum Skills BASIC WEB SKILLS Click on a link Use the back button Perform a basic search in a search engine Print a web page Go to a specified URL Scroll through a webpage Identify search results Download and view a file from a webpage

Statement of Minimum Skills Basic Email Skills Send an email message Attach a file to an email message Receive an email message

Initial Pilots Design a 2 part assessment – Self Assessment – how well can you do this? Simulation (CTDLC design, Course Technologies Different combinations of these given in 9 institutions in the winter of 2004.

Initial Pilot Results Students were not able to self-assess accurately. 10% had few of the basic skills 31% had significant weaknesses

Second Pilot Use Course Technologies simulations – 17 item Basic Skills Assessment Windows – (3)open window, open file, close file Word – (6) insert, copy, paste, save, print, format Internet- (5) enter URL, use link, scroll, back button, search Email – (3) open message, reply, send with attachment

Second Pilot Administered to 2090 students in 5 community colleges. Assessment given after their placement tests (Accuplacer) 1459 registered for classes – have demographic information for them.

Not meeting basic skills Results Only 18% got 100% Pass 72% 300 Need Review 22% 250 200 Number of Students 150 Not meeting basic skills 6% 100 50 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17

Time to Complete Those who had the skills took an average of 7 minutes Those in the “need review” or “not meeting basic skills” took an average of 10 minutes.

Item Response Theory Used IRT for Validity and Reliability Most difficult items: Send email with attachment Reply to email Enter URL in address bar Copy and Paste Use Search Engine

Demographic Analysis--Gender 75% 70% 80% 60% Share of Students 40% 20% 0% Women Avg=82.5 Men Avg=84.6

Average Family Income by Test Score Group $60,000 $51,520 $42,178 $50,000 $40,000 $27,322 Average Family Income $30,000 $20,000 $10,000 $- Not Meeting Need Review Pass

Demographic Analysis -- Age 85.1 86 83.2 84 82 80 78.0 Average Test Score 78 76.5 76 74 72 19- 20-21 22-24 25+

Demographic Analysis No significant correlation with prior college experience. Students who scored higher on writing scored higher on Basic Computer Skills (.42 correlation). Small (.25) correlation with Math skills.

Caveats Time: Students were given the assessment after their reading, writing, and math tests. Language: both in terms of ESL and the language we use for skills Demographics: It appears that percentage of those who lack skills is decreasing, but we don’t know if the two groups are comparable.

Conclusions Recommendations May be important to survey faculty again as the technological needs may have changed. Some students are entering Ct Community Colleges without important skills. Certain skills seem difficult for many students – especially email. Testing can be quick

Reports Available on Line Click on Research at www.ctdlc.org: Basic eSkills Item Response Theory Analysis Statement of Minimum Technical Literacy. Remediation: http://www.ctdlc.org/remediation/