Assessing Students Computer Skills Lessons Learned From a Survey Used in Science 100 © 2002 Karin E. Readel

Initial Purpose of Survey SCI 100 students spend about 50% of course working in same group (4-5 students) I needed some basic information from students in order to form “balanced” groups early in semester Who were my students?

Evolution of Survey Some questions unclear or redundant Early versions of survey made me realize I wanted more information Purpose of survey changed –I became interested in students’ “perception vs. reality” –Survey also provided me with information about topics I need to cover in further detail

Survey Administration Students asked to fill out survey during first lab Students are told information will be used in group formation I use hardcopies of survey to assign groups based on series of criteria Data entered (eventually) into an Access database for analysis –Overall Trends –By Computer Ability –By Class

Survey 2 Pages Long; Fill in the Blank and Rankings

The Students Total Sample Size of 638 from Fall 2001 through Fall (44%) are commuters 358 (56%) live on campus

Number of Students in Each Class

Number of Students in Each Major

Computer Access Have Internet Access at Home vs. Campus Have Printer Access at Home vs. Campus (only Fall 2002 data)

Self Ranking of Overall Computer Ability Students asked to rank themselves as “beginner”, “intermediate” or “advanced” computer users. A numerical scale (1-10) was added to the survey in Fall 2002.

Self-Ranking of Computer Ability

Self-Ranking of Computer Ability by Class

Operating System? “What operating system do you use most often?” My interpretation of whether or not it is plausible –473 Correct –165 Incorrect Out of the “correct” responses –454 MS Windows –17 Mac OS –2 Unix –2 Linux

Correct OS by Computer Ability

Research Experience on the Internet

Average Internet Research Ability Fall 2002 Data

Word Processing Software Students were asked the name and version of the software they use

Converting Files to Rich Text Format Students were asked if they knew how to convert a file to rich text format –239/638 students answered yes

RTF Conversion By Class

Virus Checking Students were asked if they knew how to scan a file or disk for viruses –373/638 answered yes

Virus Checking By Class

Web Page Experience Number of students with prior web page experience Students’ rankings of their web page experience

Most Commonly Used Software

Do Students REALLY Have Web Page Experience?

Web Page Ability by Class

Do Students Know How to FTP? Students were asked if they knew how to FTP a file 185/638 answered yes

Do Students Know How to Use Excel? 405 of 638 students have experience using Excel

Excel Ability By Class

Average Excel Ability Fall 2002 Data

Prior Blackboard Use? Students were asked if they had used Blackboard in another course

Blackboard Use by Class

Students’ Ranking of Blackboard In Fall 2002 students were asked to rank experience on scale of –Average Ranking = 6.6

Ranking of Blackboard by Computer Ability

Overall Trends Students think of themselves as Intermediate Computer Users Probably overestimating their abilities –25% don’t know what OS they use –32% don’t know how to convert a file to RTF – 45% don’t know how to scan for viruses –6% don’t know what Word Processing program they use

Overall Trends cont’d. Less than half of students have experience creating web pages Further, many students who say they have experience, are very limited –Mostly beginners and intermediates Most students don’t know how to FTP files

Overall Trends cont’d. Approximately 60% of students had used Blackboard before –More upperclassmen vs. freshmen Satisfaction level about the same between upper and lower classmen Correlation between computer ability and satisfaction with Blackboard?

Group Roles Students were asked to rank their preference in potential group roles –Leader –Researcher –Computer Guru –Facilitator –Artist –Field Person –Writer/Editor (added in Fall 2002)

Group Role Preferences Leader Researcher

Group Role Preferences Computer Guru Facilitator

Group Role Preferences Artist Field Person

Group Role Preferences Writer category was added in Fall 2002

Group Roles by Computer Ability

Group Roles by Major Leader Researcher

Group Roles by Major Computer Guru Facilitator

Group Roles by Major Artist Field Person

Group Roles by Major Writer

Overall Trends in Group Roles Researcher and Facilitator are most popular first choices Leader and Field Person fall in the middle Computer Guru and Writer are least popular overall Artist at either top or bottom

Overall Trends cont’d. Advanced users/ POLI majors more likely to choose Leader role Advanced users no more likely to choose Computer Guru role –(desire does not mirror ability?)

Future Changes Incorporate early assignment to asses Excel abilities (particularly graphing skills) Add post assessment at end of semester –Have computer skills improved? –Which ones? To what extent? Expand database to incorporate grades on different assignments –Do students with certain skills do better or worse in SCI 100?