Supporting group learning and assessment through Internet Liane Tarouco Marlise Geller Antonio R. de Vit Luciano Hack Graduate Program Informatic on Education Universidade Federal do RS - Brazil

CMDL Computer Mediated Distance Learning (classroom is routinely a far more scrutable educational environment than the physical classroom) Computer Mediated Distance Learning (classroom is routinely a far more scrutable educational environment than the physical classroom) & news groups & news groups chat chat videoconference videoconference

CSCL Computer Supported collaborative Learning Computer Supported collaborative Learning group learning is more effective group learning is more effective Learning through reflexive discussion Learning through reflexive discussion

CSCL based on Internet services Computer Supported collaborative Learning Computer Supported collaborative Learning CSCL based on Internet CSCL based on Internet asynchronous asynchronous , news groups, , news groups, electronic board electronic board synchronous synchronous chat, videoconference chat, videoconference

Collaborative learning Problems Problems heterogeneous contributions heterogeneous contributions lack of capability for monitoring all group work lack of capability for monitoring all group work

u u Learning assessment became a challenge to those who work with distance education Learning assessment

u u Evaluation process should include a great variety of evidences that go beyond the traditional final examination based on paper and pencil.

Face-to-face context Professors use more than just formal mechanisms to evaluate students. body language participation Quality of questions proposed by students are good indicators of learning

Distance Education u u What normally occurs it is the use of formal mechanisms only

Why ? u It occurs not only because of the lack of accessible mechanisms that can assist instructors on this task, but also, because many professors prefer to work with more traditional methods of evaluation. u Authentication u Legal aspects

Internet evolution u Evolution of networks and computers and especially the Internet u New mechanisms and services started to show up

Keeping track u Tools to keep track of students activities and interactions with learning environment with learning environment with the group of students with the group of students with the professor with the professor

Internet for distance education Reduction of the distribution costs; Reduction of the distribution costs; Internet facilitates collaborative writing; Internet facilitates collaborative writing; The corrections and updates are simpler to handle; The corrections and updates are simpler to handle; Several techniques for evaluation are possible using multimedia for the interaction between professor and students and between students. Several techniques for evaluation are possible using multimedia for the interaction between professor and students and between students.

Distance education assessment Assessment based on contributions for group discussions Assessment based on contributions for group discussions Tests (automatically handled by computer program) Tests (automatically handled by computer program) Term papers (analyzed by professor or assistants) Term papers (analyzed by professor or assistants) Oral or written tests conduced in the presence of the instructor (some times through videoconference) or with a remote assistant. Oral or written tests conduced in the presence of the instructor (some times through videoconference) or with a remote assistant.

Time spent with assessment

Types of assessment u case studies u exams u papers u projects All of them requires extensive time to grade and are very difficult to get automated help to do the evaluation.

Distance education technologies Ancient forms Internet based environment environment Printed text books HTML pages Printed text books HTML pages Exercises Forms and CGI Exercises Forms and CGI TV broadcast MBONE TV broadcast MBONE VHS tapes Real Video, MPEG VHS tapes Real Video, MPEG Audio tapes Real audio Audio tapes Real audio Assesment Forms and CGI Assesment Forms and CGI

Topics u Kirkpatrick's model for evaluation - set of tools to provide capability to asses according this model F consensus F tracking F voting F self-evaluation

Kirkpatricks model Reaction Learning Behaviour Results

Formative evaluation u Formative evaluation is important to make easy for the students not only inform their opinions and feelings about how course is going on but see this contributions being handled and considered.

Using quiz system u Teachers may built artifacts and link them u CGI allows dynamically building and editing

Basic functions included u u Feedback for each alternative u u Indication of correct answer u u Nothing more, just counting incidence of selections on each alternative

Complementary tools u Consensus u Tracking u Voting u Self-evaluation

Tool of consensus It allows on time creation of enquete proposing instigating questions, as well as the collection and tabulation of responses; The tool has the capability to keep track of each student's level of contribution to the discussion as well as enquete responses.

Tracking tool u u The register of learning activities (accessed page with dates and time) is important because it allows the professor to know the progress of students trough the course; u u The collected information will show that type of access the student has made (visited units, tools used, etc), as well as the time spent on it.

Voting tool u u Aimed to provide conditions for a fast feedback for the instructor on specific subjects.

Self-evaluation tool u u Allows the dynamic building of enquetes for self-evaluation by professor and by students.

Group activity handling Group activity may give opportunity for student to present contributions that some times are relegated to forgetfulness. Group activity may give opportunity for student to present contributions that some times are relegated to forgetfulness.

Consolidation It is also needed a voting system to support group decision about the consolidated results from discussion It is also needed a voting system to support group decision about the consolidated results from discussion ordering ordering reordering reordering selecting selecting discarding discarding

What was needed Development of applications to handle participants contributions Development of applications to handle participants contributions Provide support for group decision Provide support for group decision Utilization of contributions that came up during group activity Utilization of contributions that came up during group activity

Handling group activity results A set of tools was designed and developed to support distance education collaborative work. A set of tools was designed and developed to support distance education collaborative work.

Group activity results Group activity results on a log file containing interchanged texts (synchronous or asynchronously) during a group debate. Group activity results on a log file containing interchanged texts (synchronous or asynchronously) during a group debate. Example: Using an environment like Palace for chat

Other services used for group discussion WWW based chat WWW based chat Videoconferencing complemented with chat Videoconferencing complemented with chat

Developed solution Application for handle results of group discussion (brainstorming) Application for handle results of group discussion (brainstorming) consolidation of contributions consolidation of contributions summaries with more relevant ideas summaries with more relevant ideas voting process voting process

Issue Analyzer Tool supporting brainstorming Tool supporting brainstorming Used natural language recognition Used natural language recognition Grouping of similar ideas Grouping of similar ideas Organize them according with definitions of the group Organize them according with definitions of the group Client server application Client server application Supported only synchronous communication Supported only synchronous communication

Eurekha Tool initially developed for information retrieval Tool initially developed for information retrieval Uses methods for grouping textual objects Uses methods for grouping textual objects Objects are organized automatically in similar objects groups Objects are organized automatically in similar objects groups Directed to natural language Directed to natural language Designed to work with files (each text object must be presented as a different file) Designed to work with files (each text object must be presented as a different file)

ESCOOP Handle log files derived from group activity Handle log files derived from group activity Clean and cut Clean and cut remove stop words remove stop words spell checking spell checking remove names of senders remove names of senders Uses Eurekha for clustering Uses Eurekha for clustering Handle voting process Handle voting process

Conclusions u u Formative evaluation plays an important role to get distance students motivated because they feel a sense of not being lost in space u u Just the opposite, they feel that being closely monitored and having their opinions being asked often is more like having a personal tutor that pay attention on way they do and what they say.

Conclusions u u This kind of attention provided by the system helps to minimize solitude that sometimes is referred as main desisting factor from students in distance education programs

Conclusions Cognitive development improved Cognitive development improved development of better self- expression development of better self- expression help student to learn about how to work collaboratively help student to learn about how to work collaboratively stimulate problem solving critical thinking and collaborative analysis stimulate problem solving critical thinking and collaborative analysis learning through reflexive discussions learning through reflexive discussions

Results colaboration valued/considered colaboration valued/considered better follow up from teacher better follow up from teacher contributions more homogeneous contributions more homogeneous secundary results secundary results writing skills improved writing skills improved more reflexive thinking about what is being presented more reflexive thinking about what is being presented

More details / Contacts u Marlise Geller u Liane Tarouco