1. Multilevel analysis of collaborative activities based on 1
Multilevel analysis of collaborative activities
based on
a Mobile Learning Scenario
for real classrooms
Irene-Angelica Chounta, Adam Giemza, H. Ulrich Hoppe
Collide, University of Duisburg-Essen
{chounta, giemza,

2. Mobile learning: a special case of learning....
Learning on the move, within context
Activities supported by:
„location-based“ scenarios(museum, outdoor trips etc)
smartphones, tablets etc.
Qualitative analysis of the activity

3. Objectives of the study
Introduction of a mobile learning scenario in traditional school classrooms
Use of common learning analytics within a mobile learning context
Study of collaborative learning activities on multiple levels of analysis

4. a mobile learning scenario for the classroom
Computer Kit: „Learn how to assemble a Personal Computer with Mobilogue“

5. Mobilogue Architecture
Authoring environment
Activity logs
Online Management Environment

6. a mobile learning scenario for the classroom
QR codes (learning stations) marking computer parts in a localized setting
Text & Pictures
Video
Quizzes

7. Method of the study Two case studies:
Case A: preliminary study, 17 students
Case B: main study, 24 students
Task: Group of students working together to assemble a pc
Each group was supported by one mobile device
The activities were analyzed with respect to:
students practice (individual and group level)
group characteristics
learning outcome
13 – 15 years old

8. Method of the study For the analysis we used:
Activity metrics from logfiles
Average time gap among consequent actions
Average response time to quizzes
Quiz score
Experts observations
Activity transcripts (recorded by students)
Users questionnaires (collaborative experience)
Pre and Post Knowledge Tests (ten questions each)

9. Case study A - Results 17 students randomly grouped in 5 groups
Activity duration ≈ 90 minutes
Group profiles based on pre-tests:
One „strong“ team (team A)
One „weak“ team (team E)
Three average teams (teams B, C, D)
Pretest Scores
avg_group
max_idv
teamA
6.25 8
teamB
3.75 5
teamC
4.33
teamD
2.67 4
teamE 1
heterogeneous
homogeneous

10. Case study A - Results
Greatest improvement  teams of average performance (on group and individual level)
The “strong” team maintained the same score on average but scored less on the individual level - homogeneity increased
The “weak” team slightly improved – heterogeneity increased
Heterogeneous teams increase homogeneity and vice versa
Pre-test Scores
avg_group
max_idv
teamA
6.25 8
teamB
3.75 5
teamC
4.33
teamD
2.67 4
teamE 1
Post-test Scores
avg_group
max_idv
6.25 7 8
5.67
6.00
2.00 3

11. Case study A - Results
Quiz score reflects the knowledge test performance
The average time gap correlates negatively with quiz score
Teams with good knowledge background and high heterogeneity move faster
Activity metrics
avg_timegap
avg response time
Quiz score
27.63
32.02
120
36.56
56.96
37.14
23.70
100
37.91
31.55 90
38.43
57.43 80
Pre-test Scores
avg_group
max_idv
teamA
6.25 8
teamB
3.75 5
teamC
4.33
teamD
2.67 4
teamE 1

12. Case study B - Results 24 students – 6 groups of four
Groups were created by the teacher
Activity duration ≈ 90 minutes
Additions / Modifications:
Experts monitored the activity
Students kept transcripts of the group activity
Students filled in questionnaires regarding their collaborative experience

13. Case study B - Results Group profiles based on pre-tests:
One strong team (team 03)
Two weak teams (team 00 and team 05)
Three average teams (teams 01, 02, 04)
The Homogeneity of teams according to pre-test confirmed teacher‘s perception
Pre-test Scores
avg_group
max_idv
team 00 2 4
team 01
5.75 9
team 02
4.25 5
team 03
6.5 8
team 04
5.5 7
team 05 1
heterogeneous
homogeneous

14. Case study B - Results
Group performance improved on average (47%) and individual (31%)
Max knowledge gain  weak team (team 05) Min knowledge gain  strong team (team 03)
Knowledge „loss“ for the „strong“ students
Homogenity increases for heterogenous groups and vice versa (confirming case study A)
Pre-test Scores
avg_group
max_idv
team 00 2 4
team 01
5.75 9
team 02
4.25 5
team 03
6.5 8
team 04
5.5 7
team 05 1
Post-test Scores
avg_group
max_idv
3.5 7
7.5 8 6
6.5
7.25 9

15. Case study B - Results Heterogeneous teams achieved higher Quiz scores
The average time gap correlates negatively with quiz score (also observed case study A)
Heterogeneous teams move faster, i.e. Individuals take the lead
Activity metrics
avg_timegap
avg response time
Quiz score
20.89
51.88
120
23.42
49.02
24.00
45.73
110
25.84
61.99 90
40.44
72.51 70
27.75
82.56
Pre-test Scores
avg_group
max_idv
team 00 2 4
team 01
5.75 9
team 02
4.25 5
team 03
6.5 8
team 04
5.5 7
team 05 1

16. Case study B - Results
The students undertook three roles throughout the scenario:
the operator of the mobile device
the scriber and
the assembler
4 out of 6 teams followed no strict role distribution
Teams following strict role distribution appeared to have lowest knowledge gain & dissatisfaction towards collaboration

17. Discussion Learning Outcome
A mobile scenario to support collaborative activities for traditional classrooms
Multilevel analysis of the activity (various methods and multiple units of analysis) to enhance the overall outcome
Learning Outcome
The setting was more effective for weak and average students
Strong students/ teams had not significant knowledge gain
The students performance increased

18. Group Characteristics & Learning Analytics
Discussion
Group Characteristics & Learning Analytics
Heterogeneous teams achieved higher quiz scores
Quiz score did not correlate to knowledge gain or knowledge background
Heterogeneous teams moved faster through the stages of the scenario
Homogeneous teams tend to increase their heterogeneity and vice versa

19. Discussion Collaboration
Team members undertook roles with no particular plan
In the case a strict plan was followed, it led to frustration and low knowledge gain
Strong students stated they would have done better on their own
Weak/average students considered the activity as helpful

20. Conclusion and Future Work
Multilevel analysis is needed to fully understand group tactics and enhance mobile learning
Extend the field of study to further investigate:
the use of various scenarios
varying degrees of freedom to classroom activities
concurrent use of multiple devices within a team

21. Questions.
Thank you 
Thank you 