Evaluation Results of OP4L Learning Environment and Services in Slovenia Maribor, May 2012 Bar, Montenegro, 25 th – 26 th June Ines Kožuh, prof. dr. Matjaž Debevc

Research detailes The aim of our study: test OP4L learning environment and its services, assess students‘ relationship with collaborative learning evaluation: before testing OP4L learning environment after testing OP4L learning environment Company LOGO

Participants Computer Science students -1st year (Faculty of Electrical Engineering and Computer Science), volunteers signed consent form, 62 participants all male, age in years (Mdn = 20, Mean = 20.73, Min = 19, Max = 25), experience in programming. Company LOGO

Participants experience in programming (5 levels) Company LOGO

Procedure Three weeks in May 2012 Two classroom meetings: 1st meeting: before testing, instructions were given, questionnaires were filled in, 2nd meeting: after testing, questionnaires were filled in, The task: brainstorming, submission of proposed idea, programming, submission of the program, assessment of the program. Company LOGO

Measures Quantitative research metod (questionnaires), Evaluation before testing OP4L: A questionnaire on the habits of using Facebook, Facebook chat and Web resources, Evaluation after testing OP4L: A questionnaire on the habits of using Facebook, Facebook chat and Web resources (the same as before testing), A questionnaire on the experience in OP4L learning environment, A questionnaire on collaborative learning and using Facebook in favor of collaborative learning, A questionnaire on the usability of OP4L learning environment (SUS), Motivated strategies on learning questionnaire (MSLQ). Company LOGO

Measures Likert-type questions and open-ended questions questionnaires were put online. Company LOGO

Statistical analysis Descriptive statistics, definition of statistically significant relationships (Spearman‘s Rho coefficient), Kruskal-Wallis Test (differences among independent samples), SPSS Company LOGO

Results Habits of using Facebook, chat and Web resources Comparison of the results before and after the study Company LOGO

Results Habits of using Facebook, chat and Web resources Company LOGO

Results Habits of using Facebook, chat and Web resources Company LOGO

Results Experience in OP4L learning environment Assessment of OP4L services: Recommended peers (idea vs. usage) Recommended readings (idea vs. usage) idea – from pedagogical and functional point of view usage – from pedagogical and functional point of view, technical operation IdeaUsage Nr of questions Cronbach‘s Alpha Nr of questions Cronbach‘s Alpha Recommended peers Recommended readings

Results Experience in OP4L learning environment Company LOGO

Results Usability of OP4L learning environment (SUS) SUS score = Company LOGO Source: A. Bangor, P. Kortum, J.A. Miller, The System Usability Scale (SUS): An Empirical Evaluation, International Journal of Human-Computer Interaction, 24, (2008), 6, pp. 574 ‒ 594.

Results Opinions on usability of OP4L learning environment Bad responsiveness of the system. Too many clicks were needed to be able to come to the task. Problems with submitting and assessing the tasks. Communication with others was too complicated. Company LOGO

Results Correlations between usability and OP4L services Company LOGO Correlations (1)(2)(3)(4)(5) (1) System usability Correlation Coefficient1, **.505 **.465 **.581 ** Sig. (2-tailed)..000 (2) Idea Recommended Peers Correlation Coefficient **.449 **.401 ** Sig. (2-tailed) (3) Usage Recommended Peers Correlation Coefficient 1, **.656 ** Sig. (2-tailed) (4) Idea Recommended Readings Correlation Coefficient ** Sig. (2-tailed)..000 (5) Usage Recommended Readings Correlation Coefficient 1,000 Sig. (2-tailed). **. Correlation is significant at the 0.01 level (2-tailed).

Results Collaborative learning and using FB in learning assessment of collaborative learning, assessment of using Facebook in learning, Company LOGO Number of questions Cronbach‘s Alpha Collaborative learning7.869 Facebook in learning4.868

Results Correlations among three variables Company LOGO Correlations (1)(2)(3) (1) Affinity for collaborative learning Correlation Coefficient **.352 ** Sig. (2-tailed) (2) Affinity for using FB for solving complex problems Correlation Coefficient ** Sig. (2-tailed)..001 (3) Frequency of using FB for solving complex problems Correlation Coefficient Sig. (2-tailed). **. Correlation is significant at the 0.01 level (2-tailed).

Results Dispersion of data for two variables Company LOGO

Results Differences among groups in knowledge of programming Company LOGO Ranks Knowledge of programming NMean Rank Affinity for collaborative learning Affinity for using FB for solving complex problems Test Statistics a,b Affinity for collaborative learning Affinity for using FB for solving complex problems Chi-Square df22 Asymp. Sig a. Kruskal Wallis Test b. Grouping Variable: Knowledge of programming

Results MSLQ – Motivational strategies Company LOGO N = 58 students

Results MSLQ – Motivational strategies Company LOGO N = 58 students

Results MSLQ – Learning strategies Company LOGO N = 58 students

Results MSLQ – Learning strategies Company LOGO N = 58 students

Contact University of Maribor, Faculty of Electrical Engineering and Computer Science, Smetanova 17, 2000 Maribor Ines Kožuh prof. dr. Matjaž Debevc Company LOGO