User Evaluation of OUM’s HTML Modules towards proposing an Enhanced Model Lim Tick Meng Widad Binti Othman Safiah Bt Md Yusof Woo Tai Kwan Datin Halimatolhanin binti Mohd Khalid

2 OUM’s Blended Mode of Learning The quality of print modules determines OUM’s success in its implementation of ODL

Our Research is guided by these two principles Open University Malaysia has been exploring the best possible means of instructional delivery that make ODL a preferred choice of learning It is necessary to leverage technology in the best possible way to enhance students’ learning 3

Background 1.OUM is in the processing of converting the print modules (more than 1500) to HTML modules 2.Since learning using modules constitutes 80% of learning time, it is vital to ensure that in any innovational change related to the use of modules, quality of the change in relation to effective learning should always be ascertained. 3.In evaluating the effectiveness of a learning system, learners should be involved as one of the key group of evaluators 4

1. What are OUM learners’ perceived quality of the web- based HTML modules from the dimensions of technical, interface-design, learner-support and instructional adequacy respectively? 2. Is there any difference between male learners and female learners in their overall perceived quality of the web-based HTML modules? 3. How should the framework for the HTML modules be designed to enhance its efficacy as a self-managed learning tool and resource?

Moving from behaviorist to constructivist learning Chickering and Gamson (1987) – The Seven Principles of Good Practice Mccombs and Vakili (2005)’s learner-centered framework Various guidelines and standards for evaluating online learning and online learning objects (example: Elissavet & Economides, 2003; Quiter & Weber, 2004; Nicole A. Buzzetto-More & Kaye Pinhey, 2006; Sternberger, C. (2006); Thair M. Hamtini, 2008) 6

The Seven Principles of Good Practice – Adoption till Today Encouraging contact Developing reciprocity and cooperation Encouraging active learning Providing prompt feedback Emphasizing time on task Communicating high expectations Respecting diverse talents and ways of learning 7

Identify Scope of Evaluation Identify Scope of Evaluation Prepare instrument for Evaluation Prepare instrument for Evaluation Administer user evaluation (survey) at various centres Administer user evaluation (survey) at various centres

Population of the study 1.OUM learners who uses the HTML modules for learning the compulsory subjects –OUMH1103 – Learning Skills for Open and Distance Learners –OUMH1203 – English for Written Communication –OUMH1303 – English for Oral Communication 9

Sampling Method Purposive and Convenient sampling –Survey forms distributed to all learners of the 3 modules who were present at T3 (all learning centres) –Analyses were carried out on the returned forms which were completed by the fifth tutorial 10

Aspects of Evaluation Closed-Ended Items (40 items) –Technical Adequacy(7 items) –Interface-Design Adequacy(4 items) –Learner-Support Adequacy(7 items) –Instructional Adequacy(22 items) Open-Ended Items(2 items) 11

The closed ended items Adopted a four-point Likert Scale –1-Strongly Disagree –2-Disagree –3-Agree –4-Strongly Agree 12 This is to avoid users to select the NOT SURE option

Cronbach’s Alpha For Items Measuring Each Aspect of Adequacy AdequacyNo of ItemsCronbach’s Alpha Technical Interface- Design Learner- Support Instructional

Profiles of the Evaluators (Learners) 14

Learner Evaluation of Adequacies

Learner Evaluation of HTML: Module Adequacies 16

t- test to compare the rating of Male and Female Learner Evaluators The analysis shows that there is significant difference between ratings of the male Learners (M=3.12, SD=.49) and the female Learners (M=3.01, SD=.57); t(815)= 2.689, p=0.007, d=0.20) The results suggest that the gender factor does have an effect on the perceived adequacy of the HTML modules. The Cohen’s d value of 0.02 however is indicative of a small effect size. 17

Problems Raised by Respondents NOProblems using the HTML Modules Number of Respondents 1There is problem logging-in/launching the module 34 2It is tiring watching the computer for too long 23 3The system appears to be slow 21 4There is no face-to-face discussion 8 5We do not know when to use the learning objects 8 6There are not enough self-assessment activities 3 18

Suggestions by Respondents NOProblems using the HTML Modules Number of Respondents 1Should have more interactive activities and exercises 11 2Face-to-face instruction is still important 10 3The print modules should also be provided 10 19

Implications and Suggestions (1) 1.Learning objects should be linked to the module at appropriate nodes and links, not just placed at repository and dissociated from the module content. 2.It needs the subject matter expert to work hand-in-hand with the instructional designers to accomplish the above task 20

Implications and Suggestions (2) 3.To cater for learners of varying learning styles, the level of interactivity of the learning objects should be enhanced in the following ways: –Contains microworld that allows exploratory learning –Provides different responses for different inputs –Provides varying levels of entry for different ability groups 21

Implications and Suggestions (3) 4.Assessment activities should not be just providing feedback in the form of scores. It should be “intelligent” enough to provide different responses for different inputs. The responses should be meaningful in that it guides learning 5.The HTML modules should have the functionality to “inform” learners of their own performance as well as their performance as compared to the peers 22

Implications and Suggestions (4) 5.To provide an alternative to face-to-face instruction, mini-lectures in the form of Video (FLV) should be embedded 6.The source of mini lectures could be –Effort from the academics (i-lecture) –Identify quality YouTube lectures –Other Open Sources 23

24 Web-Video Lecture (YouTube Resource etc) Hyperlinked Lecture Embedded Lecture Web Resource Hyperlink Embedded LO0 Hyperlink Score Database Module Page (1) Module Page (2) Module Page (3) Online Forum and Synchronous Communication Tool Web Resource MyVLE Environment Web Environment Online Assessment

That ‘s all, Thank You