1. 20081 E-learning Lecture-10: EVALUATING THE IMPACTS OF E-LEARNING week 12- Semester-4/ 2009 Dr. Anwar Mousa University of Palestine Faculty of Information Technology

2. 20082 EVALUATING THE IMPACTS OF E-LEARNING Goals Explore issues related to the evaluation of e-learning and teaching activities. Propose a comprehensive plan for the evaluation of e- learning and teaching.

3. 20083 Goals of evaluation A major goal of any evaluation activity is to influence decision making. For any organization to be able to attain its mission, a comprehensive evaluation strategy for ascertaining the impacts of its various teaching, learning and research related activities is crucial. This strategy needs to be systematic in its approach to gathering different types of data and feedback from a range of sources, and with the help of a variety of instruments.

4. 20084 Goals of evaluation The gathering of this kind of data and feedback is also crucial to ensuring a high quality of service, and effective utilization of information and communications technology in teaching and learning. The term “evaluation” is being used here to refer to the systematic acquisition of feedback on the use, worth and impact of some activity, program or process in relation to its intended outcomes (see Naidu, 2005).

5. 20085 Goals of evaluation The most basic distinctions between various types of educational evaluation activities are drawn between formative, summative, and monitoring or integrative evaluation (see also Kirkpatrick, 1994; Naidu, 2002, 2005; Reeves, 1997, 1999).

6. 20086 Evaluation methodology You should aim to gather data from all stakeholders (i.e., students and staff) regularly using a set of evaluation instruments within A consistent evaluation framework which should include: – front-end analysis, formative, summative and integrative evaluation. You should also aim to collect a variety of data using a range of data gathering instruments. However, you would want to keep the data gathering process as simple and as less intrusive (interfering) as possible.

7. 20087 Evaluation methodology Front-end analysis comprises a set of ways by which you would plan to ascertain the readiness of students and staff and their preferences in relation to teaching and learning online. Carrying out such surveys periodically and especially prior to the full roll-out of e-learning will enable your organization to get a better handle on how to align its services to meet the needs of prospective (future) users. The information gathered will help to inform the organization on the nature of its user needs, their perceptions (awareness) and expectations, and any gaps in the provision of existing support.

8. 20088 Evaluation methodology Formative evaluation would involve gathering feedback from users and other relevant groups during the implementation process. Its purpose would be to identify problems so that improvements and adjustments can be made during the implementation stages of elearning in your organization. You may wish to plan to carry out formative evaluations routinely and regularly. It would be best that these evaluations use a consistent set of tools comprising surveys, and focus group interviews with users.

9. 20089 Evaluation methodology Summative evaluation will enable you to ascertain the full impacts and outcomes of e-learning on teaching and learning at your organization. You would usually carry this out upon the completion of an e-learning program, even though there is not likely to be a dividing line between formative and summative evaluation phases. As part of this process, your aim is to periodically assess the sum impacts of e-learning on teaching and learning activities in your organization.

10. 200810 Evaluation methodology Data gathered should reveal how e-learning is responding to challenges facing teaching and learning in your organization, and the extent to which you are achieving benchmarks and milestones which you have set. Monitoring or integrative evaluation will comprise attempts to ascertain the extent to which the use of e- learning or online learning is integrated into regular teaching and learning activities at your organization. Data gathered as part of this process will reveal the extent to which, and how teaching and learning activities in the organization have been impacted with e-learning.

11. 200811 EVALUATING THE IMPACTS OF E-LEARNING For more information on evaluation tools have a look at these sites. l http://ncode.uow.edu.au/evaltool.html l http://www.tltgroup.org l http://www.tltgroup.org/programs/ftools.html l http://www.tltgroup.org/programs/flashlight.html l http://mime1.marc.gatech.edu/MM_Tools/ evaluation.html l http://ncode.uow.edu.au/evaluation.html l http://ncode.uow.edu.au/evalart.html l http://ncode.uow.edu.au/evalcase.html l http://ncode.uow.edu.au/evalstrat.html

12. 200812 EVALUATING THE IMPACTS OF E-LEARNING For more information on evaluation tools have a look at these sites. l http://www.uwex.edu/disted/evaluation.html l http://ncode.uow.edu.au/evalother.html l http://eval.cgu.edu/ l http://ericae.net/nintbod.htm l http://www.unc.edu/cit/guides/irg-49.html l http://www.unc.edu/~elliott/evaluate.html l http://www.icbl.hw.ac.uk/ltdi/cookbook/cookbook.pdf

13. 200813 Points to Remember Evaluation refers to the systematic acquisition of feedback on the use, worth and impact of some activity, program or process in relation to its intended outcomes. Any evaluation strategy should include front-end analysis, formative, summative and integrative evaluation activities. Evaluation of e-learning is no different and should aim at gathering data from all stakeholders.