1. HEURISTIC EVALUATION OF WEB-BASED INTELLIGENT TUTORING SYSTEMS
Ioan I. ANDONE, Ph.D Professor,
Napoleon-Alexandru Al. SIRETEANU, Ph.D Lecturer
“Alexandru Ioan Cuza” University of Iaşi, Romania,

2. Objectives/Agenda Objectives: Agenda:
understand how these sytems improve learning and teaching
see the characteristics, benefits and usability metrics
Agenda:
1.Introduction
2. Fundamentals of ITSs and Web-based ITSs
3. Characteristics of heuristic evaluation
4. The benefits of the heuristics evaluation
5. Usability metrics of heuristic evaluation
6. Conclusions

3. Introduction Two distinct ways for improve learning and teaching:
1) ITSs have made teachers and students more productive;
could reduce teaching costs or increase the speed of knowledge aquirement
2) ITSs help improve the quality of learning.
ITSs are, in many respects, very similar to human tutors
They are based on Cognitive Science and Artificial Intelligence
They enable students to use tutoring resources anytime and anywere
ITSs use sumulations and other higly interactive lerning environment
 They help the retain and apply knowledge and skills more effectively in operations settings

4. Fundamentals of ITSs and Web-based ITSs
ITSs fundamentals:
They are organized into three separate software modules: expert, tutor and interface (figure 1-Components)

5. Fundamentals of ITSs and Web-based ITSs
Web-Based ITSs fundamentals:
Are becoming mainstream area of research and development
Facilitates co-operative teaching and learning, and an potential for sharing the learning resources and the tasks of creating new learning resources through collaboration (WWW)
A modular architecture supporting by authoring tools and an indexing mechanism for the repository of tutoring components
Are usually dedicated to a restricted expert system’s knowledge base
Have a closed architecture with little possibility of modifications
Recent examples: CALAT [Nakabayashi, 1997]; InterBook [Brusilovshy, 1997], VC PROLOG [Peylo, 2000], Desire2Learn [Koedinger, 2006], etc.

6. Fundamentals of ITSs and Web-based ITSs
Most of Web-based ITSs use a client/server architecture that places shared resources and functionality on servers, and use Internet to deliver student interfaces on a variety of client platforma at any location (figure 2-Architecture)

7. Characteristics of heuristic evaluation
Objectives:
definitions of the term of heuristics evaluation
present of alternative usability heuristics
seeking a new international standard for improving the efficiency of Web-based ITSs usability
the heuristic evaluation with Techsmith’s UserVue software

8. Definitions of heuristics evaluation:
R. Molich & J Nielsen [1994]:
“a form of usability inspection where usability specialists judge whether each element of a user interface follows a list of established usability heuristics”
heuristic evaluation involves a small set of evaluators which examine the interface and judge its compliance with recognized usability principles (the “heuristics”). The evaluators are usually experts in HCI or user interfaces.
This opinion is valuable also for evaluation of Web-base ITSs.
See J.Nielsen’s usability heuristics on figure 3.

9. Figure 3 J.Nielsen’s usability heuristics

10. Definitions of heuristics evaluation:
Our opinion:
We need a set of guidelines that are based on research and have some international consensus
Example: Mossier and Smith’s [1986]- 944 Guidelines for designing User Interface-the largest collection in the world
BS EN-ISO International Usability Standard proposes an alternative of seven guidelines because limitations of Nielsen’s usability heuristics (figure 4- ISO 9241 usability heuristics). These guidelines have an International Consesus and they can be applied to any Online Learning System for improving the efficiency of usability

11. Figure 4 ISO 0241 Usability heuristics

12. Notes:
 ISO 9241 is the new international standard for web usability and for Web-based ITSs usability also!
this standard will be of interest to anyone who designs, evaluates or commisions web sites for Web-based ITSs
This new web usability standard hasn’t been approved yet; it is based arround a reference model which addresses the dewsign domain only (Figure 5)
 Techsmith’s UserVue is an online service for heuristic evaluation of Web-based ITSs

13. Figure 5
Reference Model

14. The benefits of the heuristics evaluation
the design feedback is valid and useful, and it can be obtained early on the design process, whilst checking conformity to established guidelines helps to promote compatibility with similar systems
it is beneficial to carry out a heuristic evaluation on early prototypes of Web-based ITS before actual users are brought in to help with further testing
usability problems found are normally restricted to aspects of the interface module that are reasonably easy to demonstrate: use of colors, layout and information structuring, consistency of the terminology, consistency of the interaction mechanisms
the method can seem very critical as designers may only get feedback on the problematic aspects of the interface module as the method is normally not used for the identification of the ‘good’ aspects.

15. Usability metrics of heuristic evaluation
Concept:precise, quality measures used to evaluate the Web-based ITS
 The purpose:to produce a Web-based ITS that is neither under-nor over-engineered; they help for outrun the competition
Three Es usability metric:
1) Effectiveness measures
2) Efficiency measures
3) Satisfaction measures

16. Conclusions Thank You ! Happy Easter!
 more and more institutions and companies offer Web-based ITSs to students and to personnel worldwide
 we will become more knowleageable about what they works and what does not work
 strongly recommended that the Web-based ITSs to be evaluated with a heuristics method, according to the new International Standard, because the benefits
Thank You !
Happy Easter!