The design and the formative evaluation of an adaptive educational system based on cognitive style Author: Evangelos Triantafillou*, Andreas Pomportsis , Stavros Demetriadis Source: Computer & Education 41(2003) ,pp.87-103 Speaker: Hui - Chun, Chiang Date: 2004/05/15

Outline Introduction Theoretical framework Design of AES-CS Formative evaluation Implement discussion Conclusions Commends Appendix Formative evaluation:形成性評量

Introduction Designing AES-CS (Adaptive Educational System based on Cognitive Styles) A formative evaluation of AES-CS →to investigate the relationship of the learning behavior & cognitive styles →more effective & efficient →to assess the usability and appealing

Theoretical framework 1-1 Adaptive Hypermedia (AH) Hypermedia: Advantage: Supporting the new constructivist way Self regulated learning Contribute to enhance learning Promoting cognitive flexibility Disadvantage Regarding to cognitive overload Disorientation Adaptive Be important for web-based educational hypermedia Theoretical framework:理論架構：分別以Adaptive Hypermedia適性化超媒體 & Cognitive style and learning認知風格和學習來談論 Self regulated learning：自我管理學習 Constructivist way :構成的方法 Disorientation迷失

Theoretical framework 1-2 Adaptive Hypermedia System (AHS) Building a user model of the goals , preferences and knowledge of the individual user Accommodating various learner needs Learning style & Cognitive style Theoretical framework:理論架構：分別以Adaptive Hypermedia適性化超媒體 & Cognitive style and learning認知風格和學習來談論 Self regulated learning：自我管理學習 Constructivist way :構成的方法 Disorientation迷失

Theoretical framework 2-1 Cognitive style and learning Cognitive style Dealing with the form of cognitive activity, not content i.e. thinking, perceiving, remembering Learning style Broader construct Including cognitive along with affective and physiological styles Perceiving:理解

Theoretical framework 2-2 Cognitive Style: well-known division →FD/FI ( Witkin 1971) Field dependence Field independence More autonomous in interpersonal skills Be global in their perceptions Rely on external cues & be less able to differentiate an embedded figure Be holistic and require external help Required externally defined goals and reinforcements Appeared to benefit from illustrative advance organizers Prefer using program control More autonomous in cognitive restructuring skills Be analytical in their approach Rely on internal cues & be more able to differentiate an embedded figure Internal cues to help them to solve Developed self-defined goals and reinforcements Preferred illustrative post organizers Prefer learner control Perceptions:感覺 Illustrative:說明 Advance organizer :a bridging strategy that provides a connection between one unit and another, it acts as a schema for the learner to make sense out of the new concept. Post organizer: a 概要and supports the reconstruction of knowledge, it’s available after the presentation of new information

Design of AES-CS 1 System architecture and implementation It supports the course “Multimedia Technology Systems”, 4 years undergraduate students in Computer Science Dep. at the Aristotle University, Greece Be adapted to the cognitive style and to the level of knowledge acquired by the student . Be organized in the form of three basic modules Domain model Student model Adaptation module Aristotle University:亞里斯多德大學 These three components interact to adapt different aspects of the instructional process, i.e. adapting the content according to user’s prior knowledge; adapting the presentation of contents through selection combination of appropriate media; adapting the teaching strategies; modifying the selection of examples and links; recommending appropriate hyperlinks.

Design of AES-CS 2 Adapting instructional strategies AHS provides learners the ability to use different instructional modes Its design both issues of cognitive style and teaching strategy Teaching strategies refers to the instructional material and the instructional strategy AES-CS adopts the instructional strategies according to their cognitive style System screen for FI learners System screen for FD learners Take an example in a physical classroom: the ideal edu.environment , a tutor to experience and understand students’ personal goals and preferences and thus to promote their skill

Design of AES-CS 3 Adaptation flexibility The flexibility provided is the optimization of the adaptation under the adaptive systems need to be controllable by the user using the toolbar The system cannot be intelligent enough to appropriately adapt in all possible cases The Toolbar

Formative evaluation An integral part of the design methodology The judgments of the strengths and weakness of instruction Improve the effectiveness and efficiency of the instruction The types were used Expert review Content experts, designers, instructors One-to-one evaluation Small group evaluation It’s an integral part(不可缺少的部分)of the design methodology

Implement discussion This AES-CS was revised twice 1st: after the expert review and the one-to-one evaluation Five experts acted as evaluators in the expert review A teaching/training expert, an instructional design expert, a subject-mater expert, an educational technologist , a subject sophisticates Getting the prompt subject’s suggestions for improving from the reactions of experts and a debriefing session Ten fourth year undergraduate students participated in the one-to-one evaluation after the expert review It’s easy to modify the initial adaptation using the student model and interaction buttons The structure of the course is clear and easy to understand a subject sophisticates(精通): a student who has successfully completed the course

Implement discussion This AES-CS was revised twice 1st: after the expert review and the one-to-one evaluation Some significant recommendations were implemented during the revision phase: A dynamic technique resized the frame according to the mouse position Keeping the same adaptive annotation (using the color) but all the links are available to be selected by the student Giving an option regarding to adapt the background of the HTML page (before revision , the system was white background and black font) a subject sophisticates(精通): a student who has successfully completed the course

Implement discussion This AES-CS was revised twice 2nd: after the small group evaluation It’s consisted of the small group evaluation in a “real world” A lecture ”Digital image” as the example learning material The evaluation was conducted by collecting data about the instruction from a variety of sources A lecture on “Digital image” in fourth year undergraduate students studying the course Multimedia Technology Systems . ”Digital image” is to introduce the role of digital image into the development of a multimedia application. A variety of sources : different type of subjects (learner) Target population : 目標的母群體

Implement discussion This AES-CS was revised twice 2nd: after the small group evaluation Subjects: selecting from the volunteer represent the target population According to the Gauss theory: 68.28% is contained in [ the mean - S.D. ~ the mean + S.D.] in a normal distribution According to “the GEFT ( Witkin 1971 )”: classifying the participants (N=68) into FD and FI groups The score the test ranged from 0 to 18, the mean was 12.76 and S.D. was 4.35 Selecting eight students [mean score - S.D. ~ mean score + S.D.], the score was 8-17 Selecting one student from them with highest (18) & selecting one with lowest score(3) A lecture on “Digital image” in fourth year undergraduate students studying the course Multimedia Technology Systems . ”Digital image” is to introduce the role of digital image into the development of a multimedia application. A variety of sources : different type of subjects (learner) Target population : 目標的母群體 Grouped Embedded Figure Test, GEFT(藏圖測驗)，有相當的信度與效度考驗(Witkin, Otman, Raskin, & Kats, 1971; Witkin &Morre, 1974)，在電腦教學上的應用探討甚多。 Example:受試者若能從藏圖測驗(Grouped Embedded Figure Test, GEFT)迅速而正確的找出隱藏圖的位置便屬於場地獨立的類型

Implement discussion This AES-CS was revised twice 2nd: after the small group evaluation Procedure: The initial adaptation accorded to the cognitive style Belonging to FD (n=4), score range : 0~9 Belonging to FI (n=6), score range : 10~18 A pre-test determine subject’s prior knowledge on “Digital image” Working at the subjects’ own pace without time limit Finished, a post-test measures the learning gain from the instruction An attitude questionnaire determines their experience using the system A debriefing session assess the subjective satisfaction on the instructional and interface design A lecture on “Digital image” in fourth year undergraduate students studying the course Multimedia Technology Systems . ”Digital image” is to introduce the role of digital image into the development of a multimedia application. A variety of sources : different type of subjects (learner) Target population : 目標的母群體

Implement discussion This AES-CS was revised twice 2nd: after the small group evaluation Results: The evaluation at this phase →improve the system →more effective and efficient Pre-test & Post-tests have the same content in the form of open-ended iteams in each test Each correct answer gets 2 points →total score 0~18 Performance means on pre- and pose- test It’s demonstrated the students got a better performance in the post-test The evaluation offers the usability and the appealing of the courseware The learners felt challenged and wanted to use the courseware again A lecture on “Digital image” in fourth year undergraduate students studying the course Multimedia Technology Systems . ”Digital image” is to introduce the role of digital image into the development of a multimedia application. A variety of sources : different type of subjects (learner) Target population : 目標的母群體 demonstrated:證實 Usability可用性: measure of the case of system’s learned and used . Improving usability directed at improved interface The appealing of the courseware(魅力): through the attitude questionnaires and debriefing session

Implement discussion This AES-CS was revised twice 2nd: after the small group evaluation The significant recommendations: Concept map and graphic path indicator FD: it’s useful to organize the structure of the knowledge domain, suggesting these tools should be active so to be sued as an extra navigation tool FI: Don’t use the tools to organize the structure of the knowledge domain, agreeing the earlier suggestion and tools help them to easily navigate , avoiding disorientation Users have the ability to display or not the content menu All of them agreed on the usefulness to change amount of guidance and direction according to their individual needs A lecture on “Digital image” in fourth year undergraduate students studying the course Multimedia Technology Systems . ”Digital image” is to introduce the role of digital image into the development of a multimedia application. A variety of sources : different type of subjects (learner) Target population : 目標的母群體 demonstrated:證實 Usability可用性: measure of the case of system’s learned and used . Improving usability directed at improved interface The appealing of the courseware(魅力): through the attitude questionnaires and debriefing session avoiding disorientation:避免迷失

Conclusions & recommendations The design and development of an AES-CE The system has two main components The client running in the Web browser Be implemented as Web pages residing in a number of liked framed. The server running in a PC workstation The development platform used is ASP using DHTML and JavaScript language

Conclusions & recommendations Some recommendations from expert ,students, evaluation Design : AES accommodates for cognitive styles were effective and efficient Initial adaptation: It needs to be controllable by the user because it can’t be intelligent enough to appropriately adapt in all possible cases Instructions : students have the ability to access the maximum amount of guidance and instructions Structure: the concept map and the graphic path indicator should be active so to be used as an extra navigation tool

Comments Providing another way (adaptive screen) to design an adaptive learning system It only has ten sample participate the system implement, it’s not objective It emphasizes cognitive style , but we can’t see the implement to differentiate FI / FD learner in the initial.

Appendix

Fig. 1. System architecture. Be easy to construct & modify & reflect the characteristics of different students 1.”Adaptive presentation technique”: Conditional text, page variants 2.”Adaptive navigation support”: finding an suitable path in hypermedia-learning environment 3.By student’s prior knowledge It serves as a basis for structuring the content of AES-CS

Table 1 The student model

Table 2 Instructional strategies Program control versus Learner control Global versus Analytical approach The use of contextual organizer Instructions Structure Feedback Instructional strategies and the implementation

This AES-CS was revised twice Fig. 2. System screen with the initial adaptation for Field Independence (FI) learners. Blue color → Recommended Gray color → Not ready to be learned Design of AES-CS 2 Discussion – This AES-CS was revised twice

Fig. 3. System screen with the initial adaptation for Field Independence (FD) learners. Toolbar will discuss later

Table 3 Instructional strategies and their implementation

Fig. 4. The Toolbar.

Fig. 5. The frame with instructional guidance before and after the revision Before the revision (fixed size) After the revision (flexible size)

Table 4 Performance means on pre- and post-test Both the FD & FI , the performance higher in post test The improvement of the system , FD > FI

Table 5 Performance on pre- and post-test

Adaptive module The adaptive system screen for FI learner The adaptive system screen for FD learner Blue color →Recommended Gray color →Not ready to be learned