Nives Mikelic Preradovic, Sanja Kisicek, Damir Boras Marvin - A Conversational Agent Based Interface for the Study of Information Sciences Nives Mikelic Preradovic, Sanja Kisicek, Damir Boras University of Zagreb, Faculty of Humanities and Social Sciences, Department of Information Sciences nmikelic@ffzg.hr, smatic@ffzg.hr, dboras@ffzg.hr

Outline Marvin’s knowledge base and reply structure Evaluation Who is Marvin? Marvin’s knowledge base and reply structure Evaluation Future work

Who is Marvin? Paranoid android from The Hitchhiker's Guide to the Galaxy by D. Adams

Who is Marvin? Has a "brain the size of a planet" which he is seldom able to use No task he could be given would occupy even the tiniest fraction of his vast intellect afflicted with severe depression and boredom The true horror of Marvin's existence is that no task he could be given would occupy even the tiniest fraction of his vast intellect In other words: he is able to solve every world problem…

Why Marvin? Aim was to build a virtual character that: Aim was not to Is capable of engaging our undergraduate students in a meaningful conversation Has the ability to parse NLQ and, by referring to a knowledge base, generate NLA Will increase the students' interest in the Study of Information Sciences, ease the search for particular information and boost satisfaction with the presentation of information Aim was not to design a tutor that would teach particular subject matter One of the most popular uses of the chatterbots is in the online tutoring process ELIZA [7] was the first chatterbot written by Joseph Weizenbaum between 1964 and 1966. It was a simulation of a Rogerian psychotherapist operated by processing users' responses to scripts. It rephrased the user's statements as questions and posed those to the “patient”. PARRY [2] was another famous early chatterbot who attempted to simulate a paranoid schizophrenic it has been shown [6] that even fairly simple chatterbots can increase the quality of experience for the user of interactive services and applications Although today one can find chatterbots that act as very advanced tutoring systems with sophisticated knowledge bases, most of the existing systems are focused on the quality and accuracy of information and the way it is presented to users, rather than the production of the knowledge base

Marvin’s knowledge base AIML: category (pattern,template, context: that, topic) Reqursion: Marvin produces the same reply to different query formulations that share the same or similar meaning (teach-lecture-instruct) AIML templates activated by the appearance of a keyword anywhere in the input Keywords: the most probable user's queries Knowledge base AIML is the XML dialect developed by Richard Wallace  It formed the basis for what was initially a highly extended Eliza called "A.L.I.C.E." (Artificial Linguistic Internet Computer Entity), which won the annual Loebner Prize Contest[1] for Most Human Computer The basic unit of knowledge in AIML is called a category. Each category consists of an input question, an output answer, and an optional context The optional context portion of the category consists of two variants, called <that> and <topic>. The tag <that> has to match Marvin’s last utterance. Remembering one last utterance is important if Marvin asks a question. This tag was implemented to use the user's reply to point the conversation in the specific direction. The <topic> tag appears outside the category, and collects a group of categories together. the name of the lecturer, the course prerequisites (if any), the number of ECTS points for the specific course, the hours of lectures / seminars / labs per course, the status of the course (obligatory or elective) and the description of the exam. Also, Marvin gives information if a specific course is prerequisite to some other course or prerequisite to the graduate study. It also has built-in information regarding the profile of undergraduate study (such as the required number of ECTS points for each semester of the study, etc). Apart from that, Marvin has built-in information regarding the Department profile, history and staff

Marvin’s reply Tries to boost communication with the user posing the questions Topic change triggers the content of the reply Marvin attempts to follow the normal flow and nature of the communication Marvin’s replies are enhanced with comments designed to convince the user that Marvin is a student himself e.g. if user changes the name of the course in his utterance, Marvin will immediately change the content of the reply, providing the user only with the necessary information) The example of the comment is: “Oh, in my time it was different…”, “When I was taking this exam, it was 12 pages long, can you believe it?”

Evaluation The database was checked for accuracy, interpretation and relevance to the types and the level of questions being asked 50 inputs and answers per evaluator / each evaluator spent 30 minutes on average chatting with Marvin Two perspectives: competent information scientist / pretend bachelor student perspective Limitations: multi clause units, exchanges that range over more than one turn Marvin’s evaluators were four most successful undergraduate trainee teachers in the final year of the undergraduate study

Evaluation the depth of knowledge too narrow to cope with open-ended conversations with humans database should be improved with more factual world knowledge Currently: no actual knowledge of what it is talking about, avoidance strategy He or it? avoid answering a question it in fact has no answer for, and suggests a new topic, giving the illusion that it actually has something to say about the new topic

Future work Modify Marvin to take on the personality and knowledge base of the key information theorists Sensitive to the knowledge levels of student users and opened to explanatory questions Use Wikipedia information to build its conversations and offer links to articles in the field of IS Many students of information sciences experience problems in learning theory for the specific courses in both undergraduate and graduate studies. They usually find it very hard to get started with reading, need cartoon level introduction, leading on to more complex material an accessible chatterbot with a knowledge base reflecting key areas of information science could make an important educational contribution. In our future work we plan to develop a small number of knowledge bases for use in information science study, particularly in Knowledge Organization course and Theory of Information Science course Students who have a research question or essay title that needs to be researched could use the bot to generate content that can be included in their assessed work. Conversation with the bot will be recordable and can then be cut and pasted into an essay. Students will be required to edit this information into a coherent essay, just as they would with information collected from texts.