1. AnnotatEd: A Social Navigation and Annotation Service for Web-based Educational Resources Rosta Farzan & Peter Brusilovsky Personalized Adaptive Web Systems (PAWS) Lab University of Pittsburgh

2. Rosta Farzan (rosta@cs.pitt.edu) & Peter Brusilovsky (peterb@pitt.edu) PAWS Lab – University of Pittsburgh Introduction Web is lacking important features –Supported in standalone hypertext systems –Important for demanding application such as web- based education Advanced hypertext features –Content annotation Allows user to annotate non-restricted web content –Guided Tour Allows user to navigate a complex hyperspace along a predefined path –Adaptive Navigation Support Helps user to select the most relevant link to navigate

3. Rosta Farzan (rosta@cs.pitt.edu) & Peter Brusilovsky (peterb@pitt.edu) PAWS Lab – University of Pittsburgh Problem Integrating several functionalities in one system –Result > Sum of its parts Our work –Integrating Annotation and Adaptive Navigation Support –Open-corpus web educational resources AnnotatEd

4. Rosta Farzan (rosta@cs.pitt.edu) & Peter Brusilovsky (peterb@pitt.edu) PAWS Lab – University of Pittsburgh Outline Background –Web Annotation –Adaptive Navigation Support Social Navigation Support AnnotatEd –Implementation phases Evaluation Conclusion & Future Work

5. Rosta Farzan (rosta@cs.pitt.edu) & Peter Brusilovsky (peterb@pitt.edu) PAWS Lab – University of Pittsburgh Web Annotation Group 1 –Developing advanced architecture and sophisticated but easy-to-use interface Group 2 –Developing annotation systems for supporting collaboration over the web

6. Rosta Farzan (rosta@cs.pitt.edu) & Peter Brusilovsky (peterb@pitt.edu) PAWS Lab – University of Pittsburgh Web Annotation – Group 1 Support content-aware annotation on arbitrary web pages Gibeo –Add.gibeo.net to any web page –Highlighting any part of the text –Associating quality with highlighted text (important, wrong, cool,..) –Shared annotations Marginalia –JavaScript web annotation system –Highlighting any part of the text –Associating comments with the highlighted text –Public and private annotations

7. Rosta Farzan (rosta@cs.pitt.edu) & Peter Brusilovsky (peterb@pitt.edu) PAWS Lab – University of Pittsburgh Web Annotation – Group 2 Annotea –Collaborative development of Semantic Web via shared annotation –Annotations are in the form of comments, notes, explanations, … Web discussion in Microsoft Office –Collaborative annotation of any web page

8. Rosta Farzan (rosta@cs.pitt.edu) & Peter Brusilovsky (peterb@pitt.edu) PAWS Lab – University of Pittsburgh Adaptive Navigation Support Classic adaptive navigation support –Guiding learners to appropriate resources –Requires manual annotation of the content –Not Well suited for the large volume of open-corpus documents Social Navigation Support –Social navigation theory “Moving toward cluster of people” (Dourish & Chalmers, 1994) –Making use of past learners’ interactions to guide new users –Relies on collective knowledge of community

9. Rosta Farzan (rosta@cs.pitt.edu) & Peter Brusilovsky (peterb@pitt.edu) PAWS Lab – University of Pittsburgh Social Navigation Support (SNS) Traffic based SNS –“footprints” Information about number of visits –Promotes links with higher number of visits –Modified CoWeb Tracks how often a page is accessed or modified Visualized the density of aggregated access over last 24 hours Annotation based SNS –Employing users’ annotation –Promotes links to pages with users’ annotations –Educo Allows learners to associate comments with a document Keeps track of when comments are modified

10. Rosta Farzan (rosta@cs.pitt.edu) & Peter Brusilovsky (peterb@pitt.edu) PAWS Lab – University of Pittsburgh AnnotatEd Social navigation and annotation services for browsing web educational resources Accompanies the learners from page to page –Redirecting all the links through AnnotatEd server Allows annotation in the form of free format comments, highlights Offers Social Navigation Support

11. Rosta Farzan (rosta@cs.pitt.edu) & Peter Brusilovsky (peterb@pitt.edu) PAWS Lab – University of Pittsburgh SNS in AnnotatEd Augmenting links inside the pages –Measuring the past activity of the group Click stream Time spent reading each page Learners’ annotations –Visual icons Group activity Individual activity Traffic based SNS –Background color: magnitude of group visiting activity –Human icon: magnitude of individual visiting activity Darker color = higher number of visits Annotation based SNS –Background color: magnitude of group annotation activity –Sticky note icon: presence of learners annotation –Sticky note color: magnitude of individual annotation activity Implementation –Baseline, Phase II, Phase III

12. Rosta Farzan (rosta@cs.pitt.edu) & Peter Brusilovsky (peterb@pitt.edu) PAWS Lab – University of Pittsburgh Baseline Implementation Complete tracking feature Simple annotation interface –Private notes Traffic based SNS –Links to pages with user annotation were augmented for the author of the annotation Preliminary evaluation –60% of students appreciated the ability to annotate –Interested in sharing annotation and seeing annotations of others –Type of learners’ annotations Praise  37 notes Problem  36 notes General  34 notes

13. Rosta Farzan (rosta@cs.pitt.edu) & Peter Brusilovsky (peterb@pitt.edu) PAWS Lab – University of Pittsburgh Phase II Implementation Enhanced interface –Highlighting feature –Public and private annotations –Typed annotations Praise, Problem, General –Signed annotations Motivating students 90% of annotations were signed Annotation based SNS –Type and density of learners’ annotation –Annotation temperature Praise  warm Problem  cold

14. Rosta Farzan (rosta@cs.pitt.edu) & Peter Brusilovsky (peterb@pitt.edu) PAWS Lab – University of Pittsburgh AnnotatEd – Phase II

15. Rosta Farzan (rosta@cs.pitt.edu) & Peter Brusilovsky (peterb@pitt.edu) PAWS Lab – University of Pittsburgh Phase III - Implementation Interface problem in Phase II –61% of general annotations were praise Type box was too distant from save button –Problem type was used mostly to report problem to the teacher Instead of negative point about the page Bookmarking feature –Generating a report for annotated and bookmarked pages

16. Rosta Farzan (rosta@cs.pitt.edu) & Peter Brusilovsky (peterb@pitt.edu) PAWS Lab – University of Pittsburgh Evaluation Classroom study School of Information Science at University of Pittsburgh 5 semesters –This paper focused on the recent studies (2 semesters) Introductory C programming (undergrad course), Information Retrieval (graduate course) System was introduced in the class by the instructor

17. Rosta Farzan (rosta@cs.pitt.edu) & Peter Brusilovsky (peterb@pitt.edu) PAWS Lab – University of Pittsburgh Hypotheses 1.Providing navigation support for “browsing” web educational resources is important –Browsing-based access is popular among students even with presence of other access options such as map based navigation, or search 2.Annotating links with SNS cues would affect students navigation decision 3.Pages with annotation-based SNS are important pages 4.Students would appreciate the annotation functionality of the system 5.Students would appreciate Annotation based SNS

18. Rosta Farzan (rosta@cs.pitt.edu) & Peter Brusilovsky (peterb@pitt.edu) PAWS Lab – University of Pittsburgh Browsing vs. Map vs. Search AnnotatEd is connected to KnowledgeSea, a navigation map with search feature Same educational material can be accessed through –Browsing, Navigation map, Search Browsing is at least as important as map-based navigation and more important than search

19. Rosta Farzan (rosta@cs.pitt.edu) & Peter Brusilovsky (peterb@pitt.edu) PAWS Lab – University of Pittsburgh Importance of Browsing Access through browsing –Provides more context –Makes students more certain about the content of the page Comparing exact pages accessed through browsing and map-based navigation –62 pages –Desirability ~ time spent reading (TSR) the page MapBrowsingP-value TSR64.24104.840.028 Students spent significantly more time on pages accessed through browsing

20. Rosta Farzan (rosta@cs.pitt.edu) & Peter Brusilovsky (peterb@pitt.edu) PAWS Lab – University of Pittsburgh Effect of SNS on Link-Following Comparing number of accesses to links with and without traffic or annotation based SNS Significantly more access on pages with SNS specially annotation-based SNS

21. Rosta Farzan (rosta@cs.pitt.edu) & Peter Brusilovsky (peterb@pitt.edu) PAWS Lab – University of Pittsburgh Importance of Pages with Students’ Annotation Comparing average accesses to pages with and without students’ annotations before appearance of annotations AnnotatedNot Annotated p-value (t test) C programming3.152.020.03 Information Retrieval 2.581.420.0005 Annotated Pages are Important

22. Rosta Farzan (rosta@cs.pitt.edu) & Peter Brusilovsky (peterb@pitt.edu) PAWS Lab – University of Pittsburgh Subjective Data Analysis Students appreciate the annotation feature of AnnotatEd Students appreciate annotation based SNS

23. Rosta Farzan (rosta@cs.pitt.edu) & Peter Brusilovsky (peterb@pitt.edu) PAWS Lab – University of Pittsburgh Conclusion AnnotatEd provides social navigation support and annotation service Our evaluation shows that –Browsing is an important information access method and provide context-rich information access method –Pages with students’ annotation are important –Traffic and annotation based SNS affect students navigation pattern and are appreciated by students

24. Rosta Farzan (rosta@cs.pitt.edu) & Peter Brusilovsky (peterb@pitt.edu) PAWS Lab – University of Pittsburgh Future Work Evaluation with larger number of users Improving annotation interface –Pen-based annotation –Will encourage higher number of annotations Improving bookmarking feature –Employing bookmarks for providing social navigation support

25. Rosta Farzan (rosta@cs.pitt.edu) & Peter Brusilovsky (peterb@pitt.edu) PAWS Lab – University of Pittsburgh