1. THE ROLE OF ADAPTIVE ELEMENTS IN WEB-BASED SURVEILLANCE SYSTEM USER INTERFACES RICARDO LAGE, PETER DOLOG, AND MARTIN LEGINUS DOLOG@CS.AAU.DK

2. Motivation Task Evaluation Set Up Initial System GUI Evaluation Resutls New System GUI Results Conlusions Outline THE ROLE OF ADAPTIVE ELEMENTS IN WEB-BASED SURVEILLANCE SYSTEM USER INTERFACES – 9. JULY 2014 INTELLINGENT WEB AND INFORMATION SYSTEMS, DEPARTMENT OF COMPUTER SCIENCE AALBORG UNIVERSITET

3. Task of epidemic surveilence Monitoring official sources Monitoring social media not so common Support for exploring a large number of indications and signals Signals – aggregates of information items (tweets, messages, …) with number over certain treshold There are only very few studies of user interfaces in general and no adaptive user interfaces in particular Motivation THE ROLE OF ADAPTIVE ELEMENTS IN WEB-BASED SURVEILLANCE SYSTEM USER INTERFACES – 9. JULY 2014 INTELLINGET WEB AND INFORMATION SYSTEMS, DEPARTMENT OF COMPUTER SCIENCE AALBORG UNIVERSITET

4. Build a system which supports a survailance on social media Evaluate the system with experts Based on evaluation build an improved version Evaluate in the second round Context: Meco FP7 EU project Task THE ROLE OF ADAPTIVE ELEMENTS IN WEB-BASED SURVEILLANCE SYSTEM USER INTERFACES – 9. JULY 2014 INTELLINGET WEB AND INFORMATION SYSTEMS, DEPARTMENT OF COMPUTER SCIENCE AALBORG UNIVERSITET

5. Health Map: http://healthmap.org/en/ Interesting system, no data on UI or filtering studies Real Time Outbreak and Disease Surveilance Focusing on official sources, study of use not known to us Other domains News recoomendation: slightly different task – news aggregations are usually not urgent, they do not have to be monitored continuosly, … Notifications and awareness: task can be very different, but awareness is also our focus => we take on a strategy where users are drivers Visualization: for example trendig topics and topic flow on twitter => different task, requires different focus for visualization Related work THE ROLE OF ADAPTIVE ELEMENTS IN WEB-BASED SURVEILLANCE SYSTEM USER INTERFACES – 9. JULY 2014 INTELLINGET WEB AND INFORMATION SYSTEMS, DEPARTMENT OF COMPUTER SCIENCE AALBORG UNIVERSITET

6. Focus group A small group – 6-12 people Shared understanding while a possibility to voice different opinions Moderator to ensure to stay within predefined topics Our method: 2 rounds – after the initial prototype and after the changes from the first round Evaluation method THE ROLE OF ADAPTIVE ELEMENTS IN WEB-BASED SURVEILLANCE SYSTEM USER INTERFACES – 9. JULY 2014 INTELLINGET WEB AND INFORMATION SYSTEMS, DEPARTMENT OF COMPUTER SCIENCE AALBORG UNIVERSITET

7. Rating scale to assess quality or quantity Thumbs up or thumbs down Bookmarking Comments Elements considered THE ROLE OF ADAPTIVE ELEMENTS IN WEB-BASED SURVEILLANCE SYSTEM USER INTERFACES – 9. JULY 2014 INTELLINGET WEB AND INFORMATION SYSTEMS, DEPARTMENT OF COMPUTER SCIENCE AALBORG UNIVERSITET

8. 8 participants – survailence experts/representatives of international epidemics organizations Asked to create signales of their interests The system then produced recommendations After they have browsed the recommended signals and documents we have conducted the focus group with these questions: How would recommendations help you deal with the excess of information you might receive? How would you like to evaluate the recommendations you receive? How should the recommendations be placed in the system? Discussion took one and half hour Goal: to discuss and learn how to present recommendations in the surveilence task Evaluation set-up: Focus Group THE ROLE OF ADAPTIVE ELEMENTS IN WEB-BASED SURVEILLANCE SYSTEM USER INTERFACES – 9. JULY 2014 INTELLINGET WEB AND INFORMATION SYSTEMS, DEPARTMENT OF COMPUTER SCIENCE AALBORG UNIVERSITET

9. Initial Prototype/Signal Definition THE ROLE OF ADAPTIVE ELEMENTS IN WEB-BASED SURVEILLANCE SYSTEM USER INTERFACES – 9. JULY 2014 INTELLINGET WEB AND INFORMATION SYSTEMS, DEPARTMENT OF COMPUTER SCIENCE AALBORG UNIVERSITET

10. Initial Prototype/Signals and Documents THE ROLE OF ADAPTIVE ELEMENTS IN WEB-BASED SURVEILLANCE SYSTEM USER INTERFACES – 9. JULY 2014 INTELLINGET WEB AND INFORMATION SYSTEMS, DEPARTMENT OF COMPUTER SCIENCE AALBORG UNIVERSITET

11. Initial Prototype/Navigation THE ROLE OF ADAPTIVE ELEMENTS IN WEB-BASED SURVEILLANCE SYSTEM USER INTERFACES – 9. JULY 2014 INTELLINGET WEB AND INFORMATION SYSTEMS, DEPARTMENT OF COMPUTER SCIENCE AALBORG UNIVERSITET

12. Useful: Tag cloud useful Rating of signals important To improve: Search process Information they provided Content organization Users should be able to assign new tags Relevancy and irrelevancy of tags should be exposed Results THE ROLE OF ADAPTIVE ELEMENTS IN WEB-BASED SURVEILLANCE SYSTEM USER INTERFACES – 9. JULY 2014 INTELLINGET WEB AND INFORMATION SYSTEMS, DEPARTMENT OF COMPUTER SCIENCE AALBORG UNIVERSITET

13. Q1: The system advises them about missing information – help them to discover topics relevant to their interest, not exact match Q1: Recommendations help in overload Q2: Agreed on usefulness of signal evaluations Q2: Same options as for signal evaluation should be available also for documents Q2: Options for re-ranking should be available Q2: Dislike option should be available Q3: Recommendation or recommended signal as a term is not very appropriate; instead ”related signals” could be used => they would like to know reason for recommendations Answers to questions THE ROLE OF ADAPTIVE ELEMENTS IN WEB-BASED SURVEILLANCE SYSTEM USER INTERFACES – 9. JULY 2014 INTELLINGET WEB AND INFORMATION SYSTEMS, DEPARTMENT OF COMPUTER SCIENCE AALBORG UNIVERSITET

14. New GUI/Map THE ROLE OF ADAPTIVE ELEMENTS IN WEB-BASED SURVEILLANCE SYSTEM USER INTERFACES – 9. JULY 2014 INTELLINGET WEB AND INFORMATION SYSTEMS, DEPARTMENT OF COMPUTER SCIENCE AALBORG UNIVERSITET

15. New GUI/Signal Listing THE ROLE OF ADAPTIVE ELEMENTS IN WEB-BASED SURVEILLANCE SYSTEM USER INTERFACES – 9. JULY 2014 INTELLINGET WEB AND INFORMATION SYSTEMS, DEPARTMENT OF COMPUTER SCIENCE AALBORG UNIVERSITET

16. 1. the user rules and the contents of each signal’s documents; 2. the user rules and the properties (i.e., disease and location values) of each signal; 3. the user’s specific rules for locations, similar locations and each signal’s documents; 4. the user’s specific rules for locations, similar locations and the locations together with similar locations of each signal; 5. the tag set of the user and the contents of each signal’s documents; 6. the tag set of the user and the properties of each signal; 7. the date of the signal and the date when the recommendation is being computed. Explanation based on the highest score in four categories: content (1,2), location (3,4), tag (5, 6), and date (7) Factors in similarity calculations THE ROLE OF ADAPTIVE ELEMENTS IN WEB-BASED SURVEILLANCE SYSTEM USER INTERFACES – 9. JULY 2014 INTELLINGET WEB AND INFORMATION SYSTEMS, DEPARTMENT OF COMPUTER SCIENCE AALBORG UNIVERSITET

17. New GUI/Tag cloud THE ROLE OF ADAPTIVE ELEMENTS IN WEB-BASED SURVEILLANCE SYSTEM USER INTERFACES – 9. JULY 2014 INTELLINGET WEB AND INFORMATION SYSTEMS, DEPARTMENT OF COMPUTER SCIENCE AALBORG UNIVERSITET

18. The same participants All rated the changes possitively More tabs for example with a timeline Tag cloud provides additional context to the list of signal Tag cloud simplifies the browsing process 2nd Focus Group THE ROLE OF ADAPTIVE ELEMENTS IN WEB-BASED SURVEILLANCE SYSTEM USER INTERFACES – 9. JULY 2014 INTELLINGET WEB AND INFORMATION SYSTEMS, DEPARTMENT OF COMPUTER SCIENCE AALBORG UNIVERSITET

19. Visualization of recommendations and specification of preferences is at least as important as algorithms for recommendations Content organization, possibilities for implicit feedback, explanations are all means which improve the percieved recommendations Focus group with experts is a good method to conduct evaluations to receive both: confirmations for the design but also suggestions for changes Conclusions THE ROLE OF ADAPTIVE ELEMENTS IN WEB-BASED SURVEILLANCE SYSTEM USER INTERFACES – 9. JULY 2014 INTELLINGET WEB AND INFORMATION SYSTEMS, DEPARTMENT OF COMPUTER SCIENCE AALBORG UNIVERSITET

20. dolog@cs.aau.dk