1. Browse design in digital libraries: Impact on user experience Xiangmin Zhang, Yuelin Li, Jingjing Liu & Ying Zhang SCILS, Rutgers University {xzhang,lynnlee, jingjing,yinzhang}@scils.rutgers.edu LIDA, Dubrovnik, May 28, 2007

2. 2 Outline Background Background Purpose of the study Purpose of the study Methodology Methodology Results Results Discussion Discussion

3. 3 Background IEEE supported research IEEE supported research Part of a large scale usability evaluation of Xplore digital library: Part of a large scale usability evaluation of Xplore digital library: Searching & browsing designsSearching & browsing designs System performance issuesSystem performance issues In comparison with ACM and IEEE CSIn comparison with ACM and IEEE CS Started in March 2005, reports submitted to IEEE, Inc Started in March 2005, reports submitted to IEEE, Inc Part of data yet to be analyzed Part of data yet to be analyzed

4. 4 Purpose of the current study The purpose of the study has two folds The purpose of the study has two folds to find out how the interaction designs for browsing would affect the user experience, andto find out how the interaction designs for browsing would affect the user experience, and to inform the design of better user experience in digital librariesto inform the design of better user experience in digital libraries

5. 5 Research Questions Do different browse design features (browsing conference proceedings) have an effect on users ’ browsing experience? Do different browse design features (browsing conference proceedings) have an effect on users ’ browsing experience? What design features would lead to poor user experience? What design features would lead to poor user experience?

6. 6 Methodology Three DLs and their browse features Three DLs and their browse features Research design Research design ParticipantsParticipants Browsing taskBrowsing task ProcedureProcedure

7. 7 Three Digital Libraries Three operational digital libraries: Three operational digital libraries: ACM Digital Library (ACM)ACM Digital Library (ACM) IEEE Computer Society Digital Library (IEEE CS)IEEE Computer Society Digital Library (IEEE CS) IEEE Xplore Digital Library (Xplore)IEEE Xplore Digital Library (Xplore) WebsitesWebsites

8. 8 ACM

9. 9 IEEE CS

10. 10 Xplore

11. 11 Major design characteristics Design features ACM IEEE CS Xplore Browse initiation Click “proceedings” link on the home page Click “alphabetic list” to access the proceedings from home page. Click “proceedings” link on the home page. Browse options for conference proceedings Search within proceedings and browse alphabetically, then providing “search within (the name of a specific proceeding)” Alphabetically Browse by keyword and browse alphabetically, then providing “search this conference proceedings” and “search this publication” Navigation for browsing proceedings No navigation support for “Browse the ACM proceedings;” but providing a navigation bar for “Search the ACM proceedings” result page: 1 2 3 4 5 6 7 8 9 10 next” 2345678910next2345678910next No navigation facilities Providing a navigation bar as: “View: 1-25 | 26-50 | 51-75 | 76-100 | 101-125 | Next > | Back to top” if multiple pages returned for an initial letter; Providing a link “Back to top” on every page. 26-50 51-7576-100101-125 Next >Back to top26-50 51-7576-100101-125 Next >Back to top

12. 12 Research Design 36 participants from Rutgers students community 36 participants from Rutgers students community Included both graduate and undergraduate students Included both graduate and undergraduate students Science, engineering, MLIS students Science, engineering, MLIS students

13. 13 Research Design (2) Browsing Task: Xplore: Please browse the proceedings of ITCC: International conference on Information Technology (2004): Coding and Computing, and locate and save two papers about data streaming. ACM Please browse the proceedings of Annual Symposium on Computational Geometry (SCG) (2004), and locate and save two papers about data streaming. IEEE Computer Society Please browse the proceedings of ITCC: International conference on Information Technology (2004): Coding and Computing, locate two papers about data streaming, and save their abstracts

14. 14 Research Design (3) Latin Square design: randomize the order of tasks and systems Latin Square design: randomize the order of tasks and systems Every 3 participants followed the same order Every 3 participants followed the same order One student dropped One student dropped

15. 15 User Interaction Measures The amount of browsing time The amount of browsing time The number of browsing steps The number of browsing steps The number of user errors related to browsing The number of user errors related to browsing

16. 16 Results User characteristics: User characteristics: Experienced computer users, and very experienced (83%) Web search engine usersExperienced computer users, and very experienced (83%) Web search engine users Inexperienced users of the three digital libraries in the study:Inexperienced users of the three digital libraries in the study: 66% never used ACM and Xplore 66% never used ACM and Xplore 80% never used IEEE CS. 80% never used IEEE CS. Strong correlations Strong correlations

17. 17 Results: performance XploreACM IEEE CS Number of articles located 1.94 (.34) 1.86(.43) 1.00 (.82)* Overall correctness of articles located.84 (.34).89 (.27).42 (.36)* Note: values in parentheses are standard deviations. * p <.001

18. 18 Comparison of interaction measures ACM IEEE CS XploreF Mean browsing time (seconds) 276.86610.71345.5125.931* Mean # of browsing steps 12.9137.3129.408.92* Total # of browsing errors 162 *Significant difference was found at p<.01, tested by ANOVA

19. 19 User satisfaction Statement User mean ratings ACMIEEE CSXplore Satisfaction with the browse result* 5.342.314.77 Overall easy to browse*5.633.295.37 Overall satisfaction with the browse feature of the system* 5.773.575.31 Easy to learn to use5.805.205.77 * Significant difference at p <.05, ANOVA test

20. 20 Discussion Major user problems with IEEE CS design: Major user problems with IEEE CS design: No search function provided with browseNo search function provided with browse Does not meet the user ’ s expectation of using search as short-cuts. Does not meet the user ’ s expectation of using search as short-cuts. Unpleasant visual/presentation designUnpleasant visual/presentation design Major user problems with Xplore design: Major user problems with Xplore design: Proceedings not ordered either on full title or acronym.Proceedings not ordered either on full title or acronym. Suggestions to improving system designs Suggestions to improving system designs

21. 21 Further studies More tasks More tasks Visual/presentation design issues Visual/presentation design issues Experienced users Experienced users

22. 22 Thank you! Questions?