1. 1 CNI 2005 Fall BriefingTechLens TechLens: Exploring the Use of Recommenders to Support Users of Digital Libraries Joseph A. Konstan, Nishikant Kapoor, Sean M. McNee, John T. Butler GroupLens Research Project and University Libraries University of Minnesota

2. 2 CNI 2005 Fall BriefingTechLens Introduction Challenges and Opportunities u large digital collections of uneven quality and scope u continuing trend towards out-of-library usage of library collections u extensive collections of metadata è citations and other linkage data (published and personally collected) è venue data u expectations of personal service è increased prevalence of personalization

3. 3 CNI 2005 Fall BriefingTechLens Recommenders Tools to help identify worthwhile stuff u Filtering interfaces è E-mail filters, clipping services è Schedulable current awareness searches u Recommendation interfaces è Suggestion lists, “top-n,” offers and promotions u Prediction interfaces è Evaluate candidates, predicted ratings

4. 4 CNI 2005 Fall BriefingTechLens Amazon.com

5. 5 CNI 2005 Fall BriefingTechLens Wine.com Seeking

6. 6 CNI 2005 Fall BriefingTechLens Cdnow album advisor

7. 7 CNI 2005 Fall BriefingTechLens CDNow Album advisor recommendations

8. 8 CNI 2005 Fall BriefingTechLens Classic CF C.F. Engine RatingsCorrelations

9. 9 CNI 2005 Fall BriefingTechLens Submit Ratings C.F. Engine RatingsCorrelations ratings

10. 10 CNI 2005 Fall BriefingTechLens Store Ratings C.F. Engine RatingsCorrelations ratings

11. 11 CNI 2005 Fall BriefingTechLens Compute C.F. Engine RatingsCorrelations pairwise corr.

12. 12 CNI 2005 Fall BriefingTechLens Request Recommendations C.F. Engine RatingsCorrelations request

13. 13 CNI 2005 Fall BriefingTechLens Identify Neighbors C.F. Engine RatingsCorrelations find good … Neighborhood

14. 14 CNI 2005 Fall BriefingTechLens Select Items; Predict Ratings C.F. Engine RatingsCorrelations Neighborhood predictions recommendations

15. 15 CNI 2005 Fall BriefingTechLens Understanding the Computation

16. 16 CNI 2005 Fall BriefingTechLens Understanding the Computation

17. 17 CNI 2005 Fall BriefingTechLens Understanding the Computation

18. 18 CNI 2005 Fall BriefingTechLens Understanding the Computation

19. 19 CNI 2005 Fall BriefingTechLens Understanding the Computation

20. 20 CNI 2005 Fall BriefingTechLens Understanding the Computation

21. 21 CNI 2005 Fall BriefingTechLens Understanding the Computation

22. 22 CNI 2005 Fall BriefingTechLens First Steps … Established that citation web data can be used to effectively rate/recommend research papers Developed and evaluated a demonstration recommender to recommend additional citations for an existing paper (using its references) u original demo used CiteSeer u this version uses ACM digital library

23. 23 CNI 2005 Fall BriefingTechLens DL Recs C.F. Engine RatingsCorrelations

24. 24 CNI 2005 Fall BriefingTechLens DL Recs C.F. Engine RatingsCorrelations Votes

25. 25 CNI 2005 Fall BriefingTechLens DL Recs C.F. Engine RatingsCorrelations Votes

26. 26 CNI 2005 Fall BriefingTechLens DL Recs C.F. Engine RatingsCorrelations Votes

27. 27 CNI 2005 Fall BriefingTechLens DL Recs C.F. Engine RatingsCorrelations Votes Request

28. 28 CNI 2005 Fall BriefingTechLens DL Recs C.F. Engine RatingsCorrelations Votes Request

29. 29 CNI 2005 Fall BriefingTechLens DL Recs C.F. Engine RatingsCorrelations Votes Request Recommendations

30. 30 CNI 2005 Fall BriefingTechLens Demonstration #1 Steps u Select user u Select paper u Select algorithm u See recommendations

31. 31 CNI 2005 Fall BriefingTechLens What We Found Results published in McNee et al. (CSCW 2002): u Yes, we can make recommendations this way! è offline analysis showed that best algorithms could find half of recommendable withheld references in top 10, ¾ in top 40 recs è online experiments showed best algorithms gave recommendations more than half of which were relevant, and more than half of which were novel u Users like it! è more than half of users felt useful (1/4 to 1/3 said not) è 1-2 good recs out of 5 seemed sufficient for use u Different algorithms have different uses Further exploration in Torres et al. (JCDL 2004)

32. 32 CNI 2005 Fall BriefingTechLens Phase II Shifted our focus to ACM Digital Library Greater exploration of user tasks: u awareness services u keeping track of a community More automation u find own bibliography from citations u find collaborators Thinking about “researcher’s desktop”

33. 33 CNI 2005 Fall BriefingTechLens Demonstration #2 Steps: u identify self u see automated collections of citations and collaborators u show how to use collections for recommendation

34. 34 CNI 2005 Fall BriefingTechLens Moving Forward Collaboration u Computer Scientists (HCI, recommenders) u Librarians (field work, domain expertise, “real- life” service deployment) Research methods u Offline data gathering and feasibility studies u Online pilots and controlled experiments u Online field studies (including random- assignment studies)

35. 35 CNI 2005 Fall BriefingTechLens What’s Next? Short-Term Efforts u Task-specific recommendation u Understanding personal bibliographies u Privacy issues Longer-Term Efforts u Toolkits to support librarians and other power users u Exploring the shape of disciplines u Rights issues

36. 36 CNI 2005 Fall BriefingTechLens Task-Specific Recommendations Many different user needs u awareness in area of expertise u find specific work in area of expertise u explore peripheral or new area u find people with relevant expertise è reviewers, program committees, collaborators u reading list for students, newcomers è individuals or groups Different algorithms fulfill different needs

37. 37 CNI 2005 Fall BriefingTechLens

38. 38 CNI 2005 Fall BriefingTechLens Personal Bibliographies Working with RefWorks to explore bibliographies maintained by library users: u how resolvable is personally-managed bibliographic data? u where does data come from (import/type) and is there sufficient quality control? u depth and span of bibliographies u suitability for recommenders

39. 39 CNI 2005 Fall BriefingTechLens Privacy Issues Anything involving personal bibliographies, library usage is extremely sensitive u what can we do with minimal personal data (e.g., explicit queries)? è can we identify particularly sensitive cases? u can we de-personalize data for collaborative applications? u for what benefits will users give informed consent to use private data? u feasibility/efficacy of ratings in library domain

40. 40 CNI 2005 Fall BriefingTechLens The Toolkit What would it take to support complex requests? u Help me assemble a collection of the 20 papers in molecular biology that have been most influential in other sciences u Help me assemble a committee of leading humanists who together span a collection of fields and have collaborated with most of the leaders of those fields A new dimension of service for expert librarian

41. 41 CNI 2005 Fall BriefingTechLens Describe a Discipline Can we build automated tools to: u identify the most important conferences and journals for a field? u identify the most important papers? è seminal work from other fields è seminal work that established this field è new work of particular influence u identify trends in topic? u identify hubs of activity?

42. 42 CNI 2005 Fall BriefingTechLens Rights Issues Not our core expertise, but … u rights issues are critical, particularly è use of metadata, including abstracts è possible future use of reviews u also important to understand and educate authors on future uses of their work è everything from rating systems to plagiarism detection

43. 43 CNI 2005 Fall BriefingTechLens Discussion Issues of your choice, or: u privacy issues – are these a show-stopper? u will these tools change the nature of scholarship? is it already changing? è can I cite each member of the program committee? u what will it take to demonstrate the value of such tools? u pragmatic issues of interoperability

44. 44 CNI 2005 Fall BriefingTechLens Our Thanks GroupLens Research Group U of M Libraries NEC Research, ACM, RefWorks NSF Grants: DGE 95-54517, IIS 96-13960, IIS 97-34442, IIS 99-78717, and IIS 01-02229 (and we hope more to come!) All the colleagues who’ve given us feedback along the way Our research subjects/users

45. 45 CNI 2005 Fall BriefingTechLens TechLens: Exploring the Use of Recommenders to Support Users of Digital Libraries Joseph A. Konstan, Nishikant Kapoor, Sean M. McNee, John T. Butler GroupLens Research Project and University Libraries University of Minnesota