PaperLens Understanding Research Trends in Conferences using PaperLens Work by Bongshin Lee, Mary Czerwinski, George Robertson, and Benjamin Bederson Presented by Jerry Alan Fails

Outline Problem Definition PaperLens Overview Previous Work Demo User Study Questions

O(1)O(10) O(1) O( ) Problem O(100) PaperLens

Data Sets InfoVis 155 Papers 315 Authors 8 Years of Data 1995 – 2002 Attributes — author name(s), paper title, year of publication, list of references, links to original papers CHI 4073 Papers 6309 Authors 23 Years of Data 1982 – 2004 Had some missing reference data Did some cleanup to identify paper source, year of publication, title, and authors Correlations Year Topic Papers/Authors Co-authors Cited Papers/Authors

Interesting Questions Which topics have come and gone over the last 23 years in CHI? Which topics are currently hot in CHI? Which papers/authors are most frequently referenced by CHI publications? How has the most frequently referenced papers/authors changed over time? What is the relationship between researchers?

Digital Libraries ACM Digital Libraries Additional Info – bibliographic data, references, citings, index terms, collaborative colleagues, peer-to-peer Tools – discussions, similar articles, review article, save CiteSeer Bibliographic data, cited by, active bibliography, similar documents, related by co-citation Histogram showing distribution of citations IEEE Explore HCI Bibliography

Galaxy

ThemeView

Envision Query Window Graphic View Window Item Summary Window Query History

What is PaperLens?

Data Sets InfoVis 155 Papers 315 Authors 8 Years of Data 1995 – 2002 Attributes — author name(s), paper title, year of publication, list of references, links to original papers CHI 4073 Papers 6309 Authors 23 Years of Data 1982 – 2004 Had some missing reference data Did some cleanup to identify paper source, year of publication, title, and authors Correlations Year Topic Papers/Authors Co-authors Cited Papers/Authors Implications Viewing a graph!!! Tight Coupling Data & UI Different Views

Clustering Used standard clustering Criteria Titles Weighted more for InfoVis clusters References Keywords Minus Stop words Months of the year Journal and proceeding titles Version and page numbers Final Clusters InfoVis – 5 CHI – 15 Joined to create 15 from 22 Table Clusters of CHI papers by topic. ClustersNumber of Papers Lab Reports, Applications, Web618 Multimodal UI601 CSCW512 Miscellaneous407 Usability334 InfoVis323 Cognitive Factors in Design241 Anthropomorphism209 End User Programming160 Target Acquisition140 User Modeling139 Audio, Tangible UI130 User Centered Design119 VR, Input Devices75 UIMS65

InfoVis PaperLens

CHI PaperLens

User Study 8 Users (including 1 pilot) 4 computer science grad students 4 full-time researchers in HCI Age ranges Study 20 minute tutorial, with think-aloud observations 16 tasks completed sequentially

Tasks 1. Who published the only paper on Graph Visualization in 1998? 2. How many papers did S. K. Card publish at InfoVis over the 8 years in our database? 3. Who were George Robertson’s coauthors on his only paper in the database? 4. How many degrees of separation exist between S. F. Roth and S. G. Eick? 5. Which topic area has enjoyed gradual growth over the last 8 years? 6. Which topic area has all but died out in terms of papers published on that topic over the last 8 years? 7. Which topic area has had many more papers published on that topic during the last 2 years in our database? 8. Which authors are in the top 10 most frequently cited list but have not published at InfoVis? 9. How many papers of the top 10 most frequently cited papers are from InfoVis? 10. How many papers in the top 10 most frequently cited list are from CHI? 11. Which topic area references the most frequently cited paper most often? 12. Go to the most frequently cited InfoVis paper and read it’s abstract. 13. In the Dynamic Queries topic area, which author is the most frequently cited? 14. What was the last year that S. K. Card published in this database? 15. Who was the most frequently cited author in 2001? 16. How many papers did J. Mackinlay and S. K. Card publish together at InfoVis over the 8 years in our database?

Lessons Learned Simple is Good An abstract overview Multiple small, simple components showing different aspects of the data Relationships shown through interactivity and tightly coupled components Visual elements laid out along axes with well defined metrics Scaling Issues Info Vis Square paper representation Fisheye technique to reveal individual paper details Too many colors (max was 8 colors, indicating 8 co-authors) CHI Context menu for details since papers less than one pixel in size Eliminate fisheye technique Papers with >5 authors color-coded with a single color

User Satisfaction QuestionRating Overall, I am satisfied with this system.5.3 It was simple to use this system.4.3 I was able to efficiently complete the tasks and scenarios using this system.6.0 I felt comfortable using this system.5.3 It was easy to learn to use this system.4.4 The "look" of this system was pleasant.4.9 I liked using this system.5.9 The organization of information in this system was clear.3.9 Whenever I made a mistake using this system, I could recover quickly and easily.4.6 It was easy to discover trends of the topics using the "Popularity of Topic" view.6.3 It was easy to discover relationships between authors using the "Degrees of Separation Links" view.4.1 It was easy to discover the most referenced papers/authors using the "Year by Year Top 10 Cited Papers/Authors" view. 6.9 Overall, highlighting on the screen was helpful.6.1 Overall, the use of color was appropriate.5.3 Table 3. Average Likert scale ratings for PaperLens, using the scale of 1=Disagree, 7=Agree.

Questions How does PaperLens allow visualization of a graph, without showing a node-link diagram? How generalizable is this work? What design principles may transfer? Can a general architecture from this UI?