1. Professor Horst Cerjak, 19.12.2005 1 Knowledge Management Institute Mark Kröll WSDM’09 Workshop on Web Search Click Data, Barcelona, Spain Intentional Query Suggestion: Making User Goals More Explicit During Search Markus Strohmaier, Mark Kröll and Christian Körner WSCD‘09: Workshop on Web Search Click Data @ WSDM 2009 Barcelona, Spain Mark Kröll mkroell@tugraz.at Graz University of Technology, Austria

2. Professor Horst Cerjak, 19.12.2005 2 Knowledge Management Institute Mark Kröll WSDM’09 Workshop on Web Search Click Data, Barcelona, Spain Related Work  on Search Intent  Understanding goals in web search, [Broder02, Rose/Levinson04].  The Intention Behind Web Queries, [BaezaYates06].  Understanding the Relationship between Searchers’ Queries and Information Goals, [Downey08].  on Query Suggestion  Query expansion using local and global document analysis, [Xu96].  Generating query substitutions, [Jones06].  Learning latent semantic relations from clickthrough data for query suggestions, [Ma08]. Observation: There is little work that combines these two areas.  Goal Oriented Search Engine, [Liu02].  Effects of Goal-Oriented Search Suggestions, [Mostert08].

3. Professor Horst Cerjak, 19.12.2005 3 Knowledge Management Institute Mark Kröll WSDM’09 Workshop on Web Search Click Data, Barcelona, Spain Intentional Query Suggestion  Query Suggestion based on User Intent  suggesting queries that represent potential user intentions Initial QueryTraditional Query Suggestion(MSN + Yahoo) Intentional Query Suggestion pokerfree online poker, free poker games, cheating at poker, learn to play poker, househouses for sale, Hugh Laurie insure my house, build my own house Definition of explicit intentional queries (suggested queries):  contain at least one verb and  describe a plausible state of affairs that the user may want to achieve or avoid (cf.) in  a recognizable way

4. Professor Horst Cerjak, 19.12.2005 4 Knowledge Management Institute Mark Kröll WSDM’09 Workshop on Web Search Click Data, Barcelona, Spain Research Overview  How could query suggestion based on user intent be realized?  How would queries expanded by user intent influence  Click-Through?  Search results?

5. Professor Horst Cerjak, 19.12.2005 5 Knowledge Management Institute Mark Kröll WSDM’09 Workshop on Web Search Click Data, Barcelona, Spain Algorithm for Intentional Query Suggestion  by creating a mapping between implicit intentional queries(short) and explicit intentional queries (long) Output: list of suggestions Example:  design your own poker chips  learn to play home poker  cheat at poker  buy poker chips Input: Example:  poker Potential Intentions

6. Professor Horst Cerjak, 19.12.2005 6 Knowledge Management Institute Mark Kröll WSDM’09 Workshop on Web Search Click Data, Barcelona, Spain Text-based Intentional Query Suggestion  dataset containing explicit intentional queries  extracted from MSN log employing the algorithm from [Strohmaier08a]  implicit intentional queries are textually compared to all explicit intentional queries  used Jaccard Similarity Measure [BaezaYates99]

7. Professor Horst Cerjak, 19.12.2005 7 Knowledge Management Institute Mark Kröll WSDM’09 Workshop on Web Search Click Data, Barcelona, Spain Neighborhood – based Intentional Query Suggestion  based on query log data we construct a bipartite graph, where TypeQueryDate q u,1 types of diet pills2006-05-24 13:34:16 q u,2 Lipo62006-05-24 13:36:24 q u,3 lose 20 pounds in 8 weeks2006-05-24 13:37:23 q e,1 lose weight fast2006-05-24 13:38:42 q u,4 lose weight fast2006-05-24 13:39:06 q u,5 weight loss upplements2006-05-24 13:39:51 q u,6 weight loss supplements2006-05-24 13:39:56 use neighboring queries to further describe and characterize explicit intentional queries  nodes of one type correspond to explicit intentional queries and  nodes of the second type correspond to implicit intentional queries containing goals not containing goals

8. Professor Horst Cerjak, 19.12.2005 8 Knowledge Management Institute Mark Kröll WSDM’09 Workshop on Web Search Click Data, Barcelona, Spain Parametric model  Distance P d  size of neighborhood N P d =3 TypeQueryDate q u,1 types of diet pills2006-05-24 13:34:16 q u,2 Lipo62006-05-24 13:36:24 q u,3 lose 20 pounds in 8 weeks2006-05-24 13:37:23 q e,1 lose weight fast2006-05-24 13:38:42 q u,4 lose weight fast2006-05-24 13:39:06 q u,5 weight loss upplements2006-05-24 13:39:51 q u,6 weight loss supplements2006-05-24 13:39:56 Excerpt of the corresponding Bipartite Graph P d =3 T(q e,1 ) q e,1  T(q e,1 ) = {“weight”, “loss”, “supplement”, “upplements”}

9. Professor Horst Cerjak, 19.12.2005 9 Knowledge Management Institute Mark Kröll WSDM’09 Workshop on Web Search Click Data, Barcelona, Spain Experimental Demonstrator [joint work with student Ferdinand Wörister]

10. Professor Horst Cerjak, 19.12.2005 10 Knowledge Management Institute Mark Kröll WSDM’09 Workshop on Web Search Click Data, Barcelona, Spain Preliminary Evaluation of Algorithm  by conducting a human subject study  categorize the 10 top-ranked suggestions for 30 queries  Two relevance classes:  (i) potential user intention  (ii) clear misinterpretation Initial QueryIntentional Query Suggestions “ anime ”“ draw anime”, “draw manga ” “ playground mat ”“ buy playground equipment”, “build a swing set ” Initial QueryIntentional Query Suggestions “ Boston herald ”“ care for Boston fern”, “flying to Nantucket ” “ playground mat ”“ raise money for our playground” Average Precision: 0.71 Average Interrater Agreement Kappa: 0.6

11. Professor Horst Cerjak, 19.12.2005 11 Knowledge Management Institute Mark Kröll WSDM’09 Workshop on Web Search Click Data, Barcelona, Spain Step Back  presented one approach and experimental demonstrator to realize query suggestion based on user intent  reasonable precision of 71% concerning quality of suggested queries  How would queries expanded by user intent influence click through and search results?

12. Professor Horst Cerjak, 19.12.2005 12 Knowledge Management Institute Mark Kröll WSDM’09 Workshop on Web Search Click Data, Barcelona, Spain Influence on Search Results(1)  Result Set Intersection between different Query Suggestion Mechanisms:  How many URLs intersect between URL result sets? Compared URL result setsAvg. Intersection Original Queries vs. Traditional Suggestions0.1911 Original Queries vs. Intentional Suggestions0.0467 Traditional Suggestions vs. Intentional Suggestions0.0511 10 Traditional Suggestions Original Query 10 Intentional Suggestions 50 500 URL result sets (unique, top-level)

13. Professor Horst Cerjak, 19.12.2005 13 Knowledge Management Institute Mark Kröll WSDM’09 Workshop on Web Search Click Data, Barcelona, Spain Influence on Search Results(2)  Result Set Intersection within the same Query Suggestion Mechanism:  How many URLs intersect between result sets that were retrieved by the same query suggestion mechanism regarding one original query? Compared URL result setsAverage Intersection Traditional Suggestions0.103 Intentional Suggestions0.026  Results suggest that queries that express a specific intention lead to more different results than traditional query suggestions 50 10 Traditional Suggestions 10 Intentional Suggestions 50... 50...

14. Professor Horst Cerjak, 19.12.2005 14 Knowledge Management Institute Mark Kröll WSDM’09 Workshop on Web Search Click Data, Barcelona, Spain Influence on Click-Through  Do users click more frequently on suggested queries if they are explicit intentional?  Experimental Setup:  click-through events for different query lengths  created different bin sizes – 5000 queries randomly drawn from MSN log  corresponding click-through events were registered and counted Implicit Intentional QueriesExplicit Intent. Queries Query Length 123-456-10>105.33 #click- through 855,649358,32764,3135,5592,7289607,236  Results suggest that queries that express a specific intention retrieve more relevant results than implicit intentional queries of the same length

15. Professor Horst Cerjak, 19.12.2005 15 Knowledge Management Institute Mark Kröll WSDM’09 Workshop on Web Search Click Data, Barcelona, Spain Conclusions  combining Search Intent + Query Suggestion  using high level search intent [Mostert08]  employing search intent on a more detailed level appears to be a natural next step  impact on search results and behavior (preliminary experiments) results suggest higher click through for explicit intentional queries results suggest more diverse results for explicit intentional queries

16. Professor Horst Cerjak, 19.12.2005 16 Knowledge Management Institute Mark Kröll WSDM’09 Workshop on Web Search Click Data, Barcelona, Spain References [BaezaYates99] Baeza-Yates R. and Ribeiro-Neto B. Modern Information Retrieval, AddisonWesley, 1999 [BaezaYates06]Baeza-Yates, R., Calderón-Benavides, L. and González-Caro, C. 2006. The Intention Behind Web Queries, in F. Crestani, P. Ferragina and M. Sanderson, ed.,Proceedings of String Processing and Information Retrieval (SPIRE ), Springer, 98-109. [Bendersky09] Bendersky, M. and Croft, W. B., "Analysis of Long Queries in a Large Scale Search Log," Workshop on Web Search Click Data (WSCD 2009) Barcelona, Spain, February 9, 2009. [Broder02] Broder A. A taxonomy of web search. In ACM SIGIR Forum 36(2), pp. 3--10, 2002. [Downey08] Downey, D.; Dumais, S.; Liebling, D. & Horvitz, E. (2008), Understanding the relationship between searchers' queries and information goals, in 'CIKM '08: Proceeding of the 17th ACM conference on Information and knowledge mining', ACM, New York, NY, USA, pp. 449--458. [Dumais98] Dumais S., Platt J., Heckerman D., and Sahami M. "Inductive learning algorithms and representations for text categorization". In: Proceedings International Conference on Information and Knowledge Management, New York, NY, USA, ACM Press, pp 148-155, 1998 [Jansen08]Jansen, B. J., Booth, D. L. and Spink, A. Determining the informational, navigational, and transactional intent of web queries. In Inf. Process. Manage. 44(3), pp. 1251--1266, 2008. [Jones06]Jones, R.; Rey, B.; Madani, O. & Greiner, W. (2006), Generating query substitutions, in 'WWW '06: Proceedings of the 15th international conference on World Wide Web', ACM, New York, NY, USA, pp. 387--396. [He07]K.Y. He and Y.S. Chang and W.H. Lu. Improving Identification of Latent User Goals through Search-Result Snippet Classification. WI '07: Proceedings of the 2007 IEEE/WIC/ACM International Conference on Web Intelligence, 683-686, IEEE Computer Society,2007. [Ma08]Ma, H.; Yang, H.; King, I. & Lyu, M. R. (2008), Learning latent semantic relations from clickthrough data for query suggestion, in 'CIKM '08: Proceeding of the 17th ACM conference on Information and knowledge mining', ACM, New York, NY, USA, pp. 709--718. [Mostert08] Mostert, J. & Hollink, V. (2008), Effects of Goal-Oriented Search Suggestions, in 'Proceedings of the 20th Belgian-Netherlands Conference on Artificial Intelligence'. [Liu and Lieberman02] Liu H., Lieberman H. and Selker T.. GOOSE: A Goal-Oriented Search Engine with Commonsense. AH '02: Proceedings of the Second International Conference on Adaptive Hypermedia and Adaptive Web-Based Systems, 253--263, Springer-Verlag,London, UK,2002. [Rose04] Rose D. E. and Levinson D., Understanding user goals in web search. In Proc. of WWW 2004, May 17-22, 2004, New York, USA, 2004. [Strohmaier08a] Strohmaier, M., Prettenhofer, P. and Kröll, M. Acquiring explicit user goals from search query logs. In 'International Workshop on Agents and Data Mining Interaction ADMI‘ 08, in conjunction with WI '08', 2008. [Strohmaier08b] M. Strohmaier, P. Prettenhofer, M. Lux, Different Degrees of Explicitness in Intentional Artifacts - Studying User Goals in a Large Search Query Log, CSKGOI'08 International Workshop on Commonsense Knowledge and Goal Oriented Interfaces, in conjunction with IUI'08, Canary Islands, Spain, 2008. [Xu96]Xu, J. & Croft, W. B. (1996), Query expansion using local and global document analysis, in 'SIGIR '96: Proceedings of the 19th annual international ACM SIGIR conference on Research and development in information retrieval', ACM, New York, NY, USA, pp. 4--11.

17. Professor Horst Cerjak, 19.12.2005 17 Knowledge Management Institute Mark Kröll WSDM’09 Workshop on Web Search Click Data, Barcelona, Spain End of Presentation Mark Kröll Graz University of Technology, Austria mkroell@tugraz.at Thanks for your attention!

18. Professor Horst Cerjak, 19.12.2005 18 Knowledge Management Institute Mark Kröll WSDM’09 Workshop on Web Search Click Data, Barcelona, Spain Questions and Discussion