ZIYANG LIU, Peng Sun, Yi Chen Arizona State University S TRUCTURED Q UERY R ESULT D IFFERENTIATION

K EYWORD S EARCH ON S TRUCTURED D ATA  Effective techniques have been developed to help users find relevant results?  Ranking: sort the results in the order of estimated relevance  Snippet: provide a summary of each result to help users judge relevance  50% of keyword searches are information exploration queries, which inherently have multiple relevant results  Users intend to investigate and compare multiple relevant results.  How to help user compare relevant results? Keywords Search Engine Results: Relevant Data Fragments Structured Data Web Search 50% Navigation 50% Information Exploration Broder, SIGIR 02 2

R ESULTS AND S NIPPETS store city Phoenix name BHPhoto merchandises category DSLR camera brand Canon megapixel 12 category DSLR camera brand Sony megapixel 12 …… store city Phoenix name Adorama merchandises category Compact camera brand HP megapixel 14 category Compact camera brand Canon megapixel 12 …… “Phoenix, camera, store” store city name BHPhoto merchandises brand Canon camera megapixel 12 brand Canon camera Phoenix Snippet store city Phoenix name Adorama merchandises category Compact camera brand Canon megapixel 12 Snippet Snippets are unhelpful in differentiating query results. 3 (Huang et al. SIGMOD 09)

D IFFERENTIATION F EATURE S ETS (DFS) store city Phoenix name BHPhoto merchandises category DSLR camera brand Canon megapixel 12 category DSLR camera brand Sony megapixel 12 …… store city Phoenix name Adorama merchandises category Compact camera brand HP megapixel 14 category Compact camera brand Canon megapixel 12 …… DFS Feature Typevalue store: nameBHPhoto camera: brandCanon Sony camera: categoryDSLR Feature Typevalue store: nameAdorama camera: brandCanon HP camera: categoryCompact Feature: (entity, attribute, value) 4 Bank websites usually allow users to compare selected credit cards, however, only with a pre-defined feature set.

C HALLENGES OF R ESULT D IFFERENTIATION  How to automatically generate DFS that highlight the differences among results?  How to measure the quality of a set of DFSs?  DFSs should obviously maximize the difference among results. How to quantify it?  What are other desirable properties?  Can DFSs be efficiently generated from results? 5

C ONTRIBUTIONS  1 st work on automatically differentiating structured search results  Application domains: online shopping, employee hiring, job/institution hunting, etc.  Identifying 3 desiderata for good DFSs  Quantifying the differentiation power of a set of DFSs  Proving the NP-hardness of DFS generation  Tackling the problem using two local optimality criteria  Single-swap / Multi-swap optimality  Implemented XRed: X ML Re sult D ifferentiation  Empirically verified the effectiveness & efficiency of XRed 6

R OADMAP  Desiderata for good DFSs  Problem definition  Local optimality and algorithms  Experiments 7

D ESIDERATUM 1 B EING S MALL  A Small DFS is easy for user to go through and compare with other DFSs.  The size of each DFS, |D|, cannot exceed a user- specified upper bound L |D| ≤ L 8

D ESIDERATUM 2 S UMMARIZING Q UERY R ESULTS DFSs that do not summarize the results show useless & misleading differences. store city Phoenix name BHPhoto merchandises category DSLR camera brand Canon megapixel 12 category DSLR camera brand Sony megapixel 12 …… store city Phoenix name Adorama merchandises category Compact camera brand HP megapixel 14 category Compact camera brand Canon megapixel 12 …… DFS Feature TypeDFS camera:brandHP Feature TypeDFS camera:brandCanon This store sells only a few HP cameras. 9

D ESIDERATUM 2 S UMMARIZING Q UERY R ESULTS DFSs that do not summarize the results show useless & misleading differences. store city Phoenix name BHPhoto merchandises category DSLR camera brand Canon megapixel 12 category DSLR camera brand Sony megapixel 12 …… store city Phoenix name Adorama merchandises category Compact camera brand HP megapixel 14 category Compact camera brand Canon megapixel 12 …… DFS Feature TypeDFS camera:brandCanon camera:brandHP Feature TypeDFS camera:brandCanon camera:brandHP This store sells only a few HP cameras. 10

 A DFS is valid only if it summarizes the corresponding result.  Features of the same type should be included in order of occurrences.  Ratios of two features in the DFS should be roughly the same as in the result. D ESIDERATUM 2 S UMMARIZING Q UERY R ESULTS Dominance Ordered Distribution Preserved 11

D ESIDERATUM 3 D IFFERENTIATING Q UERY R ESULTS  Differentiation unit: feature type.  A feature type t in two DFSs D 1 and D 2 is differentiable if  The order of the features of type t is different.  The ratio of two features of type t is different. D 1. Camera: brand: Canon D 2. Camera: brand: HP D 1. Camera: brand: Canon D 2. Camera: brand: Canon Camera: brand: HP D 1. Camera: brand: Canon Camera: brand: HP D 2. Camera: brand: Canon Camera: brand: Canon Camera: brand: HP 12

Degree of Differentiation (DoD) of two DFSs = Number of differentiable feature types. D ESIDERATUM 3 D IFFERENTIATING Q UERY R ESULTS Feature TypeDFS store:nameBHPhoto camera:brandCanon Sony camera:categoryDSLR Feature TypeDFS store:nameAdorama camera:brandCanon HP camera:categoryCompact DoD = 3 DoD of multiple DFSs = the sum of DoD of every pair of DFS. 13

R OADMAP  Desiderata for good DFSs  Problem definition  Local optimality and algorithms  Experiments 14

DFS G ENERATION P ROBLEM  Given a set of results and a size limit L, generate a DFS for each result such that  Their DoD is maximized.  Every DFS is valid (good summary)  Every DFS’s size does not exceed L.  We proved the NP-hardness of this problem by reduction from X3C. 15

R OADMAP  Desiderata for good DFSs  Problem definition  Local optimality and algorithms  Experiments 16

L OCAL O PTIMALITY  To tackle this hard problem, instead of achieving global optimality, we propose two local optimality criteria :  Single-swap Optimality  Multi-swap Optimality 17

S INGLE S WAP  A set of DFSs is Single-Swap Optimal, if adding / changing a single feature in a single DFS (subject to validity and size limit) cannot increase the DoD. Feature TypeValue store: nameBHPhoto store: cityPhoenix camera: megapixel12 camera: categoryDSLR Feature TypeValue store: nameAdorama camera: brandCanon HP camera: megapixel12 DoD = 1 Feature TypeValue store: nameAdorama camera: brandCanon HP camera: categoryCompact DoD increases to 2 # of cameras: 200 Category: DSLR: 188 Others: 12 Brand: Canon: 103 Sony: 50 Nikon: 25 HP: 22 Megapixel: 12: : 15 14: 20 STORE 1 # of cameras: 150 Category: Compact: 140 Others: 10 Brand: Canon: 80 HP: 70 Megapixel: 12: : 5 14: 19 STORE 2 Achieved Single-Swap Optimal 18

A LGORITHM FOR S INGLE -S WAP O PTIMALITY  Start from a randomly generated DFS for each result.  Repeatedly add a feature / change a feature in a DFS.  Stop until the DoD no longer increases. Does this algorithm terminate in polynomial time? Yes: The maximum possible DoD for a set of DFSs is POLYNOMIAL. Each iteration increases the DoD at least by 1. Each iteration takes polynomial time. 19

M ULTI -S WAP O PTIMALITY  A set of DFSs is Multi-Swap Optimal, if adding / changing any number of features in a single DFS (subject to validity and size limit) cannot increase the DoD. Feature TypeValue store:nameBHPhoto store:cityPhoenix camera: megapixel12 camera:categoryDSLR Feature TypeValue store:nameAdorama camera: brandCanon HP camera:categoryCompact DoD = 2 Feature TypeValue store:nameBHPhoto camera:brandCanon Sony camera:categoryDSLR DoD increases to 3 20 # of cameras: 200 Category: DSLR: 188 Others: 12 Brand: Canon: 103 Sony: 50 Nikon: 25 HP: 22 Megapixel: 12: : 15 14: 20 STORE 1 # of cameras: 150 Category: Compact: 140 Others: 10 Brand: Canon: 80 HP: 70 Megapixel: 12: : 5 14: 19 STORE 2

A LGORITHM FOR M ULTI -S WAP O PTIMALITY  Start from a randomly generated DFS for each result.  Repeatedly add / change multiple features in a DFS.  Stop until the DoD no longer increases. We designed a novel dynamic programming algorithm, which takes pseudo-polynomial time 21 This algorithm has exponential time complexity!

E VALUATION  We have implemented Xred ( X ML Re sult D ifferentiation) and evaluated it empirically.  Data sets  Film (  Camera Retailer (synthetic)  Result generation: XSeek (  DFS size limit: 10% of # of feature types  Metrics:  Quality (DoD)  Efficiency  Comparison system: exponential algorithm that generates optimal solution. 22

DFS Q UALITY 23 Film Camera Retailer

E FFICIENCY 24 Result Size 1KB ~ 9KB # of Results 2 ~ 52 Film Camera Retailer

S CALABILITY 25

C ONCLUSIONS  We initiate the problem of automatically differentiating structured query results, which is useful for information exploration queries.  We define Differentiation Feature Set (DFS) for each result, and identify three desiderata for DFS.  We formalize the DFS generation problem, and prove its NP- hardness.  We propose two local optimality criteria: single-swap and multi-swap, and design algorithms to efficiently achieve them.  We implemented the XRed system, and verified its effectiveness and efficiency through experiments. 26

F UTURE W ORK Result differentiation is a new area and opens opportunities for new research topics.  Is there a better way of selecting feature types, e.g., by considering users’ interests?  Is there a better way of measuring the quality of DFSs besides DoD?  Are there approximation / randomized algorithms for DFS generation problem that achieve better quality / efficiency? 27

28