1. Antara Ghosh Jignashu Parikh
QUERY CLUSTERING
Antara Ghosh
Jignashu Parikh

2. OVERVIEW PROBLEM DEFINITION MOTIVATION ISSUES OUR APPROACH
Notion of Similarity
Algorithm for Computing Similarity
Clustering
Classification
Experimental Results
Future Work
Conclusion

3. PROBLEM DEFINITION
Define Query Space and find clusters of “similar” queries.
Build a Framework for
Finding Similarity
Clustering Queries
Classification of new queries

4. MOTIVATION Complex Queries are common place
Plan Space grows exponentially with number of joins and relations involved
Optimisation time to the range of 8-15 secs reported by Roy et. al.
Query Space considerably smaller than Plan Space
Short Circuit the optimiser!!

5. ISSUES Establishing the Notion of Similarity
Dealing with “different similarities”
Clustering Queries based on Similarity
Huge Query Space- Infinite?
How many Clusters are there?
Classification
Searching through large number of Clusters

6. Notion of Similarity
“Syntactico-semantic” similarity given that table sizes are comparable in terms of sizes and estimated result size.
Checking syntactico-semantic similarity
Table access through index or not
Types of operations each table is involved in, equality non-equality
No of predicates corresponding tables are involved.
Mapping compatible tables with respect to original and estimated result size.

7. Notion of similarity (contd.)
The select and join predicate of the query must be compatible.
Total projected attribute size
A special check for each table if all the attributes accessed in it is indexed.

8. Definitions E A C D B F 6 1 4 5 2 3
Degree of a table: the number of the predicates the table is involved
Index type counts of a table: the number of 2-way and 1-way indices on each predicate of this table
Operation type count: the number of equi-joins and non-equi-joins the table is involved in.
Index flag: this flag is set when all the selection and prediction predicates of a table is on indexed attributes
Validity of mapping: map table only if they have same value for all the variables above

9. Algorithm If number of tables in two queries same
1. Match semantics at query level
Check if the total degree, index count, operation type count, and index flags are same over all the tables.
2 For all valid mapping among the tables
Find the distance of tables among the mapped table based
on the formula:
distance=0.7*(absolute(s1-s2)/max(s1,s2))
+0.3*(absolute(so1-so2)/max(so1,so2))
where si=original size of table i
= original cardinality* original rowsize
soi=estimated result size of table I
=rf * original cardinality * projected row size
Find the mapping with minimal sum of such distances
If it is less than magic threshold number then the queries are called similar, otherwise not

10. Time complexity
The mapping reduces from all possible combination of table joins and type of joins in a query, to mapping of tables among two queries.
Table mapping is done at the end of algorithm, when the queries are syntactico-semantically compatible.
Tables are partitioned according to their degrees. Mapping is done among the tables of same partition.
Time complexity = p1!+p2!+p3!+……+pk!
where p1+p2 +..+pk=N and N=no of tables
Where as for all such mapping the complexity will become of order N!

11. Similar-looking queries different execution plans
select resume
from emp_resume
where empno>'000010'
select empno
from emp_resume
where empno>'000010'

12. Different looking Queries-Similar Plans!
select * from employee as a,
emp_act as b, emp_photo as c
where a.empno=b.empno and
b.empno=c.empno and
a.empno>'000000' and
b.empno<'000400‘ and c.empno
between '000010' and '000390'
select a.firstnme, a.lastname ,b.projno, c.resume
from employee as a, emp_act as b, emp_resume as c
where a.empno=b.empno and b.empno=c.empno

13. Clustering Queries
Given a set of Queries (Usually Large) generate clusters efficiently
Cannot afford several passes of the database of queries.
A single pass algorithm, similar to the leader algorithm (Hartinger 1975) is proposed.
Even BIRCH can be used

14. Algorithm For Clustering
Inputs: A set of Queries Q1 to Qn and a Threshold T
Output: K-clusters
Algorithm
Start with Q1 and make a cluster C1 with Q1 as a representative.
Let k = No. of classes
Ri = Representative of Cluster Ci
For i = 2 to n do
If Qi is similar to any one of R1 to Rk, say Rj, then
add Qi to Cluster Cj.
Else if there is no such cluster then
add produce one more cluster Ck+1 with representative
Rk= Qi.
End if
End.

15. Classification Host of classification methods can be applied
Decision Tree Based Classification
Why?
Most of the features are deterministic- Rule based system suits well
Storage Minimal- No need to store the clusters once we have the decision trees
C5.0 Decision Tree Induction Program used
Uses Training data for generating decision rules
Decision rules split the Query Space
Chooses the hierarchy of rules based on decreasing Information gain from top to bottom

16. GENERATING DECISION TREES
Set of 400 Queries Used for Training + Test for 6 clusters
50 Attributes/Features Per Query
Clusters were as under
1 (82) and 2 (72): Single Relation Queries with same semantics but different Size Ratios
3 (30): Two-Relation/Single Join Queries
4 (107) and 5 (73): Three Relation/Double Join Query Clusters differing in semantics
6 (35): Four-Relation/Triple Join Queries
Leave-one Out Cross Validation Performed

17. C5.0: Results Classification Accuracy: Time Taken
Training Set: 94.7%
Mean Leave-One Out Accuracy: 88.2%
Time Taken
Induction+Classification
Training Set (1+400): 0.1 sec » sec/q
Leave One Out ( ): 9.3 seconds
We can surely beat the query optimiser!
Most of the errors in class 1 and 2- The difference is much more fuzzy there.

18. Future work Sub-query matching
Size based mapping-calibrating weights and thresholds
Dealing with more sophisticated queries
Extensive testing for clustering and classification

19. Conclusions Proposed framework is completely scalable.
Concept of similarity can be applied to any number of relations
Time Complexity of the algorithm has been tuned for dealing with queries involving large number of relations.
Clustering and classifications schemes selected are meant for dealing with large data.
Metric of Similarity proposed is more general than using DB2 for clustering.
Query space may be smaller than Plan space but still we are dealing with “Infinity”
Thorough testing using a large real-life query set should be used to test the mettle of the algorithm.
if the query space being infinite is a dark cloud, the silver lining is- If the optimiser can handle the Plan space, we should be able to deal with the query space!