1. Advanced Database Systems
Sekolah Tinggi Ilmu Statistik (STIS)
Advanced Database Systems

2. Dr. Said Mirza Pahlevi, M.Eng.
Query Processing 1
Lecture 7
Dr. Said Mirza Pahlevi, M.Eng.

3. Today's Lecture Introduction to query processing
Phases of Query Processing
Query decomposition
Heuristical Approach to Query Optimization
Dr. Said Mirza Pahlevi, M.Eng.

4. Introduction
First Topic
Dr. Said Mirza Pahlevi, M.Eng.

5. Introduction
In network and hierarchical DBMSs, low-level procedural query language is generally embedded in high-level programming language.
Programmer’s responsibility to select most appropriate execution strategy.
With declarative languages such as SQL, user specifies what data is required rather than how it is to be retrieved.
Relieves user of knowing what constitutes good execution strategy.
Dr. Said Mirza Pahlevi, M.Eng.

6. Introduction Also gives DBMS more control over system performance.
Two main techniques for query optimization:
Comparing different strategies based on relative costs, and selecting one that minimizes resource usage.
Heuristic rules that order operations in a query;
Disk access tends to be dominant cost in query processing for centralized DBMS.
Dr. Said Mirza Pahlevi, M.Eng.

7. Query Processing
Activities involved in: parsing, validating, optimizing, and executing a query
Aims of QP:
Transform query written in high-level language (e.g. SQL), into correct and efficient execution strategy expressed in low-level language (implementing relational algebra/RA);
Execute strategy to retrieve required data.
Dr. Said Mirza Pahlevi, M.Eng.

8. Query Optimization
Activity of choosing an efficient execution strategy for processing query.
As there are many equivalent transformations of same high-level query, aim of QO is to choose one that minimizes resource usage.
Generally, reduce total execution time of query.
May also reduce response time of query.
Problem computationally intractable with large number of relations, so strategy adopted is reduced to finding near optimum solution.
Dr. Said Mirza Pahlevi, M.Eng.

9. Example: Different Strategies
Find all Managers who work at a London branch.
SELECT *
FROM Staff s, Branch b
WHERE s.branchNo = b.branchNo AND
(s.position = ‘Manager’ AND b.city = ‘London’);
Dr. Said Mirza Pahlevi, M.Eng.

10. Example: Different Strategies
Three equivalent RA queries are:
(position='Manager')  (city='London')  (Staff.branchNo=Branch.branchNo) (Staff X Branch)
(position='Manager')  (city='London')(Staff Staff.branchNo=Branch.branchNo Branch)
(position='Manager'(Staff)) Staff.branchNo=Branch.branchNo (city='London' (Branch))
SELECT * FROM Staff s, Branch b
WHERE s.branchNo = b.branchNo AND
(s.position = ‘Manager’ AND b.city = ‘London’);
Dr. Said Mirza Pahlevi, M.Eng.

11. Example: Different Strategies
Assume:
1000 tuples in Staff; 50 tuples in Branch;
50 Managers; 5 London branches;
no indexes or sort keys;
results of any intermediate operations stored on disk;
cost of the final write is ignored;
tuples are accessed one at a time.
Dr. Said Mirza Pahlevi, M.Eng.

12. Disk Access Cost
(position='Manager')  (city='London')  (Staff.branchNo=Branch.branchNo) (Staff X Branch)
Cost: ( ) + 2*(1000 * 50) =
(position='Manager')  (city='London')(Staff Staff.branchNo=Branch.branchNo Branch)
Cost: 2* ( ) = 3050
(position='Manager'(Staff)) Staff.branchNo=Branch.branchNo (city='London' (Branch))
Cost: * *5 = 1160
Cartesian product & join operations are much more expensive than selection, and third option significantly reduces size of relations being joined together.
1000 tuples in Staff; 50 tuples in Branch;
50 Managers; 5 London branches;
Dr. Said Mirza Pahlevi, M.Eng.

13. Phases of Query Processing
Second Topic
Dr. Said Mirza Pahlevi, M.Eng.

14. Phases of Query Processing
QP has four main phases:
Decomposition (consisting of parsing and validation);
Optimization;
Code generation;
Execution.
Dr. Said Mirza Pahlevi, M.Eng.

15. Phases of Query Processing
Dr. Said Mirza Pahlevi, M.Eng.

16. Dynamic versus Static Optimization
The first three phases of QP can be carried out:
Dynamically every time query is run;
Statically when query is first submitted.
Advantages of dynamic QO arise from fact that information is up to date.
Disadvantages are that performance of query is affected, time may limit finding optimum strategy.
Dr. Said Mirza Pahlevi, M.Eng.

17. Dynamic versus Static Optimization
Advantages of static QO are removal of runtime overhead, and more time to find optimum strategy.
Disadvantages arise from fact that chosen execution strategy may no longer be optimal when query is run.
Could use a hybrid approach to overcome this.
If system detects that the database statistics have changed significantly then use the dynamic QO
Dr. Said Mirza Pahlevi, M.Eng.

18. Dr. Said Mirza Pahlevi, M.Eng.
Query Decomposition
Third Topic
Dr. Said Mirza Pahlevi, M.Eng.

19. Query Decomposition
Aims are to transform high-level query into RA query and check that query is syntactically and semantically correct.
Typical stages are:
analysis,
normalization,
semantic analysis,
simplification,
query restructuring.
Dr. Said Mirza Pahlevi, M.Eng.

20. 1. Analysis
Analyze query lexically and syntactically using compiler techniques.
Verify relations and attributes exist.
Verify operations are appropriate for object type.
Dr. Said Mirza Pahlevi, M.Eng.

21. Analysis - Example This query would be rejected on two grounds:
SELECT staff_no
FROM Staff
WHERE position > 10;
This query would be rejected on two grounds:
staff_no is not defined for Staff relation (should be staffNo).
Comparison ‘>10’ is incompatible with type position, which is variable character string.
Dr. Said Mirza Pahlevi, M.Eng.

22. Analysis
Finally, query transformed into some internal representation more suitable for processing.
Some kind of query tree is typically chosen, constructed as follows:
Leaf node created for each base relation.
Non-leaf node created for each intermediate relation produced by RA operation.
Root of tree represents query result.
Sequence is directed from leaves to root.
Dr. Said Mirza Pahlevi, M.Eng.

23. Example: R.A.T.
(s.position=‘Manager’(Staff)) s.branchNo= b.branchNo(b.city=‘London’(Branch))
Dr. Said Mirza Pahlevi, M.Eng.

24. 2. Normalization
Converts query into a normalized form for easier manipulation.
Predicate can be converted into one of two forms:
Conjunctive normal form:
(position = 'Manager'  salary > 20000)  (branchNo = 'B003')
Disjunctive normal form:
(position = 'Manager'  branchNo = 'B003' )  (salary >  branchNo = 'B003')
Dr. Said Mirza Pahlevi, M.Eng.

25. 3. Semantic Analysis
Rejects normalized queries that are incorrectly formulated or contradictory.
Query is incorrectly formulated if components do not contribute to generation of result.
If some join specifications are missing
Query is contradictory if its predicate cannot be satisfied by any tuple.
(position = 'Manager'  position = Assistant' )
Algorithms to determine correctness exist only for queries that do not contain disjunction and negation.
Dr. Said Mirza Pahlevi, M.Eng.

26. Semantic Analysis For queries we can construct:
A relation connection graph.
Normalized attribute connection graph.
Relation connection graph
Create node for each relation and node for result.
Create edges between two nodes that represent a join, and
edges between nodes that represent projection.
If not connected, query is incorrectly formulated.
Dr. Said Mirza Pahlevi, M.Eng.

27. Example: Checking Semantic Correctness
SELECT p.propertyNo, p.street
FROM Client c, Viewing v, PropertyForRent p
WHERE c.clientNo = v.clientNo AND
c.maxRent >= 500 AND
c.prefType = ‘Flat’ AND p.ownerNo = ‘CO93’;
Relation connection graph not fully connected, so query is not correctly formulated.
Have omitted the join condition (v.propertyNo = p.propertyNo) .
c.maxRent >= 500
c.prefType = ‘Flat’
p.ownerNo = ‘CO93’
c.clientNo = v.clientNo
Dr. Said Mirza Pahlevi, M.Eng.

28. 4. Simplification Objectives of this stage
Detects redundant qualifications,
Eliminates common sub-expressions,
Transforms query to semantically equivalent but more easily and efficiently computed form.
Typically, access restrictions, view definitions, and integrity constraints are considered.
Assuming user has appropriate access privileges, first apply well-known idempotency rules of boolean algebra.
Dr. Said Mirza Pahlevi, M.Eng.

29. Idempotency Rules p  (p)  p p  (p)  p
p  false  false p  false  p
p  true  p p  true  true
p  ( ~ p)  false p  (~p)  true
p  (p  q)  p p  (p  q)  p
Dr. Said Mirza Pahlevi, M.Eng.

30. Example: Simplification
no result
Dr. Said Mirza Pahlevi, M.Eng.

31. Heuristical Approach to Query Optimization
Fourth Topic
Dr. Said Mirza Pahlevi, M.Eng.

32. Transformation Rules: Rule 1
Conjunctive Selection operations can cascade into individual Selection operations (and vice versa).
pqr(R) = p(q(r(R)))
Sometimes referred to as cascade of Selection.
branchNo='B003'  salary>15000(Staff) = branchNo='B003'(salary>15000(Staff))
Dr. Said Mirza Pahlevi, M.Eng.

33. Transformation Rules: Rule 2
Commutativity of Selection.
p(q(R)) = q(p(R))
branchNo='B003'(salary>15000(Staff)) = salary>15000(branchNo='B003'(Staff))
Dr. Said Mirza Pahlevi, M.Eng.

34. Transformation Rules: Rule 3
In a sequence of Projection operations, only the last in the sequence is required.
LM … N(R) = L(R)
lNamebranchNo, lName(Staff) = lName(Staff)
Dr. Said Mirza Pahlevi, M.Eng.

35. Transformation Rules: Rule 4
Commutativity of Selection and Projection.
If predicate p involves only attributes in projection list, Selection and Projection operations commute:
A1, …, Am(p(R)) = p(A1, …, Am(R))
where p {A1, A2, …, Am}
fName, lName(lName='Beech'(Staff)) = lName='Beech'(fName,lName(Staff))
Dr. Said Mirza Pahlevi, M.Eng.

36. Transformation Rules: Rule 5
Commutativity of Theta join (and Cartesian product).
R p S = S p R
R X S = S X R
Rule also applies to Equijoin and Natural join. For example:
Staff staff.branchNo=branch.branchNo Branch =
Branch staff.branchNo=branch.branchNo Staff
Dr. Said Mirza Pahlevi, M.Eng.

37. Transformation Rules: Rule 6
Commutativity of Selection and Theta join (or Cartesian product).
If selection predicate involves only attributes of one of join relations, Selection and Join (or Cartesian product) operations commute:
p(R r S) = (p(R)) r S
p(R X S) = (p(R)) X S
where p {A1, A2, …, An}
Dr. Said Mirza Pahlevi, M.Eng.

38. Rule 6 (cont’)
If selection predicate is conjunctive predicate having form (p  q), where p only involves attributes of R, and q only attributes of S, Selection and Theta join operations commute as:
p  q(R r S) = (p(R)) r(q(S))
p  q(R X S) = (p(R)) X (q(S))
Dr. Said Mirza Pahlevi, M.Eng.

39. Rule 6 (cont’) For example:
position='Manager'  city='London'(Staff Staff.branchNo=Branch.branchNo Branch) =
(position='Manager'(Staff)) Staff.branchNo=Branch.branchNo (city='London' (Branch))
Dr. Said Mirza Pahlevi, M.Eng.

40. Transformation Rules: Rule 7
Commutativity of Projection and Theta join (or Cartesian product).
If projection list is of form L = L1  L2, where L1 only has attributes of R, and L2 only has attributes of S, provided join condition only contains attributes of L, Projection and Theta join commute:
L1L2(R r S) = (L1(R)) r(L2(S))
Dr. Said Mirza Pahlevi, M.Eng.

41. Rule 7 (cont’)
If join condition contains additional attributes not in L (M = M1  M2 where M1 only has attributes of R, and M2 only has attributes of S), a final projection operation is required:
L1L2(R r S) = L1L2( (L1M1(R)) r (L2M2(S)))
Dr. Said Mirza Pahlevi, M.Eng.

42. Rule 7 (cont’) For example: and using the latter rule:
position,city,branchNo(Staff Staff.branchNo=Branch.branchNo Branch) =
(position, branchNo(Staff)) Staff.branchNo=Branch.branchNo (city, branchNo (Branch))
and using the latter rule:
position, city(Staff Staff.branchNo=Branch.branchNo Branch) =
position, city ((position, branchNo(Staff)) Staff.branchNo=Branch.branchNo
( city, branchNo (Branch)))
Dr. Said Mirza Pahlevi, M.Eng.

43. Transformation Rules: Rule 8
Commutativity of Union and Intersection (but not set difference).
R  S = S  R
R  S = S  R
Dr. Said Mirza Pahlevi, M.Eng.

44. Transformation Rules: Rule 9
Commutativity of Selection and set operations (Union, Intersection, and Set difference).
p(R  S) = p(S)  p(R)
p(R  S) = p(S)  p(R)
p(R – S) = p(S) – p(R)
Dr. Said Mirza Pahlevi, M.Eng.

45. Transformation Rules: Rule 10
Commutativity of Projection and Union.
L(R  S) = L(S)  L(R)
Dr. Said Mirza Pahlevi, M.Eng.

46. Transformation Rules: Rule 11
Associativity of Theta join (and Cartesian product).
Cartesian product and Natural join are always associative:
(R S) T = R (S T)
(R X S) X T = R X (S X T)
If join condition q involves attributes only from S and T, then Theta join is associative:
(R p S) q  r T = R p  r (S q T)
Dr. Said Mirza Pahlevi, M.Eng.

47. Rule 11 (cont’) For example: Cannot simply move the bracket!
(Staff Staff.staffNo=PropertyForRent.staffNo PropertyForRent) ownerNo=Owner.ownerNo  staff.lName=Owner.lName Owner =
Staff staff.staffNo=PropertyForRent.staffNo  staff.lName=lName (PropertyForRent ownerNo Owner)
Cannot simply move the bracket!
Dr. Said Mirza Pahlevi, M.Eng.

48. Transformation Rules: Rule 12
Associativity of Union and Intersection (but not Set difference).
(R  S)  T = S  (R  T)
(R  S)  T = S  (R  T)
Dr. Said Mirza Pahlevi, M.Eng.

49. Example: Use of Transformation Rules
For prospective renters of flats, find properties that match requirements and owned by CO93
Assumption: there are fewer properties owned by owner CO93 than prospective renters
SELECT p.propertyNo, p.street
FROM Client c, Viewing v, PropertyForRent p
WHERE c.prefType = ‘Flat’ AND
c.clientNo = v.clientNo AND
v.propertyNo = p.propertyNo AND
c.maxRent >= p.rent AND
c.prefType = p.type AND
p.ownerNo = ‘CO93’;
Dr. Said Mirza Pahlevi, M.Eng.

50. Example SELECT p.propertyNo, p.street
FROM Client c, Viewing v, PropertyForRent p
WHERE
c.prefType = ‘Flat’ AND
c.clientNo = v.clientNo AND
v.propertyNo = p.propertyNo AND
c.maxRent >= p.rent AND
c.prefType = p.type AND
p.ownerNo = ‘CO93’;
Dr. Said Mirza Pahlevi, M.Eng.

51. Example
Rule 1, 2, 6
Split conjuction of selections into individual selection (rule 1)
Reorder selection and commute the selection and cartesian products (rule 2 and 6)
Dr. Said Mirza Pahlevi, M.Eng.

52. Cartesian Product + Selection = Join
Example
Cartesian Product + Selection = Join
Dr. Said Mirza Pahlevi, M.Eng.

53. Rule 11: reorder the equijoin
Example
Rule 11: reorder the equijoin 4 50
100
100 4 50
Assumption: there are fewer properties owned by owner CO93 than prospective renters
Dr. Said Mirza Pahlevi, M.Eng.

54. Example
Rule 4 & 7
Move projections down to past the Equijoints and create new Projection as required
Dr. Said Mirza Pahlevi, M.Eng.

55. Example
p.type=‘flat’
Dr. Said Mirza Pahlevi, M.Eng.

56. Heuristical Processing Strategies
Perform Selection operations as early as possible.
Keep predicates on same relation together.
Combine Cartesian product with subsequent Selection whose predicate represents join condition into a Join operation.
Use associativity of binary operations to rearrange leaf nodes so leaf nodes with most restrictive Selection operations executed first.
Dr. Said Mirza Pahlevi, M.Eng.

57. Heuristical Processing Strategies
Perform Projection as early as possible.
Keep projection attributes on same relation together.
Compute common expressions once.
If common expression appears more than once, and result not too large, store result and reuse it when required.
Useful when querying views, as same expression is used to construct view each time.
Dr. Said Mirza Pahlevi, M.Eng.

58. Quiz 1
Buatlah R.A.T dari query berikut dan kemudian lakukan optimisasi dengan menggunakan pendekatan Heuristic! Tuliskan:
SQL query
Relational algebra
Initial R.A.T
Optimized R.A.T
Tampilkan firstname dan nomor branch dari staff yang bekerja di branch London dimana salary mereka lebih besar dari 10000!
Dr. Said Mirza Pahlevi, M.Eng.

59. Answer SQL Query Relational Algebra
SELECT fName, branchNo
FROM Branch, Staff
WHERE Branch.branchNo=Staff.branchNo AND
salary>10000 AND city='London'
Relational Algebra
fName, branchNo (city=‘London’  salary>10000
(Staff Staff.branchNo=Branch.branchNo Branch))
Dr. Said Mirza Pahlevi, M.Eng.

60. Answer staff branch S.branchNo=B.branchNo salary>10000
fName, branchNo
city=‘London’
branchNo
staff
branch
S.branchNo=B.branchNo
city=‘London’  salary>10000
fName, branchNo
Dr. Said Mirza Pahlevi, M.Eng.

61. Quiz 2
Buatlah R.A.T dari query berikut dan kemudian lakukan optimisasi dengan menggunakan pendekatan Heuristic! Tuliskan:
SQL query
Relational algebra
Initial R.A.T
Optimized R.A.T
Tampilkan propertyNo beserta fname dari ownernya, dimana property tersebut berada pada kota Glasgow, memiliki kamar lebih dari 3 dan nomor telpon ownernya berawalan 0141!
Dr. Said Mirza Pahlevi, M.Eng.

62. Answer SQL Query Relational Algebra
SELECT propertyNo, fname
FROM PropertyForRent p, PrivateOwner o
WHERE p.ownerNo=o.ownerNo AND city='Glasgow' AND
rooms > 3 AND telNo LIKE '0141%'
Relational Algebra
propertyNo, fName(city=‘Glasgow’  rooms>3  telNo like ‘0141%’
(PropertyForRent p.ownerNo=o.ownerNo PrivateOwner))
Dr. Said Mirza Pahlevi, M.Eng.

63. Answer p.ownerNo=o.ownerNo city=‘Glasgow’  rooms > 3
PropertyForRent
PrivateOwner
p.ownerNo=o.ownerNo
city=‘Glasgow’
 rooms > 3
 propertyNo, fName
 telNo like ‘0141%’
propertyNo, ownerNo
fName, ownerNo
PropertyForRent
PrivateOwner
p.ownerNo=o.ownerNo
city=‘Glasgow’  rooms > 3
 telNo like ‘0141%’
propertyNo, fName
Dr. Said Mirza Pahlevi, M.Eng.

64. Tampilkan nomor branch dan nomor property yang dikelola oleh branch tersebut dengan persyaratan bahwa jenis property adalah Flat dan jumlah kamar 4.
Dr. Said Mirza Pahlevi, M.Eng.

65. Quiz 3
Buatlah R.A.T dari query berikut dan kemudian lakukan optimisasi dengan menggunakan pendekatan Heuristic! Tuliskan:
SQL query
Relational algebra
Initial R.A.T
Optimized R.A.T
Tampilkan nomor branch dan nomor property yang dikelola oleh branch tersebut dengan persyaratan bahwa jenis property adalah Flat dan jumlah kamarnya 4!
Dr. Said Mirza Pahlevi, M.Eng.

66. Answer SQL Query Relational Algebra
SELECT branchNo, propertyNo
FROM Branch as b, PropertyForRent as p
WHERE b.branchNo=p.branchNo AND
p.type=‘Flat’ AND p.rooms=4
Relational Algebra
branchNo, propertyNo (p.type=‘Flat’  p.rooms=4
(Branch b.branchNo=p.branchNo PropertyForRent))
Dr. Said Mirza Pahlevi, M.Eng.

67. Answer branchNo, propertyNo (p.type=‘Flat’  p.rooms=4
(Branch b.branchNo=p.branchNo PropertyForRent))
propertyNo, s.branchNo
branchNo, propertyNo
s.branchNo=b.branchNo
type=‘Flat’  rooms=4
propertyNo, branchNo
branchNo
s.branchNo=b.branchNo
type=‘Flat’  rooms=4
Branch
PropertyForRent
Branch
PropertyForRent
Dr. Said Mirza Pahlevi, M.Eng.

68. Quiz 4
Buatlah R.A.T dari query berikut dan kemudian lakukan optimisasi dengan menggunakan pendekatan Heuristic! Tuliskan:
SQL query
Relational algebra
Initial R.A.T
Optimized R.A.T
Tampilkan firstname dan lastname dari staff beserta nomor property dan nomor owner dari property yang dikelola oleh staff tersebut, dengan persyaratan bahwa posisi staff adalah Assistent dan jenis Propertynya adalah Flat!
Dr. Said Mirza Pahlevi, M.Eng.

69. Answer SQL Query Relational Algebra
Tampilkan firstname dan lastname dari staff beserta nomor property dan nomor owner dari property yang dikelola oleh staff tersebut, dengan persyaratan bahwa posisi staff adalah Assistent dan jenis Propertynya adalah Flat!
Answer
SQL Query
SELECT fname, lname, propertyNo, ownerNo
FROM Staff s, PropertyForRent p
WHERE s.staffNo=p.staffNo AND s.position=‘Assistent' AND
p.type = ‘Flat’
Relational Algebra
fname, lname, propertyNo, ownerNo(s.position=‘Assistent’  p.type=‘Flat’
(Staff s.staffNo=p.staffNo PropertyForRent))
Dr. Said Mirza Pahlevi, M.Eng.

70. Answer
fname, lname, propertyNo, ownerNo(s.position=‘Assistent’  p.type=‘Flat’
(Staff s.staffNo=p.staffNo PropertyForRent))
PropertyForRent
Staff
p.staffNo=s.staffNo
p.type=‘Flat’
 fname,lname, propertyNo, ownerNo
 s.position=‘Assistant’
propertyNo, staffNo, ownerNo
fName, lname, staffNo
PropertyForRent
Staff
s.staffNo=p.staffNo
s.position=‘Assistent’ 
p.type=‘Flat’
fname,lname, propertyNo, ownerNo
Dr. Said Mirza Pahlevi, M.Eng.