Presentation is loading. Please wait.

Presentation is loading. Please wait.

Example. Bulk Nested-Loop Joins Using Buffers: e.g. 22 blocks.

Published byJoshua Wiggins Modified over 9 years ago

Similar presentations

Presentation on theme: "Example. Bulk Nested-Loop Joins Using Buffers: e.g. 22 blocks."— Presentation transcript:

1 Example

2 Bulk Nested-Loop Joins Using Buffers: e.g. 22 blocks

3

4

5

6

7

8 Sort Merge Joins Notes From Winter 2015

9 Visualizations Sailors Page 1 Page 2 Page 3 Page 4

10 Visualizations Sailors Record 1 Record 2 Record 3 Record 4 Record 5 Page 2 Page 3 Page 4

11 Visualizations Sailors Record 1 Record 2 Record 3 Record 4 Record 5 Page 2 Page 3 Page 4 Reserves

12 Simple Nested Loops Join Key idea: Take each record of S and match it with each record of R. Steps: 1.Get tuple of S. 2.Iterate through each tuple in R. SailorsReserves

13 Simple Nested Loops Join Key idea: Take each record of S and match it with each record of R. Steps: 1.Get tuple of S. 2.Iterate through each tuple in R. SailorsReserves (name = Bob, sid = 1)

14 Simple Nested Loops Join Key idea: Take each record of S and match it with each record of R. Steps: 1.Get tuple of S. 2.Iterate through each tuple in R. SailorsReserves (name = Bob, sid = 1)(sid = 3, bid = 6) (sid = 1, bid = 4) (name = Bob, sid = 1, bid = 4) Output:

15 Simple Nested Loops Join Key idea: Take each record of S and match it with each record of R. Steps: 1.Get tuple of S. 2.Iterate through each tuple in R. SailorsReserves (name = Bob, sid = 1)(sid = 3, bid = 6) (sid = 1, bid = 4) (name = Bob, sid = 1, bid = 4) Output: (sid = 1, bid = 7) (name = Bob, sid = 1, bid = 7)

16 Sort-Merge Join Key idea: Sort S and R, then merge them! Steps: 1.Sort S and R. 2.“Zip” or merge. SailorsReserves

17 Sort-Merge Join Key idea: Sort S and R on join column, then merge them! Steps: 1.Sort S and R. 2.“Zip” or merge. SailorsReserves (name = Bob, sid = 1) (name = Sam, sid = 3) (name = Sue, sid = 7) (name = Jill, sid = 2) (name = Joe, sid = 12) (name = Sue, sid = 8) (name = Yue, sid = 4)

18 Sort-Merge Join Key idea: Sort S and R on join column, then merge them! Steps: 1.Sort S and R. 2.“Zip” or merge. SailorsReserves (name = Bob, sid = 1) (name = Jill, sid = 2) (name = Sam, sid = 3) (name = Yue, sid = 4) (name = Sue, sid = 7) (name = Sue, sid = 8) (name = Joe, sid = 12)...

19 Sort-Merge Join Key idea: Sort S and R on join column, then merge them! Steps: 1.Sort S and R. 2.“Zip” or merge. SailorsReserves (name = Bob, sid = 1) (name = Jill, sid = 2) (name = Sam, sid = 3) (name = Yue, sid = 4) (name = Sue, sid = 7) (name = Sue, sid = 8) (name = Joe, sid = 12)... (sid = 1, bid = 4) (sid = 1, bid = 7) (sid = 3, bid = 6) (sid = 4, bid = 3) (sid = 8, bid = 1) (sid = 8, bid = 13) (sid = 8, bid = 15) (sid = 12, bid = 1)...

20 Sort-Merge Join Key idea: Sort S and R on join column, then merge them! Steps: 1.Sort S and R. 2.“Zip” or merge. SailorsReserves (name = Bob, sid = 1) (name = Jill, sid = 2) (name = Sam, sid = 3) (name = Yue, sid = 4) (name = Sue, sid = 7) (name = Sue, sid = 8) (name = Joe, sid = 12)... (sid = 1, bid = 4) (sid = 1, bid = 7) (sid = 3, bid = 6) (sid = 4, bid = 3) (sid = 8, bid = 1) (sid = 8, bid = 13) (sid = 8, bid = 15) (sid = 12, bid = 1)... (name = Bob, sid = 1, bid = 4) Output:

21 Sort-Merge Join Key idea: Sort S and R on join column, then merge them! Steps: 1.Sort S and R. 2.“Zip” or merge. SailorsReserves (name = Bob, sid = 1) (name = Jill, sid = 2) (name = Sam, sid = 3) (name = Yue, sid = 4) (name = Sue, sid = 7) (name = Sue, sid = 8) (name = Joe, sid = 12)... (sid = 1, bid = 4) (sid = 1, bid = 7) (sid = 3, bid = 6) (sid = 4, bid = 3) (sid = 8, bid = 1) (sid = 8, bid = 13) (sid = 8, bid = 15) (sid = 12, bid = 1)... (name = Bob, sid = 1, bid = 4) (name = Bob, sid = 1, bid = 7) Output:

22 Sort-Merge Join Key idea: Sort S and R on join column, then merge them! Steps: 1.Sort S and R. 2.“Zip” or merge. SailorsReserves (name = Bob, sid = 1) (name = Jill, sid = 2) (name = Sam, sid = 3) (name = Yue, sid = 4) (name = Sue, sid = 7) (name = Sue, sid = 8) (name = Joe, sid = 12)... (sid = 1, bid = 4) (sid = 1, bid = 7) (sid = 3, bid = 6) (sid = 4, bid = 3) (sid = 8, bid = 1) (sid = 8, bid = 13) (sid = 8, bid = 15) (sid = 12, bid = 1)... (name = Bob, sid = 1, bid = 4) (name = Bob, sid = 1, bid = 7) Output:

23 Sort-Merge Join Key idea: Sort S and R on join column, then merge them! Steps: 1.Sort S and R. 2.“Zip” or merge. SailorsReserves (name = Bob, sid = 1) (name = Jill, sid = 2) (name = Sam, sid = 3) (name = Yue, sid = 4) (name = Sue, sid = 7) (name = Sue, sid = 8) (name = Joe, sid = 12)... (sid = 1, bid = 4) (sid = 1, bid = 7) (sid = 3, bid = 6) (sid = 4, bid = 3) (sid = 8, bid = 1) (sid = 8, bid = 13) (sid = 8, bid = 15) (sid = 12, bid = 1)... (name = Bob, sid = 1, bid = 4) (name = Bob, sid = 1, bid = 7) (name = Sam, sid = 3, bid = 6) Output:

24 Sort-Merge Join Key idea: Sort S and R on join column, then merge them! Steps: 1.Sort S and R. 2.“Zip” or merge. SailorsReserves (name = Bob, sid = 1) (name = Jill, sid = 2) (name = Sam, sid = 3) (name = Yue, sid = 4) (name = Sue, sid = 7) (name = Sue, sid = 8) (name = Joe, sid = 12)... (sid = 1, bid = 4) (sid = 1, bid = 7) (sid = 3, bid = 6) (sid = 4, bid = 3) (sid = 8, bid = 1) (sid = 8, bid = 13) (sid = 8, bid = 15) (sid = 12, bid = 1)... (name = Bob, sid = 1, bid = 4) (name = Bob, sid = 1, bid = 7) (name = Sam, sid = 3, bid = 6) Output:...

25 Sort-Merge Join Key idea: Sort S and R on join column, then merge them! Steps: 1.Sort S and R. 2.“Zip” or merge. SailorsReserves (name = Bob, sid = 1) (name = Jill, sid = 2) (name = Sam, sid = 3) (name = Yue, sid = 4) (name = Sue, sid = 7) (name = Sue, sid = 8) (name = Joe, sid = 12)... (sid = 1, bid = 4) (sid = 1, bid = 7) (sid = 3, bid = 6) (sid = 4, bid = 3) (sid = 8, bid = 1) (sid = 8, bid = 13) (sid = 8, bid = 15) (sid = 12, bid = 1)... (name = Bob, sid = 1, bid = 4) (name = Bob, sid = 1, bid = 7) (name = Sam, sid = 3, bid = 6) Output:...

26 Sort-Merge Join Key idea: Sort S and R on join column, then merge them! Steps: 1.Sort S and R. 2.“Zip” or merge. SailorsReserves (name = Bob, sid = 1) (name = Jill, sid = 2) (name = Sam, sid = 3) (name = Yue, sid = 4) (name = Sue, sid = 7) (name = Sue, sid = 8) (name = Joe, sid = 12)... (sid = 1, bid = 4) (sid = 1, bid = 7) (sid = 3, bid = 6) (sid = 4, bid = 3) (sid = 8, bid = 1) (sid = 8, bid = 13) (sid = 8, bid = 15) (sid = 12, bid = 1)... (name = Bob, sid = 1, bid = 4) (name = Bob, sid = 1, bid = 7) (name = Sam, sid = 3, bid = 6) Output:...

27 Sort-Merge Join Key idea: Sort S and R on join column, then merge them! Steps: 1.Sort S and R. 2.“Zip” or merge. SailorsReserves (name = Bob, sid = 1) (name = Jill, sid = 2) (name = Sam, sid = 3) (name = Yue, sid = 4) (name = Sue, sid = 7) (name = Sue, sid = 8) (name = Joe, sid = 12)... (sid = 1, bid = 4) (sid = 1, bid = 7) (sid = 3, bid = 6) (sid = 4, bid = 3) (sid = 8, bid = 1) (sid = 8, bid = 13) (sid = 8, bid = 15) (sid = 12, bid = 1)... (name = Bob, sid = 1, bid = 4) (name = Bob, sid = 1, bid = 7) (name = Sam, sid = 3, bid = 6) Output:...

28 Sort-Merge Join Key idea: Sort S and R on join column, then merge them! Steps: 1.Sort S and R. 2.“Zip” or merge. SailorsReserves (name = Bob, sid = 1) (name = Jill, sid = 2) (name = Sam, sid = 3) (name = Yue, sid = 4) (name = Sue, sid = 7) (name = Sue, sid = 8) (name = Joe, sid = 12)... (sid = 1, bid = 4) (sid = 1, bid = 7) (sid = 3, bid = 6) (sid = 4, bid = 3) (sid = 8, bid = 1) (sid = 8, bid = 13) (sid = 8, bid = 15) (sid = 12, bid = 1)... (name = Bob, sid = 1, bid = 4) (name = Bob, sid = 1, bid = 7) (name = Sam, sid = 3, bid = 6) Output:...

29 Sort-Merge Join Key idea: Sort S and R on join column, then merge them! Steps: 1.Sort S and R. 2.“Zip” or merge. SailorsReserves (name = Bob, sid = 1) (name = Jill, sid = 2) (name = Sam, sid = 3) (name = Yue, sid = 4) (name = Sue, sid = 7) (name = Sue, sid = 8) (name = Joe, sid = 12)... (sid = 1, bid = 4) (sid = 1, bid = 7) (sid = 3, bid = 6) (sid = 4, bid = 3) (sid = 8, bid = 1) (sid = 8, bid = 13) (sid = 8, bid = 15) (sid = 12, bid = 1)... (name = Bob, sid = 1, bid = 4) (name = Bob, sid = 1, bid = 7) (name = Sam, sid = 3, bid = 6) Output:...

30 Sort-Merge Join Key idea: Sort S and R on join column, then merge them! Steps: 1.Sort S and R. 2.“Zip” or merge. I/Os: SailorsReserves (name = Bob, sid = 1) (name = Jill, sid = 2) (name = Sam, sid = 3) (name = Yue, sid = 4) (name = Sue, sid = 7) (name = Sue, sid = 8) (name = Joe, sid = 12)... (sid = 1, bid = 4) (sid = 1, bid = 7) (sid = 3, bid = 6) (sid = 4, bid = 3) (sid = 8, bid = 1) (sid = 8, bid = 13) (sid = 8, bid = 15) (sid = 12, bid = 1)...

31 Optimizing Sort-Merge Join Key idea: Internal Sort on both. Perform merge on all runs! Steps: 1.Internal sort S and R. (Pass 0) 2.Merge all runs. SailorsReserves (name = Bob, sid = 1) (name = Jill, sid = 2) (name = Sam, sid = 3) (name = Yue, sid = 4) (name = Sue, sid = 7) (name = Sue, sid = 8) (name = Joe, sid = 12)... (sid = 1, bid = 4) (sid = 1, bid = 7) (sid = 3, bid = 6) (sid = 4, bid = 3) (sid = 8, bid = 1) (sid = 8, bid = 13) (sid = 8, bid = 15) (sid = 12, bid = 1)...

32 Optimizing Sort-Merge Join Key idea: Internal Sort on both. Perform merge on all runs! Steps: 1.Internal sort S and R. (Pass 0) 2.Merge all runs. SailorsReserves (name = Bob, sid = 1) (name = Jill, sid = 2) (name = Yue, sid = 4) (name = Sue, sid = 8) (name = Jack, sid = 18) (name = Cat, sid = 22)... (name = Sam, sid = 3) (name = Sue, sid = 7) (name = Joe, sid = 12)... (sid = 1, bid = 4) (sid = 1, bid = 7) (sid = 4, bid = 3) (sid = 8, bid = 1) (sid = 8, bid = 13) (sid = 12, bid = 1)... (sid = 3, bid = 6) (sid = 8, bid = 15)...

33 External Sorting Classic interview question: how to sort if data don’t fit in memory.

34 External Sorting: 2-Way Merge Sort

35 What is a sorted run? (name = Bob, sid = 1) (name = Jill, sid = 2) (name = Sam, sid = 3) (name = Sue, sid = 6) (name = Kev, sid = 8) (name = Jack, sid = 9) (name = Joe, sid = 10) (name = Sid, sid = 12) (name = Sal, sid = 15) (name = Bit, sid = 1) (name = Bat, sid = 2) (name = Tam, sid = 3) (name = Foo, sid = 6) (name = Bar, sid = 8) (name = Bam, sid = 9) (name = Ke, sid = 10) (name = Kay, sid = 12) (name = Al, sid = 15) A sorted subset of a table. Above we have pages with tuple size = 3 There are two sorted runs both with a length of 3 pages. (We denote the size of a sorted run by how many pages the sorted run spans)

36 2 Way Merge Sort Given N pages of tuples to sort B = 3 case Pass 0: Quicksort a single page 2-way merge sort: how many passes? – 1+ceil(log_2(N)) – 1 for pass 0, ceil(log_2(N)) because pass 0 has N runs and at each level we merge 2 runs together – ceil of this because the number of runs in pass 0 might not be a a power of two, but we still need to keep merging until we're down to 1 run,

37 Generalized Sort Merge

Download ppt "Example. Bulk Nested-Loop Joins Using Buffers: e.g. 22 blocks."

Similar presentations

Database Management Systems, R. Ramakrishnan and J. Gehrke1 Evaluation of Relational Operations Chapter 12, Part A.

Database Management Systems, R. Ramakrishnan and J. Gehrke1 Evaluation of Relational Operations Chapter 12, Part A.
Equality Join R X R.A=S.B S : : Relation R M PagesN Pages Relation S Pr records per page Ps records per page.

Equality Join R X R.A=S.B S : : Relation R M PagesN Pages Relation S Pr records per page Ps records per page.
CS 540 Database Management Systems

CS 540 Database Management Systems
Join Processing in Database Systems with Large Main Memories ACM Transactions on Database Systems Vol. 11, No. 3, Sep 1986 Leonard D. Shapiro Donghui Zhang,

Join Processing in Database Systems with Large Main Memories ACM Transactions on Database Systems Vol. 11, No. 3, Sep 1986 Leonard D. Shapiro Donghui Zhang,
Join Processing in Databases Systems with Large Main Memories

Join Processing in Databases Systems with Large Main Memories
Database Management Systems 3ed, R. Ramakrishnan and Johannes Gehrke1 Evaluation of Relational Operations: Other Techniques Chapter 14, Part B.

Database Management Systems 3ed, R. Ramakrishnan and Johannes Gehrke1 Evaluation of Relational Operations: Other Techniques Chapter 14, Part B.
Database Management Systems, R. Ramakrishnan and Johannes Gehrke1 Evaluation of Relational Operations: Other Techniques Chapter 12, Part B.

Database Management Systems, R. Ramakrishnan and Johannes Gehrke1 Evaluation of Relational Operations: Other Techniques Chapter 12, Part B.
CS186 Week 0 Out of Core Algorithms. Today External Merge Sort External Hashing.

CS186 Week 0 Out of Core Algorithms. Today External Merge Sort External Hashing.
Database Management Systems, R. Ramakrishnan and J. Gehrke1 External Sorting Chapter 11.

Database Management Systems, R. Ramakrishnan and J. Gehrke1 External Sorting Chapter 11.
1 Overview of Query Evaluation Chapter 12. 2 Objectives  Preliminaries:  Core query processing techniques  Catalog  Access paths to data  Index matching.

1 Overview of Query Evaluation Chapter Objectives  Preliminaries:  Core query processing techniques  Catalog  Access paths to data  Index matching.
1 External Sorting Chapter 13. 2 Why Sort?  A classic problem in computer science!  Data requested in sorted order  e.g., find students in increasing.

1 External Sorting Chapter Why Sort?  A classic problem in computer science!  Data requested in sorted order  e.g., find students in increasing.
External Sorting “There it was, hidden in alphabetical order.” Rita Holt R&G Chapter 13.

External Sorting “There it was, hidden in alphabetical order.” Rita Holt R&G Chapter 13.
Database Management Systems, R. Ramakrishnan and J. Gehrke1 Query Evaluation Chapter 11 External Sorting.

Database Management Systems, R. Ramakrishnan and J. Gehrke1 Query Evaluation Chapter 11 External Sorting.
1 Implementation of Relational Operations Module 5, Lecture 1.

1 Implementation of Relational Operations Module 5, Lecture 1.
1  Simple Nested Loops Join:  Block Nested Loops Join  Index Nested Loops Join  Sort Merge Join  Hash Join  Hybrid Hash Join Evaluation of Relational.

1  Simple Nested Loops Join:  Block Nested Loops Join  Index Nested Loops Join  Sort Merge Join  Hash Join  Hybrid Hash Join Evaluation of Relational.
SPRING 2004CENG 3521 Join Algorithms Chapter 14. SPRING 2004CENG 3522 Schema for Examples Similar to old schema; rname added for variations. Reserves:

SPRING 2004CENG 3521 Join Algorithms Chapter 14. SPRING 2004CENG 3522 Schema for Examples Similar to old schema; rname added for variations. Reserves:
1 Optimization Recap and examples. 2 Optimization introduction For every SQL expression, there are many possible ways of implementation. The different.

1 Optimization Recap and examples. 2 Optimization introduction For every SQL expression, there are many possible ways of implementation. The different.
External Sorting R & G Chapter 11 One of the advantages of being disorderly is that one is constantly making exciting discoveries. A. A. Milne.

External Sorting R & G Chapter 11 One of the advantages of being disorderly is that one is constantly making exciting discoveries. A. A. Milne.
Query Processing 1: Joins and Sorting R&G, Chapters 12, 13, 14 Lecture 8 One of the advantages of being disorderly is that one is constantly making exciting.

Query Processing 1: Joins and Sorting R&G, Chapters 12, 13, 14 Lecture 8 One of the advantages of being disorderly is that one is constantly making exciting.
1 Optimization - Selection. 2 The Selection Operation Table: Reserves(sid, bid, day, agent) A page (block) can hold 100 Reserves tuples There are 1,000.

1 Optimization - Selection. 2 The Selection Operation Table: Reserves(sid, bid, day, agent) A page (block) can hold 100 Reserves tuples There are 1,000.

Similar presentations

Ads by Google