1. EXTERNAL SORTING ALGORITHMS AND IMPLEMENTATIONS 05011004 Ayhan KARGIN 05011027 Ahmet MERAL

2. Contents External Sorting Needs and Usage Areas External Sorting Algorithms Environments for Implementing External Sorting Algorithms Used Technologies Phases of External Sorting K-Way Merge Sort Multi-Step K-Way Merge Sort Replacement Selection Sort SimpleDB Layered Components of SimpleDB Classes of SimpleDB Query Layer of SimpleDB Relational Algebra, that SimpleDB Supports Relational Algebra Preparatory Work External Sorting on SimpleDB

3. External Sorting Needs and Usage Areas DBMS Group By, Join, Order By Data Warehouse (ETL) Data Mining Data Processing

4. External Sorting Algorithms K-Way Merge Sort Multi-Step K-Way Merge Sort Replacement Selection Sort

5. Environments for Implementing External Sorting Algorithms MinSQL PosgreSQL SimpleDB

6. Used Technologies Java VM Java SE JDBC Java RMI Eclipse

7. Phases of External Sorting Run Generation Phase Merge Phase

8. K-Way Merge Sort

9. Multi-Step K-Way Merge Sort

11. Replacement Selection

12. Test results depending on stage area size Note: Main memory is 8x. Stage AreaDisc Access CountRun TimeK (Run Count) 1x38170.69017 2x4200.7428 3x2521.0496 4x2581.4204 5x2661.9524 6x2792.3333

13. SimpleDB The Client Side That contains the JDBC interfaces and implements the JDBC driver The Basic Server Which provides complete funcionality of DB but ignores efficiency issues Extensions To the basic server that support efficient query processing.

14. Layered Components of SimpleDB Remote Perform JDBC requests received from clients. Parse Extract the tables, fields, and predicate mentioned in an SQL statement. Planner Create an execution strategy for an SQL statement, and translate it to a relational algebra plan. Query Implement queries expressed in relational algebra. Metadata Maintain metadata about the tables in the database, so that its records and fields are accessible.

15. Layered Components of SimpleDB Record Provide methods for storing data records in pages Transaction Support concurrency by restricting page access. Enable recovery by logging changes to pages. Buffer Maintain a cache of pages in memory to hold recently-accessed user data. Log Append log records to the log file, and scan the records in the log file. File Read and write between file blocks and memory pages.

16. Classes of SimpleDB

19. Query Layer of SimpleDB Relational Algebra Select Project Product Sort

20. Relational Algebra, that SimpleDB Supports Relational Algebra Select Project Product Sort SQL Where Select Join Order By

21. Relational Algebra Select SId, SName, DName from STUDENT, DEPT where MajorId=DId and DName='math' Q1: Product (STUDENT, DEPT) Q2: Select (Q2, MajorId=DId ) Q3: Select (Q2, DName='math') Q4: Project (Q3, {SId, SName, DName })

22. Relational Algebra

23. Preparatory Work We have to introduce order by operator (in SQL) to parser layer of SimpleDB. The columns, which will be sort, must be presenced in QueryData class in structural form. If sorting will be happened, sorting plans must be called by BasicQueryPlanner class. Calculating number of random accesses per file on FileMgr layer. Calculating duration of transaction with adding start time and end time to Transaction class. Calculating number of unsorted records on each transaction. We may have to compare, copy and exchange congeneric records on different scans. So, RecordExchange class is written for these purposes. StepCalculator class is written. It splits any number to two close integer multipliers. For example; from 29, 6 and 5; from 49, 7 and 7; from 50, 8 and 7; etc.

24. External Sorting on SimpleDB Materialize Plan Edward Sciores Merge Sort Plan&Scan K-Way Merge Sort Plan&Scan Multi-Step K-Way Merge Sort Plan&Scan Replacement Selection Sort Plan&Scan Multi-Step Replacement Selection Sort Plan&Scan

25. Algorithms Choice Dependency