1. An Efficient Multi-Dimensional Index for Cloud Data Management Xiangyu Zhang Jing Ai Zhongyuan Wang Jiaheng Lu Xiaofeng Meng School of Information Renmin University of China

2. Outline Motivation Query Answering on the Cloud Related Work EMINC: Index the Cloud Efficiently ◦ Node Bounding ◦ Extended Node Bounding ◦ Cost Estimation based Index Update Evaluation Conclusion & Future Work

3. Outline Motivation Query Answering on the Cloud Related Work EMINC: Index the Cloud Efficiently ◦ Node Bounding ◦ Extended Node Bounding ◦ Cost Estimation based Index Update Evaluation Conclusion & Future Work

4. Motivation Cloud systems have been justified as brilliant for web search applications ◦ Simple structure, mostly key-value pairs ◦ Flexible, efficient for analytic work However, they are insufficient for complex data management needs ◦ No powerful language as SQL ◦ Hard to process complex queries ◦ Lack of efficient index structures

5. Distributed Cloud base? BigTable HBase How to query on other attributes besides primary key?

6. Motivation As part of our Cloud-based DBMS project, we aim to build efficient index structure on the Cloud.

7. Outline Motivation Query Answering on the Cloud Related Work EMINC: Index the Cloud Efficiently ◦ Node Bounding ◦ Extended Node Bounding ◦ Cost Estimation based Index Update Evaluation Conclusion & Future Work

8. Query Answering in the Cloud Fast locating of relevant slave nodes Efficient lookup on each slave nodes

9. Outline Motivation Query Answering on the Cloud Related Work EMINC: Index the Cloud Efficiently ◦ Node Bounding ◦ Extended Node Bounding ◦ Cost Estimation based Index Update Evaluation Conclusion & Future Work

10. Related Work S. Wu and K.-L. Wu, “An indexing framework for efficient retrieval on the cloud,” IEEE Data Eng. Bull., vol. 32, pp.75–82, 2009. H. chih Yang and D. S. Parker, “Traverse: Simplified indexing on large map-reduce-merge clusters,” in Proceedings of DASFAA 2009, Brisbane, Australia, April 2009, pp. 308–322. M. K. Aguilera, W. Golab, and M. A. Shah, “A practical scalable distributed b-tree,” in Proceedings of VLDB’08, Auckland, New Zealand, August 2008, pp. 598–609.

11. Distributed Database Data slicing in DDBS ◦ Horizontal, vertical, etc. ◦ Slice based on conditions ◦ Check condition conflict on query processing Data distribution on the Cloud is different and could be very complex if expressed as set of conditions Condition check is too expensive

12. Outline Motivation Query Answering on the Cloud Related Work EMINC: Index the Cloud Efficiently ◦ Node Bounding ◦ Extended Node Bounding ◦ Cost Estimation based Index Update Evaluation Conclusion & Future Work

13. Outline Motivation Query Answering on the Cloud Related Work EMINC: Index the Cloud Efficiently ◦ Node Bounding ◦ Extended Node Bounding ◦ Cost Estimation based Index Update Evaluation Conclusion & Future Work

14. EMINC: Node Bounding Node cube of a table on a slave node ◦ Value range of table on this node IdAB 111 222 334 467 5510 Node Cube: (1,1), (6,10)

15. EMINC: Architecture Each leaf node corresponds to one node cube Use KD-Tree to maintain local index on slave nodes

16. EMINC: Query Processing Get query cube of the query and look up in the R-Tree to get relevant data nodes ◦ 1 Query Cube: (1,3),(2,4) Node Cube Query Cube Node Cube Query Cube

17. Outline Motivation Query Answering on the Cloud Related Work EMINC: Index the Cloud Efficiently ◦ Node Bounding ◦ Extended Node Bounding ◦ Cost Estimation based Index Update Evaluation Conclusion & Future Work

18. EMINC: Extended Node Bounding Problem with single bounding ◦ Bad performance for sparse node Many queries will be mislead to this node

19. EMINC: Cube Cutting Single Node Cube with Low Accuracy Multiple Node Cube with High Accuracy

20. EMINC: Cube Methods Random cuttingEqual cuttingClustering-based cutting

21. Outline Motivation Query Answering on the Cloud Related Work EMINC: Index the Cloud Efficiently ◦ Node Bounding ◦ Extended Node Bounding ◦ Cost Estimation based Index Update Evaluation Conclusion & Future Work

22. EMINC: Index Update Strategy Index update issues: ◦ Cubes may invalidate themselves after certain data update, thus need reconstruction Insertion invalidates cube ◦ Create a node cube containing new data For regular maintenance of index ◦ Cost estimation based update strategy

23. EMINC: Cost Estimation Strategy Cost of index update: ◦ Recalculate cubes on local node ◦ Transfer to master node and maintain R-Tree ◦ Query performance will be affected Benefit of index update: ◦ More accurate query directing, less waste

24. EMINC: Two Phase Method After one update: 1.Wait for a time period of deltaT 2.deltaT expires, check if an update is needed Determin deltaT Check for update Assumption : Number of queries to be processed Total size of node cubes of this node

25. EMINC: Phase One After pervious update: benefit = decrement-of-query/time* deltaT ◦ We enjoy the benefit of pervious update for deltaT time period cost = number-of-queries missed ◦ Number of queries we could process if we use pervious update time to answer queries

26. EMINC: Phase Two benefit > cost => deltaT After deltaT expires, check if an update is needed. This check involves following: ◦ Record update frequency ◦ Expected benefit ratio ◦ Performance requirement We leave this as future work

27. Outline Motivation Query Answering on the Cloud Related Work EMINC: Index the Cloud Efficiently ◦ Node Bounding ◦ Extended Node Bounding ◦ Cost Estimation based Index Update Evaluation Conclusion & Future Work

28. Evaluation 6 machines ◦ 1 as master node ◦ 5 slave nodes simulating 100~1000 nodes Each machine had a 2.33GHz Intel Core2 Quad CPU, 4GB of main memory, and a 320G disk. Machines ran Ubuntu 9.04 Server OS.

29. Evaluation: Point Query

30. Evaluation: Range Query

31. Outline Motivation Query Answering on the Cloud Related Work EMINC: Index the Cloud Efficiently ◦ Node Bounding ◦ Extended Node Bounding ◦ Cost Estimation based Index Update Evaluation Conclusion & Future Work

32. Conclusion In this paper we presented a series of approaches on building efficient multi- dimensional index on Cloud platform. We developed the node bounding technique to reduce query processing cost on the cloud platform. In order to maintain efficiency of the index, we proposed a cost estimation- based approach for index update.

33. Future Work Complete cost estimation model Take replication of data into consideration Implement in Hbase to further verify performance

34. Thanks Please visit our lab for more information: http://idke.ruc.edu.cn/