What is it? What kind of system need it?. Distributing system, cloud system etc.

Slides:



Advertisements
Similar presentations
Revisiting Co-Processing for Hash Joins on the Coupled CPU- GPU Architecture School of Computer Engineering Nanyang Technological University 27 th Aug.
Advertisements

LIBRA: Lightweight Data Skew Mitigation in MapReduce
Daniel Schall, Volker Höfner, Prof. Dr. Theo Härder TU Kaiserslautern.
SLA-Oriented Resource Provisioning for Cloud Computing
Packet Switching COM1337/3501 Textbook: Computer Networks: A Systems Approach, L. Peterson, B. Davie, Morgan Kaufmann Chapter 3.
EHarmony in Cloud Subtitle Brian Ko. eHarmony Online subscription-based matchmaking service Available in United States, Canada, Australia and United Kingdom.
Autonomic Systems Justin Moles, Winter 2006 Enabling autonomic behavior in systems software with hot swapping Paper by: J. Appavoo, et al. Presentation.
Chapter 19: Network Management Business Data Communications, 5e.
Building a Distributed Full-Text Index for the Web S. Melnik, S. Raghavan, B.Yang, H. Garcia-Molina.
Query Processing of Massive Trajectory Data based on MapReduce Qiang Ma, Bin Yang (Fudan University) Weining Qian, Aoying Zhou (ECNU) Presented By: Xin.
 Need for a new processing platform (BigData)  Origin of Hadoop  What is Hadoop & what it is not ?  Hadoop architecture  Hadoop components (Common/HDFS/MapReduce)
1 Minggu 12, Pertemuan 23 Introduction to Distributed DBMS (Chapter , 22.6, 3rd ed.) Matakuliah: T0206-Sistem Basisdata Tahun: 2005 Versi: 1.0/0.0.
Self-Correlating Predictive Information Tracking for Large-Scale Production Systems Zhao, Tan, Gong, Gu, Wambolt Presented by: Andrew Hahn.
Chapter Physical Database Design Methodology Software & Hardware Mapping Logical Design to DBMS Physical Implementation Security Implementation Monitoring.
70-290: MCSE Guide to Managing a Microsoft Windows Server 2003 Environment Chapter 11: Monitoring Server Performance.
1 Porcupine: A Highly Available Cluster-based Mail Service Yasushi Saito Brian Bershad Hank Levy University of Washington Department of Computer Science.
1 Optimizing Utility in Cloud Computing through Autonomic Workload Execution Reporter : Lin Kelly Date : 2010/11/24.
5 Creating the Physical Model. Designing the Physical Model Phase IV: Defining the physical model.
Chapter 1 An Overview of Database Management. 1-2 Topics in this Chapter What is a Database System? What is a Database? Why Database? Data Independence.
Centralized and Client/Server Architecture and Classification of DBMS
CS 443 Advanced OS Fabián E. Bustamante, Spring 2005 Porcupine: A Highly Available Cluster- based Mail Service Y. Saito, B. Bershad, H. Levy U. Washington.
Module 14: Scalability and High Availability. Overview Key high availability features available in Oracle and SQL Server Key scalability features available.
VIRTUALISATION OF HADOOP CLUSTERS Dr G Sudha Sadasivam Assistant Professor Department of CSE PSGCT.
Simplify your Job – Automatic Storage Management Angelo Session id:
Jiazhang Liu;Yiren Ding Team 8 [10/22/13]. Traditional Database Servers Database Admin DBMS 1.
Measuring zSeries System Performance Dr. Chu J. Jong School of Information Technology Illinois State University 06/11/2012 Sponsored in part by Deer &
Applying Twister to Scientific Applications CloudCom 2010 Indianapolis, Indiana, USA Nov 30 – Dec 3, 2010.
Database Architectures and the Web
Department of Computer Science Engineering SRM University
Virtual Machine Hosting for Networked Clusters: Building the Foundations for “Autonomic” Orchestration Based on paper by Laura Grit, David Irwin, Aydan.
CSC271 Database Systems Lecture # 30.
The McGraw-Hill Companies, Inc Information Technology & Management Thompson Cats-Baril Chapter 3 Content Management.
Map Reduce for data-intensive computing (Some of the content is adapted from the original authors’ talk at OSDI 04)
Task Scheduling for Highly Concurrent Analytical and Transactional Main-Memory Workloads Iraklis Psaroudakis (EPFL), Tobias Scheuer (SAP AG), Norman May.
HBase A column-centered database 1. Overview An Apache project Influenced by Google’s BigTable Built on Hadoop ▫A distributed file system ▫Supports Map-Reduce.
Cloud Computing Energy efficient cloud computing Keke Chen.
Your Data Any Place, Any Time Online Transaction Processing.
© Pearson Education Limited, Chapter 16 Physical Database Design – Step 7 (Monitor and Tune the Operational System) Transparencies.
70-290: MCSE Guide to Managing a Microsoft Windows Server 2003 Environment, Enhanced Chapter 11: Monitoring Server Performance.
1 Moshe Shadmon ScaleDB Scaling MySQL in the Cloud.
 Mainak Ghosh, Wenting Wang, Gopalakrishna Holla, Indranil Gupta.
Oracle9i Performance Tuning Chapter 1 Performance Tuning Overview.
Join Synopses for Approximate Query Answering Swarup Achrya Philip B. Gibbons Viswanath Poosala Sridhar Ramaswamy Presented by Bhushan Pachpande.
Scalable Web Server on Heterogeneous Cluster CHEN Ge.
H-Store: A Specialized Architecture for High-throughput OLTP Applications Evan Jones (MIT) Andrew Pavlo (Brown) 13 th Intl. Workshop on High Performance.
CEPH: A SCALABLE, HIGH-PERFORMANCE DISTRIBUTED FILE SYSTEM S. A. Weil, S. A. Brandt, E. L. Miller D. D. E. Long, C. Maltzahn U. C. Santa Cruz OSDI 2006.
EFFECTIVE LOAD-BALANCING VIA MIGRATION AND REPLICATION IN SPATIAL GRIDS ANIRBAN MONDAL KAZUO GODA MASARU KITSUREGAWA INSTITUTE OF INDUSTRIAL SCIENCE UNIVERSITY.
Intro – Part 2 Introduction to Database Management: Ch 1 & 2.
CPU Scheduling Presentation by Colin McCarthy. Runqueues Foundation of Linux scheduler algorithm Keeps track of all runnable tasks assigned to CPU One.
70-290: MCSE Guide to Managing a Microsoft Windows Server 2003 Environment, Enhanced Chapter 11: Monitoring Server Performance.
CCNA 2 Week 6 Routing Protocols. Copyright © 2005 University of Bolton Topics Static Routing Dynamic Routing Routing Protocols Overview.
Online Data partitioning in distributed database systems
Big traffic data processing framework for intelligent monitoring and recording systems 學生 : 賴弘偉 教授 : 許毅然 作者 : Yingjie Xia a, JinlongChen a,b,n, XindaiLu.
1EMC CONFIDENTIAL—INTERNAL USE ONLY IMPLEMENTING DEGRADED MODE.
1 Chapter 22 Distributed DBMS Concepts and Design CS 157B Edward Chen.
Unit 4: Processes, Threads & Deadlocks June 2012 Kaplan University 1.
Load Rebalancing for Distributed File Systems in Clouds.
Author: Akiyoshi Matonoy, Toshiyuki Amagasay, Masatoshi Yoshikawaz, Shunsuke Uemuray.
Em Spatiotemporal Database Laboratory Pusan National University File Processing : Database Management System Architecture 2004, Spring Pusan National University.
E-Store: Fine-Grained Elastic Partitioning for Distributed Transaction Processing Systems Jihui Yang CS525 Advanced Distributed System March 1, 2016.
INTRODUCTION TO GRID & CLOUD COMPUTING U. Jhashuva 1 Asst. Professor Dept. of CSE.
Cofax Scalability Document Version Scaling Cofax in General The scalability of Cofax is directly related to the system software, hardware and network.
Elasticity in SciDB DBMS Team Members Gunjan Sharma(MT15015) Hiya popli(MT15020)
Optimizing Distributed Actor Systems for Dynamic Interactive Services
Introduction to Distributed Platforms
Windows Azure SQL Federation
Curator: Self-Managing Storage for Enterprise Clusters
Flash Storage 101 Revolutionizing Databases
Building an Elastic Main-Memory Database: E-Store
Specialized Cloud Architectures
Presentation transcript:

What is it? What kind of system need it?

Distributing system, cloud system etc

High availability, high performance, elasticity

On demand processing, storage and network resources are provided

What is Elasticity?

An open challenge and a topic of many recent research

An elastic system that not only adds and removes nodes, but also reconfigures them in a heterogeneous manner according to the workload’s access patterns.(HBase)

Normally a manual task, this paper help to do autonomous elasticity of NoSQL

MET: Elastic System heterogeneously reconfigures nodes according to the observed workload

Algorithm detail in paper

An elastic partitioning framework for distributed OLTP DBMSs.

Serve time-varing workload due to daily, weekly or seasonal difference in demand, or because of rapid growth in demand due to a company’s business success. Many OLTP workload are heavily skewed to : hot: tuples o ranges of tuples

It automatically scales resources in response to demand spikes, periodic events, and gradual changes in an application’s workload.

Two-tier data placement strategy: cold data is distributed in large chunk, while smaller ranges of hot tuples are assigned explicitly to individual nodes.

All non-replicated tables of an OLTP database form a tree-schema based on foreign key relationship

Problem can be broken into three parts

Data migration, two tier partition.

Installed on every DBMS node in the cluster

Keys are extracted from their block and allocated to nodes individually. Participate hot keys separately from cold ranges.

Standalong program running continuously outside of the DBMS.

Detect imbalance by using CPU utilization in a main memory of DBMS.

After a brief collection period, E-Monitor switches back to lightweight mode and sends the data collected during this phase to E-Planner to generate a migration plan for the DBMS.

Monitoring only the root tuples provides a good approximation of system activity and minimizes the overhead of this phase.

Detail in the paper