Download presentation
Presentation is loading. Please wait.
Published byGarry Sparks Modified over 9 years ago
1
Jason Baker, Chris Bond, James C. Corbett, JJ Furman, Andrey Khorlin, James Larson,Jean Michel L´eon, Yawei Li, Alexander Lloyd, Vadim Yushprakh Megastore Scalable Highly Available Storage for Interactive Systems Presented By: Hamid Seyedmoradi Ayoub Hamidi Ehsan Mohamad Nezamian Advanced Database Systems SRBIAU, Kurdistan Campus 10May2012
2
Megastore Motivation Introduction NoSQL & RDBMS Megastore Paxos SRBIAU, Kurdistan Campus Advanced Database Systems2
3
Megastore wow! more than 3 billion write and 20 billion read daily key Contribution Data Model and Storage System Paxos Replication Report on Experience SRBIAU, Kurdistan Campus Advanced Database Systems3
4
AVAILABILITY & SCALE Replication For Availability, we implemented a synchronous, fault tolerant log replicator optimized for long distance links Partitioning and Locality for scale, we partitioned data into a vast space of small databases SRBIAU, Kurdistan Campus Advanced Database Systems4
5
AVAILABILITY & SCALE Replication Strategies Asynchronous Master/Slave Synchronous Master/Slave Optimistic Replication We decided to use Paxos SRBIAU, Kurdistan Campus Advanced Database Systems5
6
Technology Options SRBIAU, Kurdistan Campus Advanced Database Systems 6
7
7 Technology Options
8
AVAILABILITY & SCALE Partitioning and Locality Replication SRBIAU, Kurdistan Campus Advanced Database Systems8 Datacenters Entity Groups Partition the datastore Each entity group is synchronously replicated across datacenters ACID semantics within an entity group Looser consistency across entity groups Entity group data and replication metadata stored in scalable NoSQL datastores
9
AVAILABILITY & SCALE Partitioning and Locality Operations: SRBIAU, Kurdistan Campus Advanced Database Systems9 Entities (Units of data) Entity Group 1 Most transactions are within a single entity group Cross Entity group transactions supported via Two – Phase Commit Asynch communication between entity groups supported by Queues Global Indexes span entity groups but have weaker consistency Entity Group 2 Local Index Send queue receive
10
AVAILABILITY & SCALE Partitioning and Locality Entity Groups Selecting Entity Group Boundaries Example Email Blogs Physical Layout SRBIAU, Kurdistan Campus Advanced Database Systems10
11
Megastore API Design Philosophy Data Model Pre-Joining with Keys SCATTER Indexes Storing Clause Repeated Indexes. Inline Indexes Mapping to Bigtable SRBIAU, Kurdistan Campus Advanced Database Systems11
12
Megastore SRBIAU, Kurdistan Campus Advanced Database Systems12
13
Megastore Transactions and Concurrency Control Read current snapshot inconsistent Transaction Lifecycle 1-Read 3-Commit 5-Clean up 2-Application logic 4-Apply SRBIAU, Kurdistan Campus Advanced Database Systems13
14
Megastore Queues Two Phase Commit SRBIAU, Kurdistan Campus Advanced Database Systems14
15
REPLICATION Brief Summary of Paxos Megastore’s Approach Fast Reads Fast Writes Replica Types Witness Replica Architecture SRBIAU, Kurdistan Campus Advanced Database Systems15
16
Architecture SRBIAU, Kurdistan Campus Advanced Database Systems16
17
Data Structures and Algorithms Replicated Logs SRBIAU, Kurdistan Campus Advanced Database Systems17
18
Data Structures and Algorithms Reads Query Local Find Position Local read Majority read Catchup Validate Query Data SRBIAU, Kurdistan Campus Advanced Database Systems18
19
Data Structures and Algorithms SRBIAU, Kurdistan Campus Advanced Database Systems19
20
Data Structures and Algorithms Writes Accept Leader Prepare Accept Invalidate Apply SRBIAU, Kurdistan Campus Advanced Database Systems20
21
Feedback SRBIAU, Kurdistan Campus Advanced Database Systems21
22
END With Thanks Question ? SRBIAU, Kurdistan Campus Advanced Database Systems22
Similar presentations
© 2024 SlidePlayer.com. Inc.
All rights reserved.