1. PNUTS: Yahoo!’s Hosted Data Serving Platform
Brian F. Cooper, Raghu Ramakrishnan, Utkarsh Srivastava, Adam Silberstein, Philip Bohannon, Hans-Arno Jacobsen, Nick Puz, Daniel Weaver and Ramana Yerneni
Yahoo! Research

2. How do I build a cool new web app?
Option 1: Code it up! Make it live!
Scale it later
It gets posted to slashdot
Scale it now!
Flickr, Twitter, MySpace, Facebook, …

3. How do I build a cool new web app?
Option 2: Make it industrial strength!
Evaluate scalable database backends
Evaluate scalable indexing systems
Evaluate scalable caching systems
Architect data partitioning schemes
Architect data replication schemes
Architect monitoring and reporting infrastructure
Write application
Go live
Realize it doesn’t scale as well as you hoped
Rearchitect around bottlenecks
1 year later – ready to go!

4. Example: social network updates
Brian
Sonja
Jimi
Brandon
Kurt
What are my friends up to?
Sonja:
Brandon:

5. Example: social network updates
6 Jimi <ph..
8 Mary <re..
12 Sonja <ph..
15 Brandon <po..
<photo>
<title>Flower</title>
<url>
</photo>
16 Mike <ph..
17 Bob <re..

6. What do we need from our DBMS?
Web applications need:
Scalability
And the ability to scale linearly
Geographic scope
High availability
Web applications typically have:
Simplified query needs
No joins, aggregations
Relaxed consistency needs
Applications can tolerate stale or reordered data

7. What is PNUTS?

8. What is PNUTS? Indexes and views CREATE TABLE Parts ( ID VARCHAR,
E C
A E
B W
C W
D E
F E
A E
B W
C W
D E
E C
F E
Indexes and views
CREATE TABLE Parts (
ID VARCHAR,
StockNumber INT,
Status VARCHAR … )
E C
A E
B W
C W
D E
F E
Geographic replication
Parallel database
Structured, flexible schema
Hosted, managed infrastructure

9. Query model Per-record operations Multi-record operations
Get
Set
Delete
Multi-record operations
Multiget
Scan
Getrange
Web service (RESTful) API

10. Detailed architecture
Clients
Data-path components
REST API
Routers
Message
Broker
Tablet
controller
Storage units

11. Detailed architecture
Local region
Remote regions
Clients
REST API
Routers
YMB
Tablet controller
Storage
units

12. Tablet splitting and balancing
Each storage unit has many tablets (horizontal partitions of the table)
Storage unit may become a hotspot
Storage unit
Tablet
Overfull tablets split
Tablets may grow over time
Shed load by moving tablets to other servers

13. Query processing

14. Range queries Grapefruit…Pear? Storage unit 1 Storage unit 2
Apple
Avocado
Banana
Blueberry
SU1
Strawberry-MAX
SU2
Lime-Strawberry
SU3
Canteloupe-Lime
MIN-Canteloupe
MIN-Canteloupe
SU1
Canteloupe-Lime
SU3
Lime-Strawberry
SU2
Strawberry-MAX
Grapefruit…Pear?
Grapefruit…Lime?
Lime…Pear?
Canteloupe
Grape
Kiwi
Lemon
Router
Lime
Mango
Orange
Storage unit 1
Storage unit 2
Storage unit 3
Strawberry
Tomato
Watermelon

15. Updates SU SU SU Routers Message brokers 1 8 Sequence # for key k
Write key k
Routers
Message brokers 2
Write key k 3
Write key k 7 4
Sequence # for key k 5
SUCCESS SU SU SU 6
Write key k

16. Asynchronous replication and consistency

17. Asynchronous replication

18. Consistency model
Goal: make it easier for applications to reason about updates and cope with asynchrony
What happens to a record with primary key “Brian”?
Record inserted
Update
Update
Update
Update
Update
Update
Update
Delete
v. 1
v. 2
v. 3
v. 4
v. 5
v. 6
v. 7
v. 8
Time
Time
Generation 1

19. Consistency model Read Stale version Stale version Current version
Time
Generation 1

20. Consistency model Read up-to-date Stale version Stale version
Current version
v. 1
v. 2
v. 3
v. 4
v. 5
v. 6
v. 7
v. 8
Time
Generation 1

21. Consistency model Read ≥ v.6 Stale version Stale version
Current version
v. 1
v. 2
v. 3
v. 4
v. 5
v. 6
v. 7
v. 8
Time
Generation 1

22. Consistency model Write Stale version Stale version Current version
Time
Generation 1

23. Consistency model Write if = v.7 Stale version Stale version
ERROR
Stale version
Stale version
Current version
v. 1
v. 2
v. 3
v. 4
v. 5
v. 6
v. 7
v. 8
Time
Generation 1

24. Consistency model Mechanism: per record mastership Write if = v.7
ERROR
Stale version
Stale version
Current version
v. 1
v. 2
v. 3
v. 4
v. 5
v. 6
v. 7
v. 8
Time
Generation 1

25. Experiments

26. Experimental setup Production PNUTS code Three PNUTS regions Workload
Enhanced with ordered table type
Three PNUTS regions
2 west coast, 1 east coast
5 storage units, 2 message brokers, 1 router
West: Dual 2.8 GHz Xeon, 4GB RAM, 6 disk RAID 5 array
East: Quad 2.13 GHz Xeon, 4GB RAM, 1 SATA disk
Workload
requests/second
0-50% writes
80% locality

27. Scalability

28. Request skew

29. Size of range scans

30. Related work Distributed and parallel databases
Especially query processing and transactions
BigTable, Dynamo, S3, SimpleDB, SQL Server Data Services, Cassandra
Distributed filesystems
Ceph, Boxwood, Sinfonia
Distributed (P2P) hash tables
Chord, Pastry, …
Database replication
Master-slave, epidemic/gossip, synchronous…

31. Conclusions and ongoing work
PNUTS is an interesting research product
Research: consistency, performance, fault tolerance, rich functionality
Product: make it work, keep it (relatively) simple, learn from experience and real applications
Ongoing work
Indexes and materialized views
Bundled updates
Batch query processing

32. Thanks!
research.yahoo.com