1. Scaleout vs. Scaleup Robert Barnes Microsoft
11/22/2018
HPTS'99 Scale Out vs Scale Up

2. HPTS'99 Scale Out vs Scale Up
Why 1BTPD?
We had two main obstacles in bringing TP to Microsoft:
Lack of products and infrastructure
Industry mindshare and/or credibility
After starting to address the first obstacle, we decided to build something to demonstrate Microsoft has a credible platform for large-scale applications
CPU
50 GB Disc
5 GB RAM
11/22/2018
HPTS'99 Scale Out vs Scale Up

3. HPTS'99 Scale Out vs Scale Up
Why 1BTPD?
We liked Jim Gray’s “cyber-brick” machine and wanted to build one - only the technology wasn’t there yet
The closest thing to it was a collection of laptops - so we wanted to tie together 1000 laptops. We decided DebitCredit would be a good workload to run, and 10KTPS* DebitCredit was born
*10 K Transactions per Second
CPU
50 GB Disc
5 GB RAM
11/22/2018
HPTS'99 Scale Out vs Scale Up

4. HPTS'99 Scale Out vs Scale Up
11/22/2018
HPTS'99 Scale Out vs Scale Up

5. HPTS'99 Scale Out vs Scale Up
Why DebitCredit?
We needed a DB-only benchmark
Do not want to configure one million terminals
A simple transaction profile to stress DTC
Many little transactions
Need distributed transactions to stress DTC
TPC-B is dead (can not report numbers)
TPC-C is not server-only (need 1 million terminals) and requires distributed data transparency
So, we returned to the Datamation Benchmark (1985)
11/22/2018
HPTS'99 Scale Out vs Scale Up

6. HPTS'99 Scale Out vs Scale Up
10KTPS  1BTPD
Work began on the benchmark in mid 1995
A vendor released a 30K TPM-C number
Even though 10KTPS is 600Ktpm and DebitCredit is very different from TPC-C transactions, we needed a different metric
tps = tpm = tph = 1 Billion Transactions Per Day*
*thanks, Jim
11/22/2018
HPTS'99 Scale Out vs Scale Up

7. HPTS'99 Scale Out vs Scale Up
Scaling
11/22/2018
HPTS'99 Scale Out vs Scale Up

8. Scaleable Systems SMP and Loosely Coupled Systems
Scaleup with SMP 8P is new standard (SHV)
Scaleout with Loosely Coupled Systems
(LCS) using inexpensive parts
SMP
Super Server
Departmental
Server
LCS
of PCs
or SHVs
Personal
System
11/22/2018
HPTS'99 Scale Out vs Scale Up

9. HPTS'99 Scale Out vs Scale Up
SMP Advantages
Single system image
no change to operations (relative to uniprocessor)
no change to applications
Simple system resources
shared memory
shared disk
shared net
Load balancing in OS kernel
8x SMP soon commodity SHV
SMP
Super Server
Departmental
Server
Personal
System
11/22/2018
HPTS'99 Scale Out vs Scale Up

10. HPTS'99 Scale Out vs Scale Up
SMP Problems
Problems:
More then 8 processors not a commodity today
More than 16 processors may require partitioning, scale out needed anyway
scale-down problem (starter systems expensive or fork lift upgrade)
Potential single point of failure
Eventually stops scaling
SMP
Super Server
Departmental
Server
Personal
System
11/22/2018
HPTS'99 Scale Out vs Scale Up

11. SMP Performance Diminishing returns
Linear extrapolation for 16
11/22/2018
HPTS'99 Scale Out vs Scale Up

12. HPTS'99 Scale Out vs Scale Up
LCS Advantages
Advantages:
No hardware limit to scale, given a scaleable interconnect and an application that partitions
Uses high volume,commodity components
No single point of failure
Upgrade by incremental growth
Balancing
Load
Clients and Servers made from the same stuff.
Inexpensive:
Built with commodity components
Fault tolerance:
Spare modules mask failures
Modular growth
grow by adding small modules
Common Function
11/22/2018
HPTS'99 Scale Out vs Scale Up

13. HPTS'99 Scale Out vs Scale Up
LCS Problems
Operations complexity
When not designed for LCS, costs can be very expensive
Change in system architecture; e.g. Exchange, SQL Server (relative to uniprocessor)
Need additional system services
Load Balancing
LCS Membership
Configuration replication
Failure Retry (within LCS)
Parallelism needed for data and applications
explicit parallelism exposes partitions to the application
implicit parallelism requires transparency
Balancing
Load
Clients and Servers made from the same stuff.
Inexpensive:
Built with commodity components
Fault tolerance:
Spare modules mask failures
Modular growth
grow by adding small modules
Common Function
11/22/2018
HPTS'99 Scale Out vs Scale Up

14. HPTS'99 Scale Out vs Scale Up
givememoney.com
The givememoney.com business is brokering – their goal is to not directly handle any products
Site is currently <10 stores, plan to be hundreds in a few years
Books, music, video,computers,software,games,surplus
It takes over 90 days to add a new store
Over 90 servers make up current site
Today, checkouts/day 10K
Business plan calls for 500K by 2002
11/22/2018
HPTS'99 Scale Out vs Scale Up

15. HPTS'99 Scale Out vs Scale Up
givememoney.com FE 5 2 5 2 2
Cache Server
ASP
SSL
ASP
SSL
FARM B
FARM A
Basket/Ad/Surplus
ASP File Server
SQL Product Server
ASP File Server
SQL Product Server
Receipt/Fulfillment
Games/Music
Videos
Games/Music
Videos
Monitor and cache
Comp/Soft
Books
Comp/Soft
Books
Music
Music
Search Servers
Search Servers
11/22/2018
HPTS'99 Scale Out vs Scale Up

16. HPTS'99 Scale Out vs Scale Up
givememoney.com
“We don’t have a scaling problem – if we need more capacity, we just add a server. We have a management problem…”
11/22/2018
HPTS'99 Scale Out vs Scale Up

17. Scale Out Business Need
givememoney.com is at 10K orders/day, business plan is to grow to 500K by 2002
Moore’s law says scale improves by 4X in 3 years, givememoney.com wants 50X , over 10X Moore’s law.
Even if you believe Moore’s law solves the growth problem, it requires many forklift upgrades
We still have to scale up – as it minimizes the number of servers needed…
Fantasy element…..
11/22/2018
HPTS'99 Scale Out vs Scale Up

18. HPTS'99 Scale Out vs Scale Up
Consolidation
Not consolidation vs. scale out
Consolidation motivations
Business
Site (Space)
Operations
Storage
Reasons for never wanting to have 1 huge system
Single point of failure
Dis-economies of scale up
Mixed workload has operations challenges similar to scale out
Difficulty of dealing with growth
11/22/2018
HPTS'99 Scale Out vs Scale Up

19. Scaleout is a software challenge
Scaleup is hardware architecture & OEM driven
Optimized by Software
e.g. minimize locking, parallel threads, reduce contention on critical sections, etc.
Ultimately, scale up begins to look like scale out (partitioning)
Scaleout is software architecture driven
Optimized by hardware
e.g. SAN (system area nets), low latency/high bandwidth interconnects
Primarily an application design and operational management problem
Possible today, but mostly left to application design/development and ad hoc management tools
11/22/2018
HPTS'99 Scale Out vs Scale Up

20. HPTS'99 Scale Out vs Scale Up
Summary
Scaleup begins to look like scaleout as you add processors
Scaleout with commodity hardware is economically compelling
Every large successful web site uses scaleout – it is practical, but hard
Operations complexity is the key barrier to scaleout
11/22/2018
HPTS'99 Scale Out vs Scale Up

21. HPTS'99 Scale Out vs Scale Up
CyberWall
11/22/2018
HPTS'99 Scale Out vs Scale Up