1. Self-Tuning Memory Management of A Database System
Yixin Diao
Sigmetrics 2008 Tutorial: Introduction to Control Theory and Its Application to Computing Systems

2. DB2 Self-Tuning Memory Management
DB2 UDB Server
Technical problems
Large systems with varying workloads and many configuration parameters
Autonomic computing: systems self-management
Memory pools
DB2
Clients
Agents
Disks
Memory pools
Challenges from systems aspects
Heterogeneous memory pools
Dissimilar usage characteristics
Challenges from control aspects
Adaptation and self-design
Reliability and robustness
SIGMETRICS 2008: Introduction to Control Theory. Abdelzaher, Diao, Hellerstein, Lu, and Zhu.

3. Load Balancing for Database Memory
Resource
Consumer 1
Consumer N
Load
Balancer
Measured Output N
Measured Output 1
Resource Allocation 1
Resource Allocation N
Load Balancing
Fairness  optimal ?
Common measured output ?
Saved System Time (xi )
simPages
savedTime
BenefitPerPage (yi )
Memory Pool Size (ui )
1000
2000
3000
4000
5000
0.02
0.04
0.06
0.08
0.1
0.12
0.14
0.16
Entry size (Page)
Benefit (sec/page)
OLTP
SIGMETRICS 2008: Introduction to Control Theory. Abdelzaher, Diao, Hellerstein, Lu, and Zhu.

4. Constrained Optimization and Regulatory Control
Saved Disk Time ( xi )
Saved System Time (xi )
Regulatory Control
Overall
Mem pool 1 (x1)
BenefitPerPage (y1)
Mem pool 2 (x2)
Mem size 1 (u1)
MemoryPool1
Mem size 2 (u2)
Optimal memory allocation
Constrained Optimization
Karush-Kuhn-Tucker conditions
SIGMETRICS 2008: Introduction to Control Theory. Abdelzaher, Diao, Hellerstein, Lu, and Zhu.

5. Dynamic State Feedback Controller
State space model
Control error
Integral control error
Feedback control law
SIGMETRICS 2008: Introduction to Control Theory. Abdelzaher, Diao, Hellerstein, Lu, and Zhu.

6. Major cost: write dirty, move memory, victimize hot
Incorporating Const of Control into Controller Design
Remove these pages
Pool Size
Benefit
Ts=12449
Ts=15703
Ts=24827
Memory Pool A
before
Disk
write dirty pages to disk
after
Memory Pool B
before OS
allocate extra memory
Major cost: write dirty, move memory, victimize hot
Linear quadratic regulation (LQR)
J =  [eT(k) eTI(k)] Q [eT(k) eTI(k)]T
+ uT(k) R u(k)
Define Q and R regarding to performance
Cost of transient load imbalances
Cost of changing resource allocations
SIGMETRICS 2008: Introduction to Control Theory. Abdelzaher, Diao, Hellerstein, Lu, and Zhu.

7. Adaptive Controller Design
Local linear model
Decentralized integral control
DB2 Memory Pool
DB2 Clients
Memory
Statistics
Collector
Response Time Benefit
MIMO
Control Algorithm
Fixed
Step
4-Bit
(Oscillation)
Model
Builder
Accurate
Interval
Tuner Y N
Entry Size
Step Tuner
Greedy
(Constraint)
SIGMETRICS 2008: Introduction to Control Theory. Abdelzaher, Diao, Hellerstein, Lu, and Zhu.

8. Experimental Assessment
squid.torolab.ibm.com
Machine: IBM7028-6C4
CPU: 4x 1453MHz
Memory: 16GB
Disk: x 9.1G
Experimental Assessment
OLTP workload: multiple (20) buffer pools
Response time benefits
Memory sizes
Increase TP from ~100 to ~250
Throughput
DSS workload: various query lengths
STMM tuning
Ts = 10680s
ConfigAdvisor settings
Ts = 26342s
> 2x improvement
DSS workload: index drop
Execution time for Query 21 (10 stream avg)
1000
2000
3000
4000
5000
6000
7000 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34
Order of execution
Time in seconds
avg
= 959
= 2285
= 6206
Some indexes dropped
Reduce 63%
SIGMETRICS 2008: Introduction to Control Theory. Abdelzaher, Diao, Hellerstein, Lu, and Zhu.

9. Comparing Control and Optimization Techniques
Control-based approach
Optimization-based approach
Local linear model
Gradient method
Decentralized integral control
Projected gradient (quasi-Newton)
Constraint enforcement (projection method)
Step length (modified Armijo rule)
Strictly applies constrained optimization
Less dependence on the model
Similarity in a simplified scenario
Differences in design considerations
“Pure” average vs. convex sum
Pole location vs. Armijo rule
Steady-state gain vs. Hessian matrix
SIGMETRICS 2008: Introduction to Control Theory. Abdelzaher, Diao, Hellerstein, Lu, and Zhu.

10. Simulation Study: Comparison with Optimization Approach
Control-based approach
Optimization-based approach
Memory size
WL change
Without noise (single run)
Total saved time
Control intervals
Effect of noise (multiple runs)
More robust and better uncertainty management
Faster convergence, but more sensitive to noise
SIGMETRICS 2008: Introduction to Control Theory. Abdelzaher, Diao, Hellerstein, Lu, and Zhu.

11. Summary DB2 self-tuning memory management
Interconnection, heterogeneity, adaptation and robustness, cost of control
Constrained optimization with a linear feedback controller
Experimental assessment for OLTP and DSS workloads
SIGMETRICS 2008: Introduction to Control Theory. Abdelzaher, Diao, Hellerstein, Lu, and Zhu.