1. Progress Report
Chia-Lun Wu

2. Simulate client/server environment
2 VM (server/client)
Python script to simulate client
n users -> n threads
Call bosc_test binary
N users, each client issue M requests with K records

3. Experiment Settings Parameters
BOSC_SCAN_THREESHOLD (10000, )
IO_thread max sleep time (60s, 30s)
50 users issue 1000 requests with 10 records
Monitor throughput overtime
Random + update intensive workload
50% insert, 50% modify

4. Experiment Results When flushing, throughput degrades
Flushing to disk can’t keep up with memory updates
After flushing, dirty blocks get more (ex: > flush -> 86794)
Each user sleep 5s between requests -> dirty blocks get more later
End up flushing all the time
Need throttling mechanism to limit the rate of memory updates ?
Flushing to disk is slow
Real-time ingestion of high-frequency small-sized sensor data streams
Streaming random disk writes: > records per second per server

5. Experiment Results SCAN_THRESHOLD = 10000 SLEEP = 60s No flush
(Throughput)

6. Experiment Results
SCAN_THRESHOLD = 10000
SLEEP = 60s
Flush

7. Experiment Results
SCAN_THRESHOLD =
SLEEP = 60s
No flush

8. Experiment Results SCAN_THRESHOLD = 100000 SLEEP = 30s No flush

9. Experiment Results SCAN_THRESHOLD = 10000 SLEEP = 60s
100 Client sleep 5 s every request
Experiment Results

10. Experiment Results SCAN_THRESHOLD = 10000 SLEEP = 60s
100 Client sleep 5 s every request
Experiment Results
Dirty blocks still get more
after flushing
Flush
Flush
Flush
Flush
Flush

11. Experiment Results (50 users issue 1000 requests with 10 records)
(Average Throughput)