Increasing Performance with Nightly Inserts Mike Rawding

Setting It is common to have access to a small amount of fast storage and a large amount of slow storage RAID RamDisks SSDs Networks Assumption for this project The available fast storage is large enough to store the new insertions from one business day It is not large enough to maintain entire database

Goals Avoid writing to slow storage during the work day Have up-to-date access to information in the databases Seamless implementation User’s interactions with the database are unchanged

Schema transactions transactions transactions_main transactions_aux id custFname custLname empID itemCode price date image Int PK Varchar Int Double Datetime BLob transactions_main id custFname custLname empID itemCode price date image Int PK Varchar Int Double Datetime BLob transactions_aux id custFname custLname empID itemCode price date image Int PK Varchar Int Double Datetime BLob

Implementation transactions = transactions_main + transactions_aux 30MB/s 310MB/s

Tools SQLAPI++ Library MySQL Connector http://www.sqlapi.com/ https://www.mysql.com/products/connector/

Insert

Insert

Select

Update

Update

Update

Delete

Delete

Delete

Transfer

Transfer

Transfer

Transfer

Results Insertion times were no better when using 8MB Blobs Average insertion time to SSD = 1.2s Average insertion time to HDD = 1.2s Insertion times were reduced by 22% when using 32MB Blobs Average insertion time to SSD = 4.7s Average insertion time to HDD = 6.0s Select, Update, and Delete showed not appreciable difference

Code http://pastebin.com/e0CrDnvg https://github.com/MikeRawding/541project.git