1. MySQL Users Conf.-1 04-27-2006 MIT Lincoln Laboratory Measuring MySQL Server Performance for Sensor Data Stream Processing Jacob Nikom MIT Lincoln Laboratory The MySQL Users Conference 2006 27 April 2006 This work was sponsored by the U.S. Army Space and Missile Defense Command under Air Force Contract# FA8721-05-C-0002. Opinions, interpretations, recommendations and conclusions are that of the author and are not necessarily endorsed by the United States Government.

2. MIT Lincoln Laboratory MySQL Users Conf-2 04-27-2006 Outline Introduction Benchmark for real-time streamed data recording Benchmark for real-time streamed events processing Summary

3. MIT Lincoln Laboratory MySQL Users Conf-3 04-27-2006 Outline Introduction –Reagan Test Site (RTS) and its instrumentation –RTS Operations Coordination Center (ROCC) –ROCC architecture and real-time data flow –ROCC backend design Benchmark for real-time streamed data recording Benchmark for real-time streamed events processing Summary

4. MIT Lincoln Laboratory MySQL Users Conf-4 04-27-2006 Reagan Test Site (RTS) and its Instrumentation –Multiple RF sensors collecting data in several regions of electromagnetic spectrum –Multiple optical sensors collecting objects metric and spectral characteristics –Telemetry systems capable of tracking multiple targets –Mobile and fixed ground safety instrumentation

5. MIT Lincoln Laboratory MySQL Users Conf-5 04-27-2006 ROCC RTS Operations Coordination Center (ROCC) ROCC controls RTS instrumentation –Manages data flow from multiple sensors –Analyses the acquired data –Displays tracks and predicts the path of space objects –Stores acquired data for later analysis and reporting –Simulates past and future activities Activity Simulation Data Analysis Algorithms Database Displays Network Sensors Primary operations:

6. MIT Lincoln Laboratory MySQL Users Conf-6 04-27-2006 ROCC Publish-subscribe middleware ROCC Architecture and Real-Time Data Flow Sensors Alg_1 Alg_M ROCC Interface Box_1 ROCC Interface Box_N Historian MySQL server Operational Data Store (ODS)

7. MIT Lincoln Laboratory MySQL Users Conf-7 04-27-2006 ROCC Backend Design Network/middleware ODS MySQL server How well Historian/MySQL server pair could satisfy current performance requirements for particular hardware platform and operating system? How well it could satisfy future requirements (how scalable is the solution)? How server performance depends upon specific hardware? How server performance depends upon specific software? Questions for benchmark to answer (then) Messages Queries Historian subscription threads

8. MIT Lincoln Laboratory MySQL Users Conf-8 04-27-2006 Outline Introduction Benchmark for real-time streamed data recording –Insertion measurement benchmark (IMB) –Testing system configurations –Analysis of IMB results Comparing one and multiple tables performance Comparing local and remote servers performance Comparing MySQL 4.1 and 5.0 servers performance Comparing MySQL 4.1 and 4.0 servers performance Comparing MySQL 4.0, 4.1 and 5.0 servers performance Comparing InnoDB engine vs. MyISAM engine performance Comparing JDK 1.5.0 64-bit vs. JDK 1.4.2 32-bit performance Benchmark for real-time streamed events processing Summary

9. MIT Lincoln Laboratory MySQL Users Conf-9 04-27-2006 Insertion measurement benchmark (IMB) Goals –Multiple clients writing into the same table (up to 100 clients) –Multiple clients writing into different tables (up to 100 tables) –Different record lengths (from 10 bytes up to 10 MB – 1,000,000 times) –Reading performance was not the major issue Design parameters –Store up to 10 MB/s with various record lengths –Simulate the asynchronous message generation in the acquisition system Server Table Client 1 … 100 Multiple clients – one table (M1) Client 1 … 100 Multiple clients – multiple tables (MM)

10. MIT Lincoln Laboratory MySQL Users Conf-10 04-27-2006 Testing system configurations QuantityCPUMemoryDisksNetwork 2Quad CPU Opteron, 64-bit, 1 MB Cache 8 GBRAID 5 SCSI Ultra 320, 300 GB 144 GB, 10 K rpm, 4.7 ms 1 Gb/sec 4Dual CPU Opteron, 64-bit, 1 MB Cache 4 - 8 GBRAID5, SATA, 250 GB, 80 GB, 7.2 K rpm, 10.9 ms, UltraATA-100, 7.2 K rpm, 11.5 ms 1 Gb/sec 2Single and dual CPU Intel(R) Xeon(TM) CPU, 2.0 GHz, 2 MB Cache 2 - 3 GBUltra SCSI 320, 146 GB, 15K rpm, 3.5 ms SCSI Ultra 160, 73 GB, 10K rpm, 7.8 ms 1 Gb/sec #Operating SystemMySQL ServerJava Client 1RHEL 4 AS, kernel 2.6.9-5.ELsmp, 64-bit4.0.18-standard 32/64-bitbuild 1.4.2_03, 32-bit 2RHEL 4 ES, kernel 2.6.9-11.ELsmp 32-bit4.1.11-standard 32/64-bitbuild 1.4.2_10, 32-bit 6SLES 8 kernel 2.4.21, 64-bit5.0.16-standard 32/64-bitbuild 1.5.0_01, 64-bit Software tested Hardware tested

11. MIT Lincoln Laboratory MySQL Users Conf-11 04-27-2006 Outline Introduction Benchmark for real-time streamed data recording –Insertion measurement benchmark (IMB) –Testing system configurations –Analysis of IMB results (MyISAM tables) Comparing one and multiple tables performance Comparing local and remote servers performance Comparing MySQL 4.1 and 5.0 servers performance Comparing MySQL 4.1 and 4.0 servers performance Comparing MySQL 4.0, 4.1 and 5.0 servers performance Comparing InnoDB engine vs. MyISAM engine performance Comparing JDK 1.5.0 64-bit vs. JDK 1.4.2 32-bit performance Benchmark for real-time streamed events processing Summary

12. MIT Lincoln Laboratory MySQL Users Conf-12 04-27-2006 Comparing One and Multiple Tables Performance (MyISAM) 4-CPU Opteron – local client and server (the same machine)

13. MIT Lincoln Laboratory MySQL Users Conf-13 04-27-2006 Comparing Local and Remote Servers Performance (MyISAM) 2-CPU server; 2-CPU remote client / 2-CPU local client and server 2-CPU server; 4-CPU remote client / 2-CPU local client and server

14. MIT Lincoln Laboratory MySQL Users Conf-14 04-27-2006 Comparing MySQL 4.1 and 5.0 Servers Performance (MyISAM) 1-CPU Intel Xeonlocal client and server

15. MIT Lincoln Laboratory MySQL Users Conf-15 04-27-2006 Comparing MySQL 4.0 and 4.1 Servers Performance (MyISAM) 1-CPU Intel Xeonlocal client and server

16. MIT Lincoln Laboratory MySQL Users Conf-16 04-27-2006 Comparing MySQL 4.0, 4.1, and 5.0 Servers Performance (MyISAM) 1-CPU Intel Xeonlocal client and server

17. MIT Lincoln Laboratory MySQL Users Conf-17 04-27-2006 Comparing InnoDB vs MyISAM and JDK 1.4.2 32-bit vs JDK 1.5.0 64-bit 2-CPU Opteron client; 4-CPU Opteron server; MySQL 4.0.18

18. MIT Lincoln Laboratory MySQL Users Conf-18 04-27-2006 Introduction Benchmark for real-time streamed data recording Benchmark for processing real-time streamed events – Client/Server (CS)–based ROCC architecture – Using indexes for real-time data storing and retrieving – Records retrieval acceleration using secondary query and indexes – Event measurement benchmark (EMB) – Analysis of EMB Results – Comparing MySQL 4.1 and 5.0 servers performance (MyISAM) Summary Outline

19. MIT Lincoln Laboratory MySQL Users Conf-19 04-27-2006 Client/Server (CS)–based ROCC architecture CS drawbacks Data centralization More workload for the network – UDP vs. TCP/IP, no multicast More workload for the database server Require indexes and multilevel queriesmore complex requests Sensors CS advantages Dont need special recording component (Historian) Pull modelall components get the data when they want them The same API for the current and historical data Excellent filtering capabilities (SQL) High throughput, very simple settings, no load balancing Very standard, very low cost, low demand for resources ROCC Interface Box_N ROCC Interface Box_1 Alg_1 Alg_M ODS MySQL server

20. MIT Lincoln Laboratory MySQL Users Conf-20 04-27-2006 Using Indexes for Storing and Retrieving Real-Time Data Traditional (interactive and batch) approach 1.Store data into the relational table 2.Add indexes to the table once the data accumulation is finished 3.Retrieve the data using added indexes Real-time (continuous) approach 1. Store data into the relational table using primary key with AUTO_INCREMENT option 2. Retrieve the latest data using primary key, ORDER and LIMIT query parameters 3. Store the latest retrieved data in the temporary derived table 4. Make the search of the record using all available fields only in the derived table Advantages Simplicity Independence of indexes addition from data creation Drawbacks Adding indexes to all data in the table is a slow process Indexes will be recalculated every time a new record is stored Advantages Drawbacks Primary key has to be numeric to allow AUTO_INCREMENT option Only one index per table could be used for selecting the derived table Very fast! Index creation time is spread over all insertion operations (10% overhead) LIMIT parameter defines the width of the window (number of records for the second SELECT statement) Primary key is indexed automatically

21. MIT Lincoln Laboratory MySQL Users Conf-21 04-27-2006 Records Retrieval Acceleration Using Inner Query and Indexes recordIDcolumn1………. N301 N-1299 N-2303 ……. N-M400 N-M-1……. 1 recordIDcolumn1………. N301 N-1299 N-2303 ……. N-M400 N-M-1……. 1 recordIDColumn1………. N301 N-1299 N-2303 ……. N-M400 N-M-1……. 1 SELECT * FROM table1 ORDER BY recordID DESC LIMIT M Table for record insertion and retrieval SELECT * FROM () t WHERE t.column1<300 second query window window width = M<<N Primary autoincrement key (indexed) Last record in the table First record in the table First record in the second query window Last record in the second query window Found record

22. MIT Lincoln Laboratory MySQL Users Conf-22 04-27-2006 Event Measurement Benchmark (EMB) Goals and Design Goals –Multiple writing clients write into multiple tables time-stamping each record –Multiple reading clients read from into different tables while the writing clients continue to write into the tables –Reading clients compare each record time stamp with current time and calculate the latency –Reading clients write the results into result tables Design –Measure the latencies between freshly stored and retrieved events –Demonstrate that more than 2 MB/sec throughput could be achieved –Demonstrate the sustainable latencies were less that 300 msec

23. MIT Lincoln Laboratory MySQL Users Conf-23 04-27-2006 MySQL server MySQL database EMB Goals and Design WritingClient_3 WritingClient_2 WritingClient_1 Computer 0 Writing tables 0 Computer 2 Reading Client_6 Reading Client_5 Reading Client_4 Reading Client_3 Reading Client_2 Reading Client_1 Result tables 2 Computer 1 Reading Client_6 Reading Client_5 Reading Client_4 Reading Client_3 Reading Client_2 Reading Client_1 Result tables 1

24. MIT Lincoln Laboratory MySQL Users Conf-24 04-27-2006 Analysis of EMB Results Database server4 CPU Opteron, writing/reading clients2 CPU Opterons MySQL server version 4.1.11–standard, Network 1 Gb/sec, JDK 1.4.2_10 Latencies histograms for writing/reading threads for 4 clients Throughput = 4 MB/sec, 90% latencies within 100 msec Throughput = 2 MB/sec, 90% latencies within 100 msec Throughput = 3 MB/sec, 90% latencies within 40 msec

25. MIT Lincoln Laboratory MySQL Users Conf-25 04-27-2006 Comparing MySQL 4.1 and 5.0 Servers Performance (MyISAM) Database server2 CPU Xeon, writing and reading clientstwo 2CPU Opterons Network 1 Gb/sec, JDK 1.4.2_10 MySQL 5.0.16 MySQL 4.1.11

26. MIT Lincoln Laboratory MySQL Users Conf-26 04-27-2006 Outline Introduction Benchmark for recording real-time streamed data Analysis of Insertion Measurement Benchmark (IMB) Benchmark for processing real-time streamed events Summary

27. MIT Lincoln Laboratory MySQL Users Conf-27 04-27-2006 Summary IMB and EMB were designed to model major data flows in ROCC IMB demonstrated: –Java/MySQL client/server architecture satisfies ROCC requirements –MySQL 4.1.11 was the fastest version for insertion transactions –MyISAM showed superior performance relative to InnoDB –JDK 1.5.0 64-bit delivers much better performance than 1.4.2 32-bit EMB demonstrated: –Java/MySQL-based client/server architecture with appropriate queries and indexation could be used for ESP applications –It delivered better throughput and latencies than currently employed publish/subscribe architecture –MySQL 5.0.16 exhibited better linearity and performance for large number of writing/reading threads than 4.1.11 version Benchmarks showed that MySQL server is capable to handle both real-time data recording and real-time event stream processing