1. Marcelo R.N. Mendes

2. What is FINCoS? A Java-based set of tools for data generation, load submission, and performance measurement of event processing systems; Main Characteristics: Flexible; Independent of particular workloads; Not bound to any specific CEP product; Scalable. 2CISUC, University of Coimbra - mnunes@dei.uc.ptJuly, 2012

3. FINCoS: Architecture 3CISUC, University of Coimbra - mnunes@dei.uc.ptJuly, 2012

4. FINCoS: Components Driver Simulates external sources of events; Submits load to the SUT; Includes data generation mechanism; Sink Receives query results from CEP engine; Stores output events in log files for subsequent validation. 4CISUC, University of Coimbra - mnunes@dei.uc.ptJuly, 2012

5. FINCoS: Components Controller Main Interface with the user. Allows to: Create reusable test setups; Control Drivers and Sinks (e.g., load, start, stop); Alter workload parameters on-the- fly (e.g., event submission rates, switch phases) 5CISUC, University of Coimbra - mnunes@dei.uc.ptJuly, 2012

6. FINCoS: Components Performance Monitor Collect Metrics of Performance Runs Two Modes: Online (integrated into FINCoS Controller); Offline (log files). 6CISUC, University of Coimbra - mnunes@dei.uc.ptJuly, 2012

7. Lesson 1: Creating a Test Setup 1. Configure a Driver 1. Decomposable Workload 2. Synthetic vs. Thirdy-Party Dataset 3. Data Generation Options 4. Event Arrival Process 2. Configure a Sink July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt7

8. July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt8

9. July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt9

10. July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt10 unique identifier number of threads used for load generation save generated data to disk? to where Driver send the events machine that hosts the Driver

11. July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt11 Workload can be split in independent phases

12. July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt12

13. July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt13 Chooses the type of workload for this phase Sets events arrival process Event Mix Events are generated in a ordered way When data generation happens To ensure repeatability…

14. July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt14 Configure event types…

15. July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt15

16. July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt16

17. July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt17 Configure Event Type Schema

18. July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt18

19. July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt19

20. July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt20 Column Name

21. July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt21 Column Type

22. July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt22 Data Generation Options

23. July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt23 Add Values

24. July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt24

25. July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt25

26. July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt26

27. July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt27

28. July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt28

29. July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt29

30. July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt30

31. July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt31

32. July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt32

33. July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt33

34. July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt34

35. July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt35

36. July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt36

37. July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt37 unique identifier List of event types this Sink subscribes to from where Sink receive events machine that hosts the Sink save generated data to disk?

38. July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt38

39. July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt39

40. July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt40

41. July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt41 Done!

42. Lesson 2: Configuring a new Connection 1. Configure a reusable connection with a CEP engine or JMS Provider July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt42

43. July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt43

44. July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt44

45. July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt45

46. July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt46 unique identifier name of the CEP engine (out-of-the-box support for Esper) Properties used to connect with the CEP engine type of the connection

47. July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt47 * supporting new CEP engines requires coding

48. July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt48 Done!

49. Lesson 3: Running Tests 1. Load Drivers and Sinks 2. Start Load Submission 3. Control Load Submission NOTE: At this point the following steps must have been completed: 1. Creation of a test setup; 2. Initialization of the FINCoS Daemon Service at the machines where Drivers and Sinks are intended to run; July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt49

50. July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt50 Load components

51. July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt51

52. July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt52 Start Load Submission

53. July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt53 Sent/Received counts

54. July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt54

55. July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt55

56. July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt56

57. Lesson 4: Collecting Performance Metrics 1. Online Performance Measurement 1. During tests 2. Offline Performance Measurement 1. After test completion, use FINCoS Performance Monitor to process log files July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt57

58. July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt58

59. July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt59

60. July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt60

61. Lesson 4: Collecting Performance Metrics (...cont) 1. Online Performance Measurement 1. During tests 2. Offline Performance Measurement 1. After test completion, use FINCoS Performance Monitor to process log files July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt61

62. July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt62

63. July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt63

64. July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt64 Configure M.I. here: Optionally, save collected metrics to a file

65. July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt65

66. July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt66 Select the stream

67. July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt67 Select the metric

68. July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt68 Select exhibition scale

69. July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt69 Add Metrics to the Chart

70. July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt70 Change Visualization Options

71. Lesson 5: Setting Test Options 1. Response Time Measurement July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt71

72. July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt72

73. July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt73

74. July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt74

75. Response Time Measurement Mode July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt75 DRIVER CEP ENGINE CEP ENGINE SINK ADAPTER 2 2 3 3 4 4 send convert input convert output process input forward generate result forward send 1 1

76. Response Time Measurement Mode July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt76 DRIVER CEP ENGINE CEP ENGINE SINK ADAPTER send convert input convert output process input forward generate result forward send Δt = t4 – t1 End-to-end: From Driver up to Sink 2 2 3 3 4 4 1 1

77. Response Time Measurement Mode July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt77 DRIVER CEP ENGINE CEP ENGINE SINK ADAPTER send convert input convert output process input forward generate result forward send Δt = t3 – t2 Process Time: From immediately before the input event is forwarded to CEP engine to immediately after the result is received. 2 2 3 3 4 4 1 1

78. Download and Feedback FINCoS is available at: http://code.google.com/p/fincos/ Any suggestions or bug reports: mnunes@dei.uc.pt July, 2012CISUC, University of Coimbra - mnunes@dei.uc.pt78