Final Presentation Liat Ben-Ami Yonathan Perez Instructor: Roy Mitrany

Presentation Overview Goal Market Survey – where does HTTP Bomber fit HTTP Background How does HTTP Bomber work? Technologies Design Overview Important Modules Concurrency GUI Summary & Gained Knowledge

Goals Generate Http traffic load in order to test server performance Analyze server responses and provide analysis results with a clear graphic display

Market survey: existing products Several traffic generators: generate traffic according to user-defined workload. HP httperf, curl-loader project,.. Http Analyzers / Monitors : monitor all http traffic from and to a user-agent, analyze it and provide graphic display of the results. Http Analyzer V5 by IE Inspector, Http Debugger Pro by Softx,… HTTP Bomber is an HTTP traffic generator that also provides analysis and graphic display of the traffic to and from the tested server.

HTTP Protocol Most significant application-layer protocol in the World Wide Web. Operates over reliable transport layer protocol (TCP) Goal: Transfer data of various types (Resources) over the Internet. Resources are identified by URIs. A stateless request-response protocol

HTTP Messages Header 1: value 1 Header 2: value2 … Header N: value N

HTTP Messages – cont. Request initial line: Method + URI + protocol version Methods supported by HTTP Bomber: GET & POST Response initial line: protocol version + status code + textual explanation status codes: 1xx: informational 2xx: success 3xx: redirection 4xx: Client error 5xx: Server error

How does HTTP Bomber work? User sets execution parameters (including tested server’s name) User builds a list of requests to be sent to server User starts simulation Virtual users send requests to server and collect responses. Responses are analyzed and statistical info is gathered and displayed.

Technologies Java Eclipse GUI: javax.swing, Jigloo Jfreechart Apache Http components

Design Overview HTTP Bomber Generator Analyzer (including Graphic display) Response Recorder writeread Server GUI / API Request List

Design Overview – Cont. Modular design HttpBomber – Top hierarchy module. Its public methods are used as user interface – as API or through GUI. Generator: generate workload and communicate with the server Analyzer: gather statistical info, write report, display graphs Generator and Analyzer interact through a common synchronized data structure

HttpBomber – Sequence Diagram

HttpGenerator - Purpose Send requests to server according to user-defined workload (execution parameters and request list) and produce records of server’s responses.

HttpGenerator - Implementation Requests are sent by virtual users (threads) through a multi-threaded Apache HttpClient. Number of virtual users is defined in execution parameters. Each virtual user iterates over the request list in a cyclic order. For each request: send request, receive response, generate a record and update the response recorder. Custom handling for IO and protocol errors

Generator – Sequence Diagram

Analyzer - Purpose Gather and produce statistical info from response records, produce report file and display graphs.

Analyzer - Implementation getData thread copies new data from the synchronized response recorder, updates counters and local data structures and writes record content to the report file. Gathered info: requestsSent, errorNum, dataReceived, dataSent, number of responses per each status code Separate threads run graphs – one thread per graph. Updated according to gathered info. at end of simulation – statistical info is computed and added to report and final graphs are displayed

Report File Example ******Start Report****** Start Time: Tue Apr 26 02:05:33 IDT 2011 Workload Parameters: Tested Server: port: 80, Number of virtual users: 1, max time cap: msec max requests per user: 5, max timeout: msec, max allowed retrials per request: 2 Response Records: Request index: 0, Status Code: 404, Data Sent: 65 bytes, Data Received: bytes, Response Time: 921 msec, Time stamp: Tue Apr 26 02:05:34 IDT 2011 Request index: 1, Status Code: 405, Data Sent: 68 bytes, Data Received: bytes, Response Time: 115 msec, Time stamp: Tue Apr 26 02:05:34 IDT 2011 Request index: 2, Status Code: 200, Data Sent: 60 bytes, Data Received: bytes, Response Time: 332 msec, Time stamp: Tue Apr 26 02:05:34 IDT 2011 Request index: 0, Status Code: 404, Data Sent: 65 bytes, Data Received: bytes, Response Time: 125 msec, Time stamp: Tue Apr 26 02:05:34 IDT 2011 Request index: 1, Status Code: 405, Data Sent: 68 bytes, Data Received: bytes, Response Time: 131 msec, Time stamp: Tue Apr 26 02:05:35 IDT 2011 Total requests sent: 5 Total data received: KB Total data sent: 0.33 KB Total errors: 0 Average traffic rate: KB/sec Response Time Statistics: Minimal msec, Maximal msec, Median msec, Average msec Status Code Distribution (zero indicates IO or protocol errors): 200 : 20% 404 : 40% 405 : 40% Time: Tue Apr 26 02:05:35 IDT 2011 Simulation ended:simulation ended *********End of report********

Synchronized Response Recorder 12N N - Number of requests in request list Records of responses to request i

HttpBomber Class Diagram

Graph Display – Online Graphs

Graph Display – Offline Graphs

Graph Display - Implementation

Graph Display – Sequence Diagram

Concurrency HTTP Bomber is a highly-concurrent application Significant need for Synchronization. Minimize access to mutable shared data. Several “virtual users” threads send requests simultaneously to server and write to the Response Recorder Analyzer: Analyzer main thread, getData thread and a thread for each of the graphs

Main thread Simulation startAnalyzer thread Virtual Users Threads …. start join start getData Thread Graph Threads start done join

User Guide - GUI

User Guide – using API example public class MainTest{ public static void main(String[] args) { URI uri1 = null; URI uri2 = null; try { uri1 = new URI(" uri2 = new URI("/jiadb"); } catch (URISyntaxException e) { return; } HttpBomber bomber = new HttpBomber(); /* set execution parameters */ bomber.setExecutionParameters(" 10, 60000, 100, Long.MAX_VALUE, 2); /* build request list */ try{ bomber.addGetRequest(uri2); bomber.addStringPostRequest(uri1, "bla bla", "text/plain; charset = UTF-8"); bomber.addGetRequest(uri1); bomber.addHeader(0, "User-Agent", "HTTP Bomber"); bomber.addHeader(1, "User-Agent", "HTTP Bomber"); bomber.addHeader(2, "User-Agent", "HTTP Bomber"); }catch(Exception e){ e.printStackTrace();} /* set graphic display */ bomber.setGraphDisplay(true, true, true, true); /* run simulation */ bomber.Init(); }

Summary & Gained Knowledge What we did: Created a reliable and user-friendly application to analyze performance of web servers What we learned: Java HTTP Protocol and Apache Http Components GUI design Graphic library API Design and implementation of a complex multi-threaded software tool