Naaliel Mendes, João Durães, Henrique Madeira CISUC, Department of Informatics Engineering University of Coimbra {naaliel, jduraes,

Slides:



Advertisements
Similar presentations
Chapter 6 Server-side Programming: Java Servlets
Advertisements

Inktomi Confidential and Proprietary The Inktomi Climate Lab: An Integrated Environment for Analyzing and Simulating Customer Network Traffic Stephane.
Configuration management
Performance Testing - Kanwalpreet Singh.
Web Performance Tuning Lin Wang, Ph.D. US Department of Education Copyright [Lin Wang] [2004]. This work is the intellectual property of the author. Permission.
JSP: JavaServer Pages Juan Cruz Kevin Hessels Ian Moon.
Web Server Hardware and Software
Introduction to z/OS Basics © 2006 IBM Corporation Chapter 8: Designing and developing applications for z/OS.
Yaksha: A Self-Tuning Controller for Managing the Performance of 3-Tiered Web Sites Abhinav Kamra, Vishal Misra CS Department Columbia University Erich.
ASP.NET 2.0 Chapter 6 Securing the ASP.NET Application.
February 11, 2003Ninth International Symposium on High Performance Computer Architecture Memory System Behavior of Java-Based Middleware Martin Karlsson,
MSF Testing Introduction Functional Testing Performance Testing.
Understanding and Managing WebSphere V5
Web Page A page displayed by the browser. Website Collection of multiple web pages Web Browser: A software that displays web pages on client computer.
This chapter is extracted from Sommerville’s slides. Text book chapter
Software Faults and Fault Injection Models --Raviteja Varanasi.
M. Taimoor Khan * Java Server Pages (JSP) is a server-side programming technology that enables the creation of dynamic,
Introduction to HP LoadRunner Getting Familiar with LoadRunner >>>>>>>>>>>>>>>>>>>>>>
Dependability benchmarking for transactional and web systems Henrique Madeira University of Coimbra, DEI-CISUC Coimbra, Portugal.
Krerk Piromsopa. Advance Net-Centric Computing Technology Krerk Piromsopa. Department of Computer Engineering. Chulalongkorn University.
Chapter 16 The World Wide Web Chapter Goals Compare and contrast the Internet and the World Wide Web Describe general Web processing Describe several.
About Dynamic Sites (Front End / Back End Implementations) by Janssen & Associates Affordable Website Solutions for Individuals and Small Businesses.
Strategies for improving Web site performance Google Webmaster Tools + Google Analytics Marshall Breeding Director for Innovative Technologies and Research.
University of Coimbra, DEI-CISUC
Bottlenecks: Automated Design Configuration Evaluation and Tune.
Monitoring Latency Sensitive Enterprise Applications on the Cloud Shankar Narayanan Ashiwan Sivakumar.
SITools Enhanced Use of Laboratory Services and Data Romain Conseil
ASP.NET Web Application and Development Digital Media Department Unit Credit Value : 4 Essential Learning time : 120 hours Digital.
MSE Presentation 3 By Padmaja Havaldar- Graduate Student
 To explain the importance of software configuration management (CM)  To describe key CM activities namely CM planning, change management, version management.
Ideas to Improve SharePoint Usage 4. What are these 4 Ideas? 1. 7 Steps to check SharePoint Health 2. Avoid common Deployment Mistakes 3. Analyze SharePoint.
Software Project Documentation. Types of Project Documents  Project Charter  Requirements  Mockups and Prototypes  Test Cases  Architecture / Design.
Contents 1.Introduction, architecture 2.Live demonstration 3.Extensibility.
1 Specification and Implementation of Dynamic Web Site Benchmarks Sameh Elnikety Department of Computer Science Rice University.
Network Computing Laboratory Experiment Tutorial Network Computing Lab
Architectural Characterization of an IBM RS6000 S80 Server Running TPC-W Workloads Lei Yang & Shiliang Hu Computer Sciences Department, University of.
Architectural Characterization of an IBM RS6000 S80 Server Running TPC-W Workloads Lei Yang & Shiliang Hu Computer Sciences Department, University of.
Introduction to JavaServer Pages. 2 JSP and Servlet Limitations of servlet  It’s inaccessible to non-programmers JSP is a complement to servlet  focuses.
Chapter 6 Server-side Programming: Java Servlets
An application architecture specifies the technologies to be used to implement one or more (and possibly all) information systems in terms of DATA, PROCESS,
1 Admission Control and Request Scheduling in E-Commerce Web Sites Sameh Elnikety, EPFL Erich Nahum, IBM Watson John Tracey, IBM Watson Willy Zwaenepoel,
Microsoft Management Seminar Series SMS 2003 Change Management.
ASP (Active Server Pages) by Bülent & Resul. Presentation Outline Introduction What is an ASP file? How does ASP work? What can ASP do? Differences Between.
JS (Java Servlets). Internet evolution [1] The internet Internet started of as a static content dispersal and delivery mechanism, where files residing.
CSI 3125, Preliminaries, page 1 SERVLET. CSI 3125, Preliminaries, page 2 SERVLET A servlet is a server-side software program, written in Java code, that.
1 Chapter 12 Configuration management This chapter is extracted from Sommerville’s slides. Text book chapter 29 1.
ASP-2-1 SERVER AND CLIENT SIDE SCRITPING Colorado Technical University IT420 Tim Peterson.
Performance Testing Test Complete. Performance testing and its sub categories Performance testing is performed, to determine how fast some aspect of a.
WebWatcher A Lightweight Tool for Analyzing Web Server Logs Hervé DEBAR IBM Zurich Research Laboratory Global Security Analysis Laboratory
An Architectural Evaluation of Java TPC-W Harold “Trey” Cain, Ravi Rajwar, Morris Marden, Mikko Lipasti University of Wisconsin-Madison
Software Architecture in Practice Mandatory project in performance engineering.
Whole Test Suite Generation. Abstract Not all bugs lead to program crashes, and not always is there a formal specification to check the correctness of.
 Project Team: Suzana Vaserman David Fleish Moran Zafir Tzvika Stein  Academic adviser: Dr. Mayer Goldberg  Technical adviser: Mr. Guy Wiener.
Configuring SQL Server for a successful SharePoint Server Deployment Haaron Gonzalez Solution Architect & Consultant Microsoft MVP SharePoint Server
Cofax Scalability Document Version Scaling Cofax in General The scalability of Cofax is directly related to the system software, hardware and network.
Agenda Introduction Literature survey Hardware and software requirements System design System implementation System testing Conclusion and future enhancement.
Introduction to Information Systems SSD1: Introduction to Information Systems Unit 1. The World Wide Web Unit 2. Introduction to Java and Object- Oriented.
QlikView Sizing Walkthru - RAM. Total RAM on the QlikView Server One or more QlikView documents (QVWs) loaded on Server Unaggregated QVW data model QlikView.
E-Business Infrastructure PRESENTED BY IKA NOVITA DEWI, MCS.
SOFTWARE TESTING Date: 29-Dec-2016 By: Ram Karthick.
Abhinav Kamra, Vishal Misra CS Department Columbia University
Software Architecture in Practice
Outline Introduction Standards Project General Idea
Strategies for improving Web site performance
PHP / MySQL Introduction
Security in Java Real or Decaf? cs205: engineering software
Lecture 1: Multi-tier Architecture Overview
Moodle Scalability What is Scalability?
Admission Control and Request Scheduling in E-Commerce Web Sites
Web Servers (IIS and Apache)
Presentation transcript:

Naaliel Mendes, João Durães, Henrique Madeira CISUC, Department of Informatics Engineering University of Coimbra {naaliel, jduraes, April 2010 Evaluating and Comparing the Impact of Software Faults on Web Servers

2 Systems and Software Engineering Group Presentation (SSE) – Coimbra, Portugal – April 22, 2010Naaliel Mendes Presentation Outline Research Problem Our Approach Methodology Specification Methodology Implementation Case Study Perfomance and Dependability Results Conclusions

3 Systems and Software Engineering Group Presentation (SSE) – Coimbra, Portugal – April 22, 2010Naaliel Mendes Research Problem How a web server will behave if a software fault (ex., bug) is activated in one of their hosted web applications? No impact? Crash? Unresponsive? Abnormal resource usage? No idea? Does it matter? At this moment millions of web servers are being used, many of them hosting web applications. Developed under time constraint, by inexperienced programmers. What is the problem? Why should it be solved? No methodology to evaluate how web servers are affected by software faults present in web applications. Netcraf’s April-10 Survey: more than 205 millions of web sites

4 Systems and Software Engineering Group Presentation (SSE) – Coimbra, Portugal – April 22, 2010Naaliel Mendes Our Approach Experimental method to evaluate and compare the impact of software faults on web servers (WS) Fault injection target: Web Application (App). How do we do that? By collecting measurements on response time, response correctness, resource use, (…) of the web server when faults are injected in one of its hosted web applications. State of the web server after each software fault activation: Crash, Unresponsive, Wrong Results, Resource Use Penalty, No Impact. Two phases: 1) Golden Run 2) Fault Injection Run WS DB ClientApp

5 Systems and Software Engineering Group Presentation (SSE) – Coimbra, Portugal – April 22, 2010Naaliel Mendes Methodology Specification Generic Aimed at addressing any type of web servers and web applications. Technology-independent Can be implemented by different teams using different technologies. But you MUST monitor the web servers properties and follow the procedures we specify. Includes classical dependability benchmark elements Experiments rules, Workload, Faultload, Measures, and Instrumentation Tools.

6 Systems and Software Engineering Group Presentation (SSE) – Coimbra, Portugal – April 22, 2010Naaliel Mendes Methodology Specification Target System Architecture Web Serving System

7 Systems and Software Engineering Group Presentation (SSE) – Coimbra, Portugal – April 22, 2010Naaliel Mendes Methodology Specification Target System Components

8 Systems and Software Engineering Group Presentation (SSE) – Coimbra, Portugal – April 22, 2010Naaliel Mendes Methodology Specification Experiment Rules Phase 1. Golden run. It is used to determine the baseline performance of the system under benchmark (without software faults). Executed three times and the result is the average of the three executions Phase 2. Execution of the workload in the presence of faults to evaluate the dependability properties of the benchmark target. The number of executions depends on the number of faults

9 Systems and Software Engineering Group Presentation (SSE) – Coimbra, Portugal – April 22, 2010Naaliel Mendes Methodology Specification Workload Web application that simulates the activities of a business oriented transactional web server. Internet commerce application environment. Multiple concurrent on-line browser sessions. Involving both static and dynamic content. Access and update patterns to a database. Faultload Set of software faults that represents realistic bugs found in applications. We can find in literature some field studies on the representativeness of software faults (e.g., [Durães, 2006]).

10 Systems and Software Engineering Group Presentation (SSE) – Coimbra, Portugal – April 22, 2010Naaliel Mendes Methodology Specification Measures Client-side perspective: Response Time, Response Correctness, Throughput. Server-side perspective: Web Server Execution Duration, Web Server Crash (System Process), Log File Size (Number of Exceptions), Processor Use. Failure Modes State of the web server: Crash, Unresponsive, Resource Use Penalty, Wrong Results, and No Impact. Mapping Measures with Failure Modes How do I know that the web server crashed?

11 Systems and Software Engineering Group Presentation (SSE) – Coimbra, Portugal – April 22, 2010Naaliel Mendes Methodology Implementation Diagnostic Approach Integrity Checker (IC) verify the performance and error propagation Failure Mode: Web Server Unresponsive, Wrong Results. How do we verify error propagation? IC is free of software faults and provides excepted responses. Client knows the expected result of a request (e.g., Spain => Europe) Web Server Web App IC Response Validator Integrity Checker Web App Client Validator knows the IC expected response

12 Systems and Software Engineering Group Presentation (SSE) – Coimbra, Portugal – April 22, 2010Naaliel Mendes Methodology Implementation Diagnostic Approach Web Server Processor Monitor observe the status of the web server process, checking if it is running or not. Failure Mode: Web Server Crash. Resource Consumption Monitor observe log file size, number of exceptions, processor use (PerfMon). Failure Mode: Resource Use Penalty. Data Analyzer. Summarize the data from different systems source (output files with different formats).

13 Systems and Software Engineering Group Presentation (SSE) – Coimbra, Portugal – April 22, 2010Naaliel Mendes Methodology Implementation Workload We used a Java implementation of the TPC-W Benchmark ( TPC’s benchmarks “delivers trusted results to the industry” Easy to install and run, easy to inject software faults (in the servlet code). Used in other research works. What is TPC-W? Ecommerce workload that simulates the activities of a retail store web site. Emulated users can browse and order books from the website. Simulates the same HTTP network traffic as would be seen by a real customer using a browser. 10 users were emulated, multiple requests to the web server. Only one instance of the TPC-W application.

14 Systems and Software Engineering Group Presentation (SSE) – Coimbra, Portugal – April 22, 2010Naaliel Mendes Methodology Implementation Faultload To select which software faults to include in our faultload, we used a field study on software fault targeting web applications [Fonseca, 2008]. Each fault is emulated where it could realistically exist in the source code. The faults were injected manually. There is no software fault injection tool targeting the bytecode of Java Applications. Two of the most representative faults types were injected in different points of the web-application source-code. MIFS (Missing IF Construct plus Statements). WVAV (Wrong Value Assigned to a Variable). 250 software faults (125 of each type) were injected.

15 Systems and Software Engineering Group Presentation (SSE) – Coimbra, Portugal – April 22, 2010Naaliel Mendes Methodology Implementation Fault Injection 5-Step-Approach 1. Analysis of the source-code to find out a fault injection point. 2. Injection of the software fault (MIFS, WVAV). 3. Compilation of the faulty source-code. 4. Copying of the faulty web application to a specific directory. 5. Returning the source-code to the original state (free of fault) and repeat these steps.

16 Systems and Software Engineering Group Presentation (SSE) – Coimbra, Portugal – April 22, 2010Naaliel Mendes Methodology Implementation Examples of Fault Injection TPCW_Database.java (Class used by TPCW servlets) MIFS (Missing IF Construct Plus Statement) WVAV (Wrong Value Assigned to a Variable) Public class TPCW_Database { (...) public static syncnrozied Connection getConnection() { (...) if (maxConn == 0 || checkout < maxConn) { con = getNewConnection(); totalConnection++; } } (...) } Public class TPCW_Database { (...) private static final String jdbcPath = “jdbc:db2:tpcw” private static final String jdbcPath = “jdbc:db2:tpcwXXXXXX” (...) }

17 Systems and Software Engineering Group Presentation (SSE) – Coimbra, Portugal – April 22, 2010Naaliel Mendes Methodology Implementation Management System Brief summarization of its execution flow: Starts after the initialization of the operating system Executes preliminary operations to ensure a fresh fault injection run start. Copy the faulty web application to the web server directory that will be used in the next fault injection run Clean-up temporary directory. Starts monitoring tools, the web server (which hosts the faulty web application), and the client applications (TPCW Client and Integrity Checker Client). Executes post conditions operations. Normalize, organize, and analyze the output files generated during the fault injection run Ends by rebooting the operating system

18 Systems and Software Engineering Group Presentation (SSE) – Coimbra, Portugal – April 22, 2010Naaliel Mendes Case Study: Experimental Setup Web Servers Apache Tomcat 6.0 Sun Java System 7.0 (SJSWS) Jetty Web Server Workload TPC-W Web Application IBM DB2 Database Operating system Windows XP Hardware Platform: Server-side: P4 3.0 GHz, 1.46Gb RAM, 80Gb of HD Hardware platform and systems surrounding the BT were the same across all experiments

19 Systems and Software Engineering Group Presentation (SSE) – Coimbra, Portugal – April 22, 2010Naaliel Mendes Case Study: Performance Results Response Time (AVG) & Throughput (Res/sec) Why did response time go down? The fault activation forced a premature termination of the workload application, reducing the number of requests sent to the web server component. Why did response time go down? The fault activation forced a premature termination of the workload application, reducing the number of requests sent to the web server component.

20 Systems and Software Engineering Group Presentation (SSE) – Coimbra, Portugal – April 22, 2010Naaliel Mendes Case Study: Dependability Results Data Integrity No integrity data errors registered by our Integrity Checker Application. Web server provided expected responses for the requests targeting the application free of fault (Integrity Checker). Availability (%) The high number of exceptions thrown by the web app. consumed all resources of the web server, making it to be unresponsive.

21 Systems and Software Engineering Group Presentation (SSE) – Coimbra, Portugal – April 22, 2010Naaliel Mendes Case Study: Resource Use Results Disk Use Comparison (%) Percentual Distribution of Log File Size 1 WVAV 1 MIFS 2 WVAV ( 95, 100, 146 MB) 1 MIFS 2 WVAV ( 95, 100, 146 MB) Log File Size (Worst Case) 5 min = MB 1 hour = GB 1 day = + 41 GB Log File Size (Worst Case) 5 min = MB 1 hour = GB 1 day = + 41 GB 1 MIFS 2 WVAV ( 50, 67, 50 MB) 1 MIFS 2 WVAV ( 50, 67, 50 MB) 1 MIFS 2 WVAV 1 MIFS 2 WVAV 1 WVAV (95 MB) 1 WVAV (95 MB)

22 Systems and Software Engineering Group Presentation (SSE) – Coimbra, Portugal – April 22, 2010Naaliel Mendes Case Study: Resource Use Results Processor Use Comparison (%) 1 MIFS 1 WVAV 1 MIFS Web Servers have different behavior to use the processor resource. These results were collected using the same instrumentation tools. Web Servers have different behavior to use the processor resource. These results were collected using the same instrumentation tools.

23 Systems and Software Engineering Group Presentation (SSE) – Coimbra, Portugal – April 22, 2010Naaliel Mendes Case Study: Failure Mode Results Which is the most robust of the web servers we evaluated?

24 Systems and Software Engineering Group Presentation (SSE) – Coimbra, Portugal – April 22, 2010Naaliel Mendes Conclusions Lessons learnt Software faults present in web applications affect the performance and the dependability of web servers in a different way across web servers. In some cases the unavailability rate was around 2%. The TPC-W application tolerated many software faults. However, in some cases, it generated a large number of exceptions that contributed to make the web server unresponsive. The manual injection of the software faults makes hard, but not impossible, the reproduction of our injection campaign. We believe that the information and references we provided can guide anyone interested in reproducing our experimental setup.

25 Systems and Software Engineering Group Presentation (SSE) – Coimbra, Portugal – April 22, 2010Naaliel Mendes Conclusions Future Work Addition of new types of software faults in our faultload. Conduction of new case studies to demonstrate the portability and scalability of our approach in other environments. Advance this work and propose a dependability benchmark for web servers considering the impact of software faults in hosted applications as the main benchmark measure.

26 Systems and Software Engineering Group Presentation (SSE) – Coimbra, Portugal – April 22, 2010Naaliel Mendes Thank you! Increase the impact and visibility of your research with AMBER Raw Data Repository Visit the web site!