Evaluating Web Software Reliability By Zumrut Akcam, Kim Gero, Allen Chestoski, Javian Li & Rohan Warkad CSI518 – Group 1

Agile Development

The Agile Development Process How does Agile work? How did our class use Agile?  3 Sprints  “Stand up” meetings at beginning of each class  Retrospective at the end of each sprint

Overview

Definition of Reliability What is reliability for Web applications? The reliability for Web applications can be defined as the probability of failure-free Web operation completions.[1] Failure is “the event of a system deviating from its specified behavior like obtaining or delivering information”.[2]

Failure Sources Failures are caused from the following sources:  Host, network or browser failures: computer systems, network or software failures, etc.  Source content failures: missing, inaccessible files, JavaScript errors, etc.  User errors: improper usage, mistyped URLs. [1]

Project Goal To strengthen the reliability of Web applications by minimizing the number of source content failures. Attempt to extend work on testing the reliability of websites. Gain experience doing a research project

Sprint 1

Sprint 1 Goals Read relevant research papers Identify factors that may effect reliability analysis Determine a system to analyze reliability on Gather access and error logs

Factors That May Effect Reliability Analysis Byte Count User Count Session Count Error Count

System to Analyze Reliability On Reliability analysis via error logs Variety of reliability requirements Commercial and non-commercial We will try to record the technologies the websites employ (Apache, DNN, ISS, PHP, ColdFusion, etc..)

Sprint 2

Sprint 2 Goals Collect log files for calculation Automate processes to extra data (user, session, byte, and error counts) Convert them into excel format Log Parser

Sprint 2 Progress DNN Logs (10 Websites) PHP Logs

What is DotNetNuke (DNN).NET version of Drupal An open source platform for building websites and web applications based on Microsoft.Net technology. Leading open source ASP.NET web content management Has been downloaded over 6 million times ~100 employees 5 th Version Founded 2006

Our DNN Logs Logs from 10 Websites Window Server (Same Server) SQL Server 2008 ~1000 unique visitors per day Logs contain  User count  Limited Error count

Major Problem Our DNN Logs does not contain  Session count  Byte count

Alternative Generate our own DNN logs

Sprint 3

Server Side Technologies Used  Windows XP Professional  Microsoft Internet Information Servers (ISS)  Microsoft SQL Server 2008  DotNetNuke (DNN) Logs Generated  Client IP’s  Byte Counts (Uploaded & Downloaded)  Time-Taken  Status Code

Generating Logs Clients  Web-Crawlers  DotNetNuke Client API Inducing Errors

Results

Workload Measurement Facts Server log data consisted of 23 consecutive days of data. Page Not Found (Error 404) is the most common type of error in our logs, with 46% of total recorded errors. Accessing forbidden data (Error 403) follows with 41%. 72 unique IPs, hits total, and each hit associated with average 5020 bytes.

Error/Success Ratios HTTP Status Codes Description 200OK 206Partial Content 302Found 304Not Modified 400Bad Request 401Unauthorized 403Forbidden 404Not Found 500Internal Server Error

Status Code-Bytes Graphic

500-Internal Server Error Profile

Number of errors

Average Time Taken By Different Errors

Conclusions By Nelson Model, the site software reliability is R = 0.966, or that 96.6% of access to website is successful. This model also shows that MTBF=29.6 hits or the site averages one error for every 29.6 hits. From the number of errors chart, we can see that Server errors are very few among the other errors which shows what the reliability of the DNN server is.

Conclusions Our modelPrevious Model [1] 23 days data26 days data 96.6 success96.2 success 29.6 hits/error26.6 hits/error 148,579 bytes per error 273,927 bytes per error [1] J.Tian, S.Rudraraju, Z.Li, “Evaluating Web Software Reliability Based on Workload and Failure Data Extracted from Server Logs”,2004. [2] T.Huynh, J.Miller, “Another viewpoint on 'Evaluating Web Software Reliability Based on Workload and Failure Data Extracted from Server Logs'”,2008. [3] G. Albeanu, A. Averian, I. Duda, “Web Software Reliability Engineering”,2009.