1. Refining of Failure Detection Technique in Web Applications
Tomohiro Nakamura, Greg Friedman, Peter Bodík
Refining Failure Detection Technique for Web Applications developed by Greg Friedman (Stanford), Peter Bodik (Berkeley) etc. using more features of Web application log
Apply new feature:
Detect anomaly by Chi-Squared testing on page transition intervals
Compare characters of the new feature and the previous (page hit frequencies)
“Page transition intervals” is useful to detect failure earlier ?
“Page transition intervals” has almost same detection ratio ?

2. Background
Failure Detection Technique in Web Application presented at 2004 winter ROC retreat :
Detect anomalies every minute by comparing page hit frequencies from last 6 hours vs. last several minutes
Use Chi-Squared testing on page hit frequencies to detect anomalies
Show the most anomalous page and the page transition from/to the page to localize the failure
The result of testing on 5 sets of HTTP logs :
Failure detection rate = 100%
7 days – 1 hour earlier notification of the failure
Failure localization rate = 40%

3. Basic idea
Failure Detection Technique in Web Application presented in 2004 winter ROC retreat is powerful
But only page hit frequencies are used to detect failure
HTTP log has more features which will be meaningful
In some case, stats of page transition will change before anomaly found on page hit frequencies and using the stats makes detection / localization earlier / better
The stats of page transition intervals (average time between page hits of each session) is used for anomaly detection

4. Basic idea (cont.) Time History Current Average interval time
Page A
Page B
Page C
Page A
Page B
Page C
Page A
Page B
Page C
Page A
Page B
Page C
Page A
Page B
Page C
Average interval time
From Page A To Page B
From Page B To Page C
Average
TAB
tAB
TBC
tBC
Compare interval times of History and Current
(TAB - tAB ) 2
(TBC - tBC ) 2
Chi2-Squared Test
Chi2 = +
TAB
TBC

5. Example(1) broken signup page
Both detect 7 days earlier (“Page transition intervals” +8 hour earlier)

6. Example(2) landing looping problem
“Page transition intervals” detect earlier (with some FP)
*FP: false positive

7. Example(3) account page problem
“Page hit frequencies” detect failure 4 hours earlier

8. Conclusion & Future work
Pros and Cons of Failure Detection using “Page transition interval” :
PRO
Detect failures earlier in many cases (some cases not)
Detection ratio is likely to be almost same as the previous
CON
False positive ratio is higher in some cases
Significance range is lower than the previous in general (difficult to show severity of the detected failure)
Future work
Robustness verification (ex. Multiple log files, Errors in log files)
Investigate Ensemble Method (each method has Pros & Cons)
Investigate failure detection coverage of these tool
Other features / algorithms to detect failures
What else?