Mark Shereshevsky, Bojan Cukic Fractal Analysis of Resource Exhaustion in Operating Systems: Initial Steps Mark Shereshevsky, Bojan Cukic Jonathan Crowell.

Slides:



Advertisements
Similar presentations
ECE 8443 – Pattern Recognition EE 3512 – Signals: Continuous and Discrete Objectives: Response to a Sinusoidal Input Frequency Analysis of an RC Circuit.
Advertisements

Stat 112: Lecture 7 Notes Homework 2: Due next Thursday The Multiple Linear Regression model (Chapter 4.1) Inferences from multiple regression analysis.
IV&V Facility 1 Using Fractal Analysis to Monitor and Model Software Aging Mark Shereshevsky, Bojan Cukic, Jonathan Crowell, Vijai Gandikota West Virginia.
DFT/FFT and Wavelets ● Additive Synthesis demonstration (wave addition) ● Standard Definitions ● Computing the DFT and FFT ● Sine and cosine wave multiplication.
Chapter 4 The Relation between Two Variables
Table of Contents Exit Appendix Behavioral Statistics.
STAT 497 APPLIED TIME SERIES ANALYSIS
CMPT 855Module Network Traffic Self-Similarity Carey Williamson Department of Computer Science University of Saskatchewan.
Alon Arad Alon Arad Hurst Exponent of Complex Networks.
Self-Similarity in Network Traffic Kevin Henkener 5/29/2002.
1 Using A Multiscale Approach to Characterize Workload Dynamics Characterize Workload Dynamics Tao Li June 4, 2005 Dept. of Electrical.
Intro to Statistics for the Behavioral Sciences PSYC 1900 Lecture 6: Correlation.
Ch. 6 The Normal Distribution
CHAPTER 6 Statistical Analysis of Experimental Data
MTH-471 Integral Equations Sheikh Irfan Ullah Khan Assistant Professor Department of Mathematics COMSTAS Institute of Information Technology.
Experimental Evaluation
Fault Prediction and Software Aging
X-bar and R Control Charts
Elec471 Embedded Computer Systems Chapter 4, Probability and Statistics By Prof. Tim Johnson, PE Wentworth Institute of Technology Boston, MA Theory and.
Applied Mathematics Complex Systems Fractals Fractal by Zhixuan Li.
Seiji Armstrong Huy Luong Huy Luong Alon Arad Alon Arad Kane Hill Kane Hill.
Objectives of Multiple Regression
Statistical Methods For Engineers ChE 477 (UO Lab) Larry Baxter & Stan Harding Brigham Young University.
ECE 8443 – Pattern Recognition EE 3512 – Signals: Continuous and Discrete Objectives: Definition of a System Examples Causality Linearity Time Invariance.
Math 116 Chapter 12.
Inference for regression - Simple linear regression
Hypothesis Testing in Linear Regression Analysis
1 Statistical Analysis - Graphical Techniques Dr. Jerrell T. Stracener, SAE Fellow Leadership in Engineering EMIS 7370/5370 STAT 5340 : PROBABILITY AND.
Introduction Introduction: Mandelbrot Set. Fractal Geometry ~*Beautiful Mathematics*~ FRACTAL GEOMETRY Ms. Luxton.
TIME SERIES by H.V.S. DE SILVA DEPARTMENT OF MATHEMATICS
CPE 619 Simple Linear Regression Models Aleksandar Milenković The LaCASA Laboratory Electrical and Computer Engineering Department The University of Alabama.
Simple Linear Regression Models
CORRELATION & REGRESSION
Verification & Validation
Model of Prediction Error in Chaotic and Web Driven Business Environment Franjo Jović*, Alan Jović ** * Faculty of Electrical Engineering, University of.
Software Reliability SEG3202 N. El Kadri.
1 LES of Turbulent Flows: Lecture 1 Supplement (ME EN ) Prof. Rob Stoll Department of Mechanical Engineering University of Utah Fall 2014.
Quantitative Skills 1: Graphing
Theory of Probability Statistics for Business and Economics.
ECE 8443 – Pattern Recognition ECE 3163 – Signals and Systems Objectives: Definition of a System Examples Causality Linearity Time Invariance Resources:
Copyright © 2011 Pearson Education, Inc. The Simple Regression Model Chapter 21.
Lecture 8 Simple Linear Regression (cont.). Section Objectives: Statistical model for linear regression Data for simple linear regression Estimation.
Chapter 4 Linear Regression 1. Introduction Managerial decisions are often based on the relationship between two or more variables. For example, after.
Problem Formulation Elastic cloud infrastructures provision resources according to the current actual demand on the infrastructure while enforcing service.
CEN st Lecture CEN 4021 Software Engineering II Instructor: Masoud Sadjadi Monitoring (POMA)
1 Self Similar Traffic. 2 Self Similarity The idea is that something looks the same when viewed from different degrees of “magnification” or different.
© Copyright McGraw-Hill Correlation and Regression CHAPTER 10.
Low-Dimensional Chaotic Signal Characterization Using Approximate Entropy Soundararajan Ezekiel Matthew Lang Computer Science Department Indiana University.
Alexey A. Didyk Kherson national technical university, Ukraine
Disk Failures Eli Alshan. Agenda Articles survey – Failure Trends in a Large Disk Drive Population – Article review – Conclusions – Criticism – Disk failure.
CY1B2 Statistics1 (ii) Poisson distribution The Poisson distribution resembles the binomial distribution if the probability of an accident is very small.
Chapter 8: Simple Linear Regression Yang Zhenlin.
Stracener_EMIS 7305/5305_Spr08_ Reliability Data Analysis and Model Selection Dr. Jerrell T. Stracener, SAE Fellow Leadership in Engineering.
Principles of Extrapolation
1 6. Mean, Variance, Moments and Characteristic Functions For a r.v X, its p.d.f represents complete information about it, and for any Borel set B on the.
ECE-7000: Nonlinear Dynamical Systems 2. Linear tools and general considerations 2.1 Stationarity and sampling - In principle, the more a scientific measurement.
Stats Term Test 4 Solutions. c) d) An alternative solution is to use the probability mass function and.
1 Statistical Analysis - Graphical Techniques Dr. Jerrell T. Stracener, SAE Fellow Leadership in Engineering EMIS 7370/5370 STAT 5340 : PROBABILITY AND.
CORRELATION-REGULATION ANALYSIS Томский политехнический университет.
26134 Business Statistics Week 4 Tutorial Simple Linear Regression Key concepts in this tutorial are listed below 1. Detecting.
Fundamentals of Data Analysis Lecture 10 Correlation and regression.
…Don’t be afraid of others, because they are bigger than you
Software Reliability Models.
Correlation and Regression
Data Science introduction.
Chapter 3: Polynomial Functions
Lecture # 2 MATHEMATICAL STATISTICS
Berlin Chen Department of Computer Science & Information Engineering
Berlin Chen Department of Computer Science & Information Engineering
CPSC 641: Network Traffic Self-Similarity
Presentation transcript:

Mark Shereshevsky, Bojan Cukic Fractal Analysis of Resource Exhaustion in Operating Systems: Initial Steps Mark Shereshevsky, Bojan Cukic Jonathan Crowell Computer Sc. and Electrical Engineering Department West Virginia University (smark, cukic,

Software aging phenomenon  “Software aging" phenomenon implies that the state of the software system degrades with time.  The degradation manifests itself in performance decline (excessive paging and swapping activity etc.), possibly leading to crash/hang failures or both.  Degradation is caused, in particular, by the exhaustion of the operating system resources, such as the number of unused memory pages, the number of disk blocks available for page swapping, etc.

Earlier studies of resource exhaustion  Vaidyanathan and Trivedi describe the behavior of operating system recourses as a function of time  Slope (trend) depends on the workload state of the system,  Workload dynamics is modeled as semi-Markov process.  However, in many workload states the dynamics of the resources demonstrates very high variance resulting in very broad confidence intervals. The highly irregular and oscillatory behavior of the data makes any linear trend model insufficient.

Our research objectives  Investigate correlation between fractal properties of the resource data and the system’s workload  Develop fractal-based model of the resource exhaustion process  Apply it to real-time operating systems.  Investigate possibility of using such model for predicting system outages and for preventive maintenance planning

Data collection  Two sets of data collected from WVU machines, plus a set of data from Duke University.  WVU1: resources realMemoryfree and freeSwapSpace on a SUN server,  Data collected once per second for 488,051 seconds (approximately 5 days and 18 hours between June 13 and June 19).  WVU2: resources realMemoryfree, freePageBytes and freeVirtualMemory from an NT server,  10 6 data points were collected (using a fairly simple C++ application) from June 26, 2:50:50 PM to June 27, 6:47:45 PM.

Fractal signals: theory  Most popularly known fractals are two-dimensional “beautiful pictures” (Mandelbrot, Julia sets Koch snowflake, Serpinski triangle, “ferns” etc.) exhibiting complex structure at every scale and self-similarity: zoom any fragment of it with any resolution and get a picture which resembles the whole set.  Functions may also be fractal, i.e. have fractal graphs. Such functions are non-smooth, do not have a derivative at any point.

Example of fractal function Weierstrass function:, a = 0.5, b = 3.

Fractal signals: applications  The study of fractal functions (i.e. those whose plots exhibit fractal properties) has proved important in several scientific domains. The examples of processes described by such functions include the heartbeat oscillations; the speech signals; the network traffic; Solar wind; the stock index charts etc.

Holder exponent: theory and computation  Fractal functions have local singularities exhibiting wild oscillations at every scale.  Mathematical tool for studying the fractal functions:  Hölder exponent measures the intensity of these oscillations  The Hölder exponent of a function f(t) at the point t is defined by the formula:  The exponent ranges from 0 to 1. It equals 1 wherever the function is smooth, and it decreases to zero as the the oscillatory singularities intensify.  For Weirstrass: H w (t) = log2/log3.

Using Hölder exponent to Analyze time series  The study of experimental time series with fractal structure is often done by computing and analyzing its Hölder exponent.  As we deal with a discrete sequence of measurements we cannot apply the definition of Hölder exponent directly. We have developed an algorithm for computing the Hölder exponent of the discrete time series.  Our algorithm showed very robust performance on several test examples.

Performance of our algorithm for computing Hölder exponent Test signal: generalized Weierstrass function: with s(t) = |sin (5  t)| sampled with step The HE is known to be s(t). The Holder exponents computed by our algorithm versus actual values.

Hölder exponent computations for memory free data  Two experiments performed on the realMemoryFree time series from the SUN server.  SET1: A three-hour set of data collected in early morning hours: from 2:30 AM to 5:30 AM;  SET1: Collected in the midday: from 12 noon to 3 PM.  The measurements taken once per second (10,800 points in each time series)  Both time series show multifractal behavior.  Since the Hölder exponent plots are extremely chaotic, we also computed the histograms of the Hölder exponents, in order to visualize their distribution over the range [0, 1].

Plots of data with Hölder exponent (1) realMemoryFree from 2:30 AM to 5:30 AM (“low workload”); The Hölder exponents for the data.

Hölder exponent hystogram (1) The histogram (distribution) of the values of Hölder exponent (SET1: low workload).

Plots of data with Hölder exponent (2) realMemoryFree data from SUN server (high workload); Hölder exponents for the data sets.

Hölder exponent hystogram (2) The histogram of Hölder exponent for realMemoryFree (high workload).

Fractality of memory time series reflects workload  The histograms show clear differences in the shape of Hölder exponents distribution for these the two sets of data.  The midday hours yield realMemoryFree data with very few Hölder exponents in the upper part of the range (very few points where data are relatively smooth).  realMemoryFree time series obtained in the night hours is quite “fat” in the upper part of the range (i.e. significant portions of the signal are relatively “smooth”).  The higher workload data gives a less dispersed spectrum of Hölder exponents which are mainly concentrated in the [0.4, 0.6] range.  This is sign of higher self-similarity (a perfectly self-similar signal has a strictly constant Hölder exponent).

Future work Recognition of the systems behavioral patterns from fractal properties of resource data; Correlation between fractal characteristics of the system resource data and malfunction/failure events. Experimental application to a real-time application Substantial benefit from preventive maintenance.