Universidade Federal de Pernambuco Paulo Maciel, Rubens Matos, Bruno Silva, Jair Figueiredo, Danilo Oliveira,Iure Fé, Ronierison Maciel and Jamilson Dantas.  Mercury: Performance and Dependability Evaluation of Systems with Exponential, Expolynomial, and General Distributions. The 22nd IEEE Pacific Rim International Symposium on Dependable Computing (PRDC 2017) Mercury: Performance and Dependability Evaluation of Systems with Exponential, Expolynomial, and General Distributions Paulo R. M. Maciel et al. prmm@cin.ufpe.br www.modcs.org Centro de Informática Universidade Federal de Pernambuco

Agenda Motivation Architecture overview Mercury Language Moment Matching Hierarchical Blocks in RBD Case Study Conclusion

Motivation Since 1989 I have been working using timed models, particularly Petri nets variants. Initially, I considered more timed models derived from Place/Transition Petri nets. Then, I started applying stochastic modeling, primarily employing SPN for simulation, then to numerical/analytical solution. After, a natural step was considering Markov chains, and combinatorial models like RBD and FT. Over the years, our group has used many academic and commercial tools. Some academic tools we have adopted in our group: INA, Design CPN and CPN tools, Great SPN, TimeNet, SHARPE …

Motivation Why implementing another tool? Cons. (some) : Pros: There are many good tools already available It is likely that your (“first”) results should be worse than those provided by these established tools Pros: It is an objective means for keeping your previous research results “alive” Allow practical/real connection between consecutive research works Having control over the products (software, models, methods etc) conceived and implemented in the group by graduate students, that, after finishing their respective research projects, start a new phase in their lives, Learning (in depth) of the respective methods.

Motivation My decision was: implement it. So we began implementing the tool in 2008 by conceiving, specifying and coding a simulation kernel for SPN. From then on many functionalities and models have been included and are now supported by the tool.

Architecture: an overview

Architecture: an overview Some of the following screenshots were presented in our previous work. Mercury: An Integrated Environment for Performance and Dependability Evaluation of General Systems. IEEE 45th Dependable Systems and Networks Conference (DSN-2015). June 22 – 25, 2015. Rio de Janeiro, RJ, Brazil

Architecture: Main Editors (SPN)

Architecture: Main Editors (CTMC)

Architecture: Main Editors (DTMC)

Architecture: Main Editors (RBD)

Architecture: Main Editors (EFM)

Miscellaneous

Mercury Language

Mercury Language Model definition

Mercury Language State definition. 'up' keyword defines working states

Mercury Language Transitions definition

Mercury Language Metric definitions

Mercury Language Model instantiation

Mercury Language Rates definition

Mercury Language Model solving

Moment Matching Moment matching is used to perform numerical analysis in SPN models in case of activities with non-exponential time distributions or that only mean and standard deviation are available; Mercury shows the subnet and the distribution parameters to be used to match the first two moments informed by the user; Mercury will present the most appropriate distribution between Exponential, Erlang, Hypoexponential, Hyperexponential, Cox-1, and Cox 2.

Moment Matching Erlang Hyperexponential Exponential Hypoexponential e.g. Cox-1 Cox-2

Hierarchical Blocks in RBD RBD models have interesting features for reliability modeling, such as simplicity, versatility, and expressive power. This simplicity comes at the expense of some tradeoffs, such as imposing the use of the exponential distribution in all failure and repair rates By combining the sum of disjoint products (SDP)/ structural function (SFM) methods with stochastic simulation, the Mercury tool enables using other distributions than the exponential

Hierarchical Blocks in RBD Failure and Repair Distribution

Hierarchical Blocks in RBD Available Distributions

Hierarchical Blocks in RBD Hierarchical blocks are translated to equivalent SPN models for simulation and their results are combined in the top-level RBD model. (Scripting language)

Hierarchical Blocks in RBD Failure and Repair Erlang Distribution Exponential

Hierarchical Blocks in RBD Failure and Repair Erlang Distribution Exponential Generate script Equivalent to

Hierarchical Blocks in RBD Failure and Repair Erlang Distribution Exponential Generate script Numerical Equivalent to Simulation

Hierarchical Blocks in RBD Failure and Repair Erlang Distribution Exponential Generate Scripting Language Numerical Equivalent to pb1 pb2 Simulation Solution by Closed Forms

Hierarchical Blocks in RBD All metrics can be obtained adopting hierarchical blocks.

Case Study We developed a case study that demonstrates some of the mentioned features. We evaluate a single VoD (Video on Demand) server and network architecture to evaluate packet loss and congestion according to video and network characteristics.

Case Study Four video formats were employed; MP4 MPG AVI FLV Each video has a duration of five minutes, and an experiment to collect data from the real world setting was executed thirty times for each video.

Case Study We measured three types of data, which are useful for building the SPN model: the packet loss probability (P_err transition), mean time between video packets (TTP transition), and average network latency (NL transition).

Case Study We used moment-matching to determine which expolynomial distribution best fits the packet interarrival and latency times, based on data collected through experimental testbeds. The respective mean (µ) and standard deviations (σ) were measured. The transitions were refined using Mercury moment matching features. Hyperexponential and Erlang distributions are the expolynomial distributions most suitable to the measured packet interarrival time and network latency.

Refined model, considering Hyperexponential and Erlang subnets after using Mercury moment-matching feature

Case Study In order to verify whether data from model and real system are statistically equivalent, we adopt the bootstrap method with the testbed experiments data, with 95% confidence interval. All model results are within the confidence interval of real testbed data.

Case Study Using the proposed SPN model, we evaluated the QoS metrics for each video type on each network scenario. WiFi 3G (EVDO) 3.5G (HSPA+) 4G (LTE)

Case Study 4G has the smallest packet loss in all cases WiFi has the biggest loss, followed by 3G Although, it is important to remark that the 3G network has a large latency, that is an additional drawback besides the packet loss.

Final remarks Mercury is an academic tool. The executable code is available for academic use. If you would like to try it out, I encourage you to access www.modcs.org and go to Tools. From there, you will get a form, sign it and download it. There, you will also find a detailed manual. Presently, we have a group of 6 graduate students working on it. DTMC modeling have been recently introduced in the tool. We'd appreciate if you let us know about bugs and improvements, but we cannot guarantee maintenance support.

Thank you! Paulo Maciel