1. Universidade Federal de Pernambuco
Paulo Maciel, Rubens Matos, Bruno Silva, Jair Figueiredo, Danilo Oliveira,Iure Fé, Ronierison Maciel and Jamilson Dantas. 
Mercury: Performance and Dependability Evaluation of Systems with Exponential, Expolynomial, and General Distributions.
The 22nd IEEE Pacific Rim International Symposium on Dependable Computing (PRDC 2017)
Mercury: Performance and Dependability Evaluation of Systems with Exponential, Expolynomial, and General Distributions
Paulo R. M. Maciel et al.
Centro de Informática
Universidade Federal de Pernambuco

2. Agenda Motivation Architecture overview Mercury Language
Moment Matching
Hierarchical Blocks in RBD
Case Study
Conclusion

3. Motivation
Since 1989 I have been working using timed models, particularly Petri nets variants.
Initially, I considered more timed models derived from Place/Transition Petri nets.
Then, I started applying stochastic modeling, primarily employing SPN for simulation, then to numerical/analytical solution.
After, a natural step was considering Markov chains, and combinatorial models like RBD and FT.
Over the years, our group has used many academic and commercial tools.
Some academic tools we have adopted in our group:
INA, Design CPN and CPN tools, Great SPN, TimeNet, SHARPE …

4. Motivation Why implementing another tool? Cons. (some) : Pros:
There are many good tools already available
It is likely that your (“first”) results should be worse than those provided by these established tools
Pros:
It is an objective means for keeping your previous research results “alive”
Allow practical/real connection between consecutive research works
Having control over the products (software, models, methods etc) conceived and implemented in the group by graduate students, that, after finishing their respective research projects, start a new phase in their lives,
Learning (in depth) of the respective methods.

5. Motivation My decision was: implement it.
So we began implementing the tool in 2008 by conceiving, specifying and coding a simulation kernel for SPN.
From then on many functionalities and models have been included and are now supported by the tool.

6. Architecture: an overview

7. Architecture: an overview
Some of the following screenshots were presented in our previous work.
Mercury: An Integrated Environment for Performance and Dependability Evaluation of General Systems.
IEEE 45th Dependable Systems and Networks Conference (DSN-2015).
June 22 – 25, Rio de Janeiro, RJ, Brazil

8. Architecture: Main Editors (SPN)

9. Architecture: Main Editors (CTMC)

10. Architecture: Main Editors (DTMC)

11. Architecture: Main Editors (RBD)

12. Architecture: Main Editors (EFM)

13. Miscellaneous

14. Mercury Language

15. Mercury Language
Model definition

16. Mercury Language
State definition.
'up' keyword defines working states

17. Mercury Language
Transitions definition

18. Mercury Language
Metric definitions

19. Mercury Language
Model instantiation

20. Mercury Language
Rates definition

21. Mercury Language
Model solving

22. Moment Matching
Moment matching is used to perform numerical analysis in SPN models in case of activities with non-exponential time distributions or that only mean and standard deviation are available;
Mercury shows the subnet and the distribution parameters to be used to match the first two moments informed by the user;
Mercury will present the most appropriate distribution between Exponential, Erlang, Hypoexponential, Hyperexponential, Cox-1, and Cox 2.

23. Moment Matching Erlang Hyperexponential Exponential Hypoexponential
e.g.
Cox-1
Cox-2

24. Hierarchical Blocks in RBD
RBD models have interesting features for reliability modeling, such as simplicity, versatility, and expressive power.
This simplicity comes at the expense of some tradeoffs, such as imposing the use of the exponential distribution in all failure and repair rates
By combining the sum of disjoint products (SDP)/ structural function (SFM) methods with stochastic simulation, the Mercury tool enables using other distributions than the exponential

25. Hierarchical Blocks in RBD
Failure and Repair Distribution

26. Hierarchical Blocks in RBD
Available Distributions

27. Hierarchical Blocks in RBD
Hierarchical blocks are translated to equivalent SPN models for simulation and their results are combined in the top-level RBD model.
(Scripting language)

28. Hierarchical Blocks in RBD
Failure and Repair Erlang Distribution
Exponential

29. Hierarchical Blocks in RBD
Failure and Repair Erlang Distribution
Exponential
Generate script
Equivalent to

30. Hierarchical Blocks in RBD
Failure and Repair Erlang Distribution
Exponential
Generate script
Numerical
Equivalent to
Simulation

31. Hierarchical Blocks in RBD
Failure and Repair Erlang Distribution
Exponential
Generate Scripting Language
Numerical
Equivalent to
pb1
pb2
Simulation
Solution by Closed Forms

32. Hierarchical Blocks in RBD
All metrics can be obtained adopting hierarchical blocks.

33. Case Study
We developed a case study that demonstrates some of the mentioned features.
We evaluate a single VoD (Video on Demand) server and network architecture to evaluate packet loss and congestion according to video and network characteristics.

34. Case Study Four video formats were employed; MP4 MPG AVI FLV
Each video has a duration of five minutes, and an experiment to collect data from the real world setting was executed thirty times for each video.

35. Case Study
We measured three types of data, which are useful for building the SPN model: the packet loss probability (P_err transition), mean time between video packets (TTP transition), and average network latency (NL transition).

36. Case Study
We used moment-matching to determine which expolynomial distribution best fits the packet interarrival and latency times, based on data collected through experimental testbeds.
The respective mean (µ) and standard deviations (σ) were measured. The transitions were refined using Mercury moment matching features.
Hyperexponential and Erlang distributions are the expolynomial distributions most suitable to the measured packet interarrival time and network latency.

37. Refined model, considering Hyperexponential and Erlang subnets after using Mercury moment-matching feature

38. Case Study
In order to verify whether data from model and real system are statistically equivalent, we adopt the bootstrap method with the testbed experiments data, with 95% confidence interval.
All model results are within the confidence interval of real testbed data.

39. Case Study
Using the proposed SPN model, we evaluated the QoS metrics for each video type on each network scenario.
WiFi
3G (EVDO)
3.5G (HSPA+)
4G (LTE)

40. Case Study 4G has the smallest packet loss in all cases
WiFi has the biggest loss, followed by 3G
Although, it is important to remark that the 3G network has a large latency, that is an additional drawback besides the packet loss.

41. Final remarks
Mercury is an academic tool. The executable code is available for academic use.
If you would like to try it out, I encourage you to access
and go to Tools. From there, you will get a form, sign it
and download it.
There, you will also find a detailed manual.
Presently, we have a group of 6 graduate students working on it. DTMC modeling have been recently introduced in the tool.
We'd appreciate if you let us know about bugs and improvements, but we cannot guarantee maintenance support.

42. Thank you! Paulo Maciel