Assessing the SoP of MBE in the Embedded Systems Domain Xubo Miao MSc, School of Computing Supervisor: James R. Cordy

1. Motivation MBE aims at increasing the effectiveness of engineering by using models as key artifacts in the development process. Only little work targeting the embedded systems domain The authors did a study on the use and the assessment of the MBE in embedded systems domain

2. Related Work Agner et al. present the results of a survey on the use of UML and model-driven approaches in the Brazilian embedded software development industry. – 45% of the 209 participants use UML. And the majority are experienced developers working at medium-sized companies. – Models are mainly used for documentation, with only little use of code generation or model-centric approaches in general.

2. Related Work (Cont’d) Kirstan and Zimmermann report a case study within the automotive domain. – Positive Effects: an earlier detection of errors, a higher degree of automation, and cost savings during the initial phases of development – Negative Effects: large function models can become too complex and that interoperability between tools is difficult.

3. Two Research Questions RQ1: What is the current SoP and the assessment of MBE in the embedded systems domain? – Modeling environments, modeling languages, types of notations, purposes that models are used for RQ2: How does the use and the assessment of MBE differ between different demographic subgroups in the embedded systems domain? – E.g. automotive and avionics domain, new and highly experienced MBE users

4.1 Research Methodology – Study Design The survey questionnaire consisted of 24 closed-ended and open-ended questions – 1 st Part: contains 13 questions gathering demographic data – 2 nd Part: contains 11 questions addressed RQ1 – Both 2 parts are considered together for answering RQ2. One Minute for you to go through the questionnaire

4.1 Research Methodology – Study Design (Cont’d) Derived 24 hypotheses from the related work to guide the data analysis for RQ1 Derived 8 hypotheses from their own view on MBE after designing the questionnaire to answer the RQ2 One Minute for you to go through all hypotheses

4.2 Research Methodology – Data Collection The surveys are distributed to partners from five different projects. 121 out of 196 surveys are completed. 9 surveys are excluded based on degradation points computed from missing answers and the time to fill out each survey page. Therefore, 112 answered surveys for data analysis

5.1 Results – RQ1: State of Practice Modeling tools and languages – Tools: most use Matlab/Simulink (50) or Eclipse-based MBE tools (34) – Notations: Finite State Machines (74), sequence-based models (64) and block diagrams (61) – F unctional aspects of a system participants already use models: structure (68), discrete state specifications (48) and static interfaces (47) are most common

5.1 Results – RQ1: State of Practice Needs for Introducing MBE

5.1 Results – RQ1: State of Practice Purposes of Models

5.1 Results – RQ1: State of Practice Positive & Negative Effects of MBE

5.1 Results – RQ1: State of Practice Shortcomings of MBE

5.2 Results – RQ2: Differences by Subgroups They use Fisher’s exact test (two-tailed) with a level of significance α ≤ H2.1, H2.6, H2.7 - p ≥ 0.05 H2.4 – not enough data for each subgroup in order to compare the groups Therefore, only H2.2, H2.3, H2.5, H2.8 are statistically significant.

5.2 Results – RQ2: H2.2 Traceability (p = ), safety (p = 0.018), and reusability (p = 0.019) Supporters perceive the effects of MBE on these three aspects significantly more positive than opponents and nuetrals.

5.2 Results – RQ2: H2.3 Cost (p = 0.016) and traceability (p = 0.006). Participants who are still using MBE report more positive effects on cost and traceability than participants who stopped using MBE. A possible explanation for the few significant differences might be that participants who stopped using MBE did so because they moved to a different position.

5.2 Results – RQ2: H2.5 Quality(p = 0.011) Participants who reported that many usability issues with tools mostly or fully applies rated the effects on quality slightly less positive than participants who reported that usability issues apply at most partly.

5.2 Results – RQ2: H2.8 Supporters: 51 of MBE reported to use MBE tools more than non-MBE tools or only MBE tools, and 18 reported to use less MBE tools than non-MBE tools or only MBE tools at all. Opponents and Neutrals: 5 and 13 MBE promoters use more MBE tools in comparison to subjects neutral or opposed to MBE.

6. Conclusion MBE is widespread in the embedded domain. Models are clearly not only used for informative and documentation purposes, but also used for simulation, code generation, test case generation, traceability and timing analysis. The survey respondents reported mostly positive effects of Model-Based Engineering