1. Modeling and Reasoning about Software Systems Containing Uncertainty and Variability Marsha Chechik University of Toronto September 11, 2015 TASE’15, Nanjing, China

2. Acknowledgements Many thanks for colleagues in Toronto Michalis Famelis Rick Salay Julia Rubin Alessio DiSandro Yi Li Benson Quach Cynthia Disenfield and elsewhere in the world Andrej Wazowski Aws Albarghouthi Michal Antkewitz Krzysztof Czarnecki Levi Lucio Gehan Selim Gabrielle Taenzer Daniel Struber 2

3. Why Models? Traditional Engineering Approach – Abstract & Precise – Amenable to analysis – Complexity: Model << System Pre-development and pre-deployment analysis – Early detection -> cheaper fixes Cost < Benefit 3

4. Software Engineering Models Requirements Architecture Behaviour Static Design Use Cases Deployment Concepts Structure Abstraction Purposeful Reasoning 4

5. Model-Based Software Engineering (MBSE) Toolbox (partial list!) Support for: Development, analysis, refinement, transformation of models + Communication, documentation 5

6. Model-driven engineering under presence of uncertainty Model-driven engineering under presence of variability Uncertainty and variability combined? Some current/future work 6

7. Uncertainty Sources Modeling Verification of models containing uncertainty – Technology – Scalability / effectiveness – Dealing with analysis results Other parts of MDSE toolbox Tool support 7

8. This is Natalie. Natalie is a modeler. Natalie faces uncertainty in her everyday work 8

9. Alternative Designs Hmm, I don’t know which one, yet. 9

10. Conflicting Stakeholder Opinions What do I do until they decide? 10

11. Incomplete Information I don’t know everything about this, yet. 11

12. Uncertainty in Software Development Uncertainty about the content of the model. Many design alternatives Conflicting stakeholder opinions Incomplete information 12

13. MBSE Toolbox for Natalie? Existing tools do not apply Options: – Wait until uncertainty gets resolved (how long?) – Make a provisional decision and run with it (need backtracking!) We propose: – Defer resolution of uncertainty but incorporate uncertainty handling into the development process to allow progress – Lift standard MDE tools to handle models with uncertainty – Add new tools: Help Natalie express her uncertainty Help Natalie determine what she does not yet know Help Natalie manage and reduce uncertainty 13

14. Modeling Uncertainty: Example Solver SolverException + effect : String Unsure if it should be an inner class. Unsure if we need this field. Notation means “inner class” in UML 14

15. Some Decisions to Make Options for assigning values to states of confidence Quantitative Examples: 0, 0.3, 0.75 Good for capturing likelihood of events Hard to capture one’s own uncertainty Qualitative Examples: yes/no/don’t know strongly … strongly agree disagree Good for capturing one’s own level of uncertainty Less precise than quantitative 15

16. How to Represent Uncertainty? Ask users Goals: model uncertainty explicitly, keep syntax familiar [MiSE’13] 16

17. Representing Uncertainty Partial Models Points of uncertainty (“May elements”) explicated using syntactic annotations Solver SolverException + effect : String Unsure if it should be an inner class. Unsure if we need this field. [ICSE12] 17

18. Representing Uncertainty Partial Models Points of uncertainty (“May elements”) explicated using syntactic annotations Propositional variables: “the element exists” Solver SolverException + effect : String X Y [ICSE12] 18

19. Representing Uncertainty Solver SolverException + effect : String X Y Solver SolverException Solver SolverException Solver SolverException + effect : String Solver SolverException + effect : String x=F, y=F x=T, y=F x=F, y=Tx=T, y=T 4 concretizations: 4 ways to resolve uncertainty. 19

20. Representing Uncertainty Partial Models Points of uncertainty (“May elements”) explicated using syntactic annotations Restrictions to the set of concretizations can be captured in the “May formula” Solver SolverException + effect : String X Y X v Y 20

21. Representing Uncertainty Solver SolverException + effect : String X Y Solver SolverException Solver SolverException Solver SolverException + effect : String Solver SolverException + effect : String x=F, y=F x=T, y=F x=F, y=Tx=T, y=T X v Y 21

22. Benefits – Partial model refinement = uncertainty reduction – Formal: allows analysis and automation – Semantics-independent – applies to models of different types – Use instead of concrete models … … to allow deferral of decisions without delaying progress Partial Models A partial model represents the entire set of possible concrete models 22

23. Uncertainty Sources Modeling Verification of models containing uncertainty – Technology – Scalability / effectiveness – Dealing with analysis results Other parts of MDSE toolbox Tool support 23

24. Model Checking Model M Property P Does M satisfy P? counterexample yes the property holds! verified

25. Properties to Verify 25 Example property: “Every inner class has at least one attribute” Natalie’s favorite analysis technique How can you apply it to partial models? Class.allInstances()->forAll( c | not c.nestedIn->isEmpty() implies not c.ownedAttributes->isEmpty() ) Class.allInstances()->forAll( c | not c.nestedIn->isEmpty() implies not c.ownedAttributes->isEmpty() )

26. Model Checking for Models with Uncertainty Partial Model M Property P Does M satisfy P? yes the property holds in all concretizations! counterexample + concretizations where property fails

27. Verification: the Naïve Approach Example property: “Every inner class has at least one attribute” Enumerate: 27

28. Verification: Lifting 28 Example property: “Every inner class has at least one attribute” Natalie’s favorite analysis technique, LIFTED Applies directly to the partial model Does not enumerate concretizations Computes result using three-valued logic …all concretizations …some but not all …none Property holds for…

29. Verification: Lifting Checking syntactic properties [ICSE12] : – Language independent – SMT-based 29

30. SMT Encoding ;Concretizations (define-sort NodeType () Int) (define-sort Node () Int) (declare-fun node (Node) Bool) (declare-fun nodeType (Node) NodeType) (define-sort EdgeType () Int) (define-sort Edge () Int) (declare-fun edge (Edge) Bool) (declare-fun edgeType (Edge) EdgeType) (declare-fun src (Edge) Node) (declare-fun tgt (Edge) Node) (declare-const CLASS NodeType) (assert (= CLASS 1)) (declare-const ATTRIBUTE NodeType) (assert (= ATTRIBUTE 2)) (declare-const OPERATION NodeType) (assert (= OPERATION 3)) (declare-const DEPENDENCY EdgeType) (assert (= DEPENDENCY 4)) (declare-const ASSOCIATION EdgeType) (assert (= ASSOCIATION 5)) (declare-const NESTEDINREFERENCE EdgeType) (assert (= NESTEDINREFERENCE 6)) (declare-const SUPERCLASSREFERENCE EdgeType) (assert (= SUPERCLASSREFERENCE 7)) ;Model (declare-const Solver Node) (assert (= Solver 8)) (declare-const SolverException Node) (assert (= SolverException 9)) (declare-const Y Node) (assert (= Y 10)) (declare-const X Edge) (assert (= X 11)) ;End Model (assert (= (nodeType Solver) CLASS)) (assert (= (nodeType SolverException) CLASS)) (assert (= (nodeType Y) ATTRIBUTE)) (assert (= (edgeType X) NESTEDINREFERENCE)) (assert (= (src X) SolverException)) (assert (= (tgt X) Solver)) (assert (=> (edge X) (and (node SolverException) (node Solver)) )) ;Model is Complete (assert(forall ((c Node)) (=> (node c) (and (>= c 8) (<= c 10) ) ))) (assert(forall ((c Edge)) (=> (edge c) (and (>= c 11) (<= c 11) ) ))) ;Solver Exists (assert (node Solver)) ;SolverException Exists (assert (node SolverException)) ;May Formula (assert (or X Y)) 30

31. Verification: Lifting Checking syntactic properties [ICSE12]: – Language independent – SMT-based Checking semantic properties: – Depends on language semantics – Requires custom adaptation of verification technique – Examples: Goal satisfaction in i* models [RE12, RE14] MTSA: Model checking and other analyses over MTSs [ASE08] 31

32. Scalability of Verification Q: Is verification of models with uncertainty feasible? – How does it compare with checking each model individually? – What about comparison with the classical verification? Q: How does the level of uncertainty affect this feasibility? Experiments: with random inputs and real case studies Random inputs: varying size of the model and level of uncertainty speedup = time for set of conventional models time to check partial model 32

33. Results 33

34. Case Study: Checking Properties Why? Triangulate experimental results (random inputs) with observations from a real-word scenario Case study details: – Bug from the UMLet bugzilla – Realistic bug fixes – Uncertainty: which combination of actions that fix the bug to choose from? – We checked two properties from the literature [SMSJ03] – Resulting partial model: 27,261 elements, 220 concretizations [SMSJ03] R.V. Der Straeten, T. Mens, J. Simmonds, and V. Jonckers. “Using Description Logic to Maintain Consistency between UML Models”. In Proc. of UML’03, pp 326-340, 2003 34

35. Results 35

36. Results of Evaluation Is there a speedup? -Yes, it is consistently faster than reasoning with the set. How is speedup affected by changing model size and levels of uncertainty? -Speedup decreases with model size -Speedup increases with level of uncertainty -No slowdowns! Reasoning with Partial models vs Reasoning with a set of conventional models 36

37. Uncertainty Sources Modeling Verification of models containing uncertainty – Technology – Scalability / effectiveness – Dealing with analysis results Other parts of MDSE toolbox Tool support 37

38. Analysis Results Solver SolverException + effect : String Class.allInstances()->forAll( c | not c.nestedIn->isEmpty() implies not c.ownedAttributes->isEmpty() ) Class.allInstances()->forAll( c | not c.nestedIn->isEmpty() implies not c.ownedAttributes->isEmpty() ) Show user an example: -- A concretization where the property holds

39. Analysis Results Solver SolverException + effect : String Class.allInstances()->forAll( c | not c.nestedIn->isEmpty() implies not c.ownedAttributes->isEmpty() ) Class.allInstances()->forAll( c | not c.nestedIn->isEmpty() implies not c.ownedAttributes->isEmpty() ) Show user a counterexample: -- A concretization where the property does not hold

40. Using Analysis Results Solver SolverException + effect : String Class.allInstances()->forAll( c | not c.nestedIn->isEmpty() implies not c.ownedAttributes->isEmpty() ) Class.allInstances()->forAll( c | not c.nestedIn->isEmpty() implies not c.ownedAttributes->isEmpty() ) Generate the diagnostic core: -- Partial model of concretizations where property does/does not hold X

41. Tool Support for Responding to Analysis Greyed out: not part of the diagnostic core Part of the core

42. Tool Support for Responding to Analysis Offending May elements removed

43. Property-Based Refinement SATSolver SolverException + effect : String X Y X v Y Property Φ1: “Every inner class has at least one attribute” Goal: Create a partial model of the concretizations for which the property holds. (X v Y) Λ Φ1 We just need to update the May formula: 43

44. Uncertainty Sources Modeling Verification of models containing uncertainty – Technology – Scalability / effectiveness – Dealing with analysis results Other parts of MDSE toolbox Tool support 44

45. Uncertainty-Reducing Refinement Systematically incorporate new information … – … Manually Edit the uncertainty annotations – … Using automated transformations Verify that a transformation is always uncertainty reducing. 45 Unsure if it should be an inner class. X v Y Unsure if we need this field. I want to get rid of effect. [FASE12, JOT15]

46. Transformations 46 The transformations assume inputs that don’t contain uncertainty Like every good MBE practitioner, Natalie uses a variety of MTs But Natalie’s models do! [MODELS13]

47. Transforming Models with Uncertainty 47 Natalie should be able to use model transformations [MODELS13]

48. Transforming Models with Uncertainty 48 We need to lift Natalie’s transformations so that they can apply to models with uncertainty Existing transformation techniques do not support this! Natalie should be able to use model transformations [MODELS13]

49. Goal: “Lifting” 49 class1 + attribute : type class1 - attribute : type + getAttribute():type class1 class2 RHS LHS NAC Solver SolverException + effect : String Solver SolverException + effect : String Solver SolverException Solver SolverException -effect : String +getEffect() : String Solver SolverException + effect : String Solver SolverException

50. Goal: “Lifting” 50 class1 + attribute : type class1 - attribute : type + getAttribute():type class1 class2 RHS LHS NAC Solver SolverException + effect : String Solver SolverException + effect : String Solver SolverException ( X ∧ ¬Y ∧ ¬a ∧ ¬b) v (¬X ∧ Y ∧ ¬a ∧ b) v ( X ∧ Y ∧ a ∧ ¬b) ( X ∧ ¬Y ∧ ¬a ∧ ¬b) v (¬X ∧ Y ∧ ¬a ∧ b) v ( X ∧ Y ∧ a ∧ ¬b) Solver SolverException + - effect : String +getEffect() : String X Y a b

51. Goal: “Lifting” 51 class1 + attribute : type class1 - attribute : type + getAttribute():type class1 class2 RHS LHS NAC Solver SolverException + effect : String X Y X v Y ( X ∧ ¬Y ∧ ¬a ∧ ¬b) v (¬X ∧ Y ∧ ¬a ∧ b) v ( X ∧ Y ∧ a ∧ ¬b) ( X ∧ ¬Y ∧ ¬a ∧ ¬b) v (¬X ∧ Y ∧ ¬a ∧ b) v ( X ∧ Y ∧ a ∧ ¬b) Solver SolverException + - effect : String +getEffect() : String X Y a b

52. Lifting Technique Summary 52 Solver SolverException + effect : String X Y X v Y ( X ∧ ¬Y ∧ ¬a ∧ ¬b) v (¬X ∧ Y ∧ ¬a ∧ b) v ( X ∧ Y ∧ a ∧ ¬b) ( X ∧ ¬Y ∧ ¬a ∧ ¬b) v (¬X ∧ Y ∧ ¬a ∧ b) v ( X ∧ Y ∧ a ∧ ¬b) Solver SolverException + - effect : String +getEffect() : String X Y a b Step 1: Determine applicability (Henshin + Z3 SMT Solver) Step 2: Transform graph (Hanshin) Step 3: Transform formula (Java + Z3 input strings)

53. Making the Toolbox... MMINT: Model Management INTeractive [MODELS15] – A platform for developing (typed) model management tools – Model management environment: Diff, refine, merge, etc. Available at https://github.com/adisandro/MMINT CSP SAT/SMT Henshin Alloy 53

54. Making the Toolbox... Uncertainty- aware MMINT: Model Management INTeractive – A platform for developing (typed) model management tools – Model management environment: Diff, refine, merge, etc. Available at https://github.com/adisandro/MMINThttps://github.com/adisandro/MMINT MU-MMINT [ICSE15] – lifting of model management operators + new ones CSP SAT/SMT Henshin Alloy MU-MMINT (pronounced “Moomin”) 54

55. Summary: Uncertainty Modeling and Analysis Our choice: – Qualitative, possibilistic uncertainty modeling – Keeping concrete syntax familiar – Encoding an exact set of potential classical models – Refinement – reducing the set of possibilities – Verification – lifting of classical approaches Explicit uncertainty modeling with Partial Models + May formulas 55 ConcretizationsPartial model

56. Model-driven engineering under presence of uncertainty Model-driven engineering under presence of variability Uncertainty and variability combined? Some current/future work 56

57. Variability Context Modeling Verification of models containing variability – Technology – Scalability / effectiveness – Dealing with analysis results Other operators for handling variability 57

58. Product Lines +Dry +Dry/Delay +Delay … Goal: Help develop, manage, reuse a large number of similar but different artifact variants (products) Example: Washing Machine Co. +Heat 58

59. Software Product Line Engineering planned and predictivesoftware reuse a discipline that promotes planned and predictive software reuse K. Pohl et al., Software Product Line Engineering: Foundations, Principles, and Techniques, 2005 P. C. Clements and L. Northrop, Software Product Lines: Practices and Patterns, 2001 59

60. Modeling: Product Line Structure + Terminology Product line (annotative) represented by – Domain Model – combined parts from all products, annotated by features (presence conditions) – Feature Model – shows possible features and restrictions for product combinations Example : Washing Machine Co. Feature Model Wash Heat Dry Delay excludes Domain Model 60

61. Product Derivation (or Configuration) The process of deriving a specific software product from: – … artifacts with variability – … feature to artifacts association – … a product definition 61

62. Product Configuration – Example 1 +Dry/Delay +Heat product – Feature configuration: {Wash, Heat} Feature Model Wash Heat Dry Delay excludes +Heat 62

63. Product Configuration – Example 2 +Dry/Delay +Dry/Delay product – Feature configuration: {Wash, Dry, Delay} Feature Model Wash Heat Dry Delay excludes +Dry/Delay 63

64. Domain Model Washing Machine Product Line Locking Waiting Washing entry/TempCheck() Drying Unlocking [heatEnabled;delayEnabled] / HeaterOn() / HeaterOff(); wash.Start(); / QuickCool() [not heatEnabled;not delayEnabled]/ wash.Start(); Presence Conditions Feature Model Wash Heat Dry Delay excludes Heat Delay Heat Dry Heat Delay 64

65. Domain Model Feature Model Washing Machine Product Line: Configuring a Product Locking Waiting Washing entry/TempCheck() Drying Unlocking [heatEnabled;delayEnabled ]/ HeaterOn() / HeaterOff(); wash.Start(); / QuickCool() [not heatEnabled;not delayEnabled]/ wash.Start(); Wash Heat Dry Delay excludes Heat Delay Heat Dry Heat Delay 65

66. +Dry/Delay Variant Result: +Dry/Delay State Machine Locking Waiting Washing Drying Unlocking [ delayEnabled] / wash.Start(); / QuickCool() [ not delayEnabled]/ wash.Start(); 66

67. +Dry/Delay Variant Result: +Dry/Delay State Machine Locking Waiting Washing Drying Unlocking [delayEnabled] /wash.Start(); / QuickCool() 67 [not delayEnabled]/ wash.Start();

68. Variability Context Modeling Verification of models containing variability – Technology and its scalability / effectiveness – Dealing with analysis results Other operators for handling variability 68

69. Model Checking Model M Property P Does M satisfy P? counterexample yes the property holds! verified

70. Model Checking for Product Lines Product Line Model M Property P Does M satisfy P? yes the property holds for all products! counterexample + a list of violating products

71. Methods for Checking Systems with Variability Input: typically featured transition systems and LTL properties – “Don’t crash into a wall.” ¬  overlap(vehicle, wall) – “Each request is answered.”  (request ⇒  response) Methods: – Special-purpose model-checker [Classen – ICSE10] – Transformation to.. … Spin [Classen – STTT12] … NuSMV [Classen – ICSE11] … IC3 [Ben-David – ICSE15] 71

72. Dealing with Analysis Results Model-checking returns a list of violating products Methods: – Remove the offending products (easy to automate) – Debug and reverify to ensure that property holds for all products (no specific support) 72

73. Variability Context Modeling Verification of models containing variability – Technology – Scalability / effectiveness – Dealing with analysis results Other operators for handling variability 73

74. Lifting Transformations 74 [ICSE14]

75. Other PL Operations 75 [SPLC13, STTT15]

76. Feature Location and Migration Across Products Given a history (sequence of commits) and a functionality exercised by tests, find a “semantic slice” that is well-formed and can pass the same tests Locating features in cloned variants [ASE12, FASE14, ongoing] 76 [ASE15]

77. Detecting and Resolving Feature Interaction 77 Goal: compositional detection and resolution (using priorities) of behavioral interference properties between features Given a selection of features, understanding of the spec of the resulting system [In progress]

78. Summary: Variability Modeling and Analysis Product lines – an industrially-relevant method for compactly encoding sets of similar but different products – Difference is explicated using features Verification of product lines: – Determine in which products a property fails – Resolution – reduction of variability or ad-hoc fixing of products Other PL operations: – “lifted” versions of classical – … or special-purpose 78 Products Product Line

79. Model-driven engineering under presence of uncertainty Model-driven engineering under presence of variability Uncertainty and variability combined? Some current/future work 79

80. Two common kinds of choices in SE UncertaintyVariability ReasonIncomplete information, design alternatives, stakeholder conflicts, etc. Market demands for product variants GranularityDecisionsFeatures ExpressionPartial modelsProduct line (PL) models SemanticsSet of artifacts produced by combinations of decisions Set of artifacts produced by combinations of features 80

81. Goal Enable modeling and analysis for design-space exploration of different product lines i.e., given a space of product lines, which one should be selected (and why)? 81 Decision combinations produce possible PLs Feature combinations produce possible products

82. Uncertainty + Variability Sources Modeling Verification of models containing uncertainty – What are relevant properties? – How to check these properties? – Dealing with property violations Current Status 82

83. Uncertainty in a Washing Machine Product Line 83 Domain Model 83 Locking Waiting Washing entry/TempCheck() Drying Unlocking [heatEnabled;delayEnabled] / HeaterOn() / HeaterOff(); wash.Start(); / QuickCool() [not heatEnabled;not delayEnabled]/ wash.Start(); Feature Model Wash Heat Dry Delay excludes Heat Delay Heat Dry Heat Delay Decision point Mutex Not sure whether Heat and Delay are mutually exclusive. Decision point Mutex Not sure whether Heat and Delay are mutually exclusive. Decision point HasSpin Not sure whether to put a guard NoSpin on transition. Decision point HasSpin Not sure whether to put a guard NoSpin on transition.

84. A Design Space of Product Lines Goal: Use properties to explore the design space of PL’s 84

85. Constraining the Design Space using Properties For a product property P – Use All for critical properties and Some for desirable properties – Use Necessary when you are sure it is needed and Possible when unsure but don’t want to exclude the possibility Necessary for product line Possible for product line All products have P All products in All product lines All products in Some product line Some products have P Some product in All product lines Some product in Some product line 85

86. Necessary-Some All product lines Some product 86

87. Possible-Some Some product line Some product 87

88. Possible-All Some product line All products 88

89. Necessary-All Example Prop: State Unlocking must always be reached - a washing machine that violates this is unacceptable All product lines All product Example Analysis Check: does prop hold? 89

90. Uncertainty in a Washing Machine Product Line 90 Domain Model 90 Locking Waiting Washing entry/TempCheck() Drying Unlocking [heatEnabled;delayEnabled] / HeaterOn() / HeaterOff(); wash.Start(); / QuickCool() [not heatEnabled;not delayEnabled]/ wash.Start(); Feature Model Wash Heat Dry Delay excludes Heat Delay Heat Dry Heat Delay Decision point Mutex Not sure Heat and Delay are mutually exclusive. Decision point Mutex Not sure Heat and Delay are mutually exclusive. Decision point HasSpin Not sure whether to put guard NoSpin on transition. Decision point HasSpin Not sure whether to put guard NoSpin on transition.

91. Domain Model Uncertainty in a Washing Machine Product Line Locking Waiting Washing entry/TempCheck() Drying Unlocking [heatEnabled;delayEnabled] / HeaterOn() / HeaterOff(); wash.Start(); / QuickCool() [not heatEnabled;not delayEnabled]/ wash.Start(); Feature Model Wash Heat Dry Delay excludes 91 Heat Delay Heat Dry Heat Delay U1: Not sure Heat and Delay are mutually exclusive. U2: Not sure whether to put a guard on transition from Washing to Drying.

92. Responses to Property Violation Necessary-All?  92

93. Response 1: Relax Constraint Possible-All? Necessary-All? 93

94. Response 2: Reduce Uncertainty Reduce Uncertainty Necessary-All?  94

95. Response: Reduce Uncertainty Necessary-All? 95

96. Response: Reduce Variability Reduce Variability Necessary-All?  96

97. Response 3: Reduce Variability Necessary-All? 97

98. Implementation and Status Implementation – Proof of concept implementation in Clafer for property checking and feedback generation – Exploring Higher-Order Alloy* Current Work – Evaluating scalability of the tools and effectiveness of the methodology – Better automation of response strategies – Case studies Power Window “Real” Washing Machine 98

99. Summary: Uncertainty + Variability Uncertainty and Variability are similar but different They can be used effectively together to explore SPL design space We defined: – Four natural classes of properties that can be checked – Responses to property violations 99 ConcretizationsPartial model Products Product Line Uncertainty Variability

100. Model-driven engineering under presence of uncertainty Model-driven engineering under presence of variability Uncertainty and variability combined? Some current/future work 100

101. Understanding Sources of Uncertainty: Mining Developer Conversations 101 Mine conversations Gather Analytics Generate Action Recommendations [RSSE14]

102. Understanding Sources of Uncertainty: Mining Developer Conversations Identify unique Points of Uncertainty from question utterances Associate them with Proposed Alternatives Track the stakeholders’ arguments Recognize the Resolution of Uncertainty 102 [RSSE14]

103. Variability-Aware Transformations Problem: Graph-based transformation Lots of similar yet somewhat different rules How to represent these rules so that the transformation can be done efficiently (e.g., save time on matching to determine applicability) How to produce variability-aware rule sets from lots of rules [FASE15, additional work in progress] 75

104. Lifting Code Analyses A terrific body of work by Christen Kaestner on reinterpreting various code analyses – one at a time- on 150% code models (with #ifdefs) Problem: Given an analysis method on programs, reinterpret it on 150% representations, together with proofs of correctness (that the method gives correct analysis on each variant) Specifically, trying to lift analysis behind UFO [CAV12] a combination between over- and under-approximation [in progress] 76

105. Representation of Sets Uncertainty Product Lines Metamodels Megamodels ConcretizationsPartial model Products Product Line Instances Metamodel Members Megamodel 105

106. Support for Megamodeling 106 [MODELS15] Model management for collection of models Additional operations: – Map – Reduce – Filter

107. To Summarize 107

108. References, part 1 108 [MiSE13] Michalis Famelis, Stephanie Santosa: MAV-Vis: Notation for Model Uncertaint. MiSE13 @ ICSE13. [ICSE12] Michalis Famelis, Rick Salay, Marsha Chechik: Partial models: Towards modeling and reasoning with uncertainty. ICSE 2012: 573-583 [RE12] Rick Salay, Marsha Chechik, Jennifer Horkoff: Managing requirements uncertainty with partial models. RE 2012: 1-10 [RE14] Jennifer Horkoff, Rick Salay, Marsha Chechik, Alessio Di Sandro: Supporting early decision-making in the presence of uncertainty. RE 2014: 33-42 [ASE08] Nicholas D’Ippolito, Dario Fishbein, Marsha Chechik, Sebastian Uchitel: MTSA: The Modal Transition System Analyzer. ASE 2008: 475-476 [FASE12] Rick Salay, Michalis Famelis, Marsha Chechik: Language Independent Refinement Using Partial Modeling. FASE 2012: 224-239 [JOT15] Rick Salay, Marsha Chechik, Michalis Famelis: A Methodology for Verifying Refinements of Partial Models. Journal of Object Technology 2015. [MODELS13] Michalis Famelis, Rick Salay, Alessio Di Sandro, Marsha Chechik: Transformation of Models Containing Uncertainty. MoDELS 2013: 673-689 [ASE12] Julia Rubin, Marsha Chechik: Locating distinguishing features using diff sets. ASE 2012: 242-245 [FASE14] Daniel Strüber, Julia Rubin, Gabriele Taentzer, Marsha Chechik: Splitting Models Using Information Retrieval and Model Crawling Techniques. FASE 2014: 47-62 [ASE15] Yi Li, Julia Rubin, Marsha Chechik: Semantic Slicing of Software Version Histories: ASE 2015. to appear. [MODELS15] Rick Salay, Sahar Kokaly, Alessio Di Sandro, Marsha Chechik: Enriching Megamodel Management with Collection-Based Operators. MODELS 2015 to appear 108

109. References, Part 2 109 [CAV12] Aws Albarghouthi, Yi Li, Arie Gurfinkel, Marsha Chechik: Ufo: A Framework for Abstraction- and Interpolation-Based Software Verification. CAV 2012: 672-678 [FASE15] Daniel Strüber, Julia Rubin, Marsha Chechik, Gabriele Taentzer: A Variability-Based Approach to Reusable and Efficient Model Transformations. FASE 2015: 283-298 [ICSE15] Michalis Famelis, Naama Ben-David, Alessio Di Sandro, Rick Salay, Marsha Chechik: MU- MMINT: An IDE for Model Uncertainty. ICSE (2) 2015: 697-700 [MODELS15Tool] A. Di Sandro, M. Famelis, R. Salay, S. Kokaly, M. Chechik. MMINT: A Graphical Tool for Interactive Model Management: MODELS 2015 Demos, to appear [SPLC13] Julia Rubin, Krzysztof Czarnecki, Marsha Chechik: Managing cloned variants: a framework and experience. SPLC 2013: 101-110 [STTT15] Julia Rubin, Krzysztof Czarnecki. Cloned Product Variants: From Ad-Hoc to Managed Software Product Lines. STTT 2015. [ICSE14] Rick Salay, Michalis Famelis, Julia Rubin, Alessio Di Sandro, Marsha Chechik: Lifting model transformations to product lines. ICSE 2014: 117-128 [RSSE14] Ahmed Shah Mashiyat, Michalis Famelis, Rick Salay, Marsha Chechik: Using developer conversations to resolve uncertainty in software development: a position paper. RSSE@ICSE 2014: 1-5 [Classen-ICSE10] Andreas Classen, Patrick Heymans, Pierre-Yves Schobbens, Axel Legay, Jean-François Raskin: Model checking lots of systems: efficient verification of temporal properties in software product lines. ICSE (1) 2010: 335-344 [Classen-ICSE11] Andreas Classen, Patrick Heymans, Pierre-Yves Schobbens, Axel Legay: Symbolic model checking of software product lines. ICSE 2011: 321-330 [Classen-STTT12] Andreas Classen, Maxime Cordy, Patrick Heymans, Axel Legay, Pierre-Yves Schobbens: Model checking software product lines with SNIP. STTT 14(5): 589-612 (2012) [Ben-David-ICSE15] Shoham Ben-David, Baruch Sterin, Joanne M. Atlee, Sandy Beidu: Symbolic Model Checking of Product-Line Requirements Using SAT-Based Methods. ICSE (1) 2015: 189-199 109

110. 110

111. Uncertainty-Reducing Refinement Systematically incorporate new information Two options: Manual – Edit the uncertainty annotations [FASE’12] Using automated transformations – How do you verify that a transformation is always uncertainty reducing? [VOLT’12] 111 Unsure if it should be an inner class. X v Y Unsure if we need this field. You know what? I want to get rid of effect.

112. Analysis Example 74

113. Tool Support for Responding to Analysis Offending May elements removed

114. Tool Support for Responding to Analysis Greyed out: not part of the diagnostic core May elements part of the core

115. Reminder: Model Transformations With Graph Rewriting (“Classical”) 115 class1 + attribute : type class1 - attribute : type + getAttribute() :type class1 class2 Negative Application Condition Left Hand Side Right Hand Side EncapsulateVariable refactoring: Make fields private and add getter methods unless they belong to some inner class Example rule:

116. Graph Rewriting: Example Input Model 116 Solver SolverException + effect : String class1 + attribute : type class1 - attribute : type + getAttribute() :type class1 class2 RHS LHS NAC Match

117. Graph Rewriting: Example Input Model 117 Solver SolverException + effect : String class1 + attribute : type class1 - attribute : type + getAttribute() :type class1 class2 RHS LHS NAC NAC also matches! ABORT !

118. Graph Rewriting: Example Input Model 2 118 SolverException + effect : String class1 + attribute : type class1 - attribute : type + getAttribute() :type class1 class2 RHS LHS NAC Match

119. Graph Rewriting: Example Input Model 2 119 SolverException + effect : String class1 + attribute : type class1 - attribute : type + getAttribute() :type class1 class2 RHS LHS NAC Delete

120. Graph Rewriting: Example Input Model 2 120 SolverException - effect : String class1 + attribute : type class1 - attribute : type + getAttribute() :type class1 class2 RHS LHS NAC Add

121. Graph Rewriting: Example Input Model 2 121 SolverException - effect : String + getEffect() : String class1 + attribute : type class1 - attribute : type + getAttribute() :type class1 class2 RHS LHS NAC Add

122. Graph Rewriting: Example Input Model 2 122 SolverException - effect : String + getEffect() : String class1 + attribute : type class1 - attribute : type + getAttribute() :type class1 class2 RHS LHS NAC No more LHS matches. Stop.

123. And Now: Transforming Models With Uncertainty 123 Natalie wants to apply the rule to an input with uncertainty

124. Why Transforming Models with Uncertainty is Hard? 124 Solver SolverException + effect : String class1 + attribute : type class1 - attribute : type + getAttribute() :type class1 class2 RHS LHS NAC Match??? X Y X v Y

125. Why Transforming Models with Uncertainty is Hard? 125 Solver SolverException + effect : String class1 + attribute : type class1 - attribute : type + getAttribute() :type class1 class2 RHS LHS NAC Match??? X Y X v Y

126. Why Transforming Models with Uncertainty is Hard? 126 Solver SolverException + effect : String class1 + attribute : type class1 - attribute : type + getAttribute() :type class1 class2 RHS LHS NAC Should we delete? Should we add? X Y X v Y

127. Why Transforming Models with Uncertainty is Hard? 127 Solver SolverException + effect : String class1 + attribute : type class1 - attribute : type + getAttribute() :type class1 class2 RHS LHS NAC Existing transformation techniques cannot be used. X Y X v Y

128. Goal: “Lifting” 128 class1 + attribute : type class1 - attribute : type + getAttribute():type class1 class2 RHS LHS NAC Solver SolverException + effect : String X Y X v Y ?

129. Goal: “Lifting” 129 class1 + attribute : type class1 - attribute : type + getAttribute():type class1 class2 RHS LHS NAC Solver SolverException + effect : String Solver SolverException + effect : String Solver SolverException ?

130. Lifting Technique 130 class1 + attribute : type class1 - attribute : type + getAttribute():type class1 class2 RHS LHS NAC Solver SolverException + effect : String X Y X v Y (a) Find Match Step 1: Determine applicability

131. Lifting Technique 131 class1 + attribute : type class1 - attribute : type + getAttribute():type class1 class2 RHS LHS NAC Solver SolverException + effect : String X Y X v Y (a) Find Match (b) Make sure the rule applies to at least one concretization (requires solving a SAT problem) Step 1: Determine applicability

132. Lifting Technique 132 class1 + attribute : type class1 - attribute : type + getAttribute():type class1 class2 RHS LHS NAC Solver SolverException + effect : String X Y X v Y Solver X Step 1: Determine applicability Step 2: Transform graph SolverException + effect : String Y (a)Copy over unchanged parts

133. Lifting Technique 133 class1 + attribute : type class1 - attribute : type + getAttribute():type class1 class2 RHS LHS NAC Solver SolverException + effect : String X Y X v Y Solver SolverException + - effect : String +getEffect() : String X Y a b Step 1: Determine applicability Step 2: Transform graph (a)Copy over unchanged parts (b)Perform additions and deletions Added and deleted elements become Maybe

134. Lifting Technique 134 class1 + attribute : type class1 - attribute : type + getAttribute():type class1 class2 RHS LHS NAC Solver SolverException + effect : String X Y X v Y Step 1: Determine applicability Step 2: Transform graph Step 3: Transform formula ( X ∧ ¬Y ∧ ¬a ∧ ¬b) v (¬X ∧ Y ∧ ¬a ∧ b) v ( X ∧ Y ∧ a ∧ ¬b) ( X ∧ ¬Y ∧ ¬a ∧ ¬b) v (¬X ∧ Y ∧ ¬a ∧ b) v ( X ∧ Y ∧ a ∧ ¬b) Solver SolverException + - effect : String +getEffect() : String X Y a b Constrain Maybe elements to ensure each that concretization is correctly affected.

135. Transforming Models with Uncertainty 135 We need to lift Natalie’s transformations so that they can apply to models with uncertainty Existing transformation techniques do not support this! Natalie should be able to use model transformations [MODELS13]