Download presentation
Presentation is loading. Please wait.
1
Introduction to Search Based Software Engineering SSS SEBASE Summer School, Birmingham University, July 2007 Mark Harman King’s College London
2
Introduction to Search Based Software Engineering SSSS Second SEBASE Summer School, Birmingham University, July 2008 Mark Harman King’s College London
3
Introduction to Search Based Software Engineering How on earth can software engineers do what John says they claim do Mark Harman King’s College London
4
How on earth can software engineers do what John says they claim do - To engineer is to optimize Mark Harman King’s College London
5
Introduction to Search Based Software Engineering Thanks John Clark, University of York, Sebastian Elbaum, University of Nebraska Lincoln Rob Hierons, Brunel University Zheng Li, King’s College London Kiarash Mahdavi, King’s College London Spiros Mancoridis, Drexel University Afshin Mansouri, King’s College London Jian Ren, King’s College London Joachim Wegener, DaimlerChrysler Shin Yoo, King’s College London YuanYuan Zhang, King’s College London
6
Mark Harman Introduction to SBSE 6 Where is King’s College London? 20 minutes’ walk
7
Mark Harman Introduction to SBSE 7 Search Based Software Engineering SBSE what and why SBSE case studies SBSE how Current and future trends Advantages Multi Objective Search Insight through search Co Evolution
8
Mark Harman Introduction to SBSE 8 Search Based Software Engineering SBSE what and why SBSE case studies SBSE how Current and future trends – personal views Advantages Multi Objective Search Insight through search Co Evolution
9
Mark Harman Introduction to SBSE 9 Search Based Software Engineering SBSE what and why SBSE case studies SBSE how Current and future trends – IMHO Advantages Multi Objective Search Insight through search Co Evolution
10
Mark Harman Introduction to SBSE 10 Search Based Software Engineering SBSE what and why SBSE case studies SBSE how Current and future trends – In Mark Harman’s Opinion Advantages Multi Objective Search Insight through search Co Evolution
11
Mark Harman Introduction to SBSE 11 What is SBSE In SBSE we apply search techniques to search large search spaces, guided by a fitness function that captures properties of the acceptable software artefacts we seek.
12
Mark Harman Introduction to SBSE 12 What is SBSE In SBSE we apply search techniques to search large search spaces, guided by a fitness function that captures properties of the acceptable software artefacts we seek. Genetic Algorithms, Hill climbing, Simulated Annealing, Random, Tabu Search, Estimation of Distribution Algorithms, Particle Swarm Optimization, Ant Colonies, LP, Genetic programming, Greedy
13
Mark Harman Introduction to SBSE 13 What is SBSE In SBSE we apply search techniques to search large search spaces, guided by a fitness function that captures properties of the acceptable software artefacts we seek. Genetic Algorithms, Hill climbing, Simulated Annealing, Random, Tabu Search, Estimation of Distribution Algorithms, Particle Swarm Optimization, Ant Colonies, LP, Genetic programming, Greedy
14
Mark Harman Introduction to SBSE 14 What is SBSE In SBSE we apply search techniques to search large search spaces, guided by a fitness function that captures properties of the acceptable software artefacts we seek.
15
Mark Harman Introduction to SBSE 15 What is SBSE In SBSE we apply search techniques to search large search spaces, guided by a fitness function that captures properties of the acceptable software artefacts we seek. So it is merely another application of Optimization Techniques?
16
Mark Harman Introduction to SBSE 16 What is SBSE In SBSE we apply search techniques to search large search spaces, guided by a fitness function that captures properties of the acceptable software artefacts we seek. So it is merely another set of applications of Optimization Techniques?
17
Mark Harman Introduction to SBSE 17 What is SBSE In SBSE we apply search techniques to search large search spaces, guided by a fitness function that captures properties of the acceptable software artefacts we seek. So it is merely another rich set of applications of Optimization Techniques?
18
Mark Harman Introduction to SBSE 18 What is SBSE In SBSE we apply search techniques to search large search spaces, guided by a fitness function that captures properties of the acceptable software artefacts we seek. So it is merely another rich set of applications of Optimization Techniques?
19
Mark Harman Introduction to SBSE 19 What is SBSE In SBSE we apply search techniques to search large search spaces, guided by a fitness function that captures properties of the acceptable software artefacts we seek. Have nails … seek hammer
20
Mark Harman Introduction to SBSE 20 What is SBSE In SBSE we apply search techniques to search large search spaces, guided by a fitness function that captures properties of the acceptable software artefacts we seek. Have nails … seek hammers
21
Mark Harman Introduction to SBSE 21 What is SBSE In SBSE we apply search techniques to search large search spaces, guided by a fitness function that captures properties of the acceptable software artefacts we seek. Have nails … seek hammers Generalise, compare, develop
22
Mark Harman Introduction to SBSE 22 What is SBSE In SBSE we apply search techniques to search large search spaces, guided by a fitness function that captures properties of the acceptable software artefacts we seek. Have nails … seek hammers Generalise, compare, develop in an unbiased manner
23
Mark Harman Introduction to SBSE 23 What is Research? In research we apply social search techniques to search large search spaces, guided by a natural evolution of memes Have nails … seek hammers Generalise, compare, develop in an unbiased manner
24
Mark Harman Introduction to SBSE 24 What is Research? In research we apply social search techniques to search large search spaces, guided by a natural evolution of memes and all the usual human frailties Have nails … seek hammers Generalise, compare, develop in an unbiased manner
25
Mark Harman Introduction to SBSE 25 Why?
26
Mark Harman Introduction to SBSE 26 The Eight Queens Problem
27
Mark Harman Introduction to SBSE 27 The Eight Queens Problem
28
Mark Harman Introduction to SBSE 28 The Eight Queens Problem
29
Mark Harman Introduction to SBSE 29 The Eight Queens Problem Perfect
30
Mark Harman Introduction to SBSE 30 The Eight Queens Problem PerfectScore 0
31
Mark Harman Introduction to SBSE 31 The Eight Queens Problem
32
Mark Harman Introduction to SBSE 32 The Eight Queens Problem
33
Mark Harman Introduction to SBSE 33 The Eight Queens Problem Two Attacks
34
Mark Harman Introduction to SBSE 34 The Eight Queens Problem Two Attacks Score -2
35
Mark Harman Introduction to SBSE 35 The Eight Queens Problem
36
Mark Harman Introduction to SBSE 36 The Eight Queens Problem
37
Mark Harman Introduction to SBSE 37 The Eight Queens Problem Three Attacks
38
Mark Harman Introduction to SBSE 38 The Eight Queens Problem Three Attacks Score -3
39
Mark Harman Introduction to SBSE 39 That was easy
40
Mark Harman Introduction to SBSE 40 Generate a solution Place 8 queens on the board so that there are no attacks
41
Mark Harman Introduction to SBSE 41 Scale up: Generate a solution Place 44 queens on the board so that there are no attacks
42
Mark Harman Introduction to SBSE 42 Scale up: Generate a solution Place 400 queens on the board so that there are no attacks
43
Mark Harman Introduction to SBSE 43 Checking vs Generating Task One: Write a method to determine which is the better of two placements of N queens Task Two: Write a method to construct a board placement with N non attacking queens
44
Mark Harman Introduction to SBSE 44 Checking vs Generating Task One: Write a method to determine which is the better of two placements of N queens Task Two: Write a method to construct a board placement with N non attacking queens
45
Mark Harman Introduction to SBSE 45 Checking vs Generating Search Based Software Engineering Write a method to determine which is the better of two solutions Conventional Software Engineering Write a method to construct a perfect solution
46
Mark Harman Introduction to SBSE 46 Checking vs Generating Search Based Software Engineering Write a method to determine which is the better of two solutions Conventional Software Engineering Write a method to construct a perfect solution
47
Mark Harman Introduction to SBSE 47 Checking vs Generating Search Based Software Engineering Write a method to determine which is the better of two solutions Conventional Software Engineering Write a method to construct a perfect solution
48
Mark Harman Introduction to SBSE 48 Checking vs Generating Search Based Software Engineering Write a fitness function to determine which is the better of two solutions Conventional Software Engineering Write a method to construct a perfect solution
49
Mark Harman Introduction to SBSE 49 Checking vs Generating Search Based Software Engineering Write a fitness function to guide a search aaaa Conventional Software Engineering Write a method to construct a perfect solution
50
Mark Harman Introduction to SBSE 50 Checking vs Generating Search Based Software Engineering Write a fitness function to guide automated search Conventional Software Engineering Write a method to construct a perfect solution
51
Mark Harman Introduction to SBSE 51 Examples
52
Mark Harman Introduction to SBSE 52 Case Studies
53
Mark Harman Introduction to SBSE 53 Evolutionary Algorithms Selection Insertion Recombination Mutation Fitness evaluation Test execution End?
54
Mark Harman Introduction to SBSE 54 Evolutionary Testing Selection Insertion Recombination Mutation Fitness evaluation Test cases Test execution End?
55
Mark Harman Introduction to SBSE 55 Evolutionary Testing Selection Insertion Recombination Mutation Fitness evaluation Test cases Test execution End? Execution
56
Mark Harman Introduction to SBSE 56 Evolutionary Testing Selection Insertion Recombination Mutation Fitness evaluation Test cases Test execution End? Execution Monitoring
57
Mark Harman Introduction to SBSE 57 Temporal Testing deceleration t min t max t optimal point of time for triggering the airbag igniter Selection Insertion Recombination Mutation Fitness evaluation Individuals Test data Monitoring Fitness values Test execution End?
58
Mark Harman Introduction to SBSE 58 Temporal Testing deceleration t min t max t optimal point of time for triggering the airbag igniter Selection Insertion Recombination Mutation Duration Individuals Test data Monitoring Fitness values Test execution End?
59
Mark Harman Introduction to SBSE 59 Evolution vs Random for Temporal Testing
60
Mark Harman Introduction to SBSE 60 Search Based Testing Applications Temporal testing Coverage based testing Functional testing Regression testing Finite State Machine testing Interaction testing Exception testing Stress testing Robustness testing
61
Mark Harman Introduction to SBSE 61 Search Based Testing Applications Temporal testing Coverage based testing Functional testing Regression testing Finite State Machine testing Interaction testing Exception testing Stress testing Robustness testing Phil will talk about some of these in more detail shortly
62
Mark Harman Introduction to SBSE 62 Search Based Testing Applications Temporal testing Coverage based testing Functional testing Regression testing Finite State Machine testing Interaction testing Exception testing Stress testing Robustness testing
63
Mark Harman Introduction to SBSE 63
64
Mark Harman Introduction to SBSE 64
65
Mark Harman Introduction to SBSE 65
66
Mark Harman Introduction to SBSE 66
67
Mark Harman Introduction to SBSE 67
68
Mark Harman Introduction to SBSE 68
69
Mark Harman Introduction to SBSE 69 Finding UIOs Hierons et al. 2004-2008 s1s1 s2s2 s3s3 a/0 a/1 b/1 b/0 a/1s3s3 b/0s2s2 b/1,a/1s1s1 UIOState
70
Mark Harman Introduction to SBSE 70 Target Level 4 Level 3 Level 2 Level 1 Fitness = Approximation_Level + Local_Distance Evaluation of predicate in a branching condition if A = B Local_Distance = | A - B | Identify relevant branching statements using control dependence Target 1. Approximation level Structural Evolutionary Testing
71
Mark Harman Introduction to SBSE 71 Evolutionary Testing vs Random EffectivenessCost
72
Mark Harman Introduction to SBSE 72 Autonomous Parking System
73
Mark Harman Introduction to SBSE 73 Autonomous Parking System
74
Mark Harman Introduction to SBSE 74 Autonomous Parking System Stop
75
Mark Harman Introduction to SBSE 75 Autonomous Parking System - Input psi gap dist2space space length space width
76
Mark Harman Introduction to SBSE 76 Autonomous Parking System
77
Mark Harman Introduction to SBSE 77 Regression Testing Test case prioritzation Test suite reduction Test case selection
78
Mark Harman Introduction to SBSE 78 Regression Testing Test case prioritzation Li, Harman, Hierons, TSE 2007 Test suite reduction Test case selection
79
Mark Harman Introduction to SBSE 79 Test Case Prioritization Order test cases higher priority => earlier execution
80
Mark Harman Introduction to SBSE 80 Greedy Optimal orderings: ABC BCA CBA
81
Mark Harman Introduction to SBSE 81 Regression Testing Test case prioritzation Test suite reduction Test case selection
82
Mark Harman Introduction to SBSE 82 Regression Testing Test case prioritzation Test suite reduction Yoo and Harman, ISSTA 07 Test case selection
83
Mark Harman Introduction to SBSE 83 Regression Test Selection
84
Mark Harman Introduction to SBSE 84 Regression Testing Test case prioritzation Test suite reduction Test case selection you and someone else, TSE 2008/9 ?
85
Mark Harman Introduction to SBSE 85 Regression Testing Test case prioritzation Test suite reduction Test case selection yoo and me, TSE 2008/9 ?
86
Mark Harman Introduction to SBSE 86 Does this look familiar?
87
Mark Harman Introduction to SBSE 87 Does this look familiar?
88
Mark Harman Introduction to SBSE 88 MDG M1 M2M3 M4 M5M6 Good Partition! M1 M2 M3 M4 M5 M6 Bad Partition! M1 M2 M4 M3 M5 M6 MQ(Good Partition) > MQ(Bad Partition) The Bunch Approach Mancoridis et al. 1998 - 2005
89
Mark Harman Introduction to SBSE 89 Hill Climbing
90
Mark Harman Introduction to SBSE 90 Hill Climbing
91
Mark Harman Introduction to SBSE 91 Hill Climbing Maybe this is no longer the optimal approach …
92
Mark Harman Introduction to SBSE 92 Hill Climbing Kata Praditwong et al.
93
Mark Harman Introduction to SBSE 93 SBSE is so generic Testing Fitness function: execution time Representation: input vector
94
Mark Harman Introduction to SBSE 94 SBSE is so generic Testing Fitness function: execution time Representation: input vector Restructuring Fitness function: cohesion and coupling Representation: mapping from modules to clusters
95
Mark Harman Introduction to SBSE 95 SBSE is so generic Testing Fitness function: execution time Representation: input vector Requirements Fitness function: customer satisfaction, cost Representation: bitset of requirements
96
Mark Harman Introduction to SBSE 96 SBSE is so generic Testing Fitness function: execution time Representation: input vector Program Comprehension Fitness function: (in)direct aspect of cognition Representation: code, architecture, visualization, model
97
Mark Harman Introduction to SBSE 97 SBSE is so generic Testing Fitness function: execution time Representation: input vector Project Management Fitness function: project duration Representation: mapping from teams to WPs
98
Mark Harman Introduction to SBSE 98 SBSE is so generic Time for two more examples Program Comprehension Mahdavi et al. ICSM 05 Requirements Analysis Zhang et al. GECCO 07
99
Mark Harman Introduction to SBSE 99 SBSE is so generic Time for two more examples Program Comprehension Mahdavi et al. ICSM 05 Requirements Analysis
100
Mark Harman Introduction to SBSE 100 Concept Assignment “The process of assigning descriptive terms usually relating to computational intent.” MOVE ‘EXAMPLE’ TO PRINT-LL MOVE ‘13’ TO PRINT-CC. CALL ‘PRINT’ USING P-PRINTLINE. MOVE POLICY-NUM TO OUT-PNUM. MOVE SCHEME-REF TO OUT-SREF. CALL ‘WRITE’ USING OUT-REC. Write Print
101
Mark Harman Introduction to SBSE 101 Allowing Overlapping Concept Binding Fitness: Signal to noise ratio Representation: Space of possible overlapping bindings MOVE ‘EXAMPLE’ TO PRINT-LL MOVE POLICY-NUM TO OUT-PNUM. MOVE ‘13’ TO PRINT-CC. MOVE SCHEME-REF TO OUT-SREF. CALL ‘PRINT’ USING P-PRINTLINE. CALL ‘WRITE’ USING OUT-REC. Print Write Call
102
Mark Harman Introduction to SBSE 102 SBSE is so generic Time for two more examples Program Comprehension Requirements Analysis Zhang et al. GECCO 07
103
Mark Harman Introduction to SBSE 103 The Next Release Problem
104
Mark Harman Introduction to SBSE 104 Motorola Cell Phone Requirements
105
Mark Harman Introduction to SBSE 105 SBSE Applications TransformationCooper, Ryan, Schielke, Subramanian, Fatiregun, Williams Requirements Bagnall, Mansouri, Zhang Effort prediction Aguilar-Ruiz, Burgess, Dolado, Lefley, Shepperd ManagementAlba, Antoniol, Chicano, Di Pentam Greer, Ruhe Heap allocationCohen, Kooi, Srisa-an Regression testLi, Yoo, Elbaum, Rothermel, Walcott, Soffa, Kampfhamer SOA Canfora, Di Penta, Esposito, Villani RefactoringAntoniol, Briand, Cinneide, O’Keeffe, Merlo, Seng, Tratt Test GenerationAlba, Binkley, Bottaci, Briand, Chicano, Clark, Cohen, Gutjahr, Harrold, Holcombe, Jones, Korel, Pargass, Reformat, Roper, McMinn, Michael, Sthamer, Tracy, Tonella,Xanthakis, Xiao, Wegener, Wilkins MaintenanceAntoniol, Lutz, Di Penta, Madhavi, Mancoridis, Mitchell, Swift Model checkingAlba, Chicano, Godefroid Probe dist’ionCohen, Elbaum UIOsDerderian, Guo, Hierons ComprehensionGold, Li, Mahdavi ProtocolsAlba, Clark, Jacob, Troya Component selBaker, Skaliotis, Steinhofel, Yoo Agent OrientedHaas, Peysakhov, Sinclair, Shami, Mancoridis
106
Mark Harman Introduction to SBSE 106 SBSE Applications TransformationCooper, Ryan, Schielke, Subramanian, Fatiregun, Williams Requirements Bagnall, Mansouri, Zhang Effort prediction Aguilar-Ruiz, Burgess, Dolado, Lefley, Shepperd ManagementAlba, Antoniol, Chicano, Di Pentam Greer, Ruhe Heap allocationCohen, Kooi, Srisa-an Regression testLi, Yoo, Elbaum, Rothermel, Walcott, Soffa, Kampfhamer SOA Canfora, Di Penta, Esposito, Villani RefactoringAntoniol, Briand, Cinneide, O’Keeffe, Merlo, Seng, Tratt Test GenerationAlba, Binkley, Bottaci, Briand, Chicano, Clark, Cohen, Gutjahr, Harrold, Holcombe, Jones, Korel, Pargass, Reformat, Roper, McMinn, Michael, Sthamer, Tracy, Tonella,Xanthakis, Xiao, Wegener, Wilkins MaintenanceAntoniol, Lutz, Di Penta, Madhavi, Mancoridis, Mitchell, Swift Model checkingAlba, Chicano, Godefroid Probe dist’ionCohen, Elbaum UIOsDerderian, Guo, Hierons ComprehensionGold, Li, Mahdavi ProtocolsAlba, Clark, Jacob, Troya Component selBaker, Skaliotis, Steinhofel, Yoo Agent OrientedHaas, Peysakhov, Sinclair, Shami, Mancoridis
107
Mark Harman Introduction to SBSE 107 SBSE Applications TransformationCooper, Ryan, Schielke, Subramanian, Fatiregun, Williams Requirements Bagnall, Mansouri, Zhang Effort prediction Aguilar-Ruiz, Burgess, Dolado, Lefley, Shepperd ManagementAlba, Antoniol, Chicano, Di Pentam Greer, Ruhe Heap allocationCohen, Kooi, Srisa-an Regression testLi, Yoo, Elbaum, Rothermel, Walcott, Soffa, Kampfhamer SOA Canfora, Di Penta, Esposito, Villani RefactoringAntoniol, Briand, Cinneide, O’Keeffe, Merlo, Seng, Tratt Test GenerationAlba, Binkley, Bottaci, Briand, Chicano, Clark, Cohen, Gutjahr, Harrold, Holcombe, Jones, Korel, Pargass, Reformat, Roper, McMinn, Michael, Sthamer, Tracy, Tonella,Xanthakis, Xiao, Wegener, Wilkins MaintenanceAntoniol, Lutz, Di Penta, Madhavi, Mancoridis, Mitchell, Swift Model checkingAlba, Chicano, Godefroid Probe dist’ionCohen, Elbaum UIOsDerderian, Guo, Hierons ComprehensionGold, Li, Mahdavi ProtocolsAlba, Clark, Jacob, Troya Component selBaker, Skaliotis, Steinhofel, Yoo Agent OrientedHaas, Peysakhov, Sinclair, Shami, Mancoridis
108
Mark Harman Introduction to SBSE 108 The two SBSE ingredients Representation Fitness function
109
Mark Harman Introduction to SBSE 109 The two SBSE ingredients Representation Fitness function Should be easy We always represent Software Engineering problems in data structures
110
Mark Harman Introduction to SBSE 110 The two SBSE ingredients Representation Fitness function Often easy We often define metrics
111
Mark Harman Introduction to SBSE 111 So far SBSE is useful SBSE is generic SBSE is easy
112
Mark Harman Introduction to SBSE 112 Future Multiple objective search Co evolution Sensitivity analysis Metric validation Landscape analysis Interactive evolution
113
Mark Harman Introduction to SBSE 113 Future Multiple objective search Co evolution Sensitivity analysis Metric validation Landscape analysis Interactive evolution
114
Mark Harman Introduction to SBSE 114 Co Evolution Two populations; Two fitness functions Collaboration, and competition
115
Mark Harman Introduction to SBSE 115 Co Evolution Two populations; Two fitness functions Collaboration, and competition applied to mutation testing Adamopoulos, Harman and Hierons GECCO 2004
116
Mark Harman Introduction to SBSE 116 MutantsTest Cases
117
Mark Harman Introduction to SBSE 117 MutantsTest Cases Fitness?
118
Mark Harman Introduction to SBSE 118 MutantsTest Cases Fitness for test cases How many mutants do you kill ?
119
Mark Harman Introduction to SBSE 119 MutantsTest Cases Fitness for mutants How many test cases do you survive?
120
Mark Harman Introduction to SBSE 120 Sensitivity analysis Inputs to search are typically estimates Estimated Software Engineering attributes Which estimates matter most?
121
Mark Harman Introduction to SBSE 121 Sensitivity analysis Inputs to search are typically estimates Estimated Software Engineering attributes Which estimates matter most? applied to component selection Yoo, Ren, Harman, Not yet published
122
Mark Harman Introduction to SBSE 122 Motorola Cell Phone Requirements
123
Mark Harman Introduction to SBSE 123 Metric Robustness Metric performs under … change, noise, fuzziness Not just good solutions; robust solutions
124
Mark Harman Introduction to SBSE 124 Metric Robustness Metric performs under … change, noise, fuzziness Not just good solutions; robust solutions Metrics are fitness functions too! Harman and Clark, Metrics Symposium 2004
125
Mark Harman Introduction to SBSE 125 Landscape Analysis What landscape is denoted by the metric? Landscape properties smoothness, plateaux, …
126
Mark Harman Introduction to SBSE 126 Landscape Analysis and Improvement Improve the fitness landscape by transformation SBSE allows transformation of program under test A Testability Transformation Harman et al. 2004 – 2008
127
Mark Harman Introduction to SBSE 127 Landscape Analysis and Improvement Improve the fitness landscape by transformation SBSE allows transformation of program under test A Testability Transformation Harman et al. 2002 – 2008 An example for Structural Testing McMinn et al. ISSTA 2007
128
Mark Harman Introduction to SBSE 128 Hitchcock Landscape void alfred(double a, double b) { if (a == b) { // target 1 double c = b + 1; if (c == 0) { // target 2...
129
Mark Harman Introduction to SBSE 129 Hitchcock Landscape
130
Mark Harman Introduction to SBSE 130 Hitchcock Landscape
131
Mark Harman Introduction to SBSE 131 Interactive evolution Harman ICPC 2007 Human in the loop Caters for intangibles Can flush out implicit assumptions Co evolution of cognitive models Lots of fun things to do here …
132
Mark Harman Introduction to SBSE 132 Summary Software Engineering problems can be optimized Software Engineering problems are often search problems This should not be a surprise The SBSE approach gives us a whole new set of Tools Concepts Techniques Insights To optimize
133
Mark Harman Introduction to SBSE 133 Summary Software Engineering problems can be optimized Software Engineering problems are often search problems This should not be a surprise The SBSE approach gives us a whole new set of Tools Concepts Techniques Insights To optimize is to engineer
134
Mark Harman Introduction to SBSE 134 Search Based Software Engineering is the focus of major initiatives SEBASE $5m 5 year project Search based software engineering Led by King’s Birmingham, York, Brunel Daimler, Motorola, IBM EvoTest $4m 3 year project Evolutionary testing Led by Valencia King’s, FIRST, INRIA, Daimler, Motorola, RILA, Xenon
135
Mark Harman Introduction to SBSE 135 Summary
136
Mark Harman Introduction to SBSE 136 Summary
137
Mark Harman Introduction to SBSE 137 Conclusions Search is what you seek
138
Mark Harman Introduction to SBSE 138 Conclusions Search is what you seek www.sebase.org
139
Mark Harman Introduction to SBSE 139 Conclusions Search is what you seek www.sebase.org Questions?
Similar presentations
© 2025 SlidePlayer.com. Inc.
All rights reserved.