1. Author Software Engineering Institute
4/26/2017
Model Testing: Testing Executable Requirements, Architecture, and Design Models
Software Engineering Institute
Carnegie Mellon University
Pittsburgh, PA
Donald G. Firesmith Principal Engineer Engineer

2. Author Software Engineering Institute
Copyright 2015 Carnegie Mellon University This material is based upon work funded and supported by the Department of Defense under Contract No. FA C-0003 with Carnegie Mellon University for the operation of the Software Engineering Institute, a federally funded research and development center. Any opinions, findings and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the United States Department of Defense. NO WARRANTY. THIS CARNEGIE MELLON UNIVERSITY AND SOFTWARE ENGINEERING INSTITUTE MATERIAL IS FURNISHED ON AN “AS-IS” BASIS. CARNEGIE MELLON UNIVERSITY MAKES NO WARRANTIES OF ANY KIND, EITHER EXPRESSED OR IMPLIED, AS TO ANY MATTER INCLUDING, BUT NOT LIMITED TO, WARRANTY OF FITNESS FOR PURPOSE OR MERCHANTABILITY, EXCLUSIVITY, OR RESULTS OBTAINED FROM USE OF THE MATERIAL. CARNEGIE MELLON UNIVERSITY DOES NOT MAKE ANY WARRANTY OF ANY KIND WITH RESPECT TO FREEDOM FROM PATENT, TRADEMARK, OR COPYRIGHT INFRINGEMENT. This material has been approved for public release and unlimited distribution except as restricted below. This material may be reproduced in its entirety, without modification, and freely distributed in written or electronic form without requesting formal permission. Permission is required for any other use. Requests for permission should be directed to the Software Engineering Institute at DM
Author
Software Engineering Institute
4/26/2017

3. Verification and Validation Methods
Traditionally, V&V ≈ Analysis + Demonstration + “Inspection” + Test. There are actually more methods for verification and validation. Testing is similar to other dynamic V&V methods.

4. Author
Program
4/26/2017
Model Testing
Model testing tests executable requirements, architecture, and/or design models the same way traditional testing tests software (and systems).
A requirements, architecture, or design model of a system is executable if:
It describes the system in sufficient detail (e.g., components, functions, relations, processes, and status changes) that it can be executed as a simulation of that system.
A means exists (e.g., modeling tool, simulation engine, execution engine, or human brain) to run the model.
Model testing is NOT the same thing as Model-Based Testing (MBT)!
Model testing uses testing to uncover defects in the models.
MBT uses models to automate testing to uncover defects in the software / systems.

5. Model-Based Shift Left Testing
Author
Program
4/26/2017
Model-Based Shift Left Testing

6. Shift Left Testing Model-Based Shift Left Testing ←
Author
Program
4/26/2017
Shift Left Testing
Major trend in software testing.
Move testing earlier in development cycle (left on timeline)
Four types of shift left testing:
Traditional Shift Left Testing
Move emphasis from system-level GUI testing via record/playback to API, component integration, and unit testing
Incremental Shift Left Testing
Few deliveries/releases with testing of multiple large increments
Agile/DevOps Shift Left Testing
Many deliveries/releases with testing of multiple small increments
Model-Based Shift Left Testing ←
Test executable requirements, architecture, and/or design models (move testing left to models)

7. Model Testing

8. Author
Program
4/26/2017
Model Testing

9. Model Testing

10. Testable Models – Requirements Models
Author
Program
4/26/2017
Testable Models – Requirements Models
Example executable models include:
ConOps storyboards (simulations)
Use case path sequence diagrams
Decision trees, finites state machines, and Petri nets
Executable requirements languages
Requirements prototypes

11. Testable Models – Architecture/Design Models
Author
Program
4/26/2017
Testable Models – Architecture/Design Models
Examples executable models include:
Architecture Analysis and Design Language (AADL) models
Unified Modeling Language (UML) models and System Modeling Language (SysML) models:
Communication diagrams, interaction diagrams, sequence diagrams, state machine diagrams, activity diagrams, and timing diagrams,
Object Constraint Language (OCL) or Action Semantic Language (ASL)
Unified Profile for DODAF and MODAF (UPDM 2.0)
Hardware Description Languages
Architecture prototypes
Design models for automatic code generation