© 2011 Carnegie Mellon University System of Systems V&V John B. Goodenough October 19, 2011.

Slides:



Advertisements
Similar presentations
Configuration Management
Advertisements

Carnegie Mellon University Software Engineering Institute CERT® Knowledgebase Copyright © 1997 Carnegie Mellon University VU#14202 UNIX rlogin with stack.
© 2013 Carnegie Mellon University UFO: From Underapproximations to Overapproximations and Back! Arie Gurfinkel (SEI/CMU) with Aws Albarghouthi and Marsha.
UNCLASSIFIED © 2011 Carnegie Mellon University Building Malware Infection Trees Jose Andre Morales 1, Michael Main 2, Weilang Luo 3, Shouhuai Xu 2,3, Ravi.
© 2014 Microsoft Corporation. All rights reserved.
Chapter 4 Quality Assurance in Context
© 2010 Carnegie Mellon University B OXES : A Symbolic Abstract Domain of Boxes Arie Gurfinkel and Sagar Chaki Software Engineering Institute Carnegie Mellon.
Sponsored by the U.S. Department of Defense © 2005 by Carnegie Mellon University 1 Pittsburgh, PA Dennis Smith, David Carney and Ed Morris DEAS.
© 2013 Carnegie Mellon University Academy for Software Engineering Education and Training, 2013 Session Architect: Tony Cowling Session Chair: Nancy Mead.
© 2013 Carnegie Mellon University Measuring Assurance Case Confidence using Baconian Probabilities Charles B. Weinstock John B. Goodenough Ari Z. Klein.
© 2010 Carnegie Mellon University ® CMMI is registered in the U.S. Patent and Trademark Office by Carnegie Mellon University. V&V Principles Verification.
© Carnegie Mellon University The CERT Insider Threat Center.
© 2012 Carnegie Mellon University UFO: Verification with Interpolants and Abstract Interpretation Arie Gurfinkel and Sagar Chaki Software Engineering Institute.
© 2010 Carnegie Mellon University Acquisition Implications of SOA Adoption Software Engineering Institute Carnegie Mellon University Pittsburgh, PA
Research Development for Android Coopman Tom. What is Android?  Smartphone operating system  Google  Popular  ‘Easy to develop’  Open-Source  Linux.
© 2013 Carnegie Mellon University Static Analysis of Real-Time Embedded Systems with REK Arie Gurfinkel 1 joint work with Sagar Chaki 1, Ofer Strichman.
Systems Engineering in a System of Systems Context
Software Engineering Techniques for the Development of System of Systems Seminar of “Component Base Software Engineering” course By : Marzieh Khalouzadeh.
The Role of Software Engineering Brief overview of relationship of SE to managing DSD risks 1.
© 2011 Carnegie Mellon University Should-Cost: A Use for Parametric Estimates Additional uses for estimation tools Presenters:Bob Ferguson (SEMA) Date:November.
© 2011 Carnegie Mellon University QUELCE: Quantifying Uncertainty in Early Lifecycle Cost Estimation Presenters:Dave Zubrow PhD Bob Ferguson (SEMA) Date:November.
® IBM Software Group © 2007 IBM Corporation Achieving Harmony IBM's Platform and Methodology for Systems Engineering and Embedded Software Development.
© 2015 Carnegie Mellon University Software Engineering Institute Carnegie Mellon University Pittsburgh, PA A Cognitive Study of Incident Handling.
Jul The New Geant4 License J. Perl The New Geant4 License Makes clear the user’s wide- ranging freedom to use, extend or redistribute Geant4, even.
Introduction to Software Testing
Lecture 11 CMM CSCI – 3350 Software Engineering II Fall 2014 Bill Pine.
Software Architecture for DSD The “Uses” Relation.
Ipek Ozkaya, COCOMO Forum © 2012 Carnegie Mellon University Affordability and the Value of Architecting Ipek Ozkaya Research, Technology.
The information contained in this document represents the current view of Microsoft Corporation on the issues discussed as of the date of publication.
© 2010 Carnegie Mellon University Team Software Process.
المحاضرة الثالثة. Software Requirements Topics covered Functional and non-functional requirements User requirements System requirements Interface specification.
Conditions and Terms of Use
Michael Dermody September 2010  Capability Maturity Model Integration ◦ Is a Trademark owned by the Software Engineering Institute (SEI) of Carnegie.
『华东师范大学』 课程名称: 软件开发实践 Software Development Practice 课程类型: 实践课 第二讲: 项目管理 Lect_02: Manage the Project 主讲 : 软件学院 周勇 副 教授 日期 :
Engr. M. Fahad Khan Lecturer Software Engineering Department University Of Engineering & Technology Taxila.
Testing -- Part II. Testing The role of testing is to: w Locate errors that can then be fixed to produce a more reliable product w Design tests that systematically.
CPSC 871 John D. McGregor Module 6 Session 3 System of Systems.
Software Engineering Principles Principles form the basis of methods, techniques, methodologies and tools Principles form the basis of methods, techniques,
University of Palestine software engineering department Testing of Software Systems Testing throughout the software life cycle instructor: Tasneem.
Protecting Data Rights Under DoD Contracts October 14, 2009 NCMA Workshop Cape Canaveral Chapter Keith R. Szeliga Sheppard Mullin Richter & Hampton.
Configuration Management and Change Control Change is inevitable! So it has to be planned for and managed.
Chapter 1: Fundamental of Testing Systems Testing & Evaluation (MNN1063)
Author Software Engineering Institute
1 XIII: Electronic Resources Managing Trials and Evaluations.
© 2015 Carnegie Mellon University COCOMO 2015 November 17, 2015 Distribution Statement A: Approved for Public Release; Distribution is Unlimited Causal.
© 2004 The IPR-Helpdesk is a project of the European Commission DG Enterprise, co-financed within the fifth framework programme of the European Community.
HNDIT23082 Lecture 09:Software Testing. Validations and Verification Validation and verification ( V & V ) is the name given to the checking and analysis.
Pittsburgh, PA CMMI Acquisition Module - Page M5-1 CMMI ® Sponsored by the U.S. Department of Defense © 2005 by Carnegie Mellon University This.
ISO 9001:2015 Subject: Quality Management System Clause 8 - Operation
1 CERT BFF: From Start To PoC June 09, 2016 © 2016 Carnegie Mellon University This material has been approved for public release and unlimited distribution.
Secure Software Workforce Development Panel Session
Configuration Management
Models for Resources and Management
Using Parallelspace TEAM Models to Design and Create Custom Profiles
Author Software Engineering Institute
Michael Spiegel, Esq Timothy Shimeall, Ph.D.
Chapter 13 Architectural Design
Configuration Management
Chapter 13 Architectural Design
Metrics-Focused Analysis of Network Flow Data
Parallelspace PowerPoint Template for ArchiMate® 2.1 version 1.1
Parallelspace PowerPoint Template for ArchiMate® 2.1 version 2.0
Internal control - the IA perspective
Introduction to Software Testing
Lecture 09:Software Testing
Verification and Validation Unit Testing
Chapter 13 Architectural Design
QUELCE: Quantifying Uncertainty in Early Lifecycle Cost Estimation
© Healthcare Inspirations. All rights reserved
Developing Useful Metrics
Presentation transcript:

© 2011 Carnegie Mellon University System of Systems V&V John B. Goodenough October 19, 2011

2 © 2011 Carnegie Mellon University System of Systems Constituents are operationally independent Provide functions that are useful in their own right Constituents are managerially independent Separately acquired; have their own stakeholders Constituent interactions create SoS value SoS performs functions, has properties, and carries out purposes not residing in any constituent itself (emergent behavior) Continual evolution of constituents and SoS capabilities Based on Maier, M.W., "Architecting Principles for System of Systems," Systems Engineering, Vol. 1, No. 4, 1998, pp

3 © 2011 Carnegie Mellon University V&V Implications Constituents are operationally independent (unexpected behavior) Provide functions that are useful in their own right Constituents are managerially independent Separately acquired; have their own stakeholders Constituent interactions create SoS value SoS performs functions, has properties, and carries out purposes not residing in any constituent itself (emergent behavior) SoS is continually evolving Based on Maier, M.W., "Architecting Principles for System of Systems," Systems Engineering, Vol. 1, No. 4, 1998, pp

4 © 2011 Carnegie Mellon University V&V Implications Constituents are operationally independent (unexpected behavior) Provide functions that are useful in their own right Constituents are managerially independent (change at own pace) Separately acquired; have their own stakeholders Constituent interactions create SoS value SoS performs functions, has properties, and carries out purposes not residing in any constituent itself (emergent behavior) SoS is continually evolving Based on Maier, M.W., "Architecting Principles for System of Systems," Systems Engineering, Vol. 1, No. 4, 1998, pp

5 © 2011 Carnegie Mellon University V&V Implications Constituents are operationally independent (unexpected behavior) Provide functions that are useful in their own right Constituents are managerially independent (change at own pace) Separately acquired; have their own stakeholders Constituent interactions create SoS value (behavioral interface) SoS performs functions, has properties, and carries out purposes not residing in any constituent itself (emergent behavior) SoS is continually evolving Based on Maier, M.W., "Architecting Principles for System of Systems," Systems Engineering, Vol. 1, No. 4, 1998, pp

6 © 2011 Carnegie Mellon University V&V Implications Constituents are operationally independent (unexpected behavior) Provide functions that are useful in their own right Constituents are managerially independent (change at own pace) Separately acquired; have their own stakeholders Constituent interactions create SoS value (behavioral interface ) SoS performs functions, has properties, and carries out purposes not residing in any constituent itself (emergent behavior) SoS is continually evolving (constituent and SoS evolution) Based on Maier, M.W., "Architecting Principles for System of Systems," Systems Engineering, Vol. 1, No. 4, 1998, pp

7 © 2011 Carnegie Mellon University The SoS Perspective: Not the Usual Rules Differences are unavoidable Requirements are only temporarily stable – Lack of agreement about the problem being solved – Continual change and renegotiation of needs Different groups want different configurations/services/etc. – And they change their minds! Tradeoff decisions are not stable Configuration information will not be accurate and cannot be tightly controlled Usage will drift from what was expected Need to manage differences rather than attempt to eliminate them SoSs evolve continuously rather than discretely SoSs must be designed to tolerate the introduction of concurrent changes Which changes absolutely must be coordinated? What is the impact if miscoordination occurs? How is the impact detected, and mitigated?

8 © 2011 Carnegie Mellon University Other Perspectives Systems of systems (SoSs) Constituents are decisionally autonomous, independently interacting entities – Socio-technical ecosystems Social interactions and collective user behaviors are relevant – Software-reliant SoS failures not typically due to single causes (or coding errors) SYSTEMS OF INDEPENDENT SYSTEMS (SoISs) Large system scale Failures are inevitable – Software at scale cannot be perfected – Recovery from failure must be part of the system design (and be verified) Can involve hundreds or thousands of components

9 © 2011 Carnegie Mellon University Assurance Justified confidence that a system will behave acceptably under all conditions of actual use How to know that system behavior is adequately constrained – so failures are not catastrophic

10 © 2011 Carnegie Mellon University Assurance (for SoSs) Assurance Justified confidence that a system will behave acceptably under all conditions of actual use How to know that system behavior is adequately constrained – so failures are not catastrophic SoS V&V is more than testing V&V must involve analysis, modeling, and simulation Tests need to check the analyses, models, and simulations – Verify predictions – Uncover invalid assumptions Key question: what have you learned about untested cases when all tests run successfully?

11 © 2011 Carnegie Mellon University Other SoS Assurance Considerations Early lifecycle is not just about making requirements/design testable Need to gather information showing that hazards to SoS operation have been recognized and dealt with appropriately Need to ensure that requirements and design do not introduce hazards Continuous SoS evolution implies Continuous V&V and/or V&V focused on robustness against unanticipated changes/usage – Analysis – Run-time monitoring

12 © 2011 Carnegie Mellon University Delivered System Building the System Building the Case Acceptance Case Building Justified Confidence Released System Explanation of why confidence is justified

13 © 2011 Carnegie Mellon University Contact John B. Goodenough Fellow Telephone: U.S. Mail: Software Engineering Institute Carnegie Mellon University 4500 Fifth Avenue Pittsburgh, PA

14 © 2011 Carnegie Mellon University NO WARRANTY THIS CARNEGIE MELLON UNIVERSITY AND SOFTWARE ENGINEERING INSTITUTE MATERIAL IS FURNISHED ON AN “AS-IS" BASIS. CARNEGIE MELLON UNIVERSITY MAKES NO WARRANTIES OF ANY KIND, EITHER EXPRESSED OR IMPLIED, AS TO ANY MATTER INCLUDING, BUT NOT LIMITED TO, WARRANTY OF FITNESS FOR PURPOSE OR MERCHANTABILITY, EXCLUSIVITY, OR RESULTS OBTAINED FROM USE OF THE MATERIAL. CARNEGIE MELLON UNIVERSITY DOES NOT MAKE ANY WARRANTY OF ANY KIND WITH RESPECT TO FREEDOM FROM PATENT, TRADEMARK, OR COPYRIGHT INFRINGEMENT. Use of any trademarks in this presentation is not intended in any way to infringe on the rights of the trademark holder. This Presentation may be reproduced in its entirety, without modification, and freely distributed in written or electronic form without requesting formal permission. Permission is required for any other use. Requests for permission should be directed to the Software Engineering Institute at This work was created in the performance of Federal Government Contract Number FA C-0003 with Carnegie Mellon University for the operation of the Software Engineering Institute, a federally funded research and development center. The Government of the United States has a royalty-free government-purpose license to use, duplicate, or disclose the work, in whole or in part and in any manner, and to have or permit others to do so, for government purposes pursuant to the copyright license under the clause at