Independent Verification and Validation (IV&V) Techniques for Object Oriented Software Systems SAS meeting July 2003

COMPANY PROPRIETARY 2 APC0342 Outline IV&V of OO artifacts Identifying Risks Proposed Techniques GQM (Goal Question Metric) Application BBN (Bayesian Belief Network) Application

COMPANY PROPRIETARY 3 APC0342 Problem Statement NASA has considerable experience applying IV&V to traditional function-based software systems However the emerging use of OO brings unique challenges and risks There is a need to develop IV&V techniques for addressing these risks Verify these techniques

COMPANY PROPRIETARY 4 APC0342 Objective Understand risks unique to large OO systems Establish a framework for identifying, evaluating, and implementing IV&V techniques for mitigating these risks

COMPANY PROPRIETARY 5 APC0342 Multi -Year Goals Identify Risks Develop base set of IV&V Techniques Apply & evaluate techniques Focus additional research and match emerging techniques to prioritized needs

COMPANY PROPRIETARY 6 APC0342 Approach Phased Approach –Identify Risks, Develop Base set of Techniques, Evaluate Base set –Focused research to adapt techniques for a broader spectrum of OO projects Reasoning –Level of maturity of IV&V ’ s understanding of the problem, developers processes, and V&V techniques specific to OO system development –Limited availability of defect data specific to OO development

COMPANY PROPRIETARY 7 APC0342 Focused research to map, adapt, or develop techniques to prioritized IV&V activities based on experienced and perceived needs for improvement in efficiency, accuracy, and capacity. 1) Identify OO Development Risks 2) Develop Base IV&V Techniques –Requirements –Analysis –Design 3) Evaluate Base IV&V Techniques Continuous research & evaluationof emerging techniques Current PhaseNext Phase

COMPANY PROPRIETARY 8 APC0342 Activities to-date Evaluation of current & emerging OO IV&V techniques Identification of Risks with OO development Investigating framework for OO IV&V Developing base set of IV&V techniques

COMPANY PROPRIETARY 9 APC0342 State of IV&V Practice Rudimentary manual techniques for Use cases or OO analysis Techniques limited to static analysis No automated tools or metrics being used for requirements, analysis & design Manual techniques ensure Specifications use UML compliant notation Design is adequately documented Design specifications will meet requirements Techniques do not address the quality of OO design objectives Encapsulation, coupling, cohesion, polymorphism etc.

COMPANY PROPRIETARY 10 APC0342 Emerging Techniques Some examples of emerging techniques –Dynamic Analysis Techniques e.g. Yacoub et. al. "Dynamic Metrics for Object Oriented Designs" –Semantic Metrics e.g. Etzkorn et. al. “Towards a Semantic Metrics Suite for OOD” –Application of BBN e.g. Fenton et.el. “Bayesian Belief Network Model for the Safety Assessment of Nuclear Computer-Based Systems” State of practice –These techniques offer exciting promise for improving IV&V products –Need further enhancement for practical IV&V application

COMPANY PROPRIETARY 11 APC0342 Risks to OO Development Risk to OO development fall into three major categories –Programmatic Risks Project termination, software component integration, utilization of tools, and project failure –Technology Risks Issues with technology used to build the product, such as: Operating system, Compilers, Libraries, and middleware –Methodology Risks Type of OO methods being used, incorrect application of a method, and issues related to lifecycle phases

COMPANY PROPRIETARY 12 APC0342 Programmatic Risks Project termination –Sub par schedule and performance of product Methods abandonment –Lack of progress or OO related benefits High learning curves –Insufficient budget and schedule for training Not meeting OO objectives –Autonomy between organizations, absence of short iterative development cycle Dependencies on Tools –Cost and tool deployment requirements

COMPANY PROPRIETARY 13 APC0342 Technology Risks Object Model –Technology does not fully support the object model e.g. asynchronous communication between objects in relational databases Persistence –Poor support for object persistence e.g. OODB, relational DB, ad-hoc flat files

COMPANY PROPRIETARY 14 APC0342 Methodology Risks Incorrect application of methodology, insufficient methodology, or no methodology –Design mixed with requirements/use cases, insufficient information in design models Software performance –Language/hardware combination fails to supports performance High defect level –System usability, reliability, and safety

COMPANY PROPRIETARY 15 APC0342 GQM Approach Identify software product metrics that can support analysis goals For Use Case diagrams –GOALS- completeness, traceability, correctness –Questions-How to achieve the goals –Metrics-use case metrics such as by Marchesi Later plans for application to Class diagrams

COMPANY PROPRIETARY 16 APC0342 BBN Approach For Use Case diagrams –Use of BBN to determine readiness for project transition into the next lifecycle phase –Actors, actions, communication association, as part of network topology –Work is on going for determining characteristics of nodes, e.g. weight, child and parent levels –CPT tables Assumption: Use Cases completely expanded

COMPANY PROPRIETARY 17 APC0342 Near term Goals Develop base set of IV&V techniques for –Requirements phase products assuming Use Case Artifacts. –Analysis phase products assuming class and sequence diagrams. –Design phase products assuming class, sequence, and deployment diagrams and state charts. Solidify feasibility of applying –GQM for Requirements and Analysis Phases –BBN to gauge readiness for lifecycle phase transition