1 SQA & Reuse Katerina Goseva-Popstojanova, WVU Aaron Wilson, NASA IV&V Kalynnda Berens & Richard Plastow, GRC Joanne Bechta Dugan, UVa David Gilliam JPL

2 Projects Real-time Linux Evaluations Kalynnda Berens & Richard Plastow, GRC Performability of Web-based applications Katerina Goseva-Popstojanova, WVU Reducing Software Security Risk through an Integrated Approach, David Gilliam & John Powel, JPL Software Assurance of Web-based Applications Tim Kurtz, GRC Software Quality & Safety Assessment Using Bayesian Belief Networks, Joanne Bechta Dugan, UVa

3 Performance benchmarking on flight-like hardware: –RTLinux (free version) V3.2 pre3 –RTLinux Pro (commercial) V2.0 –RTAI V –Linux Kernel (future) –Jaluna (future) RTLinux and RTAI are –Stable –Support many processors –Require a learning curve Real-time Linux Evaluations

4 Which Real-Time Linux is best? User-space Task CreationUser-space Program ForkingTiming Jitter (Harmonic)Timing Jitter (Non-harmonic)Context Switch TimingHardware InterruptsSoftware InterruptsKernel Task CreationInter-task MessagingGet SemaphoreGet/Release SemaphoreRelease/Get Semaphore RTLinux Pro BestOKWorst BestOK Worst Good RTLinux free WorstBest OKVery Good Worst BestOKWorst RTAIOK GoodBestWorstBest GoodBest

5 Web access log analysis User session characterization Realistic workload Software/hardware resource utilization Application & hardware resource monitoring Web error log analysis Request-based and session-based error characterization Software/hardware failure/recovery characterization Performability model Session layer (user view) Service layer (software architectural view) System layer (deployment view) Reliability/ availability model Performance model Resource layer (hardware device view) Web measurement and modeling framework

6 Cost effective way to improve quality 10-35% of the total number of errors are due to only 3 files Fixing the errors with the highest frequency of occurrence is the most cost effective way to improve Web quality

7 Software Vulnerabilities Expose IT Systems and Infrastructure to Security Risks Goal: Reduce Security Risk in Software and Protect IT Systems, Data, and Infrastructure Security Training for System Engineers and Developers Software Security Checklist for end-to-end life cycle Software Security Assessment Instrument (SSAI) Security Instrument Includes: Model-Based Verification Property-Based Testing Security Checklist Vulnerability Matrix Collection of security tools NASA Reducing Software Security Risk Through an Integrated Approach

8 Womb-to-Tomb Process Coincides with Organizational Polices and Requirements Security Risk Mitigation Process in the Software Lifecycle Software Lifecycle Integration –Training –Software Security Checklist Phase 1 –Provide instrument to integrate security as a formal approach to the software life cycle –Requirements Driven Phase 2: –External Release of Software –Release Process –Vulnerability Matrix – NASA Top 20 –Security Assurance Instruments Early Development – Model Checking / FMF Implementation – Property Based Testing –Security Assessment Tools (SATs) Description of available SATs Pros and Cons of each and related tools with web sites Notification Process when Software or Systems are De-Commissioned / Retired

9 How should NASA SA assure web-based applications? Solution –Implement the same types of controls on web-apps development that are used on other types of software development –Audit and review projects web-app development activities using a set of checklists –Pilot the guidebook/checklists Deliverables –Best Practices guidebook –Checklists Software Assurance of Web-based Applications

10 GETR Decision How can we investigate and document the decision process that is used to go from... to… I have an acceptable level of belief that the system will operate as specified. Quality Assurance Test ResultsPersonal and Team CMM Prototype Performance Requirements Review Is the system good enough to release? Code InspectionRisk Assessment Formal Methods for a computer-based system Engineering Judgment Software Quality & Safety Assessment Using BBN

11 BBN model of Software Development Process

12 Technology Readiness Level Software Quality & Safety Web performability Reducing software security risk

13 Brief description of the field –Quality attributes: reliability, performance, security, maintainability, and reusability –Techniques Testing: property testing, performance testing Real system, real workload Analysis & Modeling: model checking, statistical & probabilistic analysis, BBN –Process & product

14 Potential benefits Improved decision support, prioritization, better allocation of resources Better product in a cost effective way through integrated approaches Increased fidelity without increasing complexity

15 Directions Increased coordination through unified approaches Infusion of improved techniques into current processes Improving the state of practice

16 Why Potential benefits to NASA –Fewer mission failures –Reduced complexity –Greater reuse of software artifacts and process improvements –Transference of best practices and lessons learned

17 Why not Standard traps –“There is no silver bullet” –“Teaching to the test” –Deadline vs. quality driven development –Tunnel vision –Dependencies on hardware and OS –Poor documentation and quality of data

18 Who is using this technology NASA projects that are using this technology –Security checklist at JPL –RT Linux Pro at Glenn –Web performability at NASA IV&V –Web-based process assurance at Glenn –Seal of Approval Process for PRA tools at NASA HQ Other projects outside of NASA that are using these tools/approaches –Web performability at LDCSEE –Formal security verification at Patchlink

19 Questions/Issues Reliability, availability, performance, security –Integrated approaches needed –What are the interactions & tradeoffs? Process & product Better, Cheaper, Faster –Can we have it all? –Should we pick (any) two?