DoD Software Engineering and System Assurance New Organization – New Vision Kristen Baldwin Office of the Under Secretary of Defense Deputy Director, Software Engineering and System Assurance February 14, 2007

SYSTEMS AND SOFTWARE ENGINEERING CENTER OF EXCELLENCE, DUSD(A&T) Slide 2 Recent Acquisition & Technology Reorganization as of June 1, 2006 Under Secretary of Defense Acquisition Technology & Logistics Deputy Under Secretary of Defense Acquisition & Technology Director Systems & Software Engineering Systems Engineering is well positioned in DoD

SYSTEMS AND SOFTWARE ENGINEERING CENTER OF EXCELLENCE, DUSD(A&T) Slide 3 Director, Systems & Software Engineering Mark Schaeffer SES Deputy Director Enterprise Development Bob Skalamera SES Deputy Director Developmental Test & Evaluation Chris DiPetto SES Deputy Director Software Engineering & System Assurance Mark Schaeffer (Acting) SES Deputy Director Assessments & Support Dave Castellano SES CORE COMPETENCIES SE Policy SE Guidance SE in Defense Acquisition Guidebook Technical Planning Risk Management Reliability & Maintainability Contracting for SE SoS SE Guide SE Education and Training DAU SE Curriculum SPRDE Certification Rqmt Corrosion R-TOC Value Engineering CORE COMPETENCIES DT&E Policy DT&E Guidance T&E in Defense Acquisition Guidebook TEMP Development Process DT&E Education and Training DAU DT&E Curriculum DT&E Certification Rqmt Joint Testing, Capabilities & Infrastructure Targets Oversight Acq Modeling & Simulation Energy DSOC/Acq Tech Task Force CORE COMPETENCIES SWE and SA Policy SWE and SA Guidance SoS, SA Guides SWE and SA Education and Training DAU SW Acq Curriculum Continuous Learning Modules for SWE, SoS, SA Software Engineering Acquisition Support Software Engineering Institute (SEI) Process Improvement CMMI Sponsor DoD/National Software Investment Strategy CORE COMPETENCIES Support of ACAT I and Other Special Interest Programs (MDAP, MAIS) Assessment Methodology (Program Support Reviews - PSRs) T&E Oversight and Assessment of Operational Test Readiness (AOTR) Systems Engineering and Developmental Test Planning and Support Lean/6-Sigma Training/Cert Systems and Software Engineering Organizational Core Competencies Acquisition program excellence through sound systems and software engineering Est. Aug 06

SYSTEMS AND SOFTWARE ENGINEERING CENTER OF EXCELLENCE, DUSD(A&T) Slide 4 DoD Software Center of Excellence Establishing a DoD Software Center of Excellence DoD Software Center of Excellence Support Acquisition Success Improve State-of-the-Practice of Software Engineering Leadership, Outreach and Advocacy Foster Resources to Meet DoD Needs DoD Software Center of Excellence Support Acquisition Success Improve State-of-the-Practice of Software Engineering Leadership, Outreach and Advocacy Foster Resources to Meet DoD Needs

SYSTEMS AND SOFTWARE ENGINEERING CENTER OF EXCELLENCE, DUSD(A&T) Slide 5 Elements of a DoD Strategy for Software Support Acquisition Success –Ensure effective and efficient software solutions across the acquisition spectrum of systems, SoS and capability portfolios Improve the State-of-the-Practice of Software Engineering –Advocate and lead software initiatives to improve the state-of-the- practices through transition of tools, techniques, etc. Leadership, Outreach and Advocacy –Implement at Department and National levels, a strategic plan for meeting Defense software requirements Foster Software Resources to meet DoD needs –Enable the US and global capability to meet Department software needs, in an assured and responsive manner Promote World-Class Leadership for Defense Software Engineering

SYSTEMS AND SOFTWARE ENGINEERING CENTER OF EXCELLENCE, DUSD(A&T) Slide 6 DoD Software Performance: What We’re Seeing* Software systemic issues are significant contributors to poor program execution –Software requirements not well defined, traceable, testable –Immature architectures, COTS integration, interoperability, obsolescence (electronics/hardware refresh) –Software development processes not institutionalized, planning documents missing or incomplete, reuse strategies inconsistent –Software test/evaluation lacking rigor and breadth –Schedule realism (compressed, overlapping) –Lessons learned not incorporated into successive builds –Software risks/metrics not well defined, managed *Based on ~40 program reviews in the past 2 ½ years

SYSTEMS AND SOFTWARE ENGINEERING CENTER OF EXCELLENCE, DUSD(A&T) Slide 7 Getting Started – What are we Doing? Identify software issues, needs –Software Industrial Base Study –NDIA Top Software Issues Workshop –Defense Software Strategy Summit Creating opportunities, partnerships –Established network of Government software POCs –Chartered the NDIA Software Committee –Information exchanges with Government, Academia, and Industry –Planning the Systems & Software Technology Conference, TAMPA! Executing focused initiatives –CMMI Integrity, CMMI-ACQ, CMMI Guidebook –Handbook on Engineering for System Assurance –SoS Systems Engineering Guide –Providing software support to acquisition programs –Transparent data vision for Acquisition Reform

SYSTEMS AND SOFTWARE ENGINEERING CENTER OF EXCELLENCE, DUSD(A&T) Slide 8 1.The impact of system requirements upon software is not consistently quantified and managed. 2.Fundamental system engineering decisions are made without full participation of software engineers. 3.Software life cycle planning and management by acquirers and suppliers is ineffective. 4.The quantity and quality of software engineering expertise is insufficient for dealing with the scale of complexity of modern systems. 5. Software verification techniques are costly and ineffective for dealing with the scale of complexity of modern systems. 6. There is a failure to assure correct, predictable, safe, secure execution of complex software in distributed environments. 7. Inadequate attention is given to total life cycle issues for COTS/NDI impacts on life cycle cost and risk. NDIA Top Software Issues COTS = commercial off-the-shelf NDI = non-developmental item

SYSTEMS AND SOFTWARE ENGINEERING CENTER OF EXCELLENCE, DUSD(A&T) Slide 9 1.Enforce effective software requirements development and management practices. 2.Institutionalize the integration and participation of software engineering in all system activities. 3.Establish a culture of quantitative planning and management 4.Collaborate on approaches attract, develop, and retain qualified talent to meet current and future needs in government and industry. 5. Develop guidance and training to improve effectiveness in assuring product quality across the life cycle. 6. Develop approaches, standards, and tools, addressing system assurance issues throughout the acquisition life cycle and supply chain. 7. Improve and expand guidelines for addressing total life cycle COTS/NDI issues NDIA Top Software Recommendations

SYSTEMS AND SOFTWARE ENGINEERING CENTER OF EXCELLENCE, DUSD(A&T) Slide 10 Software Strategy Summit Events October Keynote Address, the Honorable Dr. James I. Finley, Deputy Under Secretary of Defense (Acquisition and Technology) Program Executive Officer and Service/Defense Agency panels on software related acquisition issues and initiatives Plenary Session Topics -- NDIA Top Software Issues, Software Industrial Base Study, and Software Producibility Workshops –Software Acquisition and Sustainment Mr. Mike Nicol, Air Force Aeronautical Systems Center Mr. Lawrence T. Osiecki, US Army, Armament Software Engineering Center –Policy Mr. Jim Clausen, DoD CIO, Office of Commercial IT Policy Col Peter Sefcik, Jr., USAF Chief, Air Force Engineering Policy and Guidance Team Lt Col Mark Wilson, SAF/AQR Systems & Software Engineering –Human Capital Dr. Kenneth E. Nidiffer, Fellow, Systems and Software Consortium Mr. George Prosnik, Defense Acquisition University E&T Center –Software Engineering Practices Mr. Grady Campbell, Software Engineering Institute Mr. Paul R. Croll, CSC, Industry Co-Chair NDIA Software Committee

SYSTEMS AND SOFTWARE ENGINEERING CENTER OF EXCELLENCE, DUSD(A&T) Slide 11 Summary of Workshop Issues* Software Acquisition and Sustainment Software issues not addressed early in lifecycle Software requirements not well defined at program start Management has limited visibility into software development processes and status Risk areas – single point failures not adequately addressed, e.g., single software providers, incomplete data rights, key personnel stability, life cycle support of COTS Acquirers do not adequately address software sustainment and total life cycle early in the program Some agencies contract before engineering is complete, prior to system design and development Policy PMs need assistance with software policy and analysis Arbitrary separation of weapon and information technology software policies Policy implementation guidance and follow-up monitoring is limited Department needs software group with good expertise to oversee and implement policy Need capability to share policy and guidance information Human Capital Experienced system & software engineers seem missing from key DoD leadership positions Shortage of highly experienced software managers, architects, domain and technical experts Eroding depth and breath of experience for personnel in DoD Young people may consider system and software engineering as a career dead end Emerging skill set may be needed for future complex DoD systems, e.g., systems of systems Software Engineering Weak linkage between software requirements and capabilities/portfolios System development methods do not properly leverage software ability to rapidly field new capability Systems and software engineering lifecycles not always consistent or harmonized Software considerations not consistently addressed in architectures Inadequate software estimating methods, e.g., COTS/NDI; best practices not applied *Reaffirmed NDIA Top Software Issues (See NDIA issue slide)

SYSTEMS AND SOFTWARE ENGINEERING CENTER OF EXCELLENCE, DUSD(A&T) Slide 12 Summary of Workshop Recommendations* Software Acquisition and Sustainment Require earned value reporting for software Require software sustainment planning at all milestones, e.g., in Systems Engineering Plan Improve architecting of systems and software Develop software work force competency maturity plan; balance software acquisition needs against capabilities Improve software metrics reporting and risk management Emphasize acquisition “-ilities” for software Policy Set up portal to share policy & guidance information Analyze policy, cross reference results, define commonality, evaluate policy quantitatively, update guidance, e.g., information technology and weapon systems software polices Update guidance for GOTS, COTS, software best practices, portfolio and risk mgmt, as a minimum Establish DoD software group to oversee and implement policy Require program offices to have software engineering expertise Human Capital Build experience bases, e.g., software communities of practice, preferred processes, tools Consider establishing a software engineering specialty track for DoD acquisition personnel Emphasize software acquisition management skill for DoD System Engineers and Project Managers Ensure teams have both systems engineering and software development expertise Assess the results of on-going service and agency human capital efforts Encourage internal process improvement for DoD acquisition offices Software Engineering Break-up large programs/architectures Set up product lines Raise software as a major factor within the work breakdown structure Revise contracting methods to allow additional detail for software requirements Provide leadership training to include a software appreciation Provide cadre of no-cost experts to support PMs Encourage independent program assessments *Reaffirmed NDIA Top Software Recommendations (See NDIA recommendation slide)

SYSTEMS AND SOFTWARE ENGINEERING CENTER OF EXCELLENCE, DUSD(A&T) Slide 13 Preliminary Analysis – Gap Identification Many issues and recommendations require further review and analysis Inventory of Service and Defense Agency on-going initiatives not complete Analysis of existing guidance, such as, directives, instructions, plans, and standards, that are related to software management is not complete Where possible, need to quantify software acquisition issues, establish benchmarks, and identify problem causes Need to prioritize issues and determine collective best path forward

SYSTEMS AND SOFTWARE ENGINEERING CENTER OF EXCELLENCE, DUSD(A&T) Slide 14 Our Challenge Given the shortage of software resources and critical software reliance –We cannot afford to be stovepiped –We must integrate across cross-functional perspectives to improve our software capability Next Steps… –Analyze Assess summit issues and recommendations –Prioritize and begin to implement Summit recommendations –Contribute to ongoing initiatives: Systems of Systems, System Assurance, CMMI, more –Explore new areas: Agile methods for DoD, SW estimation, SW Test, more Follow up –Track and monitor the above –Conduct a 2007 Software Summit to review these actions and recommend adjustments for 2008 and beyond.