1. © 2013 Carnegie Mellon University The Software Assurance Competency Model: A Roadmap to Enhance Personal Professional Capability Software Engineering Institute Carnegie Mellon University Pittsburgh, PA 15213 Nancy R. Mead, PhD

2. 2 Software Assurance Competency Model Nancy Mead, May 20, 2013 © 2013 Carnegie Mellon University Definition of Software Assurance The following definition of software assurance is used: Application of technologies and processes to achieve a required level of confidence that software systems and services function in the intended manner, are free from accidental or intentional vulnerabilities, provide security capabilities appropriate to the threat environment, and recover from intrusions and failures. May 2013

3. 3 Software Assurance Competency Model Nancy Mead, May 20, 2013 © 2013 Carnegie Mellon University Motivation for the competency model Industry concerns: Unable to describe or assess software assurance (SwA) skills of candidate employees Insufficient number of qualified candidates New employees need additional SwA skills training in order to perform their jobs Education and training concerns: No standard SwA curricula being offered Difficult for universities to address the need Lack of resources and course materials May 2013

4. 4 Software Assurance Competency Model Nancy Mead, May 20, 2013 © 2013 Carnegie Mellon University Software Assurance Curriculum Project August 2010 Master of Software Assurance Reference Curriculum 20092013 Community Outreach AMCIS COMPSAC SSTC FISSEA CISSE keynote 170+ Members of SwA Ed Community Outreach AMCIS COMPSAC SSTC FISSEA CISSE keynote 170+ Members of SwA Ed March 2011 Goals: Develop software assurance curricula Define transition strategies for implementation Professional Society Recognition Integrated into course offerings Carnegie Mellon University Stevens Institute of Technology US Air Force Academy University of Detroit Mercy University of Houston Integrated into course offerings Carnegie Mellon University Stevens Institute of Technology US Air Force Academy University of Detroit Mercy University of Houston Undergraduate Course Outlines MSwA Syllabi Community College Education Fall 2011 Needs MSwA course materials Nine SwA core courses Curriculum Development MSwA course descriptions for other degree programs Undergraduate curriculum with specializations High school needs Needs MSwA course materials Nine SwA core courses Curriculum Development MSwA course descriptions for other degree programs Undergraduate curriculum with specializations High school needs May 2013

5. 5 Software Assurance Competency Model Nancy Mead, May 20, 2013 © 2013 Carnegie Mellon University Purpose of SwA Competency Model The Software Assurance (SwA) Competency Model was developed to support the following uses: Enable assessment of SwA capabilities of employee candidates Offer guidance for developing SwA courses for an organization Provide information about industrial competency needs and expectations for curricula development Provide guidance for SwA professional development and career planning Provide support for professional certification and licensing May 2013

6. 6 Software Assurance Competency Model Nancy Mead, May 20, 2013 © 2013 Carnegie Mellon University Sources When developing the SwA Competency Model, research included consulting the following sources beyond the curriculum: Software Assurance Professional Competency Model (Department of Homeland Security, 2012) Information Technology Competency Model (Department of Labor, 2012) A Framework for PAB Competency Models (Professional Advisory Board and IEEE Computer Society, 2012) Balancing Software Engineering Education and Industrial Needs (Ana M. Moreno et al, 2012) May 2013

7. 7 Software Assurance Competency Model Nancy Mead, May 20, 2013 © 2013 Carnegie Mellon University Validation Method As the model was being developed, it was validated by industry reviewers, who are mapping the model to typical positions/skills. This helped to identify gaps and inconsistencies. Among others, the validators included representatives from: (ISC) 2 Board of Directors Harris Corporation Symantec May 2013

8. 8 Software Assurance Competency Model Nancy Mead, May 20, 2013 © 2013 Carnegie Mellon University Definition of Competency In the SwA Competency Model, the term competency represents the set of knowledge, skills, and effectiveness needed to carry out the job activities associated with one or more roles in an employment position: Knowledge is what an individual knows and can describe (e.g., can name and define various classes of risks). Skills are what an individual can do and involves application of knowledge to carry out a task (e.g., can identify and classify the risks associated with a project). Effectiveness is concerned with the ability to apply knowledge and skills in a productive manner, characterized by attributes of behavior such as aptitude, initiative, enthusiasm, willingness, communication skills, team participation, and leadership. May 2013

9. 9 Software Assurance Competency Model Nancy Mead, May 20, 2013 © 2013 Carnegie Mellon University Levels of Competency Levels of competency are used to distinguish between different levels of professional capability relative to knowledge, skills, and effectiveness. The five levels of SwA competency are characterized as follows: L1: Technician L2: Professional Entry Level L3: Practitioner L4: Senior Practitioner L5: Expert May 2013

10. 10 Software Assurance Competency Model Nancy Mead, May 20, 2013 © 2013 Carnegie Mellon University L1: Technician The following are characteristics of level 1. A technician possesses technical knowledge and skills, typically gained through a certificate or an associate degree program, or equivalent knowledge and experience. Personnel at this level of competency may be employed in a system operator, implementer, tester, or maintenance position with specific individual tasks assigned by someone at a higher level. Main areas of competency are System Operational Assurance, System Functional Assurance, and System Security Assurance. Major tasks are tools support, low-level implementation, testing, and maintenance. May 2013

11. 11 Software Assurance Competency Model Nancy Mead, May 20, 2013 © 2013 Carnegie Mellon University L2: Professional Entry Level The following are characteristics of level 2. A professional entry level professional possesses application-based knowledge and skills and entry-level professional effectiveness, typically gained through a bachelor’s degree in computing or through equivalent professional experience. Personnel at this level of competence may perform all tasks of L1 as well as manage a small internal project, supervise and assign sub-tasks for L1 personnel, supervise and assess system operations, and implement commonly accepted assurance practices. Main areas of competency are System Functional Assurance, System Security Assurance, and Assurance Assessment. Major tasks are requirements fundamentals, component design, and implementation. May 2013

12. 12 Software Assurance Competency Model Nancy Mead, May 20, 2013 © 2013 Carnegie Mellon University L3: Practitioner The following are characteristics of level 3. A practitioner possesses a breadth and depth of knowledge, skills, and effectiveness beyond the L2 level and typically has two to five years of professional experience. Personnel at this level may perform all tasks of L2 personnel as well as set plans, tasks, and schedules for in-house projects; define and manage such projects and supervise teams on the enterprise level; report to management; assess the quality of systems; and implement and promote commonly accepted software assurance practices. Main areas of competency are Risk Management, Assurance Assessment, and Assurance Management. Major tasks are requirements analysis, architectural design, tradeoff analysis and risk assessment. May 2013

13. 13 Software Assurance Competency Model Nancy Mead, May 20, 2013 © 2013 Carnegie Mellon University L4: Senior Practitioner The following are characteristics of level 4. A senior practitioner possesses breadth and depth of knowledge, skills, and effectiveness and a variety of work experiences beyond the L3 level with five to ten years of professional experience and advanced professional development at the master’s level or with equivalent education/training. Personnel at this level may perform all tasks of L3 personnel as well as identify and explore best practices of software assurance for implementation, manage large projects, interact with external agencies, etc. Main areas of competency are Risk Management, Assurance Assessment, Assurance Management, and Assurance Across Life Cycles. Major tasks are assurance assessment, assurance management, and risk management across the life cycle. May 2013

14. 14 Software Assurance Competency Model Nancy Mead, May 20, 2013 © 2013 Carnegie Mellon University L5: Expert The following are characteristics of level 5. An expert possesses competency beyond the L4 level. He or she advances the field by developing, modifying, and creating methods, practices and principles at the organizational level or higher. An expert has peer/industry recognition. Typically, experts include a low percentage of an organization’s work force within the software assurance profession (e.g., 2 % or less). May 2013

15. 15 Software Assurance Competency Model Nancy Mead, May 20, 2013 © 2013 Carnegie Mellon University Software Assurance Body of Knowledge (SwABoK) Knowledge Area (KA)KA Competency AALC: Assurance Across Life Cycles (L3, L4, L5) The ability to incorporate assurance technologies and methods into life-cycle processes and development models for new or evolutionary system development and for system or service acquisition RM: Risk Management (L2, L3, L4, L5) The ability to perform risk analysis and tradeoff assessment and to prioritize security measures AA: Assurance Assessment (L1, L2, L3, L4) The ability to analyze and validate the effectiveness of assurance operations and create auditable evidence of security measures AM: Assurance Management (L3, L4, L5) The ability to make a business case for software assurance, lead assurance efforts, understand standards, comply with regulations, plan for business continuity, and keep current in security technologies SSA: System Security Assurance (L1, L2, L3, L4) The ability to incorporate effective security technologies and methods into new and existing systems SFA: System Functionality Assurance (L1, L2, L3) The ability to verify new and existing software system functionality for conformance to requirements and to help reveal malicious content SOA: System Operational Assurance (L1, L2, L3) The ability to monitor and assess system operational security and respond to new threats

16. 16 Software Assurance Competency Model Nancy Mead, May 20, 2013 © 2013 Carnegie Mellon University Competency Attributes of Effectiveness 1 Aptitude: The ability do a certain software assurance activity at a certain level of competence. Aptitude is not the same as knowledge or skill but rather indicates the ability to apply knowledge in a skillful way. (L2-L5) Initiative: The ability to start and follow through on a software assurance work activity with enthusiasm and determination. (L1-L5) Enthusiasm: Being interested in and excited about performing a software assurance work activity. (L1-L5) Willingness: undertaking a work activity when asked, even if it is an activity the individual is not enthusiastic about performing. (L1-L5) May 2013

17. 17 Software Assurance Competency Model Nancy Mead, May 20, 2013 © 2013 Carnegie Mellon University Competency Attributes of Effectiveness 2 Communication: expressing thoughts and ideas in both oral and written forms in a clear and concise manner while interacting with team members, managers, project stakeholders, and others. (L2-L5) Teamwork: Working professionally and willingly with other team members while collaborating on work activities. (L1-L5) Leadership: Effectively communicating a vision, strategy, or technique that is accepted and shared by team members, managers, project stakeholders, and others. (L3-L5) May 2013

18. 18 Software Assurance Competency Model Nancy Mead, May 20, 2013 © 2013 Carnegie Mellon University SwA Competency Designations 2 Risk Management Concepts Risk Management Process Software Assurance Risk Management

19. 19 Software Assurance Competency Model Nancy Mead, May 20, 2013 © 2013 Carnegie Mellon University Example from the SwA Competency Model KAUnitCompetency Activities Risk Manage ment Risk Management Concepts L1: Understand the basic elements of risk management, including threat modeling. L2: Explain how risk analysis is performed. L3: Determine the models, process, and metrics to be used in risk management for small internal projects. L4: Develop the models, processes, and metrics to be used in risk management of projects of any size. L5: Analyze the effectiveness of the use and application of risk management concepts across an organization.

20. 20 Software Assurance Competency Model Nancy Mead, May 20, 2013 © 2013 Carnegie Mellon University Uses of the model We envision the following uses: Since it is a general model, industry and government organizations can instantiate the model for their own use. Faculty can use the model in conjunction with course development and outcomes. Industry managers can use the model for assessing SwA competencies and also for recruiting and building teams. SwA professionals can use the model for career planning. New graduates can use the model to map their SwA skills to job position descriptions and interviews, as well as for planning their next career growth steps. May 2013

21. 21 Software Assurance Competency Model Nancy Mead, May 20, 2013 © 2013 Carnegie Mellon University Summary and Plans To summarize the competency model work activities: The model builds on the prior software assurance curriculum work, and on other competency models. The model has been published as an SEI report and presented at DHS meetings, as a BrightTalk Webinar, and here. An additional short paper for IEEE Security and Privacy is planned. May 2013

22. 22 Software Assurance Competency Model Nancy Mead, May 20, 2013 © 2013 Carnegie Mellon University Additional Resources Hilburn, Thomas; Ardis, Mark; Johnson, Glenn; Kornecki, Andrew; & Mead, Nancy. Software Assurance Competency Model (CMU/SEI-2013- TN-004). Software Engineering Institute, Carnegie Mellon University, 2013. http://www.sei.cmu.edu/library/abstracts/reports/13tn004.cfmhttp://www.sei.cmu.edu/library/abstracts/reports/13tn004.cfm SwA Curriculum Website: http://www.cert.org/mswa/http://www.cert.org/mswa/ May 2013

23. 23 Software Assurance Competency Model Nancy Mead, May 20, 2013 © 2013 Carnegie Mellon University Questions? May 2013

24. 24 Software Assurance Competency Model Nancy Mead, May 20, 2013 © 2013 Carnegie Mellon University Copyright 2013 Carnegie Mellon University. This material is based upon work supported by the Department of Defense under Contract No. FA8721-05-C-0003 with Carnegie Mellon University for the operation of the Software Engineering Institute, a federally funded research and development center. Any opinions, findings and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the United States Department of Defense. NO WARRANTY THIS CARNEGIE MELLON UNIVERSITY AND SOFTWARE ENGINEERING INSTITUTE MATERIAL IS FURNISHED ON AN “AS- IS” BASIS. CARNEGIE MELLON UNIVERSITY MAKES NO WARRANTIES OF ANY KIND, EITHER EXPRESSED OR IMPLIED, AS TO ANY MATTER INCLUDING, BUT NOT LIMITED TO, WARRANTY OF FITNESS FOR PURPOSE OR MERCHANTABILITY, EXCLUSIVITY, OR RESULTS OBTAINED FROM USE OF THE MATERIAL. CARNEGIE MELLON UNIVERSITY DOES NOT MAKE ANY WARRANTY OF ANY KIND WITH RESPECT TO FREEDOM FROM PATENT, TRADEMARK, OR COPYRIGHT INFRINGEMENT. This material has been approved for public release and unlimited distribution except as restricted below. Internal use:* Permission to reproduce this material and to prepare derivative works from this material for internal use is granted, provided the copyright and “No Warranty” statements are included with all reproductions and derivative works. External use:* This material may be reproduced in its entirety, without modification, and freely distributed in written or electronic form without requesting formal permission. Permission is required for any other external and/or commercial use. Requests for permission should be directed to the Software Engineering Institute at permission@sei.cmu.edu.permission@sei.cmu.edu *These restrictions do not apply to U.S. government entities.

25. 25 Software Assurance Competency Model Nancy Mead, May 20, 2013 © 2013 Carnegie Mellon University SwA Competency Designations 1 Software Life-Cycle Processes Software Assurance Processes and Practices

26. 26 Software Assurance Competency Model Nancy Mead, May 20, 2013 © 2013 Carnegie Mellon University SwA Competency Designations 3 Assurance Assessment Concepts Measurement for Assessing Assurance

27. 27 Software Assurance Competency Model Nancy Mead, May 20, 2013 © 2013 Carnegie Mellon University SwA Competency Designations 4 Making the Business Case for Assurance Managing Assurance Compliance Considerations for Assurance

28. 28 Software Assurance Competency Model Nancy Mead, May 20, 2013 © 2013 Carnegie Mellon University SwA Competency Designations 6 For Newly Developed and Acquired Software for Diverse Applications For Diverse Operational (Existing) Systems Ethics and Integrity in Creation, Acquisition, and Operation of Software Systems

29. 29 Software Assurance Competency Model Nancy Mead, May 20, 2013 © 2013 Carnegie Mellon University SwA Competency Designations 6 Assurance Technology Assured Software Development Assured Software Analytics Assurance in Acquisition

30. 30 Software Assurance Competency Model Nancy Mead, May 20, 2013 © 2013 Carnegie Mellon University SwA Competency Designations 7 Operational Procedures Operational Monitoring System Control

31. 31 Software Assurance Competency Model Nancy Mead, May 20, 2013 © 2013 Carnegie Mellon University SwA Competency Designations 8 Aptitude Initiative Enthusiasm Willingness Communication Teamwork Leadership