1. With placeholders for inputs from OSD and Services - TBD
NDIA Systems Engineering Division Annual Planning Meeting Summary and 2014 Plans Action Plan Status Committee Reports
December 11-12, 2013
With placeholders for inputs from OSD and Services - TBD

2. Introduction Status of 2013 NDIA SE Division activities
Relative to 2013 Plan
SED plan from 12/12 meeting, briefed at 2/13 meeting
Committee Reports
2013 Accomplishments
2014 Plans

3. NDIA SED 2013 Tasks - Derived from DoD/Services Initiatives (Voice of the Customer)
ASA (ALT)
OCSE
Dev Planning
SE Workforce
SE methods/ tools
Reliability
Security protection
ExportabiIity
Soldier/Small Unit Challenge
DASD (SE)
ERS
Anti-Counterfeit
Risk Mgmt
Schedule risk
Supplier risks
Security design
SE models (MBx)
SoSE
Verbal
Inputs
Provided
SAF AQRE
Policy (levels)
Process
Tradespace
Architecture
Cost Assessment
Specialty Engrg
People / Workforce (specialists vs. generalists)
DASN (RDT&E)
SEP
Agile in acquisition
SoS mission reqts
R&D assessment

4. NDIA SED 2013 Action Plan Status#
NDIA SED 2013 Tasks
Committee
Plan
1 
Quantitative risk management guidance. [DASD(SE)]
SEEC
(Brown, Rassa)
Ongoing action. Feb workshop planned.
Risk workshop planned, deferred.
No action in 2012 or 2013 – drop it?
2 
Support Engineered Resilient Systems (ERS) initiatives. Link ERS with M&S? [DASD(SE)]
SED (Rassa)
M&S
IEEE/NDIA ERS workshop in planning. Other actions pending. Division organizational emphasis?
No action yet, still pending.
3 
Anti-counterfeit – SE role, industry feedback for supply chain risk management, augment body of knowledge [DASD(SE)]
SED
(Henry)
Issue SED request for 1:1 company inputs provided direct to DASD(SE). Potential follow-up workshop. Could be company proprietary? Coordinate with other NDIA, AIA, etc.
No request from OSD; no action
4 
Design Patterns for Security – share SERC research findings (UVA)
[DASD(SE)]
Schedule Dr. Horowitz (UVA) for NDIA SED briefing.
Done, Feb 2013 SED meeting.

5. NDIA SED 2013 Action Plan Status#
NDIA SED 2013 Tasks
Committee
Plan
5 
Advanced Modeling: Model Based for “X” (MBx) – how could acquisition strategies be enhanced to use models (RFPs, etc.) [DASD(SE)]
M&S
(Coolahan)
M&S committee study or report. Collect industry input, examples.
See M&S plan.
6 
SoSE: how to demand articulation and insertion of SoS context during life cycle acquisition? [DASD(SE)]
SoS (Dahmann)
SoS committee study or report and recommendations. International issues?  7
Soldier / Small Unit Challenge
[ASA(ALT)]
SED WG
(Serna)
WG to consider soldier and small unit architecture.
Discontinued; lack of Army support 8
Design for Reliability – need RFP language. [ASA(ALT)]
LCS
(Cothran)
Consider input on RFP language.
LCS committee is inactive; no chair 9
Security / Protection for coalition operations [ASA(ALT)]
System Assurance (Croll)
SA committee to consider security issues for coalition – provide recommendations to SED for action
SA committee rechartered to SSE

6. NDIA SED 2013 Action Plan Status#
NDIA SED 2013 Tasks
Committee
Plan 10
Design for Exportability – BBP 2.0, foreign military sales (FMS), direct commercial sales (DCS) [ASA(ALT)]
No SED action 11
Reconcile Agile development with DoD acquisition practices, requirements, IPT structure, cost, skills
[DASN (RDT&E)] SW
(Draper)
Conduct workshop. Collect best practices (cross-discipline, SEI, DHS, AFEI, INCOSE, …
Partner with SEI, ADAPT. See Agile guidance. Kickoff ‘Agile and SE’ at NDIA SE Conf – WG formation in 2014. 12
Consider how to move tradespace for architecture early in the life cycle (pre-MDD)
Architecture (Sheeley)
MA (Lohse)
Start initially with white paper vision on use of architecture to bound/expand the trade space (solution agnostic)
See Arch and MA status briefings 13
Collect industry best practices for growing and retaining specialists vs. generalists
E&T (Gelosh)
How does industry keep its specialists?
See E&T briefing for status

7. NDIA SED 2014 Actions # NDIA SED 2013 Tasks Committee Plan / Status 1
Brief the existing M&S life cycle cost model report at the NDIA SED February meeting
M&S
(Bergenthal, Coolahan)
Plan for Feb 2014 NDIA SED meeting. 2
Review the Top 5 SE issues report on technical authority and consider if any NDIA actions are appropriate. Consider best practices (industry or government) to implement technical authority.
NDIA SED
(Henry, Rassa, Draper, Serna) 3
Request DoD briefing on how program startups work (Dave Ahern). Provide industry best practices on program startup/assist teams.
(Henry, Rassa)
Schedule for NDIA SED meeting briefings. 4
Support MORS affordability handbook development and community of practice initiatives.
Affordability WG / NDIA SED
Serna 5
NDIA to provide recommendations on deployment of SE and standards (e.g., RFP language, transition, governance, compliance).
(Draper / Rassa / Roedler / Henry)
Coordinate with joint IEEE/DoD/industry WG. 6
Assess new DoDI and provide recommendations on any NDIA implementation actions needed (impact analysis).
SEEC (Brown) or short term WG
Focus on industry clarification, adoption and impact analysis. 7
Collect industry best practices for quantitative risk management.
(Rassa, Forbes)
Establish risk management WG, with meetings held following NDIA SED meetings. 8
Support Engineered Resilient Systems (ERS) initiatives.
NDIA SED / IEEE
(Rassa)
Follow on from open 2013 actions. IEEE activity with NDIA support.

8. NDIA SED 2014 Actions # NDIA SED 2013 Tasks Committee Plan / Status 9
Schedule committees for special topics and detailed briefs at NDIA SED meetings.
NDIA SED (Henry)
Feb: SoS – Aug?
April: SSE (PPP) – June?
June: M&S
August: E&T
October (SE Conf plenary?): DT&E 10
Consider ways to increase NDIA SED collaboration internally (committees, other NDIA divisions) as well as externally (other organizations, company representatives)
NDIA SED
Consider a consolidated and managed list of NDIA external stakeholders at the SED level. Ensure integration without duplicating effort. 11
Several committees could use more industry and government participation – consider ways to build more involvement, given current constraints.
Collect all needs and descriptions in a consolidated request to Eng VPs? 12
Document and publicize the procedures for releasing reports through NDIA. Identify which committee products are in draft form that need to be formalized.
NDIA SED (Rassa, Britt)
Reports need to go through NDIA (Britt) prior to sending to the SED steering group for review. For web publishing send to Britt Bommelje and Rebecca Danahy. 13
Britt to provide a copy of the NDIA SED charter.
NDIA
Distributed via . 14
Look at M&S use assessment and identify areas for further tasks to support MBSE.
M&S
This could be useful to drive other research.

9. NDIA SED 2014 Actions # NDIA SED 2013 Tasks Committee Plan / Status 15
Some NDIA SED web pages are badly out of date – SED leadership and committee to update web pages under their areas of control.
NDIA SED / Committee Leads 16
ODASD(SE) to provide guidance on next steps for SE Effectiveness study (e.g., briefing SAEs)
NDIA SED / ODASD(SE)
Dona to provide feedback. 17
Consider a follow-on to the root cause analysis study of PSR findings.
Potential POCs: Bob Rassa, Jim Thompson, Pete Nolte, Gary Bliss, …

10. Committee Inputs - 2013 Task Progress / 2014 Task Plans
Architecture (Arch) Automatic Test (ATC) Development Test and Evaluation (DT&E) Education and Training (E&T) Enterprise Health Mgmt (EHM) Environment, Safety, and Occupational Health (ESOH) Human-System Integration (HSI) – no slides Joint Committee on Systems Engrg in Manufacturing (JCSEM) – no slides Life Cycle Support (LCS) – no slides Modeling and Simulation (M&S) Mission Analysis (MA) Systems Engineering Effectiveness Committee (SEEC) Software Committee System of Systems (SoS) Systems Security Engineering (SSE)

11. Architecture Committee – 2013 Task Summary
Meetings:
7 Virtual meetings
1 Face-to-face meeting
SE Conference:
15 papers
Accomplishments:
Completed the DoDAF Recommendations Report
Delivered it to Steve Henry
Posted on the website
Presented Architecture Metrics results (worked in 2012) at the INCOSE Symposium
Participated in SEI SoS Architecture workshop
Received architecture informational briefings / demos from C.Gedo, Architecture Stnds Group (DISA & GTRI), and on J6 Architecture portal from B.Piazza (JS J6)
In-work:
Reviewing whitepaper “Role of Architecture to Influence the Development Planning Tradespace” with the Mission Analysis subcommittee
Open system architecture discussions with Dr. Flietz (AFRL)

12. Architecture Committee - 2014 Task Plan
Proposed 2014 Tasks:
Complete joint Architecture/Mission Analysis white paper “Role of Architecture to Influence the Development Planning Tradespace”
Investigate status and make proposals around subject of Open System Architectures
Initiate joint discussions with Test Committee
Deliverables/Products
“Role of Architecture to Influence the Development Planning Tradespace” whitepaper
Schedule / Resources
Release whitepaper 1Q
Initiate Open System Architecture discussions 1Q
Initiate joint Architecture – Test discussions 1Q
Issues / Concerns:
Availability of committee members

13. Automatic Test Committee – 2013 Status

14. Automatic Test Committee - 2014 Task Plan Projects Working Group
Proposed 2014 Tasks:
Government/Industry Liaison Reports
Detail, Staff, Schedule & Initiate new ATC tasks:
Industry Survey for Smart Automatic Test System Opportunities
Software Architecture for Synthetic Instrumentation
Deliverables/Products
Semi-annual Reports posted on ATC Web-site:
US Army ATS Liaison Report
USAF ATS Liaison Report
US Navy/USMC-Air ATS Liaison Report
USMC-Ground ATS Liaison Report
DoD AMB Liaison Report
Commercial ATS Liaison Report
Deliverables from new tasks: TBD
Schedule / Resources
Status/Review/Present at regular ATC meetings:
March/April (adjacent DoD ATS Management Board (AMB) Meeting – Gov’t participating committee) at NDIA HQ
September in St Louis (AUTOTESTCON)
December with SED (TBD)
No resource issues or support anticipated
Issues / Concerns:
Only one concern specific to ATC:
Reduced defense budgets and fallout from the Las Vegas GSA meeting has had a significant impact on Government (AMB) participation in many events involving travel – not just NDIA - including holding virtual AMB meetings.
Response:
ATC is holding its meeting at NDIA HQ adjacent AMB scheduled meetings, and will remain flexible in scheduling our meetings to maximize attendance, participation, etc.

15. Steve Scukanec, Northrop Grumman
Developmental Test and Evaluation Committee Status and Plans December 2013
Beth Wilson, Raytheon
Steve Scukanec, Northrop Grumman
Industry Co-Chairs

16. DT&E Committee 2013 Status Activity Plans for 2013 Status/Plans
Scientific Test and
Analysis Techniques
STAT for T&E
Effective use of statistical approaches for test optimization (including DOE) with examples
White paper complete
Preparing journal publications
Presented results at SE conference
Modeling & Simulation
Evaluate test as a stakeholder
Joint meetings with SED meetings
Propose for 2014
Architecture
NDIA T&E Conf
DT&E panel, STAT report
Conference cancelled
NDIA SE Conf
DT&E tracks
Joint tracks, metrics/STAT reports
Added Task Dec 2012:
Metrics
Provide more information on leading indicators for T&E
Status: draft report
Plans: results at SE conference
Reliability
Develop a better test strategy for reliability (collaborate with IEEE Reliability Society)
Discussed at June 2013 meeting
Leads identified Nov 2013
Added Task Jun 2013:
Cyber Testing
Develop recommendations for Cyber Guidelines 2.0 in 2014
Lead and team identified
Kicked off Nov 2013 (joint with ITEA)
Chief Developmental Tester
Develop recommendations for industry interaction with Chief Developmental Tester role
Complete
In Process
Cancelled

17. DT&E Committee - 2014 Task Plan Projects Working Group
Proposed 2014 Tasks:
Cyber Testing Project
Chief Developmental Tester Project
Collaboration with IEEE Reliability
Joint meetings with SED Committees
Modeling and Simulation
Architecture
Systems Security Engineering
Education and Training
Deliverables/Products
Cyber Testing Recommendations white paper
Chief Developmental Tester white paper
Candidate KSABs for role (E&T)
Industry interaction recommendation
Tutorial on role for SE conference
Inputs to reliability standards effort
Joint meeting notes
Schedule / Resources
Cyber Testing Draft Recommendations Feb 2014
Weekly teleconferences started 12/3/2013
Full white paper later in 2014
Chief Developmental Tester
Joint meeting with E&T with SED meeting
Tutorial at SE conference
Reliability standards input for Jan/Aug meetings
Test as a Stakeholder joint meetings with SED
Modeling and Simulation
Architecture
Systems Security Engineering
Issues / Concerns:
Industry engagement
NDIA T&E conference may be cancelled (Seattle in March)
ITEA conference (Denver in September) may be postponed and moved

18. Summary of DT&E Committee 2013 Results/2014 Plans
Topic
Activity
T&E
Test Optimization
2013: Scientific Test and Analysis Techniques report/articles
Cyber Testing
: Industry recommendations for cyber testing
Chief Developmental Tester
: Industry interaction with new role SE
Performance Measurement
2013: Requirements Verification leading indicators report
Reliability
: Recommendations for better reliability testing
Modeling and Simulation
2014: T&E Perspective for Modeling/Simulation products
Architecture
2014: T&E Perspective for Architecture views
Education and Training
2014: Chief Developmental Tester role
Systems Security
Engineering
2014: Cyber testing guidelines connections to
Program Protection Planning
Completed
Current
Proposed

19. DT&E Committee 2012-2013 Statistical Test Optimization
Activity
Plans for 2013
Status/Plans
Scientific Test and
Analysis Techniques
STAT for T&E
Examples of effective use of statistical approaches for test optimization (including DOE) for an implementation framework
White paper complete
SE Conference track: 2012 and 2013
ITEA journal publication
MORS conference Nov 2013
2012 SE Conference summit/workshop thread
Tutorials on Monday 10/22
Presentations on Wednesday 10/24
Synthesis Panel on Wednesday 10/24
Follow-on Results:
DT&E Committee white paper
ITEA Journal article
SE Conference track Oct 2013

20. DT&E Committee 2012-2013 Leading Indicator Metrics
Activity
Plans for 2013
Status/Plans
Metrics
(with Performance Measurement WG)
Provide more information on leading indicators for T&E
Draft report
Presented results at SE conference
2012 SE Conference workshop
Follow-on workshop with System Performance Measurement WG
Focused on Requirements Verification not addressed in first report
Potential Measures Identified:
System Maturity Level
Verification Requirement Maturity
Technical Measures and Stakeholder Need

21. DT&E Committee 2012-2013 Reliability Testing
Activity
Plans for 2013
Status/Plans
Reliability Testing
(with IEEE)
Develop a better test strategy for reliability (collaborate with IEEE Reliability Society)
Solicit feedback on existing standards
Identify recommendations to improve reliability testing
Reliability Testing Focus
Software reliability standard
Cyber testing initiatives
Data for reliability growth measures
Input/Feedback Opportunities:
IEEE meetings January and August
Periodic revision periods
Project Leads:
Lou Gullo, Taz Daughtrey

22. DT&E Committee 2013-2014 Cyber Testing
Activity
Plans for
Status/Plans
Cyber Testing
Review what industry does for cyber testing, expand focus to include interoperability
Recommendations for Cyber Testing guidelines
Project Lead: Dave Desjardins
Joint Project with ITEA

23. DT&E Committee 2013-2014 Chief Developmental Tester
Activity
Plans for
Status/Plans
Chief Developmental Tester
Review service policies for new role and industry implementations
Propose model for industry interaction
White paper
T&E/SE conference presentations
The Secretary of Defense shall require that each major defense acquisition program be supported by—
‘‘(A) a chief developmental tester; and
‘‘(B) a governmental test agency, serving as lead developmental test and evaluation organization for the program.
Coordinating DT&E activities
Insight into Contractor activities
Oversee T&E activities
Inform government PM about contractor DT&E results
Project Lead:
Joe Manas

24. DT&E Committee Results 2008-2013
Backup
DT&E Committee Results

25. Summary of DT&E Committee Test and Evaluation Efforts
Topic
Activity
DT&E Committee
DoD T&E Policy Study
2006 – 2008: Workshops and Study Report
Improving T&E in the DoD Acquisition Process
Integrated Testing
2008 – 2010: Integrated Test Study
NDIA presentations and tutorial
ITEA journal article
RFP Language for T&E
2010 – 2011: Comments for Update to OSD Guide
Incorporating T&E into DoD Acquisition Contracts
Test Optimization
2012: Statistical Optimization Conference Thread
2013: Scientific Test and Analysis Techniques report/articles
Cyber Testing
Started 2013: Industry recommendations for cyber testing
Chief Developmental Tester
Started 2013: Industry interaction with new role

26. Summary of DT&E Committee Systems Engineering Efforts
SED Committee
Activity
SE Collaboration
Software
2009: SW T&E Summit Recommendations
System of Systems
2010 – 2011: T&E for SoS Workshop and Initiatives
2012: Final Report on Best Practices Model
Performance Measurement
2012: Leading Indicators for T&E workshop
2013: Requirements Verification leading indicators report
Modeling and
Simulation
2011: Effective Use of Modeling and Simulation for T&E
2012: Distributed Model-Based Testing
Reliability
Started 2013: Recommendations for better reliability testing

27. DT&E Committee Current Structure Since 2010
NDIA
Systems Engineering
Division
NDIA
Test and Evaluation
Division
SED
Committee
Collaborative
Projects
T&E
Themes
For
Conferences
NDIA Industrial Committee
on Test and Evaluation
(ICOTE)
NDIA
DT&E Committee
Project Results
DT&E Themes
SE Themes & Issues
T&E Themes & Issues
NDIA Systems Engineering Conference (Q4)
Collaborative Projects
NDIA Test and Evaluation Annual Conference (Q1)
Periodic T&E Committee-Focused Conferences
Cooperation with other NDIA Divisions
DT&E Committee Focus: T&E initiatives aligned with SE, DT&E

28. DoD T&E Policy Study August 2006: DT&E Committee Kickoff Policy Study:
“Improving T&E in the DoD Acquisition Process”
Industry T&E policy recommendations
Workshops:
August 2007
January 2008
Focus Areas:
1. Earlier contractor and tester involvement
2. Integrated DT/OT and DT operational relevance
3. Suitability
April 2008: Report Summarized Results:
10 Findings
15 Recommendations

29. Integrated Testing (CT/DT/OT) Implementation Framework

30. Software Test and Evaluation Software Summit
RFP Language
How Much
Testing is
Enough?
Lifecycle
and
End-to-End
Software T&E
Changing
Paradigms
Training & Competency Model
Policy, Guidance, and Standards
Tools, Automation, Methodologies, Process
2009

31. RFP Language Industry Comments for Update:
Master #
Reviewer
Line #
Comment and Rationale
Recommended Input
A/R/P
Industry Comments for Update:
Incorporating Test and Evaluation
Into Department of Defense
Acquisition Contracts
Recommendations from SW Summit

32. Test and Evaluation for Systems of Systems
2009: “Sleepless Nights”
List of Issues
2010: Workshop
2010: “Sominex”
Resulting Initiatives
2011: Best Practices
Wave Model
2012: Final Report

33. Effective Use of Modeling and Simulation for Test and Evaluation
Joint Meeting in August 2011
Distributed Testing, the Joint Mission Environment Test Capability (JMETC) and the Test and Training Enabling Architecture (TENA)
DoD M&S Community of Interest Data Management Working Group
LVC Architecture Roadmap Implementation (LVCAR‐I) Gateways Effort Applicability to T&E
OSD T&E Working Group
Raytheon Presentation on M&S for T&E
Potential Topics for November AMSWG Meeting
2011

34. Modeling & Simulation for Distributed Testing
Benefits:
Find integration issues earlier
Test to learn in ‘safe’ environment
Protect proprietary information
Facilitate DT to OT transition
Increase performance testing range in operating environments
Support end to end studies throughout the program
Barriers:
Security
Lack of persistent network
Early consideration of technical issues
Perceived value
Disconnect between the communities (M&S and T&E)
Recommendations:
Harmonize the standards for M&S and Test for the life cycle perspective (HLA, TENA, Metadata)
Create a framework for reusing and repurposing M&S through the product model
Establish M&S as part of statistical test design
Determine what tests are conducted to acquire data for model validation.
Fewer test events with better models.
Recommend the use of M&S to do I&T
Recommend establishment of JMETC as a persistent node for industry to engage in MBDI&T
Joint Meeting
August 2012
Joint Track
SE Conference October 2012
2012

35. Education and Training Committee – 2013 Task Summary
Established a Joint Competency Working Group (JCWG)
Includes members from E&T Committee and INCOSE Competency WG
Main objective is to evolve to a globally accepted* and marketed standard competency framework, based on systems engineering effectiveness, that can be used to produce competency models tailored to the needs of the customer organizations.
Developed white paper and proposal to evolve current INCOSE UK Competencies Framework along four paths:
Evolve to an SE role-based competency framework that is extensible, scalable, and tailorable by the customer organization.
Evolve by adding the concept of classes. This is where the competencies achieve their interdependence with each other.
Evolve by including a Professional Class that covers leadership and soft skill competencies.
Evolve by adding a new level of proficiency called the Senior Practitioner to the existing four levels in the INCOSE UK Competencies Framework.
* Approved for release as an INCOSE product.

36. Education and Training Committee - 2014 Task Plan
Proposed 2014 Tasks:
Develop crisp definitions of Knowledge, Skills, Abilities, Behaviors and Experience.
Develop better definitions of the six classes.
Refine definition of the Senior Practitioner level.
Develop better definitions for each version (0.5, 0.75, and 1.0).
Explore how system complexity affects SE Roles.
Develop use cases.
Ensure consistency with INCOSE Certification program and SE Handbook.
Deliverables/Products
Competencies Framework V 0.5
Competencies Framework V 0.75
Competencies Framework V 1.0
Conference Papers and Presentations
Final Report
Schedule / Resources:
Two weeks before INCOSE IW 2014, send draft Version 0.5 to JCWG members.
Conduct working session at IW 2014 to finalize Version 0.5.
Conduct working session at IS 2014 to finalize Version 0.75
Ultimate Goal: Produce Version 1.0 by IW 2015.
Issues / Concerns:
Few outside UK use framework – Need to seek broader buy-in early and throughout development process and show how we can use evolved competencies framework to clarify what a SE is, what they do, and why they are important.
Project may face analysis paralysis – Need to set reasonable POA&M, stick with schedule, discuss, move forward, exhibit strong leadership along the way.

37. EHM Committee – 2013 Task Summary
2013 Tasks Planned
Status
Accomplishments (deliverables, etc.)
- Comments
Integrated Diagnostics Handbook update
In-work
Accomplishments: OSD/ATL has been working on rewrite of Mil-Hdbk EHM participation from May Effort to continue through mid 2014

38. (EHM Committee - 2014 Task Plan
Proposed 2014 Tasks:
working with OSD and rewrite of Integrated Diagnostic handbook
Deliverables/Products
Review/Comments of Integrated Diagnostics Handbook as it progresses
Schedule / Resources
Status/Review/Present at regular EHM meetings:
March/April EHM Meeting
September EHM Meeting
December with SED
No resource issues or support anticipated
Issues / Concerns:
No Risks, Barriers or Concerns specific to EHM; However, reduced defense budgets has had a significant impact on Government, and defense contractor participation in many events

39. ESOH Committee – 2013 Task Summary
Completed 2013 Tasks:
Supplemented the new Defense Acquisition Guidebook Chapter 4, Systems Engineering
Teaming with DoD Acquisition ESOH IPT
“Handbook for Human Systems Integration (HSI) & Environment, Safety, and Occupational Health (ESOH) in Pre- Milestone A Analysis of Alternatives (AoA) and Joint Capabilities Integration and Development System (JCIDS) Activities,” published October 2013
“Guide to ESOH in the Systems Engineering Plan (SEP); Programmatic Environment, Safety, and Occupational Health Evaluation (PESHE); and National Environmental Policy Act (NEPA)/Executive Order (EO) Compliance Schedule,” published October 2013
Coordinated with Aerospace Industrial Association (AIA) on rewrite of National Aerospace Standard 411 (NAS411)
Goal of rewrite: provide detailed implementation instructions to contractors on implementation of the MIL-STD-882E, DoD Standard Practice for System Safety, Task 108, "Hazardous Materials Management Plan"
NAS 411, “Hazardous Materials Management Program,” published 30 September 2013
NAS 411-1, “Hazardous Materials Target List,” published 30 September 2013
Key element: List of hazardous materials categorized per Task 108 as Prohibited or Restricted
Planned the NDIA SE Conference ESOH Track
Combined HSI and ESOH Track and coordinated abstracts with the HSI Committee
Six abstracts to address MRAP Lessons Learned for Rapid and Urgent Program ESOH management, streamlining ESOH documentation initiatives, and updates on DoD Policy

40. ESOH Committee - 2014 Task Plan
Proposed 2014 Tasks:
Focus on improving implementation of current DoD Acquisition SE ESOH policy
Continue work with AIA on NAS list of Tracked materials
Work with DSOC/SOH Council on memo regarding 15 Nov 2013 Army CAE memo on ESOH risk management and reporting
MIL-STD-882E implementation work
Develop mechanism and collect Industry and Government feedback on policy and guidance
Quarterly webinars
Deliverables / Products
List of Tracked Hazardous Materials
Memo co-signed by P&R and DUSD (I&E)
MIL-STD-882E corrections and DIDs
Comments on improving DoD ESOH policy and guidance, and MIL-STD-882E
Webinars
Schedule / Resources
NAS list of Tracked materials – ECD Dec 2014/funding by AIA and DoD participants
Memo regarding 15 Nov 2013 Army CAE memo – ECD Jun 2014/funding by DoD participants
MIL-STD-882E implementation work – TBD
Develop mechanism and collect Industry and Government feedback on policy and guidance – ECD Dec 2014/funding by participants
Quarterly webinars – ECD June 2014/DoD funded
Issues / Concerns:
Agreement between DoD and AIA on Tracked list
Time for staffing and advocacy for proposed P&R and DUSD (I&E) memo
Funding for MIL-STD-882E update development
Time and funding for questionnaire development
OSD support

41. Modeling and Simulation Committee – 2013 Task Summary
Final report on M&S use assessment
Briefing on product given at October 2013 Systems Engineering Conference
Key spreadsheet completed and shared with ODASD/SE/SA liaison
Final report draft in progress – to be completed in mid-December
Committee meetings
Meetings held in February, April, June, and August 2013
Presentations posted on committee web site
Report on essential elements of a System Model
Subcommittee formed in August 2013
Expect final report in July 2014

42. Modeling and Simulation Committee - 2014 Task Plan
Proposed 2014 Tasks:
Complete study to determine essential elements of a “System Model” that evolves over the life cycle (started in 2013)
Provide continuing situational awareness for M&S Committee members (ongoing)
Motivate submittal of abstracts to populate full-length M&S track at 2014 SE Conference
Deliverables/Products
Final report on essential elements of a System Model
Committee meeting presentations posted on committee web site
Schedule / Resources
Final report on essential elements of a System Model – July 2014
Have some volunteers; need to recruit more
Committee meetings in February, April, June, and August 2014
Need organizations to volunteer to host
Issues / Concerns:
Finding volunteers to work on unfunded study efforts is getting much more difficult in current fiscal environment.
Continuing Government restrictions on travel and fees for professional society activities is creating difficulties in maintaining perception of value of society activities to industry participants.

43. Mission Analysis Committee – 2013 Task Summary
Collaborated with ASD R&E/Research on the integration of S&T/IR&D
Industry involvement in the Multidisciplinary University Research Initiative (MURI) process
Increase Industry awareness and use of Defense Innovation Marketplace
Developed draft whitepaper on “The Role of Architecture in the Development Planning Trade Space” with the Architecture Committee
Collaborated with the Government DPWG on Development Planning Activities Models
Best Practices, Techniques, Methodologies, and Tools
Completed all actions from June 2012 NDIA DPWG Workshop, except:
Intellectual Property as a barrier to successful DP
OCI as a barrier to successful DP
Final DPWG Workshop Report not submitted

44. Mission Analysis Committee - 2014 Task Plan
Proposed 2014 Tasks:
Support the Government DPWG and ASD R&E/Research on an as needed basis
Organizational Conflict of Interest (OCI) Task
Collaborate with the Government DPWG to address OCI
Develop a clear and concise understanding of the FAR / DFARS language
Communicate findings across Gov’t/ Industry
Complete the “Role of Architecture in Development Planning” Whitepaper with the Architecture Committee
Deliverables/Products
“Role of Architecture in Development Planning” Whitepaper w/ the Architecture Committee
Final Report on OCI Task findings and recommendations
Schedule / Resources
“Role of Architecture in Development Planning” Whitepaper – April 2014
OCI Findings and Recommendations – June, 2014
OCI Final Report – August, 2014
Issues / Concerns:
Diminishing number of contributing committee members
Developing Planning instantiation across Government and Industry has been largely successful. It is questionable as to the need to continue the Mission Analysis Committee. Resources may be better utilized elsewhere.

45. SE Effectiveness Committee – 2013 Task Summary
Widely disseminated the results of the SE Effectiveness Study through numerous channels
Released detailed survey results report
Discussed follow-on work ideas with DASD(SE)
SE self-assessment tool for programs
“Lightweight” Pre-PSR program assessment
Government presentations
DASD(SE)
USD(AT&L)
DoD SE Forum
Army SE Forum
DoD DPWG
Conference presentations
IEEE Syscon
INCOSE IS2013
NDIA GVSETS
NDIA SE Conference (panel)
Presentations for INCOSE chapters
Finger Lakes chapter
Space Coast chapter
Chicagoland chapter
Corporate presentations
Westinghouse Nuclear
Northrop Grumman Corp.
Harris Corp.
Boeing
Other presentations
MITRE
INCOSE webinar
Podcast for SEI

46. Systems Engineering Effectiveness Committee – 2014 Task Plan
Proposed 2014 Tasks:
Task 1: Develop a means of measuring risk management process value
- Enable the evaluation of risk management in programs
- Enable the comparison of alternate risk management paradigms
- Enable the improvement of risk management processes
Task 2: “Get this* out to the SAEs.”, (awaiting further instructions from DASD (SE) – some possibilities listed below)
- Periodic self-assessment of programs
Pre- Program Support Reviews (PSR) assessment of programs
Task 3: Establish collaboration effort with INCOSE SE Effectiveness Working Group
* The SE Effectiveness Study results (per The Honorary Frank Kendall, USD (AT&L))
Deliverables/Products
Task 1: Presentation at Annual SE Conference
Task 2: Report out at year-end Planning Meeting
Task 3: TBD
Schedule / Resources
Task 1: Oct 2014
Task 2: Dec 2014
Task 3: TBD
Task 1 & 2: Assigned liaison from DASD (SE) office
Issues / Concerns:
Working agreement with INCOSE SE Effectiveness Working Group

47. SoS SE Committee 2013 Task Summary
Committee inputs to TTCP Product addressing SoS at key points in the systems development process
February 2013: Initial presentation of TTCP draft
March 2013: Inputs from committee members on TTCP draft
November 2013: Shared Updated Product, with request for added inputs
December 2013: Inputs coming in now
SoS Webinar Series: SoS Engineering Collaborators Information Exchange (SoSECIE)
14 Webinars, each with between participants
Presentations are posted on SoS SE Website
Committee participation SEI workshop on SoS SW Reuse Patterns
Committee Meetings focused on Issues of Interest
February - TTCP Product Review
April – T-Area-SoS – EU SoS Research Agenda Development Projects
July – Army Victory Architecture (with Architecture Committee)
August – M&S and SoS (with M&S Committee)
Six SoS tracks at SE Conference, most joint with other committees
Drafted annex for systems of systems accepted for revision

48. SoS Committee 2014 Task Plan
Deliverables/Products
Feedback on inputs to TTCP work product
Presentations on SoS SE Committee Area on NDIA Website
Webinars slides posted on DASD SE site
Proposed 2014 Tasks
Input and Feedback from TTCP
Regular committee exchanges
Focus on new and emerging Service SoS Initiatives
Joint meeting with SSE on SoS/SSE
SoS Webinar Series: SoS Engineering Collaborators Information Exchange (SoSECIE)
Schedule / Resources
Bi-Monthly committee exchanges
Bi-Weekly Webinars
Issues / Concerns
None

49. Systems Security Engineering Committee Status and Plans December 2013
Holly Coulter Dunlap, Raytheon
Beth Wilson, Raytheon
Industry Co-Chairs

50. SSE Committee 2013 Status: NEW
Activity
Plans for 2013
Status/Plans
Added Task Apr 2013:
New SSE Committee
Continue work of Systems Assurance Committee to follow up on May 2012 Program Protection Planning (PPP) workshop
Kickoff held June 18, 2013
Committee renamed Systems Security Engineering (SSE) Committee
NDIA SE Conference:
SSE track, joint SSE/SoS track
Progress on 5 priorities identified May 2012
Follow-on Workshop planned for May 2013
Complete
In Process
Cancelled

51. SSE Committee - 2014 Task Plan Projects Working Group
Proposed 2014 Tasks:
PPP Implementation Workshop
Joint meetings with SED Committees
Developmental Test and Evaluation: Connections between PPP and cyber testing guidelines
Systems of Systems: PPP leverage points in the SoS Wave Model
Deliverables/Products
Workshop Recommendations
NDIA SE Conference Progress Briefings
Comments on PPP related guidance
Schedule / Resources
PPP Workshop Apr/May
Joint meetings with SED Committees Jun/Aug
Developmental Test and Evaluation
Systems of Systems
Issues / Concerns:
Industry and government engagement
© 2006 by Carnegie Mellon University

52. Summary of SSE Committee 2014 Plans
Topic
Activity
SSE
PPP Workshop
2014: Follow-on to 2012 Workshop
Focus on Taxonomy and Metrics
May 20-22: MITRE facility in McLean, VA
Industry Inputs
Comments on guideline documents
Inputs into PPP implementation
SED
Systems of Systems
2014: PPP leverage points in the SoS Wave Model
Developmental Test and Evaluation
2014: Cyber testing guidelines connections to Program Protection Planning
Completed
Current
Proposed

53. Systems Security Engineering
2013
Systems Security Engineering
Restart Former Systems Assurance Committee
New Systems Security Engineering Committee
Kick-off June 18th
Track at SE Symposium
Planning follow-on workshop in 2014 on Program Protection Plan

54. NDIA SE Conference 2013 Issue Short Title NDIA SE Conference Paper 1
Taxonomy
16290 – Critical Program Information Test Vector (Geoff “Ninja” Donatelli, Raytheon) 2
Metrics
16185 – Software Assurance and NDAA 2013: Software Code Quality Checking (John Keane DoD VA IPO, Vik Chauhan Deloitte Consulting) 3
Contracts and
Acquisition Strategy
16223 – System Security Engineering and Comprehensive Program Protection (Melinda Reed, OSASD SE) 4
Threat and
Attack Vectors
16051 – Engineering Your Software for Attacks (Bob Martin, Mitre)
16077 – Security Engineering in a Systems of Systems Environment (George Rebovich, Mitre)
16001 – Strategic Cybersecurity Threat Analysis Framework: Know Your Enemy to Defeat Your Enemy (Michele Myauo, Microsoft)
16111 – A Supply Chain Attack Framework to Support DoD Supply Chain Security Risk Management (Dr. John Miller, Mitre) 5
Education
16153 – A Practical Educational Approach to Program Protection Planning (Dr. Don Gelosh, Worcester Polytechnic Institute)