1. Analysis of Alternatives: Statute, Policy and Best Practices
10/3/2011
Analysis of Alternatives: Statute, Policy and Best Practices
Dr. Mark Husband
Professor of Systems Engineering and Cost Analysis
; DSN
January 10, 2012

2. Perspective on what an ideal AoA provides Some Best Practices
10/3/2011
Outline
Statute and policy
Perspective on what an ideal AoA provides
Some Best Practices
Example Study Team
10/03/11

3. Policy (5000.02) is being updated to implement law (WSARA)
10/3/2011
Statutes and Policies
Sections 2366a and 2366b of Title 10, USC
NDAA 2006
Operation of the Defense Acquisition System
DoDI December 8, 2008 USD(AT&L)
Weapons Systems Acquisition Reform Act of 2009
Public Law May 22, 2009 NDAA FY09
Joint Capabilities Integration and Development System
CJCSI G March 1, CJCS
Policy ( ) is being updated to implement law (WSARA)
10/03/11

4. A major defense acquisition program must have an AoA*
10/3/2011
2366a and 2366b Certification
“§ 2366a. Major defense acquisition programs: certification required before Milestone A
(a) Certification—A major defense acquisition program may not receive Milestone A approval…. or otherwise be initiated prior to Milestone B approval…until the Milestone Decision Authority certifies…
(4) that an analysis of alternatives has been performed consistent with study guidance developed by the Director of Cost Assessment and Program Evaluation;”
“§ 2366b. Major defense acquisition programs: certification required before Milestone B
(a) Certification—A major defense acquisition program may not receive Milestone B approval…, until the milestone decision authority—
(1) has received a business case analysis and certifies on the basis of the analysis that—
(A) the program is affordable when considering the ability of the Department of Defense to accomplish the program's mission using alternative systems;…
(3) further certifies that— …
(B) the Department of Defense has completed an analysis of alternatives with respect to the program;”
A major defense acquisition program must have an AoA*
Note: U.S. Code, Current through Public Law , enacted July 1, TITLE 10, SUBTITLE A, PART IV, CHAPTER 139
*Excludes current ops rapid acquisitions / MS C and beyond (Also includes MAIS programs)
10/03/11

5. Example documents for 2366b Business Case Analysis
10/3/2011
Example documents for 2366b Business Case Analysis
ARA memo
TRA report and memo
AoA
PDR report
ORD
CDD JROCM
CDD
Class Justification & Approval
ASR
Service Full Funding Memos
SCP
ICE
MNS PB
Example for SDB II MS B Certification: Memo for USD(AT&L) from AT&L/ARA: “Submittal of Business Case Analysis (BCA) and Other Substantiating Information to Support the Section 2366b Certification for the Small Diameter Bomb Increment II (SDB II) Program,” dated Aug 4, 2010.
10/03/11

6. DoDI 5000.02 Policy Requirements
10/3/2011
DoDI Policy Requirements
Materiel
Development
Decision
Engr & Manufact
Update AoA
as Needed
Gap
Analysis
Initial
Capabilities
Document
(ICD)
Materiel Solution
AoA
AoA Guidance Issued
Technology
Milestone A B C
ICD and the AoA study guidance shall guide the AoA and Materiel Solution Analysis Phase activity
The AoA shall focus on identification and analysis of alternatives, measures of effectiveness, cost, schedule, concepts of operations, and overall risk
The AoA shall assess the critical technology elements associated with each proposed materiel solution, including technology maturity, integration risk, manufacturing feasibility, and, where necessary, technology maturation and demonstration needs
The study plan will be coordinated with the MDA and approved by DPA&E prior to the start of the AoA
The final AoA shall be provided to the DPA&E not later than 60 days prior to the DAB or ITAB milestone reviews
Note: The text above is a list of direct extracts from DoDI dated December 8, 2008.
10/03/11

7. DoDI 5000.02 Policy Requirements
10/3/2011
DoDI Policy Requirements
Materiel
Development
Decision
Engr & Manufact
Update AoA
as Needed
Gap
Analysis
Initial
Capabilities
Document
(ICD)
Materiel Solution
AoA
AoA Guidance Issued
Technology
Milestone A B C
The DCAPE shall evaluate the AoA. The assessment, in collaboration with OSD and Joint Staff, shall assess the extent to which the AoA:
a. Illuminated capability advantages and disadvantages;
b. Considered joint operational plans;
c. Examined sufficient feasible alternatives;
d. Discussed key assumptions and variables and sensitivity to changes in these;
e. Calculated costs; and,
f. Assessed the following:
(1) Technology risk and maturity;
(2) Alternative ways to improve the energy efficiency of DoD tactical systems with end items that create a demand for energy, consistent with mission requirements and cost effectiveness; and
(3) Appropriate system training to ensure that effective and efficient training is provided with the system
Note: The text above is a list of direct extracts from DoDI dated December 8, 2008.
10/03/11

8. WSARA “Responsibilities.—
10/3/2011
WSARA
“Responsibilities.—
The Director of Cost Assessment and Program Evaluation shall serve as the principal official within the senior management of the Department of Defense for the following: …
(4) Formulation of study guidance for analyses of alternatives for major defense acquisition programs and performance of such analyses, as directed by the Secretary of Defense”1
“Study Guidance for Analysis of Alternatives.—
The Director of Cost Assessment and Program Evaluation shall take the lead in the development of study guidance for an analysis of alternatives for each joint military requirement for which the Chairman of the Joint Requirements Oversight Council is the validation authority.
In developing the guidance, the Director shall solicit the advice of appropriate officials within the Department of Defense and ensure that the guidance requires, at a minimum—
full consideration of possible trade-offs among cost, schedule, and performance objectives for each alternative considered; and
an assessment of whether or not the joint military requirement can be met in a manner that is consistent with the cost and schedule objectives recommended by the Joint Requirements Oversight Council.”2
1. Para (a), section 101, Title I, Public Law ; 2. ibid., para (d), section 201, Title II
10/03/11

9. What does an ideal AoA Provide?
10/3/2011
What does an ideal AoA Provide?
An ideal AoA provides essential information on the decision space available to make executive decisions
Prioritized, quantified, contextualized capability gaps
Feasible alternatives that are affordable and mitigate the capability gaps
The trades within and across alternatives among cost, schedule, and performance
Transparent analysis of recognized quality
Sensitivity analysis—how robust are the solutions?
AoA provides well-described and understood “options” for the decision-maker
10/03/11

10. Stand-up a Service Executive Steering Body
10/3/2011
AoA Best Practices
Stand-up a Service Executive Steering Body
Good Practice: Service Executive Steering Body is formed and includes key stakeholders (e.g. requirements, acquisition, resources)
Result:
Ensures balanced approach needed for success
Stakeholder integration throughout process mitigates risk of “stovepiped” analysis and solutions
Provides a link to the CAPE-led “Study Advisory Group” which monitors execution of the AoA
Poor Practice: Service Executive Steering Body not formed
Result:
Service more likely to proceed down path unaware of issues that may delay or kill effort
SAG: Study Advisory Group—key stakeholders (SES level), AT&L, ASDs, Service reps, chaired by the CAPE (SES level). They review the guidance and monitor the execution of the AoA.
10/03/11

11. Screen out infeasible candidates early in the effort
10/3/2011
AoA Best Practices
Cost and affordability considered upfront
Good Practice: Cost and affordability boundaries established early in the analysis
Result:
Facilitates exploration of viable trade space
Analytical resources not expended on infeasible solutions
Includes O&S
Includes impact on schedule—how long to build within cost limits
Poor Practice: Cost and affordability not considered or not addressed until end of AoA
Result:
Exploration of viable trade space may be minimized or not accomplished
Key aspects of viable trade space may be missed because analytical resources are applied toward exploring infeasible solutions
Screen out infeasible candidates early in the effort
10/03/11

12. AoA Best Practices Transparent analysis
10/3/2011
AoA Best Practices
Transparent analysis
Good Practice: Site visits to centers of analysis and open analyst interaction
Result:
Facilitates understanding of analysis enabling leadership to focus on the implication of results
Fosters sound analysis which “co-opts” CAPE and AT&L
Service can address OSD concerns before study time and effort invested
Poor Practice: No or limited analyst interaction
Result:
Understanding of analysis impeded
Confidence in analysis diminished
Leadership time diverted to analyst reviews
Re-work more likely
Recent site visits to Army Materiel Systems Analysis Activity proved useful in understanding methodology of on-going AoAs
10/03/11

13. Obtaining information from contractors / allies
10/3/2011
AoA Best Practices
Obtaining information from contractors / allies
Good Practice: Use request for information
Result:
Facilitates discussion of trade space to inform AoA
Faster access to information
May provide data as input to modeling and simulation
Poor Practice: Use request for proposal
Result:
May prejudge AoA
Encumbers process with open competition constraints
Use of contractor information is legal and desirable but must be well-protected to avoid compromise that puts SESB at risk
10/03/11

14. AoA Study Team Structure (example)
10/3/2011
AoA Study Team Structure (example)
The study director establishes the study team to plan and execute the AoA. Study team membership is determined by the needs of the AoA; members with appropriate skills are usually drawn from many organizations. Members often include contractors who provide critical skills and resources. The team focuses on defining alternatives, then assessing and comparing their operational effectiveness, life cycle costs and risks. Organizations who typically contribute members to an AoA study team include:
Operating Command (OC)
• Financial Management/Comptroller (FM)
• Manpower and Personnel (A1)
• Intelligence, Surveillance, and Reconnaissance (A2)
• Air and Space Operations (A3)
• Maintenance and Logistics (A4)
• Plans and Programs (A5)
• Communications and Information (A6)
• Installations and Mission Support (A7)
• Requirements (A5/A8)
• Analysis, Assessments, and Lessons Learned (A9)
• Security (A3)
• Weather (A3/5)
• Engineering (A7)
Implementing Command (IC)
• AFMC/A3/FM
• OAS (AFMC/A9)
• Product Centers
• Laboratories
• Air Logistics Centers (ALCs)
• System Program Offices (SPOs)
Other AF Organizations
• AF/A2
• AF/A5XW
• SAF/AQ/FM
• AF Cost Analysis Agency (AFCAA)
• MAJCOMs
• AF Operational Test & Evaluation Center (AFOTEC)
• AF/A9
• AF Flight Standards Agency (AFFSA)
• Global Cyberspace Integration Center (GCIC)
• Air Force Intelligence, Surveillance, Reconaissance Agency (AFISRA)
• AF Global Weather Center (AFGWC)
Other DOD Organizations
• USA, USN, USMC
• Combatant Commanders (COCOMs)
• Defense Intelligence Angency (DIA)
• Defense Threat Reduction Agency (DTRA)
• Defense Logistics Agency (DLA)
• National Geospatial Intelligence Agency (NGA)
Non-DoD Organizations
• Department of Homeland Security (DHS)
• Department of State (DoS)
• Department of Energy (DoE)
• Department of Interior (DoI)
• National Aeronautics and Space Administration (NASA)
• Contractors (KTRs)
• Federal Aeronautics Administration (FAA)
• Department of Transportation (DoT)
• National Imagery Mapping Agency (NIMA)
Oversight/Advisory Organizations
• OSD-level integrated product teams (IPTs)
• AF Council (AFC)
• AF Requirements and Operational Capabilities Council (AFROCC)
• Technical Review Group (TRG)
• OAS (AF CoE)
• OSD/PA&E
Office of Aerospace Studies:
10/03/11