1 Program Management Systems Committee Joint Industry/Government Meeting August 13, 2008 Service Contracts Subcommittee

2 Subcommittee Charter Problem Statement EVMS is being applied to a variety of services type work scenarios and there is no consistent guidance or policy on how EVMS should be implemented Charter for Subcommittee: To develop and recommend policy, process guidance, and/or training to support the implementation of EVMS on services type work Targeted Exit Criteria: White Paper - “Guidance for Implementing EVMS on Services Work 2

3 Subcommittee – Members Twenty four members Five from Government SEC Treasury, FMS DOL DOI DOD

4 Accomplishments to Date Compiled list of “problems, issues, and/or challenges” with implementing EVMS on services work PMSC members completed a survey in April 2007 EVM is feasible and has value on services work, with some reservations Conducted a Workshop at the IPMC in November 2007 EVM is feasible and has value on services work, with some reservations Conducted a Workshop at PMI-CPM conference in May Demonstrated how EVM could be applied to the Program Management function Initial draft White Paper

5 White Paper EVM is useful for managing services work; however its application should be scaled to balance the benefits of EVM with program risks and the cost of EVM implementation. It is important to seek ways to define objective measures to quantify earned value for services, It may be more beneficial, at least in some cases, to use EVM at the program level rather than at the individual contract level. EVM is compatible with PBSA.

6 What We Know OMB requires EVM for all capital acquisition and other major acquisition programs involving development effort, however this requirement applies to all contract types, without exception. EVMS requirements have been applied to a broad variety of contracts for services, and these requirements frequently present significant management challenges. Most suppliers have policies that require EVMS to be used to manage both government and commercial projects, at least those posing significant risk, regardless of contract type but these policies often exclude T&M/LOE contracts unless the contract requires it. Services in support of Capital programs are often acquired using T&M/LOE contracts and the contracts include EVMS clauses. Additional guidance for both the application and use of EVM to manage services would be welcomed by the program management community.

7 What We Know Contracts for services include performance standards that must be met; if not, the supplier may be subject to payment reductions or reduced profits. Government and Industry are today working more closely than ever to ensure that program cost, schedule and technical objectives are met. Programs falling short of their objectives may risk reductions in funding or even cancellation. The use of EVMS on contract types to which it has not been traditionally applied, i.e., on FFP or T&M contracts; on efforts combining development and services; or on efforts where not only completion criteria but also specific performance standards must be met, requires flexibility that many EVM systems weren’t designed for and typically don’t provide.

8 Why are Service Contracts a Challenge to EVMS application The work is continuous and repetitive in nature; The work is difficult to schedule and budget; The effort is often process or resource oriented, lacking specific completion criteria; and customer priorities frequently change. These characteristics make traditional EVM application difficult or challenging. However, many contracts for services are performance- based and have clear themselves to management using EVM without posing the types of issues to be addressed in this white paper

9 Service Contracts Difficult to Manage Using EVM Staff augmentation – These contracts frequently describe the labor categories, skills/qualifications, and number of hours to be delivered during the contract’s period of performance. However, the customer determines the Statement of Work and schedule, thus making it impossible for the contractor to develop a CWBS, schedule the work, resource load the schedule to develop the budget, identify completion criteria for EV measurement, and analyze performance against the PMB. At best, on this type of effort, the contractor can only develop a resource plan in anticipation of the customer’s needs.

10 Services Difficult to Manage Using EVM System engineering and technical assistance (SETA) – SETA contracts are frequently set up in a fashion similar to staff augmentation jobs, although they do usually contain a SOW rather than a set of labor categories with deliverable hours. Nevertheless, the exact nature of the work that will be required is not known well in advance and thus it is difficult to establish a detailed schedule and performance measurement baseline except, perhaps, in short- term increments. Logistics support/depot maintenance – Issues with these types of contracts include an inability to predict, in advance, the maintenance drivers, as well as to accurately predict the event drivers that determine maintenance requirements, e.g., flight operations. These efforts are characterized by frequently changing priorities and therefore the EVMS used would require either shorter-term planning windows or frequent re-planning to make it useful as a management tool.

11 Services Difficult to Manage Using EVM Managed services, e.g., the operation of a Help Desk. This type of effort is characterized by difficulty in predicting the volume and type of effort that will be required. Further, Service Level Agreements (SLAs) are typically imposed and used in place of EV metrics to manage these efforts.

12 Contract Element Required for EVM A discrete SOW, from which an end product-oriented work breakdown structure can be defined, and a schedule. From these, a time-phased, resource loaded plan for executing the required work within the timeframe specified in the contract can be developed. As stated in the NDIA PMSC EVMS Application Guide: “Where work is discrete, EVM can be effectively employed. Where work is T&M/LOE, performance must be evaluated using other means that are not related to schedule milestones or measurement of progress.” “For contracts issued in support of a program where the supplier does not control and manage the work scope and schedule (such as a staff augmentation contract), EVMS requirements should not be applied to the contract itself but at the program level where scope, schedule and cost are present.” (Source: NDIA PMSC EVMS Application Guide).

13 OFPP's “Guide to Best Practices for Performance- Based Service Contracting” Job Analysis: "involves determining what the agency's needs are, and what kinds of services and outputs are to be provided by the contractor." Job analysis may involve– Organization analysis: involves reviewing the agency's needs and identifying the services and outputs required from the contractor. Work analysis: involves further analyzing the required outputs by breaking down the work into its lowest task level and linking tasks in a logical flow of activities. Performance analysis and standards: assigns a performance requirement to each task, which involves determining how a service can be measured and what performance standards and quality levels apply. Directives analysis: involves screening directives to determine which should be utilized, either in whole or in part.

14 Performance–Based Service Acquisition Guidance Use a WBS to develop the requirements, cost and schedule for the contract Tell contractor what the Government wants not “How To” do the work Define clear measurable performance standards for all the work Develop a Quality Assurance Surveillance Plan Clearly define the method of measuring performance Establish good and poor performance parameters Require re-performance of unsatisfactory work, if feasible Take appropriate action if work is not delivered Past Performance rating Reduction in price or fee

15 Seven Steps To Performance-Based Acquisition 1.Establish the team 2.Decide what problem needs solving 3.Examine private-sector and public sector solutions 4.Develop a PWS or SOO 5.Decide how to measure and manage performance 6.Select the right contractor 7.Manage performance

16 Seven Steps Guide

17 CONTROL ACCOUNT CONTROL ACCOUNT CONTROL ACCOUNT CONTROL ACCOUNT Work Packages CONTRACT WORK BREAKDOWN STRUCTURE LEVEL 1 FIRE CONTROL RADAR TRAINING RECEIVER SYSTEM ANTENNA RECEIVER PHASE SHIFTER RECEIVER CASE POWER SUPPLYAPPLICATION S/W LEVEL 2 LEVEL 3 LEVEL 4 Responsibility Assignment Matrix (PMB Step 1) COMPANYCOMPANY TESTTEST ENGENG MFGMFG DESIGNDESIGN MECHANICAL DESIGN Electrical DESIGN DRAFTING/ CHECKING ORGANIZATIONAL STRUCTURE The control account level is determined by the responsibility assignment matrix (RAM). The RAM is defined by combining the contract WBS with the organizational structure.

18 FAR Part 37 – Service contracting Use performance based acquisition methods to the maximum extent practicable, except for— (i) Architect-engineer services (ii) Construction (iii) Utility services or (iv) Services that are incidental to supply purchases. Use the following order of precedence (Public Law , section 821(a)); (i) A firm-fixed price performance-based contract or task order. (ii) A performance-based contract or task order that is not firm-fixed price. Requires justification (iii) A contract or task order that is not performance-based. Requires justification All PBSA’s require a Quality Assurance Plan

19 T&M and LOE Contracts Not Meant to be Long-Term – FAR 16.6 T&M/LOE contract may only be used when it is not possible at the time of placing the contract to estimate accurately the extent or duration of the work or to anticipate costs with any reasonable degree of confidence. T&M/LOE contract provides no positive profit incentive to the contractor for cost control or labor efficiency. Appropriate Government surveillance of contractor performance is required to give reasonable assurance that efficient methods and effective cost controls are being used. T&M?LOE contract may be used only if— The CO prepares a Determination and Findings that no other contract type is suitable – Approved by the Head of the Contracting Activity

20 Successful Application of EVM to Services Requires Change in Business Practices Government must Fully implement PBSA Clearly define the work Provide WBS Provide Schedule Reduce the number of T&M/LOE contracts Industry must Adopt current EVM Systems to accommodate Services. When necessary, if Government does not, Define the work Develop the WBS Establish the schedule

21 Change Common Approach Control Account for PMO Work Package 1: Project Management Resources: Fixed, Project Manager One charge number, LOE Work Package 2: Project Control Resources: Fixed; Scheduler, Financial Analyst, EV Specialist, Data Entry Clerk One charge number, LOE

22 Project Management Service Program Management Office Project Management Scheduling Finance Project Support Office Project Management “Common Approach”

23 How could we do it?

24 Planning Considerations Isolate the must do work Target a process for improvement Isolate the important high risk work Collect costs to objectively demonstrate correlations (cause & effect) Understand what affects quality Understand where the money is going Manage it, Make it, Fix it, Maintain it, Operate it, Improve it

25 Project Management Service Admin Mgt Technical Direction / Mgt Planning/ Statusing/ Analyzing/ Reporting Monthly Processing Verification &Validation Training CAM /User Support Program Management Office Project Management Operations Support Baseline Support Project Support Office EVMS Improvements PMB Creation PMB Modification Tool & Utility Development Research & Informal Train Training Course Development Conferences & Seminars In-house Company Mandated

26 PM Basis of Estimate (BOE) Administrative Management Based on percentage times the number of FTEs (5-8%) Technical Direction / Management Based on percentage of technical work (7-12%) Planning, Statusing, Analyzing, Reporting Based on Should Costs, Business Processes, # of CAPs, Risk, Scope Stability, EVM Maturity, etc

27 Program Support BOE Operations Number of Control Accounts Business process requirements Data entry: number of activities, resources, charges Status Review: CAP input validation & analysis Variance Review: Corrective action validation and documentation Report frequency, level of detail, format, content Nature of Work EVM Maturity of Operations

28 Industry Response to Acquisition Advisory Panel –March 7, 2007 Recommendation 2: FAR Part 7 and 37 should be modified to include two levels of Performance-based Acquisitions: Transformational and Transactional. OFPP should issue more explicit implementation guidance and create a PBSA “Opportunity Assessment” tool to help agencies identify when they should consider using performance-based acquisition vehicles. MULTI-ASSOCIATION RESPONSE Industry supports the recommendation to create two categories of PBSAs to distinguish transformational from transactional acquisitions. We also support the development of an “Opportunity Assessment” tool for determining when PBSA is appropriate, but are concerned about how the tool will be developed and would caution against the development of a simple check box-type tool. There are too many variables that can determine the appropriateness of a PBSA. For example, while an agency might have a “transformational” requirement, it may not be possible for the agency to baseline their particular measurement and whether the measurement is even realistic and important to the end goals.

29 When should PBSA or EVM be used? PBSA Stand alone contracts - FFP contracts EVMS Separate major acquisition Part of a major acquisition Cost and incentive type contracts