1. 0 0 0 Making Better Best Value Tradeoff Decisions Breakout Session # WC12-F10 Marge Rumbaugh, CPCM, Fellow and Janie Maddox, CPCM, Fellow Tuesday, July 31, 2012 4:00– 5:15

2. 11 1 Review the Tradeoff Process Explain Evaluation Factors Describe Tradeoff Analysis Discuss GAO Report Findings: Best Value Tradeoff Decisions Explore Case Studies: Tradeoff Lessons Learned Topics Discussed

3. 22 2 Used when it is in the Government’s best interest to award to an offeror that does not have the lowest price/cost or highest technical rating. Permits tradeoffs among cost and non-cost evaluation factors. The Tradeoff Process Government must justify why additional technical merit is worth a higher price.

4. 3 The Tradeoff Process Is it worth the extra Price Premium??? PRICE High Low TECHNICAL HighLow Is it worth the Cost Savings??

5. 4 Mandatory Evaluation Factors Price or cost to the government; Quality of the product or service; Past performance; and Small business participation for acquisitions over $650,000. 4

6. 5 Effective Evaluation Factors Agencies tend to focus on the same broad categories: –Technical, –Management –Cost or price, and –Past performance. 5

7. 6 Effective Evaluation Factors The factor must be variable, The variance must be measurable, and The factor must be determinant. 6

8. 7 Effective Evaluation Factors The evaluation factor should provide a reasonable expectation of variance among offerors. Discriminators illustrate the variance among offerors in a quantitative or qualitative manner. 7

9. 8 Evaluation Factor Relative Importance –Significantly more important than, –Approximately equal to, or –Significantly less important than cost or price. The RFP must state if all evaluation factors other than cost or price when combined are: 8

10. 9 Making The Tradeoff Decision The source selection decision must not only identify the differences between proposals, but also their strengths, weaknesses, and risks relative to the stated evaluation factors. 9

11. 10 Tradeoff Analysis Base the source selection decision on the trade-off analysis, comparing the proposal strengths and weaknesses against the RFP’s evaluation criteria. 10

12. 11 Tradeoff Analysis Possible outcomes: –lowest-priced proposal superior based on non-cost factors. –no meaningful distinctions between the non-cost proposals. –lowest-priced proposal not superior based on non-cost factors. 11

13. 12 Source Selection Decision Model Source: U.S. Army Source Selection Guide, June 2001 12

14. 13 GAO Report: Tradeoff & Price Premium DoD paid few price differentials despite best value tradeoff solicitations that emphasized non-cost factors. GAO Report GAO-11-8 October 2010 13

15. 14 GAO Report: Tradeoff Challenges Developing meaningful evaluation factors; Time investment to conduct tradeoff evaluations; and Acquisition staff’s Business judgment. GAO Report GAO-11-8 October 2010 14

16. Case Study: FirstLine Transportation Security Transportation Security Administration (TSA) issued an RFP for airport screening services Period of performance: 12 months base + four 1-year options FPAF Contract type Court of Federal Claims No. 11-375C (Sept 27, 2011) 15

17. Case Study: FirstLine Transportation Security The Government will make the award decision on a Best Value basis. The Government will award to the responsible Offeror whose offer conforming to the solicitation will be most advantageous to the Government, price and other factors considered. 16

18. Case Study: Evaluation Criteria Evaluation Factor Order of ImportanceScoring Method ComplianceMost ImportantPass/fail Management Adjective Screening Services Security Training Transition Past Performance PriceLeast importantFair, reasonable & balanced 17 These Factors Are listed in descending order of importance. Combined are more important than Price

19. Case Study: Rating System Adjectives Management, Screening, Security, & Transition –Outstanding –Good –Acceptable –Unacceptable Past performance –Acceptable –Neutral –Unacceptable 18

20. Case Study: Proposal Ratings Evaluation FactorFirstlineAkal Management ApproachOutstandingGood Screening ServicesOutstandingGood Security TrainingOutstandingGood TransitionOutstandingGood Past PerformanceAcceptable Strengths331 Weaknesses01 Price$1,160,000$1,000,000 19

21. Case Study: Source Selection Decision FirstLine: better proposal –Technical superiority –Less government intervention –Less operational risk –Higher price by 16% Akal: acceptable proposal –Acceptable level of technical competence –Lower price Both proposals fully sufficient in meeting requirements 20

22. Court of Federal Claims Decision In a tradeoff source selection, the relevant question is not if the lowest price proposal will meet the RFP’s technical requirements. The government selected a technically inferior proposal – justified the decision by stating that both offerors were acceptable –noted a significant price difference. 21

23. Court of Federal Claims Decision The technical merits of the competing offers never approached equal. The agency deviated from the RFP’s requirements: –Minimized the importance of non-price factors –Elevated importance of price –Awarded contract based on lowest-price technically acceptable basis not tradeoff 22

24. Court of Federal Claims Decision When selecting a low-price technically inferior proposal in a best-value tradeoff procurement where non-price factors are more important than price, it is not sufficient for the government to simply state that a proposal’s technical superiority is not worth the price premium. 23

25. FirstLine Case Study Lessons Learned Apply evaluation standards consistently –Don’t minimize strengths Document business judgments in tradeoff evaluation. Don’t use Lowest Price Technically Acceptable as a basis for the source selection when the RFP states Tradeoff. 24

26. 25 Case Study: Technology Concepts & Design, Inc. TCDI challenges the weaknesses evaluated in its proposal. The agency didn’t provide either contemporaneous documentation or subsequent explanation supporting its assessment of this weakness in TCDI’s proposal. 25

27. 26 Lesson Learned: Technology Concepts & Design, Inc. GAO ruled the agency didn’t have a reasonable basis for assigning the weakness in TCDI’s proposal. Lesson Learned: When making the tradeoff decision, the agency should consider the proposal’s relative quality and document that determination. 26

28. 27 Case Study: PlanetSpace, Inc. NASA Space Station cargo transportation services The RFP specified two evaluation factors: –price and –mission suitability. Two contracts awarded 27

29. 28 Case Study: PlanetSpace, Inc. The SSA’s source selection statement: –Didn’t provide any quantitative estimate of the price differences between 3 proposals. –Identified a concern with the risks in PlanetSpace’s proposal. 28

30. 29 Case Study: PlanetSpace, Inc. Orbital’s proposal offered superior mission suitability at a higher price. The SSA had to determine which of the two proposals (Orbital Sciences or PlanetSpace) provided the best value. 29

31. 30 Case Study: PlanetSpace Inc. SSA Tradeoff evaluation: –PlanetSpace had a lower price. –SSA not sure PlanetSpace could address challenges (risks) in its management approach. –Typical trade-off analysis not done. 30 Bottom Line: Low likelihood PlanetSpace could perform the contract.

32. 31 Lessons Learned: PlanetSpace, Inc. SSA declaration filed after protest: –The disparity in risk between proposals was great and – the services so critical; –almost no price advantage could justify selecting PlanetSpace’s riskier proposal. Ruling: It was a properly conducted trade-off analysis. 31

33. 32 Lesson Learned: PlanetSpace, Inc. Trade-off analysis is not intended to be a legal or analytical straightjacket. Lesson Learned: Be specific in the source selection documentation the first time. 32

34. 33 Conclusion Identify rationale for trade-offs. Compare proposals focusing on key differences: strengths, weaknesses, risks. Explain benefits offered by the superior proposal & why it is or is not significant enough to warrant any additional cost. Follow the relative importance identified in the RFP. 33

35. 34 Questions? 34