Source Selection Training

Source Selection Training
Evaluation of Mission Capability Welcome to the AF Evaluation of Mission Capability and Proposal Risk in Source Selection Training Session. This is a just-in-time briefing presented for your information as you prepare to evaluate the Mission Capability (technical) volume in a source selection. This training is open to all Source Selection team members. Updates made for AFAC , 28 Jan 2008 March 2008 9/22/2018

Source Selection Organization
SSA SSAC SSET Chair Mission Capability PCAG Cost/Price Contract Advisors Source Selection Evaluation Team This is a wiring diagram of the typical source selection organization. The source selection evaluation team usually consists of all the functions in the blue box. In some cases, some of the boxes may be combined. AFFARS MP5315.3, Informational Guidance (IG) paragraph (1) states, “For PEO acquisitions or source selections greater than $100M, the team structure normally consists of the Source Selection Authority, Source Selection Advisory Council, the Source Selection Evaluation Team and any advisors.” 9/22/2018

Mission Capability Team
Mission Capability Factor Mission Capability Technical Rating Mission Capability Risk Rating Subfactor Subfactor This chart takes a closer look at the Mission Capability team which is the focus of our training. The key points to notice are: -The Mission capability team is responsible for evaluating two separate evaluations of the Mission Capability factor – Mission Capability Technical Rating and Mission Capability Risk Rating. -Both of these evaluations are based on the same Mission Capability subfactors you established in the RFP. You will see that the two evaluations are different but provide a complete picture of the offeror’s proposal. First, how will they meet requirements? Second,what is the risk of their approach to meeting requirements? Do you think they will succeed based on their approach to meet requirements and manage the program? Subfactor Subfactor Subfactor 9/22/2018

Source Selection Process Overview
ASP Competitive Range Briefing Pre-FPR Decision Decision Request for FPR Ratings Released Pre- RFP RFP Release Proposal Receipt Interim Ratings and ENs released EN Responses FPR Receipt Award Debriefings Oral Presentations EXCHANGES CO controls after release of RFP Req Devel Risk Assessment Acq Strategy DRFPs Conferences One-on-One Mtgs Market Research Acquisition Plan Cost/Price Risk exchanges with offerors Discussions (only w/offerors in CR) CO Responses to Questions RFP Amendments All proposals eval Comp range deliberated No revisions allowed Communicate only with those uncertain if in/out of CR Must discuss significant weaknesses deficiencies other aspects to enhance award potential Proposal revisions allowed This is an overview chart for the complete Source Selection process. Depending on the size and complexity of the acquisition, the acquisition team may be required to do an Acquisition Strategy Briefing. Other briefings that may be required are Initial Evaluation Briefing, Pre-FPR or Clearance Briefing, and Decision Briefing. These briefings may be combined briefings to the SSA and SSAC. Acquisitions over $100M will have the following briefings to the SSA and SSAC: Initial Evaluation Briefing (The process ends here if awarding without discussions.) Pre-FPR Briefing unless awarding without discussions Decision Briefing unless awarding without discussions AFFARS MP states that as a minimum, at the initiation of and again at the conclusion of discussions, the SSET through the CO shall indicate, to or discuss with, each offeror in the competitive range the following: Adverse past performance to which the offeror has not had an opportunity to respond, uncertainties, weaknesses, deficiencies, and strengths. This should be accomplished by providing the offeror its own mission capability, cost/price risk (if applicable) and past performance ratings. The final disclosure of ratings to the offeror prior to requesting final proposal revisions shall reflect the results of discussions with the offeror. Offerors eliminated from the competitive range may choose to receive a de-briefing then or wait until after award—they can have only one de-briefing. After award of contract, the other offerors may receive a de-briefing. PAR SSDD Communications for greater understanding leading to CRD Debriefings of successful and unsuccessful offerors. Same rating charts as shown to SSA 9/22/2018

Mission Capability Proposal Evaluation Process
Quick Look Proposals Received Individual Analysis Worksheets Quick Look Evaluation Notices Discussions Subfactor Summaries Competitive Range Briefing Analysis Worksheets Final Proposal Revisions Subfactor Summaries Clearance Pre-FPR Activities This chart covers the steps of the proposal evaluation process focusing on your job as a member of the Mission Capability team. We will cover each step in this process to make sure you understand your roles and responsibilities in the proposal evaluation. We will also make sure you understand how important you are to the process of evaluating proposals and ultimately selecting the best contractor to do the job. Decision Briefing Contract Award SSA Debriefings 9/22/2018

Quick Look Who submitted offers? Is proposal complete?
Ensure page count limitations Have any exceptions to RFP requirements been noted? Obvious showstoppers Do proposals need to be loaded into any evaluation tool? “Quick Look” information will be provided to the SSA The Contracting Office will take a quick look at all proposals, making sure each proposal is complete, deleting excess pages, and making a complete list of offerors including subcontractors. The SSET Chairperson will provide list of offerors to the SSA. 9/22/2018

Individual Analysis Worksheets Individual Analysis Worksheets Proposals Received Quick Look Evaluation Notices Analysis Worksheets Discussions Competitive Range Briefing Subfactor Summaries Subfactor Summaries Clearance Pre-FPR Activities Final Proposal Revisions The next step for the Mission Capability Team is the individual analysis worksheets. Decision Briefing Debriefings Contract Award SSA Decision 9/22/2018

Evaluation Process Evaluator - individual analysis worksheet
Read RFP Read proposals Using analysis worksheets, assess proposal against RFP requirements and evaluation criteria Document how proposal exceeds, meets, or fails to meet the mission capability requirements Document strengths, deficiencies, weaknesses Subfactor Chief - Prepare Subfactor Summary Write Evaluation Notices (EN) for unresolved issues Assign ratings SSET Chair - Prepare Evaluation Briefing Brief results and recommendations to SSA - This chart is simply a narrative discussion of the previous chart. 9/22/2018

Evaluation Steps Read RFP sections L and M
Read Offeror’s Executive Summary: Grasp overarching messages or themes, and general organization of proposal (if Executive Summary was requested) Review all relevant portions of proposal Collaborate with PCAG and cost team members to: Resolve issues Clarify proposal content Review proposal against evaluation criteria in RFP Section M to assess whether proposal: Exceeds Meets … Section M Does Not Clearly Meet Requirement Does Not Meet Here are the steps you will take to evaluate proposals. It is always a good idea to start by re-reading the RFP. You may not have looked at it for awhile, so it is good to once again become familiar with your evaluation criteria before you start evaluating proposals. Keep the RFP handy during the evaluation process. When you get to the proposals, read the summary first and then get down to the details on the first proposal to be evaluated. 9/22/2018

Keys to Developing Initial MC Technical and Risk Ratings
Complete the Evaluation of One Proposal Prior to Moving to the Next One Ensures integrated assessment of proposal Facilitates evaluation against MC subfactor As opposed to comparing proposals to one another More efficient as all evaluators are reviewing the same proposal MC team members can assist each other in finding information or answering questions The technical team should complete evaluation of one proposal at a time. That means each member assigned to evaluate a subfactor would do their individual evaluation, the team chief would lead the team in consensus and accomplish the subfactor summary plus the ENs for that subfactor. 9/22/2018

Keys to Developing Initial MC Technical & Risk Ratings
Evaluate only information contained in the proposal Evaluate proposal on the basis of what we asked for, not what we would like to see Don’t expand the scope of evaluation beyond original intent Evaluate proposals against Evaluation Subfactors in RFP (Sections L and M)--not against one another Ensure consistent evaluation across all proposals You can only evaluate what you read in the proposal. You may have knowledge that the offeror has some other capability or has done something but you can not evaluate based on your knowledge—ONLY evaluate what the offeror has written in his proposal. Mission Capability Team Chief must ensure consistent evaluation on all proposals. You do not compare proposals. You MUST evaluate each one consistently against Section M criteria. 9/22/2018

Mission Capability Technical Evaluation
Individual Evaluator completes analysis for their Subfactor No collaboration with other Mission Capability evaluators Collaboration will be at Subfactor summary level Individual Evaluator evaluates each Mission Capability Subfactor in accordance with Section M of RFP and completes Individual Analysis Worksheet Proposed approach evaluated as: Exceeding Requirement Meeting Requirement Not Clearly Meeting Requirement Failing to Meet Requirement The Mission Capability Team Chief will decide how to organize the team—by subfactor chiefs with certain evaluators evaluating one subfactor or all evaluators may evaluate all subfactors. Each evaluator must document his/her evaluation of the subfactor, draft ENs, and participate with other evaluators in reaching consensus of the overall evaluation and ENs for each subfactor. The Mission Capability Team Chief is responsible for reaching consensus of the evaluators on each of the Mission Capability subfactors. The Mission Capability Team Chief will make sure that the subfactor summary level documentation clearly shows the consensus results and disposition any individual evaluator’s difference with the consensus. The SSET Chairperson should decide which offeror will be evaluated first, second, etc. It is best for the Mission Capability Team to complete the evaluation of one offeror before any one starts the evaluation of the next offeror. 9/22/2018

Evaluate the Exceeds Requirements to determine which meet the Strength definition A strength requires you to document all three aspects of this definition: A significant aspect of an offeror’s proposal that has merit and exceeds specified performance or capability requirements; Advantageous to the government; and Will be included in the contract or is inherent in the offeror’s process Document the Strengths The technical evaluators must review a strength to make sure that it meets all 3 parts of the strength definition. They must be able to explain how it meets each part in the documentation, on the briefing charts and orally to the SSA. 9/22/2018

Evaluate each Failed to Meet Requirement to determine Deficiencies Deficiency: A material failure of a proposal to meet a Government requirement Document the Deficiencies The evaluators must also explain a deficiency. What government requirement did the offeror fail to meet in the proposal? 9/22/2018

Evaluate each item previously marked as Not Clearly Meeting Requirement and determine if it is an Uncertainty Uncertainty: A doubt regarding whether an aspect of the proposal meets a material performance or capability requirement. It requires additional information from the offeror to further explain the proposal before the evaluator can complete his/her review and analysis and should generate the issuance of an EN. Document the Uncertainties AFFARS MP5315.3, paragraph 8.16 defines Uncertainty as follows: “Uncertainty” is a doubt regarding whether an aspect of the proposal meets a material performance or capability requirement. It requires additional information from the offeror to further explain the proposal before the evaluator can complete his/her review and analysis and should generate the issuance of an EN. Information Guidance paragraph 8.16 states that proposals including an uncertainty about meeting a material performance or capability requirement will normally be rated “yellow.” Proposals including an uncertainty about potentially exceed a material performance or capability requirement will normally be rated “green.” Unlike being non-responsive in the case of IFB, in negotiated procurements, teams should address any shortfalls, questions, or uncertainties during discussions and provide the offeror the opportunity to satisfy the government’s requirement. In cases where award without discussions occur an offeror may not have the opportunity to address such issues in order to clearly meet the government requirement. In these cases, the team should carefully weigh the benefit of awarding without discussions over what otherwise might be another competitive proposal if discussions were to be held and proposal revisions accepted. 9/22/2018

Sec M – Mission Capability Technical Evaluation
Mission Capability technical ratings focus on Strengths, Deficiencies, and Uncertainties in the offeror’s proposal for a Mission Capability Subfactor Narrative Assessment shall include: Strength Deficiency Uncertainty Write Evaluation Notice if necessary Verify proposal meets all minimum technical requirements Interface with PCAG and Cost Team AFFARS MP5315.3, section 8, contains the above definitions of strength, deficiency and uncertainty. The definition of deficiency is set forth in FAR The SSET Chairperson should hold interface meetings with the Mission Capability Team Chief, PCAG Chairperson and the Cost Team Lead frequently. Recommend you keep track of action items from these meetings to know when the action has been completed. 9/22/2018

Mission Capability Technical Ratings
Mission Capability Team Chief or Subfactor Chief is responsible for: Combining inputs from all MC evaluators Reaching consensus of the evaluators on each of the Mission Capability Subfactors Making sure that the Subfactor Summary Level documentation clearly shows the consensus results and disposition of any individual evaluator’s difference with the consensus Assigning Color Rating 9/22/2018

Sec M -- Mission Capability
Technical Ratings BLUE-EXCEPTIONAL: Exceeds specified minimum performance or capability requirements in a way beneficial to the government. A proposal must have one or more strengths and no deficiencies to receive a blue. GREEN-ACCEPTABLE: Meets specified minimum performance or capability requirements. A proposal must have no deficiencies to receive a green but may have one or more strengths. YELLOW-MARGINAL: There is doubt regarding whether an aspect of the proposal meets a specified minimum performance or capability requirements, but any such uncertainty is correctable. AFFARS MP , Paragraph ,Table 1 sets forth the Mission Capability Technical Rating colors, ratings and descriptions. MP states that through discussions, the government evaluators should obtain the necessary information from offerors with interim Yellow/Marginal ratings to resolve outstanding issues within the offer. Yellow/Marginal ratings should be rare by the time of the final evaluation. MP also states that if an offeror’s proposal demonstrates a material failure to meet a Government requirement, this is a deficiency in the offeror’s proposal resulting in a Red/Unacceptable rating and the proposal is not awardable. RED-UNACCEPTABLE: Fails to meet specified minimum performance or capability requirements. The proposal has one or more deficiencies and is not awardable. 9/22/2018

Mission Capability Risk Evaluation
Risk is assessed at the Mission Capability Subfactor level Focuses on the Weaknesses associated with an offeror’s proposed approach Assessment considers potential for disruption of schedule, increased cost, degradation of performance, the need for Government oversight, and the likelihood of unsuccessful contract performance When a Strength is identified in a Mission Capability Technical Rating, assess if offeror’s proposed approach would likely cause an associated Weakness which may impact schedule, cost or performance (MP ) You evaluate risk against each mission capability subfactor without “rolling up” the rating. MP , Paragraph states that mission capability risk rating focuses on the weaknesses associated with an offeror’s proposed approach. A risk rating is assigned to each mission capability subfactor. Assessment of a mission capability risk considers potential for disruption of schedule, increased cost, poor performance, the need for increased Government oversight and the likelihood of unsuccessful contract performance. For any weakness identified, the evaluation shall address the offeror’s proposed mitigating and why that approach is or is not acceptable. 9/22/2018

Mission Capability Risk Evaluation
Evaluation Worksheets and Summaries shall document: Weakness: A flaw in the proposal that increases the risk of unsuccessful contract performance. Significant Weakness: A flaw that appreciably increases the risk of unsuccessful contract performance. Deficiency: Combination of significant weaknesses in a proposal that increases the risk of unsuccessful contract performance to an unacceptable level. (FAR ) Risk Mitigation proposed by the offeror. For ANY weakness identified, the Evaluator MUST: Generate ENs for significant weaknesses, weaknesses, and deficiencies Address the offeror’s proposal for mitigation, if available and document why that approach is or is not acceptable You evaluate risk against each subfactor without “rolling up” the rating. Note that the term weakness is used only in connection with the assessment of risk. MP , Paragraph states that mission capability risk rating focuses on the weaknesses associated with an offeror’s proposed approach. A risk rating is assigned to each mission capability subfactor. Assessment of a mission capability risk considers potential for disruption of schedule, increased cost, poor performance, the need for increased Government oversight and the likelihood of unsuccessful contract performance. For any weakness identified, the evaluation shall address the offeror’s proposed mitigating and why that approach is or is not acceptable. 9/22/2018

Mission Capability Risk Ratings
Has little potential to cause disruption of schedule, increased cost, or degradation of performance. Normal contractor effort and normal government monitoring will likely be able to overcome any difficulties. Can potentially cause disruption of schedule, increased cost or degradation of performance. Special contractor emphasis and close government monitoring will likely be able to overcome difficulties. Likely to cause significant disruption of schedule, increased cost or degradation of performance. Extraordinary contractor emphasis and rigorous government monitoring may be able to overcome difficulties. The existence of a significant weakness or combination of weaknesses that is very likely to cause unmitigated disruption of schedule, drastically increased cost or severely degraded performance. Proposals with an unacceptable rating are not awardable. *+ At the option of evaluators/teams, a plus “+” rating may be used as an option when risk is evaluated to be in the upper boundaries but is not high enough to merit the next higher rating. Low* Moderate* High* AFFARS MP , Table 2, Mission Capability Risk Ratings *+A plus rating may be used as an option when risk is evaluated to be in the upper boundaries but not enough to merit the next higher rating. “When assigning the risk rating, teams should endeavor to rate proposals as low, moderate, high, or unacceptable. However, if additional stratification within the Low, Moderate or High Risk ratings is desired, evaluators/teams may (optionally annotate this by adding a plus “+” to the risk rating.” The use of a (+) shall not apply to the Unacceptable risk rating. Mission capability risk can be summarized by one question— “Can the offeror actually perform the work as proposed?” Unacceptable 9/22/2018

Mission Capability Risk Considerations / Examples
Unproven processes May be technically desirable, but could introduce program risk State of the art technical solution May drive increased schedule, cost risk Example: Requirement: Light weight, high strength material Proposed solution: Exotic alloy MC Subfactor rating “Blue” as technical requirement exceeded Proposal Risk rating “High” as material is scarce and difficult to obtain Here are some examples of proposal approaches that could lead to higher risk for the program. Based on what you know about your program and offerors, think about things that might lead to higher, or lower risk. If you performed a risk assessment early in your program planning, the results of that assessment should point to things you might look for or see in proposals. 9/22/2018

Technical & Risk: The Differences
MC Technical MC Risk Focus Does the proposed performance meet or exceed requirements? What are the risks of meeting solicitation requirements if performed as proposed? Issue What is quality of outcome? Are there Advantages, benefits What risks/mitigations are inherent in offeror’s approach? Assess- ment Strengths Meets requirements Uncertainties Deficiencies No weaknesses Weakness Significant weakness Deficiency Mission Capability color rating is comparison of proposed approach against Section M criteria for MC “Does their approach meet/exceed our requirement?” Mission Capability risk is an assessment of the risk of implementing the proposed approach “Will their approach work?” Integrated evaluation of approach, risk mitigation, schedule, and cost “Will their approach result in the need for increased government oversight?” Sources of MC Risk: Under-manning the effort, Incorrect skill mix Poor integration of tasks or processes as reflected in the IMP Poor integration within IMS (i.e., activity relationships aren’t logical) Rating Blue Green Yellow Red Low Moderate High Unacceptable 9/22/2018

MISSION CAPABILITY (MC)
Analysis Worksheet Source Selection: __________________________________________________________ Evaluator: _______Offeror: _________ Factor: Subfactor: MISSION CAPABILITY (MC) Component of Performance or Capability Requirement: What is Offered: Technical Rating How Proposal Exceeds, Meets, or Fails to Meet Performance or Capability Requirements: Strengths (MC): (Start narrative with "Strengths:" Deficiencies (MC) : (Start narrative with "Deficiencies:") Uncertainties (MC): (Areas requiring additional information) Risk Rating) Significant Weakness (MC): (Start narrative with “Significant weakness”) Weaknesses (MC): (Start narrative with "Weaknesses:") Deficiencies*(MC): (Start narrative with “Deficiencies”) Mitigation (MC): (Start narrative with "Mitigation:") Evaluation Notice Required? *A deficiency could be a result of a significant weakness (or combination of weaknesses) that is very likely to cause unmitigated disruption of schedule, drastically increased cost or severely degraded performance. Information Guidance number 70 provides this template. It is a “Best Practice” example but not required. 9/22/2018

Individual Analysis Worksheets Sample Statements
Examples of a “strength” statement Poor: a. Current teaming arrangements exceed minimum requirements b. Proposed comprehensive risk management toolset is a strength Better: a. The offeror exceeds our requirements by signing MOUs and pre-negotiated subcontracts with key players. This is advantageous because it enables a quick start-up after contract award b. All team members (including subs) agree to use same risk tool. This is a significant benefit to the government because it makes the integrated risk assessment more efficient and further mitigates the overall risk to the government. Here are come examples of statements to support strengths. Note whether they meet the individual parts of the definitions we talked about earlier. 9/22/2018

Notices Individual Analysis Worksheets Proposals Received Quick Look Evaluation Notices Analysis Worksheets Discussions Competitive Range Briefing Subfactor Summaries Subfactor Summaries Clearance Pre-FPR Activities Final Proposal Revisions Now we will discuss the Evaluation Notices (ENs). Decision Briefing Debriefings Contract Award SSA Decision 9/22/2018

Evaluation Notices (ENs) Types of Exchanges
Clarifications* As needed with AWOD Adverse past performance, not previously addressed PPI relevance Minor or clerical errors Communications* To determine competitive range Shall Adverse PPI is determining factor in exclusion May - Inclusion or exclusion in Competitive Range is uncertain Enhance Government understanding of proposal Facilitate Government evaluation process Address ambiguities, perceived weaknesses, omissions, mistakes, etc. Negotiations / Discussions Conducted with offerors in the competitive range Goal is to get best value Discuss weaknesses, deficiencies, and other aspects of the proposal to enhance award Discuss efforts above mandatory minimums Conducted either orally or in writing or both Areas may also include issues of compliance with RFP other than evaluation factors As you can see there are three types of Evaluation Notices (ENs). They can be used only in certain specific circumstances. The information you can request will be limited depending on your situation. Always ask your Contracting Officer for help in this area. AWOD: Award Without Discussions Communications should only be used with offerors who are “on the bubble” in terms of their inclusion in the competitive range. Communications are not allowed for the purposes of clarifying proposal characteristics that, if clarified, might allow for award without the conduct of discussions. Proposals with deficiencies (red ratings) may be included in the competitive range. Discussions with these offerors must cover all deficiencies. If an offeror corrects all deficiencies, then that proposal may be awardable. PPI = Past Performance Information *Neither clarifications nor communications allow for an opportunity for proposal revision 9/22/2018

Issuing Evaluation Notices
Used to better understand offeror’s proposal or notify offerors about aspects of their proposal Should be clearly and concisely written with specific references to RFP and proposal documents ENs must identify the type of exchange: Clarification, Communication, or Discussion PCO assists in determining the proper classification ENs must also identify deficiencies Evaluator who initiated EN records final actions, formulates a conclusion and documents the evaluation PCO controls all exchanges with offerors Here are some good “rules-of-thumb” to follow when you consider ENs. If you think you need information to complete your evaluation, document it as a part of your analysis. Make sure this information gets to your subfactor chief and the Contracting Officer. They will ultimately decide whether a particular EN should go out based on approval of the SSA. 9/22/2018

Evaluation Notice (EN)
_____FAR (a) Clarification* Offeror_______________________ _____FAR (b) Communications* Control#______________________ _____FAR (c) Discussions _____Deficiency *Government will not accept proposal revisions as a result of Clarification or Communication exchanges Request for Proposal REFERENCE (Specify Request for Proposal paragraph number, Section M and Section L reference, etc.) GOVERNMENT COMMENT: Factor _________________________________________________________________ Subfactor _________________________________________________________________ PROPOSAL REFERENCE: (Specify offeror’s document, Proposal Volume, paragraph, and page number) SUMMARY: Description of issue in question and specific request for additional/supplemental information needed to clarify or correct the issue. Include references to the solicitation if necessary. EVALUATOR: (Note: The evaluator's name should not be included on the copy sent to the offeror.) OFFEROR RESPONSE: EVALUATOR ASSESSMENT OF OFFEROR RESPONSE: Address impact (including impact on offeror ratings, if any) and evaluate response. MP contains a “Best Practice” example. It states the template is provided as Informational Guidance. Different formats for ENs may be used; however, you must identify the type of EN and if the EN is a result of a deficiency. 9/22/2018

Examples of EN Summary Poor EN: Better EN:
Need clarification on where the risk management process was demonstrated “Current staffing for this task is 21 people. The proposed staffing of 9 people is inadequate.” Better EN: Describe where risk management process was used previously and why that program is analogous to this effort. Describe the process improvements to be implemented in order to achieve the efficiencies in personnel proposed. The key here is to clearly describe what problem you have with the proposal and what you want the offeror to provide to take care of that problem. Do not say the number of people proposed is not correct because we currently have XX people doing the job. 9/22/2018

Individual Analysis Worksheets Proposals Received Quick Look Evaluation Notices Subfactor Summaries Analysis Worksheets Discussions Competitive Range Briefing Subfactor Summaries Subfactor Summaries Clearance Pre-FPR Activities Final Proposal Revisions Subfactor Summaries is the next topic we will discuss. Decision Briefing Debriefings Contract Award SSA Decision 9/22/2018

Subfactor Summary Completed by subfactor chief with team concurrence
Combines inputs from all MC evaluators Compilation of subfactor analysis worksheets Assigns color code for MC technical rating and risk rating Completed before Decision to award without discussions or establish competitive range Request for Final Proposal Revision Final SSA Briefing Subfactor summary charts are shown to offerors As “interim ratings” prior to award To support debriefings The subfactor chief takes the information in all individual analysis worksheets and uses it to assign a color rating to represent how well the contractor meets requirements and a risk rating to represent how the contractor’s approach to meeting requirements might affect performance, cost, and schedule. Your evaluation, and the support for your assessments is the key to the rating the subfactor chief will assign. 9/22/2018

Subfactor Summary ___ Initial Summary ___ Pre-Final Proposal Revision Summary ___ Final Summary Source Selection: Author: Offeror: Factor: Subfactor: Proposal Description: MISSION CAPABILITY (MC) Technical Rating: Strengths Details: Strengths Summary: Deficiencies Details: Deficiencies Summary: Uncertainties (Areas Requiring Additional Info): RISK Rating: Deficiencies Detail*: Deficiencies Summary*: Mitigation Efforts/Weaknesses Details: Mitigation Efforts/Weaknesses Summary: Comments: Reviewed by: *A deficiency could be a result of a significant weakness (or combination of weaknesses) that is very likely to cause unmitigated disruption of schedule, drastically increased cost or severely degraded performance. This Subfactor Summary template is provided as attachment to AFFARS MP You are not required to use this template. 9/22/2018

Individual Analysis Worksheets Proposals Received Quick Look Evaluation Notices Competitive Range Briefing Analysis Worksheets Discussions Decision Briefing Subfactor Summaries Subfactor Summaries Clearance Pre-FPR Activities Final Proposal Revisions The Competitive Range Briefing will be the topic we will discuss next. Decision Briefing Debriefings Contract Award SSA Decision 9/22/2018

Competitive Range Reduction of number of Offerors with whom discussions will be held Each proposal is rated against all evaluation factors SSA establishes the Competitive Range Competitive Range is comprised of all most highly rated proposals, unless further reduced for efficiency Successive Competitive Range Determinations possible MP5315.3, Paragraph , states the SSA shall establish the competitive range, approve entering discussions, and approve release of evaluation notices. Clarification and Communication ENs can be released prior to the competitive range. The SSA may designate the Contracting Officer as the approval authority for release of evaluation notices by so stating in the source selection plan. FAR (c) states that prior to entering into discussions, a competitive range determination must be made by the CO. SSA/SSAC will be briefed and SSA will read and authorize release of the ENs. 9/22/2018

Competitive Range Briefing
May recommend elimination of one or more offerors from the Competitive Range Must provide detail sufficient to support PCO recommendation for SSA approval SSET presentation of Initial Evaluation Results for all Factors Brief all Strengths, Deficiencies, Uncertainties and Weaknesses SSA approval to enter Discussions Release ENs Should be available for review and approval by SSA SSA approval of Interim Ratings for release to Offerors SSA’s approval for Award Without Discussions If awarding without discussions, SSAC chair will provide a source selection recommendation to the SSA Also provide a minority opinion if there is significant disagreement among the SSAC members Even if no offerors are being eliminated the SSA will receive an “Competitive Range Briefing” that details the interim evaluation status of all offerors and clearly identifies all issues requiring issuance of ENs. The SSA must approve the release of ENs before discussions are opened. Examples of those people who would attend the briefing are: SSA, SSAC, SSET (all members) and source selection advisors. Charts—Recommend listing strengths, deficiency, uncertainties, weaknesses etc., in descending order of importance on charts for each subfactor. SSA will decide at the conclusion of this briefing to: a. Award without discussions, or b. Conduct discussions with all offerors or make a competitive range determination eliminating one or more offerors before conducting discussions. Remember your evaluations and written justification in your individual worksheets? Those are the foundation for the recommendation to the SSA. You may have to defend your assessment, so be thorough, be clear, and be ready! Your opportunity to excel with the SSA! 9/22/2018

Individual Analysis Worksheets Proposals Received Quick Look Evaluation Notices Discussions Analysis Worksheets Discussions Competitive Range Briefing Subfactor Summaries Subfactor Summaries Clearance Pre-FPR Activities Final Proposal Revisions The next step is discussions. Decision Briefing Debriefings Contract Award SSA Decision 9/22/2018

Discussions Discussions with Offerors remaining in competitive range
Resolve evaluation notices Discussions for MC Technical and Risk normally covers: Deficiencies Uncertainties Weaknesses SSET will provide actual interim ratings to the offeror Same briefing charts will be provided/briefed to the offeror Face to face discussions and/or telephonic discussions may be necessary Offerors are permitted to update their proposals You will use these revisions as a part of your final evaluation You will have to document any discussions and results for your area Here is what will happen if your team goes into discussions. A few key points: The Contracting Officer is the POC for all discussions with the offerors Your evaluations will go to the offerors: colors, risk ratings, and written justification. You will be explaining these to the offeror at the beginning and again at the end of discussions, so once again make sure your assessment is well documented and based on the evaluation criteria in the RFP and the offeror’s proposal. A proposal with a Red rating in any subfactor could be correctable in discussions. Normally face to face, or as a minimum telephonic, discussions are the best for ensuring any outstanding issues are clearly understood. Offerors will be able to update their proposals, and you will evaluate the changes. Make sure you get your questions answered now. Based on discussions, you will update your analysis worksheets, revise ratings (color and risk) if justified, and provide these revised ratings to the offerors. 9/22/2018

Source Selection Process During Discussion Phase
Discussions must be conducted With each Offeror in the Competitive Range Issue ENs Receive response to ENs from Offeror Evaluate response Follow up may be face to face Discussions or ENs Evaluate responses received Use the process of ENs, Follow-on ENs and Discussions to make sure the two parties have a clear understanding of what the offeror is proposing and what the Government’s problem is with what has been proposed. See AFFARS MP5315.3, IG paragraph (ref 1), for assistance on discussions. Here are some points from that paragraph: Discussions are required for those areas of a proposal that are deficient, where weaknesses exist, or where other aspects of the offeror’s proposal are significant enough to affect the selection decision, and/or where information presented is unclear. These areas make include issues of compliance with the requirements of the RFP other than evaluation factors. Discussions must be sufficiently robust to ensure a complete understanding of the proposal, and may include real time face-to-face dialogue as necessary. Discussions may be conducted orally or in writing or both determined by the nature of the issues to be addressed. Remember oral discussions also have to be documented in writing for the official record. Before concluding discussions teams should consider the release of a “pre-FPR” to include the ratings, the model contracts, and a specific delineation of any outstanding issues. This provides offerors the opportunity to resolve any potential issues prior to release of the FPR, and would logically reduce the likelihood of a substantial post-FPR discussion. 9/22/2018

Individual Analysis Worksheets Proposals Received Quick Look Evaluation Notices Analysis Worksheets Discussions Competitive Range Briefing Subfactor Summaries Clearance Pre-FPR Activities Subfactor Summaries Clearance Pre-FPR Activities Final Proposal Revisions The next area we will discuss in the activities prior to issuance of the request for the Final Proposal Revision (FPR). Decision Briefing Debriefings Contract Award SSA Decision 9/22/2018

Clearance Pre-FPR Briefing
Obtain Contract Clearance from Clearance Approval Authority to request the Final Proposal Revision (MP5315.3, Paragraph 5.6.6) SSA is NOT the Clearance Approval Authority Obtain SSA’s authorization for release of Request for Final Proposal Revisions SSET briefs completed evaluation to the SSA/SSAC This includes your revised ratings as a result of discussions Review of contract documentation including performance metrics AFPEO/CM PEO must make performance based decision prior to FPR release AFFARS (e)(1) requires Contract Clearance for competitive acquisitions by the clearance authority for the SSA to make the decision to award without discussions OR to request final proposal revisions. The Clearance Approval Authority is the same as stated early for Business Clearance and set forth in AFFARS USAF Management and Oversight of Acquisition of Services (MOASP) requires that all services acquisitions contain outcome based objectives and appropriate metrics that ensure timely and accurate assessments of the contractor’s performance. The objectives and over arching metrics should be developed by the requiring activities, addressed in the AP and approved in the ASP, included in the RFP and made a part of the contract. The designated official should review the objectives and metrics for the contractor’s performance as well as making a performance based decision prior to award of the contract. It is best to make this decision prior to request for FPR. Except for major weapon system and space program acquisitions, the Air Force Acquisition Executive has appointed Air Force Program Executive Officer for Services (AFPEO/CM) as the designated official to review all services acquisitions. Therefore, the PEO for AFPEO/CM must make the performance based decision including performance metrics prior to award of the contract. 9/22/2018

Individual Analysis Worksheets Proposals Received Quick Look Evaluation Notices Analysis Worksheets Discussions Competitive Range Briefing Subfactor Summaries Final Proposal Revisions Subfactor Summaries Clearance Pre-FPR Activities Final Proposal Revisions This chart covers the steps of the proposal evaluation process focusing on your job as a member of the Mission Capability team. We will cover each step in this process to make sure you understand your roles and responsibilities in the proposal evaluation. We will also make sure you understand how important you are to the process of evaluating proposals and ultimately selecting the best contractor to do the job. Decision Briefing Debriefings Contract Award SSA Decision 9/22/2018

Request for Final Proposal Revision
Issued at conclusion of discussions Updated ratings shall be disclosed to offerors prior to request for FPR Issued to all offerors in Competitive Range May release & explain/discuss draft to ensure understanding of all issues Establishes common cut-off date Proposal revisions form baseline for Final Evaluation Technical Baseline Terms and Conditions Business Arrangement Gives offeror opportunity to submit a Final Proposal Revision AFFARS MP5315.3, IG paragraph (ref 1): Before concluding discussions teams should consider the release of a “pre-FPR” to include the ratings, the model contracts, and a specific delineation of any outstanding issues. This provides offerors the opportunity to resolve any potential issues prior to release of the FPR, and would logically reduce the likelihood of a substantial post-FPR discussion. AFFARS MP5315.3, IG paragraph (ref 2): Team may use the actual briefing charts that were used to brief the SSA as a method to provide the offeror the results of their ratings at the initiation of discussions and prior to final proposal revision request. All Changes Should be Traceable to Original Proposal 9/22/2018

Final Proposal Evaluation
SSET must evaluate Final Proposal Revision to Mission Capability Technical and Risk Ratings, Past Performance and Cost/Price including Cost/Price Risk if applicable Document Evaluation Reassess ratings including Strengths, Deficiencies, and Weaknesses If an SSAC is used, the SSAC shall, with assistance of SSET chairperson, provide the comparative analysis of offers when the SSA assigns the responsibility for preparing that analysis to the SSAC AFFARS MP5315.3, IG paragraph (ref 2) says the team should jointly evaluate the final proposals using the Rating Team Worksheets, or other similar document. Only one worksheet is complete for each subfactor per offeror (unless the team is evaluating subfactors; in this case, the team should use one sheet for each subfactor per offeror). The “final” evaluation block of the worksheet should be checked. AFFARS MP5315.3, paragraph 6.5 says the SSAC shall, with the assistance of the SSET chairperson, provide the required comparative analysis of offers when the SSA assigns the responsibility for preparing that analysis to the SSAC. 9/22/2018

Individual Analysis Worksheets Proposals Received Quick Look Evaluation Notices Analysis Worksheets Discussions Competitive Range Briefing Subfactor Summaries Subfactor Summaries Clearance Pre-FPR Activities Final Proposal Revisions This chart covers the steps of the proposal evaluation process focusing on your job as a member of the Mission Capability team. We will cover each step in this process to make sure you understand your roles and responsibilities in the proposal evaluation. We will also make sure you understand how important you are to the process of evaluating proposals and ultimately selecting the best contractor to do the job. Decision Briefing Final Evaluation Briefing Debriefings Contract Award SSA Decision 9/22/2018

Decision Briefing Briefing to SSA/SSAC must include the following:
Include all the up front charts from the Competitive Range Briefing Color Ratings for Mission Capability Technical Rating for each Subfactors Mission Capability Risk Rating for each Subfactor Past Performance Rating Cost/Price Cost Risk Rating if applicable Contracting considerations and any exception to T&C’s Summary chart covering all remaining offeror’s ratings Only Final Ratings shown Sufficiently detailed narrative descriptions of each offeror’s Strengths, Deficiencies, Weaknesses, and Past Performance SSAC (if used) or SSET makes a source selection recommendation Summary Chart a. AFFARS MP covers Source Selection Decision (Documentation Requirements) 6.1 A Decision Briefing shall be conducted whenever the SSA is other than the CO and Paragraph 7.9 states that whenever the SSA is other than the CO, a source selection decision briefing is mandatory and shall be included in the source selection file. Note: The decision briefing depicts the final assessment only. While the SSET will brief the resolution of prior deficiencies and weaknesses, the charts contain no indications of ratings assessed at the time of the competitive range determination. b. AFFARS MP5315.3, IG paragraph 7.9 gives information about the content of the briefing. 1. Matrices displaying color ratings for mission capability technical rating subfactors and mission capability risk rating for each subfactor, past performance evaluation, Cost/Price Risk (if required) and cost/price analysis for al offerors 2. Supporting narrative in bullet form characterizing: -All strengths, deficiencies, and weaknesses --Strengths, deficiencies and weaknesses which contributed to the color ratings and risk ratings are expected to include an indication of the potential benefit to, or undesirable impact upon, the government. -Confidence information --Include those positive and negative aspects which affect the performance confidence assessment -To be considered by the SSA regarding the comparison of offerors’ proposals and past performance -Cost or price evaluation including reasonableness and any realism assessment 3. Also cover the following: distinguishing aspects of the acquisition, funding issues, contracting considerations, exceptions to terms and conditions, recap of factors & relative importance, evaluation criteria for each factor/subfactor, summary of offerors’ proposed approaches, SSET’s comments for trade off analysis for the SSA/s consideration in making an integrated assessment of best value, any analyses by the SSAC, and the source selection recommendation of the SSAC if used or the SSET, and any minority opinion. c. AFFARS MP , paragraph covers the requirement for a recommended source to the SSA. Rationale for the recommendation shall be based upon the evaluation criteria from RFP and shall represent the SSET’s or SSAC’s recommendation to the SSA. In the event that there is significant disagreement among the SSAC members regarding the recommendation that should be presented to the SSA, a minority opinion shall be presented providing the SSA sufficient information to fully consider the minority view. 9/22/2018

Factor / subfactor summaries
Documentation Flow Individual analysis worksheets Factor / subfactor summaries Evaluation briefing & proposal analysis report SSDD 9/22/2018

Documentation SSDD – Source Selection Decision Document (SSDD) is the document that reflects the Source Selection Authority’s integrated assessment and selection decision PAR – Proposal Analysis Report (PAR) is a report that fully documents the results of the evaluation of each proposal and the comparative analysis of all proposals within the competitive range. Subfactor/Factor summaries (already covered) Individual Analysis worksheets (already covered) These are the parts of the pyramid on the previous chart. Note that your individual analysis worksheets are the foundation for the entire structure. You play a key role in this source selection and in the final decision. NOTE: As a technical evaluator, you will also be responsible for evaluations to support the Cost/Price Team, especially if the contract type is cost reimbursement. Those evaluations will assess whether the proposed hours support the content and schedule, whether the types/quantities of materials and/or facilities are appropriate, any parametric estimate inputs, and any other areas appropriate for the acquisition. Your evaluation of the proposed costs will be an important input to the Price documentation. 9/22/2018

Required Documentation
Evaluation Evaluation Notices Assessment of Risk Strengths Deficiencies Uncertainties Weaknesses Past Performance Cost/price Analysis Briefing Charts Proposal Analysis Report (PAR) Source Selection Decision Document (SSDD) AFFARS MP5315.3, paragraph 7, AF source selection documents: 7.1. SSP and any revisions 7.2. DRFP if issued along with comments received and government responses thereto 7.3. RFP and any amendments including the Request for Final Proposal Revision. 7.4. Proposals 7.5. Evaluation worksheets and summaries. After each member of the evaluation team has completed his or her review of a proposal, the evaluation must be documented and included in the source selection file. Competitive Range Determination When the SSA is other than the CO, this determination may be documented in a briefing. Briefing information/charts shall be in sufficient detail to support the CO recommendation and included in source selection file. Include any documentation with the Competitive Range Briefing regarding approval to release ENs, or enter into discussions. Evaluation Notices, responses, and Government evaluation Clearance Documentation Clearance documentation in addition to the determination to award without discussions or the final proposal revision request approval shall be included in the contract file. Decision Briefing Whenever the SSAT is other than the CO, a source selection decision briefing is mandatory and shall be included in the source selection file. Proposal Analysis Report (PAR) Simplified Source Selection Report for source selections <$10M at discretion of SSA Source Selection Decision Document (SSDD) 7.13 Source Selection Debriefing Documents 9/22/2018

Records Retention AFFARS MP5315.3, para 4.2.2.3
Maintain source selection evaluation records Once presented to SSA in any form, that evaluative material and any related supporting evaluative material becomes an official record that must be maintained and must not be altered. Can update, revise, or change that evaluation information in subsequent documentation, but the original record must remain distinct Prior to presentation to the SSA, evaluative materials are “working papers” and may be changed/modified by their author as necessary Records Retention. Everyone is responsible to ensure that appropriate documentation of the evaluation is maintained. AFFARS MP5315.3, paragraph reads as follows: “Maintain source selection evaluation records. When evaluation information developed by any member of the source selection team is presented in any form to the SSA, that evaluative material and any related supporting evaluative material becomes an official record that must be maintained and must not be altered. Updates, revisions, or changes to that evaluation information must be captured in subsequent documentation in a way that the original record remains distinct. Evaluative materials are considered working papers prior to their disclosure to the SSA. These working papers may be changed or modified by their author as necessary in order to support the evaluation process.” 9/22/2018

Debriefings Conducted separately for each offeror that requests debrief Strengths, weaknesses, and/or deficiencies. Your written assessments will be presented to the unsuccessful offeror to support the ratings – your final chance to excel! Also released to each of the unsuccessful offerors are the redacted assessments from the successful offeror’s briefing charts. Overall ranking of offeror (if applicable) Ratings of successful offeror and debriefed offeror Reasonable responses to relevant questions After the SSA makes a decision, all offerors are notified. Both successful and unsuccessful offerors can request a debriefing for final feedback on their proposal. You may participate in the debriefings. Your evaluations, assessments, and rationale will be provided to the offerors. You may be called upon to help answer questions. If your evaluations and support are clear, concise, and complete you will be better able to answer questions. Also, the offerors will understand your evaluation, and they may learn lessons for future competitions. 9/22/2018

… and in Conclusion Factor Chief & PCO Subfactor Chiefs Evaluators
Approve ENs, Assessments Review Ratings (Colors, Risks, Price) Draft Briefing Charts Assessment Factor Chief & PCO Review/Approve ENs Assessments Subfactor Chiefs Approve ENs, Assessments Draft Ratings (Colors, Prop. Risk) Develop ENs Review Comments and Draft “Assessments “ Disregard w/disposition Combine w/other comments Modify with rationale Assessments Evaluators Evaluate each Mission Capability Subfactor against the evaluation criteria in Section M of the RFP. Proposed Approach Rated as Follows: Exceeds Requirement Meets Requirement Not Clearly Meets Requirement Fails to Meet Requirement Evaluate the Exceeds Requirements to Determine Strength (IAW Strength Definition) Document Strengths, Deficiencies, Uncertainties Analyze Mission Capability Risk Based Upon Offeror’s Approach Document Weaknesses Document Significant Weakness Address any Risk Mitigation Write Evaluation Notice if necessary Verify Proposal Meets all Minimum Technical Requirements Recommend ENs = Feedback Evaluation Assistance Advisors Evaluation Assistance Evaluation Assistance Evaluation Assistance Evaluation Assistance 9/22/2018

Questions or Feedback Questions? Feedback?
Comments/Recommendations can be posted at: The training modules will be reviewed/updated periodically based on your inputs 9/22/2018

Source Selection Training

Similar presentations

Presentation on theme: "Source Selection Training"— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

Source Selection Training

Similar presentations

Presentation on theme: "Source Selection Training"— Presentation transcript:

Similar presentations

About project

Feedback