1. 1 Best Value Project Model Kenneth T. Sullivan PhD, MBA & Performance Based Studies Research Group Del E. Webb School of Construction School of Sustainable Engineering and the Built Environment Ira A. Fulton Schools of Engineering Arizona State University www.pbsrg.com

2. W W W. P B S R G. C O M 2 What If You Are Going In For Brain Surgery… Would you?  Find the cheapest surgeon?  After you found the surgeon you wanted to hire, then ask that surgeon if he/she could lower their price a little?  Would you tell them that they should do it faster?  Would you tell them how to perform the surgery / what tools they can use?  Would you tell them that you have a better way of doing the surgery?  Would you tell them which nurses/doctors they can use?  Would you hire other individuals to tell/direct the surgeon how to do the surgery? Would you be nervous if?  The surgeon asked you how you would like him/her to do the surgery?  The surgeon asked how long you would like the surgery to take?  If the surgeon knew you had asked for the wrong procedure, but did it any way, because hey, the customer is always right?

3. W W W. P B S R G. C O M 3 What If You Are Going In For Brain Surgery… Wouldn’t you rather?  Find the best surgeon available?  Let the surgeon ask you the right questions so he/she can gather the right info?  Let the surgeon tell you how long it should take to perform the surgery?  The surgeon let you know what the potential risks are and what he/she is going to do about them, but what could happen even if he/she does everything they can…and all this before he/she starts?  The surgeon pick out the best nurses/doctors to assist him/her?  Let the surgeon tell you what to do after the surgery so it all works out like it is supposed to?  Let the surgeon let you know of some possible alternatives that could be better in the long term? Wouldn’t you feel better if?  The surgeon found something different during the operation that he/she used their expertise and took the best course of action?  The surgeon would just sedate you and then wake you up when it was over?

4. W W W. P B S R G. C O M Worldwide as a leader in Best-Value Systems  Conducting research since 1994  900+ Projects  $4.4 Billion Services & Construction  5% Increase in Vendor profit  98% On-time, On-Budget, Customer satisfaction  PMI, NIGP, IFMA, IPMA  Tests in Netherlands, Botswana/Africa  ASU – investments of over $100M due to BV PBSRG’s Research Results (Performance Based Studies Research Group)

5. W W W. P B S R G. C O M 5 Research Clients General Dynamics University of Minnesota General Services Administration (GSA) Heijmans, Netherlands Ministry of Transportation, Netherlands State of Alaska Brunsfield (Malaysia) University of Alberta State of Oklahoma State of Idaho Idaho Transportation Department State of Oregon US Army Medical Command USAF Logistics Command University of New Mexico University of Idaho Arizona Parks and Recreation US Solar Neogard Arizona State University US Corps of Engineers Arizona Public Service (APS) Salt River Project (SRP) Rochester Utility Boise State University Idaho State Lewis & Clark City of Phoenix, AZ City of Peoria, AZ Hennepin County, MN City of Roseville, MN Olmstead County, MN Fann Environmental EVIT School District Fulbright Program /University of Botswana, Africa US Embassy, Bank of Botswana RMIT, Melbourne Australia Aramark, Canon, Qwest, ISP, Chartwells, AP, Pearson Various Contractors and Consultants

6. W W W. P B S R G. C O M Tested Application Areas Construction & Design/Engineering  Large GC, infrastructure, municipal, laboratory, education, hospital, financial, large specialty, etc.  Small GC, renovation, repair, maintenance, roofing, demolition, etc.  DBB, CMAR, DB, IDIQ, JOC, Low Bid  Development, supply chain IT  Networking, Data Centers, Software (COTS, custom), ERP, etc. Facility Services  Maintenance, industrial moving, custodial, waste management, services, etc. Business, Municipal, & University Services  Dining, Document Management, Multi-Media Rights, Fitness Equipment, Online Education, Bookstores, Material Recycling, Furniture, etc. Health Insurance, Medical Services Manufacturing 6

7. W W W. P B S R G. C O M 7 Working Commission 117 & Journal International Efforts & Partners 5 years 15 tests for infrastructure Two major GCs Fulbright Scholar University of Botswana PIPS tests RMIT Teaching IMT PBSRG platform Tongji University Brunsfield Complete Supply Chain Univ of Alberta

8. W W W. P B S R G. C O M 8 What makes our research message unusual….. Simplistic Uses logic Efficiency: less decision making, less management, and better results (best value and high profits) It is more important for the vendor who does the work to know what to do than it is for client ’ s representative to know what the vendor should do Measurement

9. W W W. P B S R G. C O M 9 Industry Structure High I. Price-Based/Traditional II. Value Based IV. Unstable Market III. Negotiated-Bid Specifications, standards and qualification based Management & Inspection Best Value (Performance and price measurements) Quality control Competition Probability of Performance Low Qualified vendors invited Owner selects vendor Negotiates with vendor Vendor performs Vendor minimizes risk Client minimizes risk

10. W W W. P B S R G. C O M 10 Vendor 1 Vendor 2 Vendor 3 Vendor 4 Performance High Low Risk High Low Performance High Low Risk High Low Impact of Minimum Standards & Requirements Vendor 1 Vendor 2 Vendor 3 Vendor 4

11. W W W. P B S R G. C O M High Low Performance Owners “The lowest possible quality that I want” Vendors “The highest possible value that you will get” Minimum Perception Problems with Traditional Systems High Low Performance Maximum

12. W W W. P B S R G. C O M Initial conditions Final conditions Traditional Management Time Laws D2 D1M&C D3 D1: Client makes decisions on budget, time, and expectation D2: Client consultant/professional makes more decisions to make expectations true D3: Vendors attempt to use the lowest possible price to minimize the risk caused by the decision making of client & consultant/professional M&C: The client attempts to force vendor to make expectations happen

13. W W W. P B S R G. C O M Initial conditions Final conditions Project Management Model Time Laws M1 C1M2 C1: Client Expectations based decisions and various factors – may or may not be “realistic” M1: Measured vendor plan that more accurately describe the initial conditions replaces C1 – converts to a predictive contract M2: RMP/WRR measures deviation & performance to plan M3: Final performance measurement M3

14. W W W. P B S R G. C O M 14 Inefficiency vs Efficiency Micro-Management Performance dictated by technical information Specification is the requirement Inspection by client Client ’ s professional is the expert and has control No performance measurements Increase flow of information Relationships (partnering, deals, give and take) used to solve issues Need more people (inefficient) No accountability Leadership Performance dictated by performance information Specification is only the intent Quality control by contractor Vendor has control Performance measurements Decrease flow of information High performance vendors used to minimize risk Need less people (efficient) Accountability Can you make the transition?

15. W W W. P B S R G. C O M 15 Performance-Based Functions I. Price-Based II. Performance-Based IV. Unstable Market III. Negotiated Competition Performance LowHigh  Treat as a Commodity  Volume Based  No Accountability  Finger Pointing  Management & Inspection  Minimum Standards  Client minimizes risk  Value & Performance  Maximize Profit  Vendor Accountability  Minimized Management & Inspection  Quality Control  Vendor minimizes risk

16. W W W. P B S R G. C O M 16 Best Value Overview Complete business model for organizations & projects A best value selection and management tool (developed and tested over 17 years) It can be applied to any type of system, organization, structure, procurement, project, or need Best Value is not just a procurement method. It is a selection and management tool that can be applied in:  Business Services (IT, dining, consultants, equipment, doc mgmt, insurance, etc.)  Facility Services (maintenance, roofing, janitorial, landscaping, supplies, etc.)  Design, bid, build (DBB), Design build (DB), Construction manager at risk (CMAR)  A/E & Design, Job Order Contracting (JOC), Indefinite Delivery Indefinite Quantity BV is not a computer software package, but rather a combination of IMT principles that allows a client to make an informed decision

17. W W W. P B S R G. C O M What does the Best Value Model do? Makes things simple (measurement, dominant information) Minimizes the fuel of bureaucracy (decision making, non-dominant information, management, control, and direction) Creates transparency Allows organizations/vendors to be highly efficient and successful Proposes that to accurately identify what “is” and then to have a plan to efficiently meet the needs will minimize risk 17

18. W W W. P B S R G. C O M 18 What is the model? Identify the expert with as little effort as possible, using measurement and differential Transfer risk and control to the expert through preplanning and risk minimization, focusing on risk that are not controlled Hire the expert Use alignment, planning, & measurement in place of management, control, and direction Create a performance information environment to drive accountability and change Proactive vs. Reactive Supply chain (us mentality) Logic vs. Experience Predictable vs. Chance

19. W W W. P B S R G. C O M 19 Best Value System Identification of Potential Best-Value Pre Planning and Risk Management Measurement of Deviation from the Expectation PHASE 1 PHASE 2 PHASE 3

20. W W W. P B S R G. C O M 20 Best Value System Identification of Potential Best-Value Pre Planning and Risk Management Measurement of Deviation from the Expectation PHASE 1 PHASE 2 PHASE 3

21. W W W. P B S R G. C O M What are we trying to accomplish? Scenario 1 Scenario 2 Question: If Purchasing wants to buy a “green circle”, in which scenario is hiring the right “green circle” easiest to justify?

22. W W W. P B S R G. C O M 22 Filter 1 Past Performance Information Filter 2 Current Capability Filter 4 Prioritization (Identify Best Value) Filter 5 Cost Reasonableness & Dominance Check Filter 6 Pre- Planning & Risk Min Time Quality of Vendors Filter 3 Interview Key Personnel Award High Low BV Process Measurement of Risk & Performance During the Contract

23. W W W. P B S R G. C O M Evaluation Criteria Past Performance Information (not rated) [Scope Plan – if needed(rated)] “Technical” Risk Plan (rated) “Non-Technical” Risk Plan (rated) Value Added Plan (rated) [Milestone Schedule (not rated/not weighted)] Cost / Financial Proposal (not rated) Interviews (rated)

24. W W W. P B S R G. C O M Weighting Client must be comfortable General Guideline  30% Price (but always have dominance check!!!!!)  70% Performance Performance Criteria (typical order of importance)  Interview  Non-Technical Risk Plan  About equal  Technical Risk Plan  VA  PPI 24

25. W W W. P B S R G. C O M Submitting a proposal Identify best person / team via measurement Get in a room, plan it out Identify & prioritize risk w/ solutions What else does the client need or could be better Cost it out / financial proposal Then write up your proposal 25

26. W W W. P B S R G. C O M 26 Filter 1 Past Performance Information Filter 2 Current Capability Filter 4 Prioritization (Identify Best Value) Filter 5 Cost Reasonableness & Dominance Check Filter 6 Pre- Planning & Risk Min Time Quality of Vendors Filter 3 Interview Key Personnel Award High Low Measurement of Risk & Performance During the Contract Filter 1 – Past Performance Information

27. W W W. P B S R G. C O M Critical Individuals Past Performance Information will be collected on all critical team components, which includes: Components  The Firm (Company/Firm)  Boss or Money Decider (if service) (Individual)  Project Manager (Individual)  Site Manager (Individual)  Critical Sub-Parties (Firms or Individuals) 27

28. W W W. P B S R G. C O M 28 Survey Form

29. W W W. P B S R G. C O M 29 Filter 1 Past Performance Information Filter 2 Current Capability Filter 4 Prioritization (Identify Best Value) Filter 5 Cost Reasonableness & Dominance Check Filter 6 Pre- Planning & Risk Min Time Quality of Vendors Filter 3 Interview Key Personnel Award High Low Measurement of Risk & Performance During the Contract Filter 2 – Current Information

30. W W W. P B S R G. C O M Filter 2 Components Technical Risk Plan Non-Technical Risk Plan Value Added Schedule (not weighted, not rated) Cost / Financial Proposal 30

31. W W W. P B S R G. C O M 31 Technical Risk Plan TECHNICAL RISK – Project risk that other vendors have due to lack of experience and expertise (capability is more important than project detail) PURPOSE  To identify vendor’s technical expertise and risk of other vendors caused by lack of capability  Differentiate vendor based on dominant expertise Length: 2 page maximum

32. W W W. P B S R G. C O M 32 Non-Technical Risk Plan Non-technical risk: any non-technical risk by other participants which could cause time or cost deviation Purpose: capability of vendor to think in the client’s best interest; vendor understand that this is the most important risk; vendor take charge of the project; vendor minimizes transactions of the project; vendor becomes proactive instead of reactive; vendor preplans activities that they do not control Length: 2 page maximum

33. W W W. P B S R G. C O M 33 Value Added Value added: Difference from average vendor in vendor’s capability to add value to the project; what can help project have more value; beyond the scope of the project “technical” requirements Understand need better than client can specify Adds value or minimize cost ; increases value of delivered service for client Length: 2 pages maximum

34. W W W. P B S R G. C O M Milestone Schedule The milestone schedule should identify key action steps and milestone dates for delivery of the service or transition.  Milestone schedule for all phases needed (1-page)  Anonymous  No template – follow your own preference 34

35. W W W. P B S R G. C O M 35 Format of the Plans In order to minimize any bias, the Plans must NOT contain any names that can be used to identify who the proponent is (such as proponent name, personnel names, project names, etc). A template is provided and must be used. Proponents are NOT allowed to re-create, re-format, or modify the template. The plans should not contain marketing material. The Technical Risk Plan must NOT exceed 2 pages. The Non-Technical Risk Plan must NOT exceed 2 pages. The Value Added Options must NOT exceed 2 pages The Schedule most NOT exceed 1 page The Pricing must be submitted as to the template

36. W W W. P B S R G. C O M 36 Plan 1  We will use our 20 years of experience in working with mechanical systems to minimize the risk of the heating and cooling system design. Plan 2  We have identified the design of the heat/cooling system as a risk. It has not been used before in the area. Will ensure that the system performance and installation is verified in the pre-award period.  We have bid using best rated mechanical contractor in the area (rated at 9.8 out of 10.0, next best rated 9.1)  Mechanical contractor identified modifications to the design to improve output and sustainability of the system with the following impacts (mechanical system cost minimized by 15% - see VA#1)  Mechanical system provided by one manufacturer, and manufacturer will be commissioned by the manufacturer, contractor, and general contractor will take full responsibility of commissioning the system Example of Solutions Risk: Design of Heating/Cooling System Type: Technical Risk

37. W W W. P B S R G. C O M 37 Plan 1  We will work with the user to minimize the impact of noise from demolition. Plan 2  We have planned to demolition during off hours and weekends. This will have a slight impact on our cost (less than 1%), but the impact to customer satisfaction justifies this.  We will also install rubber sheets on the floors to diminish noise and vibrations.  Both solutions can be performed within your budget.  Both solutions have been used on multiple previous projects w/ high levels of customer satisfaction (9.4/10). Example of Solutions Risk: Noise from Demolition Type: Technical Risk

38. W W W. P B S R G. C O M 38 Plan 1  Coordination with [water company] is critical. We will coordinate and plan with [water company] as soon as the award is made to make sure that we get water to the site to irrigate the fields. Plan 2  We will coordinate and schedule the water with [water company]. However, based on past experience there is a high risk they will not meet the schedule (the water company does not meet schedule over 90% of the time).  We will have temporary waterlines setup and ready to connect to the nearby fire hydrant to irrigate until [water company] is ready.  We will also have water trucks on-site if there is problems with connecting the lines. Example of Solutions Risk: Getting water to the site Type: Technical Risk

39. W W W. P B S R G. C O M 39 Plan 1  We will develop various alternative design solutions for different storm events. We will then evaluate the effect of potential solutions, and then decide which one is the best. Plan 2  Implementation of a 50-year drainage system instead of a 100-year system could save $1 million in project costs and reduce the conveyance capacity requirement by approximately 30%, which will reduce maintenance costs and right-of-way requirements. Example of Solutions Risk: Project Drainage Type: Technical Risk

40. W W W. P B S R G. C O M 40 Example of Solutions Risk: Concrete Escalation Type: Non-Technical Plan 1  The owner can be assured all risks associated with material escalations will be eliminated because we offer the benefit of an experienced project team that includes the most detailed, prequalified and extensive list of subcontractors and suppliers, from around the world. Plan 2  The cost of concrete has been rising drastically. Since this project requires a substantial amount of concrete, cost is a risk. To minimize this risk, we have secured and signed a contract with a local concrete manufacturer to prevent any increase in cost during the duration of this project.

41. W W W. P B S R G. C O M 41 Example of Solutions Risk: Loss of Radio Flagship in Major Market Type: Non-Technical Plan 1  We will work very hard to maintain excellent affiliate relationships. If we lose a radio station (e.g. it changes its format) we will move quickly to replace the lost station. If we cannot quickly replace a flagship station, we can be very creative and could even consider purchasing all local inventory from a new flagship station. Plan 2  In the past 10 yrs, on over 50 accounts, 7 radio stations format changes have occurred. The following solution is optimal.  We own and will maintain two radio contracts covering the area, where signals can be switched if required. The flagship station will be the station with the stronger signal and greater coverage.  If a station is lost we will have a equal replacement within 2 months. If within two months a replacement is not contracted we will purchase inventory from another station or discount the cost of an inventory purchase and add it to our payments to the client.

42. W W W. P B S R G. C O M 42 Plan 1  Our internal food safety standards are recognized as being far more stringent than government regulatory requirements. In the unlikely event of a food-borne illness, our strong relationships with local, state, and national health agencies will ensure and 24-hour response. Plan 2  If a food safety issue arises, vendor will effectively minimize the client’s risk of exposure by:  1) Vendor’s system will issue a safety alert and related directives to 10,000+ units and all ASU email accounts in less than 15 minutes.  2) The vendor will place a lock within in its foodservices purchasing system on any food with risk so it cannot be purchased,  3) The vendor will remove all potentially harmful products within the first hour of notice.  4) The vendor will identify as many purchasers as possible through credit receipt names and the client system to notify them individually. Warnings will be placed around campus within two hours of discovery. Example of Solutions Risk: Safe Food Supply/Food Born Illness Type: Non-Technical

43. W W W. P B S R G. C O M Risk that is not identified by client professional, competing vendors, and expert contractor will be identified and solved within 3 days maximum (unless longer time is justified and requested) Contractor shall immediately notify the designer and owner the same day as discovery of potential cost and time impact Contractor shall find best possible options to minimize risk, with accompanying cost and time All cost and time figures will be competed among the best performers, and listed with solutions Contractor shall propose the best solution with justification and present to client representative If client representative approves, no action required, as contractor will create change order work, justification, pricing, and proposal within the specified time, and proceed to accomplish the work If client objects, contractor will work around issue and document cost and time deviation on a weekly basis Example of Solutions Risk: Unidentified Risk Type: Non-Technical

44. W W W. P B S R G. C O M Example of Value Added

45. W W W. P B S R G. C O M 45 Example: Value Added Items Reroofing this building will not stop all water leaks. The majority of the leaks are caused by cracks in the parapet walls, broken/missing glass, and poor caulking. For an additional $20K and 3 weeks in schedule we can replace and repair all of these items.

46. W W W. P B S R G. C O M 46 Plans Summary The All Plans are 2 pages maximum each The Plans must NOT contain any names. No marketing / No technical information Avoid general risks / solutions Identify value added options (and explain why) The Plans becomes part of the final contract. The Plans provide a high performing vendor an opportunity to prove their expertise & prove they are not a commodity. If nobody can clearly differentiate themselves in the Plans, the prioritization will be based on other factors.

47. W W W. P B S R G. C O M 47 Filter 1 Past Performance Information Filter 2 Current Capability Filter 4 Prioritization (Identify Best Value) Filter 5 Cost Reasonableness & Dominance Check Filter 6 Pre- Planning & Risk Min Time Quality of Vendors Filter 3 Interview Key Personnel Award High Low Measurement of Risk & Performance During the Contract Filter 3 – Interviews

48. W W W. P B S R G. C O M 48 Interview Format The client will actually “interview” each critical individual. This is not a “presentation” All individuals will be interviewed separately. The client will not allow group interviews with the proponents team. The client will not allow any other individuals to be present during interviews Generally, interview times will last about 15-25 minutes per individual A standard set of questions will be generated and asked to each individual. The client has the option to clarify any answers (or ask additional questions to clarify an answer). No substitutions will be allowed. Individuals listed on Proposal Form must be present. If a team member is not present for the interview, they will jeopardize the teams competitiveness.

49. W W W. P B S R G. C O M Primary Questions 1. Why were you selected for this project? 2. How many similar projects have you worked on? Individually and as a Team? 3. Describe a similar project you have developed/worked on to the current project. 4. What is different about this project from other projects that you have worked for? 5. Draw out the process for this project by major milestone activities. 1.Identify, prioritize, and how you will minimize the risks of this project. 2.What risks don’t you control? How will you minimize those risks? 3.What do you need from the client and when do you need it? 6. How are you going to measure your performance during the project? 7. What value do you bring to the project in terms of differences based on dollars, quality, expertise, or time?

50. W W W. P B S R G. C O M 50 Filter 1 Past Performance Information Filter 2 Current Capability Filter 4 Prioritization (Identify Best Value) Filter 5 Cost Reasonableness & Dominance Check Filter 6 Pre- Planning & Risk Min Time Quality of Vendors Filter 3 Interview Key Personnel Award High Low Measurement of Risk & Performance During the Contract Filter 4 – Identify the Potential Best Value

51. W W W. P B S R G. C O M Example – Assessment is based on actuals

52. W W W. P B S R G. C O M 52 Filter 1 Past Performance Information Filter 2 Current Capability Filter 4 Prioritization (Identify Best Value) Filter 5 Cost Reasonableness & Dominance Check Filter 6 Pre- Planning & Risk Min Time Quality of Vendors Filter 3 Interview Key Personnel Award High Low Measurement of Risk & Performance During the Contract Filter 5 – Dominance Check w/ Cost Reasonableness

53. W W W. P B S R G. C O M 53 Dominance Check & Cost Reasonableness Best-Value is the lowest price Best-Value is within X% of next highest ranked firm Best-Value can be justified based on other factors Best-Value is within budget Yes No Yes Best Value Prioritization Best Value Prioritization Yes No Go with Alternate Proposal or Cancel Proceed to Pre-Award Yes No Yes No Proceed to highest ranked proposal within budget Dominance Check Also

54. W W W. P B S R G. C O M Dominance Check (on price) If cost of best value is over [10%] above the next submitted best value, dominant information must be provided to show that they are the best value, otherwise the next best value is the best value If cost is within [10%] of the next best value, they are the best value

55. W W W. P B S R G. C O M 55 Filter 1 Past Performance Information Filter 2 Current Capability Filter 4 Prioritization (Identify Best Value) Filter 5 Cost Reasonableness & Dominance Check Filter 6 Pre- Planning & Risk Min Time Quality of Vendors Filter 3 Interview Key Personnel Award High Low Measurement of Risk & Performance During the Contract Filter 6 – Pre-Award Pre-Planning and Risk Minimization

56. W W W. P B S R G. C O M 56 Best Value System Identification of Potential Best-Value Pre Planning and Risk Management Measurement of Deviation from the Expectation PHASE 1 PHASE 2 PHASE 3

57. W W W. P B S R G. C O M 57 Filter 1 Past Performance Information Filter 2 Current Capability Filter 4 Prioritization (Identify Best Value) Filter 5 Cost Reasonableness Filter 6 Pre- Planning & Risk Min Time Quality of Vendors Filter 3 Interview Key Personnel Award High Low BV Process Measurement of Risk & Performance During the Contract Non-Detailed Detailed Non-Detailed

58. W W W. P B S R G. C O M 58 Pre Award Period What is it / Why is it important Period of time allotted to potential best value vendor (aka the Expert) to:  Present their project/service plan  Set a plan for its delivery / clarify that your proposal is accurate  Identify the risks and issues that could cause the plan to deviate  Identify what you don’t know and when you will know it and how the plan could change based upon what you discover  Set plans to minimize those risks from occurring  Address all the concerns and risks of the client

59. W W W. P B S R G. C O M 59 Pre Award Period What is it / Why is it important Period of time allotted to potential best value vendor (aka the Expert) to:  Know how they are being successful and adding value (measurement)  What metrics you will use and how you will report them  What is the current baseline condition we are comparing against  Identify what you need from the client and have a plan for getting it  Have completely aligned expectations between all parties so everyone knows what is going to transpire and what they are supposed to do

60. W W W. P B S R G. C O M 60 Pre Award Period What is it / Why is it important Period of time allotted to potential best value vendor (aka the Expert) to:  Coordinate the schedule  Write the contract in terms of performance information  Get contract signed

61. W W W. P B S R G. C O M Pre Award Period Start Very High Level Cost Verification Included in Proposal Excluded from Proposal Major Assumptions Major Client Risks/Concerns High Level Project Work Plan Client Risks/Concerns PA Schedule Uncontrollable Risks Response to all risks Roles and Responsibilities Value Added Ideas Coordination Review Functionality Mid Level Performance Reports / Metrics Additional Documentation Technical Details Project Schedule High level demos PA Document End

62. W W W. P B S R G. C O M 62 Pre Award Document (RM Plan) The PA Document should contain the following: 1.Scope  Clear and Detailed Project Scope (what is and what is not included) – Set Baseline Expectation 2.Risks  A list of Risks vendor does not control with plans to minimize  Identified Risks List  A list of all client/user risks with plans to minimize & all vendor’s risks 3.Milestone schedule 4.Coordination 5.Client Action Item List 6.Weekly Risk Report 7.Performance Metrics 8.Fee 9.Other: Agreed to Value Adding Options, Original Plans, Interview Minutes, etc…

63. W W W. P B S R G. C O M 63 Pre Award Summary Meeting Is not a “Q&A” meeting  All issues resolved  All coordination complete  All risks that are not in vendors control have been identified  All value added options have been addressed PA Meeting is to summarize all of the coordination that has been complete and walk through the PA Document Upon successful completion of the PA Meeting, the client will make the award

64. W W W. P B S R G. C O M 64 Filter 1 Past Performance Information Filter 2 Current Capability Filter 4 Prioritization (Identify Best Value) Filter 5 Cost Reasonableness & Dominance Check Filter 6 Pre- Planning & Risk Min Time Quality of Vendors Filter 3 Interview Key Personnel Award High Low Measurement of Risk & Performance During the Contract AWARD

65. W W W. P B S R G. C O M 65 Best Value System Identification of Potential Best-Value Pre Planning and Risk Management Measurement of Deviation from the Expectation PHASE 1 PHASE 2 PHASE 3

66. W W W. P B S R G. C O M 66 Filter 1 Past Performance Information Filter 2 Current Capability Filter 4 Prioritization (Identify Best Value) Filter 5 Cost Reasonableness & Dominance Check Filter 6 Pre- Planning & Risk Min Time Quality of Vendors Filter 3 Interview Key Personnel Award High Low Measurement of Risk & Performance During the Contract Measure Risk & Performance Measurement of Deviation from the Expectation Management by Risk Minimization

67. W W W. P B S R G. C O M 67 Weekly Reporting System Excel Spreadsheet that tracks only unforeseen risks on a project Client will setup and send to vendor once Award/NTP issued Vendor must submit the report every week (Friday). The final project rating will be impacted by the accuracy and timely submittal of the WRS

68. W W W. P B S R G. C O M 68 Unforeseen Risks PERFORMANCE SUMMARY Vendor Performance Client Performance Individual Performance Project Performance RISK MANAGEMENT PLAN Risk Risk Minimization Schedule WEEKLY REPORT Risk Unforeseen Risks METRICS Time linked Financial Operational/Client Satisfac. Environmental Measurement of Deviation from the Expectation Management by Risk Minimization

69. W W W. P B S R G. C O M 69 Filter 1 Past Performance Information Filter 2 Current Capability Filter 4 Prioritization (Identify Best Value) Filter 5 Cost Reasonableness & Dominance Check Filter 6 Pre- Planning & Risk Min Time Quality of Vendors Filter 3 Interview Key Personnel Award High Low Measurement of Risk & Performance During the Contract Post Project Rating Updating of PPI

70. Best Value DB Process Balmoral Project 1. RFP Submittals PPI Design Firm – Lead Architect – Mechanical Engineer Contractor – Project Manager – Site Superintendent Technical Capability / Risk Risk Plan Value Added Price $ Other UofA Req 2. Interviews Design Firm Lead Architect Design Firm Mechanical Eng Contractor PM Contractor Site Superintendent Shortlist (if necessary) 3. Selection Single Proponent 4. Pre Award Risk Management Plan Rendering 5. Award 6. Weekly Risk Reporting 7. Performance Measurement Close Out

71. W W W. P B S R G. C O M Where we need your help Critical personnel?  Lead Designer  Mechanical Engineer  GC Project Manager  GC Site Superintendent  Mechanical Sub  Others? For interview, do we need to speak with the cost estimator? For the pre-award, having a partial rendering, how much will be needed? How long should we give the designers?  2 weeks?, 1 week? 3 weeks? 71

72. W W W. P B S R G. C O M Best Value Vendor Characteristics Preplans and Minimizes Risk on Each Initiative  Has a plan and knows the risks to the plan  Is transparent  Communicates clearly  Asks good questions, knows what they don’t know Measures Performance and drives accountability Uses Dominant Information to Differentiate themselves/show value added Educates the Client and helps the client be a better client Educates themselves and has a continually enhanced vision Holds themselves and the client accountable Their plan is aligned so that when they win, the client automatically wins 72

73. W W W. P B S R G. C O M Best Value Client Characteristics Ensures their needs and concerns are known by the vendor Ensures the vendor has a plan that addresses each need/concern/risk Is a facilitator to the vendors development of the plan(s) Enforces the best value structure  Weekly risk reporting is being done  Each risk is given a client satisfaction rating  Measurements by vendor are being done  Do not be pulled into making decisions you do not need to make Educates themselves and the vendor  Avoids reversion  Transfers risk and control Holds the vendor and themselves accountable Ensures the plan is aligned so they win and the vendor wins 73

74. W W W. P B S R G. C O M 74 City of Peoria, AZ Results Number of Best-Value Procurements: 65 Estimated Budget: $586 Million  (DB/CMAR/JOC)  (Wastewater, Office Buildings, Fire Station, Parks, Roadway)  (AE Services, Radio, Maintenance, Software) Average Number of Proposals per Project: 6 Number of Completed Projects: 10 ($193 Million)  Overall C/O Rate: 0.01% (compared to 7%)  Final Results: 100% Satisfaction  Final Results: 9.1 Rating (10 max)  5 Projects where money returned

75. W W W. P B S R G. C O M 75 Rio Vista Project

76. W W W. P B S R G. C O M 76 Results Top Award 2007 Gold Award for Project Leadership Rio Vista Recreation Center

77. W W W. P B S R G. C O M 77 Fire Station 7 “Gold Medal Design Excellence” (Fire Chief Magazine – 2007) “Design Excellence Merit Award” (Fire Rescue Magazine – 2007) Masonry Guild Design Excellence Award - 2008

78. W W W. P B S R G. C O M 78 City of Peoria AZ Results CriteriaLow BidBest ValueDifference% Change Number of Projects 389NA Awarded Cost $74,181,566$187,935,047NA Actual Cost $79,315,696$188,683,729NA Average % Over Budget 7%0.4%-6.5%-94% Average Change Order % 14%0.5%-13.5%-96% % Projects On Budget (Yes/No) 3%66%63%2100% Awarded Duration (Days) 60163792NA Actual Duration (Days) 81354013NA Average % Delay 35%6%-29.4%-83% % Projects On Time (Yes/No) 37%44%7%19% Owner Satisfaction 20%93%73%365%

79. W W W. P B S R G. C O M Overview of SHIP Test Objectives Establish contract with SHIP provider for college students in Idaho BSU Objectives:  Maximize value (performance and cost) of SHIP  Have an environment of risk minimization and performance measurement  Minimize client effort in selection and management  Minimize decision making  Education of PIPS  Measurement of differential BSU would like to create a “consortium” of universities/colleges in Idaho for a single SHIP contract 79

80. W W W. P B S R G. C O M Deliverables Major project deliverables include: 1.Set and Educate Project core team and 2.Set BSU Strategic Plan 3.Capture current level of performance and cost  Plan providers  Cost structure  Program structure and details  Identify differentials, gaps, and overlaps 4.Create RFP 5.Educate Vendors 6.Run Selection and Interviews 7.Run Pre-Planning and Risk Management 8.Award & Transition 9.Establish and maintain measurement system 80

81. W W W. P B S R G. C O M Stat e of Idah o Overview Create a statewide Student Health Insurance Plan (SHIP) consortium Create a statewide Student Health Insurance Plan (SHIP) consortium  Boise State University (BSU)  Idaho State University (ISU)  Lewis-Clark State College (LCSC) 3-Year Contract | $36 Million 3-Year Contract | $36 Million Measurements of Success Measurements of Success 1.Reduce internal University program administration costs 2.Maintain or increase Customer Satisfaction (University & Students) 3.Maintain or increase cost-effectiveness of program to students

82. W W W. P B S R G. C O M Stat e of Idah o What Should We Include In RFP? Request For Information (RFI)  General request to vendors  Ask vendors what information they need to see in the RFP to create and provide an accurate proposal  Has no contractual implications, just providing information to the client RFIRFP

83. W W W. P B S R G. C O M Stat e of Idah o Selection Criteria & Weights Responding contractors were evaluated on: Responding contractors were evaluated on:  Premiums (Student, Spouse, Dependents) (200 Points)  Interviews (350 Points)  Program Administrator  Claims Administrator  Waiver Administrator  Data Base Manager  Marketing Manager  Risk Assessment and Value Added (RAVA) plan (250 Points)  Risk Assessment – ability to identify and minimize potential risk unique to this project  Value Added Option – ability to add value to the project in terms of time, money or quality  Scope Plan (50 Points)  Concise synopsis of the work that will be performed (major tasks, steps, or work packages).  Vendors impression of how they will achieve the objectives of the consortium  Past Performance Information (150 Points)  Firm  Program Administrator

84. W W W. P B S R G. C O M Stat e of Idah o 84 Filter 1 Past Performance Information Filter 2 Proposal & RAVA Plan Filter 4 Prioritize (Identify Best Value) Filter 5 Pre-Award Phase (Pre-Plan) Filter 6 Weekly Report & Post-Rating Time Quality of Vendors Filter 3 Interview High Low Summary of Proposal Submittal Proposal Includes: 1)Cost/Financial Information 2)Risk Plans (3) 3)Scope Plan (2) 4)PPI

85. W W W. P B S R G. C O M Stat e of Idah o Coverage/Plan Characteristics Consortium goal was to standardize coverage between all three University's (to maximum extent possible). However, deviations were made as necessary (BSU athletic coverage, ISU RX Coverage, Capitated Fee, etc) Consortium goal was to standardize coverage between all three University's (to maximum extent possible). However, deviations were made as necessary (BSU athletic coverage, ISU RX Coverage, Capitated Fee, etc) Consortium goal was to increase plan characteristics (to provide better coverage for students) Consortium goal was to increase plan characteristics (to provide better coverage for students)

86. W W W. P B S R G. C O M Stat e of Idah o SHIP Analysis

87. W W W. P B S R G. C O M Stat e of Idah o Analysis of Proposals

88. W W W. P B S R G. C O M Stat e of Idah o Overall Best-Value Results Best-Value Results: Best-Value Results:  Student Premium has decreased by 2% (-$26)  Spouse & Dependent Premium has decreased by 19% (-$519)  In general, Benefits/Coverage have been increased Previous Program:Previous Program: –Student Premiums increased $124/year (past 4 years) –Spouse & Dependent Premiums increased $126/year

89. W W W. P B S R G. C O M Case Study: ASU Food Services Contract $32 Million Dollars (Over 10 Years)

90. W W W. P B S R G. C O M After 1 Year: Monitoring Based on Measurements Increase sale of food by 14% Increased cash to ASU by 23% Minimized management cost by 80% Increased customer satisfaction by 37% Increased capital investment by 100%

91. W W W. P B S R G. C O M Performance Metrics – Combined ASU FY 2010

92. W W W. P B S R G. C O M ASU Dining Performance Summary

93. W W W. P B S R G. C O M 93 U of MN Objectives The UMN has a goal to be recognized as a top research institution in the world In 2005, CPPM partnered with the PBSRG (ASU) to implement the PIPS Best Value Process CPPM’s Objectives of the Best-Value Program are to:  Contract to high performers  Respond faster to customer needs  Increase performance (on time, on budget, high quality)  Increase efficiency of procurement (spend taxpayers money more efficient)  Create a fair and open process for all vendors

94. W W W. P B S R G. C O M 94 CPPM Strategic Plan First organization to establish and follow a Strategic Plan Ultimate Goal: CPPM take over entire program and is successful in implementing and sustaining the program.  Year 1 – Pilot Testing  Year 2 – Evaluation and Continued Testing  Year 3 – Expansion  Year 4 – Expansion  Year 5 – Infusion & Transition  Year 6 – Transition

95. W W W. P B S R G. C O M 95 CPPM Strategic Plan Year 1 Identify and educate core group Identify qualified vendors Implement best-value Analyze pilot projects Year 2 Continue testing best-value Evaluate core group and refine Expand test to different trades (General Construction) Educate more internal CPPM staff Implement a weekly project tracking system Refine list of qualified vendors Educate and debrief qualified vendors on initial project results Year 3 Allow other CPPM personnel to test Automated online Directors Report Monitor all CPPM projects (LB & BV) Expand testing (A/E Services) Identify performance of UMN PM’s, Procurement, other critical areas, etc. Train CPPM on all BV components Year 4 CPPM acquire and perform all best-value functions (educate and train) PBSRG assist on areas of weakness CPPM handle analysis and tracking of all weekly reports Implement best-value on a larger scale Educate other UMN groups (Energy, Zones, Permitting, Codes, ect)

96. W W W. P B S R G. C O M 96 Transitional Plan

97. W W W. P B S R G. C O M 97 Award Analysis:  Number of Best-Value Procurements: 161  Awarded Cost: $50.6M (11% below average cost)  Average Number of Proposals: 4  Projects Where Best-Value was also Lowest Cost: 53%  85% of projects were awarded to vendor with highest / second highest RAVA Plan (7.3 vs 5.9) Performance Information:  Contractor Impacts: 0% Change Orders / 4% Delay  Vendor post project rating: 9.6  Average Contractor Increase in Profit: 5% Current Construction Results

98. W W W. P B S R G. C O M 98 PM 1PM 2PM 3PM 4 Procurement Officer 1Procurement Officer 2 Director Contractor 1 Contractor 2 Contractor 3 Contractor 4 Contractor 3 Contractor 6 Contractor 1 Contractor 8 Contractor 9 Contractor 7 Contractor 2 Contractor 4 Contractor 8 Contractor 9 Contractor 2 Program Report Director 1Director 2 PM 1PM 2PM 3PM 4 Vice President

99. W W W. P B S R G. C O M 99 Report – Overall Program

100. W W W. P B S R G. C O M 100 Report - Directors

101. W W W. P B S R G. C O M 101 Report - End Users

102. W W W. P B S R G. C O M 102 Report – Internal PM’s

103. W W W. P B S R G. C O M 103 Report - Contractors

104. W W W. P B S R G. C O M 104 Report – Yearly Analysis

105. W W W. P B S R G. C O M 105 Report – Top 10 Riskiest Projects

106. W W W. P B S R G. C O M 106 Report – Analysis of Risks

107. W W W. P B S R G. C O M Research from Contractor Delays Contractor Risks% Delivery of Materials Delayed28% Installation errors26% Incorrect material ordered or delivered11% Alteration of installation needed9% Manufacture didn't have sufficient materials9% Misunderstanding of Construction Documents6% Door Frames incorrect size4% Soil compaction2% 107 52% of risks due to errors in materials delivered