Performance Based Studies Research Group State of Oklahoma Risk Management Training How To Minimize Your Risk As A Consultant www.pbsrg.com

State of Oklahoma Strategic Plan Compete based on value (proven performance and expertise) Transfer risk and control to vendor Minimize client decision making, directives, and control Vendor writes majority of the contract Vendor manages contract Measurement, measurement, measurement

State Vendor Expectations Proactive and Accountable Vendor has control Focus on risk in the seams Risk – Plans to minimize risk – impact – customer satisfaction Performance measurement – uses dominate information Performance information/risk identification disengages the bureaucracy Win-Win Vendor’s success is just as important as State’s success Goal alignment “No contract” – “Vendor writes their own” Contract is a guide If we have to go to the contract then everyone has already lost Should never have to go the contract

2009 State of Oklahoma Consultant Selection and Risk Management Process Filter 1 Registration: Past Perf. Info. Filter 2 Scope RAVA plan Filter 3 Interviews Filter 4 Best Value Prioritization Filter 5 Pre-planning Phase: WR and Scope Filter 6 Weekly Report & Post-Rating High Criteria: Interview RAVA PPI Qual. Criteria: RAVA Tech. PPI Qual. Shortlist Award Identify BV Low Time 4

Final Rating After the project is complete, the client will evaluate the project. The Final Rating will be heavily impacted by: Change orders Project delays Poor quality Owner surprises Complaints Ability to submit accurate and timely weekly reports In Best-Value Procurement, the vendors goal is to get a high rating in order to stay competitive. Before we discuss the Pre Award Phase, it is important for the vendor to understand that this project will be evaluated upon completion. The Final Rating will be heavily impacted by: Change orders Project delays Poor Quality Owner surprises Complaints This rating will impact the vendors past performance database by 50%, so it is important for the vendor to get a high rating to stay competitive. 5 5

50% Modification 6 6

State of Oklahoma Risk Management Process Vendor takes into account all client requirements and constraints Vendor creates a baseline (cost, schedule, scope, quality) Vendor identifies all risks that they do not control that could impact baseline plan and shows a plan to minimize them. Vendor takes into account all client concerns and shows how they will be minimized. If any deviation occurs during the project it is documented and tracked on the weekly report. Vendor will be rated by the client on their deviation from the baseline.

The Weekly Report An excel document that tracks pre-identified risks/concerns and actual risks/concerns on a project. Pre-Identified Risks/Concerns: Any concerns of the client at any point in the project Any risks identified before the project begins Actual Risks/Concerns Risks that are unforeseen Risks that the contractor feels has a good probability of occurring All information on the weekly report must be in “Dominant” fashion. “Dominant” means: Simple, clear, concise, accurate….. Easy to understand without a lot of information.

Weekly Report Schedule Inserted in the “Schedule&Budget” tab of the weekly The schedule should go from task order award till construction close-out (even if contractor was only awarded till the design/workplan phase) The schedule items should include the following types of activities: Major milestones High risk activities The schedule should be updated every week. It should have incorporated impact of risks It should accurately reflect the current status of the project

Reporting Pre-Identified Risks Before a project begins the vendor should have done the following: Talked with all client parties and documented their concerns Identified any potential risks that the vendor does not control Created a plan to minimize all concerns and risks Placed all concerns/risks and minimization plans in the weekly report in the RMP tab Throughout the course of a project if anyone has concerns or identifies more potential risks the vendor should add them to the RMP tab with plans to minimize them. If a pre-identified risk becomes a potential problem on the project it is then documented on the Risks tab as well.

Reporting Actual Risks Actual risks should be identified on the “RISKS” tab of the weekly report When an actual risk is identified the following should documented: How it became a risk If it was not pre-identified, an explanation of why it was not A dated history of actions performed to minimize the risk Impact to cost and time Owner rating of risk Until a risk is resolved the following must be updated: Estimated impact to time and cost Actions performed to minimize the risk Owner rating of the risk Every resolved risk with an impact, must clearly be linked to a modification on the “Schedule&Budget” tab.

Industry Structure Best Value is best predictable performance High III. Negotiated-Bid II. Value Based Owner selects vendor Negotiates with vendor Vendor performs Best Value (Performance and price measurements) Quality control Best Value is best predictable performance Performance IV. Unstable Market I. Price Based Specifications, standards and qualification based Management & Inspection Award is based on subjective minimum which has to be enforced Low Competition High

Problem with Priced Based Systems: proliferates decision making Owners “The lowest possible quality that I want” Contractors “The highest possible value that you will get” High Low Performance High Low Performance Maximum Minimum

Impact of Minimum Standards Blind vs Visionary Contractor 1 Contractor 2 Contractor 3 Contractor 4 Performance High Low Risk High Low Risk Performance Contractor 1 Contractor 2 Contractor 3 Contractor 4 Low High Decision making: what is the minimum standard, and do all contractors meet the minimum standards

Traditional PM vs. New PM Model Risks Control Don’t Control Me & Them Us Risks Control Don’t Control

Accurate perception leads to less decision making and expectations Initial conditions Final conditions Laws Laws Time

Best Value Selection Contractor 1 Contractor 2 Contractor 3 Performance High Low Risk Contractor 1 Contractor 2 Contractor 3 Contractor 4 Performance High Low Risk

Industry performance and capability Vendor X Customers Highly Trained Outsourcing Owner Partnering Owner Medium Trained Minimal Experience Price Based

Impact on Professionalism High I. Price Based II. Value Based IV. Unstable Market III. Negotiated-Bid Specifications, standards and qualification based Management & Inspection Best Value (Performance and price measurements) Quality control Competition Performance Low Owner selects vendor Negotiates with vendor Vendor performs Minimized decision making Maximized decision making Best Value Environment More accountability More expertise More professionalism Price Based Environment Less accountability Less expertise Less professionalism Less perceived “value added” Movements to make designer work for contractors due to lack of value added

Dominant information What does the most qualified, experienced people in a firm do? Get work, or do work? Why is this? It is more important in a destabilized industry to get work then to do work…. Downside….what happens to our professionalism????

ASTM Standard Setting

ASTM Standard Setting

What Will Manufacturer ‘C’ Do? Standards motivate manufacturers to lower quality

There is something wrong with the delivery of services….. No one knows how bad the problem really is….. Entire system is broken…. Requires more management…. Performance/value is decreasing…. Relationships are more important than results….

As management, control, and direction become more important….. Skill 1 Skill 2 Skill 3 Skill 4 ….it becomes less important to be skilled, accountable, and able to minimize risk

“Manager’s Code” The movement of risk.....when making decisions Is It Working? NO YES Don’t Mess With It! Did You Mess With It? YES YOU IDIOT! NO Anyone Else Knows? Will it Blow Up In Your Hands? YES You’re SCREWED! YES Can You Blame Someone Else? NO NO Look The Other Way NO Hide It Yes NO PROBLEM!

Impact on Designer’s Value and Professionalism High I. Price Based II. Value Based IV. Unstable Market III. Negotiated-Bid Specifications, standards and qualification based Management & Inspection Best Value (Performance and price measurements) Quality control Competition Performance Low Owner selects vendor Negotiates with vendor Vendor performs Minimized decision making Maximized decision making Best Value Environment More accountability More expertise More professionalism Dominantly clear that the designer can add the most value Price Based Environment Less accountability Less expertise Less professionalism Less perceived “value added” Movements to make designer work for contractors due to lack of value added

What is the difference? Do designers spend more time in reacting to bureaucracy or doing a good technical job? 2 1 2 3 4 1 3 4

Traditional PM vs. New PM Model Risks Control Don’t Control Me & Them Us Risks Control Don’t Control

Evolution of Delivery Processes Design-Bid-Build (DBB) Design Bid Build Design-Build (DB) Bid Design-Build Construction Manager At Risk (CMAR) Design Bid Build

Construction Manager At Risk Where is value added????? Design-Bid-Build (DBB) Design Bid Build Design-Build (DB) Bid Design-Build Construction Manager At Risk (CMAR) Design Bid Build

Who should be the center of the Universe in construction? In design? Outsourcing High Performing Vendor Client Client Professional Low Performing Vendor Direction, control, review

CMAR Best Practices 0% 30% 60% 100% Designer 1 3 5 Contractor 2 4 Design Intent Baseline $ Designer Final Design 1 3 5 Contractor 2 4 Scope Cost Est. Re-Scope w/Modifications Bonding capacity

Self Regulating Loop (Six Sigma DMAIC Generated) Actions Minimize data flow Minimize analysis Minimize control Requirements (DBB, DB, CMAR, DBO) Scope Risk Assessment/Value Added Plan M R Past Performance Information M R Interview Key Personnel Identify value (PPI, RA, Interview) M M V R Preplanning, Quality Control Plan M 50% Efficient Service = Minimize Risk = Self Measurement = Identify Value M R V 50% M M Measure again M R

2009 State of Oklahoma Consultant Selection Process Filter 1 Registration: Past Perf. Info. Filter 2 Scope RAVA plan Filter 3 Interviews Filter 4 Best Value Prioritization Filter 5 Pre-planning Phase: WR and Scope Filter 6 Weekly Report & Post-Rating High Criteria: Interview RAVA PPI Qual. Criteria: RAVA Tech. PPI Qual. Shortlist Award Identify BV Low Time 35

Past Performance Surveys/ Past Performance Sheet Design Firm Compiles Information Design Firm compiles and sends surveys and past performance sheet back to State Email/Fax Survey Call To Verify The final step is for your past clients to fax the survey back to the current client Mail survey back to Design Firm State of Oklahoma Past Client 36

Past Performance Information Form 254 Design firms Critical sub elements Lead project managers

Traditional Management Initial conditions Final conditions D M&C Laws Laws Time D Risk is deviation from expected measurements

When do we determine the expectation so we can measure performance? Initial conditions Final conditions M M Laws Laws Time M M Risk is deviation from expected measurements

Scope Will be rated in relative fashion Conservative plan on how the work will be done If other options are available, they will be considered along with the risk 30,000 foot level description Final design description Milestone schedule

Risk Assessment and Value Added Plan (RAVA) The Risk Assessment and Value Added (RAVA) Plan consists of two pieces The Risk Assessment The Value Added The Risk Assessment section is used to identify high performing vendors that can: Identify and minimize risk before the service has started Deliver plan to minimize risk during the life of the service. The RA Plan should clearly address the following items: List and prioritize major risk items (areas that may cause the service not to meet the expectations of the Government). Each vendor should focus on risks it does not control and Explain how the vendor will minimize the risk. 41

Risk Assessment Plan Steps What events cause risk How will the design firm minimize the risk from happening If the risk happens, what will the designer do Transfers control and the opportunity to minimize risk to the design firm

RAVA Plan Rated Blind The RAVA Plan must NOT exceed 2 pages. The RAVA Plan must not have ANY NAMES (such as vendor or manufacturer names, personnel names, project names, product names, or company letterhead) Do NOT include brochures, marketing information, or product names! The RAVA Plans will be evaluated “blind” in order to minimize any bias. 43

Value Added Differentiation Answer the questions: “What value do I bring that differentiates me from my competitors?” The Value Added Differentiation section of the RAVA Plan is to allow a vendor to: Show how it will add value (that its competitors cannot) The impact of that value in simple, provable terms How the value added will be measured – must have an impact on dollars, time, and/or satisfaction of the Government Marketing material is worthless and will negatively impact scores Value Added is where vendors provide “great ideas” 44

How The Submittal Process Works Average Score Proposal Form (1 page) Proposal Form & Other Documentation Submittal Contracting Officer Proposal Form (1 page) RAVA and Tech. Prop. (2 Pages Each) Evaluation Members

Interviews The following individual will be interviewed Project Manager/Lead Designer All interviews will be individual The same individuals used for the past performance survey will be the same individuals interviewed and will be the same individuals that will be assigned to the project No bait and switch

Sample Interview Questions Why were you selected for this project? How many similar projects have you worked on? What is different about this projects from other clients that you have worked for? What are you doing differently on this project than you have done before? Identify, prioritize, and how you will minimize the risks of this project. What risks don’t you control? How will you minimize those risks? What are your personal goals on this project? How are you going to measure your performance? What value do you bring to the project in terms of differences based on dollars, quality, or time? How will you measure performance for this project?

Best Value Selection Past Performance Risk Assessment (don’t control) / Value Added Scope/Technical Proposal (2 Page proposal/milestone schedule) Qualifications Interview

Rating Scheme (1-10, 1(not qualified, 5(can’t differentiate/don’t know), 10 (dominantly better) Filter 1 Registration: Past Perf. Info. Filter 2 Selection: RAVA plan Filter 3 Interviews Filter 4 Best Value Prioritization Filter 5 Pre-planning Phase: WR and Scope Filter 6 Weekly Report & Post-Rating High Criteria: Interview RAVA PPI Qual. Criteria: RAVA Tech. PPI Qual. Shortlist Award Identify BV Low Time 49

Risk Management Plan Sheet (RMP) Weekly Report Unforeseen Risks Risk Management Plan Sheet (RMP) Pre-Identified Risk Risk Minimization Risk Sheet (Actual) Risk that actually occurs Unforeseen Risks QUALITY ASSURANCE Checklist of Risks Sign and Date PERFORMANCE SUMMARY Vendor Performance Client Performance Individual Performance Project Performance

What are the measurements of a Designer Scope and cost Plan with minimal risk Plan to minimize risk Knowing this with minimal effort Has the designer done this before Do they know what they don’t know on a future project Can they clearly articulate their vision

Traditional PM Model: Influence and Control (micromanagement) Reactive Focus on changing people Followers are the constraint Requires maximum amount of resources Relieves management from accountability of nonperformance Inefficient and nonperforming

New PM model: Leadership Model (measurements) Proactive Alignment (measurement) Requires Understanding (acceptance of measurements) Leader/manager and system is the constraint (blindness) Focus is on setting up a quality controlled system based on performance measurements, best value requirements, and dominant information Efficient (measurement)

State of Oklahoma Best Value Process for Designers and Contractors Filter 1 Past Performance Information Filter 2 Proposal & Risk / Value Plan Filter 4 Prioritize (Identify Best Value) Filter 5 Pre-Award Phase (Pre-Plan) Filter 6 Weekly Report & Post-Rating Time Quality of Vendors Filter 3 Interview Award High Low

Differences in new system Provide past performance information (PPI) from references PPI on organization and critical components Submittals (scope and RAVA plan) will be two pages each One additional page for milestone schedule Interview will be with the project manager How do you know you can do it How are you doing it How are you taking control of the project and minimizing risk and decision making of client How did you pick your engineering subvendors

Requirements on Competing Firms More or less time invested on a project? More or less detail? Risk they control or do not control? Dominant differential or technical qualifications? Risk minimization done when? More risky approach or conservative control?

Designer should separate the requirement from the risk Propose the vision that they can do with minimial risk Propose to immediately analyze any “potentially risky options” in the beginning of the design contract Only pursue “risky options” if there is dominant information that justifies the “perceived risk” Cost should be based on “vision” and “requirements” Makes all other parties accountable and gives control of the design back to the design professional

Delivery of Design Services Status Quo Risks Control Don’t Control Me & Them Control, manage, direct, and inspect Increase the flow of information Inefficient, ineffective Maximizes technical issues Initial conditions Final conditions Time Laws X M New Design Delivery Model Align Transfer control to the contractor Preplanning Quality Control Measure Minimize flow of information Minimizes technical issues Us Risks Control Don’t Control Initial conditions Final conditions Time Laws M M M M