Development and Use of the OCM 1 © University of Wisconsin-Madison

Development and Use of the OCM 2 © University of Wisconsin-Madison

C , David H. Gustafson and Harold J. Steudel 3 Learning Objectives Upon completing this lesson, you will be able to: Understand the theoretical nature and development of the Organizational Change Manager (OCM) Understand your results from the OCM and how to use them to improve your likelihood of project success Appreciate some level of confidence in the results you will get on your project

C , David H. Gustafson and Harold J. Steudel 4 What is the OCM forecasting model & how was it developed?

Subjective Bayesian Model P(H 1 |D 1.. Dn) = P(D 1 |H 1 ) x P(Dn|H 1 ) x P(H 1 ) P(D 1 ….Dn) Key elements – Prior Odds: P(H 1 )/P(H 2 ) – Likelihood Ratio: P(D 1 |H 1 )/P(D 1 |H 2 ) – Posterior Odds: P(H 1 |D 1.. Dn)/ P(H 2 |D 1.. Dn) Estimated by Experts

C , David H. Gustafson and Harold J. Steudel 6 Identify Factors & Measures Interview each expert – You must predict projects’ success or failure. – Can only talk to me. I will get answers. – What questions before predicting? – What answers would make you happy/sad? Review literature to supplement interviews. Create “straw model”, and meet and fight till agree.

C , David H. Gustafson and Harold J. Steudel 7 Likelihood Ratio Estimation Estimation - Experts – Estimate likelihoods – Estimate likelihood ratios – Compare responses. – Modify likelihood estimates Compare & discuss differences across experts Re-estimate likelihoods

C , David H. Gustafson and Harold J. Steudel 8 Example Likelihood Estimation Direct Estimate: Suppose you had the records of 100 successful change projects. How many would have project launches “mandates” that are: – Proactive, – Neglectful and – Nonexistent (against).

C , David H. Gustafson and Harold J. Steudel 9 Proactive Leaders carefully thought about this, Assigned a change agent (Champion), Gave a very clear aim for the project, Made not changing unacceptable and set a firm deadline. Neglectful Leaders initiated the project, Assigned the change agent, Didn't clearly define need, task or deadlines Nonexistent (Against) Leaders were against project from start The “Project Launch” Factor

C , David H. Gustafson and Harold J. Steudel 10 Likelihood ratio estimation I have the records of two projects out in the hall. One is was a success; one a failure. In which one were senior leaders more likely to:  carefully think about the project before picking it,  set a very clear aim,  remove status quo as an option and  give a champion needed responsibility & authority. How much more likely? Much? Some? A LITTLE! Almost even?

C , David H. Gustafson and Harold J. Steudel 11 Example: Internal Comparison You said that a success would be a “little more likely” to have a proactive mandate. But your direct estimates say that a proactive mandate is 3.5 times more likely. A rather big difference. Please resolve this.

C , David H. Gustafson and Harold J. Steudel 12 Compare Across Experts Look for substantial differences Ask them to talk about the differences Give them the opportunity to revise. Use the average their likelihood estimates

C , David H. Gustafson and Harold J. Steudel 13 Example: Internal Comparison

C , David H. Gustafson and Harold J. Steudel 14 OCM Values for Project Launch Factor Number of “Yes”OCM Factor Value

Bayesian Factors……. 1/6 * 1/1.4 * etc. for the 15 factors

C , David H. Gustafson and Harold J. Steudel 16 What exactly does it predict? What is Success? Six months after changes are made: they will still be in place and those affected will (for the most part) say: “It worked. I am glad we made the change” OCM Testing and Validation

C , David H. Gustafson and Harold J. Steudel 17 Study I Fourteen nursing homes Team watched state surveyors: scored OCM Six months later returned; measured change Correlated scores with # deficiencies fixed. Correlation was.80.

C , David H. Gustafson and Harold J. Steudel 18 Study II - Method 323 senior health care leaders: US, Canada and Netherlands (e.g. administrators, medical directors) attending professional meetings. Completed a survey: project they know well. 194 implemented >6 months & results known. (Big Success, Modest Success, Modest Failure, Big Failure)

C , David H. Gustafson and Harold J. Steudel 19 Assessed predictive accuracy Divide outcomes in two groups X Big Success or Pretty Successful X Disappointing or Big Failure ROC analysis chose three cutoff regions X< -1.0 (29% of cases) X ( 8%) X>+1.0 (63%) Success & failure rates in each category?

C , David H. Gustafson and Harold J. Steudel 20 Accuracy of Change Predictor

C , David H. Gustafson and Harold J. Steudel 21 Accuracy of OCM 66 failures: OCM predicted 77% accurately 151 successes: predicted 88% accurately

C , David H. Gustafson and Harold J. Steudel 22 The OCM “Predictor” Score portrays your chances of success. For instance, a +6 score means that projects with this score succeed 6 times more frequently than they fail. And a score of -8 means that projects with this score fail 8 times more frequently than they succeed. Prediction of Propensity for Successful Change

Based on the Team’s responses, your organization scored (Process) and (Cultural) on a scale from –10 to +10 for this project.

Prediction of Propensity for Successful Change

OCM Score vs. Success & Failure Rates

C , David H. Gustafson and Harold J. Steudel 26 We find that scores in the +9 to +10 range rarely occur in the “real world” and probably indicate that the team doing this scoring has not been brutally honest with themselves. Special Note

C , David H. Gustafson and Harold J. Steudel 27 Key Benefits Helps teams identify & correct roadblocks Helps monitor progress of change effort Helps select changes to address Helps allocate implementation resources A metric for identifying positive & negative implementation patterns in organization

Interpretation of YOUR Overall OCM Scores For OCM scores over 3 – If your team has OCM Predictor score over 3, your team does not need to make any immediate improvements to increase your likelihood of having a successful project. However, the improvement could be made. These improvements can be considered “optional” at this time, provided the currently strong OCM factors do not deteriorate. For OCM scores under 3 – If your team has OCM Predictor score under 3, your team should consider if it is possible to make some improvements in order to increase your likelihood of having a successful project.

How to Gain the Opportunities? How Can YOU Use the OCM Results to Improve Your Team’s Chances of Success?