Better Organizations Through Better Science ® © 2009 Valtera Corporation. All Rights Reserved. Management-Based Evidence SIOP New Orleans April 2, 2009 Benjamin Schneider, Ph.D. Senior Research Fellow, VALTERA

2 © 2009 Valtera Corporation. All Rights Reserved. Outline Putting the burden on researchers. This talk is not for you if you could care less about management paying attention to your findings. Issues to consider Sponsor characteristics Strategic Focus Doability Administration and data analysis Receptivity Feedback Follow-through and follow-up Summary and Conclusion

3 © 2009 Valtera Corporation. All Rights Reserved. Putting the Burden on Researchers Evidence-based management puts the burden on management –Don’t operate on hunches –Do operate on evidence Misses the paradox: –Management achieves by having good hunches (Gladwell) –Researchers know what should be done; management knows whether it can be done –Management knows the nuances for local implementation Conclusion: Listen carefully to your sponsor before beginning the research if you want the data to be used

4 © 2009 Valtera Corporation. All Rights Reserved. Sponsors Permit you to do research and/or hire you because they like you; your credentials are secondary Sponsors are the key to action after the research; you must please them through the process or they will not use your findings Sponsors must be intimately consulted in every stage of the process so they feel comfortable with the resultant findings: They must understand the findings in their terms so you must know who they are and what there terms are They lay their career on the line to implement findings so listen very carefully

5 © 2009 Valtera Corporation. All Rights Reserved. Strategic Focus If you do not show how your work is directly, or at worst indirectly related to your sponsor’s critical problem fugedaboudet. Sponsor wants an employee opinion survey—find out why. Sponsor wants a new selection process; find out what the present one is not doing sponsor wants you to help her make change—find out from what, to what, why that, and why now? Generic anything we do is out; strategic everything we do is in. A strategic focus is how management thinks and so should we: Employee opinion surveys Personnel selection Goal-setting and rewards systems Climate and culture efforts at measurement and change

6 © 2009 Valtera Corporation. All Rights Reserved. Strategic Focus Examples Organizational climate-climate for what? Service Safety Innovation Employee engagement towards what ends? Customers? Employee Turnover Personnel selection to achieve what goals? Efficiency Quality

7 © 2009 Valtera Corporation. All Rights Reserved. Doability Financial resources? Lexus vs. Fit Customized? CEO commitment—how far and in what ways? Public sign-off on change? Make the project as important as financial results? Need for a line-based project team, members of which are like those who will be affected Communications to anyone/everyone who will be involved Touching base to encourage line support throughout Tracking project process, data analysis and feedback preparation

8 © 2009 Valtera Corporation. All Rights Reserved. Receptivity Project goals in the language and conceptual space of the organization Careful diagnosis of the key strategic goals, norms, roles & functions –Mission statement, myths and stories and their focus and message –Introduce all projects so they “fit” the context—good use of the project team Implemented tools must also fit the context Deal head on with cynicism associated with previous efforts Surveys in the language of the context Tests, interviews and simulations that are highly relevant to the context –Face validity saves face –Line interviewers and assessors Have no secrets—because there are no secrets

9 © 2009 Valtera Corporation. All Rights Reserved. Administration and Data Analysis Administration Preceded by letter from CEO about the importance attached to the project—and that there will be follow-up (unlike previous surveys) If paper, administer in groups If on-line get time off to do it Data analysis Management likes item data—”top box” or “top boxes” Compare item data to the competition (norms) testing theory using scales is fine but not relevant to management Driver analysis Do driver analysis against strategic outcomes Do driver analysis at the level of the team or group or unit where action happens Focus in on actionable items (NOT: Overall how satisfied …) Driver analysis focuses on drivers, not just low-scoring items

10 © 2009 Valtera Corporation. All Rights Reserved. Feedback Invite the CEO and his or her TMT so they hear the presentation and witness the reactions of the unit leaders Test the findings and implications with the project team first Always present the good news first and that the purpose of everything is to build on strengths Put the most important findings up front, with implications Present the process by which the evidence was gathered Arrange a separate meeting with the CEO to discuss change issues Unit leaders give feedback to their units Train them to do the feedback with the focus on the positives and the future and not the past Make unit leaders responsible for presenting their action plans to at least the project team and hopefully the CEO (or his or her representatives)

11 © 2009 Valtera Corporation. All Rights Reserved. Summary and Conclusions Evidence-based management puts the burden on management to take action based only on evidence Researchers should shoulder the burden of producing management based evidence Management-based evidence is: – generated in the world of managers –focuses on their strategic needs –pays far more attention to the research process than is typical –presents data in ways that managers find useful and understand –operates with a basic philosophy of change as the outcome All of my data based publications have been paid for by management so this approach is not a limitation on publishing