1. Measuring Team Effectiveness Presented by Jill A. Marsteller, PhD, MPP Based on work by Stephen Shortell, Jill Marsteller, Michael Lin, Marjorie Pearson, Shinyi Wu, Peter Mendel, Shan Cretin, and Mayde Rosen. The Role of Team Effectiveness in Improving Chronic Illness Care, RAND/ UC Berkeley Improving Chronic Illness Care Evaluation. Medical Care, November 2004.

2. Outline of presentation Define team effectiveness Situate in team performance theory Context Factors, items & reliability Show associations -- antecedents and consequences Conclusion

3. Team Effectiveness Perceived organizational support Team self-assessed skill Goal agreement and participative norms Team autonomy/process ownership Information/help available

5. Context in which measures were tested 40 teams participating in the Improving Chronic Illness Care Evaluation (ICICE) Surveyed members of multi-disciplinary teams As many as 12 occupational categories Size 1 to 14 members One disease-specific team per organization Organizations are hospitals, physician groups, clinics, health plans, or health systems Participation was voluntary

6. Team Effectiveness Instrument Original creator: G. Ross Baker at U. Toronto (ross.baker@utoronto.ca) (32)ross.baker@utoronto.ca We reduced number of items (23), defined own factors ANOVA confirmed aggregation of individual- level responses to team level Responses 1-7 (strongly agree to strongly disagree) Varimax rotation, eigenvalues 1.0, clean loading 0.4

7. Factors, Items & Reliability Organizational support (Alpha=0.85) This organization makes sure people have the skills and knowledge to work in teams A team that does a good job in this organization does not get any special rewards or recognition (reverse coded) Senior management in the organization strongly supports our work Senior management regularly reviews our progress in making change Senior managers in my organization see success in this project as a high priority for the organization

8. Factors, Items & Reliability Team Self-Assessed Skill (Alpha=0.90) Our team has been able to use measurement very effectively to design and test changes After we have completed a change, team members are excellent in reflecting and learning from the results Members of our team were very successful in using information from our change cycles to design new tests of change In making changes, our team was able to easily adapt change ideas to match the needs of our organization Our team applied enough knowledge and skill to the work to get the work done well

9. Factors, Items & Reliability Goal Agreement & Participative Norms (Alpha=0.90) Project team members agreed on the projects overall goals The projects goals were understood by all the project team members Most members of my team got a chance to participate in decision-making Certain individuals in this group had special skills and knowledge that the rest of us count on The contribution of every group member was listened to and considered

10. Factors, Items & Reliability Overall Perceived Team Effectiveness (Alpha=0.95) Organizational support Team self-assessed skill Goal agreement and participative norms Team autonomy (Alpha=0.81) Information/help available

11. Descriptives MeasureNMeanStd. Dev. MinMax Team Size406.533.04314 Team Skill405.030.903.136.73 Goal Agree and Partic Norms405.850.644.577 Organizational Support404.780.932.206.27 Overall Perceived Team Effectiveness 405.240.743.476.64

12. Predicting Team Effectiveness Independent Variable Overall Perceived Team Effectiveness Coefficient (sig. lvl) Constant1.11 Team Size-0.06* Team Champion0.69*** Patient Satisfaction Focus0.49** Cultural Balance3.10* Asthma0.09 Physician Percentage on Teams1.27* N40 F5.29 p-value0.0006 Adj R-Sq0.4 *indicates p<0.10; ** indicates p<0.05; *** indicates p<0.01

13. Predicting Quality Improvement Activity Independent VariableNumber of ChangesDepth of Changes Coefficient (sig. lvl) Constant-49.62-11.13 Team Size5.793.90*** Team Size Squared-0.22-0.19** Patient Satisfaction Focus-21.58***-4.79** Cultural Balance118.85*17.54 Asthma-10.75-3.52** Overall Perceived Team Effectiveness 12.00***4.69*** N40 F3.285.66 p-value0.01220.0004 Adj R-Sq0.260.42 * indicates p<0.10; ** indicates p<0.05; *** indicates p<0.01

14. Change in QI Clinical Process Score Patient Survey~1300 patients, 29 organizations Higher score greater improvement in process measures than average Team effectiveness and team skill are consistently associated with greater process improvement

15. Conclusion Applies to multiple settings Versatile Appropriate measures of team-level phenomena Easily understood questions High Cronbachs Alphas Well supported by theory Relate as expected to QI Questions are actionable Available free on the web, www.rand.org/health/icice