Louisiana Math & Science Teacher Institute (LaMSTI) Overview of External Evaluation and Development of Self-Report Measures of Instructional Leadership and Teacher Capital Stephen Meyer RMC Research Corporation MSP Learning Network Conference, January 23, 2012

Evaluation Questions (Paraphrased) Partnership Driven Program participation and completion rates; Impact on professional learning community focused on mathematics and science; Impact on participant engagement in leadership roles and activities Teacher Quality, Quantity, and Diversity Impact on teacher content knowledge, dispositions, and social and other types of capital; Impact on teacher practice; Extent to which LaMSTI reflects and supports the racial and ethnic diversity of the region Challenging Courses and Curricula LaMSTI courses (reflect research and curriculum standards; effectively engage of participants; meet needs of participants and their students); Dissemination of LaMSTI course materials Evidence- Based Design and Outcomes Use and support the use of data and research that addresses practitioner needs and optimizes STEM learning; Factors that impede or facilitate progress toward LaMSTI goals; Impact on student performance in participant classrooms; Impact on participant job satisfaction Institutional Change and Sustainability LaMSTI partnerships that are effective and sustained; Progress toward sustaining and “scaling up” activities and strategies; External adoption of LaMSTI project materials

Methods Participant Surveys, Content Knowledge Assessments, and Focus Groups Participant Surveys, Content Knowledge Assessments, and Focus Groups Project Leader and Faculty Interviews/ Focus Groups Project Leader and Faculty Interviews/ Focus Groups Course and Participant Classroom Observations Course and Participant Classroom Observations Administrator Interviews Administrator Interviews Follow-Up Participant Phone Interviews Follow-Up Participant Phone Interviews Document Review Document Review Student Achievement Data Student Achievement Data

Participant Survey Measures Professional Development Participation Professional Development Participation Preparedness/Ability to Teach (Math/Science) Preparedness/Ability to Teach (Math/Science) Confidence in Content and Pedagogical Content Knowledge (Math/Science) Confidence in Content and Pedagogical Content Knowledge (Math/Science) Participation in Professional Learning Activities Participation in Professional Learning Activities Job Satisfaction Job Satisfaction School Support for Math and Science Teaching School Support for Math and Science Teaching Teacher Background Information Teacher Background Information

Findings Average to high baseline measures of preparedness and confidence to teach math/science Average to high baseline measures of preparedness and confidence to teach math/science Strong school support for teaching math/science Strong school support for teaching math/science Participants report: Participants report: –LaMSTI courses have addressed gaps in knowledge, introduced challenging content, helped them understand connections across grade levels and disciplines, and increased understanding of Common Core standards –Acquired knowledge and skills they could apply in their own classrooms

Measuring Instructional Leadership and Teacher Capital Evaluation Questions To what extent and how does participation in LaMSTI increase participant engagement in leadership roles and activities? To what extent and how does participation in LaMSTI increase participant engagement in leadership roles and activities? To what extent and how does participation in LaMSTI affect teachers’ content knowledge, dispositions, and social and other types of capital? To what extent and how does participation in LaMSTI affect teachers’ content knowledge, dispositions, and social and other types of capital?

Instructional Leadership and Teacher Capital in LaMSTI A critical feature underlying the LaMSTI model is the development of teacher leaders. As avenues for fostering effective instructional leadership, LaMSTI seeks improve the social, cultural, and symbolic capital of teachers. For example, the project seeks to foster participant: 1) social capital by improving participant connectedness to professional community; 2) cultural capital by improving participant ability to participate in conversations about STEM subjects and teaching; and 3) symbolic capital by conferring a credential and the prestige associated with participation the project.

Teacher Leadership “The process by which teachers, individually or collectively, influence their colleagues, principals, and other members of school communities to improve teaching and learning practices with the aim of increased student learning and achievement” (York-Barr & Duke, 2004, p. 288).

Teacher Leadership: Dimensions of Practice 1) Coordination and management; 2) School or district curriculum work; 3) Professional development of colleagues; 4) Participation in school change & improvement; 5) Parent and community involvement; 6) Contributions to the profession; and 7) Preservice teacher education. (York-Barr & Duke, 2004)

Conditions for Teacher Leadership Teacher leadership influenced by school context and culture; roles and responsibilities; and structures (York-Barr, & Duke, 2004) Teacher leadership influenced by school context and culture; roles and responsibilities; and structures (York-Barr, & Duke, 2004) Culture of schools can create barriers to leadership that involves interaction and exchanging advice among colleagues (Smylie, 1992) Culture of schools can create barriers to leadership that involves interaction and exchanging advice among colleagues (Smylie, 1992) Teacher collaboration is a form of distributed leadership within a school, mediated by artifacts, routines, tools, and structures (Spillane, 2006) Teacher collaboration is a form of distributed leadership within a school, mediated by artifacts, routines, tools, and structures (Spillane, 2006)

Measures Related to Teacher Leadership Confidence in Ability to Support Colleagues (RMC Research, 2006) Confidence in Ability to Support Colleagues (RMC Research, 2006) Engagement in Leadership Roles (Stokes, Helms & Maxon, 2003) Engagement in Leadership Roles (Stokes, Helms & Maxon, 2003) Advice-Seeking by Colleagues (RMC Research, 2009) Advice-Seeking by Colleagues (RMC Research, 2009) Instructional Leadership (Adapted from Louis, Dretzke, & Wahlstrom, 2010) Instructional Leadership (Adapted from Louis, Dretzke, & Wahlstrom, 2010) Challenges Related to Teacher Leadership (Stokes, Helms & Maxon, 2003) Challenges Related to Teacher Leadership (Stokes, Helms & Maxon, 2003)

Measuring Teacher Social, Cultural, and Symbolic Capital

Social Capital Networks and associations that are valuable for information exchange and resource acquisition (Bourdieu, 1986; Lee & Croninger, 1999). Networks and associations that are valuable for information exchange and resource acquisition (Bourdieu, 1986; Lee & Croninger, 1999). Social network analysis - used to study development of teacher leaders in the context of MSP programs (e.g., Pitts & Spillane, 2009) Social network analysis - used to study development of teacher leaders in the context of MSP programs (e.g., Pitts & Spillane, 2009)

Social Capital Social networks support five aspects of teacher learning: (1) openness to new ideas/ concepts; (2) understanding, perceiving and modifying practices to fit within a particular context; (3) sharing complex and tacit knowledge; (4) the capacity for a learning community to support reform and change; and (5) sustaining commitment to the community or activity (Baker-Doyle & Yoon, 2011) Social networks support five aspects of teacher learning: (1) openness to new ideas/ concepts; (2) understanding, perceiving and modifying practices to fit within a particular context; (3) sharing complex and tacit knowledge; (4) the capacity for a learning community to support reform and change; and (5) sustaining commitment to the community or activity (Baker-Doyle & Yoon, 2011)

Cultural Capital Bourdieu (1990) described linguistic capital as one type of cultural capital, referring to one’s mastery of and relation to language which represents a means of communication and self- presentation acquired from one's surrounding culture. Development and recognition of cultural capital happens within a particular “field” within which cultural competence, or knowledge of particular tastes, dispositions, or norms, is both produced and given a value. Bourdieu (1990) described linguistic capital as one type of cultural capital, referring to one’s mastery of and relation to language which represents a means of communication and self- presentation acquired from one's surrounding culture. Development and recognition of cultural capital happens within a particular “field” within which cultural competence, or knowledge of particular tastes, dispositions, or norms, is both produced and given a value.

Symbolic Capital Resources available to an individual on the basis of honor, prestige or recognition (Bourdieu, 1986). Resources available to an individual on the basis of honor, prestige or recognition (Bourdieu, 1986). Formal or informal positions or titles within organizations and credentials can serve as symbolic representations of legitimacy, expertise, authority, ability, or respect. Formal or informal positions or titles within organizations and credentials can serve as symbolic representations of legitimacy, expertise, authority, ability, or respect. Within K-12 educational settings, social and symbolic capital contribute to effective leadership and teaching among administrators by facilitating access to quality networks and signifying the acquisition of reputable, legitimate schooling experiences (Bartee, 2007) Within K-12 educational settings, social and symbolic capital contribute to effective leadership and teaching among administrators by facilitating access to quality networks and signifying the acquisition of reputable, legitimate schooling experiences (Bartee, 2007)

Measures Related to Teacher Social, Cultural, and Symbolic Capital No existing self-report measures No existing self-report measures Developing items based on LaMSTI objectives, research, and theory... Developing items based on LaMSTI objectives, research, and theory... Access to resources that support work as a teacher Access to resources that support work as a teacher Regular exposure to new ideas and concepts Regular exposure to new ideas and concepts Access to a strong learning community Access to a strong learning community Collaboration with colleagues who support work as a teacher Collaboration with colleagues who support work as a teacher Ability to communicate a deep understanding of how to teach STEM content to others; talk about current education research Ability to communicate a deep understanding of how to teach STEM content to others; talk about current education research Improved reputation and recognition as leader as result of participation in LaMSTI Improved reputation and recognition as leader as result of participation in LaMSTI

Next Steps Develop/adapt item scales to align with LaMSTI objectives Develop/adapt item scales to align with LaMSTI objectives Test reliability/validity Test reliability/validity e.g., relation to other survey measures and teacher and student outcomes e.g., relation to other survey measures and teacher and student outcomes

Contact Information Stephen Meyer RMC Research th Street, Suite 2100 Denver, CO