Research ● Evaluation ● School Improvement ● Learning Innovation ● Educator Effectiveness ● Systems Transformation Evaluating STEM Professional Development.

Slides:



Advertisements
Similar presentations

Advertisements

Progress Towards Reading Success: The Reading First Evaluation Prepared by: Amy Kemp, Ph.D. Research Associate and Patricia A. Muller, Ph.D. Associate.
Leon County Schools Performance Feedback Process August 2006 For more information
REL Appalachia and the Virginia Middle School Research Alliance Justin Baer, Director, REL Appalachia Virginia School-University Partnership Steering Committee.
Team 6 Lesson 3 Gary J Brumbelow Matt DeMonbrun Elias Lopez Rita Martin.
April 6, 2011 DRAFT Educator Evaluation Project. Teacher Education and Licensure DRAFT The ultimate goal of all educator evaluation should be… TO IMPROVE.
“The Primary Goal of the Cleveland Metropolitan School District is to become a premier school district in the United States of America.” The Cleveland.
How to Evaluate Your Health Literacy Project Jill Lucht, MS Project Director, Center for Health Policy
Becoming a High Impact Board Susan Salter Director of Board Development Alabama Association of School Boards.
Deanne Gannaway Facilitating Change in Higher Education Practices.
+ Project IDEAL in Minnesota: Building a Community of ABE DL Practice Minnesota ABE DL Toolkit.
STEM Education Reorganization April 3, STEM Reorganization: Background  The President has placed a very high priority on using government resources.
Project Aim To provide training for Early Childhood Care Providers (ECCPs) on Applied Behavior Analysis (ABA) principles within the EIBI autism classroom,
Knowing What We Learn and Demonstrating Our Success: Assessment and Evaluation of FLCs Andrea L. Beach, Ph.D. Western Michigan University Presented at.
Community Planning Training 1-1. Community Plan Implementation Training 1- Community Planning Training 1-3.
Developing an Effective Evaluation to Check for Understanding Susan E. Schultz, Ph.D. Evaluation Consultant PARK Teachers.
+ Distance Learning PD History and Development Model: Building a Community of ABE DL Practice Minnesota ABE DL Toolkit.
Enhancing Geoscience Education at Minority-Serving Institutions AMS Diversity Projects Dr. James Brey Director, Education Program | American Meteorological.
THE MYSTERY OF GETTING RESEARCH INTO USE… THE ONGOING MYSTERY OF GETTING RESEARCH INTO USE.
What is Effective Professional Development? Dr. Robert Mayes Science and Mathematics Teaching Center University of Wyoming.
Local Evaluation Overview and Preliminary Findings Diane Schilder, EdD.
So What Can I Expect When I Serve on an NEASC/CPSS Visiting Team? A Primer for New Team Members.
Student Senate A look at leadership… Kirstie Colin Reid McCormick Whitney Marlatt.
Interstate New Teacher Assessment and Support Consortium (INTASC)
Louisiana Math & Science Teacher Institute (LaMSTI) Overview of External Evaluation and Development of Self-Report Measures of Instructional Leadership.
Overview: FY12 Strategic Communications Plan Meredith Fisher Director, Administration and Communication.
Skunk Works Evaluation Tools: How do we know if we are having an impact?
Instructional/Program Design Produced by Dr. James J. Kirk Professor of HRD Western Carolina University.
Master Action Plan By Asifa Naz Khan. Subject Specialist English, TICSS, Lahore.
OHS Region X TTA Created by ICF International for OHS Region X.
Fundamentals of Evaluation for Public Health Programs ROBERT FOLEY, M.ED. NIHB TRIBAL PUBLIC HEALTH SUMMIT MARCH 31,
© 2010 by Nelson Education Ltd.
Building a Successful Professional Development Model Presented by: Howard Landman Project Director “Eastern Connecticut Elementary Science Coaching Consortium”
The Impact of the MMP on Student Achievement Cindy M. Walker, PhD Jacqueline Gosz, MS University of Wisconsin - Milwaukee.
Connecting the Dots PLC AfL DI Higher Order Thinking TLCP Multi- Literacies Arts Technology Inquiry BIP SEF SIP.
Juggling the Program Management Ball 1. One day thou art an extension educator… The next day thou art a county director or district extension director…how.
 How do we begin?  What are the different types of evaluation research?  How do these different types fit together?  What purpose do they serve?
Stakeholder consultations Kyiv May 13, Why stakeholder consultations? To help improve project design and implementation To inform people about changes.
1. Housekeeping Items June 8 th and 9 th put on calendar for 2 nd round of Iowa Core ***Shenandoah participants*** Module 6 training on March 24 th will.
Ipod Project Welcome Back Session September 27, :30- 4:30 PM (SEA # NA–Credit) Lenoir County Public Schools Preparing all students to be competitive.
Situation Analysis Determining Critical Issues for Virginia Cooperative Extension.
Lessons Learned about Going to Scale with Effective Professional Development Iris R. Weiss Horizon Research, Inc. February 2011.
Public Lessons Zigen Rural English Teacher Training Program Lin Xian, Shanxi, China Summer 2011.
April 8, 2004Washington University Teaching Center Self-Evaluation as an Instructor; Course Redesign Regina Frey, Director Washington University Teaching.
NOVA Evaluation Report Presented by: Dr. Dennis Sunal.
LPSA Zone Leaders Meeting Illinois Science Teachers Association (ISTA) Illinois Section of the American Association of Physics Teachers (ISAAPT) Illinois.
Welcome 2011 California Statewide Medical and Health Exercise.
Evaluation of the Noyce Teacher Scholarship Program 2010 NSF Noyce Conference Abt Associates Inc. July 9, 2010.
Student Learning when Teaching with Technology: Aligning Objectives, Methods, and Assessments Copyright Information Copyright Karen St.Clair & Stan North.
Professional Development Opportunities for the New Math Standards.
Teacher Growth and Assessment: The SERVE Approach to Teacher Evaluation The Summative or Assessment Phase.
Assessment of Student Learning in General Education AAHE/NCA 2003 Assessment Workshop Omaha, Nebraska ● June 2003.
NOVA Evaluation Report Presented by: Dr. Dennis Sunal.
ANNOOR ISLAMIC SCHOOL AdvancEd Survey PURPOSE AND DIRECTION.
Mathematics Performance Tasks Applying a Program Logic Model to a Professional Development Series California Educational Research Association December.
Exploring the Role of Clinical Supervision in Teacher Development Bede McCormack, LaGuardia Community College and Laura Baecher,
Lead Teach Learn PLC Fundamental III: Climate and Engagement 11/13/15.
Initial Project Aims To increase the capacity of primary schools in partnership with parents to implement a sustainable health and sexuality education.
A Framework for Assessing Needs Across Multiple States, Stakeholders, and Topic Areas Stephanie Wilkerson & Mary Styers REL Appalachia American Evaluation.
Yvonne Abel, Abt Associates Inc. November 2010, San Antonio, TX Enhancing the Quality of Evaluation Through Collaboration Among Funders, Programs, and.
TELL Survey 2015 Trigg County Public Schools Board Report December 10, 2015.
CaMSP Science Assessment Webinar Public Works, Inc. Sharing Lessons Learned in the Development and Use of Science Assessments for CaMSP Teachers and Students.
ELA Grade 11/12 Cohort Common Core Transition Training SY March 7, 2014 Professional Development Center (PDC) Judy Henderson, Emily Jimenez, Elizabeth.
Professional Development: Imagine Difference Shapes and Sizes
Designing Professional Development for Elementary School Teachers
Melanie Taylor Horizon Research, Inc.
MaryCatherine Jones, MPH, Public Health Consultant, CVH Team
School Leadership Team Conference: HQS 3
This presentation will include:
Facilitating Change (AET 560)
Presentation transcript:

Research ● Evaluation ● School Improvement ● Learning Innovation ● Educator Effectiveness ● Systems Transformation Evaluating STEM Professional Development in Informal Education Settings Sheila A. Arens & Jesse Rainey

Research ● Evaluation ● School Improvement ● Learning Innovation ● Educator Effectiveness ● Systems Transformation Train-the-trainer approach: individuals from informal science education settings provided training on climate change and how to deliver information about climate change to the public. NASA Climate Day / Earth Ambassadors Trained trainers become Earth Ambassadors, expected to design and implement Climate Day events at their informal “home sites. ” Events typically were part of other events being held at the Earth Ambassadors’ institutions (lots of variation in delivery / implementation) NASA-sponsored

Research ● Evaluation ● School Improvement ● Learning Innovation ● Educator Effectiveness ● Systems Transformation PD for informal educators aligned well with K-12 best practices in PD: ongoing and intensive connected to subject matter involved subject matter experts supported educators to develop strategies for eliciting prior knowledge encouraged use formative assessment information to guide instruction involved active learning Professional Development for Earth Ambassadors

Research ● Evaluation ● School Improvement ● Learning Innovation ● Educator Effectiveness ● Systems Transformation Supplemental Materials

Research ● Evaluation ● School Improvement ● Learning Innovation ● Educator Effectiveness ● Systems Transformation Recruited Sites

Research ● Evaluation ● School Improvement ● Learning Innovation ● Educator Effectiveness ● Systems Transformation Evaluation Questions & Data Collection Overarching Research Question Ancillary Questions Data Collection Method(s), Source(s) of Data & Instrument Development Needed What were the successes and challenges of the Earth Ambassadors training?  Are there changes in Earth Ambassador’s attitudes about climates following training?  What are Earth Ambassador’s experiences during and following training?  Surveys  Interviews (stratified sample of Earth Ambassadors regarding experiences during and following training)  Artifacts from Earth Ambassador trainings What were the successes and challenges of the Climate Day Events?  What are the perceptions of participants in the Climate Day Events?  What successes and challenges did Earth Ambassadors experience as they implemented their Climate Day Events?  Were Climate Day Events implemented as planned and what are the perceptions of participants regarding the Event? (sample)  What additional training or resources do Earth Ambassadors need to improve the successes or efficacy of their Events?  Participant surveys  Earth Ambassador survey (key Program staff)  Observations (sample two Climate Day Events)  Artifacts from Climate Day events

Research ● Evaluation ● School Improvement ● Learning Innovation ● Educator Effectiveness ● Systems Transformation Overarching Findings Earth Ambassadors perceived themselves to be more knowledgeable about climate change, ways to be “greener,” and strategies for using the materials. Earth Ambassadors considered themselves as more confident in presenting information to the public. Event participants noted an increased understanding of climate and climate change, increased confidence to talk with others about climate, and an interest in attending more climate-related events.

Research ● Evaluation ● School Improvement ● Learning Innovation ● Educator Effectiveness ● Systems Transformation Examining EA “Reach” Using Geocoding

Research ● Evaluation ● School Improvement ● Learning Innovation ● Educator Effectiveness ● Systems Transformation Focus on a “case” Client expressed interest in learning more from some of the sites. In collaboration with them, we determined to… conduct additional data collection about implementation at a sample of sites, and more closely example one of the sites where the “reach” of the program had the most potential. Kopernik Observatory & Science Center, Vestal, NY: This example is a realization of the Earth Ambassadors theory of change that moves from training to information to dissemination and to planned community action—and has led to positive steps toward addressing issues of climate change. Collect additional extant data (on the location and the politics / policies in the area) Collect additional data o Surveys, interviews o Observation

Research ● Evaluation ● School Improvement ● Learning Innovation ● Educator Effectiveness ● Systems Transformation

September 2013: Conference on Extreme Weather, Climate Change, Mitigation and Adaptation: An “action-oriented” conference aimed at municipal officials in the Southern Tier of New York State. June 2014: “Public Night” Event: Focused on extreme weather and adaptation, and featured a speaker from a local university and winners of a student essay contest about climate change. September 2014: Director and Fire Coordinator for a county public safety facility spoke about the community’s experiences with the floods of 2006 and 2011 and lessons learned in preparing for the next flood. October 2014: Preparing New York’s Communities for Climate Change: Speaker from the New York State Department of Environmental Conservation will discuss recommendations from the New York State Office of Climate Change, including the key role local governments play in responding to climate change, and the adoption of the Climate Smart Communities pledge. Additional data collected at this site provided information on how the program impacts perceptions and planned actions among government leaders and outcomes within their respective communities. Site Activities & Additional Data Collection

Research ● Evaluation ● School Improvement ● Learning Innovation ● Educator Effectiveness ● Systems Transformation Observation Protocol for Climate Day Event Approximate number of attendees: Activity: Time: Total Minutes: List of materials and resources used (indicate whether Climate Day materials or supplemental / created by site): Brief Protocol for Interview with Earth Ambassador:  Provide general information on how the Earth Ambassador organized their Climate Day Event (e.g., structure of event, professionals contacted, publicity, etc.):  Were the events implemented as planned?  What successes and challenges did the EA experience as the Climate Day event was implemented? Workshop #1: ○ None of the time ○ A little ○ Sometimes ○ A lot ○ Throughout ○ N/A  How often did presenter focus on hands-on / interactive participant involvement?  Did the presenter encourage and value audience input (e.g., questions, comments)?  How often was lecture (as a teaching technique) used?  How often was breaking into smaller groups (as a teaching technique) used? ○ Not at all ○ A little ○ Somewhat ○ Very ○ Extremely ○ N/A  Was the science content appropriate for the developmental levels of the audience members?  Was presenter able to address audience questions? Some items adapted from Goodman Research Group, Inc. (2009). Final evaluation report: Community Ambassadors in Science Exploration (CASE). Retrieved from

Research ● Evaluation ● School Improvement ● Learning Innovation ● Educator Effectiveness ● Systems Transformation Survey for Kopernik Conference Participants To what extent did…[Presenter A] provide sufficient information regarding causes of extreme weather and climate change? What was one key "take away" from this conference for you? Thinking back to what you learned at the conference, what three or four things did you hope to accomplish when you returned to your community? What additional support would you need in order to take action—or additional action— related to minimizing the damage due to extreme weather and climate change in your community? Phone Interview Protocol for Kopernik Conference Participants In your survey responses, you indicated that since the conference you were able to X (actions described in survey responses). What challenges have you experienced as a result of these actions? What successes have you experienced? What type of reaction have you received from the community? From stakeholders? Do you have additional action plans related to due to mitigating greenhouse gases or minimizing the damage from extreme weather? What resources or information would you need in order to accomplish these plans?

Research ● Evaluation ● School Improvement ● Learning Innovation ● Educator Effectiveness ● Systems Transformation

Perceived Importance of Community Engagement “Citizens have to be informed and want to adapt to the changes…The information has to be regularly supplied to the community, otherwise it is out of sight, out of mind.” “It’s very important to do this type of education and it’s valuable to have local people to know who and where to target…The local politicians are generally ignorant or just disregard it. We’re in a flood-prone area so it’s good to go after the people who are directly affected by it because then they have a greater understanding of the connection between climate and weather.” “If I were with NASA, I’d ask, how can we work more effectively with public schools and institutions? What can they influence? …make it in the best interest of research universities to see community-based education as a top priority for which they will receive money and recognition.” “Many people in U.S. aren’t very literate in solar energy but we can fix that. On-the-ground work is the most effective work, and I think the [EA program] is a great idea.”

Research ● Evaluation ● School Improvement ● Learning Innovation ● Educator Effectiveness ● Systems Transformation Thank you!!