The Effects of Multiple Art Experiences on Long term Social Emotional Outcomes of Urban Elementary School Students: Year 1 Results Jay P. Greene, Heidi H. Erickson, Angela R. Watson, and Molly I. Beck AEFP 43rd Annual Conference

Overview Randomly assign classes of 4th and 5th graders to receive 3 culturally enriching field trips throughout the school year and measure the impact on student social-emotional and academic outcomes. Results: Increase desire to be cultural consumers Increase in Math and ELA test scores Increase survey effort Null effect for art participation and social perspective taking

Motivation: Field trips are a long standing tradition for schools Increase in schools canceling field trips (Ellerson & McCord, 2009) Teachers report decline in arts education and field trip, particularly among disadvantaged students (Government Accountability Office, 2009) Principal reported pressure from accountability standards and tight budgets Cultural institutions report fewer student groups attending and that adult attendance at art institutions is also declining (Rabkin & Hedberg, 2010)

Previous literature Limited rigorous research on the benefits from field trips Observational studies Students involvement in the arts is associated with higher academic performance (Ruppert, 2006; Jægar and Møllegarrd, 2017) Meta-analysis on arts integration programs find 4 percentage point increase in achievement (Ludwig, Boyle, & Lindsay, 2017) Student who attend multiple cultural institutions experienced academic benefits in the short term (Lacoe, Painter, & Williams, 2016) Gold standard studies Students randomly assigned to receive a field trip to an art museum experienced an increase in tolerance, critical thinking, and desire to consume art (Greene et al., 2014; Bowen, Greene, & Kisida, 2014; Kisida, Greene, & Bowen, 2014) Student randomly assigned to attend live theater demonstrated higher levels of content knowledge, tolerance, and social perspective taking (Greene et al, forthcoming)

Research Question Do students experience social emotional and academic benefits from multiple field trips to cultural institutions? What we add to the literature Experimental design Multiple trips to 3 different art institutions Large, urban school district serving primarily minority and low income students Survey and administrative data for multiple years Hypotheses Expect positive gains in social emotional constructs such as social perspective taking Expect positive gains in student desire to consume arts Expect no significant effect in academic achievement

Research Design: Randomized Control Trial 4 elementary schools in large urban school district Randomly assign 4th or 5th grade students to serve as treatment or control School 1 4th Grade- Treatment 5th Grade- Control School 2 School 3 4th Grade- Control 5th Grade- Treatment School 4

Research Design: Randomized Control Trial Treatment: Field trips to Atlanta Symphony Orchestra, Alliance Theater, and High Museum of Art Control: Business as usual, field trip to 1 of the 3 art partners at Woodruff Art Center 4th & 5th Grade Randomized Baseline Survey in Fall 3 Field Trips Follow-up Survey in Spring

Data 4 Elementary schools, 4th & 5th grades, in a large urban school district ~550 Students Student Surveys Interest in art consumption and participation Social Perspective Taking Effort Non-response and careless answers Administrative records Demographics, end of year test scores, courses, attendance, discipline records

Pre-Treatment Comparisons of Treatment and Control Groups Variables Control (mean) Treatment (mean) Difference (C-T) Observations Demographics: Age in years 10.91 10.88 0.03 429 Female 53.54% 57.17% -3.63% 530 Non-white 99.37% 98.83% 0.54% 534 Students with Disabilities 17.73% 15.03% 2.70% 468 Attendance 2016 95.09% 95.48% -0.40% 2017 95.57% 95.72% -0.15% 532 Transferred within year 5.88% 6.85% -0.97% 4.29% 6.57% -2.28% Any Disciplinary Infraction 10.56% 8.22% 2.35% 2016 Testscores, combined -0.36 -0.29 -0.07 457 ELA -0.40 -0.11 455 Math -0.31 -0.21 -0.10 Science -0.20 -0.09 Social Studies -0.30 -0.33 454 Baseline Effort Measures Careless Answers 0.17 0.14* Non-response 0.24 0.13 0.11 Social Perspecitve Taking 0.00 0.14 -0.15* 529 Composite Cultural Consumption -0.13 0.18 -0.31*** 539 Composite Cultural Participation -0.04 0.06 0.10** Recall Previous Field Trip 68.89% 75.21% -6.32% 526 *** p<0.01, ** p<0.05, * p<0.1

Research Methodology Randomized Control Trial: Compare students who were randomly assigned to receive 3 art related field trip to control students who received one Average Treatment Effect 𝑌 𝑖𝑠 = 𝛽 0 + 𝛽 1 𝑇𝑟𝑒𝑎 𝑡 𝑖𝑠 + 𝛽 2 𝐵𝑎𝑠𝑒𝑙𝑖𝑛 𝑒 𝑖𝑠 + 𝑋 𝑖 𝛽 3 + 𝜃 𝑠 + 𝜀 𝑖𝑠 𝑋 𝑖 is a vector of student characteristics Average test score, combined Z-test in prior year tested subjects Age Gender Student with Disability Attendance prior year 𝜃 𝑠 is a fixed effect for each school Attendance= days attend/days enrolled

Treatment Effect on Student End of Year Standardized Exams   (1) (2) (3) Composite Test Score ELA Math Treatment 0.124*** 0.123** 0.107* (0.046) (0.053) (0.060) Composite Test Scores 2016 0.841*** 0.770*** 0.787*** (0.023) (0.028) (0.033) Female 0.022 0.111** -0.070 (0.047) (0.052) SWD 2016 -0.163* -0.236** -0.066 (0.093) (0.100) (0.108) Attendance 2016 0.985* 0.509 1.315* (0.550) (0.598) (0.675) Constant -1.017** -0.692 -1.191* (0.511) (0.555) (0.634) Observations 441 R-squared 0.769 0.693 0.630 Fixed effects for the four participating schools are included in each model. Age is not included in these models because age could not be calculated from administrative records. Standard errors clustered at teacher level in parentheses *** p<0.01, ** p<0.05, * p<0.1

Treatment Effect on Student Effort, Cultural Consumption and Participation, and Social Perspective Taking   (1) (2) (3) (4) (5) Careless Answers Non-Response Consumption Participation SPT Treatment -0.231*** -0.156* 0.340*** 0.084 0.113 (0.084) (0.077) (0.105) (0.119) (0.131) Careless Answers 2016 0.399*** (0.054) Non-response 2016 0.025 (0.056) Cultrual Consumption 2016 0.487*** (0.042) Cultural Participation 2016 0.551*** (0.046) SPT 2016 0.330*** Composite Test Scores 2016 -0.120*** -0.191*** -0.035 0.028 (0.037) (0.050) (0.058) Observations 379 374 R-squared 0.225 0.082 0.316 0.384 0.131 Fixed effects for the four participating schools, student age, gender, SWD status, and prior year attendance are included in each model. Standard errors clustered at teacher level in parentheses *** p<0.01, ** p<0.05, * p<0.1

Summary Large, statistically significant, positive effect on tests scores, student effort, and art consumption Positive but statistically insignificant effect on Social Perspective Taking and art participation Year 2 Added school engagement questions to survey Larger sample 6 new elementary schools New cohort at original 4 schools Treatment in 4th grade in yr. 1 will get 3 more fieldtrips for total of 6

Thank You Questions or Comments Heidi H. Erickson hh018@email.uark.edu

Supplementary Information

Sample Items: Art Consumption If your friends or family wanted to go to an art museum, how interested would you be in going? How interested are you in visiting an art museum? Visiting art museums is fun. I plan to visit art museums when I am an adult. Art is interesting to me. I feel like I don’t belong when I’m at an art museum. I feel comfortable talking about art. I would tell my friends that they should visit an art museum. Would you like more art museums in your town? Do you think your friend would enjoy a field trip to an art museum (such as the High Museum of Art)?

Sample Items: Social Perspective Taking How often do you attempt to understand your friends better by trying to figure out what they are thinking? How often do you try to think of more than one explanation for why someone else acted as they did? Overall, how often do you try to understand the point of view of other people? How often do you try to figure out what emotions people are feeling when you meet them for the first time? In general, how often do you try to understand how other people view the situation?

Descriptive Statistics for Constructing Outcomes Scales   # of Items Cronbach's Alpha Social Perspective Taking Fall 7 0.76 Spring 0.77 Cultural Consumption Cultural Consumption Composite 29 0.91 Cultural Consumption Art 10 0.86 Cultural Consumption Theater 0.81 Cultural Consumption Symphony 9 0.89 0.93 0.90 Cultural Participation Cultural Participation, Composite 12 0.82 Cultural Participation, Art 4 0.84 Cultural Participation, Theater Cultural Participation, Symphony 0.75 0.83 0.85 Notes: The scales are constructed by averaging each item in the scale. Each item in the scales was standardized before averaging all items in the scale. The entire scale was then standardized again.

Consent Forms Received Study consent rates by treatment and control   Consent Forms Received Enrollment Overall Consent Rates Treatment 307 352 87.22% Control 252 340 74.12% Overall Consent 559 692 80.78% Enrollment represents students were who eligible to participate in the study. The treatment and control group both consist of two groups of fourth graders and two groups of fifth graders.

Treatment Effect on Cultural Consumption   (1) (2) (3) (4) Composite Symphony Theater Art Treatment 0.340*** 0.217** 0.535*** 0.116 (0.105) (0.095) (0.101) (0.092) Cultrual Consumption 2016 0.487*** (0.042) Symphony Consumption 2016 0.547*** (0.039) Theater Consumption 2016 0.304*** (0.055) Art Consumption 2016 0.478*** (0.037) Composite Test Scores 2016 -0.035 -0.072 -0.058 0.044 (0.050) (0.046) (0.053) (0.057) Observations 379 R-squared 0.316 0.343 0.225 0.322 Fixed effects for the four participating schools are included in each model. Robust standard errors in parentheses *** p<0.01, ** p<0.05, * p<0.1

Benefits from Field Trips Based off past literature Cultural consumption and participation (Greene et al, 2014; Greene et al, forthcoming) Social Perspective Taking (Gehlbach et al., 2008; Greene et al., forthcoming) New to our project Administrative Records Test scores, attendance, course selection, grades, discipline records Survey effort as a proxy for student engagement Careless answers and item non-response (Zamarro et al., 2016; Hitt, Trivitt, Cheng, 2016) Hypotheses Expect positive gains in social emotional constructs such as social perspective taking Expect positive gains in student desire to consume arts Expect no significant effect in academic achievement