FIRST ROBOTICS AS A MODEL FOR EXPERIENTIAL PROBLEM-BASED LEARNING: A COMPARISON OF STUDENT ATTITUDES AND INTERESTS IN SCIENCE, MATHEMATICS, ENGINEERING, AND TECHNOLOGY Donald Sanford Griffith, Jr. May 2005

INTRODUCTION Significance of the Study As reported by the Third International Mathematics and Science Study (TIMSS), students in the United States are lagging behind other nations in the areas of mathematics and science (U.S. Department of Education, 1999). Undergraduate admissions requirements for engineering schools require students to have a solid background in mathematics and science (U.S. Department of Labor, 2002). United States Department of Labor statistics also indicate that there will be the need for 20 percent more engineers by the end of this decade (Southern Regional Education Board, 2001).

INTRODUCTION Significance of the Study The number of students who plan to major in engineering upon college entrance has continued to decrease, ACT Policy Report (Noeth, Cruce & Harmston, 2003). Students enrolled in engineering programs hit a 17-year low in 1999, according to the National Society of Professional Engineers. Enrollment in engineering technology programs in 2000 was about half what it was in 1990 (Southern Regional Education Board, 2001).

Statement of the Problem The problem addressed in this study was whether student participation in the six-week challenge of the FIRST Robotics Competition had an effect on high school students’ attitudes and interest in science, mathematics, engineering, and technology. The possible relationships between the six-week challenge of the FIRST Robotics Competition and high school students’ attitudes and interest in science, mathematics, engineering, and technology is uncertain and may be important to the overall success of students in high school courses, post-secondary education and related careers.

Purpose of the Study The purpose of this study was to investigate whether student participation in the six-week challenge of the FIRST Robotics Competition, positively changed high school students’ attitudes and interest in science, mathematics, engineering, and technology after their participation. It was anticipated that gender and race of the high school students, in the context of the FIRST Robotics experience, would influence attitudes and interest in science, mathematics, engineering and technology in either a positive or negative manner.

Research Questions Primary Questions 1.Do high school students that participate in the six-week challenge of the FIRST Robotics Competition have more positive changes in their attitudes regarding science, mathematics, engineering, and technology after their participation than students who do not participate in the program? 1.Do high school students that participate in the six-week challenge of the FIRST Robotics Competition have more interest in science, mathematics, engineering, and technology after their participation than students who do not participate in the program?

Subsidiary Questions 1.Does the gender of high school students have an effect on their interest in science, mathematics, engineering, and technology? 1.Does the race of high school students have an effect on their interest in science, mathematics, engineering, and technology?

Null Hypothesis 1.There is no difference in the attitudes of high school students that participate in the six-week challenge of the FIRST Robotics Competition regarding science, mathematics, engineering, and technology after their participation, than students who do not participate in the program. 1.There is no difference in the interest level of high school students that participate in the six-week challenge of the FIRST Robotics Competition regarding science, mathematics, engineering, and technology for students after their participation, than students that do not participate in the program.

Assumptions For this study the following was assumed: 1. high school students participating in the research study would provide accurate and valid responses on the pre- and post- survey instruments. 2. the variation of science, mathematics, engineering, and technology teachers students have experienced, and courses students have completed was equally distributed in the population. 3. the variation of science, mathematics, engineering, and technology extra-curricular activities students have experienced were equally distributed in the population. 4. the variation of students with previous FIRST Robotics Competition experience were equally distributed in the experimental group sample.

Literature Review FIRST has participated in 7 research studies since 1996 seeking to answer the question, “Is the program working?” In each study, data were collected from participants involved in FIRST. In each of these studies, FIRST students were not compared to other student populations; therefore there was not conclusive evidence that the change in students’ attitudes or interests was due to their participation in FIRST.

Literature Review Educational philosophies, theories, and methodologies supporting the FIRST Robotics Competition were considered as the rationale for identifying topics of inquiry in this review. The topics of inquiry identified were as follows: philosophies of education, experiential learning, constructivism, situated cognition, problem-based learning, and project-based learning within the context of high school students’ attitudes and interests in science, mathematics, engineering, and technology.

Research Locations High school sites were located in urban and rural locations throughout the state of South Carolina. High schools sites had students participating as team members with a FIRST Robotics team that was registered to participate in the 2005 FIRST Robotics Competition. High schools and FIRST Robotics teams selected had participated in the 2004 FIRST Robotics Competition.

Distribution of FIRST Robotics Teams Participating in the Study

Distribution of Sponsoring High Schools Participating in the Study

Research Locations During the six-week period of the study, students participating in FIRST Robotics were involved designing, developing, and producing a robot for the 2005 FIRST Robotics Competition while students at the sponsoring high schools were enrolled in a science, mathematics, engineering, or technology course.

Sampling A convenience sample was drawn from the assessable population of 2005 FIRST Robotics teams and sponsoring high schools in the state for this study. Students selected were attending public high schools during the school year. Students participating in the study represented twelve of the nineteen 2004 FIRST Robotics teams and twenty of the thirty-five sponsoring high schools. The twelve FIRST Robotics teams had a total of one hundred eighty-three participating students. The sponsoring high schools had a total of five hundred fifty participating students.

Instrumentation Copied segments of the What Do YOU Think? survey designed by the Goodman Research Group in 2000 for FIRST, to investigate attitudes and interest in science, mathematics, engineering, and technology of the study participants. Administered pre- and post-surveys to participants based conclusions and recommendations included in the Goodman Research Group 2000 Final Report To FIRST : “…any data being collected from students as part of a program evaluation be accomplished via written mail surveys not web- based surveys.” “…future research in the direction of tracking students over time and comparing them to students who do not participate in the program.” Administered the What Do YOU Think? pre- and post-surveys using Optical Mark Reader (OMR) forms to record participant responses.

Procedures Public high schools in a southeastern state identified. Letters of support received. Procedures for data collection detailed. Approval to conduct study received from Clemson University IRB. Pilot study performed. Study pre- and post- surveys administered. Statistical procedures performed. Findings compared to findings reported by the Goodman Research Group, Inc Final Report to FIRST.

Statistical Procedures (OMR OS/390 Data Set Analysis: 9.1 SAS Software©) the frequency for responses to each question was examined. chi-square procedure was used to examine the distribution patterns of the independent demographic characteristics. ANOVA was conducted to determine if differences existed between the pre- and post-survey responses to each question within each group, and if differences existed between these two groups. ANOVA was conducted to determine if differences existed between the pre- and post-survey response averages within each group, and if differences existed between these two groups.

Participants Group Pre-survey Post Survey Loss Participant % Experimental 181/ /132 (52) (FIRST) Control 546/ /375 (177) (SMET) Totals 727/ /507 (229) 70.35

Gender and Race Pre-survey Post-survey Gender Percentages Group Female Male Female Male Experimental (FIRST) Control (SMET) Race Percentages Minority White Minority White Experimental (FIRST) Control (SMET)

Survey Instrument 7 Attitudinal Categories teamwork science and mathematics knowledge of science and math careers, persist in science, mathematics, engineering, and technology enjoyment of high school science and math classes the working world self-image 3 Interest Categories career field interests students’ expected activities for the year after high school graduation students’ highest level of education

Survey Response Selection Four items are based on a 5 point Likert type scale from (1) a lot to (5) doesn’t apply (to the item). Twenty-three items of the sixty survey items are based on a 5 point Likert type scale from (1) strongly agree to (5) strongly disagree (with the item). A B C D E Strongly Agree Neither Agree Disagree Strong Agree nor Disagree Disagree 14. I enjoy participating in team orientated activities. Ten items are based on a 4 point Likert type scale from (1) very likely to(4) very unlikely (with the item). Two items are based on a 4 point Likert type scale from (1) a lot to (4) nothing (with the item). A B C D A lot Some A little Nothing 33. Science? The remaining twenty-one items required the student to select one from multiple choices.

Research Question 1 Do high school students that participate in the six-week challenge of the FIRST Robotics Competition have more positive changes in their attitudes regarding science, mathematics, engineering, and technology after their participation than students who do not participate in the program? A one-way analysis of variance (ANOVA) was conducted to examine differences between pre- and post-survey dependent variables within the experimental group (FIRST), and the control group (SMET). A two-way ANOVA was conducted to examine findings specific to the participants by comparing between the FIRST and SMET groups. Test statistic used to determine level of significance for each dependent variable in experimental and control group ( p-value α=.05 ).

One-way ANOVA Examine differences between pre- and post-survey dependent variables within each group Dependent variable 54: Managing your time effectively is an important part of being successful in your job. Source df Sum of Squares Mean Square F Value Pr > F Model Error Corrected Total

Two-way ANOVA Examine differences between pre- and post-survey dependent variables between each group Control * Test Group (SMET 546 / FIRST 181) Examine category means, pre- and post-survey question averages in each group FIRST Group Category 6: Statements about the working world. Source df Sum of Squares Mean Square F Value Pr > F Model Error Uncorrected Total

Research Question 2 Do high school students that participate in the six-week challenge of the FIRST Robotics Competition have more interest in science, mathematics, engineering, and technology after their participation than students who do not participate in the program? Response frequencies for the experimental group (FIRST) and control group (SMET), pre- and post survey responses were explored in order to address research question 2. The response frequency for each question on the pre- and post-survey was calculated as a percentage for each group.

Increases in career field interests between pre-survey and post-survey, FIRST experimental group, reported by gender. Percentage Field Gender Gender Percentage Boy Girl Pre-Survey Boy Girl Post-survey Change Architecture Business, Finance Math Engineering

Decreases in career field interests between pre-survey and post- survey, SMET control group, reported by gender. Percentage Field Gender Gender Percentage Boy Girl Pre-Survey Boy Girl Post-survey Change Architecture Business, Finance Math Engineering

Increases in career field interests between pre-survey and post-survey Gender differences of the SMET group Percentage Field Gender Gender Percentage Boy Girl Pre-Survey Boy Girl Post-survey Change Arts, Design Social Science Medicine Law

Comparison of FIRST and SMET post-survey career field interests between students, positive difference of interests for FIRST students. Gender differences are reported for each group. Percentage Field Gender FIRST Gender SMET Percentage Boy Girl Post-Survey Boy Girl Post-survey Change Engineering Math Computers Architecture Science

Comparison of the FIRST and SMET groups, pre-survey and post-survey interest in attending a 4-year college, service academy, or university after leaving high school. Gender differences are reported for each group. Percentage Pre-Survey Post-survey Net Change Group Gender Gender Gender Boy Girl Boy Girl Boy Girl FIRST SMET

Comparison of the FIRST and SMET groups, pre-survey and post-survey interest in attending a 2-year junior or community college after leaving high school. Gender differences are reported for each group. Percentage Pre-Survey Post-survey Net Change Group Gender Gender Gender Boy Girl Boy Girl Boy Girl FIRST SMET Although there was a decrease of 1.95% indicated by girls in the FIRST group regarding their interest to attend a 2-year junior or community college after leaving high school there was an increase of 9.67% by girls in the FIRST group between pre- and post-survey responses, regarding their expectation to be attend a 4-year college, service academy, or university.

Comparison of the FIRST and SMET groups, post-survey responses, students’ highest expected level of education. Gender differences are reported for each group. Percentage Post-survey Expectation Group Gender Total Boy Girl 4-year college degree FIRST SMET Master degree FIRST SMET Ph.D. or M.D. FIRST SMET

Comparison of the FIRST and SMET groups, post-survey responses, students’ expectation of a 4-year college degree. Gender differences are reported for each group. Percentage Expectation: Race Gender 4-year college degree White Minority Boy Girl Pre Post Pre Post Pre Post Pre Post FIRST SMET

Null Hypothesis 1 There is no difference in the attitudes of high school students that participate in the six-week challenge of the FIRST Robotics Competition regarding science, mathematics, engineering, and technology after their participation, than students who do not participate in the program. The null hypothesis concerning attitude differences of high school students that participate in the FIRST Robotics six-week challenge as compared to students that do not participate in the program was not rejected. High pre-survey dependent variable scores provided by students in the FIRST group did not allow for significant gain in each of the seven-attitudinal categories. Findings indicated that there were significant attitudinal differences between students in the experimental group (FIRST), and students the control group (SMET). Students in the FIRST group had statistically significant higher attitude means than students in the SMET group on both pre- and post-survey’s in the seven-attitudinal categories.

Null Hypothesis 2 There is no difference in the interest level of high school students that participate in the six-week challenge of the FIRST Robotics Competition regarding science, mathematics, engineering, and technology for students after their participation, than students that do not participate in the program. The null hypothesis concerning interest differences of high school students that participate in the FIRST Robotics six-week challenge as compared to students that do not participate in the program was rejected. The frequency for responses to each question in the three interest categories of the pre- and post-survey were calculated for the experimental and control group to evaluate differences. The results were expressed in percentages and indicated that there were significant differences in the respondent scores for the pre-survey versus the post-survey in the FIRST group.

Subsidiary Question 1 and 2 Does the gender of high school students have an effect on their interest in science, mathematics, engineering, and technology? Does the race of high school students have an effect on their interest in science, mathematics, engineering, and technology? The chi-square procedure was used to examine the distribution patterns of the independent demographic characteristics along the dependent variables. Descriptive analysis was conducted for the data on race and gender. Subsidiary questions 1 and 2 were addresses within the independent samples t test analysis demonstrating no significant differences on pre-survey or post-survey scores by either gender or race.

The Importance of This Study The findings of this study suggest that FIRST students’ have, and continue to have, high positive attitudes and interests in science, mathematics, engineering, and technology. Positive attitudes and interest in these subjects lead to higher academic achievement. The consequences are that students tend to enroll in advanced courses and pursue careers related to science, mathematics, engineering, and technology.

Enjoyment Science and Math The post-survey reported increases in enjoyment of science and mathematics for students in the experimental group (FIRST). The post-survey indicated that more than 79% of the FIRST respondents enjoyed science, and 81% enjoyed math a lot, compared to 54% and 59% respectively reported by students in the control group.

Develop Interest in Science and Math FIRST respondents, 84% were either likely or very likely to have plans for taking advanced high school or college level classes in each of the four fields. Students indicated the greatest likelihood of taking college level classes in science, mathematics, and computers. In contrast, the control group respondents reported on the pre- and post-survey that 63% were either likely or very likely to have plans for taking advanced high school or college level classes in each of the four fields.

Immediately After High School The post-survey reported that over 60% of the students in the FIRST group planned on attending a four-year college, service academy, or university immediately after graduating high school, an increase of 9% from the pre-survey. These post-survey results are compared to 39% for the SMET group of students, an increase of 1.5%. Over 60% of FIRST students responded on the post-survey, that over the course of their educational careers, they would earn at least a Bachelor’s degree, 22% expected a Master’s degree, and 21% an advanced professional or terminal degree.

Educational Expectations The study indicated that there was a difference in white and minority students’ expectations about their highest level of education. Minority students in both groups indicated on the pre- and post- surveys that they were less likely than white students to expect a 4-year college degree. The same trend was observed regarding girls with one exception; girls in the FIRST group responded on the post-survey that they were more likely than boys to expect a 4-year college degree after participating in the FIRST Robotics Competition six-week challenge experience.

Consistent With Previously Reported Findings Study findings that were consistent with findings reported in the Goodman Research Group’s Final Report to FIRST (Goodman Research Group, 2000) Differences in study findings were reported in two areas. students’ perceived knowledge of science and mathematics careers, and students’ intentions to persist in science, mathematics engineering, and technology courses and related careers.

Summary Results of the study offer some evidence that the FIRST Robotics Competition six-week challenge experience has short-term attitudinal and interest impact on students. Students in the FIRST group:  reported high positive attitudes regarding science, mathematics, teamwork, and the working world.  were more interested and felt positive about their abilities, in science and mathematics compared to students in the control group.  attitudes were more positive regarding teamwork and the world of work compared to students in the control group.  likely to have plans for taking advanced high school or college level classes in science, mathematics, engineering, technology.

Recommendations for Future Study  Investigate the long-term outcomes of student participation in FIRST Robotics.  An investigation of student participation in FIRST Robotics Competition regional and championship events. FIRST Robotics Competition event experiences involve students in the performance phase of the competition.  Expanding the study to include FIRST Robotics Competition student participants enrolled in other than public high schools, e.g., private, parochial, charter, etc., may provide very useful insights.  Follow-up studies in various geographic areas of the country may provide additional information and prove useful in forming the basis for future inquiry regarding the study variables.