1. General Education Science Assessment Leadership Team
Assessment Day May 2017
General Education Science Assessment
Leadership Team
Victor Bondzie, Timothy Barnett, and Rita Luther

2. This assessment day focuses on at-risk students

3. Graduation Rates (%) 6.2% 10.7% 9.2% 3.4% Other? SE Asian
Is the data interpreted per time point as 19.4%, 32.2%, and 36.6% of AA, H, C students, respectively graduate in five years?
C and H students show 10.7% and ~9.2%,resp increase over the 8 year period in 5 yr grad rate, while AA student show only ~6.2% increase.
6.2%
10.7%
9.2%
3.4%

4. Five Year Graduation Rate By Success in First Five Courses
Graduation Rates (%)
Five Year Graduation Rate By Success in First Five Courses
Are the five course any course or Gen Ed courses only? Any course
If successful in specified #classes then that is your rate of graduate
8.9%
7.7%
4.2%
0.8%

5. Success Rates by Gender/Ethnicity
All Fall 2016 Courses – College-Wide
Ethnicity
Gender
Is success rate for the first five courses and are students FTIC?
How well they do in their classes (A, B or C)
Failure rate D, W, F, I

6. Four At-Risk Student Populations Identified:
From Achieving the Dream to the Academic Initiative Review at Valencia College
Four At-Risk Student Populations Identified:
Students not successful in any of their first five courses or do not enroll twice in their first four terms are not as likely to graduate.
New students attending college for the first time (FTIC)
“Student success is largely determined by a student’s experience during his or her first-year in college” (Upcraft, et al. 2005)
Students taking math and identified as in need of support (developmental)
2015 Strategic Report found that 46% of student success in developmental math from Fall 2011 to Summer 2013 semesters.
Students who are among those that show declining rates of persistence, such as African American males.

7. Fall 2016 Survey of At-Risk Students Indicated:
Some students are using and trying to follow advice given by support services.
Students want more feedback and in some cases more direction from faculty.
They want to learn strategies for engaging faculty, learning support and advising at Valencia College.
Developmental Education Research
English courses: Survey of students (n=1,387 13% response rate), 1 focus group (n=7), and 7 student interviews
Math courses: Survey of students (n=1,226 12% response rate), 3 focus groups (n=23), and 5 student interviews
Nichole Jackson, Assistant Director of Learning Assessment, Valencia Institutional Assessment

8. Students who are at risk tell us they need:
Support transitioning from high school to college
Strategies making the most of services available
Preparation to achieve in courses higher than their current ability level
Encouragement, incentives, or requirements in order to seek help
A connection to support services:
that will deepen their knowledge
be available for them to use repeatedly and regularly in person and online

9. Agenda Items Review Learning Outcomes Review 2017 Spring Data
Discuss Outcomes Resulting From Data
Discuss Future Directions of the Science Gen. Ed. Work

10. REVIEW OF LEARNING OUTCOMES
Outcome #1: Critical Thinking- Effectively analyze, evaluate, synthesize, and apply information and ideas from diverse sources and disciplines.
Faculty developed ( ) and employed Biology, Chemistry, and Physics scenarios to address Observation, Hypothesis, Data and Conclusions.
Outcome #2: Quantitative and Scientific Reasoning- Students should be able to use processes, procedures, data, or evidence to solve problems and make effective decisions - Not yet attempted.

11. REVIEW OF 2017 SPRING DATA

12. Current Expectations
The students should be able to apply the scientific method.
Expectations assessed through student analysis of the scientific method in the context of a specific scientific problem.
We expect that at least 3000 student participants.
We also expect that 70% or more of our students will earn a grade of 75% or higher.

13. Summary of Spring 2017 Data Overall Responses
Fall 2015
Fall 2016
Spring 2016
Spring 2017
Students Invited
11442
12448
11296
11862
Total Responses
3179
3222
2406
2892
% Responded
27.78
25.88
21.30
24.38
Minimum One Question Completed Taking ≥ 4 Minutes
Fall 2015/Spring 2016-Chemistry
Fall 2016/Spring 2017-Physics
Fall 2015
Fall 2016
Spring 2016
Spring 2017
Total Responses
2230
2300
1590
1973
% Responses
19.49
18.48
14.08
16.63
Mean Score
1.76
2.33
2.00
2.34
No. Scores ≥ 75% (% Total)
587 (26.32%)
1051 (45.70%)
544 (34.21%)
905 (45.87%)

14. Science Gen Ed Assessment “Passing” Rates
% Students Scoring ≥ 75%
Biology Assessment
Chemistry Assessment
Physics Assessment

15. Distribution of Scores

16. Percent Correct Per Question
% Students
Not all questions correct only a single question correct
Individual question basis

17. Participation By Campus

18. Faculty Participation
33%
30%
20%
16%

21. QUESTION BREAKDOWN

22. Question #1: Which of the following observations support conditions for the existence of water? Observations Question #2: A scientist graphed the event of a single temporary drop in the brightness of a star that was observed during an observation period of 1,500 days this way: The scientist created 4 additional graphs to show the timing of brightness changes in four other stars that were also observed for 1,500 days each. If each planet is described by a distinct drop, which of the following graphs would be most consistent with the presence of two planets circling a star, rather than just 1 planet? Hypothesis/Prediction/Data Analysis Question #3: Which of the following statements is true from the experimental data obtained by the Kepler team? Data Analysis Question #4: Which of the following statements can reasonably be concluded? Conclusions

23. Question #1: Which of the following observations support conditions for the existence of water?
Fall 2016
Spring 2017
Answer
# Responses
% Responses
A) The core of these new planet is similar to our own Earth
191 8%
162
B) The majority of the planets are smaller than four times the size of the Earth
121 5% 94
C) All of the recent discovered worlds are located in multi-planet systems
145 6%
149
D) Four of the newly discovered planets orbit in their sun’s habitable temperature zone
1843
80%
1568
79%
Total
2300
100%
1973

24. Question #2: A scientist graphed the event of a single temporary drop in the brightness of a star that was observed during an observation period of 1,500 days this way: The scientist created 4 additional graphs to show the timing of brightness changes in four other stars that were also observed for 1,500 days each. If each planet is described by a distinct drop, which of the following graphs would be most consistent with the presence of two planets circling a star, rather than just 1 planet?
Fall 2016
Spring 2017
Answer
# Responses
% Responses A)
391
17%
329 B)
562
24%
471
C) Correct Answer
1215
53%
1021
52% D)
127 6%
149 8%
Total
2300
100%
1973

25. Question #3: Which of the following statements is true from the experimental data obtained by the Kepler team?
Fall 2016
Spring 2017
Answer
# Responses
% Responses
A) Approximately 35 of the 715 new planets are bigger than the size of Neptune
964
42%
854
43%
B) Approximately 7% of the 715 new planets are less than two-and-a-half times the size of Earth
288
13%
267
14%
C) Approximately 500 of the 715 new planets are smaller than the size of Neptune
332
266
D) Approximately 95% of the 715 new planets are located in multi-planet systems
716
31%
586
30%
Total
2300
100%
1973

26. Question #4- Which of the following statements can reasonably be concluded?
Fall 2016
Spring 2017
Answer
# Responses
% Responses
A) There is a high probability that if two cats are seen that they could be two lions
464
20%
374
19%
B) There is a high probability that if more than two cats are seen there would be a lion and multiple lionesses
1331
58%
1172
59%
C) There is a high probability that if two cats are seen they could be two lionesses
111 5% 97
D) There is an equal probability that if two cats are seen that they could be two lions or two lionesses
394
17%
330
Total
2300
100%
1973

27. Spring 2017: Question #1 Disaggregated Data By Ethnicity and Gender
# Responses (%) Males
# Responses (%) Females
Answer
African American
Hispanic
Caucasian
A) The core of these new planet is similar to our own Earth
4 (6%)
16 (7%)
9 (5%)
25 (12%)
40 (9%)
27 (8%)
B) The majority of the planets are smaller than four times the size of the Earth
5 (7%)
8 (3%)
7 (4%)
18 (9%)
25 (5%)
16 (5%)
C) All of the recent discovered worlds are located in multi-planet systems
12 (5%)
29 (14%)
44 (10%)
25 (7%)
D) Four of the newly discovered planets orbit in their sun’s habitable temperature zone
57 (80%)
205 (85%)
175 (88%)
133 (65%)
346 (76%)
281 (81%)
Total
71 (100%)
241 (100%)
200 (100%)
205 (100%)
455 (100%)
349 (100%)

28. Spring 2017: Question #2 Disaggregated Data By Ethnicity and Gender
# Responses (%) Males
# Responses (%) Females
Answer
African American
Hispanic
Caucasian A)
10 (14%)
31 (13%)
26 (13%)
30 (15%)
68 (15%)
65 (19%) B)
16 (23%)
50 (21%)
40 (20%)
61 (30%)
127 (28%)
81 (23%)
C) Correct Answer
40 (57%)
142 (59%)
121 (61%)
90 (44%)
216 (48%)
181 (52%) D)
4 (6%)
18 (7%)
13 (7%)
24 (12%)
42 (9%)
22 (6%)
Total
70 (100%)
241 (100%)
200 (100%)
205 (100%)
455 (100%)
349 (100%)

29. Spring 2017: Question #3 Disaggregated Data By Ethnicity and Gender
# Responses (%) Males
# Responses (%) Females
Answer
African American
Hispanic
Caucasian
A) Approximately 35 of the 715 new planets are bigger than the size of Neptune
24 (34%)
114 (47%)
110 (55%)
57 (28%)
161 (35%)
163 (47%)
B) Approximately 7% of the 715 new planets are less than two-and-a-half times the size of Earth
8 (11%)
23 (10%)
20 (10%)
40 (20%)
81 (18%)
37 (11%)
C) Approximately 500 of the 715 new planets are smaller than the size of Neptune
33 (14%)
18 (9%)
25 (12%)
79 (17%)
56 (16%)
D) Approximately 95% of the 715 new planets are located in multi-planet systems
31 (44%)
71 (29%)
52 (26%)
83 (40%)
134 (29%)
93 (27%)
Total
71 (100%)
241 (100%)
200 (100%)
205 (100%)
455 (100%)
349 (100%)

30. Spring 2017: Question #4 Disaggregated Data By Ethnicity and Gender
# Responses (%) Males
# Responses (%) Females
Answer
African American
Hispanic
Caucasian
A) There is a high probability that if two cats are seen that they could be two lions
11 (15%)
41 (17%)
22 (11%)
54 (26%)
99 (22%)
60 (17%)
B) There is a high probability that if more than two cats are seen there would be a lion and multiple lionesses
40 (56%)
158 (66%)
146 (73%)
101 (49%)
258 (57%)
208 (60%)
C) There is a high probability that if two cats are seen they could be two lionesses
2 (3%)
12 (5%)
6 (3%)
19 (4%)
15 (4%)
D) There is an equal probability that if two cats are seen that they could be two lions or two lionesses
18 (25%)
30 (12%)
26 (13%)
28 (14%)
79 (17%)
66 (19%)
Total
71 (100%)
241 (100%)
200 (100%)
205 (100%)
455 (100%)
349 (100%)

31. Fall 2016: Question #1-4 Disaggregated Data By Grade (% Percent Correct)
Spring 2017
Grade
Q #1
Q #2
Q #3
Q #4 A
641 (86%)
451 (60%)
391 (52%)
483 (64%)
584 (84%)
407 (58%)
384 (55%)
445 (64%) B
633 (80%)
413 (52%)
341 (43%)
446 (56%)
525 (80%)
328 (50%)
283 (43%)
385 (59%) C
483 (79%)
296 (49%)
221 (36%)
343 (56%)
339 (77%)
204 (47%)
163 (37%)
253 (58%) D
114 (70%)
77 (48%)
45 (28%)
84 (52%)
89(71%)
62 (50%)
38 (30%)
68 (54%) F
39 (70%)
20 (36%)
17 (30%)
26 (46%)
42 (66%)
28 (44%)
12 (19%)
33 (52%)

32. STUDENT SCORING BY SUNDRY VARIABLES

33. Student Scoring by Sundry Variables
Format with titles

34. Student Scoring by Sundry Variables
Scores by Number of Science Courses Completed Prior to Fall 2016

35. Student Scoring by
Sundry Variables
Scores by Discipline

36. SUMMARY/DISCUSSION OF SPRING 2017 DATA
Student responses and distribution of answers overall were similar between Fall and Spring 2017 Sci Gen Ed Assessments
Data indicated a consistent discord between ethnic groups within males and female student populations for correct answer selections in the Sci Gen Ed Assessment with african american students showing the lowest percentage of students selecting correct answers. This was followed by hispanic students and finally caucasian students showing the highest percentage of students selecting correct answers.
Data also indicated gaps between male and female students in question performance for african american and hispanic female students. Caucasian female students had a higher percentage of students selecting correct answers among females and in some cases overlapping with the performance of some male populations.

37. SUMMARY/DISCUSSION OF SPRING 2017 DATA
Trends shown in proportion of student with correct answers based on course grades
Sundry variables:
Comparable frequencies of students completed the tutorial between Fall 2016 and Spring 2017
Mean scores of those completing the tutorial are slightly higher, but not a striking difference (too early)
Slightly higher mean scores for students that completed more science courses, but not a striking difference.
Physics, chemistry, biology, earth science, horticulture (Score performance differences from higher to lower)

38. FUTURE DIRECTIONS OF SCI GEN ED WORK
How best do we assist at-risk students? (Mentoring, tutoring, NSE, other ideas)
Where do we go from here…
The fall and spring data is almost identical. Can we run one assessment each fall and know that it will be representative of students overall? (This would give you much more time to move from results in fall to action plan development in spring towards assessment day).
Should we run biology assessment (edited) in Fall 2017?
How do we get full adjunct faculty buy-in to assessment to increase student participation?
We need new volunteers to serve on the Assessment team with Rita for two years
Further discussions on assessment day.
Run in Fall only since data between semesters is similar and do some action thing in spring
Bob-Instead of multiple choice, do a lab report, random sampling to grade.
Full vs sampling data
If lab report, may be more motivated to complete.
Provide uniform rubric and sample lab report (good example)

39. Thank you!