1. NKU Executive Doctoral Co-hort August, 2012 Dot Perkins
STAT MINI-PROJECT
NKU Executive Doctoral
Co-hort
August, 2012
Dot Perkins

2. Gallatin County High School
Implemented E-prep in to improve ACT results
My hypothesis is that students who utilize E-PREP will score statistically higher on the ACT in March 2012 than students who did not utilize E-PREP, i.e., the previous three junior classes ( 2011, 2010, 2009)

3. I will compare the 2012 junior class ACT scores to the previous three years (2011, 2010, 2009) junior classes ACT scores.
These classes did not have access to E-PREP. I will also compare the sophomore PLAN scores to each classes’ ACT scores to determine possible student growth or decline

4. E-PREP
An on-line tool to quickly improve skills and confidence on the ACT test.
Each course features engaging, expert and personalized instruction, available 24/7 through on-demand videos and interactive lessons
Student had access 24/7, 365
The high school scheduled E-PREP three times during the school year to benchmark

5. PLAN & ACT CLASS OF 2013 Summary statistics:
Column n
Mean
Variance
Std. Dev.
Std. Err.
Median
Range
Min
Max Q1 Q3
PLAN 2013 95 17 16 11 27 14 20
ACT 2013 19 24 35 15 22
*Gain of 2 points typical) growth. Shows promise. Not conclusive.

6. Boxplot of All PLAN and ACT Results for Class of 2013, 2012, 2011

7. CLASS OF 2013 ACT – PLAN DIFFERENCE COMPARED TO CLASS OF 2012 & 2011 ACT & PLAN DIFFERENCE
Column n
Mean
Variance
Std. Dev.
Std. Err.
Median
Range
Min
Max Q1 Q3
PLAN 2013 95 17 16 11 27 14 20
ACT 2013 19 24 35 15 22
PLAN 2012 & 2011
187 10 29 18
ACT 2012 & 2011 33 21

8. PLAN
ACT
DIFFERENCE
2013(E-PREP Access)
17.095 19
+1.905
2012 & 2011
16.24
18.28
+2.04

9. Here are the hypothesis tests for combined ACT scores and growth
Here are the hypothesis tests for combined ACT scores and growth.   Hypothesis test results: μ1 : mean of ACT μ2 : mean of ACT 2012 & μ1 - μ2 : mean difference H0 : μ1 - μ2 = 0 HA : μ1 - μ2 > 0 (without pooled variances)
VERY slight indication of significance. Indicates difference in sample mean ACT scores is slightly unusual, if there is truly no difference in mean ACT scores for all students in these grades. On average, E-PREP group .7 higher than non-E-PREP group
Difference
Sample Mean
Std. Err. DF
T-Stat
P-value
μ1 - μ2
0.1211

10. ACT Growth –(minus) PLAN: How much growth?
Hypothesis test results: μ1 : mean of 2013 growth μ2 : mean of 2011/12 Growth μ1 - μ2 : mean difference H0 : μ1 - μ2 = 0 HA : μ1 - μ2 > 0 (without pooled variances)
Difference
Sample Mean
Std. Err. DF
T-Stat
P-value
μ1 - μ2
0.6651
ACT Growth –(minus) PLAN: How much growth?
Non-EPREP actually increased slightly more from PLAN
To ACT (****Starter higher*****)

11. Here are the hypothesis tests for ACT and growth comparisons for each year (2013 vs 2012, 2013 vs 2011)   Hypothesis test results: μ1 : mean of ACT μ2 : mean of ACT μ1 - μ2 : mean difference H0 : μ1 - μ2 = 0 HA : μ1 - μ2 > 0 (without pooled variances)
VERY slight indication of significance. Indicates difference in sample mean ACT scores is slightly unusual, if there is truly no difference in mean ACT scores for all students in these grades.
Difference
Sample Mean
Std. Err. DF
T-Stat
P-value
μ1 - μ2
0.1047

12. Hypothesis test results: μ1 : mean of 2013 Growth μ2 : mean of 2012 Growth μ1 - μ2 : mean difference H0 : μ1 - μ2 = 0 HA : μ1 - μ2 > 0 (without pooled variances)
Difference
Sample Mean
Std. Err. DF
T-Stat
P-value
μ1 - μ2
0.5323
P-value: No evidence that there is growth/change
between the two groups E-PREP and Non-E-PREP
groups.
*Slight indication that ACT scores are higher but no
Conclusive evidence that E-PREP impacted.

13. Hypothesis test results: μ1 : mean of ACT μ2 : mean of ACT μ1 - μ2 : mean difference H0 : μ1 - μ2 = 0 HA : μ1 - μ2 > 0 (without pooled variances)
Difference
Sample Mean
Std. Err. DF
T-Stat
P-value
μ1 - μ2
0.2393
P-value: No indication of a difference between ACT of 2013 & Not unusual .

14. Hypothesis test results: μ1 : mean of 2013 Growth μ2 : mean of 2011 Growth μ1 - μ2 : mean difference H0 : μ1 - μ2 = 0 HA : μ1 - μ2 > 0 (without pooled variances)
Difference
Sample Mean
Std. Err. DF
T-Stat
P-value
μ1 - μ2
0.7653
P-value: No evidence of a difference in change between 2013 & 2011.

15. Quantitative Data Examination
A summative sample of ACT test results data was obtained and compiled from Gallatin County High School
Conclusions:
*E-PREP was implemented with all juniors, sophomores, and freshmen. This on-line program has potential because E-PREP group exhibited higher ACT scores. The data are also inconclusive because there is no difference in the growth. (There are higher ACT scores for the first E-PREP class. ***May just have had a good class last year)

16. Quantitative Data Examination
Checking assumptions: samples were selected independently; number of individuals in each group exceeds 30; standard deviation of ACT and growth scores for all students in each class is unknown. Potential threat to validity is the lack of random selection.
(How was the program used? Consistently?)

17. Qualitative Data Examination
A focus group of eight randomly selected juniors was held to discuss with them their impressions, observations, likes and dislikes of using E-PREP as a tool to improve their performance on the ACT. The conversation was recorded and transcribed.

18. QUESTIONS
1. How many times did you take E-PREP? How much time was involved? Did you ever utilize E-PREP at home? The public library? On your cell phone?
2. After experiencing E-PREP, did you find it rigorous/difficult/challenging? Why or why not?
3. Did you give E-PREP your best effort? Why or why not?
4. As a result of participating in E-PREP, do you think your ACT score improved? Why or why not?
5. Were there other strategies/things that you did to improve your ACT score along with using E_PREP? If so, what were these?

19. QUESTIONS 6. Did the technology work for you?
7. Did your teacher, principal, assistant principal, or guidance counselor discuss your E-PREP results with you? How? When?
8. If your teacher reviewed your E-PREP results with you, did instruction within your classroom change as a result of theses results in order to help you improve your ACT score? How?
9. If our current sophomores who will be juniors use E-PREP, do you think this program would benefit them in their ACT preparation? Why or why not?
10. What could be improved in using E-PREP if anything? Why?

20. Qualitative Data Conclusions
After convening this focus group of students to discuss E-PREP the following are common observations and impressions:
*Students desire a set schedule of when E-PREP will be given. Let them know when they will be expected to perform.
*Give students time to do the tutoring and individual lessons that E-PREP offers to help students improve.
*Individualize instruction so students can improve once they get their E-PREP results back. Students want teachers to go over their results with them (Feedback).

21. Students suggested counting E-PREP as part of grade
*Students suggested counting E-PREP as part of grade. Recognize students who are working hard and doing what they’re supposed to do.
*Time – Give students time to read questions they missed and the correct answers.
*Need full implementation – E-PREP will do more. We don’t know how to use it all yet.
*All teachers need to take E-PREP seriously

22. NOTE
The variability in administration approaches and use of E-PREP from class to class may undermine the validity of statistical analyses. Suggest greater uniformity of use, if results are to be studied statistically.

23. Final Conclusions
*Consensus of the focus group is that E-PREP is a valuable tool and does impact ACT results by giving students confidence in the types of questions that are asked on this assessment and the time constraints that are imposed with this test.
*E-PREP will continue to be implemented next year and more data will be collected to determine value of this program on improving ACT scores among our juniors.