1. Business Statistics and Operations Management: Teaching Strategies and Assessment of Student Learning in Traditional vs. Online Classes
presented by Dr. Burcu Adivar
joint work with Dr. Li Chen
Fayetteville State University
College of Business and Economics
Department of Management, Marketing & Entrepreneurship
2017 DSI Annual Conference
Nov 18, 2017
Washington, DC

2. Outline Objective Description of the study
Effects of age, gender and race on learning outcomes of the quantitative courses
Comparative analysis of online vs. face-to-face teaching
Correlation analysis
Multivariate analysis
Conclusions

3. Objective
To better understand student learning experience in quantitative courses
To measure the effect of quantitative courses’ learning outcomes on the CLA exam performance
To use analytics before proposing curriculum changes and redesigning assessment of learning outcomes
To explain the determinant of CLA scores and use predictive models to increase student and faculty awareness for better preparedness

4. Description of the Data
MGMT335 Operations Management is a core course for students in College of Business and Economics
BADM 216 Business Statistics is the prerequisite course
Face-to-face and online sections are offered every Fall, Spring and summer
We are using Canvas system and same syllabus, same text for all sections in our teaching.
Canvas system provides
number of page views
number of participation in the course website
number of messages sent to the instructor online
amount of time he has spent in the course website
course grade, student pictures (gender, age, race)
Number of ontime, late and missed assignments
Banner system provides
student transcripts
statistics grade
Dean’s Office provides
CLA score cards for students
Total score
154 Student records from 5 sections of OM:
Fall 2016 (F2F, ONL), Spring 2017(F2F, ONL), Summer 2016 (F2F)

5. Age Factor STAT_Grade OM_Grade CLA_Score Age N Mean Min Max % of Total
Variance Y
110
2.94 1 4
71.89%
0.86
78.8 3
103
72.06%
282 34
1029
746
1347
77.84%
20663 A 44
2.88
28.11%
1.06
76.4
3.15
102
27.94%
526 10
996
782
1187
22.16%
19688
All
154
2.92
100.00%
0.91
78.1
350
1021
20171

6. Gender Factor STAT_Grade OM_Grade CLA_Score Gender N % of Total Mean
Std Dev
Min
Max F 94
61.33%
2.94
0.95 1 4
61.40%
78.6
17.9
3.15
103 29
66.57%
1032
134
782
1281 M 60
38.67%
2.9
0.96
38.60%
77.4
20.1 3
102 15
33.43%
1002
159
746
1347
All
154
100.00%
2.92
78.1
18.7 44
1021
142

7. Race Factor STAT_Grade OM_Grade CLA_Score Race N Mean Min Max
% of Total
Variance A 95
2.71 1 4
68.72%
0.83
73.7 3
102
70.21%
400 28
1003
746
1347
66.35%
17383 W
3.57
26.74%
0.25
91.4 70
103
25.65%
69.3 11
1104
908
1281
28.71%
20579 H 5
3.4 2
4.55%
0.8
82.7 98
4.14%
184
1044
901
1187
4.94%
40898
All
128
2.92
100.00% 78
370 41
1032
19951

8. Online vs. Face to face STAT_Grade OM_Grade CLA_Score Type N Mean Min
Max
% of Total
Variance
F2F 65
2.77 1 4
40.00%
0.77
78.7
3.15
99.7
42.51%
260 27
1015
746
1347
60.99%
20432
ONL 89
3.03
60.00%
0.99
77.7 3
103
57.49%
419 17
1031
842
1271
39.01%
20835
All
154
2.92
100.00%
0.91
78.1
350 44
1021
20171

9. Page Views vs. F/O
Graph Builder

10. Participations vs. F/O
Graph Builder

13. Correlations Multivariate Page Views Participations On Time Missing
Activity time
STAT_Grade
CLA_Score
OM_Grade
1.0000
0.5276
0.4661
0.0720
0.3519
0.4189
0.8060
0.2647
0.1174
0.5238
0.2587
0.1098
0.4344
0.0135
0.0078
0.2108
0.0348
0.3668
0.5257
0.4521
Multivariate

14. Correlation Probability
Page Views
Participations
On Time
Missing
Activity time
STAT_Grade
CLA_Score
OM_Grade
<.0001
0.3749
0.7078
0.8504
0.0009
0.4480
0.0716
0.0012
0.4779
0.8678
0.0002
0.1024
0.9236
0.1697
0.6679
0.0143
0.0021
Multivariate

15. Boosted Tree for CLA_Score
RSquare
RMSE N
0.792 44
Term
Number of Splits SS
Portion
STAT_Grade 12
0.2642
OM_Grade 14
0.1643
Participations
0.1555
Page Views 16
0.0981
Number of Messages 10
0.0726
Activity time
0.0567
communication
0.0507
On Time 11
0.0420
Age 24
0.0410
Race
0.0188
Type
0.0168
Late 3
0.0123
Attendance 1
0.0036
Gender 2
0.0018
Missing
0.0015

16. Prediction Profiler
Boosted Tree for CLA_Score

17. Effect Summary Summary of Fit Fit Group > Response OM_Grade
Source
LogWorth
PValue
Missing
25.537
STAT_Grade
11.221
Type
8.976
Summary of Fit
RSquare
RSquare Adj
Root Mean Square Error
Mean of Response
Observations (or Sum Wgts)
154
Source DF
Sum of Squares
Mean Square
F Ratio
Lack Of Fit 52
2.4602
Pure Error 98
79.778
Prob > F
Total Error
150
<.0001*
Max RSq
0.8540
Fit Group > Response OM_Grade
Fit Group > Response OM_Grade

18. Parameter Estimates Analysis of Variance Source DF Sum of Squares
Mean Square
F Ratio
Model 3
Error
150
120.2
Prob > F
C. Total
153
<.0001*
Parameter Estimates
Term
Estimate
Std Error
t Ratio
Prob>|t|
Intercept
20.88
<.0001*
Missing
-12.97
STAT_Grade
7.47
Type[F2F]
6.51
Fit Group > Response OM_Grade

20. Boosted Tree for OM_Grade
RSquare
RMSE N
0.858
154
Term
Number of Splits SS
Portion
Missing 24
0.4247
STAT_Grade 29
0.2307
Page Views
0.1060
On Time 19
0.0965
Participations 6
0.0690
communication 10
0.0243
Attendance 9
0.0213
Type 11
0.0102
Activity time
0.0055
Late 5
0.0052
Age
0.0034
Number of Messages 2
Gender
0.0000
Boosted Tree for OM_Grade

21. Prediction Profiler
Boosted Tree for OM_Grade

22. Regression results

23. Conclusions
Our data shows wide difference among students (for example, variance is large among students' OM and CLA grade, for page views, participation and communication, online students have larger range)
Our data helps us identify key factors for students performance. (for example, the correlation analysis shows that page views, participation and on time submission have strong correlation with OM grade) 
Our analysis helps instructors. (for example, instructors can remind students early when they observe students' performance on those key factors are not good)