1. Mary R. Hedges The University of Auckland DEE 11 - LSE

2. Outline Background and motivation ECON 191 Course Data OLS Results Next Steps

3. Background BCom & BProp required full principles split 1 semester Microeconomics followed by 1 semester macroeconomics Degree restructured to introduce: Common core between BCom, BProp & BBIM Make room for two integrated courses (B1 & B2) Goal to make degree more applied. Economics reduced to one core course

4. ECON 191 Needed to meet multiple objectives Prepare for theory degree in economics Prepare students not progressing Meet ANZICA requirements Attract students to an economics major Structure settled on: 8 weeks microeconomics 4 weeks macroeconomics

5. ECON 191 Assessment Goals Large numbers so marking load was a concern but also... wanted students to get early feedback on their understanding Wanted students to be able to apply theory to real world problems (not seen before) Wanted them to remember what they had learned transformational in threshold concept terms or ‘less is more’

6. Assessment Structure 11 weekly quizzes (10 marks) Best ten counted Unlimited attempts but only the best would count Assesses key theory and basic understandings 5 tutorials (10 marks) Applied theory to real world problems Stepped through process of application but... Then had to attempt another one their own and hand it in. Mid-semester test (30 marks) Final exam (50 marks)

7. Early Motivation The mid-semester test, after 6 completed quizzes, generated the following results: 74% pass, 22% fail and 4% DNC. This is in comparison to the same course semester 2, 2010 where the test results were: 49% pass, 48% fail and 3% DNC. Similarly ECON 101 has a test on the same content and the 2010 results for those papers were: Semester 1: 51% pass, 42% fail and 6% DNC. Semester 2: 54% pass, 40% fail and 6% DNC. All of these courses had the same course co-ordinator who was responsible for all of these tests, they all cover the same content, and the courses were taught by the same person (though not all streams) therefore these comparison are reasonable that the only major change was the introduction of weekly quizzes.

8. No Plussage This semester had 23 students out of 594 who did not sit the test (30% weighting) and of those only one of them then sat the exam (and had applied for an aegrotat on medical grounds). The other 22 students did not complete the bulk of the course work in this course. As a comparison: ECON 191, Semester 2, 20116/97 did not sit the test but 4 of these did sit the exam. ECON 101, Semester 1, 201090/1447 did not sit the test and 31 of these did sit the exam. ECON 101, Semester 2, 201039/628 did not sit the test and 20 of these did sit the exam.

9. Analysis Collected data from two datasets: Course assessment data Demographic data from the registry database Initially ran OLS regression on data in order to indentify the best metrics to use in bi-probit models later.

10. OLS - Demographics Initially tried papers taken (no.) and papers taken squared but neither were significant. Ability (measured by cumulative GPA always strongly significant. Being male, young and not a domestic student were all significant at 10% level only and stayed so throughout all the analysis.

11. Quizzes & Tutorials Coefficients a Model Unstandardized Coefficients Standardized Coefficients tSig. BStd. ErrorBeta 1(Constant)12.9054.5532.834.005 BCOM-.329.692-.016-.475.635 BBIM-.112.898-.004-.125.901 Ability2.824.164.61017.177.000 Papers_taken.367.868.012.423.672 Domestic-1.418.712-.058-1.992.047 Pakeha1.2251.647.057.744.457 Mapas.5041.795.016.281.779 Asian.3741.626.019.230.818 Male1.342.534.0672.513.012 Age-.150.078-.052-1.928.054 Tut_mark.306.140.0762.185.029 Quiz_mark.804.162.1874.952.000 a. Dependent Variable: Exam_mark

12. Best Metric for Engagement? Results were very similar when we used the following alternative metrics: # of tutorials attended Total quiz time Median quiz time Avg quiz time # quizzes attempted Reasons for using (and not using all of them).

13. Next steps... Choose best OLS metric Use in a bi-probit model Of those that did x more or less than average... Their performance in y was above or below average of that group. Also have school ability ranking but not complete dataset and worry it will shrink our sample too much.

14. Conclusions Marks for tutorials and weekly quizzes definitely: Increased engagement through the course Were significant in improving final exam performance (and therefore final grade) Even after controlling for ability. New assessment programme seems to be working and tweaks are now being made.