1. Oral Exams to Improve and Assess Actuarial Student Learning
Diana Skrzydlo
Continuing Lecturer, Dept of Statistics and Actuarial Science, University of Waterloo
ARC, Aug 7, 2015

2. Outline What is an Oral Exam? Motivation
Advantages and Challenges of Oral Exams
My Use of Oral Exams in the Actuarial Context
Feedback from Students
Improvements and Recommendations

3. What is an Oral Exam? Various formats used in different fields
Medicine
Thesis defense
Language courses
Viva voce
Etc
Essential point: two-way oral communication

4. Motivation Shortcomings of traditional written tests
Not able to ask for clarification
Not easy to test higher levels of learning
Compare, synthesize, analyze, evaluate
Oral communication: desirable skill in and of itself!
Marking is time-consuming
Oral exams address many of these concerns

5. Benefits of Oral Exams Learning experience for the student
Get a much better sense of depth of knowledge
Preparing for it helps students prepare for written test as well
Preparation for interviews/workplace
Can give help if question is misunderstood/missing some information

6. Disadvantages of Oral Exams
Language barrier
Nerves
Not as objective (the “halo” effect)
Time
Academic dishonesty

7. How to Overcome Disadvantages
Give students clear expectations and lots of time/practice to prepare
Have a clear marking scheme/rubric
Have it not worth an extremely large %
Allow students to bring in notes
Only use when class size permits
Have a bank of questions and randomize

8. My Experience with Oral Exams
As a grad student, research project and took an oral exam in Gord Willmot’s ACTSC 966
As an instructor, have given oral exams in:
STAT 430/830 (Experimental Design)
ACTSC 455/855 (Advanced Life Insurance Practice)
ACTSC 613 (Probability and Statistics for Actuaries)
STAT 334 (Probability Models for Business and Accounting)

9. My Format Exam is worth 5% overall
Open book (can bring in absolutely anything)
5 questions, each out of 5 marks
15 minutes individually scheduled
Can ask for clarification, or I may ask follow-up
Randomized bank of questions ensuring I cover course topics equally
After all exams but before written final, I post point-form “ideal” answers

10. Sample Questions Define ___
Discuss the advantages/disadvantages of ___
Compare and contrast ___ vs ___
Describe the procedure for ___
How would ___ change if ___
Students told they will be asked one of each

11. Sample Questions - Actuarial
Define: non-diversifiable risk, stress test, hybrid pension plan, credited interest rate
Adv/Disadv: stochastic simulation, DC or DB pension plan, ILN model for interest rates
Compare: UL vs Equity-Linked insurance, actuarial reserve vs dynamic hedge portfolio
Describe process: MC simulation, calculating normal cost in DB pension, profit testing, calculating hedge portfolio, generating RSLN-2

12. Sample Questions - Statistics
Define: independence, generating function, MLE, hypothesis test, Markov chain
Adv/Disadv: CLT, paired comparison experiment, F-test in ANOVA table
Compare: Binomial vs Poisson, MME vs MLE, CI vs PI, Type I vs Type II error
Describe process: joint transformation, LSE estimation, Bayesian estimation, generating random walk/Brownian Motion

13. Observations of Students in Exam
Some reported being quite nervous at start but calmed down with encouragement
Many said it was good practice for interviews
Liked the ability to know if they were on the right track – immediate feedback
I find it extremely valuable to assess learning
Sometimes they come up with things I never even considered
A very small number did not prepare at all and it is obvious

14. Comments from Students post Exam
“I think it was a very beneficial component of the course. I think it is important in actuarial science to not only be able to do the math but to be able to explain concepts. It is also good preparation for interviews. I have been asked in interviews to describe certain concepts and the format was very similar to the questions asked in the Oral Exam. Overall it was a positive experience and quite unique in the program.” “The environment was welcoming, understanding, and supportive. If we didn't know the answer right away, you provided guidance to help us along, which made the situation more comfortable. It helped me when studying for the final exam and in my future courses because I actually had to learn to understand the concepts in order to communicate them effectively.”

15. Comments from Students post Exam
“Being able to understand and explain the ideas behind the concepts made it easier to study the more technical aspects leading up to the exam. For example, it is a lot easier to understand and/or memorize the formulas if you understand the underlying idea behind the calculation. From an oral communication perspective, I feel that it was also helpful as well. The evaluation almost felt like an interview - a 1 on 1 session where I have to communicate basic ideas, explain processes, and think on my feet. Overall, I feel that it is a good evaluation that you should keep in your smaller courses - there really aren't many other courses that offer this kind of evaluation, and I think it really helps the student.”

16. Comments from Students post Exam
“The oral exam is really a breakthrough of current exam system. What I heard most from my co-op terms is that UW students have very good technical skills but not soft skills. Oral exam is a good way to show the importance of communication of ActSci students. The questions and the environment is also similar to the real interviews, so it helps for that too. I would recommend you to spread this oral exam to the whole department or faculty so more people can benefit from it.”
“I would say the oral exam helped me understand the material, especially the interpretation of concepts. Our written exams only test our technical skills, so it is a good opportunity to get insight into the content.”

17. Potential Improvements
Give students more practice beforehand
Rather than have it completely open book, suggest students prepare summary notes
Audio recording for objectivity
Improve randomization of questions to account for difficulty

18. Recommendations
Oral Exams can be used effectively in small classes for upper year students
Best if students have lots of time/instruction/ guidelines/practice to prepare
Sufficiently conceptual material important
Take steps to minimize disadvantages
Emphasize purpose/importance as a learning activity as well as an assessment

19. Questions/comments welcome
Thank you
Questions/comments welcome