Knowledge Retention from Introductory CS Courses Anthony Baxter Paul Piwowarski University of Kentucky

2 Instructors of senior-level computer science courses at the University of Kentucky have noted deficiencies in the knowledge of senior CS students in basic programming concepts and practices

3 These included: 1. Pointers/use of dynamic storage 2. Pass by reference vs. pass by value and scope of variables 3. Make file and program organization 4. Basic knowledge of machine architecture

4 Cntinued: 5. Debugging 6. Lack of understanding of object- oriented design 7. Inexperience in developing large projects

5 The three required introductory CS courses cover these topics: 1. CS115  Has a large percentage of non-CS majors enrolled (75-80%)  Does not assume that students have previous programming experience  Uses C++ and Microsoft Visual Studio  Lab assignments, debugging taught

6 2. CS215  Continuation of CS115 (same textbook, programming environment, labs)  Has a large percentage of non-CS majors enrolled  OO programming  Data structures  Complexity (Big-O notation)

7 3. CS216  Less than 50% CS majors  Unix platform and tools  Program organization and make file  Program assignments start with C++ and shell scripts  Interpretive language assignments (Perl)  Last program exceeds 1000 lines with multiple interacting programs in different languages  Software engineering/testing  Introduction to Java/Internet

8 1. Lower division instructors insist that students are learning concepts (measured by program assignments, homeworks, exams) 2. Upper division instructors insist that students have not learned concepts

9 Instructors of CS499 (senior project capstone course) gave a quiz to the students over a number of semesters. The quiz (composed of questions from CS216 exams) covered items 1-4 of the list: 1. Pointers/use of dynamic storage 2. Pass by reference vs. pass by value and scope of variables 3. Make file and program organization 4. Basic knowledge of machine architecture

10

11 1. From program assignment results, exam results, students learned these concepts in introductory courses 2. The quiz indicates about a 50% drop in senior comprehension of the concepts

12 “When you have eliminated all which is impossible, then whatever remains, however improbable, must be the truth.” Sherlock Holmes

13 Therefore: Between the introductory courses, and the senior level courses (typically 2-3 years), the student minds have been taken over by creatures from space (see for example, “Invasion of the Body Snatchers” 1956)

14

15 However before notifying “The National Enquirer”, we decided to present the results to the senior students, and ask their opinion on them.

16

17  100% of the students said the material was covered in earlier courses  100% of the students said that they understood the material at that time

18 Reasons that the students gave for their poor performance

19 Nature of the quiz and how administered (no advance notice, did not affect their grade) “The main reason I missed questions on the quiz was a lack of prior preparation.” “I suppose I didn't take the CS 499 quiz as seriously as I would have if it had been graded.” “Just didn’t put too much effort into it.”

20 Lack of recent classes with a programming component after the introductory courses “After the introductory courses that teach these programming concepts, there are no intensive programming courses to reinforce them.” “I haven't used the material recently” “I haven't even had a programming course in the last two years”

21 Students use reference material “…you can sit at home with all your reference material and do the work.” “Like other students said, this is isn’t stuff I would need to do without having references close by to help me.” “If a situation comes up where we need it in the real world, we have reference books, we have Google.”

22 Students are expected to use many different languages “I don't spend a significant amount of time coding in C or C++.” “Most of my class programming projects have involved Java, SQL, Prolog, and other languages.” “…all of my classes last semester covered material in Java, so the use of pointers and make files isn't necessarily easily recalled for an 8am pop quiz.”

23 Students were also asked to comment about senior instructor perceptions of their knowledge in areas not measured by the quiz

24 “Most have never used a debugging tool. Even when given a tool, they typically do not understand what the tool is telling them.” (26% disagreed) Sample student comments: “I agree somewhat. We haven't had the education in class on using gdb or other debugging tool. “ “Some basic debugging, but never taught how to use one.” “I have not used a debugging tool since CS115. All my debugging is done via print statements.”

25 “Students lack an understanding of object-oriented design.” (60% disagreed) Sample student comments: “I disagree. I think we know how object- oriented design can be used to great capacity in building clean, efficient code.” “Was emphasized in early courses.” “Agree. I didn't fully understand this concept till last semester.”

26 “Many students seem intimidated by large projects. Most tell me they have never written a large project (near or exceeding a thousand lines of code)” (38% disagreed) Sample student comments: “I'm intimidated by large projects in general, but have done them.” “This is true, until my senior year I never really did any large projects.” “True, most programs written for class are not huge projects.”

27Summary  After the introductory courses, there are no intensive programming courses. Courses teach concepts in specialized areas (algorithms, compilers, etc.).  There is no reason to memorize information. Students had examples, both online and from previous courses.  A quiz is an imperfect measure of student knowledge, especially a pop quiz that the students knew would not affect their grade.

28 Summary (continued)  There is a significant gap in time (typically two to three years) between the introductory CS courses, and the senior-level courses, and a number of non-CS courses are also taken in this period.  Students are required to use many different languages, not just C/C++ that was the basis of the quiz.

29 Possible Solutions  Add more programming content to courses after the introductory courses.  Reemphasize subject matter covered in the introductory courses that they are expected to use in more advanced classes.  Provide online references and tutorials on core material.  Ensure that programming assignments have high standards and students use good techniques and programming paradigms.

30 Possible Solutions (continued)  Integrate formal testing requirements to more assignments  Have projects that illustrate the benefits of OO techniques. The importance of OO programming is best learned by experience.  Introduce team assignments into more courses.

31 In Conclusion Research may lead to finding causes of phenomena, but will not get you published in “The National Enquirer”