Invariant Based Programming in Education Tutorial, FM’08 Linda Mannila

Background Challenges at CS departments ◦ First year students have weak skills in logic, formal reasoning, writing proofs etc. ◦ Students do not see a connection between theoretical and practical courses Goal ◦ Giving students hands-on skills in constructing proofs and reasoning about correctness early on ◦ Linking theory with practice

IBP in Education Course for PhD students in case study sessions with faculty, students, researchers Undergraduate course, given twice, starting in spring 2007

Undergraduate Course Elective (~15 students/course) Prerequisities: ◦ Introductory programming ◦ Logic 17 interactive, example-driven sessions ◦ 4 (7) sessions with SOCOS Addition,..., linear search,..., sorting Evaluation ◦ questionnaires, interviews, observations, assignments, exam

Experienced benefits (+) Introduction to program correctness and formal verification Good introduction to formal verification and how tools can be used in that context Helps remove errors in the algorithm that could lead to bugs A practical method for introducing program correctness IBP summarizes the proof conditions in a good way IBP is intuitive Introduction to a more abstract view of programming Learning to think about how a program works in general, without a given programming language Learned to think about a program as states and transitions instead of merely as transitions as is usually the case More tangible overview of the program structure Learning to draw a program makes it easier to see its structure Makes it easy to keep the track of the various parts of a program New and useful contents

Experienced difficulties (-) Syntax and notation ◦ It’s difficult to formulate one’s programs according to the standard ◦ Since I have programmed previously e.g. the Java way of expressing things is quite ingrained Proofs ◦ Proving programs by hand is very work intense ◦ Proving complex programs is difficult Finding the correct invariant and postcondition Using SOCOS

Preliminary error analysis Only minor problems related to the ◦ Method (e.g. the syntax) ◦ Algorithm Main problem ◦ Weak logical background ◦ Interpreting and constructing quantified expressions ◦ Understanding when to use quantifiers ◦ Understanding which one to use Problems with logic affect other aspects ◦ Unclear invariants (not necessarily weak) ◦ Unclear proofs

Conclusions Learning IBP is not difficult ◦ Students get the big picture rather quickly ◦ Appreciate the diagrammatic notation Main difficulties related to predicate logic ◦ More focus on using logic in practice ◦ Students need more practice in reading and constructing quantified expressions Remedy? ◦ Split the course into two parts: 1) manual and 2) with tool support  More time for practice

Assignment Results (2007) Scores ◦ Average 25.5 (max 40) ◦ Seven students > 30 points Most common errors ◦ Syntax related (e.g. using Java like syntax or not following the given proof format) Other errors ◦ Incomplete invariants ◦ Problems with the algorithm ◦ Using undeclared variables ◦ One “off by one” error

Error analysis (2008) Jonas Erikson, Master’s Thesis Analyzed student solutions to four assignments (all in all ~50 solutions) Three main error categories: ◦ Syntax (related to the diagram representation) ◦ Reasoning (related to misconceptions and weak understanding) ◦ Logical background (related to having a weak background in expressing logical statements)

Total number of errors Nested loops

Related to syntax

Related to reasoning Nested loops

Related to logical background Define own predicates, use quantifiers to a larger extent

Discussion Could IBP be introduced at high school level? ◦ As a course in mathematics? ◦ Would the method need to be simplified? How?