Computer Science School of Computing Clemson University Specification and Reasoning in SE Projects Using a Web IDE Charles T. Cook (Clemson) Svetlana V. Drachova-Strang (Limestone College) Yu-Shan Sun (Clemson) Murali Sitaraman (Clemson) Jeffrey C. Carver (Alabama) Joseph E. Hollingsworth (IU Southeast) This research is funded in part by NSF grants CCF , CCF , DUE , and DUE

Computer Science School of Computing Clemson University Part I: Overview

School of Computing Clemson University About Clemson  School of Computing has about 600 undergrads and 200 grads  Clemson University has 17,000 students  Located in a town (also Clemson) with an official population of about 13,000 in South Carolina  Has a football stadium with capacity 85,000 3

School of Computing Clemson University This talk  This talk is about undergrad courses, though the ideas have been used in both undergrad and grad courses for many years 4

School of Computing Clemson University Goals of the SE Projects  Role of formal specifications as contracts in team software development and integration  Role of formal specifications in reasoning about software correctness  Other uses Specification-based test case design 5

School of Computing Clemson University Intro. and Advanced Projects  Introductory projects involve only use of and reasoning with existing library components 2 weeks of lectures  Advanced projects involve development and reasoning about new components 3-5 weeks of lectures  Other variations 6

School of Computing Clemson University Sample Courses and Institutions  Sample introductory projects Alabama (Software Engineering) Cleveland State (Software Engineering)  Sample advanced projects Clemson (Software Engineering) Denison (Independent study projects)  Other variations Depauw (Theory), NC State (Data structures), Southern Wesleyan (Data Structures), Ramapo College (Programming Languages),… 7

School of Computing Clemson University Clemson University Details  Two-course sequence Sophomore-Level, CP SC 215: Software Development Foundations Junior/Senior-Level: CP SC 372: Software Engineering  Experimentation and assessment Piloting (2007/2008) Institutionalization (2009 to present) 8

School of Computing Clemson University Soft. Dev. Foundations Course  Intro to Java, object-based computing, software engineering, design patterns, etc. 10 weeks (includes usual materials and projects)  Intro to formal specifications and reasoning (interspersed with above topics) 4 weeks  Uses RESOLVE-style specifications in a Java context 9

School of Computing Clemson University Software Engineering Course  Software life cycle, process models, requirements analysis and design 8 weeks (includes usual materials and projects)  Transition from informal to formal 1 week  Specification-based component development and quality assurance 5 weeks  Uses RESOLVE 10

School of Computing Clemson University Clemson University Key Points  Specification/reasoning included in syllabi for two required courses for majors (2009)  Graduating student learning outcome modified to include “development of bug-free software according to specifications” (2012) 11

Computer Science School of Computing Clemson University Part II: Example Projects

School of Computing Clemson University RESOLVE  An integrated specification and programming language for verified, component-based software development  A Verifying compiler  A github project  A freely-available web interface to use the verifying compiler: 13

School of Computing Clemson University Sample Intro Assignments  Generate VCs and prove the given Queue Remove_Last operation.  Make each of the following changes and explain what is unprovable. Comment out the first Dequeue operation. Change the maintaining clause (loop invariant) to #Q = o Q. Change the decreasing clause (termination progress metric) to |T|. 14

School of Computing Clemson University Sample Advanced Assignments  Implement and verify: Extension operations on Queues to Insert_After and Remove_After Sequence_Template using Queue_Template with extensions Multiple implementations of Queue_Template satisfying given internal contracts  Assignments to put it all together Involve a dozen components and teams of 3 students 15

School of Computing Clemson University Sample Components  Stack_Template  Queue_Template  Preemptable_Queue_Template  Sequence_Template  List_Template  Search_Store_Template  Map_Template  Prioritizer_Template  … 16

Computer Science School of Computing Clemson University Part III: A Web IDE Demo

School of Computing Clemson University Getting Started  Tab: Web IDE  Google: RESOLVE web IDE RESOLVE verifier Clemson RESOLVE 18

School of Computing Clemson University Sample Intro Assignments  Generate VCs and prove the given Queue Remove_Last operation.  Make each of the following changes and explain what is unprovable. Comment out the first Dequeue operation. Change the maintaining clause (loop invariant) to #Q = o Q. Change the decreasing clause (termination progress metric) to |T|. 19

School of Computing Clemson University Elements of the Assignment  Queue_Template concept specification  A specification of Remove_Last Operation  An annotated implementation of Remove_Last operation 20

School of Computing Clemson University Mathematical Modeling  Concepts provide mathematical models for programming objects  To write formal specifications, we need to model the state mathematically  Some objects we use in programming, such as Integers and Reals, have implicit models  For others, such as stacks, queues, lists, etc., we need to conceive explicit mathematical models

School of Computing Clemson University Mathematical Modeling of Queues Concept Queue_Template(type Entry; Max_Length: Integer); uses String_Theory; Type Family Queue is modeled by … Operation Enqueue… Operation Dequeue… … end Queue_Template;

School of Computing Clemson University Concept Queue_Template(type Entry; Max_Length: Integer); uses String_Theory; Type Family Queue is modeled by Str(Entry); exemplar Q; constraints |Q| <= Max_Length; initialization ensures Q = ; … end Queue_Template; Mathematical Modeling of Queues

School of Computing Clemson University Operation Remove_Last (updates Q: Queue; replaces E: Entry); requires |Q| /= 0; ensures #Q = Q o ; A Specification of Remove_Last

School of Computing Clemson University Procedure Remove_Last (updates Q: Queue; replaces E: Entry); Var T: Queue; Dequeue (E, Q); While (Length(Q) /= 0) do Enqueue(E,T); Dequeue(E,Q); end; Q :=: T; end Remove_Last; An Implementation of Remove_Last

School of Computing Clemson University Assume #Q = Dequeue (E, Q); While (Length(Q) /= 0) T EQ Iter. 1  10 Iter.2 20 Iter.3 30 Iter.4 40  do Enqueue(E,T); Dequeue(E,Q); end; Understanding Loop Invariant

School of Computing Clemson University Assume #Q = Dequeue (E, Q); While (Length(Q) /= 0) T EQ Iter. 1  10 Iter.2 20 Iter.3 30 Iter.4 40  Loop maintains this invariant: #Q = T o o Q Understanding Loop Invariant

School of Computing Clemson University Procedure Remove_Last (updates Q: Queue; replaces E: Entry); Var T: Queue; Dequeue (E, Q); While (Length(Q) /= 0) maintaining #Q = T o o Q; decreasing |Q|; do Enqueue(E,T); Dequeue(E,Q); end; … An Implementation of Remove_Last

School of Computing Clemson University Verification  Press Verify Button  Generate and prove automatically a series of verification conditions (VCs)  Students understand why the VCs arise  Understand connections between contracts, code, and proofs  Can prove VCs 29

School of Computing Clemson University Impact of Incorrect Code  The first statement, call to Dequeue, is removed.  Leads to one unprovable VC (Verification Condition)  VC 0_1: Base Case of the Invariant of While Statement in Procedure Remove_Last: Remove_Last_Realiz.rb(8) Goal: Q = ((empty_string o ) o Q) Givens: … 30

Computer Science School of Computing Clemson University Part IV: Assessment

School of Computing Clemson University Assessment Summary  For details See the paper See Drachova Ph. D. dissertation (2013, Clemson), available at website  A quick summary Likert items to assess perceived benefits of web IDE (avg. 4.0/5.0) Project grades (avg. from 80%); about the same as non-formal parts RCI-item based analysis 32

School of Computing Clemson University What reasoning skills are necessary? Reasoning Concept Inventory Boolean LogicStandard Logic Symbols, Standard Proof Techniques Discrete Math StructuresSets, Strings, Numbers, Relations, and other mathematical theories as needed Precise SpecificationsMathematical Descriptions of Software interfaces for clients and implementers. Math models for structures. Pre and Post conditions for operations. Modular ReasoningEach Module needs to be proven correct only once. Correctness ProofsMathematical Assertions equivalent to the correctness of the program. Application of Proof Techniques to the program

School of Computing Clemson University Clemson RCI-Based Evaluation 34 Sem#Avg.% scoring 70% or higher RCI %85% RCI %59% RCI %67% 3.4.3: Precise specifications 4.2: Design by contract 4.3.1: Internal contracts

School of Computing Clemson University SE Project Benefits on RCI- Based Learning Outcomes 35 Avg.% scoring 70% or higher Before project (mid-term) 59%25% After project (final) 84%75% RCI Topics: 3.4.2, 3.4.3: Precise specifications Sample size: 24 students, Spring ‘12

School of Computing Clemson University Summary  Students can practice formal specification and reasoning principles effectively using the RESOLVE web IDE and its verifier There are some benefits for even a minimal intro (3 lectures plus project) Significant benefits with longer-term exposure  IDE and materials online