1. S. Haridi and P. Van Roy1 Declarative Programming Techniques Seif Haridi KTH Peter Van Roy UCL

2. S. Haridi and P. Van Roy2 Overview What is declarativeness? –Classification, advantages for large and small programs Iterative and recursive programs Programming with lists and trees –Lists, accumulators, difference lists, trees, parsing, drawing trees Reasoning about efficiency –Time and space complexity, big-oh notation, recurrence equations Higher-order programming –Basic operations, loops, data-driven techniques, laziness, currying User-defined data types –Dictionary, word frequencies –Making types secure: abstract data types The real world –File and window I/O, large-scale program structure, more on efficiency Limitations and extensions of declarative programming

3. S. Haridi and P. Van Roy3 Declarative operations (1) An operation is declarative if whenever it is called with the same arguments, it returns the same results independent of any other computation state A declarative operation is: –Independent (depends only on its arguments, nothing else) –Stateless (no internal state is remembered between calls) –Deterministic (call with same operations always give same results) Declarative operations can be composed together to yield other declarative components –All basic operations of the declarative model are declarative and combining them always gives declarative components

4. S. Haridi and P. Van Roy4 Declarative operation Arguments Results Declarative operations (2) rest of computation

5. S. Haridi and P. Van Roy5 Why declarative components (1) There are two reasons why they are important: (Programming in the large) A declarative component can be written, tested, and proved correct independent of other components and of its own past history. –The complexity (reasoning complexity) of a program composed of declarative components is the sum of the complexity of the components –In general the reasoning complexity of programs that are composed of nondeclarative components explodes because of the intimate interaction between components (Programming in the small) Programs written in the declarative model are much easier to reason about than programs written in more expressive models (e.g., an object-oriented model). –Simple algebraic and logical reasoning techniques can be used

6. S. Haridi and P. Van Roy6 Why declarative components (2) Since declarative components are mathematical functions, algebraic reasoning is possible i.e. substituting equals for equals The declarative model of chapter 4 guarantees that all programs written are declarative Declarative components can be written in models that allow stateful data types, but there is no guarantee

7. S. Haridi and P. Van Roy7 Classification of declarative programming Declarative programming Descriptive Programmable Observational Definitional Declarative model Functional programming Nondeterministic logic programming Deterministic logic programming The word declarative means many things to many people. Let’s try to eliminate the confusion. The basic intuition is to program by defining the what without explaining the how

8. S. Haridi and P. Van Roy8 Descriptive language  s  ::= skip empty statement |  x  =  y  variable-variable binding |  x  =  record  variable-value binding |  s 1   s 2  sequential composition |local  x  in  s 1  end declaration Other descriptive languages include HTML and XML

9. S. Haridi and P. Van Roy9 Descriptive language Seif 48 XML Other descriptive languages include HTML and XML

10. S. Haridi and P. Van Roy10 Kernel language  s  ::= skip empty statement |  x  =  y  variable-variable binding |  x  =  v  variable-value binding |  s 1   s 2  sequential composition |local  x  in  s 1  end declaration | proc {  x   y 1  …  y n  }  s 1  end procedure introduction |if  x  then  s 1  else  s 2  end conditional |{  x   y 1  …  y n  } procedure application |case  x  of  pattern  then  s 1  else  s 2  end pattern matching The following defines the syntax of a statement,  s  denotes a statement

11. S. Haridi and P. Van Roy11 Why the KL is declarative All basic operations are declarative Given the components (substatements) are declarative, –sequential composition –local statement –procedure definition –procedure call –if statement –try statement are all declarative

12. S. Haridi and P. Van Roy12 Structure of this chapter Iterative computation Recursive computation Thinking inductively Lists and trees Control abstraction Higher-order programming User-defined data types Secure abstract data types Modularity Functors and modules Time and space complexity Nondeclarative needs Limits of declarative programming procedural abstraction data abstraction

13. S. Haridi and P. Van Roy13 Iterative computation An iterative computation is a one whose execution stack is bounded by a constant, independent of the length of the computation Iterative computation starts with an initial state S 0, and transforms the state in a number of steps until a final state S final is reached:

14. S. Haridi and P. Van Roy14 The general scheme fun {Iterate S i } if {IsDone S i } then S i else S i+1 in S i+1 = {Transform S i } {Iterate S i+1 } end IsDone and Transform are problem dependent

15. S. Haridi and P. Van Roy15 The computation model STACK : [ R={Iterate S 0 }] STACK : [ S 1 = {Transform S 0 }, R={Iterate S 1 } ] STACK : [ R={Iterate S 1 }] STACK : [ S i+1 = {Transform S i }, R={Iterate S i+1 } ] STACK : [ R={Iterate S i+1 }]

16. S. Haridi and P. Van Roy16 Newton’s method for the square root of a positive real number Given a real number x, start with a guess g, and improve this guess iteratively until it is accurate enough The improved guess g’ is the average of g and x/g:

17. S. Haridi and P. Van Roy17 Newton’s method for the square root of a positive real number Given a real number x, start with a guess g, and improve this guess iteratively until it is accurate enough The improved guess g’ is the average of g and x/g: Accurate enough is defined as: | x – g 2 | / x < 0.00001

18. S. Haridi and P. Van Roy18 SqrtIter fun {SqrtIter Guess X} if {GoodEnough Guess X} then Guess else Guess1 = {Improve Guess X} in {SqrtIter Guess1 X} end Compare to the general scheme: –The state is the pair Guess and X –IsDone is implemented by the procedure GoodEnough –Transform is implemented by the procedure Improve

19. S. Haridi and P. Van Roy19 The program version 1 fun {Sqrt X} Guess = 1.0 in {SqrtIter Guess X} end fun {SqrtIter Guess X} if {GoodEnough Guess X} then Guess else {SqrtIter {Improve Guess X} X} end fun {Improve Guess X} (Guess + X/Guess)/2.0 end fun {GoodEnough Guess X} {Abs X - Guess*Guess}/X < 0.00001 end

20. S. Haridi and P. Van Roy20 Using local procedures The main procedure Sqrt uses the helper procedures SqrtIter, GoodEnough, Improve, and Abs SqrtIter is only needed inside Sqrt GoodEnough and Improve are only needed inside SqrtIter Abs (absolute value) is a general utility The general idea is that helper procedures should not be visible globally, but only locally

21. S. Haridi and P. Van Roy21 Sqrt version 2 local fun {SqrtIter Guess X} if {GoodEnough Guess X} then Guess else {SqrtIter {Improve Guess X} X} end end fun {Improve Guess X} (Guess + X/Guess)/2.0 end fun {GoodEnough Guess X} {Abs X - Guess*Guess}/X < 0.000001 end in fun {Sqrt X} Guess = 1.0 in {SqrtIter Guess X} end end

22. S. Haridi and P. Van Roy22 Sqrt version 3 Define GoodEnough and Improve inside SqrtIter local fun {SqrtIter Guess X} fun {Improve} (Guess + X/Guess)/2.0 end fun {GoodEnough} {Abs X - Guess*Guess}/X < 0.000001 end in if {GoodEnough} then Guess else {SqrtIter {Improve} X} end end in fun {Sqrt X} Guess = 1.0 in {SqrtIter Guess X} end

23. S. Haridi and P. Van Roy23 Sqrt version 3 Define GoodEnough and Improve inside SqrtIter local fun {SqrtIter Guess X} fun {Improve} (Guess + X/Guess)/2.0 end fun {GoodEnough} {Abs X - Guess*Guess}/X < 0.000001 end in if {GoodEnough} then Guess else {SqrtIter {Improve} X} end end in fun {Sqrt X} Guess = 1.0 in {SqrtIter Guess X} end The program has a single drawback: on each iteration two procedure values are created, one for Improve and one for GoodEnough

24. S. Haridi and P. Van Roy24 Sqrt final version fun {Sqrt X} fun {Improve Guess} (Guess + X/Guess)/2.0 end fun {GoodEnough Guess} {Abs X - Guess*Guess}/X < 0.000001 end fun {SqrtIter Guess} if {GoodEnough Guess} then Guess else {SqrtIter {Improve Guess} } end end Guess = 1.0 in {SqrtIter Guess} end The final version is a compromise between abstraction and efficiency

25. S. Haridi and P. Van Roy25 From a general scheme to a control abstraction (1) fun {Iterate S i } if {IsDone S i } then S i else S i+1 in S i+1 = {Transform S i } {Iterate S i+1 } end IsDone and Transform are problem dependent

26. S. Haridi and P. Van Roy26 From a general scheme to a control abstraction (2) fun {Iterate S IsDone Transform} if {IsDone S} then S else S1 in S1 = {Transform S} {Iterate S1} end fun {Iterate S i } if {IsDone S i } then S i else S i+1 in S i+1 = {Transform S i } {Iterate S i+1 } end

27. S. Haridi and P. Van Roy27 Sqrt using the Iterate abstraction fun {Sqrt X} fun {Improve Guess} (Guess + X/Guess)/2.0 end fun {GoodEnough Guess} {Abs X - Guess*Guess}/X < 0.000001 end Guess = 1.0 in {Iterate Guess GoodEnough Improve} end

28. S. Haridi and P. Van Roy28 Sqrt using the Iterate abstraction fun {Sqrt X} {Iterate 1.0 fun {$ G} {Abs X - G*G}/X < 0.000001 end fun {$ G} (G + X/G)/2.0 end } end This could become a linguistic abstraction

29. S. Haridi and P. Van Roy29 Recursive computations Recursive computation is one whose stack size grows linear to the size of some input data Consider a secure version of the factorial function: fun {Fact N} if N==0 then 1 elseif N>0 then N*{Fact N-1} else raise domainError end end This is similar to the definition we saw before, but guarded against domain errors (and looping) by raising an exception

30. S. Haridi and P. Van Roy30 Recursive computation proc {Fact N R} if N==0 then R=1 elseif N>0 then R1 in {Fact N-1 R} R = N*R1 else raise domainError end end

31. S. Haridi and P. Van Roy31 Execution stack [{Fact 5 r 0 }] [{Fact 4 r 1 }, r 0 =5* r 1 ] [{Fact 3 r 2 }, r 1 =4* r 2, r 0 =5* r 1 ] [{Fact 2 r 3 }, r 2 =3* r 3, r 1 =4* r 2, r 0 =5* r 1 ] [{Fact 1 r 4 }, r 3 =2* r 4, r 2 =3* r 3, r 1 =4* r 2, r 0 =5* r 1 ] [{Fact 0 r 5 }, r 4 =1* r 5, r 3 =2* r 4, r 2 =3* r 3, r 1 =4* r 2, r 0 =5* r 1 ] [r 5 =1, r 4 =1* r 5, r 3 =2* r 4, r 2 =3* r 3, r 1 =4* r 2, r 0 =5* r 1 ] [r 4 =1* 1, r 3 =2* r 4, r 2 =3* r 3, r 1 =4* r 2, r 0 =5* r 1 ] [r 3 =2* 1, r 2 =3* r 3, r 1 =4* r 2, r 0 =5* r 1 ] [r 2 =3* 2, r 1 =4* r 2, r 0 =5* r 1 ]

32. S. Haridi and P. Van Roy32 Substitution-based abstract machine The abstract machine we saw in Chapter 4 is based on environments –It is nice for a computer, but awkward for hand calculation We make a slight change to the abstract machine so that it is easier for a human to calculate with –Use substitutions instead of environments: substitute identifiers by their store entities –Identifiers go away when execution starts; we manipulate store variables directly

33. S. Haridi and P. Van Roy33 Iterative Factorial State: {Fact N} (0,{Fact 0})  (1,{Fact 1})  …  (N,{Fact N}) In general: (I,{Fact I}) Termination condition: is I equal to N fun {IsDone I FI } I == N end Transformation : (I,{Fact I})  (I+1, (I+1)*{Fact I}) proc {Transform I FI I1 FI1} I1 = I+1 FI1 = I1*FI end

34. S. Haridi and P. Van Roy34 Iterative Factorial State: {Fact N} (0,{Fact 0})  (1,{Fact 1})  …  (N,{Fact N}) Transformation : (I,{Fact I})  (I+1, (I+1)*{Fact I}) fun {Fact N} fun {FactIter I FI} if I==N then FI else {FactIter I+1 (I+1)*FI} end end {FactIter 0 1} end

35. S. Haridi and P. Van Roy35 Iterative Factorial State: {Fact N} (0,{Fact 0})  (1,{Fact 1})  …  (N,{Fact N}) Transformation : (I,{Fact I})  (I+1, (I+1)*{Fact I}) fun {Fact N} {Iterate t(0 1) fun {$ t(I IF)} I == N end fun {$ t(I IF)} J = I+1 in t(J J*IF) end } end

36. S. Haridi and P. Van Roy36 Iterative Factorial State: {Fact N} (1,5)  (1*5,4)  …  ({Fact N},0) In general: (I,J) Invariant I*{Fact J} == (I*J)*{Fact J-1} == {Fact N} Termination condition: is J equal to 0 fun { IsDone I J} I == 0 end Transformation : (I,J)  (I*J, J-1) proc { Transform I J I1 J1} I1 = I*J J1 = J1-1 end

37. S. Haridi and P. Van Roy37 Programming with lists and trees Defining types Simple list functions Converting recursive to iterative functions Deriving list functions from type specifications State and accumulators Difference lists Trees Drawing trees

38. S. Haridi and P. Van Roy38 User defined data types A list is defined as a special subset of the record datatype A list Xs is either –X|Xr where Xr is a list, or –nil Other subsets of the record datatype are also useful, for example one may define a binary tree (btree) to be: –node(key:K value:V left:LT right:RT) where LT and BT are binary trees, or –leaf This begs for a notation to define concisely subtypes of records

39. S. Haridi and P. Van Roy39 Defining types  list  ::=  value  |  list  [] nil defines a list type where the elements can be of any type  list T  ::= T |  list  [] nil defines a type function that given the type of the parameter T returns a type, e.g.  list  int    btree T  ::= node(key:  literal  value:T left:  btree T  right:  btree T  ) [] leaf(key:  literal  value:T) Procedure types are denoted by proc{T1 … Tn} Function types are denoted by fun{T1 … Tn}:T and is equivalent to proc{T1 … Tn T} Examples: fun{  list   list  }:  list 

40. S. Haridi and P. Van Roy40 Lists General Lists have the following definition  list T  ::= T |  list  [] nil The most useful elementary procedures on lists can be found in the Base module List of the Mozart system Induction method on lists, assume we want to prove a property P(Xs) for all lists Xs 1.The Basis: prove P(Xs) for Xs equals to nil, [X], and [X Y] 2.The Induction step: Assume P(Xs) hold, and prove P(X|Xs) for arbitrary X of type T

41. S. Haridi and P. Van Roy41 Constructive method for programs on lists General Lists have the following definition  list T  ::= T |  list T  [] nil The task is to write a program {Task Xs 1 … Xs n } 1.Select one or more of the arguments Xs i 2.Construct the task for Xs i equals to nil, [X], and [X Y] 3.The recursive step: assume {Task … Xs i …} is constructed, and design the program for {Task … X|Xs i …} for arbitrary X of type T

42. S. Haridi and P. Van Roy42 Simple functions on lists Some of these functions exist in the library module List : 1.{Nth Xs N}, returns the N th element of Xs 2.{Append Xs Ys}, returns a list which is the concatenation of Xs followed by Ys 3.{Reverse Xs} returns the elements of Xs in a reverse order, e.g. {Reverse [1 2 3]} is [3 2 1] 4.Sorting lists, MergeSort 5.Generic operations of lists, e.g. performing an operation on all the elements of a list, filtering a list with respect to a predicate P

43. S. Haridi and P. Van Roy43 The Nth function Define a function that gets the N th element of a list Nth is of type fun{$  list T   int  }:  T , Reasoning: select N, two cases N=1, and N>1: N=1:{Nth Xs 1}  Xs.1 N>1: assume we have the solution for {Nth Xr N-1} for a smaller list Xr, then {Nth X|Xr N}  {Nth Xr N-1} fun {Nth Xs N} X|Xr = Xs in if N==1 then X elseif N>1 then {Nth Xr N-1} end

44. S. Haridi and P. Van Roy44 The Nth function fun {Nth Xs N} X|Xr = Xs in if N==1 then X elseif N>1 then {Nth Xr N-1} end fun {Nth Xs N} if N==1 then Xs.1 elseif N>1 then {Nth Xs.2 N-1} end

45. S. Haridi and P. Van Roy45 The Nth function Define a function that gets the Nth element of a list Nth is of type fun{$  list T   int  }:  T , fun {Nth Xs N} if N==1 then Xs.1 elseif N>1 then {Nth Xs.2 N-1} end There are two situations where the program fails: –N > length of Xs, (we get a situation where Xs is nil) or –N is not positive, (we get a missing else condition) Getting the n th element takes time proportional to n

46. S. Haridi and P. Van Roy46 The Member function Member is of type fun{$  value   list  value  }:  bool , fun {Member E Xs} case Xs of nil then false [] X|Xr then if X==E then true else {Member E Xr} end end X==E orelse {Member E Xr} is equivalent to if X==E then true else {Member E Xr} end In the worst case, the whole list Xs is traversed, i.e., worst case behavior is the length of Xs, and on average half of the list

47. S. Haridi and P. Van Roy47 The Append function fun {Append Xs Ys} case Xs of nil then Ys [] X|Xr then X|{Append Xr Ys} end The inductive reasoning is on the first argument Xs Appending Xs and Ys is proportional to the length of the first list declare Xs0 = [1 2] Ys = [a b] Zs0 = {Append Xs0 Ys} Observe that Xs0, Ys0 and Zs0 exist after Append A new copy of Xs0, call it Xs0’, is constructed with an unbound variable attached to the end: 1|2|X’, thereafter X’ is bound to Ys

48. S. Haridi and P. Van Roy48 The Append function proc {Append Xs Ys Zs} case Xs of nil then Zs = Ys [] X|Xr then Zr in Zs = X|Zr {Append Xr Ys Zr} end declare Xs0 = [1 2] Ys = [a b] Zs0 = {Append Xs0 Ys} Observe that Xs0, Ys and Zs0 exist after Append A new copy of Xs0, call it Xs0’, is constructed with an unbound variable attached to the end: 1|2|X’, thereafter X’ is bound to Ys

49. S. Haridi and P. Van Roy49 Append execution (overview) Stack: [{Append 1|2|nil [a b] zs0}] Store: {zs0,...} Stack: [ {Append 2|nil [a b] zs1} ] Store: {zs0 = 1|zs1, zs1,... } Stack: [ {Append nil [a b] zs2} ] Store: {zs0 = 1|zs1, zs1=2|zs2, zs2,...} Stack: [ zs2 = [a b] ]Store: {zs0 = 1|zs1, zs1=2|zs2, zs2,...} Stack: [ ]Store: {zs0 = 1|zs1, zs1=2|zs2, zs2= a|b|nil,...}

50. S. Haridi and P. Van Roy50 Reverse fun {Reverse Xs} case Xs of nil then nil [] X|Xr then {Append {Reverse Xr} [X]} end Xs0 = 1 | [2 3 4] Xs1 reverse of Xs1[4 3 2] append [4 3 2] and [1]

51. S. Haridi and P. Van Roy51 Length fun {Length Xs} case Xs of nil then 0 [] X|Xr then 1+{Length Xr} end Inductive reasoning on Xs

52. S. Haridi and P. Van Roy52 Merging two sorted lists {Merge [3 5 10] [2 5 6]}  [2 3 5 5 6 10] fun{Merge  list T   list T  }:  list T , where T is either  int ,  float , or  atom  fun {Merge Xs Ys} case Xs # Ys of nil # Ys then Ys [] Xs # nil then Xs [] (X|Xr) # (Y|Yr) then if X =< Y then X|{Merge Xr Ys} else Y|{Merge Xs Yr} end end end

53. S. Haridi and P. Van Roy53 Sorting with Mergesort {MergeSort  list T  }:  list T  ; T is either  int ,  float ,  atom  MergeSort uses a divide-and-conquer strategy 1.Split the list into two smaller lists of roughly equal size 2.Use MergeSort (recursively) to sort the two smaller lists 3.Merge the two sorted lists to get the final result

54. S. Haridi and P. Van Roy54 Sorting with Mergesort L L1 L2 L11 L12 L21 L22 S11 S12 S21 S22 S S1 S2 split merge

55. S. Haridi and P. Van Roy55 Sorting with Mergesort {MergeSort  list T  }:  list T  ; T is either  int ,  float ,  atom  MergeSort uses a divide-and-conquer strategy 1.Split the list into two smaller lists of roughly equal size 2.Use MergeSort (recursively) to sort the two smaller lists 3.Merge the two sorted lists to get the final result fun {MergeSort Xs} case Xs of nil then nil [][X] then Xs else Ys Zs in {Split Xs Ys Zs} {Merge {MergeSort Ys} {MergeSort Zs}} end end

56. S. Haridi and P. Van Roy56 Split proc {Split Xs Ys Zs} case Xs of nil then Ys = nil Zs = nil [] [X] then Ys = Xs Zs = nil [] X1|X2|Xr then Yr Zr in Ys = X1|Yr Zs = X2|Zr {Split Xr Yr Zr} end end

57. S. Haridi and P. Van Roy57 Exercise The merge sort, described above is an example of divide and conquer problem-solving strategy. Another simpler (but less efficient) is insertion sort. This sorting algorithm is defined as follows: 1.To sort a list Xs of zero or one element, return Xs 2.To sort a list Xs of the form X1|X2|Xr, 1.sort X2|Xr to get Ys 2.insert X1 in the right position in Ys, to get the result

58. S. Haridi and P. Van Roy58 Converting recursive into iterative computation Consider the view of state transformation instead: S 0  S 1  S 2 ...  S i ...  S f At S 0 no elements are counted At S 1 one elements is counted At S i i elements are counted At S n where n is final state all elements are counted The state is in general a pair (I, Ys) I is the length of the initial prefix of the list Xs that is counted, and Ys is the suffix of Xs that remains to count Consider % Length :: fun{$ }: fun {Length Xs} case Xs of nil then 0 [] X|Xr then 1+{Length Xr} end

59. S. Haridi and P. Van Roy59 Converting recursive into iterative computation The state is in general a pair (I, Ys) I is the length of the initial prefix of the list Xs that is counted, and Ys is the suffix of Xs that remains to count fun {IterLength I Xs} case Xs of nil then I [] X|Xr then {IterLength I+1 Xr} end fun {Length Xs} {IterLength 0 Xs} end x 1 x 2... x i x i+1... x n Ys Xs Assume Ys is Y|Yr (I, Y| Yr)  (I+1, Yr) {Length Xs} = I + {Length Y| Yr} = (I+1) + {Length Yr} =... = N+{Length nil}

60. S. Haridi and P. Van Roy60 State invariants PGiven a state S I that is (I, Ys): the property P(S I ) defined as {Length Xs} = I + {Length Yr} is called state invariant fun {IterLength I Xs} case Xs of nil then I [] X|Xr then {IterLength I+1 Xr} end fun {Length Xs} {IterLength 0 Xs} end x 1 x 2... x i x i+1... x n Ys Xs Induction using state invariants: P 1.prove P(S 0 ) (0, Xs) PP 2.assume P(S I ), prove P(S I+1 ) PP 3.conclusion: for all I, P(S I ) holds (in particular P(S final ) holds

61. S. Haridi and P. Van Roy61 Reverse fun {Reverse Xs} case Xs of nil then nil [] X|Xr then {Append {Reverse Xr} [X]} end This program does a recursive computation Let us define another program that is iterative (Initial list is Xs) (nil, x 1 | x 2... | x i | x i+1 |... | x n |nil) (x 1 |nil, x 2... x i x i+1... x n ) (x 2 |x 1 |nil,... x i x i+1... x n )

62. S. Haridi and P. Van Roy62 Reverse fun {IterReverse Rs Xs} case Xs of nil then Rs [] X|Xr then {IterReverse X|Rs Xr} end fun {Reverse Xs} {IterReverse nil Xs} end

63. S. Haridi and P. Van Roy63 Nested lists  nestedList T  ::= nil [] T |  nestedList T  []  nestedList T  |  nestedList T  we also add the condition that T  nil, and T  X|Xr for some X and Xr Given an input Ls of type  nestedList T  count the number of elements in Ls Induction of the type  nestedList T 

64. S. Haridi and P. Van Roy64 Nested lists fun {IsCons Xs} case Xs of X|Xr then true else false end end fun {IsLeaf X} {Not {IsCons X}} andthen X \= nil end fun {LengthL Xs} case Xs of nil then 0 [] X|Xr andthen {IsLeaf X} then 1 + {LengthL Xr} [] X|Xr andthen {Not {IsLeaf X}} then {LengthL X} + {LengthL Xr} end

65. S. Haridi and P. Van Roy65 Nested lists fun {Flatten Xs} case Xs of nil then nil [] X|Xr andthen {IsLeaf X} then X|{Flatten Xr} [] X|Xr andthen {Not {IsLeaf X}} then {Append {Flatten X} {Flatten Xr}} end

66. S. Haridi and P. Van Roy66 Accumulators Accumulator programming is a way to handle state in declarative programs. It is a programming technique that uses arguments to carry state, transform the state, and pass it to the next procedure. Assume that the state S consists of a number of components to be transformed individually: S = (X,Y,Z,...) For each procedure P, each state component is made into a pair, the first component is the input state and the second components is the output state after P has terminated S is represented as (X in, X out,Y in, Y out,Z in, Z out,...)

67. S. Haridi and P. Van Roy67 Accumulators Assume that the state S consists of a number of components to be transformed individually: S = (X,Y,Z,) Assume P1 to Pn are procedures: proc {P X 0 X Y 0 Y Z 0 Z} : {P1 X 0 X 1 Y 0 Y 1 Z 0 Z 1 } {P2 X 1 X 2 Y 1 Y 2 Z 1 Z 2 } : {Pn X n-1 X Y n-1 Y Z n-1 Z} end The procedural syntax is easier to use if there is more than one accumulator accumulator

68. S. Haridi and P. Van Roy68 Example Consider a variant of MergeSort with accumulator proc {MergeSort1 N S0 S Xs} –N is an integer, –S0 is an input list to be sorted –S is the remainder of S0 after the first N elements are sorted –Xs is the sorted first N elements of S0 The pair (S0, S) is an accumulator The definition is in a procedural syntax because it has two outputs S and Xs

69. S. Haridi and P. Van Roy69 Example (2) fun {MergeSort Xs} {MergeSort1 {Length X} Xs _ Ys} Ys end proc {MergeSort1 N S0 S Xs} if N==0 then S = S0 Xs = nil elseif N ==1 then X|S = S0 [X] else % N > 1 S1 Xs1 Xs2 NL = N div 2 NR = N - NL {MergeSort1 NL S0 S1 Xs1} {MergeSort1 NR S1 S Xs2} Xs = {Merge Xs1 Xs2} end

70. S. Haridi and P. Van Roy70 Multiple accumulators Consider a stack machine for evaluating arithmetic expressions Example: (1+4)-3 The machine executes the following instructions push(1) push(4) plus push(3) minus 1 4 5 5 3 2

71. S. Haridi and P. Van Roy71 Multiple accumulators (2) Example: (1+4)-3 The arithmetic expressions are represented as trees: minus(plus(1 4) 3) Write a procedure that takes arithmetic expressions represented as trees and output a list of stack machine instructions and counts the number of instructions proc {ExprCode Expr Cin Cout Nin Nout} Cin: initial list of instructions Cout: final list of instructions Nin: initial count Nout: final count

72. S. Haridi and P. Van Roy72 Multiple accumulators (3) proc {ExprCode Expr C0 C N0 N} case Expr of plus(Expr1 Expr2) then C1 N1 in C1 = plus|C0 N1 = N0 + 1 {SeqCode [Expr2 Expr1] C1 C N1 N} [] minus(Expr1 Expr2) then C1 N1 in C1 = minus|C0 N1 = N0 + 1 {SeqCode [Expr2 Expr1] C1 C N1 N} [] I andthen {IsInt I} then C = push(I)|C0 N = N0 + 1 end

73. S. Haridi and P. Van Roy73 Multiple accumulators (4) proc {ExprCode Expr C0 C N0 N} case Expr of plus(Expr1 Expr2) then C1 N1 in C1 = plus|C0 N1 = N0 + 1 {SeqCode [Expr2 Expr1] C1 C N1 N} [] minus(Expr1 Expr2) then C1 N1 in C1 = minus|C0 N1 = N0 + 1 {SeqCode [Expr2 Expr1] C1 C N1 N} [] I andthen {IsInt I} then C = push(I)|C0 N = N0 + 1 end proc {SeqCode Es C0 C N0 N} case Es of nil then C = C0 N = N0 [] E|Er then N1 C1 in {ExprCode E C0 C1 N0 N1} {SeqCode Er C1 C N1 N} end

74. S. Haridi and P. Van Roy74 Shorter version (4) proc {ExprCode Expr C0 C N0 N} case Expr of plus(Expr1 Expr2) then {SeqCode [Expr2 Expr1] plus|C0 C N0 + 1 N} [] minus(Expr1 Expr2) then {SeqCode [Expr2 Expr1] minus|C0 C N0 + 1 N} [] I andthen {IsInt I} then C = push(I)|C0 N = N0 + 1 end proc {SeqCode Es C0 C N0 N} case Es of nil then C = C0 N = N0 [] E|Er then N1 C1 in {ExprCode E C0 C1 N0 N1} {SeqCode Er C1 C N1 N} end

75. S. Haridi and P. Van Roy75 Functional style (4) fun {ExprCode Expr t(C0 N0) } case Expr of plus(Expr1 Expr2) then {SeqCode [Expr2 Expr1] t(plus|C0 N0 + 1)} [] minus(Expr1 Expr2) then {SeqCode [Expr2 Expr1] t(minus|C0 N0 + 1)} [] I andthen {IsInt I} then t(push(I)|C0 N0 + 1) end fun {SeqCode Es T} case Es of nil then T [] E|Er then T1 = {ExprCode E T} in {SeqCode Er T1} end

76. S. Haridi and P. Van Roy76 The lecture Based on Chapter 5 (updated version exist on the web page) Difference lists Programming with trees Higher order programming Modularity and program structure Stand-alone applications

77. S. Haridi and P. Van Roy77 Difference lists (1) A difference list is a pair of lists, each might have an unbound tail, with the invariant that the one can get the second list by removing zero or more elements from the first list X # X% Represent the empty list nil # nil% idem [a] # [a]% idem (a|b|c|X) # X % Represents [a b c] [a b c d] # [d]% idem

78. S. Haridi and P. Van Roy78 Difference lists (2) When the second list is unbound, an append operation with another difference list takes constant time fun {AppendD D1 D2} S1 # E1 = D1 S2 # E2 = D1 in E1 = S2 S1 # E2 end local X Y in {Browse {AppendD (1|2|3|X)#X (4|5|Y)#Y}} end Displays (1|2|3|4|5|Y)#Y

79. S. Haridi and P. Van Roy79 A FIFO queue with difference lists (1) A FIFO queue is a sequence of elements with an insert and a delete operation. –Insert adds an element to one end and delete removes it from the other end Queues can be implemented with lists. If L represents the queue content, then inserting X gives X|L and deleting X gives {ButLast L X} (all elements but the last). –Delete is inefficient: it takes time proportional to the number of queue elements With difference lists we can implement a queue with constant-time insert and delete operations –The queue content is represented as q(N S E), where N is the number of elements and S#E is a difference list representing the elements

80. S. Haridi and P. Van Roy80 A FIFO queue * with difference lists (2) Inserting ‘b’: –In: q(1 a|T T) –Out: q(2 a|b|U U) Deleting X: –In: q(2 a|b|U U) –Out: q(1 b|U U) and X=a Difference list allows operations at both ends N is needed to keep track of the number of queue elements fun {NewQueue} X in q(0 X X) end fun {Insert Q X} case Q of q(N S E) then E1 in E=X|E1 q(N+1 S E1) end end proc {Delete Q Q1 X} case Q of q(N S E) then S1 in X|S1=S Q1 = q(N-1 S1 E) end end fun {EmptyQueue} case Q of q(N S E) then N==0 end end

81. S. Haridi and P. Van Roy81 Flatten (revisited) fun {Flatten Xs} case Xs of nil then nil [] X|Xr andthen {IsLeaf X} then X|{Flatten Xr} [] X|Xr andthen {Not {IsLeaf X}} then {Append {Flatten X} {Flatten Xr}} end Remember the Flatten function we wrote before? Let us replace lists by difference lists and see what happens.

82. S. Haridi and P. Van Roy82 Flatten with difference lists (1) Flatten of nil is X#X Flatten of X|Xr is Y1#Y where –flatten of X is Y1#Y2 –flatten of Xr is Y3#Y –equate Y2 and Y3 Flatten of a leaf X is (X|Y)#Y Here is what it looks like as text

83. S. Haridi and P. Van Roy83 Flatten with difference lists (2) proc {FlattenD Xs Ds} case Xs of nil then Y in Ds = Y#Y [] X|Xr then Y0 Y1 Y2 in Ds = Y0#Y2 {Flatten X Y0#Y1} {Flatten Xr Y1#Y2} [] X andthen {IsLeaf X} then Y in (X|Y)#Y end fun {Flatten Xs} Y in {FlattenD Xs Y#nil} Y end Here is the new program. It is much more efficient than the first version.

84. S. Haridi and P. Van Roy84 Reverse (revisited) Here is our recursive reverse: Rewrite this with difference lists: –Reverse of nil is X#X –Reverse of X|Xs is Y1#Y, where reverse of Xs is Y1#Y2, and equate Y2 and X|Y fun {Reverse Xs} case Xs of nil then nil [] X|Xr then {Append {Reverse Xr} [X]} end

85. S. Haridi and P. Van Roy85 Reverse with difference lists (1) The naive version takes time proportional to the square of the input length Using difference lists in the naive version makes it linear time We use two arguments Y1 and Y instead of Y1#Y With a minor change we can make it iterative as well fun {ReverseD Xs} proc {ReverseD Xs Y1 Y} case Xs of nil then Y1=Y [] X|Xs then Y2 in {ReverseD Xr Y1 Y2} Y2 = X|Y end end R in {ReverseD Xs R nil} R end

86. S. Haridi and P. Van Roy86 Reverse with difference lists (2) fun {ReverseD Xs} proc {ReverseD Xs Y1 Y} case Xs of nil then Y1=Y [] X|Xs then {ReverseD Xr Y1 X|Y} end end R in {ReverseD Xs R nil} R end

87. S. Haridi and P. Van Roy87 Difference lists: summary Difference lists are a way to represent lists in the declarative model such that one append operation can be done in constant time –A function that builds a big list by concatenating together lots of little lists can usually be written efficiently with difference lists –The function can be written naively, using difference lists and append, and will be efficient when the append is expanded out Difference lists are declarative, yet have some of the power of destructive assignment –Because of the single-assignment property of the dataflow variable Difference lists originate from Prolog

88. S. Haridi and P. Van Roy88 Trees Next to lists, trees are the most important inductive data structure A tree is either a leaf node or a node containing one or more subtrees While a list has a linear structure, a tree can have a branching structure There are an enormous number of different kinds of trees. Here we will focus on one kind, ordered binary trees. Later on we will see other kinds of trees.

89. S. Haridi and P. Van Roy89 Ordered binary trees  btree T1  ::= tree(key:T1 value:  value   btree T1   btree T1  ) [] leaf Binary: each non-leaf node has two subtrees Ordered: keys of left subtree < key of node < keys of right subtree key:3 value:x key:1 value:ykey:5 value:z leaf

90. S. Haridi and P. Van Roy90 Search trees Search tree: A tree used for looking up, inserting, and deleting information Let us define three operations: –{Lookup X T}: returns the value corresponding to key X –{Insert X V T}: returns a new tree containing the pair (X,V) –{Delete X T}: returns a new tree that does not contain key X

91. S. Haridi and P. Van Roy91 Looking up information There are four cases: –X is not found –X is found –X might be in the left subtree –X might be in the right subtree fun {Lookup X T} case T of leaf then notfound [] tree(key:Y value:V T1 T2) andthen X==Y then found(V) [] tree(key:Y value:V T1 T2) andthen X<Y then {Lookup X T1} [] tree(key:Y value:V T1 T2) andthen X>Y then {Lookup X T2} end

92. S. Haridi and P. Van Roy92 Inserting information There are four cases: –(X,V) is inserted immediately –(X,V) replaces an existing node with same key –(X,V) is inserted in the left subtree –(X,V) is inserted in the right subtree fun {Insert X V T} case T of leaf then tree(key:X value:V leaf leaf) [] tree(key:Y value:W T1 T2) andthen X==Y then tree(key:X value:V T1 T2) [] tree(key:Y value:W T1 T2) andthen X<Y then tree(key:Y value:W {Insert X V T1} T2) [] tree(key:Y value:W T1 T2) andthen X>Y then tree(key:Y value:W T1 {Insert X V T2}) end

93. S. Haridi and P. Van Roy93 Deleting information (1) There are four cases: –(X,V) is not in the tree –(X,V) is deleted immediately –(X,V) is deleted from the left subtree –(X,V) is deleted from the right subtree Right? Wrong! fun {Delete X T} case T of leaf then leaf [] tree(key:Y value:W T1 T2) andthen X==Y then leaf [] tree(key:Y value:W T1 T2) andthen X<Y then tree(key:Y value:W {Delete X T1} T2) [] tree(key:Y value:W T1 T2) andthen X>Y then tree(key:Y value:W T1 {Delete X T2}) end

94. S. Haridi and P. Van Roy94 Binary tree deletion A A

95. S. Haridi and P. Van Roy95 Binary tree deletion A X A ? Deleting X from a binary tree. The problem is how to patch up the tree after X is removed

96. S. Haridi and P. Van Roy96 Binary tree deletion X Filling the gap after removal of X Y Y remove X transfer Y

97. S. Haridi and P. Van Roy97 Deleting information (2) The problem with the previous program is that it does not correctly delete a non-leaf node To do it right, the tree has to be reorganized: –A new element has to take the place of the deleted one –It can be the smallest of the right subtree or the largest of the left subtree fun {Delete X T} case T of leaf then leaf [] tree(key:Y value:W T1 T2 ) andthen X==Y then case {RemoveSmallest T2} of none then T1 [] Yp#Wp#Tp then tree(key:Yp value:Wp T1 Tp) end [] tree(key:Y value:W T1 T2) andthen X<Y then tree(key:Y value:W {Delete X T1} T2) [] tree(key:Y value:W T1 T2) andthen X>Y then tree(key:Y value:W T1 {Delete X T2}) end

98. S. Haridi and P. Van Roy98 Deleting information (3) To remove the root node Y, there are two possibilities: One subtree is a leaf. Result is the other subtree. Neither subtree is a leaf. Result is obtained by removing an element from one of the subtrees. T1 Y Y T2T1 Yp Tp leaf

99. S. Haridi and P. Van Roy99 Deleting information (4) The function {RemoveSmallest T} removes the smallest element from T and returns the triple Xp#Vp#Tp, where (Xp,Vp) is the smallest element and Tp is the remaining tree fun {RemoveSmallest T} case T of leaf then none [] tree(key:X value:V T1 T2) then case {RemoveSmallest T2} of none then X#V#T2 [] Xp#Vp#Tp then Xp#Vp#tree(key:X value:V T1 Tp) end

100. S. Haridi and P. Van Roy100 Deleting information (5) Why is deletion complicated? It is because the tree satisfies a global condition, namely that it is ordered The deletion operation has to work to maintain the truth of this condition Many tree algorithms rely on global conditions and have to work hard to maintain them

101. S. Haridi and P. Van Roy101 Depth-first traversal An important operation for trees is visiting all the nodes There are many orders in which the nodes can be visited The simplest is depth-first traversal, in which one subtree is completely visited before the other is started Variants are infix, prefix, and postfix traversals fun {DFS T} case T of leaf then skip [] tree(key:X value:V T1 T2) then {DFS T1} {Browse X#V} {DFS T2} end

102. S. Haridi and P. Van Roy102 Breadth-first traversal A second basic traversal order is breadth-first In this order, all nodes of depth n are visited before nodes of depth n+1 The depth of a node is the distance to the root, in number of hops This is harder to implement than depth- first; it uses a queue of subtrees fun {BFS T} fun {TreeInsert Q T} if T\=leaf then {Insert Q T} else Q end end proc {BFSQueue Q1} if {EmptyQueue Q1} then skip else X Q2={Delete Q1 X} tree(key:K value:V L R)=X in {Browse K#V} {BFSQueue {TreeInsert {TreeInsert Q2 R} L}} end in {BFSQueue {TreeInsert {NewQueue} T}} end

103. S. Haridi and P. Van Roy103 Balanced trees We look at ordered binary trees that are approximately balanced A tree T with n nodes is approximately balanced if the height of T is at most c.log 2 (n+1), for some constant c Insertion and deletion operations are in the order of O(log n) Red-black trees preserve this property: the height of a RB- tree is at most 2.log 2 (n+1) where n is the number of nodes in the tree

104. S. Haridi and P. Van Roy104 Red Black trees Red Black trees have the following properties: 1.Every node is either red or black 2.If a node is red, then both its children are black 3.Each path from a node to all descendant leaves contains the same number of black nodes It follows that the shortest path from the root to a leaf has all black nodes and is of length log(n+1) for a tree of n nodes The longest path has alternating black-red nodes and of length 2.log(n+1) Insertion and deletion algorithms preserve the ’balanced’ tree property

105. S. Haridi and P. Van Roy105 Example 26 4117 2114 4730 1910 7 28162338 12 11

106. S. Haridi and P. Van Roy106 Insertion at leaf 10 5 12 10 ? or 10 ? 5 12 10 or Has to be fixed

107. S. Haridi and P. Van Roy107 Insertion at internal node (left) 10 5 6 ?? 12 Insertion may lead to violation of red-black property

108. S. Haridi and P. Van Roy108 Insertion at internal node (left) Y X T1 Z T3T3 T4T4 Y XZ T3T3T4T4 Right Rotate T2T2 Correction is done by rotation (to the right in this case) T1T2T2

109. S. Haridi and P. Van Roy109 Insertion at internal node (left) Y X T1 Z T3T3 T4T4 Y XZ T3T3T4T4 Right Rotate T2T2 Correction is done by rotation (to the right in this case) T1T2T2

110. S. Haridi and P. Van Roy110 Insertion at internal node (right) Y Z T1 X T3T3 T2T2 Left Rotate T4 Y XZ T3T3T4T4 T1T2T2

111. S. Haridi and P. Van Roy111 Insertion at internal node (right) Z Y T1 X T3T3T2T2 Left Rotate T4 Y XZ T3T3T4T4 T1T2T2

112. S. Haridi and P. Van Roy112 Program ::= red [] black ::= nil [] node( ) declare fun {Insert X T} node(C Y T1 T2) = {InsertRot X T} in node(black Y T1 T2) end

113. S. Haridi and P. Van Roy113 Program fun {InsertRot X T} case T of nil then node(red X nil nil) [] node(C Y T1 T2) andthen X == Y then T [] node(C Y T1 T2) andthen X < Y then T3 = {InsertRot X T1} in {RotateRight node(C Y T3 T2)} [] node(C Y T1 T2) andthen X > Y then T3 = {InsertRot X T2} in {RotateLeft node(C Y T1 T3)} end

114. S. Haridi and P. Van Roy114 Insertion at internal node (left) Y X Z T3T3 T4T4 Y XZ T3T3T4T4 Right Rotate T1T1T2T2 T1T1T2T2 fun {RotateRight T} node(C Z LT T4) = T in case LT of node(red Y node(red X T1 T2) T3) then node(red Y node(black X T1 T2) node(black Z T3 T4)) [] node(red X T1 node(red Y T2 T3)) then node(red Y node(black X T1 T2) node(black Z T3 T4)) else T end end

115. S. Haridi and P. Van Roy115 Insertion at internal node (right) Z Y X T3T3 T4T4 Y XZ T3T3T4T4 Left Rotate T1T1T2T2 T1T1 T2T2 fun {RotateLeft T} node(C X T1 RT) = T in case RT of node(red Z node(red Y T2 T3) T4) then node(red Y node(black X T1 T2) node(black Z T3 T4)) [] node(red Y T2 node(red Z T3 T4)) then node(red Y node(black X T1 T2) node(black Z T3 T4)) else T end end

116. S. Haridi and P. Van Roy116 Drawing trees

117. S. Haridi and P. Van Roy117 Reasoning about efficiency Computational efficiency is two things: –Execution time needed (e.g., in seconds) –Memory space used (e.g., in memory words) The kernel language + its semantics allow us to calculate the execution time up to a constant factor –For example, execution time is proportional to n 2, if input size is n To find the constant factor, it is necessary to measure what happens in the implementation (e.g., « wall-clock » time) –Measuring is the only way that really works; there are so many optimizations in the hardware (caching, super-scalar execution, virtual memory,...) that calculating exact time is almost impossible Let us first see how to calculate the execution time up to a constant factor

118. S. Haridi and P. Van Roy118 Big-oh notation We will give the computational efficiency of a program in terms of the « big-oh » notation O(f(n)) Let T(n) be a function that is the execution time of some program, measured in the size of the input n Let f(n) be some function defined on nonnegative integers T(n) is of O(f(n)) if –T(n)  c.f(n) for some positive constant c, except for some small values of n  n 0 Sometimes this is written T(n)=O(f(n)). Be careful! –If g(n)=O(f(n)) and h(n)=O(f(n)) then it is not true that g(n)=h(n)!

119. S. Haridi and P. Van Roy119 Calculating execution time We will use the kernel language as a guide to calculate the time Each kernel operation has an execution time (that may be a function of the « size » of its arguments) To calculate the execution time of a program: –Start with the function definitions written in the kernel language –Calculate the time of each function in terms of its definition Assume each function’s time is a function of the « size » of its input arguments –This gives a series of recurrence equations There may be more than one equation for a given function, e.g., to handle the base case of a recursion –Solving these recurrence equations gives the execution time complexity of each function

120. S. Haridi and P. Van Roy120 Kernel language execution time  s  ::= skip k |  x  =  y  k |  x  =  v  k |  s 1   s 2  T(s 1 ) + T(s 2 ) |local  x  in  s 1  end k+T(s 1 ) | proc {  x   y 1  …  y n  }  s 1  end k |if  x  then  s 1  else  s 2  end k+ max(T(s 1 ), T(s 2 )) |{  x   y 1  …  y n  } T x (size(I({y 1,…,y n })) |case  x  of  pattern  then  s 1  else  s 2  end k+ max(T(s 1 ), T(s 2 )) Execution time T(s) of each kernel language operation  s  : Each instance of k is a different positive real constant size(I({y 1,…,y n })) is the size of the input arguments, defined as we like In some special cases,  x  =  y  and  x  =  v  can take more time

121. S. Haridi and P. Van Roy121 Example: Append proc {Append Xs Ys Zs} case Xs of nil then Zs = Ys [] X|Xr then Z Zr in Zs = Z|Zr {Append Xr Ys Zr} end end Recurrence equation for recursive call: –T Append (size(I({Xs,Ys,Zs}))) = k1+max(k2, k3+T Append (size(I({Xr,Ys,Zr}))) –T Append ( size (Xs)) = k1+ max(k2, k3+T Append ( size (Xr)) –T Append ( n) = k1+max(k2, k3+T Append ( n-1) –T Append (n) = k4+T Append ( n-1) Recurrence equation for base case: –T Append (0) = k5 Solution: –T Append (n) = k4.n + k5 = O(n)

122. S. Haridi and P. Van Roy122 Recurrence equations A recurrence equation is of two forms: –Define T(n) in terms of T(m 1 ), …, T(m k ), where m 1, …, m k < n. –Give T(n) directly for certain values of n (e.g., T(0) or T(1)). Recurrence equations of many different kinds pop up when calculating a program’s efficiency, for example: –T(n) = k + T(n-1)(T(n) is of O(n)) –T(n) = k1 + k2.n + T(n-1)(T(n) is of O(n 2 )) –T(n) = k + T(n/2)(T(n) is of O(log n)) –T(n) = k + 2.T(n/2)(T(n) is of O(n)) –T(n) = k1 + k2.n + 2.T(n/2)(T(n) is of O(n.log n)) Let us investigate the most commonly-encountered ones –For the others, we refer you to one of the many books on the topic

123. S. Haridi and P. Van Roy123 Example: FastPascal Recursive case –T FP (n) = k1+ max(k2, k3+T FP (n-1)+k4.n) –T FP (n) = k5+k4.n+T FP (n-1) Base case –T FP (1) = k6 Solution method: –Assume it has form a.n 2 +b.n+c –This gives three equations in three unknowns: k4-2.a=0 k5+a-b=0 k6=a+b+c –It suffices to see that this is solvable and a  0 Solution: T FP (n) is of O(n 2 ) fun {FastPascal N} if N==1 then [1] else local L in L={FastPascal N-1} {AddList {ShiftLeft L} {ShiftRight L}} end

124. S. Haridi and P. Van Roy124 Example: Mergesort Recursive case: –T MS (size(Xs)) = k1 + k2.n + T(size(Ys)) + T(size(Zs)) –T MS (n) = k1 + k2.n + T(  n/2  ) + T(  n/2  ) –T MS (n) = k1 + k2.n + 2.T(n/2) (assuming n is a power of 2) Base cases: –T MS (0) = k3 –T MS (1) = k4 Solution: –T MS (n) is of O(n.log n) (can show it also holds if n is not a power of 2) Do you believe this? –Run the program and measure it! fun {MergeSort Xs} case Xs of nil then nil [][X] then Xs else Ys Zs in {Split Xs Ys Zs} {Merge {MergeSort Ys} {MergeSort Zs}} end end

125. S. Haridi and P. Van Roy125 Memory space Watch out! There are two very different concepts Instantaneous active memory size (memory words) –How much memory the program needs to continue executing successfully –Calculated by reachability (memory reachable from semantic stack) Instantaneous memory consumption (memory words/sec) –How much memory the program allocates during its execution –Measure for how much work the memory management (e.g., garbage collection) has to do –Calculate with recurrence equations (similar to execution time complexity) These two concepts are independent

126. S. Haridi and P. Van Roy126 Calculating memory consumption  s  ::= skip 0 |  x  =  y  0 |  x  =  v  memsize(v) |  s 1   s 2  M(s 1 ) + M(s 2 ) |local  x  in  s 1  end 1+M(s 1 ) |if  x  then  s 1  else  s 2  end max(M(s 1 ), M(s 2 )) |{  x   y 1  …  y n  } M x (size(I({y 1,…,y n })) |case  x  of  pattern  then  s 1  else  s 2  end max(M(s 1 ), M(s 2 )) Memory space M(s) of each kernel language operation  s  : size(I({y 1,…,y n })) is the size of the input arguments, defined as we like In some cases, if, case, and  x  =  v  take less space

127. S. Haridi and P. Van Roy127 Memory consumption of bind memsize(v): –integer: 0 for small integers, else proportional to integer size –float: 0 –list: 2 –tuple: 1+n (where n=length(arity(v))) –other records: (where n = length(arity(v))) Once for each different arity in the system: k.n, where k depends on the run-time system For each record instance: 1+n –procedure: k+memsize(s) where is the procedure body and k depends on the compiler memsize(s): –Roughly proportional to number of statements and identifiers. Exact value depends on the compiler.

128. S. Haridi and P. Van Roy128 Higher-order programming Higher-order programming = the set of programming techniques that are possible with procedure values (lexically-scoped closures) Basic operations –Procedural abstraction: creating procedure values with lexical scoping –Genericity: procedure values as arguments –Instantiation: procedure values as return values –Embedding: procedure values in data structures Control abstractions –Integer and list loops, accumulator loops, folding a list (left and right) Data-driven techniques –List filtering, tree folding Explicit lazy evaluation, currying Later chapters: higher-order programming is the foundation of component-based programming and object-oriented programming

129. S. Haridi and P. Van Roy129 Procedural abstraction Procedural abstraction is the ability to convert any statement into a procedure value –A procedure value is usually called a closure, or more precisely, a lexically-scoped closure –A procedure value is a pair: it combines the procedure code with the environment where the procedure was created (the contextual environment) Basic scheme: –Consider any statement –Convert it into a procedure value: P = proc {$} end –Executing {P} has exactly the same effect as executing

130. S. Haridi and P. Van Roy130 Procedural abstraction fun {AndThen B1 B2} if B1 then B2 else false end

131. S. Haridi and P. Van Roy131 Procedural abstraction fun {AndThen B1 B2} if {B1} then {B2} else false end

132. S. Haridi and P. Van Roy132 A common limitation Most popular imperative languages (C, C++, Java) do not have procedure values They have only half of the pair: variables can reference procedure code, but there is no contextual environment This means that control abstractions cannot be programmed in these languages –They provide a predefined set of control abstractions (for, while loops, if statement) Generic operations are still possible –They can often get by with just the procedure code. The contextual environment is often empty. The limitation is due to the way memory is managed in these languages –Part of the store is put on the stack and deallocated when the stack is deallocated –This is supposed to make memory management simpler for the programmer on systems that have no garbage collection –It means that contextual environments cannot be created, since they would be full of dangling pointers

133. S. Haridi and P. Van Roy133 Genericity Replace specific entities (zero 0 and addition +) by function arguments The same routine can do the sum, the product, the logical or, etc. fun {SumList L} case L of nil then 0 []X|L2 then X+{SumList L2} end fun {FoldR L F U} case L of nil then U []X|L2 then { F X {FoldR L2 F U}} end

134. S. Haridi and P. Van Roy134 Map fun {Map Xs F} case Xs of nil then nil [] X|Xr then {F X}|{Map Xr F} end

135. S. Haridi and P. Van Roy135 Filter fun {Filter Xs P} case Xs of nil then nil [] X|Xr andthen {P X} then X|{Filter Xr P} [] X|Xr then {Filter Xr P} end

136. S. Haridi and P. Van Roy136 Instantiation Instantiation is when a procedure returns a procedure value as its result Calling {FoldFactory fun {$ A B} A+B end 0} returns a function that behaves identically to SumList, which is an « instance » of a folding function fun {FoldFactory F U} fun {FoldR L F U} case L of nil then U [] X|L2 then {F X {FoldR L2 F U}} end in fun {$ L} {FoldR L F U} end end

137. S. Haridi and P. Van Roy137 Embedding Embedding is when procedure values are put in data structures Embedding has many uses: –Modules: a module is a record that groups together a set of related operations –Software components: a software component is a generic function that takes a set of modules as its arguments and returns a new module. It can be seen as specifying a module in terms of the modules it needs. –Delayed evaluation (also called explicit lazy evaluation): build just a small part of a data structure, with functions at the extremities that can be called to build more. The consumer can control explicitly how much of the data structure is built.

138. S. Haridi and P. Van Roy138 Control Abstractions declare proc {For I J P} if I >= J then skip else {P I} {For I+1 J P} end {For 1 10 Browse} for I in 1..10 do {Browse I} end

139. S. Haridi and P. Van Roy139 Control Abstractions proc {ForAll Xs P} case Xs of nil then skip [] X|Xr then {P X} {ForAll Xr P} end {ForAll [a b c d] proc{$ I} {System.showInfo "the item is: " # I} end} for I in [a b c d] do {System.showInfo "the item is: " # I} end

140. S. Haridi and P. Van Roy140 Control abstractions fun {FoldL Xs F U} case Xs of nil then U [] X|Xr then {FoldL Xr F {F X U}} end Assume a list [x1 x2 x3....] S0  S1  S2 U  {F x1 U}  {F x2 {F x1 U}} .... 

141. S. Haridi and P. Van Roy141 Control abstractions fun {FoldL Xs F U} case Xs of nil then U [] X|Xr then {FoldL Xr F {F X U}} end What does this program do ? {Browse {FoldL [1 2 3] fun {$ X Y} X|Y end nil}}

142. S. Haridi and P. Van Roy142 Tree folding

143. S. Haridi and P. Van Roy143 Explicit lazy evaluation

144. S. Haridi and P. Van Roy144 Currying

145. S. Haridi and P. Van Roy145 Data abstraction A data abstraction is a set of values and an associated set of operations –There are two kinds of data abstraction, abstract data types and objects. Here we will talk about abstract data types. It is called abstract because it is completely described by its set of operations regardless of its implementation This means that it is possible to change the implementation of the data abstraction without changing its use The data abstraction is thus described by a set of procedures These operations are the only thing that a user of the abstraction can assume

146. S. Haridi and P. Van Roy146 Example: stack Assume we want to define a new abstract data type  stack T  whose elements are of any type T fun {NewStack}:  stack T  fun {Push  stack T   T  }:  stack T  proc {Pop  stack T   T   stack T  } fun {IsEmpty  stack T  } :  bool  These operations normally satisfy certain conditions {IsEmpty {NewStack}} = true for any E and T, {Pop {Push S E} E S} holds {Pop {EmptyStack}} raises error

147. S. Haridi and P. Van Roy147 Stack (implementation) fun {NewStack} nil end fun {Push S E} E|S end proc {Pop E|S E1 S1} E1 = E S1 = S end fun {IsEmpty S} S==nil end proc {Display S} S end

148. S. Haridi and P. Van Roy148 Stack (another implementation) fun {NewStack} nil end fun {Push S E} E|S end proc {Pop E|S E1 S1} E1 = E S1 = S end fun {IsEmpty S} S==nil end fun {NewStack} emptyStack end fun {Push S E} stack(E S) end proc {Pop stack(E S) E1 S1} E1 = E S1 = S end fun {IsEmpty S} S==emptyStack end fun {Display S}... end

149. S. Haridi and P. Van Roy149 Dictionaries The datatype dictionary is a finite mapping from a set T to  value , where T is either  atom  or  integer  fun {NewDictionary} –returns an empty mapping fun {Put D Key Value} –returns a dictionary identical to D except Key is mapped to Value fun {CondGet D Key Default} –returns the value corresponding to Key in D, otherwise returns Default fun {Domain D} –returns a list of the keys in D

150. S. Haridi and P. Van Roy150 Implementation fun {NewDictionary} nil end fun {Put Ds Key Value} case Ds of nil then [Key#Value] [] (K#V)|Dr andthen Key==K then (Key#Value) | Dr [] (K#V)|Dr andthen K>Key then (Key#Value)|(K#V)|Dr [] (K#V)|Dr andthen K<Key then (K#V)|{Put Dr Key Value} end

151. S. Haridi and P. Van Roy151 Implementation fun {CondGet Ds Key Default} case Ds of nil then Default [] (K#V)|Dr andthen Key==K then V [] (K#V)|Dr andthen K>Key then Default [] (K#V)|Dr andthen K<Key then {CondGet Dr Key Default} end fun {Domain Ds} {Map Ds fun {$ K#_} K end} end

152. S. Haridi and P. Van Roy152 Tree implementation fun {Put Ds Key Value} %... similar to Insert end fun {CondGet Ds Key Default} %... similar to Lookup end fun {Domain Ds} proc {DomainD Ds S1 S0} case Ds of leaf then S1=S0 [] tree(key:K left:L right:R...) then S2 S3 in S1=K|S2 {DomainD L S2 S3} {DomainD R S3 S0} end R in {DomainD Ds R nil} R end

153. S. Haridi and P. Van Roy153 Secure abstract data types: Stack is not secure fun {NewStack} nil end fun {Push S E} E|S end fun {Pop S E} case S of X|S2 then E=X S2 end end fun {IsEmpty S} S==nil end proc {Display S} S end

154. S. Haridi and P. Van Roy154 Secure abstract data types II The representation of the stack is visible: [a b c d] Anyone can use an incorrect representation, i.e., by passing other language entities to the stack operation, causing it to malfunction (like a|b|X or Y=a|b|Y) Anyone can write new operations on stacks, thus breaking the abstraction-representation barrier How can we guarantee that the representation is invisible?

155. S. Haridi and P. Van Roy155 Secure abstract data types III We need to make the representation visible only in part of the program and hidden from the rest Idea: put values inside a protection boundary We will see that there are two ways to use this boundary: –Stationary value. The value never leaves the boundary. Operations can enter the boundary to calculate with the value. This is like a bank that allows transfers but never lets money outside its doors. –Mobile value. The value can be taken out of the boundary and put back in. Operations with proper authorization can take the value out. Once out, calculations can be done on the value. This is like a bank that only stores money; clients have to take money out to use it.

156. S. Haridi and P. Van Roy156 Secure abstract data types IV Let us build the second solution (value enters and leaves the boundary) –The first solution (stationary value) can be built with higher-order programming – we will see it later when we introduce encapsulated state It is not possible to build the second solution in the declarative model we have seen so far (think about it!) We need to extend the model. Let us add just one new concept, a new basic type called a name –A name is like an atom except that it cannot be typed in on a keyboard or printed (no print representation; can only be used inside a program)! –The only way to have a name is if one is given it explicitly There are just two operations on names: –N={NewName} : returns a fresh name (name creation) –N1==N2 : returns true or false (name comparison)

157. S. Haridi and P. Van Roy157 Secure abstract data types V proc {NewWrapper ?Wrap ?Unwrap} Key={NewName} in fun {Wrap X} {Chunk.new w(Key:X)}% A chunk is a restricted form of record end fun {Unwrap W} W.Key% The only operation allowed is field selection end We want to « wrap » and « unwrap » values This puts them inside and takes them out of the boundary Let us use names to define a wrapper & unwrapper:

158. S. Haridi and P. Van Roy158 Secure abstract data types VI We need to add two new concepts to the basic computation model: Names (unforgeable constants) and Chunks (restricted records) {NewChunk Record} –returns a value similar to record but its arity cannot be inspected –{Arity foo(a:1 b:2)} is [a b] {NewName} –a function that returns a new symbolic (unforgeable), i.e. cannot be guessed –foo(a:1 b:2 {NewName}:3) makes impossible to access the third component, if you do not the arity {NewChunk foo(a:1 b:2 {NewName}:3) } –Returns what ?

159. S. Haridi and P. Van Roy159 Secure abstract data types: A secure stack With the wrapper & unwrapper we can build a secure stack local Wrap Unwrap in {NewWrapper Wrap Unwrap} fun {NewStack} {Wrap nil} end fun {Push S E} {Wrap E|{Unwrap S}} end fun {Pop S E} case {Unwrap S} of X|S2 then E=X {Wrap S2} end end fun {IsEmpty S} {Unwrap S}==nil end proc {Display S} {Unwrap S} end end

160. S. Haridi and P. Van Roy160 Capabilities and security We say a computation is secure if it has well-defined and controllable properties, independent of the existence of other (possibly malicious) entities (either computations or humans) in the system What properties must a language have to be secure? One way to make a language secure is to base it on capabilities –A capability is an unforgeable language entity (« ticket ») that gives its owner the right to perform a particular action and only that action –In our model, all values are capabilities (records, numbers, procedures, names) since they give the right to perform operations on the values –Having a procedure gives the right to call that procedure. Procedures are very general capabilities, since what they do depends on their argument –Using names as procedure arguments allows very precise control of rights; for example, it allows us to build secure abstract data types Capabilities originated in operating systems research –A capability can give a process the right to create a file in some directory

161. S. Haridi and P. Van Roy161 Secure queue I fun {NewQueue} X in q(0 X X) end fun {Insert Q X} case Q of q(N S E) then E1 in E=X|E1 q(N+1 S E1) end end proc {Delete Q Q1 X} case Q of q(N S E) then S1 in X|S1=S Q1 = q(N-1 S1 E) end end fun {EmptyQueue} case Q of q(N S E) then N==0 end end

162. S. Haridi and P. Van Roy162 Secure queue II The representation of the queue is visible q(N S E) N is the number of elements in the difference list S#E Any one can violate the representation, e.g. by using q(M S E) where M is different from N Any one can assume this representation and use it instead of the abstract operations, thus breaking the abstraction- representation barrier How to secure that the representation is invisible ?

163. S. Haridi and P. Van Roy163 Secure queue III local W UW in {NewWrapper W UW} fun {NewQueue} X in {W q(0 X X)} end fun {Insert C X} q(N S E) = {UW C} E1 in E=X|E1 {W q(N+1 S E1)} end..... end local W UW in {NewWrapper W UW}.... proc {Delete C1 ?C2 ?X} q(N S E)= {UW C1} S1 in S=X|S1 C2 = {W q(N-1 S1 E)} end fun {EmptyQueue C} q(N S E)= {UW C} in N==0 end end

164. S. Haridi and P. Van Roy164 Software components What is a good way to organize a large program? –It is confusing to put all in one big file Partition the program into logical units, each of which implements a set of related operations –Each logical unit has two parts, an interface and an implementation –Only the interface is visible from outside the logical unit, i.e., from other units that use it –A program is a directed graph of logical units, where an edge means that one logical unit needs the second for its implementation Such logical units are vaguely called « modules » or « software components » Let us define these concepts more precisely

165. S. Haridi and P. Van Roy165 Basic concepts Module = part of a program that groups together related operations into an entity with an interface and an implementation Module interface = a record that groups together language entities belonging to the module Module implementation = a set of language entities accessible by the interface but hidden from the outside (hiding can be done using lexical scope) Module specification = a function whose arguments are module interfaces, that creates a new module when called, and that returns this new module’s interface –Module specifications are sometimes called software components, but the latter term is widely used with many different meanings –To avoid confusion, we will call our module specifications functors and we give them a special syntax (they are a linguistic abstraction)

166. S. Haridi and P. Van Roy166 Standalone applications A standalone application consists of one functor, called the main functor, which is called when the application starts –The main functor is used for its effect of starting the application and not for the module it creates –The modules it needs are linked dynamically (upon first use) or statically (upon application start) –Linking a module: First see whether the module already exists, if so return it. Otherwise, find a functor specifying the module according to some search strategy, call the functor, and link its arguments recursively. Functors are compilation units, i.e., the system has support for handling functors in files, in both source and compiled form

167. S. Haridi and P. Van Roy167 Constructing a module Let us first see an example of a module, and then we will define a functor that specifies that module A module is accessed through its interface, which is a record We construct the module MyList MergeSort is inaccessible outside the module Append is accessible as MyList.append declare MyList in local proc {Append …} … end proc {MergeSort …} … end proc {Sort …} … {MergeSort …} … end proc {Member …} … end in MyList=‘export‘(append:Append sort: Sort member: Member …) end

168. S. Haridi and P. Van Roy168 Specifying a module Now let us define a functor that specifies the module MyList The functor MyListFunctor will create a new module whenever it is called The functor has no arguments because the module needs no other modules fun {MyListFunctor} proc {Append …} … end proc {MergeSort …} … end proc {Sort …} … {MergeSort …} … end proc {Member …} … end in ‘export‘(append:Append sort: Sort member: Member …) end

169. S. Haridi and P. Van Roy169 Syntactic support for functors We give syntactic support to the functor The keyword export is followed by the record fields The keyword define is followed by the module initialization code There is another keyword not used here, import, to specify which modules that the functor needs functor export append: Append sort: Sort member: Member … define proc {Append …} … end proc {MergeSort …} … end proc {Sort …} … {MergeSort …} … end proc {Member …} … end end

170. S. Haridi and P. Van Roy170 Example standalone application Compile Dict.oz giving Dict.ozf Compile WordApp.oz giving WordApp* functor import Open Dict QTk define … (1. Read stdin into a list) … (2. Calculate word frequencies) … (3. Display result in a window) end functor export new:NewDict put:Put condGet:CondGet entries:Entries define fun {NewDict}... end fun {Put Ds Key Value} … end fun {CondGet Ds Key Default} … end fun {Entries Ds} … end end File Dict.oz File WordApp.oz

171. S. Haridi and P. Van Roy171 Library modules A programming system usually comes with many modules –Built-in modules: available without needing to specify them in a functor –Library modules: have to be specified in a functor

172. S. Haridi and P. Van Roy172 The real world File I/O Window I/O Programming with exceptions More on efficiency –Garbage collection is not magic –External references

173. S. Haridi and P. Van Roy173 Large-scale program structure Modules: an example of higher-order programming Module specifications

174. S. Haridi and P. Van Roy174 Limitations and extensions of declarative programming Is declarative programming expressive enough to do everything? –Given a straightforward compiler, the answer is no Examples of limitations Extensions –Concurrency –State –Concurrency and state When to use each model –Use the least expressive model that gives a simple program (without technical scaffolding) –Use declarative whenever possible, since it is by far the easiest to reason about, both in the small and in the large