Building “Real World” Software in Academia Matthias Felleisen PLT, Rice University

July 16, 1999 What is the “Real World”? … the Problem? n The TeachScheme! Project n Problems with Scheme n Building DrScheme n Technical Problems u Today: Extensibility or Single Point of Control

July 16, 1999 The TeachScheme! Project n Everyone should learn to program n … should do so early n … and it should be in Scheme n Scheme programming teaches critical skills: algebraic problem solving n Every student benefits …. n … not just the few who continue to program

July 16, = = 9 (+ 4 5) (+ 4 5) 9 Program Add (Output); Begin Writeln (4 + 5) Writeln (4 + 5)End. 9 Algebra SchemePascal Example #1 Arithmetic

July 16, 1999 f(x) = 4 + x ( define ( f x ) ( + 4 x )) Program f (Input, Output) ; Var x : Integer ; x : Integer ;Begin Readln ( x ) ; Readln ( x ) ; Writeln ( 4 + x ) Writeln ( 4 + x ) End. Algebra Pascal Scheme Example #2 A Simple Function

July 16, 1999 The Problems We Are Facing n psychology of teachers, students and parents n sociology of AP, principals, and administrators n a technical problem: Scheme’s syntax isn’t all that simple

July 16, 1999 Scheme’s Syntax is Bad #1 (define (length alist) (cond ((empty? alist) 0) (else 1 + (length (rest alist)))))

July 16, 1999 Emacs #1

July 16, 1999 Scheme’s Syntax is Bad #2 (define (length alist) (cond (null? (alist) 0) (else (+ 1 (length (rest alist))))))

July 16, 1999 Emacs #2

July 16, 1999 Our Solution: DrScheme n a hierarchy of languages u each level matches a student’s skill level u each level provides matching error messages n … plus a few tools: u a syntax checker & alpha renamer u a symbolic stepper u a type-flow analysis n currently used at around 80 universities/schools by around 8000 to students

July 16, 1999 d = (define (f v …) e) e = v | (f e …) | (primitive e …) | (cond (e e) …) f = name v = name d = (define-struct v (v …)) e = (local (d …) e) d = (define v e) | (set! v e) | (cond (e …) …) The Language Hierarchy (Excerpts) each level extends the level below

July 16, 1999 DrScheme

July 16, 1999 DrScheme #1

July 16, 1999 DrScheme #2

July 16, 1999 DrScheme Tools: Bindings

July 16, 1999 DrScheme Tools: Alpha Renaming

July 16, 1999 DrScheme Tools: Type Analysis

Building Extensible Software Matthias Felleisen PLT, Rice University

July 16, 1999 The Technical Problem: n We need extensible software components: u extensible parser u extensible evaluator u … n By developing each language level once and reusing it for all extensions, we save a huge amount of work (and maintenance work) n Principle: Single Point of Control

July 16, 1999 The Technical Solutions: n What does “extensible component” mean? n The programmer’s programmer’s perspective u composing patterns u composing patterns, again n The programming language perspective

July 16, 1999 Extensible Software Components Base Modified Component cut, paste, edit Extensible Base Extension add / link Matthias Felleisen:

July 16, 1999 Extensible Components are Good n No access to source code needed: critical for a Web-based model of software production & sales n Even if source code is accessible: programming for extensibility forces programmers to create a single point of control for many “functions” n Finally, it increases the reuse potential.

July 16, 1999 Extensibility is Not Guaranteed n Rumor 1: OOPLs guarantee extensibility. n Rumor 2: OO Design patterns do. n Rumor 3: Scheme does it :-) Krishnamurthi and Felleisen [FSE98]

July 16, 1999 The “Big” Problem for Extensibility E = var | (lambda (var) E) | (E E) void f(E some_e) { … } atyp g(E some_e) { … } E h(String s) { … } btyp m(E some_e) { … } E o(E an_e) { … } functional extension | (let (var E) E) | (if E E E) variant extension

July 16, 1999 Previous Attempts to Solve “It” n Reynolds (76): “It” is a problem … n Guy Steele (93): quasi-monads for interpreters n Cartwright and Felleisen (94): extensible interpreters n Hudak, Jones, and Liang (92-97): extensible “geometry server” for Navy

July 16, 1999 Solution 1: The Magic of Composing Patterns

July 16, 1999 A Closer Look at Objects and Classes n Classes make variant extensions straightforward. n The visitor pattern makes functional extensions straightforward. n Let’s combine these two virtues!

July 16, 1999 e = c | v | (lambda (x...) e) | (e e) |... Datatypes as Class Hierarchies

July 16, 1999 Datatype Visitors forConstants forVariables forProcedures forApplications forConstants forVariables forProcedures forApplications forConstants forVariables forProcedures forApplications

July 16, 1999 A Concrete Visitor class Constant extends Expression { void toVisitor(VisitorIF aVisitor) { aVisitor.forConstants(this) ; } } class PrintVisitor implements VisitorIF { void forConstants(… aConstant) { … } void forVariables( …aVariable) { … } void forProcedures(… aProcedure){ … } void forApplication(… anApplication){ … } Expression k; … k.toVisitor(new PrintVisitor());...

July 16, 1999 A Variant Extension forConstants forVariables forProcedures forApplications Expression Conditional (if e e e) forConditionals

July 16, 1999 So What’s the Problem? forConstants forVariables forProcedures forApplications Expression Conditional (if e e e) forConditionals new PrintVisitor(…) The Fix: Use Virtual Constructor Pattern

July 16, 1999 The Solution Imposes n Composing patterns: u Interpreter Pattern (natural) u Visitor Pattern u Virtual Constructor Pattern n Complex interactions between patterns n Numerous run-time type checks

July 16, 1999 Solution 2: More Magic with Patterns

July 16, 1999 Another Closer Look at Objects and Classes n Classes make variant extensions straightforward. n Inheritance can add functionality. n Let’s combine these two virtues!

July 16, 1999 Datatypes and Clients Parser for Expression Expression

July 16, 1999 Adding Functionality to Datatypes

July 16, 1999 So What’s the Problem Now? Parser for Expression Expression Method1 Ouch -- It’s the wrong kind of expression !

July 16, 1999 Linking the Parser via a Factory Parser for Expression Expression Method1 Expression Factory Expression Factory

July 16, 1999 The Second OO Solution Imposes n Composing patterns: u interpreter pattern u abstract factory u virtual constructor (or two AF) n Complex interactions between patterns n Numerous run-time type checks

July 16, 1999 Summary of OO Solutions n Programming for extensibility and for single point of control is feasible in existing OOPLs n But, the code is difficult to produce and maintain. n It suggests a new challenge to language designers.

July 16, 1999 Solution 3: The Language Designer at Work

July 16, 1999 How a Language Designer Can Help: A programming language should support both classes and hierarchical modules with external connectors.

July 16, 1999 Modules A B C imports module definitions exports

July 16, 1999 D E A B C Linking and Nesting Modules

July 16, 1999 Classes in Modules Conditionals (cond (e e) …) exports: classes, abstract classes, and interfaces imports: classes, abstract classes, and interfaces

July 16, 1999 Clients of Datatypes new Constants(…) ; … new Procedure(…)

July 16, 1999 Adding Variants, Again Conditionals (cond (e e) …) link to old clients, link to new clients now

July 16, 1999 Adding Functionality, Again Conditionals (cond (e e) …) new_method link to old clients, link to new clients now

July 16, 1999 Modules are “Natural” Conditionals (cond (e e) …) new_method … as if God had told us about all possible extensions ….

July 16, 1999 Lessons for Language Designers n Programmers must be able to connect components through external mechanisms. n Classes must extend interfaces, not fixed classes. n Modules must link to signatures, not fixed modules.

July 16, 1999 Hardware designers have known this idea for a long time. Why are language designers behind?

July 16, 1999 “Good” Programming Languages n MzScheme provides modules and classes with external connectors (units and mixins). Typing for MzScheme-like modules exists Flatt and Felleisen [PLDI98] n OCAML provides modules and classes with external connectors (functors and classes), but types are still in the programmer’s way. n Typing is possible, but OCAML is is to weak

July 16, 1999 The Costs of External Connectors n Layout of objects is unknown due to unknown superclass. n Intermodule optimization of datatypes becomes more difficult. n But, the cost is the nearly same as if we had programmed to an interface (which is what we are supposed to do anyway)

July 16, 1999 Future Work: n Work on more technical details (types, performance) n Work with language designers and “revisers” to affect their u technical understanding u psychology u sociology (And yes, these are the same problems that we face with TeachScheme)

July 16, 1999 Thank You and Credits n Robby Findler (*) n Matthew Flatt (Utah) n Cormac Flanagan (DEC SRC) n Shriram Krishnamurthi (*) n Dan Friedman (Indiana) n Corky Cartwright (Rice)