1. FRP for Real Paul Hudak Yale University Department of Computer Science April 2001 Joint work with: Zhanyong Wan Walid Taha

2. FRP  Fran is a DSL for graphics and animation.  Frob is a DSL for robotics.  FranTk is a DSL for graphical user interfaces.  FRP (functional reactive programming) is the essence of Fran, Frob, and FranTk: Fran = FRP + graphics engine + library Frob = FRP + robot controller + library FranTk = FRP + Tk substrate + library  FRP has two key abstractions: Continuous time-varying behaviors. Discrete streams of events.

3. Domain-Specific Languages Functional Programming FRP Functions, types, etc. (Haskell) Continuous behaviors and discrete reactivity Specialized languages Fran FVision Graphics, Robotics, GUIs, VisionApplications FranTk Frob

4. Behaviors  Continuous behaviors capture any time-varying quantity, whether: input (sonar, temperature, video, etc.), output (actuator voltage, velocity vector, etc.), or intermediate values internal to a program.  Operations on behaviors include: Generic operations such as arithmetic, integration, differentiation, and time-transformation. Domain-specific operations such as edge-detection and filtering for vision, scaling and rotation for animation and graphics, etc.

5. Events  Discrete event streams include user input as well as domain-specific sensors, asynchronous messages, interrupts, etc.  They also include tests for dynamic constraints on behaviors (temperature too high, level too low, etc.)  Operations on event streams include: Mapping, filtering, reduction, etc. Reactive behavior modification (next slide).

6. Reactive Control of Continuous Values One animation example that demonstrates key aspects of FRP: growFlower = stretch size flower where size = 1 + integral bSign bSign = 0 `until` (lbp ==> -1 `until` lbr ==> bSign).|. (rbp ==> 1 `until` rbr ==> bSign)

7. Frob  Recall that: Frob = FRP + robot controller + robot/vision library  Programming robots is a lot like programming an animation!  … except that: The robot doesn’t always do what you want it to do. Error / anomalous conditions are more common. Real-time issues are more dominant. Sensor input is critically important, but unreliable. Robots have different response characteristics: Often must react more quickly. Often are slower than graphics hardware.

8. Robots with Vision

9. (our old robots) Nomadic Technologies SuperScout Computing: PC running Linux Hugs Radio Modem Vision 16 Sonars Bumpers Wheel Controls

10. Autonomous Coordinated Motion  Natural behavior amongst living animals: flocking, herding, schooling, swarming  Specific tasks of interest to us: congregation, navigation, “escortation”, formation motion, obstacle avoidance, dispersion, etc.  Key technologies of interest: computational vision and control FRP

12. Example of Coordinated Motion  Problem: Specify local control strategy for two differential-drive robots in interleaving trajectories, where each robot only knows the relative position of the other.  Can be achieved by two-step simplification: Non-holonomic constraint on differential-drive robot is eliminated by considering a moving frame of reference. Relative to that frame, each robot exhibits identical behavior: simply circle the other robot.  Frob permits abstract formulation of solution. Two independent critically-damped PI controllers. Local motion assumes holonomic vehicle; i.e. differential drive robot can be treated as omni-directional robot.

13. Local Behavior desired distance desired rotation vFrame vLat vRot moving frame of reference

15. Code Snippet interleaveC dist omega0 vFrame = let … distError = distOther - dist vLat = vector2Polar (kpDist * distError + kiDist * integralB distError) angOther vRot = vector2Polar (omega0*distOther/2) (angOther - pi/2) in velocityV (vFrame + vLat + vRot)

16. History of FRP Research  TBag, Active VRML: Conal Elliott, ’95-97.  Fran: Elliott & Hudak, ICFP ’97.  Various implementations: Eliiott, DSL ’97, PLILP ’99.  Semantics: Anthony Daniels, PhD Thesis ‘99.  Frob: Peterson, Hager, Hudak, Elliott, ICRA ‘99, PADL ’99. Used in robotics course at Yale in ‘99, ‘01.  Fvision: Peterson, Hager, Hudak, Reid, ICSE ’99, PADL ’01.  SOE’s “FAL”: Hudak, stream-based implementation of Fran-like language, ’00.  FranTk: Fran-based GUI, Meurig Sage, ICFP ’00.  Frappé: Java-based FRP, Antony Courtney, PADL ’01.  Yale FRP: core language + growing library of graphics, robotics, simulator, etc. code, ’00-01 (public release planned soon).  Semantics: Wan, Hudak, PLDI ’00, showed correspondence of denotational and operational semantics.

17. Real-Time FRP  How do we make FRP run fast?  How do we make guarantees about both time and space behavior?  How does FRP relate to other models of hybrid automata?  Can FRP be used for embedded systems?  And at a more abstract level: What is an operational semantics for FRP? Our goal: Real-Time FRP, an abstract, restricted subset of FRP, with guaranteed bounds on execution time and space, and deadlock free.

18. Syntax of RT-FRP  Syntax of lambda terms: (“lifted” terms are just “Maybe” type)  Syntax of values:  Syntax of signals: Note: “Event a” in FRP is isomorphic to “Behavior (Maybe a)”. In RT-FRP we just combine them and call them “signals”.

19. Types  Syntax of types:  Contexts:  Judgments: “e is a functional term of type g” “s is a signal carrying values of type g” Or in Haskell: e :: g, and s :: Behavior g

20. Some Typing Rules

21. More typing rules  Reactivity:  Non-recursive binding:  Recursive binding:

22.  Note: let signal x = s in ext x == s

23. Recursion  There are two “prototypical” kinds of recursion in FRP: (1) (2)  Using recursive signals these can be expressed as: (1) (2)

24. But what about integral?  Define using a delay in a recursive signal.  By way of example, here is the running maximum of a signal:

25. Definition of Integral … using the forward Euler method:

26. From FRP to RT-FRP

27. and more…

28. Operational Semantics  A program is executed in discrete steps, driven by time-stamped input values.  Given a time t and input i, a term s evaluates to a value v and transitions to a new term s ’.  We write this as: and or, more compactly as: where:

29. Program Execution  A proper RT-FRP program never terminates. Its meaning is the infinite sequence of values, or “observations” that it yields.  The sequence: s 0  s 1,v 1 ; s 1  s 2,v 2 ; s 2  s 3,v 3 ; … is abbreviated: s 0  v 1, s 1  v 2, s 2  v 3, s 3 …  Meta-comment: Program meaning depends on the time-stamped input pairs (t j,i j ). t 0,i 0 t 2,i 2 t 1,i 1 t 0,i 0 t 1,i 1 t 2,i 2

30. Some Eval Rules

31. More Eval Rules (ev-signal)

32. Some Transition Rules

33. More Transition Rules

34. and more… (tr-switch-occ) (tr-switch-noc)

35. Tail Signals  Recursive signals are nice, but we’d like something better:  With let-continuation, we can define tail-recursive signals that can also pass values.  Similar to standard model of hybrid automata.  Note: the syntax prevents unbounded term growth as in: letcont k x = s0 until x=ev then k x + 1 not possible

36. For Example  A simple model of a thermostat: let signal temp = let cont k1 () = until when (temp>t) => k2 k2 () = until when (temp k1 in t0 until startev => k1 in ext temp

37. Key Results Type safety / preservation thus no core dumps! Each step takes constant time thus no time leaks! Term size cannot grow thus no space leaks! In addition, with a notion of well-formed recursion, progress is guaranteed. thus no deadlock!

38. Preventing Deadlock  FRP programs are naturally “infinite” – signals are streams of values with unbounded extent. This is a good thing!  With recursion, however, terms can become “stuck”: let signal x = ext x in ext x  Solution: define notion of well-formed recursion that disallows “direct” recursions, and thus prevents deadlock.  Key idea: in “let signal x = s1 in s2”, we require that s1 is in W {x}, where: W x = input | time | ext e where e contains no X | delay v s | let signal y = W x in W x | W x switch on x = ev in W x | …  In other words, a delay must appear somewhere.

39. Future Work on RT-FRP  Enrich language for practical use.  Compile into lower-level code (C, etc.).  Examine sensor / behavior fusion.  Consider embedded systems.  Better formulation of well-formed recursion.  Test case: compile to PIC microcontroller on our “soccer-bots”.

40. How to Hide a Flock of Turkeys