CSE S. Tanimoto Visual Languages 1 Visual Languages (Outline) Motivation Control Flow Diagrams Data Flow Diagrams Iconic Programming, Iconic Representation Visual Languages for Computer-based Human Communication Programming by Demonstration

CSE S. Tanimoto Visual Languages 2

3 Motivation Programming is difficult because it’s hard to imagine what the state of the program is. Programming is difficult because language constructs are cryptic. Most PL concepts were developed in the text-only days, before graphical interfaces were widespread. Human visual perception is capable of high-bandwidth input if images, rather than text, are used. Visual languages can sometmes be learned more quickly.

CSE S. Tanimoto Visual Languages 4 Control Flow Diagrams Flow charts: originally used only for specification and documentation later, executable flow charts were developed. product = 1 n > 1 product *= n n-- Print product Read n N Y

CSE S. Tanimoto Visual Languages 5 Data Flow Diagrams Captures potential parallelism Usually requires scheduling Allows cueing of types by line attributes cos sin Input double * 5 sqr +Output Input imageOutput imageTo Black&White

CSE S. Tanimoto Visual Languages 6 Iconic Programming Icons can sometimes be learned more quickly and remembered more easily than textual names. An icon can represent an object or an operation. “Icons on strings” -- replacement of the boxes in flowcharts and data flow diagrams by icons Scripts composed of “iconic sentences” Example: Pictorial Language for Animation by Youngsters (PLAY) -- Runyan & Tanimoto, 1985.

CSE S. Tanimoto Visual Languages 7

8

9 Iconic Representation Icons can sometimes be learned more quickly and remembered more easily than textual names. Icon design involves metaphor -- use of analogy. Stylization -- graphical simplication and exaggeration. Decontextualization -- the trashcan is not in the kitchen. Abstract concepts are difficult to represent. Composing new images takes longer than new names.

CSE S. Tanimoto Visual Languages 10

CSE S. Tanimoto Visual Languages 11 Example Iconic Language Vedo-Vedi: An iconic language for computer-based human communication. Prototype supports multi-lingual communication Meaning is expressed in an animation, with subtitles developed by S. Tanimoto & C. Bernardelli, 1998.

CSE S. Tanimoto Visual Languages 12 Programming by Demonstration The user performs a sample computation. The system may generalize the sequence. Requires an interface in which users can manipulate data directly. Controlling the generalization process is tricky.

CSE S. Tanimoto Visual Languages 13 Example Systems Gael Curry: Programming by Abstract Demonstration -- UW, Stagecast Creator: A simulation environment using programming by demonstration and rule-based simulation. (based on Cocoa & KidSim). Avail. at ToonTalk: A robot programming environment supporting concurrent computation and complex control structures via a visual programming-by-demonstration method. (developed by Ken Kahn, available at