Visual Languages (Outline) Motivation Control Flow Diagrams Data Flow Diagrams Iconic Programming, Iconic Representation Visual Languages for Computer-based Human Communication Programming by Demonstration CSE 341 -- S. Tanimoto Visual Languages

CSE 341 -- S. Tanimoto Visual Languages

CSE 341 -- S. Tanimoto Visual Languages Motivation Programming is difficult because it’s hard to imagine what the state of the program is. Programming is difficult because language constructs are cryptic. Most PL concepts were developed in the text-only days, before graphical interfaces were widespread. Human visual perception is capable of high-bandwidth input if images, rather than text, are used. Visual languages can sometimes be learned more quickly. CSE 341 -- S. Tanimoto Visual Languages

CSE 341 -- S. Tanimoto Visual Languages Control Flow Diagrams Flow charts: originally used only for specification and documentation later, executable flow charts were developed. Read n product = 1 N n > 1 Y Print product product *= n n-- CSE 341 -- S. Tanimoto Visual Languages

CSE 341 -- S. Tanimoto Visual Languages Data Flow Diagrams Captures potential parallelism Usually requires scheduling Allows cueing of types by line attributes sin sqr Input double + Output cos * 5 Input image To Black&White Output image CSE 341 -- S. Tanimoto Visual Languages

CSE 341 -- S. Tanimoto Visual Languages Iconic Programming Icons can sometimes be learned more quickly and remembered more easily than textual names. An icon can represent an object or an operation. “Icons on strings” -- replacement of the boxes in flowcharts and data flow diagrams by icons Scripts composed of “iconic sentences” Example: Pictorial Language for Animation by Youngsters (PLAY) -- Runyan & Tanimoto, 1985. CSE 341 -- S. Tanimoto Visual Languages

CSE 341 -- S. Tanimoto Visual Languages

CSE 341 -- S. Tanimoto Visual Languages

Iconic Representation Icons can sometimes be learned more quickly and remembered more easily than textual names. Icon design involves metaphor -- use of analogy. Stylization -- graphical simplification and exaggeration. Decontextualization -- the trashcan is not in the kitchen. Abstract concepts are difficult to represent. Composing new images takes longer than new names. CSE 341 -- S. Tanimoto Visual Languages

CSE 341 -- S. Tanimoto Visual Languages

Example Iconic Language Vedo-Vedi: An iconic language for computer-based human communication. Prototype supports multi-lingual communication Meaning is expressed in an animation, with subtitles developed by S. Tanimoto & C. Bernardelli, 1998. CSE 341 -- S. Tanimoto Visual Languages

Programming by Demonstration The user performs a sample computation. The system may generalize the sequence. Requires an interface in which users can manipulate data directly. Controlling the generalization process is tricky. CSE 341 -- S. Tanimoto Visual Languages

CSE 341 -- S. Tanimoto Visual Languages Example Systems Gael Curry: Programming by Abstract Demonstration -- UW, 1977. Stagecast Creator: A simulation environment using programming by demonstration and rule-based simulation. (based on Cocoa & KidSim). Avail. at www.stagecast.com. ToonTalk: A robot programming environment supporting concurrent computation and complex control structures via a visual programming-by-demonstration method. (developed by Ken Kahn, available at www.toontalk.com). CSE 341 -- S. Tanimoto Visual Languages

Example Commercial Software Development Language Topological sort algorithm in Prograph. Prograph, by Pictorius, Inc. of Halifax, N.S., Canada. (uses data-flow paradigm) CSE 341 -- S. Tanimoto Visual Languages

Issues with Visual Languages Scaling up: Diagrams can turn into “spaghetti”. Editability: Programming by demonstration avoids an editable representation. Naming without text: iconic naming conventions tend not to be as natural for programmers as textual ones. Visual representations for complex concepts: Metaphors are more difficult to design and understand for abstract notions than simple ones. CSE 341 -- S. Tanimoto Visual Languages