Chapter 3 Start Semiotic Engineering Tenets of Semiotic Engineering basis for theory to evaluate, to design

Ch. 3 Semiotic Engineering Introduction: Designers perform interaction to critique interfaces as a regular part of their research and practice, But the field of HCI lacks a proper discipline of interaction criticism. By interaction criticism we mean rigorous, evidence- based interpretive analysis that explicates relationships among elements of an interface and the meanings, affects, moods, and intuitions they produce in the people that interact with them; the immediate goal of this analysis is the generation of innovative design insights. We summarize existing work offering promising directions in interaction criticism to build a case for a proper discipline. We then propose a framework for the discipline, relating each of its parts to recent HCI research

Standard process of HCI Standard process of how to practice interaction design: problem definition, user study, sketching through prototype design, and evaluation

This process is refined to address: aspects of interaction design, in particular usability, performance, and user satisfaction. Increasingly, we are hearing calls for innovation in more subjective areas, such as Aesthetics affective computing experience design intimate and embodied interaction

Improvement needed interaction design needs to improve its awareness of “the symbolic level of mood and meaning, of sociability and civility”; we need “a language true to the medium of computation, networks, and telecommunications.”

Problem = the lack of an expressive language of interaction design Needs: rigorous interpretive analysis that explicates how elements of the interface, through their relationships to each other, produce certain meanings, affects, moods, and intuitions in the people that interact with them. We say “rigorous” to stress that interaction criticism, like the best film and literary criticism, transcends anything-goes subjectivism and offers instead systematic, evidence-based analyses of subjective phenomena. The purpose of such an analysis is to generate actionable insights for interaction design.

Framework as the study of interfaces Stylistic references Standards and conformance to tradition Materiality and remediation Genre Functional versus cultural dimensions of an interface Representational techniques Challenges to user expectations Capacity for unanticipated use

goal of becoming a thoughtful, reflective designer A sensibility regarding the qualities of designs and design processes A developed language, which appears to mean a technical analytic vocabulary Reflective thinking, which emphasizes the interpreting subject’s awareness of her- or himself in the development of one’s own thoughts Retrospective reflection, which explores the “arguments and ideas that could explain a design”

Goal from this class To build a good language for the interface which: Is futuristic Is manipulated by user in novice then expert manner Is dynamically representative of the concept Is progressively learned Has elements known to user and aspect never seen/experienced by user and the unknown aspects are “bridge” intuitive. (“bridge” = via user experimentation)

Good languages have levels Conceptual Level Goal Semantic Level Meaning Syntactical Level Grammar Lexical Level Tokens, Alphabet

Interaction Diagogues Person System Goal: Save the file Subgoal: find the “saving functionality” Action token: Mouse travel to disc image on left upper corner

Next cycle of human - system Input Received: The disc image appears selected and “Save (Ctrl + S)” is given in help box Perception: I’m on the save functionality.   Subgoal: Start the save function Desired token: Select the disc image Action token: Press the left mouse button

Then the next cycle is: Input Received: Dialogue Box appears Perception: This is the save dialogue   Next sub-goal: Name the file with unique description name Processing: the default name is not Document 1 but the first line of the file and this is okay… but the location of the file is not in the correct folder New sub-goal: get the file into folder SWE 4783 Action: select the scroll bar, press left mouse button, and travel down till I see the SWE 4783 folder.

Give the left side data: Input Received: Perception:   Next Sub-goal: Action: Action token:

Foley and Van Dam Four Levels of Cognition (thinking) Conceptual level: User's mental model of the interactive system Semantic level: Describes the meanings conveyed by the user's command input and by the computer's output display Syntactic level: Defines how the units (words) that convey semantics are assembled into a complete sentence that instructs the computer to perform a certain task Lexical level: Deals with device dependencies and with the precise mechanisms by which a user specifies the syntax

Command Language Grammar from Moran Moran (1981) describes Command Language Grammar (CLG) as having three components and six levels (see below).

Moran’s Levels Task level: The user comes to the system with a set of tasks that he wants to accomplish. The purpose of the Task level is to analyze the user’s needs, and to structure his task domain in a way that is amenable to an interactive system. The output of this level is a structure of specific tasks that the user will set for himself with the aid of the system. Semantic level: A system is built around a set of objects and manipulations of those objects. To the system these are data structures and procedures; to the user they are conceptual entities and conceptual operations on these entities. The Semantic level lays out these entities and operations. They are intended to be useful for accomplishing the user’s tasks, since they represent the systems functional capability. Thus, the Semantic level also specifies methods for accomplishing the tasks in terms of these conceptual entities and operations. Syntactic level: The conceptual model of a system is embedded in a language structure, the command language, for users to communicate with the system. All command languages are built out of a few syntactic elements; commands, arguments, contexts and state variables. The Syntactic level lays out these elements. The “meaning” of each command of the system is defined in terms of operations at the Semantic level, and the methods at the Semantic level are recorded in terms of Syntactic level commands. Interactional level: The dialogue conventions for the user-system interaction must ultimately be resolved as a sequence of physical actions- key presses and other primitive device manipulations by the user and display actions by the system. The Interaction level specifies the physical actions associated with each of the Syntactic level elements, as well as the rules governing the dialogue

HCI = twofold computer-mediated metacommunication About aspect of communication itself How it gives users a key (hint) for interpreting How their senders mean Nonverbal cues Presence affordance One-shot message – complete content encoded

Three fundamental consequences Designer involved at interaction time Qualitative dimensions Process of encoding the message as a computing system captures and freezes only part Will evolve overtime Computationally encoded portion is the One-shot metacommunication message to users.

Semiotic Engineering Design Space Designer’s messageSystem’s Sign user’s role

Designing Signs triggers: What aspects of the sender? What aspects of the receiver? Context of communication? Communication code? Channel? Message? See page 87-88