1. Human Computer Interaction
Unit III
Human Computer Interaction
Ubiquitous computing: smart devices, environments and interaction 1 1

2. Ubiquitous computing: smart devices, environments and interaction
HCI: Overview
Part A: eHCI Use in some common smart device types
Part B iHCI for accompanied smart devices
Part C: iHCI for wearable & implanted smart devices
Part D: Human Centred Design
Part E: User Models and iHCI Design
Ubiquitous computing: smart devices, environments and interaction 2 2

3. Ubiquitous computing: smart devices, environments and interaction
HCI: Overview
HCI, eHCI & iHCI 
eHCI use in 4 Widely Used Devices
iHCI use in accompanied smart devices
iHCI use in wearable and implanted smart devices
Human Centred Design (HCD)
User Models: Acquisition & Representation
iHCI Design
Ubiquitous computing: smart devices, environments and interaction 3 3

4. Ubiquitous computing: smart devices, environments and interaction4 4

5. Ubiquitous computing: smart devices, environments and interaction
HCI: Introduction
Term HCI, widely used, since onset of Personal Computing era in 1980s.
However groundwork for field of HCI started earlier, during onset of the industrial revolution
Tasks became automated and powered-assisted
-> triggers an interest in studying human-machine interaction
Some tasks require little human interaction during operation, e.g., clothes-, dish- washing etc
Other tasks are very interactive, e.g., face washing, playing the violin, etc
Ubiquitous computing: smart devices, environments and interaction 5 5

6. Ubiquitous computing: smart devices, environments and interaction
H,C & I
Basic concepts of HCI are:
Humans – single or multiple user with diverse physical and mental abilities, interacting cooperatively or competitively
Computers / devices – not just PCs but embedded computing devices (dust, tabs,pads, boards)
Interaction – directed via command or by manipulating virtual object, also involve natural interaction such as speech, gesture etc.
Ubiquitous computing: smart devices, environments and interaction 6 6

7. Ubiquitous computing: smart devices, environments and interaction
HCI: Motivation
Machines (systems) aid human performance, but systems that interact poorly with humans will be a poor human aid.
Need design models & process that are (user) interactive
The motivation for HCI is clear; to support more effective use (Dix, 2004a) in three ways
Useful: a User task that user requires to be done.
Usable: Do the task easily, naturally, safely
Be used: enrich the user experience by making it attractive, engaging, fun etc.
Ubiquitous computing: smart devices, environments and interaction 7 7

8. HCI: Usability vs. Usefulness
Summarised as Heckel's law and Heckel's inverse law:
Heckel’s law:
The quality of the user interface of an appliance is relatively unimportant in determining its adoption by users if the perceived value of the appliance is high.
Heckel’s inverse law:
The importance of the user interface design in the adoption of an appliance is inversely proportional to the perceived value of the appliance
Ubiquitous computing: smart devices, environments and interaction 8 8

9. Ubiquitous computing: smart devices, environments and interaction
Explicit HCI (eHCI)
eHCI design: explicit interaction during a device’s normal operation.
What are the Dominant eHCI UIs
Pure eHCI
Context-free
Focus on H2C (Human-to-Computer) Interaction
Ubiquitous computing: smart devices, environments and interaction 9 9

10. Ubiquitous computing: smart devices, environments and interaction
iHCI
Concept of implicit HCI (iHCI)
Proposed by Schmidt (2000)
Defined as “an action, performed by the user that is not primarily aimed to interact with a computerized system but which such a system understands as input”.
Our definition of iHCI bit different:
inputs with an implicit or implied context
Users behaviour in given situation
Gestures as clap to control device as well as express an emotion
Ubiquitous computing: smart devices, environments and interaction 10 10

11. Ubiquitous computing: smart devices, environments and interaction
iHCI
iHCI is more about C2H (Computer to Human) Interaction
iHCI assumes Chas a certain Model of H user
Model of H used as additional input
Need to share implicit context between human and system
Implicit interaction naturally supports hidden device design.
Ubiquitous computing: smart devices, environments and interaction 11 11

12. Ubiquitous computing: smart devices, environments and interaction
Overview
HCI, eHCI & iHCI
eHCI use in 4 Widely Used Devices 
iHCI use in accompanied smart devices
iHCI use in wearable and implanted smart devices
Human Centred Design (HCD)
User Models: Acquisition & Representation
iHCI Design
Ubiquitous computing: smart devices, environments and interaction 12 12

13. UI and HCI Designs for 4 Common DevicesPC
Mobile Phone
Games Console but many sub-types
TV / Projectors
Ubiquitous computing: smart devices, environments and interaction 13 13

14. Diversity of ICT Interaction
Size : (Hand, Centimetre, Micro, body sized or larger)
Haptic Intput : (Two handed vs. One Handed Vs. Hands free)
Interaction modalities : (Single Vs. Multiple)
Posture of Human Operator : (Laying, sitting, standing, walking, running)
Distance of output display to input control : Centimetre to metres
Position during Operations : Fixed Vs. Mobile
Connectivity : (Stand alone vs. networked vs. wired vs. wireless)
Tasking : single tasks devices
Ubiquitous computing: smart devices, environments and interaction 14 14

15. UI Type: Personal Computer Interface
MEMEX System
Commond Line Visual Interface
WIMPS Interface / Direct Manipulation interface
Ubiquitous computing: smart devices, environments and interaction 15 15

16. Ubiquitous computing: smart devices, environments and interaction
PC UI use in Mobiles
Using a conventional PC UI approach won’t be optimum for mobile computing & ubiquitous computing - need a different approach, Why?
Smaller Display Area
Variety of Resource Constained input and output Devices
Handling limited key input : Multitap,T9, FastTap, Soft keys
Handling limited output
Audible outputs – Vehicle nevigation System
Ubiquitous computing: smart devices, environments and interaction 16 16

17. UI Type: Games Console Interfaces
Games consoles: an important driver and can contribute to UbiCom in a number of ways
More Natural interface
Many different types of Games Console Interface
Forsters (2005) Seven different generation of Game Consoles
7th generation includes Nintendo Wii - replacement of D- pad
Ubiquitous computing: smart devices, environments and interaction 17 17

18. UI Type: Localised Remote Control Interfaces
Characteristics
Input controller and device separation
Input device interfaces
Wireless link between input control device and device
Ubiquitous computing: smart devices, environments and interaction 18 18

19. Ubiquitous computing: smart devices, environments and interaction
UIs often introduce modes to minimise the number of UI controls (e.g., toggle buttons vs. radio buttons)
AM and PM are modes!
Up and down buttons introduce modes
UI that contain more controls & less modes appear more complex but are often easier to operate
Changing the gesture to trigger action (e.g., adaptive menus) are more useful if they are inclusive rather than exclusive
Ubiquitous computing: smart devices, environments and interaction 19

20. Ubiquitous computing: smart devices, environments and interaction
Overview
HCI, eHCI & iHCI
eHCI use in 4 Widely Used Devices
iHCI use in accompanied smart devices 
iHCI use in wearable and implanted smart devices
Human Centred Design (HCD)
User Models: Acquisition & Representation
iHCI Design
Ubiquitous computing: smart devices, environments and interaction 20 20

21. iHCI use in Accompanied Smart Devices: Topics
Single vs. Multi-Modal Visual Interfaces
Gesture Interfaces
Reflective versus Active Displays
Combining Input and Output User Interfaces
Auditory Interfaces
Natural Language Interfaces
Ubiquitous computing: smart devices, environments and interaction 21 21

22. Computer input & output modalities
Ubiquitous computing: smart devices, environments and interaction 22 22

23. Multi-Modal Interaction: Design
Two main approaches
Data for each modality can be processed separately, then combined at the end.
Data for each modality can be processed & combined concurrently
Ubiquitous computing: smart devices, environments and interaction 23 23

24. Ubiquitous computing: smart devices, environments and interaction
Gesture Interfaces
What are Gestures?
Expressive, meaningful body motions
Involving physical movements. - fingers, arms, hands, face, body
With the intent of conveying meaningful information about interacting with the environment.
Contactful Gestures Vs. Contactless gestures
2D gestures Vs. 3D Gestures
Directly sensed versus indirectly sensed
Ubiquitous computing: smart devices, environments and interaction 24 24

25. Gesture Interfaces: HCI->HPI->HHI->HCI
Ubiquitous computing: smart devices, environments and interaction 25 25

26. Reflective versus Active Displays
Can we produce ICT displays that support more of the properties of physical paper?
Display design mimics paper
Epaper display design differs from actual paper
Ubiquitous computing: smart devices, environments and interaction 26 26

27. ElectroPhoretic Displays or EPDs
Ubiquitous computing: smart devices, environments and interaction 27 27

28. Combining Input and Output User Interfaces
UIs discussed so far, input devices are separated from the output devices
State of the input is available as a visual cue only.
How can we combine / link input and output better?
Ubiquitous computing: smart devices, environments and interaction 28 28

29. Ubiquitous computing: smart devices, environments and interaction
Touchscreen
What are touchscreens?
Displays where position of contact with screen is detected
Via pointed physical objects such as pens, fingers, etc
Events can then be generated for an associated visual object at that position and
Associated actions can then be triggered.
Ubiquitous computing: smart devices, environments and interaction 29 29

30. Ubiquitous computing: smart devices, environments and interaction
Touchscreen
Touchscreen behaves as 2D, planar smart skin.
Wherever it is touched, a virtual object can be activated.
Types of touchscreens
Resistive
Capacitive
Surface acoustic waves etc.
Touch screen can behave as a:
soft control panel and user interface
that is reprogrammable
which can be customised to suit a range of applications and users
Ubiquitous computing: smart devices, environments and interaction 30 30

31. Tangible User Interface (TUI)
(TUI) is a UI that augments the real physical world by coupling digital information to everyday physical objects and environments.
Tangible user interfaces are also referred to as
passive real-world props,
graspable user interfaces,
manipulative user interfaces
embodied user interfaces
Ubiquitous computing: smart devices, environments and interaction 31 31

32. Ubiquitous computing: smart devices, environments and interaction
DataTiles Project
Allows users to manipulate data in form of tangible “tiles”
Combinations of data streams and functions make it possible to create new applications
Ubiquitous computing: smart devices, environments and interaction 32 32

33. Ubiquitous computing: smart devices, environments and interaction
Organic Interfaces
Similar to Tangible Interfaces
3 characteristics which characterize organic UIs.
- Display can be the input device
- display can take on any shape
- displays can change
Typically use Organic Light-Emitting Diode (OLED) type materials
Ubiquitous computing: smart devices, environments and interaction 33 33

34. Auditory Interfaces: Non-Speech Based
Challenges : Presence of audio noise, access control for voice activation,NLP of voice input
2 basic auditory interfaces:
Speech based
Non-speech based
Non-speech auditory interfaces:
Earcons, synthetic auditory sequences
Ubiquitous computing: smart devices, environments and interaction 34 34

35. Natural Language Interfaces
Generally, interaction can be more easily processed and understood if it defined using an expressive language that has a well-defined syntax or grammar and semantics
requires that users already know the syntax.
Ubiquitous computing: smart devices, environments and interaction 35 35

36. Ubiquitous computing: smart devices, environments and interaction
Overview
HCI, eHCI & iHCI
eHCI use in 4 Widely Used Devices
iHCI use in accompanied smart devices
iHCI use in wearable and implanted smart devices 
Human Centred Design (HCD)
User Models: Acquisition & Representation
iHCI Design
Ubiquitous computing: smart devices, environments and interaction 36 36

37. Hidden UI via Wearable and Implanted Devices
In the Posthuman model, technology can be used to extend a person's normal conscious experience and sense of presence, across space and time.
There are 3 types of post-human technology:
Accompanied
e.g. ???
Wearable
e.g., ???
Implants
E.g., ???
Ubiquitous computing: smart devices, environments and interaction 37 37

38. Ubiquitous computing: smart devices, environments and interaction
Wearable computers
Wearable interfaces include a combination of ICT devices & modalities
Wearable computers are especially useful when?
Focus is on multi-modal interaction which includes visual interaction.
Ubiquitous computing: smart devices, environments and interaction 38 38

39. Ubiquitous computing: smart devices, environments and interaction
Wearable computers
Visual modal systems are divided according to how humans interact with the system: ??
Visual interaction can be classified into
command
non-command interfaces.
Non-command vision-based (human motion) analysis systems generally have four stages:
motion segmentation
object classification
tracking
interpretation.
Ubiquitous computing: smart devices, environments and interaction 39 39

40. Wearable Computer: WearComp and WearCam
Many researchers contributed to the advancement of wearable computing
Perhaps the most important Pioneer of Wearable Computing is Steve Mann
His 1st early main application focussed on recording personal visual memories that could be shared with other via the Internet.
Ubiquitous computing: smart devices, environments and interaction 40 40

41. Wearable Computer: WearComp and WearCam
Photo courtesy of Wikimedia Commons,
Ubiquitous computing: smart devices, environments and interaction 41 41

42. Wearable computing: Mann’s definition
Mann (1997): 3 criteria to define wearable computing.
Eudaemonic criterion
Existential criterion
Ephemeral criterion
Ubiquitous computing: smart devices, environments and interaction 42 42

43. Wearable computing: Types
Some different type of wearable computers??
N.B. Not all these meet Mann’s criteria
Ubiquitous computing: smart devices, environments and interaction 43 43

44. Head(s)-Up Display or HUD:
presents data without blocking the user's view
pioneered for military aviation - now used in commercial aviation and cars.
2 types of HUD
Fixed HUD:
Head-mounted HUD
Ubiquitous computing: smart devices, environments and interaction 44 44

45. EyeTap & Virtual Retinal Display
Instructors can add more detail about these here or delete this slide.
Ubiquitous computing: smart devices, environments and interaction 45 45

46. Brain Computer Interface (BCI) or Brain Machine Interfaces (BMI)
HCI focuses on indirect interfaces from human brain via human actuators
BCI are direct functional interfaces between brains and machines
BCI represents ultimate natural interface
Would you choose to make use of one when they become available in the future?
Ubiquitous computing: smart devices, environments and interaction 46 46

47. Brain Computer Interface (BCI) or Brain Machine Interfaces (BMI)
Direct vs. Indirect coupling design choices ??
See also BANs in Chapter 11
Brain versus nerve direct coupling design choices??
Ubiquitous computing: smart devices, environments and interaction 47 47

48. Ubiquitous computing: smart devices, environments and interaction
Computer Implants
Opposite of wearing computers outside the body is to have them more directly interfaced to the body.
Many people routinely use implants
????
Of specific interest is developing devices that can adapt to signals in the human nervous system.
By connecting electronic circuitry directly to the human nervous system,
???
Ubiquitous computing: smart devices, environments and interaction 48 48

49. Ubiquitous computing: smart devices, environments and interaction
Cyborg 2
Electrode array surgically implanted into Warwick’s left arm and interlinked into median nerve fibres is being monitored.
Ubiquitous computing: smart devices, environments and interaction 49 49

50. Ubiquitous computing: smart devices, environments and interaction
BCI
Instructors can add more detail about experiments here or delete this slide
Ubiquitous computing: smart devices, environments and interaction 50 50

51. Ubiquitous computing: smart devices, environments and interaction
PostHuman Model
Use of alterative technology mediated realities
A feeling of presence in the experience provides feedback to a person about the status of his or her activity.
The subject perceives any variation in the feeling of presence and tunes its activity accordingly.
Ubiquitous computing: smart devices, environments and interaction 51 51

52. PostHuman Model and Reality
People can experience alternative realities depending on:
the type of environment people are situated in
on their perception of the environment.
Reality can be:
Technology mediated, e.g., ???
Chemically mediated, , e.g., ???
Psychologically mediated, , e.g., ???
Ubiquitous computing: smart devices, environments and interaction 52 52

53. Ubiquitous computing: smart devices, environments and interaction
Realities: VR, AR and MR
(Revision of Section )
Ubiquitous computing: smart devices, environments and interaction 53 53

54. Ubiquitous computing: smart devices, environments and interaction
Virtual Reality (VR)
VR seeks to immerse a physical user in a virtual 3D world
VR uses a computer simulation of a subset of the world and immerses the user in it using UIs based upon: ??
VR seeks to enable humans to interact using a more natural interaction that humans use in the real world
Ubiquitous computing: smart devices, environments and interaction 54 54

55. Augmented Reality (AR)
Electronic images are projected over the real world so that images of the real and virtual world are combined.
VR considered as a subset of AR?
Early E.g. head-mounted display by Sutherland (1968).
Similar systems are in use today in types of military aircraft.
Ubiquitous computing: smart devices, environments and interaction 55 55

56. Telepresence & Telecontrol
Telepresence allow a person in 1 local environment to: ?? .
Telecontrol refers to the ability of a person in 1 place to
???
Ubiquitous computing: smart devices, environments and interaction 56 56

57. Ubiquitous computing: smart devices, environments and interaction
Overview
HCI, eHCI & iHCI
eHCI use in 4 Widely Used Devices
iHCI use in accompanied smart devices
iHCI use in wearable and implanted smart devices
Human Centred Design (HCD) 
User Models: Acquisition and Representation
iHCI Design
Ubiquitous computing: smart devices, environments and interaction 57 57

58. Conventional Design versus HCD
Ubiquitous computing: smart devices, environments and interaction 58 58

59. Conventional Functional System Design
Ubiquitous computing: smart devices, environments and interaction 59 59

60. Human Centred Design (HCD)
Focus on types of UbiCom System & environments:
Need to make the type of user explicit: human users
In contrast, automatic / autonomous systems
Ubiquitous computing: smart devices, environments and interaction 60 60

61. Human Centred Design (HCD)
ISO standard human centred design life-cycle involves 4 main sets of activities:
Define context of use
Specify stake-holder and organisational requirements
Multiple alternative (UI) designs need to be built.
Designs need to be validated against user requirements.
Ubiquitous computing: smart devices, environments and interaction 61 61

62. Human Centred Design (HCD)
Ubiquitous computing: smart devices, environments and interaction 62 62

63. A Fuller Range of System & User Requirements / Use Contexts
HCD System & User requirements
-> Wider requirements than back-end functional requirements
HCD Methodologies are a powerful way to get the wide range of environment requirements & use contexts for UbiCom systems
What is the fuller ranges of UbiCom / HCD requirements?
Ubiquitous computing: smart devices, environments and interaction 63 63

64. A Fuller Range of System & User Requirements / Use Contexts
???
Physical Environment
Users
Types
Task & goals
User interface
Social
Usability & User experience:
Usability:
User experiences
Ubiquitous computing: smart devices, environments and interaction 64 64

65. HCD: Use Context / Requirements
Ubiquitous computing: smart devices, environments and interaction 65 65

66. HCD: Usability as a User Requirement
Usability is defined as ??
Usability is not a single, one-dimensional property of a user interface.
Usability is a combination of factors. ISO explicitly mentions no. of factors
???
These usability factors can often be expanded further sub- properties,
Ubiquitous computing: smart devices, environments and interaction 66 66

67. Ubiquitous computing: smart devices, environments and interaction
HCD: Stake-Holders
End-user is obvious stake-holder in HCD Design
Who are the other stake-holders in the personal memory scenario?
Are there additional stake-holder requirements?
Ubiquitous computing: smart devices, environments and interaction 67 67

68. HCD: Acquiring User Context/User Requirements
Several dimensions for get user requirements during HCD life-cycle
In Controlled conditions (Lab) vs. in the field
Direct user involvement (e.g., interview, questionnaire) vs. indirect (e.g., observations)
Individual users vs. user groups, HCI / domain experts vs. predictive user models (no users)
Ubiquitous computing: smart devices, environments and interaction 68 68

69. HCD: Methods to Acquire User Requirements
Which Methods?
?????
Analysis of data gathered depends on:
Amount of time, level of detail, uncertainty etc
Knowledge the analysis requires
Ubiquitous computing: smart devices, environments and interaction 69 69

70. Usability Requirements & Use Contexts Examples
For each of the scenarios in chapter 1, e.g., the personal video memories, define the use context and usability requirements.
Ubiquitous computing: smart devices, environments and interaction 70 70

71. HCI / HCD versus User Context Awareness
Are these the same or similar concepts?
See User context awareness (Chapter 7)
See HCI / HCD (Chapter 5)
Ubiquitous computing: smart devices, environments and interaction 71 71

72. HCD: System Model for Users vs. Users’ Model of System
What model of the system does it project to the user?
What model does the user have of the system?
What if these models differ?
Ubiquitous computing: smart devices, environments and interaction 72 72

73. HCD Design: Conceptual Models & Mental Models
Amazing number of everyday things & objects
????
Very challenging for people to learn to operate and understand many devices of varying degrees of complexity if the interaction with each of them is unique.
Complexity of interacting with new machines cm be reduced. How?
Ubiquitous computing: smart devices, environments and interaction 73 73

74. HCD Design: Conceptual Models
Discuss some example conceptual models
Ubiquitous computing: smart devices, environments and interaction 74 74

75. HCD Design: Affordances
Complexity of interacting with new systems is reduced if:
they have parts that provide strong clues on how to operate themselves.
These are referred to as affordances
What are examples of physical UI affordances?
Ubiquitous computing: smart devices, environments and interaction 75 75

76. HCD Design: Virtual Affordances
Many analogue physical objects being replaced by virtual computer UIs
Virtual UI affordances are being increasing important.
How to design virtual UI affordances?
Can link virtual objects or widgets in it to related & familiar physical world objects
Challenges in linking widgets to familiar physical objects?
Ubiquitous computing: smart devices, environments and interaction 76 76

77. HCD: Multiple Prototype Designs
Example: Consider PVM Scenario (Chapter 1)
What type of design?
Is there only 1 type of design for recording / playing / transmitting multimedia? ?
Consider the requirements:
Ubiquitous computing: smart devices, environments and interaction 77 77

78. Ubiquitous computing: smart devices, environments and interaction
HCD: Evaluation
Summative versus Formative Evaluation
Summative
Conventional
To verify Design
Formative
HCD
Ubiquitous computing: smart devices, environments and interaction 78 78

79. HCD: System Evaluation Methods
Can use similar techniques to gathering user requirements.
Ubiquitous computing: smart devices, environments and interaction 79 79

80. Ubiquitous computing: smart devices, environments and interaction
Overview
HCI, eHCI & iHCI
eHCI use in 4 Widely Used Devices
iHCI use in accompanied smart devices
iHCI use in wearable and implanted smart devices
Human Centred Design (HCD)
User Models: Acquisition & Representation 
iHCI Design
Ubiquitous computing: smart devices, environments and interaction 80 80

81. User Modelling: Design Choices
Implicit vs. explicit models
User instance (Individual) modelling versus user (stereo)type modelling
Static versus dynamic user models
Generic versus application specific models
Content-based versus collaborative user models
Ubiquitous computing: smart devices, environments and interaction 81 81

82. User Modelling Design: Implicit vs. Explicit models
Systems can either use
Explicit feedback
Implicit feedback
Often these can be combined to improve the user model.
Hybrid user models may also be used.
Stereotype user model to reduce individual user values by trying to suggest default values from matching user stereotypes.
Ubiquitous computing: smart devices, environments and interaction 82 82

83. Indirect User Input and Modelling
Context Aware user models
Use history of previous interactions
Personalise user input to filter user selections of contents and Services
Use context values such as location, entity, activity and time. And understand the current situation.
Ubiquitous computing: smart devices, environments and interaction 83 83

84. Direct User Input and Modelling
User requirements & user model built using:
Single-shot versus Multi-shot user input
Static versus Dynamic input
Ubiquitous computing: smart devices, environments and interaction 84 84

85. Ubiquitous computing: smart devices, environments and interaction
User Stereotypes
Past user interactios and predict user attribute values with confidence.
Stereotype: infers user model from small number of facts using a larger set of facts from a group user model.
Design challenge when user who may not fit a stereotype and dealing with too many inputs or incomplete inputs.
Ubiquitous computing: smart devices, environments and interaction 85 85

86. Modelling Users’ Planned Tasks and Goals
Users often interact purposely with a system in a task- driven way, to achieve a particular goal.
Several ways to analyse and model user tasks:
Hierarchical Task Analysis or HTA
Etc
Ubiquitous computing: smart devices, environments and interaction 86 86

87. HCD: Functional Requirements
Ubiquitous computing: smart devices, environments and interaction 87 87

88. Multiple User Tasks and Activity Based Computing
Use tasks as part of activities that require access to services across multiple devices,
Devices can be used by different types of people
Users are engaged in multiple concurrent activities
Users are engaged in activities which may occur across multiple physical environments,
Activities may be shared between participants
Activities on occasion need to be suspended and resumed.
Ubiquitous computing: smart devices, environments and interaction 88 88

89. Situation Action versus Planned Action Models
2 basic approaches to task design
Planned actions:
Pre-Planning
HTA Analysis
Situated action:
Assessment at each step what the optimum next local action is that bring user nearer to his or her goal.
Ubiquitous computing: smart devices, environments and interaction 89 89

90. Ubiquitous computing: smart devices, environments and interaction
Overview
HCI, eHCI & iHCI
eHCI use in 4 Widely Used Devices
iHCI use in accompanied smart devices
iHCI use in wearable and implanted smart devices
Human Centred Design (HCD)
User Models: Acquisition & Representation
iHCI Design 
Ubiquitous computing: smart devices, environments and interaction 90 90

91. Ubiquitous computing: smart devices, environments and interaction
iHCI
iHCI Model Characteristics
User Context Awareness
Intuitive and Customised Interaction
Personalisation
Affective Computing
iHCI Design Heuristics and Patterns
Ubiquitous computing: smart devices, environments and interaction 91 91

92. Ubiquitous computing: smart devices, environments and interaction
Types of User Model
Several related terms & kinds of user model are differentiated
User Models
Personal Profiles
User contexts
Application / User requirements
System Models
Mental Models
Conceptual Models
Ubiquitous computing: smart devices, environments and interaction 92 92

93. User Context Awareness
User context aware can be exploited to beneficially lessen the degree of explicit HCI needed.
User context-awareness is a sub-type of general context- awareness (Chapter 7)
User context-awareness can include:
Social environment context
Users’ physical characteristics and capabilities for HCI
User presence in a locality or detected activity
User identity (Section ).
User planned tasks and goals (Section 5.6.4).
Users’ situated tasks (Sections 5.6.5, 5.6.6).
User emotional state (Section 5.7.5)
Ubiquitous computing: smart devices, environments and interaction 93 93

94. Intuitive and Customised Interaction
Are current computer systems dominated by MTOS based devices & use of desktop UI metaphor intuitive?
E.g., ??
E.g., ??
E.g., etc
Ubiquitous computing: smart devices, environments and interaction 94 94

95. Intuitive and Customised Interaction
Moran & Zhai propose 7 principles to evolve desktop model into more intuitive model for UbiCom
From Office Container to Personal Information Cloud
From desktop to a diverse set of visual representations
From Interaction with 1 device to interaction with many
From Mouse & Keyboard to  Interactions & modalities
Functions may move from Applications to Services
From Personal to Interpersonal to Group to Social
From low-level tasks to higher level activities
Ubiquitous computing: smart devices, environments and interaction 95 95

96. Ubiquitous computing: smart devices, environments and interaction
Personalisation
Personalisation: tailoring applications & services specifically to an individual’s needs, interests, preferences
Adaptation of consumer product, electronic or written medium, based on person profile
Applications of personalisation
targeted marketing
product & service customisation including information filtering
CRM
Ubiquitous computing: smart devices, environments and interaction 96 96

97. Personalisation: Benefits
???
Ubiquitous computing: smart devices, environments and interaction 97 97

98. Personalisation: Challenges (Cons)
???
Ubiquitous computing: smart devices, environments and interaction 98 98

99. Ubiquitous computing: smart devices, environments and interaction
Personalisation
Personalisation: a more complete model of user-context that is more reusable and persists:
????
2 key issues:
design of model so that it can be distributed and shared
dynamic vs. static task-driven user preference contexts
Ubiquitous computing: smart devices, environments and interaction 99 99

100. Personalisation: Mechanisms
Instructors can add more slides about how personalisation mechanisms, e.g., recommender systems, work here or delete this slides
Ubiquitous computing: smart devices, environments and interaction
100
100

101. Affective Computing: Interactions using Users’ Emotional Context
Affective computing: computing relates to, arises from, or influences emotions.
Applications include:
???
Design challenges for affective computing with those for:
determining the user context
developing more complex human-like intelligence models
Ubiquitous computing: smart devices, environments and interaction
101
101

102. Ubiquitous computing: smart devices, environments and interaction
Affective Computing
Pickard (2003) identified six design challenges:
Range & modalities of emotion expression is broad
People’s expression of emotion is idiosyncratic & variable
Cognitive models for human emotions are incomplete
Sine qua non of emotion expression is the physical body but computers not embodied in the same way
Emotions are ultimately personal and private
No need to contaminate purely logical computers with emotional reactiveness
Ubiquitous computing: smart devices, environments and interaction
102
102

103. iHCI: Design Heuristics and Patterns
Many different higher-level HCI design usability / user experience criteria have been proposed by different HCI designers to promote good design of HCI interaction.
Many different HCI heuristics (rules of thumb derived from experience) have proposed to support HCI criteria
Specific guidance is needed to engineer UIs to comply with these usability & user experience HCI principles.
UI design patterns can support HCI usability principles and then be mapped into lower-level more concrete design patterns
Ubiquitous computing: smart devices, environments and interaction
103
103

104. iHCI: Design Heuristics and Patterns
Example iHCI patterns include:
Ubiquitous computing: smart devices, environments and interaction
104
104

105. iHCI: Design Patterns & Heuristics
Instructors can propose many more examples here or delete this slide.
Ubiquitous computing: smart devices, environments and interaction
105
105

106. iHCI: Engineering iHCI Design Patterns
Can propose simplify design models along 2 dimensions that are interlinked
Organisation / structural models versus time-driven interaction models
Front-end / Presentation (UI) interaction versus back-end system actions that support this interaction
Need to organise UI widgets or objects at UI
Need to organise and link presentation to actions
Need to design interaction with these widgets
(see next slide as an example)
Ubiquitous computing: smart devices, environments and interaction
106
106

107. iHCI: Engineering iHCI Design Patterns
Image Search
Ubiquitous computing: smart devices, environments and interaction
107
107

108. Ubiquitous computing: smart devices, environments and interaction
Overview
HCI, eHCI & iHCI 
eHCI use in 4 Widely Used Devices 
iHCI use in accompanied smart devices 
iHCI use in wearable and implanted smart devices 
Human Centred Design (HCD) 
User Models: Acquisition & Representation 
iHCI Design 
Ubiquitous computing: smart devices, environments and interaction
108
108

109. Ubiquitous computing: smart devices, environments and interaction
Summary
A human centred design process for interactive systems specifies four principles of design: the active involvement of users and a clear understanding of user and task requirements; an appropriate allocation of function between users and technology based upon the relative competence of the technology and humans; iteration is inevitable because designers hardly ever get it right the first time; a multi-disciplinary approach to the design.
Human centred design life-cycle involves user participation throughout four main sets of activities: defining user tasks and the (physical, ICT) environment context; defining user and organisational requirements; iterative design prototyping and validation against the requirements.
Ubiquitous computing: smart devices, environments and interaction
109
109

110. Ubiquitous computing: smart devices, environments and interaction
Summary
To enable humans to effectively interact with devices to perform tasks and to support human activities, systems need to be designed to support good models of user interfaces and processes of human computer interaction.
Users can be modelled directly and indirectly. User task models can be modelled as task plans or as situated actions. iHCI design concerns three additional concerns: support for natural (human computer) interaction; user models including models of emotions which can be used to anticipate user behaviour and user context awareness including personalisation.
Some design patterns and heuristics oriented towards iHCI are described.
Ubiquitous computing: smart devices, environments and interaction
110
110