Semiotics and semantic: tools for an effective appropriation of information, communication and health technologies Carine Duteil-Mougel, Anne Geslin-Beyaert, Laurent Billonnet, Nicole Pignier, Didier Tsala-Effa, Jean-Michel Dumas University of Limoges, Limoges, France Medetel 2012 The international eHealth, Telemedecine and Health ICT forum April 2012, Luxembourg

Outline Introduction Introduction Learning from experimentation I. I. What about semiotics? II. II. What about semantics? III. III. The object - subject interaction IV. IV. The appropriation of Communicant objects V. V. The Voice User InterfacesConclusion To the human-machine semiotic&semantic interface (HMSSI)

Learning from experimentation…  ICT : a wish but still an unsuitable system; o Feeling of disappointment. o Poor consideration of human being, user friendliness, personalization, dignity and respect. o Feeling of intrusion and stigmatization.  Training and information needed for a better appropriation of equipments. 1. Need for an ethical dimension. 2. Appropriation of ICT tools is a condition of success. … to the semiotic and semantic approach

1.Definition  Semiotics is a human science which analyzes systems of signification, whatever the language expressing them: texts, pictures, spaces, etc. 2. Semiotics of the ICT includes two main directions: i. multimedia, multimodal and hypertextual contents; ii.graphic and material interfaces. In both cases, the semiotician questions the conditions of perception of a text or an object on a digital support. I. What about semiotics?

3. Semiotics of the ICT in the field of health  The way the user, with his current skills and the gestural, cultural experiences he developed in the practices of his everyday life, perceives digital objects intended to help him.  As a consequence, the ICT object is acceptable only insofar as it interacts with the person through his own experiences (ex: a light path does not fit and tell the story of the person). The semiotician thinks out the coherence between the design of an ICT device and the perception the user can have of the interaction with this device.

1. Definition:  Semantics is the most recent of the branches of linguistics and is still in the process of being constituted.  We have adopted a semantic approach (developed within the Saussurian tradition) that we consider to be the most sensitive to diversity: interpretative semantics.  By privileging the study of meaning, interpretative semantics focuses on the Cultural Objects, rather than the sign, and defines meaning as interpretation. 2. Objectives: i. To describe a great diversity of Cultural Objects ii.To specify the incidence of social practices II. What about semantics?

3. Semantics of ICTs:  As Cultural Objects, ICTs belong to the human sphere (concept of Umwelt formulated by Jakob von Uexküll). This surround is composed of polysemiotics performances (involving many signs systems).  The meaning of these objects is then studied in relation with the actions to which they are associated.  The study of users’ discourses is very interesting from this point of view, and should lead to formulate recommendations for a more efficient appropriation of the ICTs, and a real implication of the persons. II. What about semantics?

 The user (subject) establishes a junction with the technological object: the relationship is called a “con-junction”. III. The object - subject interaction Social practice Presentational level Semiotic level Physical level Ex. mental representations Ex. speech and gesture (nonverbal communication) Ex. movement Semiotic mediation Semiotic mediation  Referring to the three levels stake in social practices (presentational, semiotic, physical), the object could become a mediator between the user and its environment. The term of “Semiotic mediation” is thusly used

 In terms of semiotics of anthropic zones, the appropriation expresses the way to go from the distal zone to the practical proximal zone. IV. The appropriation of Communicant objects Identity Zone (coincidence) Identity Zone (coincidence) Proximal Zone (adjacency) Proximal Zone (adjacency) Distal Zone (strangeness) Distal Zone (strangeness) FETISH Object is seen as familiar FETISH Object is seen as familiar IDOL Object is seen as technical IDOL Object is seen as technical  Our Hypothesis: the Voice User Interfaces (VUI) which have recourses to spoken language, make easier the pass from IDOL to FETISH, in comparison to the Graphical User Interfaces (GUI). The object which has the FETISH status is seen as "familiar" and that make easier its acceptation and its daily use. The problematic is to tip the ICT objects from IDOL to FETISH status. First person: I NOW Here First person: I NOW Here Second person: You NEAR PAST and FUTURE There Second person: You NEAR PAST and FUTURE There Third person: He/She/It/ That PAST and FUTURE Elsewhere Third person: He/She/It/ That PAST and FUTURE Elsewhere

 The Recognition of Continuously Spoken Sentences allows the user to address oneself from a natural way to the system without any preliminary specific learning to use the system.  The recourse of the natural dialogue (unlimited) begs the interrogation about the semantic interpretation, and underlines the role of the situational context.  Objective: Predict functional Voice User Interfaces which take into account the expectation and the use of languages (lexicon, syntax, semantics, pragmatics, etc.) V. The Voice User Interfaces

To the human-machine semiotic&semantic interface (HMSSI) Based on the vocal and/or the gesture approaches Several mono or multimodal interfaces techniques  dematerialized interfaces vocal interfaces (analysis and synthesis) virtual interfaces using gesture recognition  augmented reality and virtual assistive information haptic, visual or auditive augmentation as a an assistance haptic, visual or auditive augmentation as a validation  intuitive tactile sensitive processes as a projection of an intuitive gesture Conclusion

Some more information?… Contact  Laurent Billonnet Professor at the University of Limoges Professor at the University of Limoges ENSIL engineering School Director of the BSc « Home Automation for Elderly and Disabled People » Director of the BSc « Home Automation for Elderly and Disabled People »