Grenoble Informatics Laboratory (LIG) HCI Engineering Research Group (IIHM) Helping the design of Mixed Systems Céline Coutrix PhD Supervisor: Laurence Nigay

2 Problem statement merge physical and digital worlds lot of applications

3 Motivation Analysis design prototyping evaluation

4 Goals Support for the design –Description –Exploration –Comparison Support for prototyping

5 Methodology Related Work Concrete Mixed Systems: - RAZZLE - ORBIS - Snap2Play Software Tool - Flexible - Extendable - … Mixed Interaction Model - Descriptive - Generative - Comparative Conceptual Work Software Tools Examples of systems Design Software Development Evaluation

6 Example: ORBIS

7 Mixed Interaction Model Physical object Physical properties Digital object Digital properties Mixed object Physical properties Digital properties ?

8 Mixed Interaction Model Interaction Modality m = (d, l) speech = (microphone, natural language) Mixed object Physical properties Digital properties ?

9 Mixed Interaction Model Linking Modality Physical properties Digital properties Input device Output device Output language Input language

10 Design Space Exploration List of Pictures

11 Comparison of Design Alternatives acquired materialized non acquired non materialized acquired non materialized non acquired materialized

12 Comparison of Design Alternatives List of Pictures

13 Design Space Exploration List of Pictures

14 size resolution dynamic static local global Comparison of Design Alternatives Capitalizing existing characteristics

15 Design Space Exploration List of Pictures Image Material Shape

16 Design Space Exploration

17 Design Space Exploration

18 Design Space Exploration List of Pictures Image Material Shape Orientation

19 Comparison of Design Alternatives non sensed generated sensed non generated sensed generated non sensed non generated

20 Comparison of Design Alternatives Image Orientation of the shape Material Shape Orientation of the Image

21 Design Space Exploration List of Pictures Top (camera, computer vision) (accelerometer, transformation 1 ) (magnetometer, transformation 2 ) Image Material Shape Orientation

22 Outline Support for the design Support for prototyping Validation

23 Support for Prototyping

24 Support for Prototyping

25 Support for Prototyping

26 Support for Prototyping Integration of lower-level toolkits –ARToolKit –Phidgets –Interface-Z MIDI sensors

27 Support for Prototyping

28 Support for Prototyping

29 Support for Prototyping

30 Outline Support for the design –Description –Exploration –Comparison Support for prototyping Validation

31 Validation 1.Mixed Interaction Model evaluation Power/EvaluationEmpiricalTheoretical Descriptive Modeling existing systems Comparative Classifying existing systems Comparison with other taxonomies Generative Design of concrete systems Ergonomic evaluation in a real exploratory design situation Exploratory design task [Cognitive Dimensions of Notations Framework] Interconnection between existing approaches [Olsen]

32 Validation 2.Software Tool evaluation empiricaltheoretical Pairs of designer+developer prototyping the same system working with/without the toolkit Evaluating User Interface System Research [Olsen]

33 Ongoing Work Related Work Concrete Mixed Systems : - RAZZLE - ORBIS - Snap2Play Software Tool - Flexible - Extendable - … Mixed Interaction Model - Descriptive - Generative - Comparative Conceptual Work Software Tools Examples of systems Design Software Development Evaluation

Grenoble Informatics Laboratory (LIG) HCI Engineering Research Group (IIHM) QUESTIONS?