1 MultiCom, a platform for the design and the evaluation of interactive systems. MultiCom, a platform for the design and the evaluation of interactive systems. J. Caelen, CLIPS-IMAG, GRENOBLE

2 Needs The designers of interactive systems need tools and methods for system design and evaluation. The academic research teams are sought frequently by the industry for studies on user needs, and on user evaluation. It would be more benefit, to create a common platform for the both partners. The platform would be more a service centre devoted to the experimentation than a classical academic or industrial institution.

3 Objective A platform for the design and the test of interactive systems, allowing to : - study the usage and the usability of the interactive systems under real conditions of using - design and assess complex systems to respect with the usage and the usability - analyse the human behaviour (under definite scenarios of using) from collected data

4 Expected benefits Improved feasibility assurance of development projects, Improved procedures, methods and concepts, Reduced development costs and time, Improved product quality and increased flexibility and adaptability, Progress towards the establishment of software engineering standards, Improved guarantees to end-users that a product has been developed following best software and cognitive engineering practice.

5 Rationale l MultiCom is an approach to software development which emphasises the use of operational models throughout the lifecycle. MultiCom is a complete platform for the design and the evaluation of the interactive systems, including all the stages of the development cycle and supporting tools for sociologic observations and ergonomic tests (by taking into account human factors). l MultiCom provides training to the end-users as soon as the system comes in the exploitation phase.

6 The complete lifecycle AObserving the current usage, predicting the future usage, BDefining the specification notebook, CDefining the using scenario and assessing it, DRefining the specification on the diagnosis delivered in C, E Developing a mock up, FEvaluating the use and the usability of the mock up, GModifying after debriefing and debugging, HDeveloping another version of the mock up, I Returning at the step F while it is necessary, J Evaluating the final mock up, K Developing a prototype, L Pushing it on the market, M Training end users if needed, N Analysing the market returns.

7 The pre-development lifecycle Specific approach : to enrich the product concept as soon as possible, before the first stage of the pre- development, using a complete parameters palette (a) sociological questionnaires, (b) human factors grid and task analyses, (c) data collected from users Concept Specification Diagnosis Criteria Functions Virtual interface Scenario Test Functions Mock up 1 Devel. Test Mock up 2 Criteria Correct.

8 The design cycle To integrate as soon as possible the human factors in the design process : three phases Information Elicitation Phase Design Synthesis Phase Design Specification Phase Concept Specification Diagnosis Criteria Functions Virtual interface Scenario Test Functions Mock up 1 Devel. Test Mock up 2 Criteria Correct.

9 MultiCom Tools l Simulation l Measure l Analysis The platform : rooms workstations audio-video

10 Tools : simulation l The simulation consists in : — simulate a product/system using some virtual reality tools, — or simulate a situation (using wizard of Oz method), — or simulate the situation and the product together.

11 Tools : measure l The simulation l The measure consists in : — choosing adequate criteria, — selecting parameters, — defining scenarios and test protocols, — collecting data about the user behaviour, — applying criteria on the selected parameters.

12 Tools : analysis l The simulation l The measure l The analysis consists in : — making functional analysis or task analysis before the specification phase, — applying reverse engineering method during the specification phase in order to validate it, — applying statistical methods on collected data after the development phase, — studying spoken dialogues if any.

13 l By prediction : from socio-cognitive theory : to specify the design and to specify the system. Materials : questionnaires, task trees, sociological criteria grid Methods

14 Methods l By prediction By simulation : (virtual prototype, wizard of Oz, etc.) in order to refine functionality and to control the level of usability of the prototype, during the development phase.

15 Methods l By prediction l By simulation l By direct observation on the prototype working : in applying assessment protocol, Materials : scenario editor, observation scripts and annotation, data collecting software, data analysis tools, simulation tools.

16 Methods l By prediction l By simulation l By direct observation : l By indirect observation : in verifying the conformity of the system to the specifications. Materials : ergonomic expertise.

17 Benefits to the developing countries l Last years, usability budgets have been steadily growing... l Costs for a usability labs ? Not really a problem... – Sharing equipment – Sharing specialists – Portable equipment l Usability laboratories can also be used for certain variants of heuristic evaluation (for the design phase but also for the adaptation to the user and for the “computerisation” phase) The single-most important decision in usability engineering is simply to do it !

18 Conclusion MultiCom is… l A competence network (sociology, ergonomics, computer science) l A complete platform of experimentation and measure l A training centre (seminars, short courses) l A resource centre offering services on sociology and human factors areas