1. GAN-MVL 2.1 Heuristic Evaluation
Silvia Gabrielli, Roberto Ranon, Markus Hodapp

2. TOC Introduction: Heuristic evaluation method
The Scenario used for the evaluation
Main results
Proposal for future evaluations (Lab simulations)

3. Heuristic Evaluation
Expert review: system compliance with a set of usability heuristics [Nielsen ’94]
Visibility of system status
Match between system and the real world
User control and freedom
Consistency and standards
Error prevention
Recognition rather than recall
Flexibility and efficiency of use
Aesthetic and minimalist design
Help users recognize, diagnose, and recover from errors
Help and documentation
Quick turnaround for the iterative design process
Typically 3-5 experts involved (75-80% of system flaws detected)

4. GAN-MVL HE
5 Usability Experts involved [University of Udine & University of Mannheim)
GAN-MVL prototype 2.1 [
Type of functionalities tested:
a) Help, download and installation of VI plugins
b) Collaboration with a remote expert via Skype
c) Remote access to an Instrument (FuncGen) via high-resolution video
d) Remote control of a VI
e) Interaction with Logbook, Calendar, Resource Browser, Http Wizard, Tunnel Monitor, User Profile
Experts were provided with a HE template to prepare their Report

5. GAN-MVL HE GAN-MVL HE Scenario 1a: Access to GAN-MVL as a User
Tasks performed:
Registration as a new user
Activation of an account and change of password
Browsing of the Knowledge Management area, of the Help system to find and read the User Guide
In the GAN Portal, Skype contact of the GAN-MVL Administrator to get support and access to the Function Generator instrument
Download and installation of the High resolution viewer (from the download area)
Remote check of the value of the signal generated by the Function Generator instrument
Request (to the Admin.) to change current account from user to administrator
Booking of an event in the calendar
Scenario 1b: Use of GAN-MVL as Administrator
Download and installation of LabView Runtime engine software for remote control of the Function Generator instrument
Modification of the value of signal generated by the instrument
Editing of a message in the Logbook about the operation previously performed
Use of the HTTP Wizard to add a new instrument to the Control Room toolbar
Access (from the GAN node) to the Tunnel Monitor to control (and eventually drop) the connections currently active (as in the case of an emergency)
Logout
GAN-MVL HE

6. HE results Synthesis of main flaws detected, reporting: Issue title
Description of each issue found
Where each issue occurs
Number of experts that notified each issue
Severity ranking assigned (Low-Medium-High)
Recommendation(s) provided
Results

7. Laboratory simulations & tests
In-depth analysis of key issues identified by HE (and SOA)
Experimental observation of breakdowns in system use
Based on predefined collaborative scenarios of use
Participation of a confederate user to elicit key situations
(Longitudinal) Log data and observations collected

8. Laboratory simulations & tests
Proposal for a future study on :
GAN-MVL support to remote troubleshooting activities
Expert-Operator audio-video interaction during troubleshooting
Expert-Operator sharing of bidirectional 2D representations of instruments (Labview VI)
Expert-Operator sharing and control of BD visual representations of VI
Analysis of:
time taken to arrive at a clear definition of the problem
time taken to establish a common referent, to provide instructions, to solve the problem
behavioural aspects of the interaction, communication patterns, interactional difficulties observed
post session feedback, participants observations
conclusion and insights for further testing

9. Laboratory simulations & tests
Support from WP8 partners needed for:
Defining a realistic and complex troubleshooting scenario to observe
Set up appropriate devices-instruments for the troubleshooting scenario
Identifying participants to involve in the test (expert-operator roles)
Results:
Will inform future lab simulations & tests
Will inform GAN-MVL final release
Will support the design of the field tests