GAN-MVL 2.1 Heuristic Evaluation Silvia Gabrielli, Roberto Ranon, Markus Hodapp

TOC Introduction: Heuristic evaluation method The Scenario used for the evaluation Main results Proposal for future evaluations (Lab simulations)

Heuristic Evaluation Expert review: system compliance with a set of usability heuristics [Nielsen ’94] Visibility of system status Match between system and the real world User control and freedom Consistency and standards Error prevention Recognition rather than recall Flexibility and efficiency of use Aesthetic and minimalist design Help users recognize, diagnose, and recover from errors Help and documentation Quick turnaround for the iterative design process Typically 3-5 experts involved (75-80% of system flaws detected)

GAN-MVL HE 5 Usability Experts involved [University of Udine & University of Mannheim) GAN-MVL prototype 2.1 [http://gan.grid.elettra.trieste.it:8080/gridsphere] Type of functionalities tested: a) Help, download and installation of VI plugins b) Collaboration with a remote expert via Skype c) Remote access to an Instrument (FuncGen) via high-resolution video d) Remote control of a VI e) Interaction with Logbook, Calendar, Resource Browser, Http Wizard, Tunnel Monitor, User Profile Experts were provided with a HE template to prepare their Report

GAN-MVL HE GAN-MVL HE Scenario 1a: Access to GAN-MVL as a User Tasks performed: Registration as a new user Activation of an account and change of password Browsing of the Knowledge Management area, of the Help system to find and read the User Guide In the GAN Portal, Skype contact of the GAN-MVL Administrator to get support and access to the Function Generator instrument Download and installation of the High resolution viewer (from the download area) Remote check of the value of the signal generated by the Function Generator instrument Request (to the Admin.) to change current account from user to administrator Booking of an event in the calendar Scenario 1b: Use of GAN-MVL as Administrator Download and installation of LabView Runtime engine software for remote control of the Function Generator instrument Modification of the value of signal generated by the instrument Editing of a message in the Logbook about the operation previously performed Use of the HTTP Wizard to add a new instrument to the Control Room toolbar Access (from the GAN node) to the Tunnel Monitor to control (and eventually drop) the connections currently active (as in the case of an emergency) Logout GAN-MVL HE

HE results Synthesis of main flaws detected, reporting: Issue title Description of each issue found Where each issue occurs Number of experts that notified each issue Severity ranking assigned (Low-Medium-High) Recommendation(s) provided Results

Laboratory simulations & tests In-depth analysis of key issues identified by HE (and SOA) Experimental observation of breakdowns in system use Based on predefined collaborative scenarios of use Participation of a confederate user to elicit key situations (Longitudinal) Log data and observations collected

Laboratory simulations & tests Proposal for a future study on : GAN-MVL support to remote troubleshooting activities Expert-Operator audio-video interaction during troubleshooting Expert-Operator sharing of bidirectional 2D representations of instruments (Labview VI) Expert-Operator sharing and control of BD visual representations of VI Analysis of: time taken to arrive at a clear definition of the problem time taken to establish a common referent, to provide instructions, to solve the problem behavioural aspects of the interaction, communication patterns, interactional difficulties observed post session feedback, participants observations conclusion and insights for further testing

Laboratory simulations & tests Support from WP8 partners needed for: Defining a realistic and complex troubleshooting scenario to observe Set up appropriate devices-instruments for the troubleshooting scenario Identifying participants to involve in the test (expert-operator roles) Results: Will inform future lab simulations & tests Will inform GAN-MVL final release Will support the design of the field tests