Evaluation of applications in main control room

Relatively small group of users Evaluation at GSI Relatively small group of users Users can theoretically be contacted at any time Limited by beamtime requirements Sometimes negative experience of giving in the past Operators want to give feedback  placed on notice board Feedback is given directly by users to application developer Few testings conducted. Feedback and evaluation results not publicly visible. No structured interpretation and consolidation of findings No clear back channel to application development

Evaluation vs. feedback Evaluation results can differ from feedback Interaction data Design feedback Jacob Nielsen: “To design the best UX, pay attention to what users do, not what they say. Self-reported claims are unreliable, as are user speculations about future behavior. Users do not know what they want.”

User-centered design Identify the need for human centered design Specifiy context of use System satisfies specified requirements Specifiy requirements Evaluate designs Produce design solutions

Evaluation methods Heuristic evaluation Usability testing Surveys & questionnaires (e.g. SUS, UEQ) Experience maps Contextual interviews / inquiry Focus groups & workshops

Questions Which evaluation methods should be used? What should be the role of operators during the process? Collect data globally or application-specific?