Usability Assessment Framework for ACGs using AHP USABILITY ASSESSMENT FRAMEWORK FOR CODE GENERATORS USING ANALYTICAL HIERARCHY PROCESS Joshua R. Dolecal Dahai Liu Remzi Seker Andrew Kornecki Department of Human Factors and Systems Embry-Riddle Aeronautical University Daytona Beach, Florida

Usability Assessment Framework for ACGs using AHP 2 Outline Usability Code Generation Rationale Framework AHP Results

Usability Assessment Framework for ACGs using AHP 3 Usability Also called: Usability Engineering, Human Factors, Ergonomics, and User Centered Design. UCD is defined as the practice of designing products so that users can perform required use, operation, service, and supportive tasks with a minimum of stress and maximum of efficiency. (Woodson,1981) Nielson (1993) defines Usability as the overall acceptability of a system.

Usability Assessment Framework for ACGs using AHP 4 Usability Criteria Learnability –How easy something is to learn Efficiency –Of the product (how well it aids users in performing tasks) Memorability –How easy it is to remember skills learned/used Errors –Error prevention, correction, and identification Satisfaction –How well users are satisfied with the product

Usability Assessment Framework for ACGs using AHP 5 Process of Generating Code Select template Create a model/chart/diagram Select output language (C/C++, Java, Ada…) Generate Code (Stephens, 2002)

Usability Assessment Framework for ACGs using AHP 6 Rationale ACG is relatively new ACGs will change the way companies work and SW is developed Usability need to be addressed, for special users –Experts (Computer Programmers) Little research done

Usability Assessment Framework for ACGs using AHP 7 Framework

Usability Assessment Framework for ACGs using AHP 8 Framework General Usability Criteria –(Nielson, 1993) Basic well accepted usability Specific Usability Criteria –(Stephens, 2002 & Whalen, and Heimdahl, 1998 & Maclay, 2000, etc.) –Elements that are unique to ACGs according to the current literature

Usability Assessment Framework for ACGs using AHP 9 Method AHP framework constructed –Usability criteria and –ACG functional analysis 8 Graduate students gave pairwise comparison. (~6 months exposure with ACGs) AHP weight Consistency

Usability Assessment Framework for ACGs using AHP 10 Analytical Hierarchy Process 3 Main steps –Establish a hierarchy –Calculate pair-wise comparison weights –Check for consistency –Apply weights to gathered data (objective/subjective) Process can be Automated using Expert Choice –

Usability Assessment Framework for ACGs using AHP 11

Usability Assessment Framework for ACGs using AHP 12 AHP Values Thomas Saaty (1982)

Usability Assessment Framework for ACGs using AHP 13 Thomas Saaty (1982)

Usability Assessment Framework for ACGs using AHP 14 4 Step Consistency Process Thomas Saaty (1982)

Usability Assessment Framework for ACGs using AHP 15 Consistency When dealing with many criteria/variables, people are not going to be perfectly consistent. It is up to you to review their decisions with them such that the decision maker understand transitivity. –E.g. If A > B and B > C then A > C

Usability Assessment Framework for ACGs using AHP 16 Results

Usability Assessment Framework for ACGs using AHP 17 Results

Usability Assessment Framework for ACGs using AHP 18 Results

Usability Assessment Framework for ACGs using AHP 19 Results Semantic consistency and User satisfaction received the highest weights, but traversing up the hierarchy we see the Translator and then Specific Usability Criteria received the highest weight A customized answer was produced –Best product for your situation –Does not label products as better than others –Allows a combination of subjective & objective data

Usability Assessment Framework for ACGs using AHP 20 Questions, Comments?