This slide deck is part of a presentation given by Saul Greenberg at a UIST 2007 panel on Evaluating user interface systems research (see paper by Dan.

Slides:

Advertisements

Similar presentations

Design, prototyping and construction

Advertisements

CSCI 4163 / CSCI 6610, Winter 2014 Human-Computer Interaction web.cs.dal.ca/~hawkey/4163 Dr. Kirstie Hawkey,

Analytical methods for IS professionals ISYS3015 What is qualitative research?

The Sketchbook. idea 1 Why Sketches The problem typically fixate on first idea local hill climbing idea 1.

Department of Computer Science

Usability Evaluation Considered Harmful Some of the time

Saul Greenberg CPSC 481 Foundations and Principles of Human Computer Interaction James Tam.

CS 498 Senior Seminar Students will research a current topic in computer science, write a paper on that topic, and make an oral presentation.

USABILITY AND EVALUATION Motivations and Methods.

Saul Greenberg CPSC 481 Foundations and Principles of Human Computer Interaction James Tam.

Graphical User Interfaces Design and usability Saul Greenberg Professor University of Calgary Slide deck by Saul Greenberg. Permission is granted to use.

Foundations and Principles of Human Computer Interaction Slide deck by Saul Greenberg. Permission is granted to use this for non-commercial purposes as.

Retrieval Evaluation. Introduction Evaluation of implementations in computer science often is in terms of time and space complexity. With large document.

Recall The Team Skills 1. Analyzing the Problem (with 5 steps) 2. Understanding User and Stakeholder Needs 1. Interviews 2. Workshops 3. Brainstorming.

Introduction to HCI Marti Hearst (UCB SIMS) SIMS 213, UI Design & Development January 21, 1999.

1 Contextual Interview Shahnewaz A. Jolly CPSC 681: Research Methods in Human Computer Interaction Instructor: Dr. Saul Greenberg Date: November 4, 2009.

CHI 2009 Review Process Changes area-based submissions and sub-committees.

NOTE: To change the image on this slide, select the picture and delete it. Then click the Pictures icon in the placeholder to insert your own image. PROJECT.

Chapter 8: Problem Solving

Freehold Borough Teacher Evaluation Training KICKOFF PRESENTATION March 8, 2013 Presented by: Joy Forrest (FLC) Jennifer Donnelly (PAE) Rich Pepe (FIS)

Human-Computer Interaction IS/HCC 760 Fall 2011 Shaun Kane.

CS 498 Senior Seminar Students will research a current topic in computer science, write a paper on that topic, and make an oral presentation.

What is the Next Generation of Human-Computer Interaction? Introduction.

Section 1: The Nature of Science

W/ slides from Greenberg & Buxton’s CHI 2008 presentation “Usability Evaluation Considered Harmful (Sometimes)”

Human-Computer Interaction IS/HCC 760 Fall 2012 Shaun Kane.

Chapter 4 Paradigms (additional materials). Beginnings – Computing in 1945 Harvard Mark I –Picture from

Assessing the Frequency of Empirical Evaluation in Software Modeling Research Workshop on Experiences and Empirical Studies in Software Modelling (EESSMod)

Software Design Experiences 2009, adapted from d.school.bootcamp.2008 DP 0 The Wallet Project.

Identify a Health Problem Qualitative Quantitative Develop Program -theory -objectives -format -content Determine Evaluation -design -sampling -measures.

What is Science? Chapter 1, Lesson 1. Using one or more of your senses and tools to gather information. observing.

Welcome! SSCI-S 100a Lecture 1. Today’s agenda Introduce myself Introduce the course Introduce some of the key terms.

Chapter 1 Introduction to Research in Psychology.

© 2006 by The McGraw-Hill Companies, Inc. All rights reserved. 1 Chapter 1 Introduction to Research in Communication Research: –Process of asking questions.

Sociology 12. Outcome analyze a variety of appropriate sociological research methods Describe common sociological research methods

Teaching with CHRONOS Data and Tools A Framework for Design Cathy Manduca Science Education Resource Center Carleton College June 13, 2006.

CPSC 581 Human Computer Interaction II Interaction Design.

© 2013 TILA 1 Organizing telecollaboration projects TILA Teacher Training Teacher as researcher.

CPD 3 - Advanced Publishing Skills 1 - How to Get Published and to Continue to Get Published in Leading Academic Journals Professor Tarani Chandola with.

School of Engineering and Information and Communication Technology KIT305/607 Mobile Application Development Week 7: Usability (think-alouds) Dr. Rainer.

STEM Fundamentals Foundation Concepts for Teaching Problem Solving The Model #302 telephone was the standard for forty years. How long will this one last?

MIS40850 Comparative Software Engineering Process Frameworks - Part 2 -

October 4, 2012.

Methodology Overview basics in user studies Lecture /slide deck produced by Saul Greenberg, University of Calgary, Canada Notice: some material in this.

Welcome! SSCI-E 100a Lecture 1.

Research Methods for Computer Science

Human Computer Interaction Slide 1

Children’s Engineering

Computational Reasoning in High School Science and Math

CPSC 581 Human Computer Interaction II

Methodology Overview 2 basics in user studies Lecture /slide deck produced by Saul Greenberg, University of Calgary, Canada Notice: some material in this.

Research Methods: Observation

General Education Assessment Subcommittee Report

Recall The Team Skills Analyzing the Problem (with 5 steps)

AP computer science PRINCIPLES

Paradigms (additional materials)

QNT 561 Education for Service/tutorialrank.com

Design, prototyping and construction

9/6 welcome Find a seat. When someone joins your table, introduce yourself and tell them what you are most excited for this year

WHAT IS COMPARATIVE POLITICS?

DP The Wallet Project d.school.bootcamp.2008.

What processes do scientists use when they perform scientific investigations? Chapter Introduction.

WHAT IS COMPARATIVE POLITICS?

User System Interface CSC

Interface Design and Usability

Design, prototyping and construction

Paradigms (additional materials)

Suggested Layout ** Designed to be printed on A3 paper in an assortment of colours. This is directly linked to the Computer Science Specification.

CS 791Graduate Topics in Computer Science [Software Engineering]

Presentation transcript:

This slide deck is part of a presentation given by Saul Greenberg at a UIST 2007 panel on Evaluating user interface systems research (see paper by Dan Olson in ACM DL) Its content was based on a paper written by Saul Greenberg and Bill Buxton, available at: http://grouplab.cpsc.ucalgary.ca/papers/2007/07-UsabilityHarmful-Greenberg-Report/07-usabilityHarmful-report2007-879-31.pdf

I need to do an evaluation

What’s the problem?

It won’t get accepted if I don’t. Duh!

Dogma Course Introduction

Good HCI Course Introduction

Time Learning Brian Gaines Breakthrough Replication Empiricism Theory major creative advances Replication ideas mimicked and altered; increasing experience Empiricism lessons formulated as empirical design rules Theory underlying causal systems Automation accepted Maturity used routinely Brian Gaines

Evaluation sweet spot Interface validation and refinement* Breakthrough Replication Empiricism Theory Automation Maturity Interface validation and refinement* *70% quantitative, 30% qualitative

Concepts, Sketches Breakthrough Replication Empiricism Theory Automation Maturity Concepts, Sketches

Memex Bush Concept

Unimplemented and untested design. Microfilm is impractical Unimplemented and untested design. Microfilm is impractical. The work is premature. Resubmit after you build and evaluate this design.

Sketchpad Sutherland Working idea

Complex controls, overly simplistic drawings Recommend you evaluate a trained draftsman using Sketchpad vs. paper and pencil with real drafting tasks

Augment Engelbart Deployed idea

While ‘tested’ by your team, this is not an objective validation.

What to do?

Reflection Bill Buxton, Sketching the User Experience

Cultural Appropriation Breakthrough Replication Empiricism Theory Automation Maturity Cultural Appropriation

Hypertext

What to do?

Case / field studies Cultural probes Reflection Extreme uses…

Evaluation Course Introduction

More Appropriate Evaluation Course Introduction

We decide what is good research

Cultural appropriation Time Learning Breakthrough Breakthrough Replication Replication Empiricism Empiricism Theory Theory Automation Automation Maturity Maturity But is this lack of tools really a bottleneck? Well, in the 1980s, Computer Scientist Brian Gaines introduced a model of how science technology develops over time, and how people learn from this development It unfolds from <READ> Breakthrough -> Invention Replication -> Research Empiricism -> Product Innovation Theory -> Product Lines Automation -> Low cost products Maturity -> Throw away products Sketches Interface validation Cultural appropriation Brian Gaines

grouplab.cpsc.ucalgary.ca/~papers/