1 Instructor: Vincent Duffy, Ph.D. Associate Professor of IE Lab 1 – Part 1 Review - Lifelong Learning Fri. Feb. 2, 2007 IE 486 Work Analysis & Design II

2 10 ways to look at a research paper systematically for critical analysis v 1. research ideas/question -what is the purpose? v 2. what background support? - literature review v 3. theoretical basis for analyzing question/hypothesis? v 4. applicability-practical contribution? v 5. theoretical contribution? v 6. appropriate methodologies for carrying out study? (determining variables, data collection, method used to test hypotheses, validity of measures and reliability) v 7. appropriate statistical analyses and assumptions? v 8. presentation of results: what do they really mean? v 9. conclusions drawn: are they reasonable? v 10. future work/research directions: any possibilities?

3 10 ways to look at a research paper systematically for critical analysis v 1. research ideas/question -what is the purpose? Confirm idea that humans treat computers as social actors Differentiate responses from different cultures A lot of research previously done re: computers, but not in relation to differences in cultures

4 10 ways to look at a research paper systematically for critical analysis v 2. what background support? - literature review v Previous studies by one author, and colleagues of the author (connecting idea that people interact with computers – in relation to social norms v People treat computer as social actors – teammate v Nass et al., 1996; Fogg; v Little systematic research in regard to the cultural issue v ‘Why’ appears to be answered.

5 10 ways to look at a research paper systematically for critical analysis v 3. theoretical basis for analyzing question/hypothesis? v Theory of reciprocity v Computers as social actors – as an extension of social- psychology literature v Paradigm developed by testing …individual social norms

6 10 ways to look at a research paper systematically for critical analysis v 4. applicability-practical contribution? v Results can be applied to interfaces for different cultures v Since humans use computers every day, it can be useful to know more about how we interact… v Results re: collectivist vs. individualist… v Marketing/brand loyalty issues. v Software on a computer…language may change, but other attributes maybe should also be considered for modification…

7 10 ways to look at a research paper systematically for critical analysis v 5. theoretical contribution? v Other studies may not consider cultural differences, but automatically assume applicability to another culture v (raises challenge to the research community to verify generalizability) v Challenge is addressed in this study for one specific social norm, giving example by which others may be considered and further studied. reciprocity (different than simply considering gesture or language, but more in terms of ways people communication, or assist one another).

8 10 ways to look at a research paper systematically for critical analysis v 6. appropriate methodologies for carrying out study? (determining variables, data collection, method used to test hypotheses, validity of measures and reliability) v Experiment, two different experiments to isolate the effects… v First experiment w/U.S. participants only; then repeated… v ? ?, ?, ?, v Ok if we trust their literature support; otherwise we don’t know – not a good paper for a student trying to learn about experimental methodologies.

9 10 ways to look at a research paper systematically for critical analysis v 7. appropriate statistical analyses and assumptions? v Statistical analyses shown in abstract: frequencies, percentages.

10 10 ways to look at a research paper systematically for critical analysis v 8. presentation of results: what do they really mean?

11 10 ways to look at a research paper systematically for critical analysis v 9. conclusions drawn: are they reasonable?

12 10 ways to look at a research paper systematically for critical analysis v 10. future work/research directions: any possibilities?