1. Methodology and Explanation XX50125 Lecture 3: Interviews and questionnaires Dr. Danaë Stanton Fraser

2. Methodology and Explanation 2005 Lectures 1.Introduction to Methods 2.Experiments 3.Interviews and questionnaires 4.Usability testing 5.Ethnography

3. Methodology and Explanation 2005 Why Interview? Good for obtaining information about –Opinions –Feelings –Goals (organisational, personal and system) –Things you didnt know you needed to know Bad for gathering (large volumes of) data

4. Methodology and Explanation 2005 Interview Preparation Decide objectives Read background material Decide whom to interview Set up the interview Decide interview structure Write down questions you will ask and pointers for questions you might ask

5. Methodology and Explanation 2005 Open Questions E.g. How do you find using this technology in your lectures? Little control Hard to analyse Hard to keep focussed on interview objectives May unsettle interviewee Time consuming and can be difficult to analyse However: May gain more depth, rich data May tell you things you had not considered May give you ideas for future projects

6. Methodology and Explanation 2005 Closed Questions E.g. How many courses do you teach? How many times have you used this technology? May settle interviewee Easy to analyse Easy to maintain focus Observation and use of data may be more efficient However: Data may not be as rich

7. Methodology and Explanation 2005 Question Pitfalls Leading questions –How bad is the current system? Double questions/poor structure –How often does the system fail, and how do you fix it when it does? Stay focussed. If you are unlikely to be able to make use of the answer dont ask the question!

8. Methodology and Explanation 2005 Question Structure Pyramid –Start with closed questions, finish with open –Good for settling nervous interviewees as answers are easy early on Funnel –Start with open questions, finish with closed –Good for aggressive interviewees as opinions are asked for early on Diamond –Closed -> Open -> Closed, Good all-round technique Structured/unstructured –All interviews may vary depending on how focused you intend to be on the prepared structure

9. Methodology and Explanation 2005 Starting the Interview Dress Greeting – first time –Who you are –Why you want to interview them –What objectives you want to achieve from the interview –That the information they give is confidential and how it might be used Subsequent greetings should discuss an issue from or summarise the previous interview Dont just be yourself

10. Methodology and Explanation 2005 During the Interview Introduce each line of questioning Probing –E.g. Why? –Shows that you are listening –Good for keeping interview focus –Double-check information and notes Taking notes –Not too many –Maintain eye contact –Go back over important points Use of sound recorders

11. Methodology and Explanation 2005 Focus Groups Normally 3-10 people A representative sample of typical users Enables people to put forward ideas in a supportive environment Facilitator guides and prompts Discussion recorded Low-cost but need good facilitator

12. Methodology and Explanation 2005 Questionnaires For wider coverage Need to know the right questions (through interviews and background) Focus on objectives Dont ask unnecessary questions Keep it short and facilitate response Open and Closed questions Scaling

13. Methodology and Explanation 2005 Likert Scale Short statements about the features to be evaluated Divide the items into groups with same number of positive and negative statements in each group Decide on scale – even or odd, number of points strongly agree neutraldisagreestrongly disagree

14. Methodology and Explanation 2005 Questionnaire difficulties The halo effect –Respondent answers everything the same by not differentiating between questions –Mix questions up on different subjects Leniency –Some respondents rate everything too well –Move the average category left (or right) of centre and make scale descriptors stronger Central tendency –Respondents rate everything as average –Increase the number of points on the scale and adjust the scale descriptors to cover a smaller range of opinions

15. Methodology and Explanation 2005 Also…evaluation during active use System refinement as a result of experience or in response to changes in users –interviews and focus group discussions –continuous user-performance data logging look at both frequent and infrequent error messages analyse sequences of actions to suggest improvements or new actions BUT respect peoples rights and consult them first! –User feedback mechanisms on-line forms, email and bulletin boards workshops and conferences