1. Data gathering

2. Four key issues Setting goals
Decide how to analyze data once collected
Relationship with participants
Clear and professional
Informed consent when appropriate
Triangulation
Use more than one approach
Pilot studies
Small trial of main study
The main reason for gathering data is to obtain information about something. It could be how do you like this application to whether the redesign of the website achieved its aims. Whatever the reason may be, it must be very clearly identified as these influence the nature of the data gathering session and the data gathering techniques that will be used and how we analyze the data. The goal can be expressed formally using some mathematical notation or using a simple description. It must be clear and concise.
A great way to establish a good relationship is to use an informed consent form. The consent form ensures the rights of both the data gatherer(they can use the data in their analysis, publish it ) and the data provider(the data provided will not be detrimental to them). How do you work with young children? This kind of consent form is not required when we are gathering requirements. Why?
Incentives will help in gathering data.
Triangulation is a strategy than demands the use of more than one data gathering technique to tackle a goal or using more than one data analysis approach. Questionnaires can be sent to a larger user group, interview a specific user group and observe some sample people from that target user group. It provides different perspectives and some findings may be corroborated, leading to a better conclusion.
The aim of a pilot study is to ensure that the study is viable. Potential problems can be identified in advance and corrected. It avoids wasting time, money and precious resources. Anyone involved in a pilot study cannot be used in the main one. Why? This helps avoid distorting the final study.

3. Data recording Notes, audio, video, photographs Notes plus photographs
Audio plus photographs
Video
The data recording technique that you choose, depends on the context, the time available and the sensitivity of the situation. It is not good to conduct interview and take notes at the same time. Avoids personal bias(you capture only what is deemed important by the note taker). Audio is less intrusive than video, but transcribing it can be a pain. Video has the advantage of capturing both visual and audio data. The problem with video is you focus on what can be captured using the lens, that you might miss out on things happening beyond the camera view. Video is the most complete form of data gathering and is highly reliable.

4. Interviews
Unstructured - are not directed by a script. Rich but not replicable.
Structured - are tightly scripted, often like a questionnaire. Replicable but may lack richness.
Semi-structured - guided by a script but interesting issues can be explored in more depth. Can provide a good balance between richness and replicability.
Interviews can be thought of as a conversation with a purpose. The above three types of interviews are different based on the amount of control that the interviews wields.
Unstructured: They are real useful when you are exploring a new concept. It is open ended, but it is better to have a plan for the type of questions that you want covered. A lot of unnecessary data may be gathered, which may be very difficult to analyze. Mostly open questions
Structured interviews are very useful when goals are clearly understood and specific questions can be identified. Questions in a structured interview must be worded in the exact manner for all participants. Mostly closed questions
Interviews, we must make sure that we do not lead the users into a specific answer with the way we phrase our questions, body language, etc. Make sure you give enough time for people to answer. Avoid gender stereotypes and long questions, jargons etc.
Choose your questions carefully to avoid bias. This is very valid, because people might sometimes give answers that show them in the best light rather than what they actually think.
Follow the usability study process, communicate the goals, reassure the participants, have a exploratory period, the main session, and cool off towards the end.

5. Focus Groups
3-10 People
People develop opinions within a social context
It is commonly used in political campaigns(Spinning Boris)marketing and social research. The discussion is led by a trained facilitator and the 3-10 people invited are usually representative of the target population. It allows diverse and sensitive opinions to be revealed and people often develop opinions when they hear other people talk about a specific issue. A preset agenda with sufficient flexibility to follow unanticipated issues is needed.

6. Questionnaires Questions can be closed or open
Closed questions are easier to analyze, and may be done by computer
Paper, and the web used for dissemination
Sampling can be a problem when the size of a population is unknown as is common online
Questionnaires are really useful at getting answers to specific questions from a large group of people who are in various areas. They are used on their own or can be used in conjunction with other data gathering techniques to deepen your understanding of a particular situation.
How do you decide between a structured interview and a questionnaire? It all comes down to the motivation of the respondent. If the respondent is motivated to answer their questions, then you may want to use a questionnaire, but if the level of motivation is not high enough, then you might want to use a structured interview.

7. Questionnaire design Question Order. Clear Instructions Length
Decide on whether phrases will all be positive, all negative or mixed.
Do you need different versions of the questionnaire for different populations?
Encourage a good response
Many questionnaires typically begin by asking for basic demographic information, sort of like a pre-test questionnaire. This information is really valuable when it comes down to analyzing the results. The background information provides valuable context to the questionnaire. If two answers conflict, it could be explained by their perspectives on the situation. Make sure you collect information that is useful to the study.
The impact of the questions can be influenced by the question order.
Strike a balance between using white space and keeping the questionnaire compact.
Make sure purpose of study is clear
Promise anonymity
Ensure questionnaire is well designed
Offer a short version for those who do not have time to complete a long questionnaire
If mailed, include a stamped addressed envelope
Follow-up with s, phone calls, letters
Provide an incentive
40% response rate is high, 20% is often acceptable

8. Question and response format
‘Yes’ and ‘No’
Checkboxes that offer many options
Rating scales
Likert scales
semantic scales
3, 5, 7 or more points?
Open-ended responses
Semantic differential scales are used less frequently than Likert scales. They explore bipolar attitudes to a particular item and provide a pair of words that provide different meanings. For example clear …………………………………confusing
How many scale points do we use, 3 or 5 or 7. Long scales help provide a nice discrimination, while it might be difficult for people to find the difference between some scale points. An odd number provides a clear central point, while an even number forces a decision and prevents them from sitting on a fence.

9. Advantages of online questionnaires
Responses are usually received quickly
No copying and postage costs
Data can be collected in database for analysis
Time required for data analysis is reduced
Errors can be corrected easily
versus web-based. An one can be targeted at very specific people, but it is just an electronic version of a paper based survey and it is not very interactive and does not provide data validation and analsyis.

10. Problems with online questionnaires
Preventing individuals from responding more than once
Individuals have also been known to change questions in questionnaires
The sampling technique with online questionnaires is not very scientific. The term to use is convenience sampling. It just involves people who were available rather than ones who fit the representative population.
Some studies have shown that people are more revealing and honest when answer online questionnaires than when answering paper based questionnaires. For instance students tend to rate their profs less favorable in web based evaluation forms than on paper based ones. Why?

11. Observation Direct observation in the field Structuring frameworks
Degree of participation (insider or outsider)
Ethnography
Direct observation in controlled environments
Indirect observation: tracking users’ activities
Diaries
Interaction logging
Insider vs Outsider
Observation is a useful data gathering tool that can be used at any stage in the product development. It can be used early on to observer people using the current system. It gives us a good idea of the context of tool usage, the goals and the problems or it can be used at a later stage to watch people interact with a prototype. The investigator may directly observe the user or might get access to records the user makes while using the tool.
Field observation can be better than observing in a usability lab, because it is possible to see the real feelings(frustration or delight) with the product. Also the user has more control in a natural setting. But because an insider is present, people may not be willing to share their true annoyances.
Observation is one technique, where it is real important to have a clear goal. There are a lot of things happening and we need to be observe the right thing, but at the same time we should be flexible.
An observer might choose to just watch without participating in the activity, or can choose to be a full time participant. In the latter case, you have to make sure you are objective with your results.-

12. Structuring frameworks to guide observation
- The person. Who? - The place. Where? - The thing. What?
The Goetz and LeCompte (1984) framework: - Who is present? - What is their role? - What is happening? - When does the activity occur? - Where is it happening? - Why is it happening? - How is the activity organized?
Who is using it? What are they doing with it and where are they using it?

13. Direct observation in a controlled setting
Think-aloud technique
Indirect observation
Diaries
Interaction logs
Ethical concerns with logging.
When do you stop observation?

14. Choosing and combining techniques
Depends on
The focus of the study
The participants involved
The nature of the technique
The resources available
The techniques must be compatible with the focus of the study, for example the type of data you want to gather, different data gathering techniques can give you different kinds of information, do you want opinions , observable behavior or facts etc.
The participants involved, children versus adults, location of the target population, time needed to complete the study, if you need a lot of time, then you might want to observe, the level of motivation can decide between questionnaires and interviews.