Topic – 05 Part I Collecting primary data & credibility of research finding A- (i) Questionnaires (ii) Interviews (iii) Observations B- Credibility of research finding Part II Sampling techniques (Probability versus Non-probability sampling)

Part I Research interview Definition ‘An interview is a purposeful discussion between two or more people’ Kahn and Cannell (1957) Types of interview/Three major types Structured interview Semi-structured interview Unstructured/In-depth interview

Structured & semi-structured interview Data collection technique in which an interviewer physically meets the respondent, reads them the same set of questions in a predetermined order, and records his or her response, usually with pre-coded answers. Semi-structured interview: Wide-ranging category of interview in which the interviewer commences with a set of interview themes but is prepared to vary the order in which questions are asked and to ask new questions in the context of the research situation. The responses/information/ data are recorded by note-taking, or perhaps by tape-recording the conversation.

Unstructured/in-depth interview Loosely structured and informally conducted interview that may commence with one or more themes to explore with participants but without a predetermined list of questions to work through. These are used to explore in depth a general area in which one is interested; these are therefore also known as in-depth interview. The interviewee is given the opportunity to talk freely about events, behavior and beliefs in relation to the topic area. (See Box 9.3 for further details)

Approaches to questioning (in semi-structured and unstructured interviews) Open questions The use of open questions allows participants to define and describe a situation or event as they wish. Example: How has corporate strategy changed over the past five years? Probing questions Questions used to further explore responses that are of significance to the research topic. Example: What external factors caused the corporate strategy to change? Specific and closed questions These types of question may be used to obtain specific information or to confirm a fact or opinion. Example: How many people responded to the customer survey? (Specific question) Did i hear you say that the new warehouse opened on 25 march? (close question requiring ‘yes’ or ‘No’ answer)

Observation as a data collection method ‘Observation involves the systematic observation, recording, description, analysis and interpretation of people’s behaviour’ Observation: The two major types Participant observation: Observation in which the researcher attempts to participate fully in the lives and activities of the research subjects and thus becomes a member of the subjects’ group Structured observation: Structured observation is concerned with the frequency of events. It is characterized by a high level of predetermined structure and quantitative analysis.

Structured observation Data collection and analysis Choosing an ‘off the shelf’ coding schedule Box 8.1 provides a checklist of questions to ask when choosing an ‘Off the Shell’ coding schedule. Designing your own coding schedule Guidelines for developing your own coding schedule is available in Table 8.1 Combining both types of schedule A specimen for recording sheet for observing behaviour in groups is provided in Figure 8.2

Credibility of Research Findings Important considerations Reliability? Validity? Generalizability?

Can we measure reliability? Yes, using computer Reliability can be assessed by posing three questions: Will the measure yield the same results on other occasions? Will similar observations be reached by other observers? Is the measure/instrument stable and consistent across time, space and researcher in yielding findings? Can we measure reliability? Yes, using computer 4-Threats to reliability (i) Subject/participant error (ii) Subject/participant bias (iii) Observer error and (iv) Observer bias

Validity Whether the findings are really about what they appear to be about. Validity depends upon: *History (same history or not), *Testing (if respondents know they are being tested), * Mortality (participants’ dropping out), * Maturation (tiring up), and * Ambiguity (about causal direction).

Generalizability Logic leaps and false assumptions The extent to which research results are generalizable. Logic leaps and false assumptions Research design is based on a flow of logic and number of assumptions, which must stand to closest scrutiny

(A: Semi-structured & unstructured interviews) Data Quality Issues (A: Semi-structured & unstructured interviews) Lack of standardization in the use of semi-structured and unstructured/in-depth interview cases may lead to raise concerns, related to: Reliability: Whether alternative researchers would reveal similar information/results; concerns about reliability relate to the issues of bias. Forms of bias: Interviewer bias: comments, tone, or non-verbal behaviour of interviewer may create bias in the way that interviewees respond to questions being asked; or researcher may demonstrate bias in interpreting responses; or interviewer remains unable to develop trust of interviewees, and they provide limited information. Interviewee or response bias: interviewees may feel information is sensitive; he/she is not empowered to deliver information related to organization; interview is too-time consuming.

(Semi-structured & unstructured interviews) Data Quality Issues (Semi-structured & unstructured interviews) Generalizability: It may not be possible to make generalizations about the entire population where results are based on a small and unrepresentative number of cases. Validity: High level of validity is possible to the extent to which the researcher gains access to their participants’ knowledge and experience and because of the interaction between researcher and interviewee which allows meanings to be probed, topics to be covered from a variety of angles and questions made clear to the respondents.

Overcoming Data Quality Issues (Semi-structured & unstructured interviews) The following points help avoid sources of bias Your own preparation and readiness for the interview The level of information supplied to the interviewee The appropriateness of your appearance at the interview The nature of opening comments made when interview commences Your approach to questioning The impact of your behaviour during the course of interview Your ability to demonstrate attentive listening skills Your scope to test understanding Your approach to recording information

(B) Structured observation Threats to validity and reliability Subject error Time error Observer effects and strategies to overcome this: – habituation and – minimal interaction

Part - II Sampling techniques Probability sampling? Versus Non-probability sampling?

Non-probability sampling and Non-probability sampling Probability samples: ones in which members of the population have a known chance (probability) of being selected Non-probability samples: instances in which the chances (probability) of selecting members from the population are unknown Probability sampling requires a sampling frame, and when a sampling frame is not possible, non-probability sampling is used Sampling frame: Sampling frame is a complete list of all the cases in the population, from which sample will be drawn (Recall we have already discussed sampling frame error). Where no suitable list exists, researcher will have to compile his/her own sampling frame. It is important to ensure that sampling frame is unbiased, current and accurate

Two Major Sampling techniques Probability (representative) and Non-probability (Judgemental) Source: Saunders et al. (2009) Figure 7.2 Sampling techniques

SIMPLE RANDOM SAMPLING Simple random sampling: the probability of being selected is “known and equal” for all members of the population Blind Draw Method (e.g. putting names “placed in a hat” and then drawn randomly) Random Numbers Method (all items in the sampling frame given numbers, numbers then drawn using table or computer program) Advantages: Known and equal chance of selection Easy method when there is an electronic database Disadvantages: (Overcome with electronic database) Complete accounting of population needed Cumbersome to provide unique designations to every population member

Systematic Sampling Systematic sampling: way to select a probability-based sample from a directory or list. This method is at times more efficient than simple random sampling; here sampling interval is used Sampling interval (SI) = population list size (N) divided by a predetermined sample size (n) How to draw: 1) Calculate SI, 2) Select a number between 1 and SI randomly, 3) Go to this number as the starting point and the item on the list here is the first in the sample, 4) Add SI to the position number of this item and the new position will be the second sampled item, 5) continue this process until desired sample size is reached.

Cluster Sampling Method by which the population is divided into groups (clusters), any of which can be considered a representative sample. These clusters are mini-populations and therefore are heterogeneous. Once clusters are established, a random draw is done to select one (or more) cluster(s) to represent the population. Area (next slide) and systematic sampling (discussed earlier) are two common methods.

Cluster Sampling – Area Method Divide the geo area into sectors (sub-areas) and give them names/numbers, determine how many sectors are to be sampled (typically a judgment call), randomly select these sub-areas. Do either a census or a systematic draw within each area. To determine the total geo area estimate, add the counts in the sub-areas together and multiply this number by the ratio of the total number of sub-areas divided by number of sub-areas.

Stratified Sampling Method The population is separated into homogeneous groups/segments/strata and a sample is taken from each. The results are then combined to get the picture of the total population. This method is used when the population distribution of items is skewed. It allows us to draw a more representative sample. Hence, if there are more of certain type of items in the population, the sample will have more of this type; and if there are fewer of another type, there will be fewer of that type, in the sample.

Non- probability sampling techniques (Judgemental sampling) Quota sampling Purposive sampling Snowball sampling Self-selection sampling Convenience sampling

selection compared to others Judgment samples Samples that require a judgment or an “educated guess” on the part of the researcher as to who should represent the population. Also, “judges” (informed individuals) may be asked to suggest who should be in the sample. Subjectivity enters in here, and certain members of the population will have a smaller, little or no chance of selection compared to others

Referral and Quota Sampling Methods Referral samples (snowball samples): samples which require respondents to provide the names of additional respondents Members of the population who are less known, disliked, or whose opinions conflict with the respondent have a low probability of being selected. Quota samples: samples that set a specific number of certain types of individuals to be interviewed Often used to ensure that convenience samples will have desired proportion of different respondent classes

Convenience Sampling Method Convenience samples: samples drawn at the convenience of the interviewer. People tend to make the selection at familiar locations and to choose respondents who are like themselves. Error occurs: 1) in the form of members of the population who are infrequent or nonusers of that location, and 2) who are not typical in the population 3) who are disliked