1. Information Gathering Using Quantitative Methods
Peter Lyman
IS 208 Information Management
SIMS
February 8, 2001

2. Today’s topics 1. Research design. 2. Building a survey instrument.
3. Implementing an interview protocol.

3. ASSIGNMENT Create a research design for your class project.
Define objectives
Design instrument
Create protocol
Test design on a classmate.

4. Quantitative research design
1. Defining your goal & objective.
2. Choosing a method: quantitative or qualitative?
3. Building a survey instrument.
4. Pre-testing the instrument.
Defining an interviewing protocol.
Interviewing
7. Coding the results
8. Statistical Analysis
9. Dissemination of the findings

5. 1. Defining the objective
How will new information add value?
for whom? when? how?
Market research
Formative evaluation = feedback for a project in progress
Final assessment = measuring the outcomes of a project
Longitudinal studies

6. Goals and objectives…
When you gather information, you also are sending a message to your respondents.
Your questions inform people about an issue
Responding to a questionnaire is a form of participation
Your findings change the perception of a project.

7. 2. quantitative or qualitative? Types of methods.
Questionnaires
Structured Interviews
Content analysis
of documents and records
Metric analysis of videotape
Focus Groups
Open ended Interviews
Oral histories
Participant Observation
Ethnography

8. 2. quantitative or qualitative? Costs and benefits
numeric
breadth
structured data
replicable
tends to be faster
good for defining scope and range of opinions
textual
depth
defines cultural nuance and intensity of feelings
case studies
tends to take time
good for identifying possible survey questions & language

9. Quantitative or qualitative?
Research designs often combine methods
Ethnography identifies issues and language for survey research
Focus group themes often probe survey research findings
Assessment combines structured surveys and open ended interviews

10. 3. Creating a survey instrument
Word the questions to avoid ambiguity and anticipate cultural differences in meaning.
Question sequence should be logical.
Question formats can elicit more/less complex information (T/F, Lickert scales, etc)
Make sure responses create clear numeric codes.

11. 4. Pre-testing
Does the wording make sense to different kinds of respondents?
Is it hard to administer the instrument?
Is the interviewer able to get people committed to taking the questionnaire?
Is the interviewer professional?

12. 5. The survey protocol/setting
How will the survey be implemented?
Face to face interviews
phone interviews
computer surveys
mail surveys
ISSUES: Response rate, suitability of instrument, cost.

13. 5. protocol /record keeping
What kind of record keeping is needed?
If open ended, write down responses
audio and videotape may add value, but increase respondent resistance, and add cost in coding and interpretation.

14. 5. protocol/sample Sample
statistical validity depends upon sample size
statistical validity depends upon random sample
(Accept no substitutes, follow up to catch right subject
is your sample motivated to take your survey?

15. 6. Interviewing/consent and motivation
An introduction should quickly describe who is doing the research and why, to overcome privacy concerns and engage the respondent.
Federally sponsored/University research must be certified to meet “informed consent” standards.

16. 6. Interviewing: establish a relationship
Engage the subject, but stay professional
Never deviate from script
Do not interrupt
learn to listen actively
No personal views
Don’t interpret
Don’t improvise
These are sources of error.
Have a protocol to answer questions!!

17. 6. Interviewing/notations
After the interview, note the quality of the relationship, interview, and any possible error (e.g., interruptions, sample problems, social pressure).
Take notes immediately, no longer than 24 hours.

18. 7. Coding Structured interview requires simple, often machine scoring.
Open ended questions require accurate (often double blind) coding following pre-defined categories.

19. 8. Analysis Statistical analysis is coin of the realm.
Learn to listen to the data, to see the unexpected.
Learn to contemplate the data to figure out what it is saying. Data visualization (graphics) helps you to recognize what it is saying.

20. 9. Dissemination Executive summary = what are the headlines?
Graphics often tell the story best.
In formative evaluation, it’s often necessary to show the data, to get people to listen.
In assessment people often don’t want to hear bad news.

21. Frequent sources of error
Pleasing/deceiving the interviewer
Wrong type of interview (e.g., phone call during dinner)
Flawed interviewing technique
Sample
Not listening to the data

22. Next time.
Qualitative methods. How to run and interpret a focus group. In the readings, focus on:
understanding group process issues (how are these reflected in group projects in SIMS?)
What are key focus group techniques
What you can/cannot learn from focus groups.