1. Research Process Define Problem, Research Objectives
HOW?
Overall Method
Survey
Experiment
Case Study
Secondary Data
What?
Concepts
Variables
Measures
Who?
Population
Sampling
Data Gathering
Analysis
Application

2. STEPS IN A SURVEY 1. Define problem and study objectives
2. Identify information needs & study population(s)
3. Determine basic design/approach
- cross sectional vs longitudinal
- on-site vs household vs other
- self-admin. vs personal interview vs phone
- structured or unstructured questions
4. Questionnaire design
5. Choose sample (frame, size, sampling design)
6. Estimate time, costs, manpower needs, etc.

3. Survey Implementation
7. Proposal & “Human subjects” review
8. Line up necessary resources
9. Pre-test instruments and field procedures
10. Data gathering and follow-up procedures
11. Coding, cleaning and data processing
12. Analysis: preliminary, then final.
13. Communication and presentation of results.

4. Definition & Measurement
“measurement is the beginning of science, … until you can measure something, your knowledge is meager and unsatisfactory” Lord Kelvin
Nominal/Conceptual Definition - define concept in terms of other concepts, links concepts without tying them to real world
Operational definition - equates definition with measurement, specify procedures/operations to generate the concept.

5. Levels of Measurement

6. Examples of Measurement scales- Income
NOMINAL
Middle income if income is between $30K and $50K
Not middle if less than $30K or more than $50K.
ORDINAL
LOW is less than 30K
MID if between 30K and 50K
HIGH if greater than 50K
INTERVAL
Income in dollars from Line 17 of 1997 tax return.
RATIO
Income in dollars + $10,000.

7. Questionnaire Design
1. Preliminary Info
Information needed
Who are subjects
Method of communication
2. Question Content
3. Question Wording
4. Response Format
5. Question Sequencing/Layout

8. What Info?
Demographic, Socioeconomic, Physical
Cognitive - Knowledge & beliefs
Affective - attitudes, feelings, preferences
Behavioral - actions

9. Question Content 1. Is this question necessary? useful?
2. Are several questions needed on this subject? Avoid double barreled questions.
3. Do respondents have information to answer the question? Use filter questions to screen.
4. Does question need to be more concrete, specific and related to subject's personal experience? Is a time referent provided?
5. Is question sufficiently general? Do you want recent behavior or "typical behavior"?
6. Do replies express general attitudes or specific ones?
7. Is content loaded or biased
8. Are subjects willing to answer?
9. Can responses be compared with existing information?

10. Wording
1. Will words be uniformly understood? Simple language. Avoid technical phrases, jargon and abbreviations.
2. Does question adequately express the alternatives?
3. Is the question misleading due to unstated assumption or unseen implications.
4. Is wording biased, emotional, or slanted?
5. Will wording be objectionable to respondents?
6. Should you use more or less personalized wording?
7. Ask in a more direct or more indirect way?

11. Form of Response 1. Open or closed-ended 2. If closed,
ordered or unordered;
number of categories,
type of cue,
forced or unforced choice
3. Response categories
mutually exclusive
exhaustive.

12. Sequencing & layout
1. Will this question influence responses to others?
2. Is question led up to in a natural way?
3. Placement to create interest, improve response rate.
4. Branching, skipping, and transitions on questionnaires.

13. 1. Simple fill in the blank
1. Simple fill in the blank. Obtaining a straightforward number or other easily understood response.
How old are you? ___________
(years).
In what county is your permanent residence?
_______________ ( county)
How much money did you spend on this trip?
$ ________________

14. 2. Open ended: To avoid leading subject, to obtain wide range of responses in subject’s own words, or when you don’t know kinds of responses to expect.
What is your primary reason for visiting the park today? _______________________________________.

15. 3. Partially closed ended
3. Partially closed ended. List major response categories while leaving room for others.
Which of the following community recreation facilities do you most frequently use? (check one).
 neighborhood parks/playgrounds
 swimming pools
 community centers
 natural areas
 tennis courts
 other (please specify) ___________________

16. 4. Checklists: Allow subjects to check multiple responses
4. Checklists: Allow subjects to check multiple responses. Categories exhaustive & mutually exclusive
Which of the following winter recreation activities have you participated in during the past month? (check all that apply)
 Cross-country skiing
 Downhill skiing
 Snowmobiling
 Ice Skating
 Sledding or Tobogganing

17. 5. Likert Scales: Versatile format for measuring attitudes.
Please check the box that best represents your level of agreement or disagreement with each of the following statements about downhill skiing:
Strongly Strongly
Agree Agree Neutral Disagree Disagree
Downhill skiing is...
exciting     
dangerous     
expensive     
Can replace “agree” with “importance” “satisfaction”, “interest” “preference” and other descriptors to fit the attitude you wish to measure.

18. 6. Rank Ordering: To measure preferences or priorities
6. Rank Ordering: To measure preferences or priorities. Limit to short lists.
Rank the following states in terms of your interest as possible travel destinations for a summer vacation trip. (Place a 1 beside the state you would most like to visit, place a 2 besides your second choice, and a 3 beside your third choice.)
______ Michigan
______ Wisconsin
______ Minnesota

19. 7. Filter Question. To screen for eligibility or knowledge prior to asking other questions.
Did you stay overnight on your most recent trip?
 NO  YES
If Yes, How many nights did you spend away from home? ________
To next question

20. 8. Semantic Differential scale
8. Semantic Differential scale. Measure perception or image of something using a set of polar adjectives.
For each of the characteristics listed below, mark an X on the line where you feel downhill skiing falls with respect to that characteristic.
exciting ___ ___ ___ ___ ___ ___ dull
expensive ___ ___ ___ ___ ___ ___ inexpensive
safe ___ ___ ___ ___ ___ ___ dangerous
(Could repeat with cross country ski and snowmobiling and compare perceptions; or Coke and Pepsi.

21. AGE : Operational Definition B
Give the subject a slip of paper with the following question.
ENTER YOUR AGE IN YEARS AT YOUR LAST BIRTHDAY ______.
The number written in the blank is the person's age.

22. EXAMPLE 1.
CONCEPTUAL DEFINITION - Length of table is the distance from one end to the other.
OPERATIONAL DEFINITION - Place a yardstick along one edge of the table, place additional yardsticks end to end until one extends over the other edge. Read the numeric marking for inches on the final yardstick at the point where it is exactly over the other edge of the table (call this X inches). Count the number of yardsticks you have used (call this N). Compute (N-1) * 36 + X. Repeat this process on the edge perpendicular to this one. The length of the table in inches is the larger of these two numbers.

23. EXAMPLE 2.
CONCEPTUAL DEFINITION - Age- the number of years that have passed since a person's date of birth.
OPERATIONAL DEFINITION - A. Ask an individual for his name, place of birth (county and state), and names of both parents. Go to the county records office for the given county and find the record for the person with the given name. If there are more than one such records, check for names of parents. Identify the date of birth from this record. Subtract the year of birth from If the month and day of birth has not yet been reached in the current year, subtract 1.

24. Levels of Measurement

25. Reliability and Validity of Measures
RELIABILITY = Absence of random error (noise), refers to repeatability of the measure. Do we get same answer each time we make measurement?
Inter-rater, test-retest, split half, parallel forms, Internal consistency- Cronbach’s alpha, KR
VALIDITY = Absence of systematic error (bias). Are we measuring what we purport to be measuring?
Content validity, face validity
Criterion-related validity: predictive, concurrent, convergent, discriminant
Construct validity: MTMM

26. Validity vs Reliability

27. Statistics & measurement level
A dichotomous variable (0 or 1) can be treated as interval

28. Misuse of Statistics Visitor Characteristics across Wilderness Areas
Variable
San Gorgonio
John Muir F p
Gender
1.2
1.1
.43
.51
Age
3.9
3.8
.52
.47
Educ
4.2
6.2
.01
Ethnicity
.00
.96
Income
3.49
3.5
.98
.32

29. Notes Gender: 1=male, 2=female
Age ranges 1=0-12, 2=13-18, 3=19-30, =31-50, 5=51-65, 6=66+
Educ: =0-8th, =HS, =2 yr coll, =4 year college, =postgrad
Ethnicity: 1=Anglo, 2=Latino, 3=Black, 4= Asian
Income: 1= 0-$13,00, =14-25, 3=26-50, 4=51-75, =$76+