Research Process Define Problem, Research Objectives HOW? Overall Method Survey Experiment Case Study Secondary Data What? Concepts Variables Measures Who? Population Sampling Data Gathering Analysis Application

STEPS IN A SURVEY 1. Define problem and study objectives 2. Identify information needs & study population(s) 3. Determine basic design/approach - cross sectional vs longitudinal - on-site vs household vs other - self-admin. vs personal interview vs phone - structured or unstructured questions 4. Questionnaire design 5. Choose sample (frame, size, sampling design) 6. Estimate time, costs, manpower needs, etc.

Survey Implementation 7. Proposal & “Human subjects” review 8. Line up necessary resources 9. Pre-test instruments and field procedures 10. Data gathering and follow-up procedures 11. Coding, cleaning and data processing 12. Analysis: preliminary, then final. 13. Communication and presentation of results.

Definition & Measurement “measurement is the beginning of science, … until you can measure something, your knowledge is meager and unsatisfactory” Lord Kelvin Nominal/Conceptual Definition - define concept in terms of other concepts, links concepts without tying them to real world Operational definition - equates definition with measurement, specify procedures/operations to generate the concept.

Levels of Measurement

Examples of Measurement scales- Income NOMINAL Middle income if income is between $30K and $50K Not middle if less than $30K or more than $50K. ORDINAL LOW is less than 30K MID if between 30K and 50K HIGH if greater than 50K INTERVAL Income in dollars from Line 17 of 1997 tax return. RATIO Income in dollars + $10,000.

Questionnaire Design 1. Preliminary Info Information needed Who are subjects Method of communication 2. Question Content 3. Question Wording 4. Response Format 5. Question Sequencing/Layout

What Info? Demographic, Socioeconomic, Physical Cognitive - Knowledge & beliefs Affective - attitudes, feelings, preferences Behavioral - actions

Question Content 1. Is this question necessary? useful? 2. Are several questions needed on this subject? Avoid double barreled questions. 3. Do respondents have information to answer the question? Use filter questions to screen. 4. Does question need to be more concrete, specific and related to subject's personal experience? Is a time referent provided? 5. Is question sufficiently general? Do you want recent behavior or "typical behavior"? 6. Do replies express general attitudes or specific ones? 7. Is content loaded or biased 8. Are subjects willing to answer? 9. Can responses be compared with existing information?

Wording 1. Will words be uniformly understood? Simple language. Avoid technical phrases, jargon and abbreviations. 2. Does question adequately express the alternatives? 3. Is the question misleading due to unstated assumption or unseen implications. 4. Is wording biased, emotional, or slanted? 5. Will wording be objectionable to respondents? 6. Should you use more or less personalized wording? 7. Ask in a more direct or more indirect way?

Form of Response 1. Open or closed-ended 2. If closed, ordered or unordered; number of categories, type of cue, forced or unforced choice 3. Response categories mutually exclusive exhaustive.

Sequencing & layout 1. Will this question influence responses to others? 2. Is question led up to in a natural way? 3. Placement to create interest, improve response rate. 4. Branching, skipping, and transitions on questionnaires.

1. Simple fill in the blank 1. Simple fill in the blank. Obtaining a straightforward number or other easily understood response. How old are you? ___________ (years). In what county is your permanent residence? _______________ ( county) How much money did you spend on this trip? $ ________________

2. Open ended: To avoid leading subject, to obtain wide range of responses in subject’s own words, or when you don’t know kinds of responses to expect. What is your primary reason for visiting the park today? _______________________________________.

3. Partially closed ended 3. Partially closed ended. List major response categories while leaving room for others. Which of the following community recreation facilities do you most frequently use? (check one).  neighborhood parks/playgrounds  swimming pools  community centers  natural areas  tennis courts  other (please specify) ___________________

4. Checklists: Allow subjects to check multiple responses 4. Checklists: Allow subjects to check multiple responses. Categories exhaustive & mutually exclusive Which of the following winter recreation activities have you participated in during the past month? (check all that apply)  Cross-country skiing  Downhill skiing  Snowmobiling  Ice Skating  Sledding or Tobogganing

5. Likert Scales: Versatile format for measuring attitudes. Please check the box that best represents your level of agreement or disagreement with each of the following statements about downhill skiing: Strongly Strongly Agree Agree Neutral Disagree Disagree Downhill skiing is... exciting      dangerous      expensive      Can replace “agree” with “importance” “satisfaction”, “interest” “preference” and other descriptors to fit the attitude you wish to measure.

6. Rank Ordering: To measure preferences or priorities 6. Rank Ordering: To measure preferences or priorities. Limit to short lists. Rank the following states in terms of your interest as possible travel destinations for a summer vacation trip. (Place a 1 beside the state you would most like to visit, place a 2 besides your second choice, and a 3 beside your third choice.) ______ Michigan ______ Wisconsin ______ Minnesota

7. Filter Question. To screen for eligibility or knowledge prior to asking other questions. Did you stay overnight on your most recent trip?  NO  YES If Yes, How many nights did you spend away from home? ________ To next question

8. Semantic Differential scale 8. Semantic Differential scale. Measure perception or image of something using a set of polar adjectives. For each of the characteristics listed below, mark an X on the line where you feel downhill skiing falls with respect to that characteristic. exciting ___ ___ ___ ___ ___ ___ dull expensive ___ ___ ___ ___ ___ ___ inexpensive safe ___ ___ ___ ___ ___ ___ dangerous (Could repeat with cross country ski and snowmobiling and compare perceptions; or Coke and Pepsi.

AGE : Operational Definition B Give the subject a slip of paper with the following question. ENTER YOUR AGE IN YEARS AT YOUR LAST BIRTHDAY ______. The number written in the blank is the person's age.

EXAMPLE 1. CONCEPTUAL DEFINITION - Length of table is the distance from one end to the other. OPERATIONAL DEFINITION - Place a yardstick along one edge of the table, place additional yardsticks end to end until one extends over the other edge. Read the numeric marking for inches on the final yardstick at the point where it is exactly over the other edge of the table (call this X inches). Count the number of yardsticks you have used (call this N). Compute (N-1) * 36 + X. Repeat this process on the edge perpendicular to this one. The length of the table in inches is the larger of these two numbers.

EXAMPLE 2. CONCEPTUAL DEFINITION - Age- the number of years that have passed since a person's date of birth. OPERATIONAL DEFINITION - A. Ask an individual for his name, place of birth (county and state), and names of both parents. Go to the county records office for the given county and find the record for the person with the given name. If there are more than one such records, check for names of parents. Identify the date of birth from this record. Subtract the year of birth from 1998. If the month and day of birth has not yet been reached in the current year, subtract 1.

Levels of Measurement

Reliability and Validity of Measures RELIABILITY = Absence of random error (noise), refers to repeatability of the measure. Do we get same answer each time we make measurement? Inter-rater, test-retest, split half, parallel forms, Internal consistency- Cronbach’s alpha, KR VALIDITY = Absence of systematic error (bias). Are we measuring what we purport to be measuring? Content validity, face validity Criterion-related validity: predictive, concurrent, convergent, discriminant Construct validity: MTMM

Validity vs Reliability

Statistics & measurement level A dichotomous variable (0 or 1) can be treated as interval

Misuse of Statistics Visitor Characteristics across Wilderness Areas Variable San Gorgonio John Muir F p Gender 1.2 1.1 .43 .51 Age 3.9 3.8 .52 .47 Educ 4.2 6.2 .01 Ethnicity .00 .96 Income 3.49 3.5 .98 .32

Notes Gender: 1=male, 2=female Age ranges 1=0-12, 2=13-18, 3=19-30, 4=31-50, 5=51-65, 6=66+ Educ: 1=0-8th, 2=HS, 3=2 yr coll, 4=4 year college, 5=postgrad Ethnicity: 1=Anglo, 2=Latino, 3=Black, 4= Asian Income: 1= 0-$13,00, 2=14-25, 3=26-50, 4=51-75, 5=$76+