© 2007 Knowledge Networks, Inc. Presented at the 2007 American Association of Public Opinion Research Conference Mike Dennis, Senior Vice President Rick Li, Project Director Joe Hadfield, Senior Research Analyst Government & Academic Research Knowledge Networks We wish to thank Tom Smith and staff at NORC for sharing the question wording and results of the GSS national priority battery in the 2006 survey Comparing KnowledgePanel SM Online and Phone Experiments with In-Person General Social Survey Prepared for:

1 Background Knowledge Networks (KN) experiment in 2000:  When “Don’t Know” was presented on screen, considerably more KN respondents than GSS respondents selected “Don’t Know.” KN experiments in 2002:  Re-designed presentation of “Don’t Know” options.  When “Don’t Know” was not shown on screen AND respondents were instructed to skip a question as “Don’t Know” at the start of the survey, DK rates were similar between KN experiment and GSS in-person survey.  Substantively, KN and GSS results were similar, except for a few items.  General pattern: more respondents in KN experiment indicated “Too much,” and more respondents in GSS indicated “Too little.”  Larger differences existed on items about urban under class (Blacks, drugs, big cities, welfare) and foreign aid.

2 Background KN experiments in 2006: Continues and expands research in 2000 and 2002:  Added phone as an additional mode of data collection.  Phone and online modes both had a nationally representative sample from KnowledgePanel SM.  17 items in standard wording only. Variant wording dropped due to budgetary constraints.  KN online: Respondents were instructed at beginning of survey to skip questions to indicate “Don’t Know.”  KN phone: Respondents must consent online before getting phone survey.

3 GSS 2006 and KN 2006 Experiments GSS 2006KN 2006 OnlineKN 2006 Phone Data collection organization NORCKnowledge Networks Sample SourceArea probability sample RDD KnowledgePanel SM Mode of Data Collection In-personInternetPhone Field periodFeb–June 2006March–May 2006May–June 2006 Question ItemsEntire GSS‘National Priority’ items standard wording Sample Size4,5101, Don’t Know treatment Not read, volunteered DK Not shown, R instructed to skip Not read, volunteered DK

4 Unweighted Sample Demographics (KN Online and KN Phone) CPS KN Online (N=1,428) KN Phone (N=600) Male48%47%43% Female52%53%57% 18–2922% 15% 30–4431%29%26% 45–5926%27%34% 60+21% 26% Less than high school17%16%12% High school32% 29% Some college27% Bachelor’s degree or higher24%26%32%

5 Unweighted Sample Demographics (KN Online and KN Phone) CPS KN Online (N=1,428) KN Phone (N=600) White70% 79% Black11% 7% Other3%2% Hispanic13% 9% 2+ Races3% Northeast19% 21% Midwest23%21%26% South36%37%30% West23% Non-Metro16%17%21% Metro84%83%79%

6 Spending Priority Items (Standard Wording) The space exploration program Improving and protecting the environment Improving and protecting the nation's health Highways and bridges Social Security Mass transportation Parks and recreation Assistance for childcare Supporting scientific research Improving the nation's education system The military, armaments and defense Solving the problems of the big cities Halting the rising crime rate Dealing with drug addiction Improving the conditions of Blacks Foreign aid Welfare Are we spending too much, too little, or about the right amount on ____?  Too much  Too little  About the right amount  Don’t Know

7 Summary of “Don’t Know” Rates % of “Don’t Know” Respondents

8 “Too Little” Rates: KN 2006 vs GSS 2006 Dealing with drug addiction (In- person: 62%; Online: 48%) Assistance for childcare (In-person: 55%; Online: 45%) Solving problems of big cities (In-person: 48%; Online: 38%) Supporting scientific research (In-person: 44%; Online: 34%) Improving the conditions of Blacks (In-person: 37%; Online: 26%)

9 Smallest Differences Average Differences Average Absolute Percentage Points Difference Between KN Online 2002, KN Online 2006, KN Phone, GSS 2002, and GSS 2006 (Across All 17 Standard Wording Spending Priority Items)

10 Conclusions Results from study in 2002 were replicated:  “Don’t Know” not shown online and instructions to skip questions produce online “Don’t Know” rates similar to in-person GSS survey.  Small systematic differences: Respondents from KN’s online survey consistently less likely to select “Too little” and more likely to select “Too much” than respondents of in-person GSS survey.  Spending items on urban underclass (blacks, big cities, crimes, drugs, and welfare) and foreign aid continue to show similar large differences between the online and in-person modes. Similarities between phone and in-person modes suggest effects of modes of data collection:  Average differences between in-person and phone modes are smaller than those between in-person and online modes.  Systematic differences do not exist between in-person and phone modes.  Differences on urban underclass and foreign aid spending items decreased or disappeared.

11 Thank You! For Additional Information, Contact Mike Dennis at All KN papers presented at AAPOR will be posted to: