Conducting Surveys at Cornell University Marin Clarkberg, Associate Director Institutional Research and Planning Yasamin Miller, Director Survey Research Institute

Who We Are Institutional Research and Planning (IRP) Within the Division of Budget and Planning; goal to inform institutional decision-making Administers regular, large-scale surveys to students and other University constituencies Survey Research Institute (SRI) Full service survey enterprise at Cornell Designed, hosted, and analyzed hundreds of surveys for non-profit, government, corporate, and Cornell clients

Rationales for surveys Why do a survey? Increasing call to have “real data” and to assess processes and outcomes Looks easy (and inexpensive… and maybe even fun) Why not do a survey? 1.Lots of data is already available 2.Survey fatigue 3.Not as easy at it might seem at first 4.Serious limitations on what survey data can actually tell you about processes and outcomes

Why you shouldn’t: 1. Data already exists Everyone has some data… Has existing data been thoroughly analyzed and understood? In addition, the University archives an enormous amount data about students (and other constituencies) Students’ academic records IRP Surveys, other surveys

Why you shouldn’t: 2. Survey fatigue Surveys are more common IRP surveying regularly since 2000 CIT-hosted surveys, e.g. WebSurveyor: oOver 1600 surveys in Survey response rates are down Senior Survey response rates: 61% in 1998; 50% in 2002; 45% in 2006 Student (staff, faculty) time is a university resource

Why you shouldn’t: 3. There is such a thing as survey expertise Question design Bad questions give you bad data Instrument design Respondents often bail out of unsatisfactory surveys Sample design Samples are often adequate to the task Sampling saves all kinds of resources

Why you shouldn’t: 4. Survey research has major limitations Ability to generalize with survey data a function of response patterns Respondents may differ in important ways from nonrespondents Surveys cannot demonstrate causation

What are the alternatives to surveys? Ransack existing sources of data Multiple sources of data help “triangulate” If nothing else, you can learn more about what you don’t know (and thus what remains to be learned) Look at alternate modes of collecting data Some questions better addressed with observation, focus groups, interviews, etc.

So when is a survey appropriate? Existing data is well-understood Unanswered questions clearly identified Scope of survey will minimize the imposition on respondents As short as possible, asked to as few as possible Appropriate survey expertise in involved in designing the study Allow ample time for coordination, consultation, design and pre-testing Shared understanding of study limitations Start small and manage expectations

How to Survey 1.Develop a reasonable timeline 2.Define your research questions 3.Design the survey instrument and sampling plan 4.Develop a data security plan 5.Notify and secure approvals 6.Data analysis and reporting

1. Develop a reasonable timeline Putting questions on the web, collecting responses, and even data analyses are the easy parts Having established a clarity of task, four to six months is not an unreasonable amount to develop and pre-test a survey instrument

4. Develop a data security plan Anonymous Data Identities of respondents are never captured Fewer data security concerns Impossible to know who responded (or how often) Impossible to link survey data with other data sources Confidential Data Identities of respondents are kept, but secured Necessary to have very secure file storage Possible to link survey data to other data of interest

5. Notify and Secure Approvals IRP Survey Calendar Notification courtesy Institutional Review Board If a project is “research” (i.e. develops or contributes to generalizable knowledge), it needs to be reviewed and approved by the IRB.IRB Data Stewards Students: the Office of the RegistrarRegistrar Student and Academic Services SAS-Research Group

Resources on campus Find the help you need Institutional Research and Planning (IRP) oServes the University oStewards of much existing student survey data oAvailable to consult with institutional studies (sampling plans, instrument design and review) Survey Research Institute (SRI) oComprehensive survey services from initial planning to data analysis and reporting oIRP uses SRI for survey hosting and administration Institutional Review Board for Human Participants oFirst and last authority on mandated review requirements and processes – ask them.