1 Research Design Procedures

2 Research Design Forming your action plan Deciding on the Who and When Defining all concepts and terms

3 Research Design Three purposes for research: Exploration Description Explanation Or Descriptive- existing conditions Normative- observed vs. intended Impact- can it be attributed to programs?

4 Exploration To gain familiarity with a topic Typically done to: Satisfy a researcher’s curiosity and desire to understand “Test the water” for a more extensive study Develop methodology to be used in a subsequent study

5 Description Provide context for situations and events Typically based on observation and reporting Observation is systematic Example: U.S. Census

6 Explanation Attempts to address the question of “why” Tries to get at reasons and underlying causes Example: Not “would you vote for McCain” but “why would(n’t) you vote for McCain?”

7 GAO- Designing Evaluations Considerations 1. Kind of information to be acquired 2. Sources of information (i.e. types of respondents) 3. Methods of sampling (i.e. random) 4. Methods of collecting data (i.e. interviews, surveys) 5. Timing and frequency of information collection 6. Basis for comparing outcomes 7. Analysis plan

8 Research Design: Experiments Experiments aim to control one variable or set of variables in order to determine their relationship to and/ or impact on another set of variables. Types of studies: Experiment Uses a random sample Quasi-experiment Does not use a random sample, must try to correct for error through statistical tests

9 Impact of a Program/Service Involving an Experimental Design IMPACT = Outcome of experimental group receiving the treatment compared to control group Information Literacy Instruction

10 Classic Design Two group pre- and post- test One experimental group One “untreated” control group Compare outcomes to assess impact Problems with this?

11 Solomon 4 Group Test Classic Design expanded to include two sets: One set has experimental group and control group who both receive pre- and post-tests One set has experimental group and control group who receive only post- tests. Advantages over classic model?

12 Solomon Four Group Design BeforeTreatmentAfter Group 1no yesyes Control 1no yesyes Group 2yes yesyes Control 2yes noyes

13 Time Series Design Repeats testing twice (or more) to establish a trend in the data independent of the experiment Pretest Treatment Posttest Pretest Treatment Posttest REPEAT Experimentyesyesyes yes yes yes Control Group yes no yes yes no yes

14 EXAMPLES Pretest/posttest design with control group pretesttreatment posttest Experimentyes yes yes Control Groupyes no yes Pretest/posttest design without control group pretesttreatment posttest Experiment Ayes yes yes Experiment Byes yes yes

15 Case Study Basis of selection: representative, typical, cluster, probability, etc. Multiple methods of data collection

16 Populations and Statistical Sampling

17 Populations Population- the entire group/ universe under study Sample- a portion of a population of possible information sources Sampling- methods for selecting these sources

18 Research Design: Action Plan (continued) Who is studied population Sample Is sample reflective of population? Where Sampling? When Sampling?

19 Who (or What) is being studied? Units of analysis: the what or whom being studied. In social research the most typical units of analysis are individual people. Can be: individuals, groups, organizations, social interactions, social artifacts Examples: Library Users or Non-Users First-year Students Senior Citizens Public Libraries Also ILLs Biographies, Mysteries, Audiobooks (i.e. collections) Web sites

20 Beware: Pitfalls of Analysis Ecological Fallacy: Something learned about a group says something about the individuals making up that group. Reductionism An attempt to explain phenomena in terms of limited or lower-order concepts.

21 Who is being studied How to select a sample?

22 Sampling- Three Options Census- collecting information from the entire group making up a population Like the decennial census Judgment sampling- making conscious choices Convenience Sampling- what’s available Probability/ Statistical Sampling- left to chance, each member of a population has an equal chance of being chosen

23 Sampling: Purpose Representativeness Sample has roughly the same distribution of characteristics as the population from which it is drawn. Nevertheless, each sample will differ from each other, as well as from the population Can determine the amount of error

24 Probability Sampling (1) Random sampling: Each member of the population has an equal and known probability of being selected Systematic sampling: Each member of the population is either assembled or listed, a random start is designated and then members of the population are selected at equal intervals… nth intervals

25 Probability Sampling (2) Stratified: Each member of the study population is assigned to a group or stratum, then a simple random sample is selected from each group or stratum

26 Probability Sampling (3) Cluster: Each member of the study population is assigned to a geographically-defined group or cluster. Clusters are then selected at random, and members of a selected group are represented in the sample.  Role of GIS and TIGER files 

27 Non-probability Sampling Convenience: selecting cases based on their availability “Typical” cases: selecting cases already known and not considered “extreme” Snowball: group members identify additional members to be included in sample Quota: sample is in same proportion as population

28 The Sample How selected Sample size Determine the actual individuals or “things” included

29 Sample Size A larger sample does not necessarily mean better results, but Too small a sample can lead to error

30 Sample Size- 3 Considerations Precision (sampling error)- the range in which the “true value” is estimated to be: ±5 Confidence Level (Central Limit Theorem)- when a population is repeatedly sampled, the average value is = to the true value, and values in each survey will be normally distributed: 95% confidence level Degree of Variability- distribution of attributes within a population. The more homogenous the population, the smaller the sample size.

31 Sample Size

32 Sampling Customers (Example) Present Lost Never-gained Nonuser

33 Users/Uses of Electronic Resources (More Examples) Home page users in general Users of a database

34 Questions of When How might time effect our study? How do we choose a time frame? What is an appropriate time frame based on the research problem? Address time through: Cross-sectional studies Longitudinal studies

35 Cross-Sectional Studies A study based on observations representing a single point in time. A “snapshot” Best for exploratory and descriptive studies U.S. Census Explanatory cross-sectional studies aim at drawing causal relationships over time based on observations made at one time. Issues?

36 Longitudinal Study Permits observations of the same phenomena over an extended period. Researcher may “follow” a group over time Researcher may become part of a group Researcher may study artifacts developed over time

37 Types of Longitudinal Studies Trend Studies Type of longitudinal study that examines changes within a population over time Cohort Studies Examines a specific subpopulation (cohort) as they change over time- often based on age. Panel Studies Examines the same set of people over time

38 The Learning Organization ngorg.html ngorg.html

39 Group Activity Selecting a Sample Archive/ Special Library Public Library Academic Library/ School Library Identify a research question Define your population Describe how you would select a sample Could you design an experiment around this project?