1. GS/PPAL 6200 3.00 Section N Research Methods and Information Systems January 13, 2015 Professor Brenda Spotton Visano Office: 130 McLaughlin Voice Mail: (416) 736-2100 ext. 20470 E-mail: spotton@yorku.caspotton@yorku.ca

2. Agenda Introduction to Data Sources – Guest Speaker: Mr. Walter Giesbrecht, Librarian, Scott Library, York University Review of Last Class Introduction to Research Methods (cont’d) – Criteria for Evaluating Research – Research Orientation (Methods) – Research Design – Checklist for Reviewing Research

3. Review of First Class Objectives of First Class: – Brief Introduction to Course – Introduction to Each Other’s Policy Interests – Motivating the Study of Policy Research Methods Achieving the Objectives – Information contained on Course Outline – Group discussion of sample survey questions to illustrate and motivate issues in ensuring data integrity and ethical use – Short videos to illustrate how common understanding and supporting evidence depends on currency of data and perspective – Deductive and Inductive Reasoning; Sample Fallacies to be avoided

4. Results of First Class Survey Total number of Respondents, N=17 Topics of Interest for Policy Research: – Justice: Social justice: 6 Criminal Justice/Access: 2 – Judiciary/Legislation: 2 – Other: Child Welfare, Public Dental Health, Privacy Issues – Undecided: 4 Research Methods of Interest for Policy Research: – Qualitative: 5 – Quantitative: 3 – Both Qualitative and Quantitative (including meta-analysis): 5 – Undecided: 4 Some Familiarity with Basic Statistics: – Yes: 11 (includes hesitant answers such as “I think so”) – No: 6 Some Familiarity with Basic Excel: – Yes: 14 – No: 3 Introduction to Excel : https://support.office.com/en-us/article/Excel-2013-training-courses-videos-and-tutorials-aaae974d-3f47-41d9-895e- 97a71c2e8a4a?ui=en-US&rs=en-US&ad=US

5. Parting Question #1 from Last Class What do we know? How do we know what we know? How do we discover what we don’t know? The positivist view: knowledge is only phenomena confirmed by the senses, subjected to and supported by empirical testing, and “value-free” – an external, objective reality The post-positivist view: knowledge includes the subjective meanings – interpreting versus explaining or, interpreting for explaining, and social reality is constructed, mutable, and created by our actions

6. Parting Question #2 from Last Class Does interpretation play a role in research? Is the researcher independent of the research subject or does the background (education, beliefs, values) of the researcher influence what is observed? Values can influence: choice of research areas, formulation of research questions, choice of methods, formulation of research design and data collection, analyses, interpretation and conclusions.

7. Research in Public Policy Excerpted from Morçöl and Ivanova (2010) Categories of MethodsQuantitative OrientationQualitative Orientation Empirical Inquiry - Design Methods Experimental, Cross- sectional, Longitudinal Case study Empirical Inquiry - Data Collection Methods Surveys, Secondary DataQualitative (long, in-depth, or semi-structured) Interviews Empirical Inquiry - Data Analysis Methods Statistical, Regression, or Time-series Analyses (Computer-assisted) Qualitative Data Analyses Empirical Inquiry - Combined Methods Game Theory, Simulations, Systems Analysis, Meta- Analyses, Network Analyses Case study, Legal Analyses, Archival, Ethnography, Grounded Theory Methods of Decision Making and Planning Cost-benefit, Decision Analyses, Linear Programming Brainstorming, Delphi

8. Criteria for Evaluating Research Reliability/Replicability: Would the same results be achieved by the same measurement technique applied to the same research subject by the same or different researchers? Is the study able to be replicated with the same results? Validity: are the research conclusions “valid”?

9. Validity - Basics Are we testing what we claim to be measuring? E.g., Do IQ tests measure intelligence? (construct validity) Is there a causal relationship between the dependent and independent variables tested? (internal validity) Can we generalize the cause-effect beyond the research study to other people/places/times? (external validity)

10. Research Design Experimental design: random assignment to control and experimental groups, manipulate isolated independent variable, observe outcomes Quasi-experimental design: omit random assignment, observe outcomes from “natural” experiments, e.g. Cross-sectional design e.g., unstructured interviews: observations taken at a point in time, Longitudinal design: track observations over time Case study design: detailed analysis of a single case (person, program, etc)

11. Research Design, Orientation, Validity DesignQuantitative Orientation Qualitative Orientation Internal Validity External Validity ExperimentalClinical trialsNot applicable HighHigh if random assignment and low if social research involves deception Cross- sectional design Survey, structured observation, content analysis, regression analysis Interviews, Focus groups Problematic since independent variable cannot be manipulated High if sample random Longitudinal design Time series analysis EthnographiesProblematic, ibid High if initial sample random Case studySurveyInterviewsLow

12. Checklist for Reviewing Research Case study: “Analyzing E-Government Research…” Heeks and Bailur (2006) Subject: E-government research For discussion in class: – Research Question: – Philosophy: – Research Design: – Data Collection Method: – Data Analysis Method: – Sample Size and Representativeness: – Conclusions: – Reliability/Replicability: – Validity: