Causal Structure, Endogeneity, and the Missing Data Problem in Modeling the Impact of Information and Communication Technology Use on Society Tuesday,

Slides:



Advertisements
Similar presentations
Inferential Statistics and t - tests
Advertisements

REGRESSION, IV, MATCHING Treatment effect Boualem RABTA Center for World Food Studies (SOW-VU) Vrije Universiteit - Amsterdam.
Brief introduction on Logistic Regression
Reliable Causal Inference via Genetic Matching: A new matching method jointly developed by Alexis Diamond and Jasjeet Sekhon Alexis Diamond Software by.
The choice between fixed and random effects models: some considerations for educational research Claire Crawford with Paul Clarke, Fiona Steele & Anna.
Longitudinal and Multilevel Methods for Models with Discrete Outcomes with Parametric and Non-Parametric Corrections for Unobserved Heterogeneity David.
Independent t -test Features: One Independent Variable Two Groups, or Levels of the Independent Variable Independent Samples (Between-Groups): the two.
FIN822 Li11 Binary independent and dependent variables.
Introduction to Statistics: Political Science (Class 7) Part I: Interactions Wrap-up Part II: Why Experiment in Political Science?
The World Bank Human Development Network Spanish Impact Evaluation Fund.
Workshop: Statistical Consulting & Research Center November 1, 2012.
1Prof. Dr. Rainer Stachuletz Limited Dependent Variables P(y = 1|x) = G(  0 + x  ) y* =  0 + x  + u, y = max(0,y*)
By Wendiann Sethi Spring  The second stages of using SPSS is data analysis. We will review descriptive statistics and then move onto other methods.
Doing Experiments An introduction 1. Empirical social science, including economics, is largely nonexperimental, using data from situations occurring in.
1 Def: Let and be random variables of the discrete type with the joint p.m.f. on the space S. (1) is called the mean of (2) is called the variance of (3)
QUALITATIVE AND LIMITED DEPENDENT VARIABLE MODELS.
ANCOVA Psy 420 Andrew Ainsworth. What is ANCOVA?
Clustered or Multilevel Data
Impact Evaluation: The case of Bogotá’s concession schools Felipe Barrera-Osorio World Bank 1 October 2010.
FIN357 Li1 Binary Dependent Variables Chapter 12 P(y = 1|x) = G(  0 + x  )
Topic 3: Regression.
Analyzing quantitative data – section III Week 10 Lecture 1.
© Institute for Fiscal Studies The role of evaluation in social research: current perspectives and new developments Lorraine Dearden, Institute of Education.
Multiple Linear Regression
Today Concepts underlying inferential statistics
Psych 524 Andrew Ainsworth Data Screening 2. Transformation allows for the correction of non-normality caused by skewness, kurtosis, or other problems.
Richard M. Jacobs, OSA, Ph.D.
Survey Experiments. Defined Uses a survey question as its measurement device Manipulates the content, order, format, or other characteristics of the survey.
ANCOVA Lecture 9 Andrew Ainsworth. What is ANCOVA?
1 Selection bias and auditing policies on insurance claims Leuven, July 20, 2006 Jean Pinquet, Montserrat Guillén & Mercedes Ayuso.
Correlational Designs
Instrumental Variables: Problems Methods of Economic Investigation Lecture 16.
Correlational Research Chapter Fifteen Bring Schraw et al.
Estimating Causal Effects from Large Data Sets Using Propensity Scores Hal V. Barron, MD TICR 5/06.
The Health Consequences of Incarceration Michael Massoglia Penn State University.
Maximum Likelihood Estimation Methods of Economic Investigation Lecture 17.
Regression. Types of Linear Regression Model Ordinary Least Square Model (OLS) –Minimize the residuals about the regression linear –Most commonly used.
The Choice Between Fixed and Random Effects Models: Some Considerations For Educational Research Clarke, Crawford, Steele and Vignoles and funding from.
AFRICA IMPACT EVALUATION INITIATIVE, AFTRL Africa Program for Education Impact Evaluation David Evans Impact Evaluation Cluster, AFTRL Slides by Paul J.
C M Clarke-Hill1 Analysing Quantitative Data Forming the Hypothesis Inferential Methods - an overview Research Methods.
Does Trade Cause Growth? JEFFREY A. FRANKEL AND DAVID ROMER*
Discrete Choice Modeling William Greene Stern School of Business New York University.
ITEC6310 Research Methods in Information Technology Instructor: Prof. Z. Yang Course Website: c6310.htm Office:
Adjusted from slides attributed to Andrew Ainsworth
SW 983 Missing Data Treatment Most of the slides presented here are from the Modern Missing Data Methods, 2011, 5 day course presented by the KUCRMDA,
Epistemology and Methods Data Selection, Operationalization, and Measurement May
7.4 DV’s and Groups Often it is desirous to know if two different groups follow the same or different regression functions -One way to test this is to.
Chapter 6 Introduction to Multiple Regression. 2 Outline 1. Omitted variable bias 2. Causality and regression analysis 3. Multiple regression and OLS.
Chapter Eight: Quantitative Methods
Randomized Assignment Difference-in-Differences
REBECCA M. RYAN, PH.D. GEORGETOWN UNIVERSITY ANNA D. JOHNSON, M.P.A. TEACHERS COLLEGE, COLUMBIA UNIVERSITY ANNUAL MEETING OF THE CHILD CARE POLICY RESEARCH.
1 Empirical methods: endogeneity, instrumental variables and panel data Advanced Corporate Finance Semester
Assumptions of Multiple Regression 1. Form of Relationship: –linear vs nonlinear –Main effects vs interaction effects 2. All relevant variables present.
ANCOVA.
Nonparametric Statistics
[Part 5] 1/43 Discrete Choice Modeling Ordered Choice Models Discrete Choice Modeling William Greene Stern School of Business New York University 0Introduction.
Alexander Spermann University of Freiburg, SS 2008 Matching and DiD 1 Overview of non- experimental approaches: Matching and Difference in Difference Estimators.
Chapter 17 Basic Multivariate Techniques Winston Jackson and Norine Verberg Methods: Doing Social Research, 4e.
Experimental and Quasi-Experimental Research
12 Inferential Analysis.
Impact evaluation: The quantitative methods with applications
Introduction to Microeconometrics
12 Inferential Analysis.
Evaluating Impacts: An Overview of Quantitative Methods
The European Statistical Training Programme (ESTP)
Chapter: 9: Propensity scores
Alternative Scenarios and Related Techniques
Regression Part II.
Presentation transcript:

Causal Structure, Endogeneity, and the Missing Data Problem in Modeling the Impact of Information and Communication Technology Use on Society Tuesday, December 01, 2015 Hun Myoung Park University Information Technology Services Indiana University

HICSS-41, January 7-10, Outline  ICT Use and Society  Competing Perspectives  Review of Traditional Approaches  Nature of Problems  Alternative Approaches  Data and Illustrations  Findings  Implications

HICSS-41, ICT Use and Society  Does ICT use influence society?  Positive, negative, or negligible effect?  Technological determinism Optimistic perspective Pessimistic perspective  Skeptical perspective

HICSS-41, Optimistic Perspective ICT UseSociety  Positive impact on society  Transformation Theory  Rheingold (1993); Grossman (1995); Morris (1999)  “Getting the general public engaged”

HICSS-41, Pessimistic Perspective ICT UseSociety  Negative impact on society  Reinforcement theory  David (1999, 2005); Norris (2001)  Digital inequality (digital divide)  “Engaging the engaged” rather than the disenfranchised

HICSS-41, Skeptical Perspective ICT UseSociety  I CT use shaped by society  R eflection of the real world  N ormalization theory  M argolis and Resnick (2000); Bimber (2001, 2003); Uslaner (2004)  ” Politics as usual ”

HICSS-41, Conflicting Evidence, How?  Conflicting empirical results depending on perspectives  What is wrong?  Failure to deal with the nature of problems properly  How do we assess the impact of ICT use (treatment effect) more correctly?

HICSS-41, Review: T-test (ANOVA)  Comparing means/proportions  Scott (2006)  Impact of ICT use: mean difference  Simplicity and easy interpretation  Two groups are assumed to have same characteristics except for the treatment

HICSS-41, Review: Linear Regression  Least squares dummy variable model (LSDV)  Jennings and Zeitner (2003); Uslaner (2004); Welch and Pandey (2007)  Impact: dummy coefficient δ  What if the dummy d are related to disturbance ε?

HICSS-41, Review: Binary Response Model  Binary logit and probit model for binary dependent variables  Bimber (2001, 2003) and Thomas and Streib (2003)  Impact: a discrete change of d, difference in predicted probabilities  Large N required

HICSS-41, Nature of Problems  Measurement issues: categorical and binary DVs  Limited DVs (self-selected)  Ambiguous causal structure  Endogeneity: d and ε are related  The “missing data problem” in nonexperimental research

HICSS-41, Causal Structure ICT UseSociety ICT UseSociety  Unidirectional versus bidirectional  Interactive and jointly determined?  Iterative and virtuous circle: Norris (2000)

HICSS-41, Endogeneity  ICT use may not be exogenous  Disturbance ε is related to the ICT use d  violation of key OLS assumption  Jointly determined in a system  Instrumental variable (IV) approach?

HICSS-41, Missing Data Problem  A subject is either ICT user (participant) or nonuser, not both.  NOT necessarily means many missing values in data  Users and nonusers may have different characteristics, which are not controlled in research (survey): self-selection bias

HICSS-41, Nonexperimental Design OBS pre  Treatment  OBS post_treatment OBS pre OBS post_control Treatment (?) OBS users OBS nonusers  Randomized control group pre-post test design  Non-randomized post test only design  Is ICT use a real treatment?

HICSS-41, Propensity Score Matching 1  Rosenbaum and Rubin (1983, 1984)  Binary Probit model to compute predicted probabilities  Match users and nonusers who have similar likelihood (propensity score)  Pair matching/subclassification; one-to- one pair matching w/o replacement  Controlling many covariates using one dimensional propensity score

HICSS-41, Propensity Score Matching 2  Rosenbaum and Rubin (1984); Dehejia and Wahba (1999)  Matching  (paired) T-test

HICSS-41, Treatment Effect Model  Subjects decide whether or not to receive treatment: selection bias  Selection equation estimates predicted probabilities of ICT use  Impact is the dummy coefficient adjusted by correlation of ICT use and the dependent variable  When ρ=0, the impact is δ

HICSS-41, Recursive Bivariate Probit Model  Maddala (1983), Greene (1998)  Two equations with an endogenous IV variable, ICT use  Correlation between disturbances  If ρ≠0, both direct/indirect effects are considered in RBPM  If ρ=0, binary response model (BRM) examines direct impact only

HICSS-41, Specification (RBPM)

HICSS-41, Secondary Data  The PEW Internet and American Life Project 2004 Post-Election Internet Tracking Survey (Crosssectional) N=2,146  The American National Election Studies Longitudinal data of 1996, 1998, 2000, 2004 N=6,014

HICSS-41, Illustration 1: E-government Use  IV (d): whether citizens look for information from government websites  DV: whether citizens sent about voting (deliberative civic engagement)  DV: Attendance at a rally during the election campaign (action- oriented)

HICSS-41, Illustration 1: E-government Use  Average effect: 9.8% vs. 2.2%  Discrete change: 15.3% vs. 3.3% Method Rally T-test 17.1% (1,243) 6.6% (1,320) PSM (Pair) 9.8% (509) 2.2% (558) BRM (Probit) 14.1% (1,030) 3.3% (1,090) RBPM 15.3% (931) 3.3% (974)

HICSS-41, Illustration 1: E-government Use

HICSS-41, Illustration 2: Internet Use  IV (d): whether citizens have used the Internet for political information  DV: discussing politics (deliberative civic engagement)  DV: whether citizens gave money to a candidate (action-oriented engagement)

HICSS-41, Illustration 2: Internet Use  Average effect: 10.1% vs. 4.4%  Discrete change: 8.3% vs. 5.2% MethodDiscussGive Money T-test 21.0% (5,419) 6.3% (5,425) PSM (Pair) 10.1% (1,091) 4.4% (1,090) BRM (Probit) 9.9% (4,956) 5.4% (4,959) RBPM 8.3% (4,956) 5.2% (4,959)

HICSS-41, Illustration 2: Internet Use

HICSS-41, Finding 1: T-test vs. PSM  Robust estimation of PSM at the expense of loss of N  T-test overestimates the impact on deliberative civic engagement due to missing data problem  No big difference in action- oriented engagement

HICSS-41, Finding 2: BRM vs. RBPM  BRM overestimates the impact on deliberative civic engagement: endogeneity matters  Both direct and indirect effects  No big difference in action- oriented engagement; the impact of ICT use is direct

HICSS-41, Finding 3: Deliberative Engagement  Both direct and indirect effects considered  Overall impact depends on signs and magnitude of effects  They may have opposite signs that cancel out each other  BRM may report misleading results

HICSS-41, Implication and Conclusion  Types of civic engagement to be differentiated; variety of civic engagement (Verba et al. 1995)  Characteristics of dependent variables carefully examined  Causal structure, endogeneity, missing data problem, and sample size considered  Specific use of ICT applications differentiated as well

HICSS-41, Questions?  Question or suggestion?