Marissa Gargano Sarrynna Sou Susan Edwards Chris Cummiskey Complex Data Analysis Improve your data analytic abilities using Stata and Early Grade Reading and Mathematics Assessment Data Sunday March 5, 8:30 – 14:45 Georgia 2 (South Tower) CIES 2017 Downtown Sheraton Atlanta, GA Marissa Gargano Sarrynna Sou Susan Edwards Chris Cummiskey Marissa Gargano Sarrynna Sou Susan Edwards Chris Cummiskey

Before we data dive Introductions around the room: Name, Affiliation, Position, Favorite thing to do in ATL. What Stata version do you have (12, 13, 14)? Stata-Use/Comfort Level. Statistical Background Experience with data from EGRA/EGMA. What you hope to get out of this class.

Assumption for Participation Currently have your computer with you. Currently have Stata on your computer. Computer has access to the internet. Currently have Microsoft Work and Excel on you computer. Have conducted analysis in Stata. Have a general grasp of statistics. Are not terrified with the word: “regression”.

Workshop Brief: Review the Agenda Two Sections: Tanzania Cross-sectional Survey [Morning] Kenya Impact Evaluation Survey [Afternoon]

Data Access Agreements Make sure everyone has signed the data access agreement… Make sure everyone has the Workshop Folder copied over to their computer.

Install “Groups” into your Stata. Connect to the internet… Open Stata, type the following into your command bar ssc install groups

Lets get Started on Simple Survey Statistics. For more than 56 years, statistics research has been one of our primary specialties. Our statisticians, epidemiologists, and bio-statisticians conduct complex statistical analyses to support wide ranging research programs in both laboratory and social sciences.

Census vs. Random Sample Portion of population Entire population Calculate estimated values Calculate true values Will always have some uncertainty No uncertainty No need to be statistically significant! Need for statistical significances To expensive Not practical Cost effective

Background: Simple Random Sample vs. School-Based Complex Random Complex (Cluster) Sample Directly sample students Sample school  student More than 1 stage sampled Only 1 stage sampled Not practical Practical Too expensive (not cost effective) Cost effective

Sample Analysis: Two Types Descriptive describe the sample (not the population from which the same came) Inferential Project the sample to the population Population Cluster Effect Sample Sample weights

Inferential Statistics: It is all about 2 simple things Point Estimates: (i.e. means, percentages) Use Sample Weights To make the sample representative of the population Accounts for over any over/under representation in the sample Minimize bias Precision Estimates: (i.e. standard error) Use sufficient sample size and small cluster size Try to make the point estimates as precise as possible Minimize the uncertainty of the point estimate. Inferential statistics

Inferential Statistics: Put Point and Precision together Once you have the proper Point Estimate and Precision Estimate, you can create various statistical figures: 95% Confidence Intervals Representative of the population and sufficiently tight. Formal statistical tests Test for difference among groups Test for correlations

“Inferential Analysis”: Failure to account for sample methodology If students were sampled with SRS Analyzed accounting for the proper sample methodology Not weighted Does not account for cluster effect Weighted Accounts for cluster effect * p<0.05 Incorrectly estimate the population’s mean orf. Public looks better than private. Overly confident in the precision of the means orf means [incorrectly conclude statistically significant differences]. Source: RTI International, Grade 2 Early Grade Reading Assessment (EGRA) and Snapshot of School Management Effectiveness in Indonesia (EdData II), March-April 2014.

Stata Coding: svy: mean orf, over(private) If students were sampled with SRS Analyzed accounting for the proper sample methodology svy: mean orf, over(private) mean orf, over(private) * p<0.05 Incorrectly estimate the population’s mean orf. Public looks better than private. Overly confident in the precision of the means orf means [incorrectly conclude statistically significant differences]. Source: RTI International, Grade 2 Early Grade Reading Assessment (EGRA) and Snapshot of School Management Effectiveness in Indonesia (EdData II), March-April 2014.

What is “svy”? “svy” for “survey design” stata command used for complex sample analysis. equivalent to SPSS’s “csaplan” (complex sample analysis plan) IF set up correctly: Accounts for the over/under representation in the sample Using the sample weights Accounts for the cluster effect Using the Taylor linearized variance estimation Very easy to use once the svyset as been specified and saved to the dataset.

What is “svyset”? “svyset” for “survey design set up” stata command used at data processing stage. tells Stata how the complex sample was drawn (so you can analyze using the “svy” command). very easy to mess up if: The data processer has little experience with complex sample The data processer has not been involved with the sample methodology and drawing the sample The data processer has not been involved in the questionnaire design, pilot data collection.

OK, TOO MUCH THEORY… LETS GET TANGIBLE in TZ Open a do-file. Save it to the Tanzania “Do Files” Folder: <path>\Analysis Workshop\Materials\Tanzania_2013\Do Files\ 1_Analyze_Tanzania-Data_CIES2017_Analysis-Workshop.do

In the Do-File Read in the Tanzania Census Data [School Level] use “…<path>\CIES-2017 Complex Analysis Workshop\Materials\Tanzania_2013\Data\ Census_List-Frame For Tanzania2013-National Survey_PLSE7.dta”, clear Read in the Tanzania Sampled Data [Student Level] use “...<path>\CIES-2017 Complex Analysis Workshop\Materials\Tanzania_2013\Data\ PUF_3.Tanzania2014-National_grade2_EGRA-EGMA-SSME_English-Kiswahili.dta”, clear

10 Minute Break?