1. Md. Waheed Alam Transparency International Bangladesh 26 August 2016 1 Quantitative Research Techniques and Tools

2. Why quantitative techniques? Exploration of a phenomenon or a research question with measureable numbers, graphs and charts Testing of a theory and hypothesis Can generalize its findings to wider population 2

3. Theory of Measurement  Measurement Validity Definition of validity - The best available approximation of truth of a given proposition, inference and conclusion through use of a certain measurement constructs like questionnaire, checklist, sampling, hypothesis tests and model building  Translation Validity – accurate translation through operationalization Face validity - whether the operationalization of a concept is a good construct; for example if you what to measure corruption whether the construct of the corruption is acceptable to experts or knowledgeable persons Content validity – check the operationanalizaiton against the relevant content domain for the construct; if you want to measure corruption whether the construct contain fundamental features of corruption 3

4. Cont……  Criterion related validity- performance of operationalization against some criterions Predictive validity – operationalization’s ability to predict casual relationship (math ability and performance in engineering profession) Concurrent validity – operationalization’s ability to distinguish between groups (manic-depressive and paranoid schizophrenic) Conversant validity – degree to which operationalization is similar to other operationlization (talents and performance ) Discriminate validity - degree to which operationalization is not similar to other operationlization (correlation between level of corruption and development) 4

5. Cont………  Threats to validity Confounding - entanglement of a measurement with other attributes History - influenced by earlier experience Testing – biased response due to being tested Maturation – subject (respondent) for measurement is mature enough Selection bias – preferential section Regression toward mean – response to any average value considering all influencing variables  Reliability - degree of measurement error Systemic error – factors that systematically effects measurement (i.e faulty instrument) Random error- error due to chance 5

6. Survey research and sampling External validity- approximate truth of conclusion that involve generalization Population vs sample  Probability sampling techniques Simple random sampling – random selection from the whole population Systematic sampling - random selection through systematic manner Stratified sampling - random selection from different strata Cluster sampling – random selection of clusters (relatively homogeneous within cluster) Multi-stage sampling – random selection at different stages 6

7. Cont…  Non probability sampling techniques (judgmental) Expert sampling (maximum variation sampling, homogeneous/heterogenoeous sampling, typical case sampling, extreme case sampling, critical case sampling) Snowball sampling Quota sampling 7

8. Sample size determination  Sample size determination formula for estimation of proportion for a complex survey design 8 Here, n= Sample Size p= proportion of success of an indicator 1-p= proportion of failure of the indictor z= Sample variate at certain confidence interval (usually at 95% confidence interval) e= Margin of error Design effect= loss of efficiency for adopting a complex survey design

9. 9 ************* Estimation of gender of students **************** svyset [pweight=w], psu(sch_code) fpc (domain) strata (partner) pweight: w Strata 1: partner SU 1: sch_code FPC 1: dom Number of strata = 4 Number of obs = 1600 Number of PSUs = 160 Population size = 38562.1 Design df = 156 -------------------------------------------------------------------------- | Linearized | Proportion Std. Err. [95% Conf. Interval] -------------+------------------------------------------------------------- gen_stu | Boys|.5125975.0130668.4867867.5384082 Girls |.4874025.0130668.4615918.5132133 ---------------------------------------------------------------------------- Proportion Estimation

10. 10 **************** Average age of students*********************. svy: mean stu_age (running mean on estimation sample) Survey: Mean estimation Number of strata = 4 Number of obs = 1600 Number of PSUs = 160 Population size = 38562.1 Design df = 156 ------------------------------------------------------------------------------------ | Linearized | Mean Std. Err. [95% Conf. Interval] -------------+-------------------------------------------------------------------- Av. Student age | 11.14644.0422097 11.06306 11.22981 -------------------------------------------------------------------------------------- Mean Estimation

11. Cautionary Notes for Probability Sampling Techniques and Estimation Frame should be a representation one A complete frame is required for random selection National surveys are usually a mixture of all probability techniques Certain number of samples have to be chosen to achieve reliability. It depends on variability of the indicator or issue under investigation, error level and reliability level Reliability estimate of survey result is Margin of Error= 1.96x SE 11

12. Hypothesis testing and model building  Mostly used for randomized research design, evaluation research and model building  Different hypothesis testing and model building tools z test for equality of proportions from two independent samples t test for equality of means of two independent samples Chi square test for exploring relationships of nominal level variables or attributes ANOVA for testing equality means from more than two independent sample Regression analysis to measure effects of explanatory variables on response variable 12

13. 13 prtest listen_b, by(gen_stu) Two-sample test of proportions 1: Number of obs = 796 2: Number of obs = 804 ------------------------------------------------------------------------------ Variable | Mean Std. Err. z P>|z| [95% Conf. Interval] -------------+---------------------------------------------------------------- Boys |.9811558.0048195.9717097.9906018 Girls|.9900498.0035004.9831891.9969104 -------------+---------------------------------------------------------------- diff | -.008894.0059565 -.0205686.0027806 | under Ho:.0059516 -1.49 0.135 ------------------------------------------------------------------------------ diff = prop(1) - prop(2) z = -1.4944 Ho: diff = 0 Ha: diff 0 Pr(Z z) = 0.9325 Independent Proportion Test

14. 14 Chi-square Test. tab reading_b partner, chi2 | partner reading_b | CODEC RDRS-1 RDRS-2 VERC | Total -----------+--------------------------------------------+---------- Achieved | 341 365 353 395 | 1,454 | 23.45 25.10 24.28 27.17 | 100.00 | 85.25 91.25 88.25 98.75 | 90.88 -----------+--------------------------------------------+---------- Not achieved | 59 35 47 5 | 146 | 40.41 23.97 32.19 3.42 | 100.00 | 14.75 8.75 11.75 1.25 | 9.13 -----------+--------------------------------------------+---------- Total | 400 400 400 400 | 1,600 | 25.00 25.00 25.00 25.00 | 100.00 | 100.00 100.00 100.00 100.00 | 100.00 Pearson chi2(3) = 48.5689 Pr = 0.000

15. 15 Independent Mean Test ttest all_competency, by(gen_stu) Two-sample t test with equal variances ------------------------------------------------------------------------------ Group | Obs Mean Std. Err. Std. Dev. [95% Conf. Interval] ---------+------------------------------------------------------------------------------------------- Boys | 796 23.88819.1022722 2.885455 23.68744 24.08895 Gilrs | 804 24.20647.0975983 2.767388 24.01489 24.39805 ---------+------------------------------------------------------------------------------------------- combined | 1600 24.04812.0707585 2.830338 23.90934 24.18691 ---------+----------------------------------------------------------------------------------------- diff | -.3182767.1413389 -.5955058 -.0410476 ----------------------------------------------------------------------------------------------------- diff = mean(1) - mean(2) t = -2.2519 Ho: diff = 0 degrees of freedom = 1598 Ha: diff 0 Pr(T |t|) = 0.0245 Pr(T > t) = 0.9878

16. 16 ANOVA oneway all_competency partner, tabulate | Summary of all_competency partner | Mean Std. Dev. Freq. ------------+------------------------------------------- CODEC | 22.7825 2.9345737 400 RDRS-1 | 24.9 1.8971024 400 RDRS-2 | 22.59 3.2861676 400 VERC | 25.92 1.056594 400 ------------+-------------------------------------------------- Total | 24.048125 2.8303385 1600 Analysis of Variance Source SS df MS F Prob > F ----------------------------------------------------------------------------------------------- Between groups 3183.01687 3 1061.00562 175.91 0.0000 Within groups 9626.2775 1596 6.03150219 ------------------------------------------------------------------------------------------------ Total 12809.2944 1599 8.01081574

17. OLS Regression Estimate svy: regress all_competency stu_age i.gen_stu total_membr i.father_edu i.mother_edu i. partner_nume log_income (running regress on estimation sample) Survey: Linear regression Number of strata = 4 Number of obs = 1540 Number of PSUs = 160 Population size = 37411.72 Design df = 156 F( 34, 123) = 4.38 Prob > F = 0.0000 R-squared = 0.2493 ---------------------------------------------------------------------------------------------- | Linearized all_compet~y | Coef. Std. Err. t P>|t| [95% Conf. Interval] -------------+-------------------------------------------------------------------------- stu_age |.1528979.1211077 1.26 0.209 -.0863245.392120 2.gen_stu (Girls) |.4126436.176139 2.34 0.020.0647185.7605687 total_membr |.0294805.0327414 0.90 0.369 -.0351932.0941543 | father_edu | 2 | -.1279239.19124 -0.67 0.505 -.505678.2498301 3 | -.5354086.6588213 -0.81 0.418 -1.83677.765953 4 | -.0252157.408959 -0.06 0.951 -.8330273.7825959 5 | -.6240197.6201092 -1.01 0.316 -1.848914.6008743 | mother_edu | 2 |.6362581.3493689 1.82 0.070 -.0538459 1.326362 3 | 1.190869.4335243 2.75 0.007.3345336 2.047204 4 |.7474457.6069215 1.23 0.220 -.4513986 1.94629 14 | 1.882447.6540573 2.88 0.005.5904958 3.174398 | partner_nume | RDCRS-1 | 1.995578.3818351 5.23 0.000 1.241344 2.749812 R DRS-2| -.2113174.5381251 -0.39 0.695 -1.274269.8516344 VERC | 2.999773.3763169 7.97 0.000 2.256438 3.743107 | log_income |.2460301.1778006 1.38 0.168 -.1051772.5972374 _cons | 18.10372 2.14046 8.46 0.000 13.8757 22.33175 ------------------------------------------------------------------------------ 17

18. Limitations Can answer how much or how many but can not answer why Mostly structured Highly technical and lack of compliance to technical rigor might produce erroneous results Need technological support like computer and software Involves much time and money 18

19. Reference Research Methods, William M. K. Trochim, Cornell University This book is available at (http://www.socialresearchmethods.net/kb/) 19

20. Q & A 20