1. GS/PPAL 6200 3.00 Section N Research Methods and Information Systems A QUANTITATIVE RESEARCH PROJECT - (1)DATA COLLECTION (2)DATA DESCRIPTION (3)DATA ANALYSIS

2. A Quantitative Research Project: Generic Overview Research Topic: What is the main issue? Research Questions: Descriptive? Relational? Causal? (How will we measure key variables?) Quantitative Research Design: Experimental? Cross-sectional? Longitudinal? Data Collection Method: Survey? Secondary Data? Census or Sample? (Data Collection Instruments) Data Analysis: Descriptive Statistics? Analysis of Variation? Regression Analysis?

3. A Quantitative Research Project: An Example Research Topic: Academic Performance Research Questions: How well do graduating students perform academically? What explains that performance? Measure “academic performance” by graduating CGPA Research Design: Cross-sectional analysis of graduating students in a given year ?Data Collection: Survey (a random sample of) students graduating in 2014 ?Data Description: Describe the data with basic statistics ?Data Analysis: Reasons for attending university and performance; Total hours studied and CGPA

4. DATA COLLECTION PROCESS 1.Census or Sample? 2.How to obtain the data? 3.How do we know when we know what we know? These questions must be considered together. The answer to Question 1 informs the answers to Questions 2 and 3. The answer to Question 2 informs the answer to Questions 1 and 3… you get the idea.

5. Census or Sample? What is the total theoretical population in which you are interested? What is the accessible population? Is it feasible or practicable to conduct a census on the accessible population? A census would tell us the actual information for all graduating students in a given year, which is a sample of all students graduating for all time A sample of all the students graduating in a given year is then a sample of a sample

6. How will we know if or when we know something? Understanding the information is complicated by the uncertainty inherent in the data we collect Construct Validity Issues: Is CGPA a good indicator of academic performance? Is the mean (average) CGPA we observe for the census population equal to the “true” mean? Is the mean CGPA we observe for a sample equal to the census population mean?

7. The Challenge We can never know if we are observing the “true” population mean (average) of CGPA since any observed population mean will deviate plus or minus σ (= a “standard deviation”) Any census of a graduating class will only be a sample of the “true” population of all graduating classes A sample of the census population – as a sample of the sample – introduces more uncertainty

8. Uncertainty complicates “knowing” Uncertainty Source #1: If we are seeking to explain the key factors determining the CGPA of graduates, we have to account for the fact that the observed CGPA might deviate from the true mean by σ Uncertainty Source #2: If it is infeasible to conduct a census of all graduating students in even one year, and all we can do is sample the sample, then we have additional uncertainty related to the size of the sample

9. Uncertainty from Sampling We know there is one inescapable source of uncertainty (Uncertainty Source #1) The sampling error (Uncertainty Source #2) complicates this uncertainty … but in a predictable way. We know the larger (smaller) the sample size, the smaller (greater) the uncertainty from any sampling error IF we use a simple random sampling method

10. Some Vocabulary for this Uncertainty We can never know if we are observing the true value of CGPA or some value plus (minus) deviations due to (1) some unexplainable shock (Uncertainty Source #1) – so we talk about a “Confidence Interval” We know that sampling error (Uncertainty Source #2) is possible – so we talk about our analysis of the sample in terms of its “Margin of Error”

11. What we know in the face of this Uncertainty For a 95% Confidence Interval we know our census population mean would be close to the true population mean 95% of the times and …we can have the confidence that the census population mean plus or minus a random error will contain the true population mean 95% of the time

12. What we know (cont’d) When sampling error is possible, and we have only sample statistics to estimate census population values, we must adjust our understanding of the 95% CI for this additional uncertainty If - we have a Margin of Error of 10% for a 95% Confidence Interval … Then - 90% of the estimated sample Confidence Intervals in repeated random sampling of the census population will contain the true population mean (average) value 95% of the time

13. How do we Sample? Whom do we Sample? If the relevant population is all York University students graduating in 2014 with undergraduate degrees, then the census population is approximately 10,000 students To collect information from all 10,000 students is not practicable; therefore consider sampling Sampling Technique: Probability Sampling - Simple Random Sampling Sampling Frame: How to select participants? – Once Ethics Approval obtained… – University database contains student contact information and CGPAs of all graduating students; a random number generator can perform the randomization for selection; …

14. Sample Size For inferential statistics, “small” is n < 30 Sample Size: How many students should we sample? Decision is guided by two competing goals: – maximize the probability that we obtain correct information on the relevant variables and – minimize the cost of our study

15. Sample Size Guide For a 95% confidence interval (Margin of Error of 5%), a sample of 400 is needed …95% CI, Margin of Error of 10%, n = 100 …95% CI, Margin of Error of 3%, n = 1000 …95% CI, Margin of Error of 1%, n = 10,000

16. DATA COLLECTION PROCESS - RECAP WHAT: We will measure “academic performance” by the student’s CGPA on graduation WHO: It is not practicable to survey all graduating students, so we will choose a sample of students HOW: We want to conduct a statistical analysis so we will collect data from a simple random sample of students for a sampling frame defined by the University’s student records WHY: To test a hypothesis about the factor(s) that influence CGPA but… – We know that the CGPA mean we observe will be imprecise as a measure of the true mean due to random deviation and due to sampling error – More accurate information is costly – So…we choose our sample size guided by this tradeoff

17. DATA COLLECTION INSTRUMENT: Survey Questionnaire & Results To obtain the data from your simple random sample of (100, 400, or 1000) graduating students in 2014 we need a survey instrument and code book To develop the survey questions, we might first – Conduct a small non-probabilistic sample in a focus group to get a better sense of the key factors influencing academic performance – Conduct a small pilot study to test our survey instrument and to practice the analysis See the proposed survey instrument and coded results of small n=41 sample