1. BY THE NUMBERS: QUANTITATIVE DATA ANALYSIS Sheika Kendi Georgia State University

2. Macalester College “Quantitative methods are distinguished by their emphasis on numbers, measurement, experimental design, and statistical analysis. Researchers typically work with a small number of predetermined response categories to capture various experiences and perspectives of individuals. Often emphasis is on analyzing a large number of cases using carefully constructed instruments that have been evaluated for their reliability and validity” (Patton, 1990). Techniques include questionnaires, structured interviews, and tests.” (Palomba and Banta, p. 337)

3. WHY USE QUANTITATIVE METHODOLOGY Ease of Data Collection Paper/Pencil Administration Web-based Survey Software

4. WHY USE QUANTITATIVE METHODOLOGY Ease of Data Analysis Less Time Consuming Process Easier to Train Staff Software Access (Excel Analysis ToolPak - descriptive statistics, histogram, correlation, t-test, ANOVA, regression, random number generation, etc.)

5. WHY USE QUANTITATIVE METHODOLOGY Large Number of Students Generalizablity of Findings

6. WHY USE QUANTITATIVE METHODOLOGY Combining Indirect and Direct Data Indirect data – Student perceptions, attitudes, skills, and knowledge Direct data – GPA, major, ethnicity/race, class standing, gender, program participation, etc.

7. HOW TO USE QUANTITATIVE METHODOLOGY Random Samples - representative Random Assignment - bias

8. HOW TO USE QUANTITATIVE METHODOLOGY Reliability and Validity of an Instrument Reliability – measure of consistency Construct/Face Validity – measure of instrument’s ability to operationalize program theory

9. HOW TO USE QUANTITATIVE METHODOLOGY Reliability and Validity of an Instrument Program Theory – In Evaluation: A systematic approach, Rossi, Lispey, and Freeman defined program theory to be the “plan of operation, the logic that connects [program] activities to the intended outcomes, and the rationale for why [a program] does what it does” (2004, p. 44).

10. QUANTITATIVE METHODOLOGY: USING AN EXISTING INSTRUMENT Process of Selecting Instruments Colleagues & Internet Expected Outcome/Item List Permission From Survey Designer Reliability & Validity of Instrument Instrument Modification

11. QUANTITATIVE METHODOLOGY: CREATING AN INSTRUMENT Survey Design Guidelines Miami Dade College Common Mistakes (double barrel and ill- defined terms) Length of survey Survey Pilot

12. QUANTITATIVE METHODOLOGY: CREATING AN INSTRUMENT Likert Scales Design Direction Numerical versus Categorical Number of Response Options Actionable items

13. BOLSTERING RESPONSE RATE Survey Invitation Multiple Reminders Captive Audience Incentives Culture of Assessment

14. QUANTITATIVE DATA ANALYSIS Level of Participation Average Scores/Ratings Mean Comparison Relationship between variables Prediction of outcome CAUTION: Statistical vs. Practical Significance & Causation

15. RESOURCES Statistics For People Who (Think They) Hate Statistics Dr. Neil J. Salkind Web Center for Social Research Methods: Selecting Statistics http://www.socialresearchmethods.net/selstat/ssstart.htm Statnotes: Topics in Multivariate Analysis, by G. David Garson http://faculty.chass.ncsu.edu/garson/PA765/statnote.htm