Florida Legislature Office of Program Policy Analysis & Government Accountability 1 Overview of Quantitative Research Methods

Florida Legislature Office of Program Policy Analysis & Government Accountability Training Overview Describes when quantitative analyses are appropriate Describes when quantitative analyses are appropriate Provides an overview of four techniques Provides an overview of four techniques Describing the central tendency and variation in a group Describing the central tendency and variation in a group Analyzing trends in program expenditures Analyzing trends in program expenditures Regression analysis Regression analysis Survival analysis Survival analysis 2

Florida Legislature Office of Program Policy Analysis & Government Accountability 3 Elements of a Finding Criteria Criteria Condition Condition Significance Significance Cause Cause Recommendation Recommendation

Florida Legislature Office of Program Policy Analysis & Government Accountability 4 Questions Addressed by Quantitative Analysis How typical? How typical? What fraction? What fraction? How many or how much? How many or how much? What has changed? What has changed? How does one group compare with other groups? How does one group compare with other groups?

Florida Legislature Office of Program Policy Analysis & Government Accountability 5 The Power of Quantitative Analysis Where anecdotes can provide rich descriptions of a few, quantitative analysis can be used to describe large aggregates Where anecdotes can provide rich descriptions of a few, quantitative analysis can be used to describe large aggregates Quantitative analysis enjoys the legitimacy of the “objectivity” promised by mathematical analysis Quantitative analysis enjoys the legitimacy of the “objectivity” promised by mathematical analysis

Florida Legislature Office of Program Policy Analysis & Government Accountability 6 Techniques Describing groups - central tendency and variation Describing groups - central tendency and variation Analyzing trends in program expenditures Analyzing trends in program expenditures Regression analysis Regression analysis Survival analysis Survival analysis

Florida Legislature Office of Program Policy Analysis & Government Accountability Central Tendency and Variation Mean (average) Mean (average) Median Median Mode Mode Range Range 7

Florida Legislature Office of Program Policy Analysis & Government Accountability 8 Teacher Salaries: District A Avg. = $40,300

Florida Legislature Office of Program Policy Analysis & Government Accountability 9 Teacher Salaries: District B Avg. = $40,300

Florida Legislature Office of Program Policy Analysis & Government Accountability 10 Teacher Salaries: District C Avg. = $40,300

Florida Legislature Office of Program Policy Analysis & Government Accountability 11 Teacher Salaries: District D Avg. = $40,300

Florida Legislature Office of Program Policy Analysis & Government Accountability Analyzing Trends in Expenditures Annual percent change Annual percent change Average annual percent change Average annual percent change Disaggregating expenditures Disaggregating expenditures 12

Florida Legislature Office of Program Policy Analysis & Government Accountability Measures of Change Annual percent change Annual percent change ($ T2 - $ T1 ) / $ T1 or ($ T2 / $ T1 ) - 1 Average annual percent change Average annual percent change ($ T5 / $ T1 ) 1 / (# years*) – 1 * # of years = 4 in this case 13

Florida Legislature Office of Program Policy Analysis & Government Accountability 14 Trends in Medicaid Expenditures Fiscal YearTotal Medicaid Expenditures Average Annual % Increase Medicaid Prescription Drug Expenditures Average Annual % Increase $6,139,478,988$608,513, $6,281,428,233$730,569, $6,611,527,446$845,774, $6,946,629,422$1,027,454, $7,763,865,8176.0%$1,313,358, %

Florida Legislature Office of Program Policy Analysis & Government Accountability 15 Disaggregating Expenditures Into Factors Driving Expenditures Factors Driving Drug Expenditures FY FY Average Annual % Increase Total Expenditures (Volume x Price) $608,513,970$1,313,358, % Volume (# of prescriptions)17,689,06822,158,9125.8% Population (# enrollees)1,150,1621,155,0790.1% Utilization (scripts / enrollee) % Price$34.40$ %

Florida Legislature Office of Program Policy Analysis & Government Accountability 16 Regression Analysis Y = a + B(X1) + B(X2) + B(X3) +... Regression analysis estimates Y given values of X1, X2, X3, etc. The Bs indicate the independent effect the Xs have on Y.

Florida Legislature Office of Program Policy Analysis & Government Accountability 17 Estimating Hospice Expenditures Total Expenditures Hospices reporting expenditures (N=37) $910,272,301 Hospices not reporting expenditures (N=6) $0

Florida Legislature Office of Program Policy Analysis & Government Accountability 18 Estimating Hospice Expenditures

Florida Legislature Office of Program Policy Analysis & Government Accountability 19 Estimating Hospice Expenditures Total Expenditures Hospices reporting expenditures (N=37) $910,272,301 Hospices not reporting expenditures (N=6) $193,073,565 Total$1,103,345,866

Florida Legislature Office of Program Policy Analysis & Government Accountability 20 Average 3-Yr. Costs Per Person Total $ Nursing Home $ Waiver $ Other $ Non- waiver $39,896$25,556$0$14,340 Waiver$59,497$3,937$53,575$1,985 Examining Cost Effectiveness

Florida Legislature Office of Program Policy Analysis & Government Accountability 21 Avg. Total $ Avg. Months Avg. Monthly $ Adj. Monthly $ Non- waiver $39, $2,046$2,558 Waiver $59, $2,271$2,657 Difference$225$99 Examining Cost Effectiveness

Florida Legislature Office of Program Policy Analysis & Government Accountability Survival Analysis Kaplan-Meier Kaplan-Meier Cox regression Cox regression 22

Florida Legislature Office of Program Policy Analysis & Government Accountability 23 Survival Analysis – How Long Until?

Florida Legislature Office of Program Policy Analysis & Government Accountability Review 1.Under what conditions is the mean (average) not the best measure of central tendency? 2.What are the best measures of central tendency under these conditions? 24

Florida Legislature Office of Program Policy Analysis & Government Accountability Review Why is using the geometric mean for the average annual percent change more appropriate than average of the annual percent changes in column B? 25 YearExpendituresAnnual percent change Avg. annual percent change (geometric mean) 2006$500, $525, % 2008$526, % 2009$535, % 2010$542, %2.04%

Florida Legislature Office of Program Policy Analysis & Government Accountability Review What are some instances where it would be important to adjust for the characteristics of members of a group when comparing multiple groups on an outcome (e.g., cost effectiveness)? What are some instances where it would be important to adjust for the characteristics of members of a group when comparing multiple groups on an outcome (e.g., cost effectiveness)? Would regression analysis be appropriate? Would regression analysis be appropriate? 26

Florida Legislature Office of Program Policy Analysis & Government Accountability Review Survival analysis techniques are a better alternative to estimating how long it takes an event to occur (e.g., release from a treatment facility). What is the disadvantage of the more commonly used average length of stay for persons released in a year? 27