1. David C. Chang, PhD, MPH, MBA Director of Outcomes Research UCSD Department of Surgery Introduction to Outcomes Research Methods and Data Resources

3. Surgery and public health

4. Problem in surgical clinical research Unregulated FDA regulation applies only to “devices” (whether a real device, or a molecular device in the form of a drug) Procedural medicine are not regulated Many reasons: complexity, difficulty in standardizing, difficulty of enforcement (“surgeons know best” attitude) Self-regulation

5. Erroneous literature

6. RCTs often too late “Tipping Point” EVAR-1, DREAM OVER

7. Social responsibility It is our responsibility in academic medicine, to shoulder the responsibility that, in other fields of medicine, has been assumed by the FDA To ensure that only good treatment modalities are applied to patients

8. Biggest barrier to good research? Not having a correctly constructed hypothesis Incorrect design Don’t know how to get data Fear of statistics

9. Typical questions Components What/why/when/how Verb Condition “Why is the sky blue?” “What is the typical presentation of appendicitis?” Open-ended

10. Open-ended questions Descriptive analysis Observational study = no comparison = no statistical test Only one denominator May have more than one numerator, generating more than one ratio All ratios are calculated with the same denominator

11. Descriptive statistics P value not applicable to compare different parts of the same population

12. Value and pitfall To explore the unknown When you know nothing, the first step is to explore and document the numbers Risk of over-generalizing

14. Inferential statistics P value applicable for comparing parts of two populations

15. What is a hypothesis? Question ≠ hypothesis Questions: usually open-ended Hypothesis: usually is closed-ended, asking for a yes/no answer Statistical testing can only give yes/no answers

16. The process – study design Study design phaseData preparationAnalysis phase Question developmentSelect databaseUnivariate Define populationLink databaseBivariate Define subsetSelect data elementsMultivariable Define outcomeGenerate new data elementsSensitivity Define primary comparisonSubset analysis Define covariates

17. Steps in constructing a hypothesis Specify the outcomes (O in PICO) Common oversight: Often focus on the P, but vague about O (a typical question, “What is the outcome (?) of xyz patients?”) Specify the comparisons (C in PICO) Not done in open-ended questions Specify covariates (control variables, adjustment)

18. Hypothesis statement y = b1X1 + b2X2 + b3X3 Death = age + race + gender + insurance…

19. Inclusion/exclusion criteria Just like a clinical trials (“eligibility criteria”) Diagnosis and/or procedure codes? Common mistake

20. Comparison

21. Outcome Mortality? Rare Complications Length of stay Charges Be judicious

22. Covariates / independent variables Patient demographcis Patient comorbidity Surgeon volume Hospital volume Hospital type (teaching vs non-teaching) Area (rural vs urban)

23. Hierarchy of influence on surgical outcomes Technique and Management Patient Surgeon Hospital Region Nation Outcomes research Clinical trials

24. The process – data preparation Study design phaseData preparationAnalysis phase Question developmentSelect databaseUnivariate Define populationLink databaseBivariate Define subsetSelect data elementsMultivariable Define outcomeGenerate new data elementsSensitivity Define primary comparisonSubset analysis Define covariates

25. Overview of public and semi-public databases Multi-specialty Administrative Databases Nationwide Inpatient Sample (NIS) Medicare, Medicaid California OSHPD Clinical Databases National Surgical Quality Improvement Program (NSQIP) Specialty-specific Trauma National Trauma Databank (NTDB) Oncology Surveillance, Epidemiology, and End Results (SEER) National Cancer Databank (NCDB) Transplant United Network for Organ Sharing (UNOS)

26. Administrative databases Advantages Large patient numbers Less selection bias Can be linked to other databases containing other non- medical information Disadvantages Limited clinical course information Limited surgical procedure information

27. NSQIP/non-NSQIP in-hospital mortality

29. Select data elements

30. Generate new data elements Most time consuming step of outcomes analysis Not every component of your research question is readily available in the database For example, comorbidity Charlson Index, Elixhauser Index Some common concepts actually undefined Readmission?

31. What is a “re-admission”? Not all “admissions” are “re-admissions” 30-day? Elective? Transfers? Diagnosis-specific? Preventable?

32. The process – analysis Study design phaseData preparationAnalysis phase Question developmentSelect databaseUnivariate Define populationLink databaseBivariate Define subsetSelect data elementsMultivariable Define outcomeGenerate new data elementsSensitivity Define primary comparisonSubset analysis Define covariates

33. Hypothesis statement y = b1X1 + b2X2 + b3X3 Death = age + race + gender + insurance…

34. Table 1: Descriptive analysis

35. Table 2: Bi-variate analysis (unadjusted comparison)

36. Table 3: Multivariable analysis (adjusted analysis)

37. Analysis for Table 1

38. P value not applicable to compare different parts of the same population

39. Analysis for Table 1 % for categorical data Mean/median/SD for continuous data For exploratory studies, descriptive studies, case series, etc., this would be the end of the process Reminder, avoid overgeneralizing

41. Analysis for Table 2

42. Think about data types… Continuous data Categorical data (Ordinal data)

43. Analysis for Table 2 Two questions to think about when picking a stats test… What is my outcome/dependent variable? What is my independent/input variable? What type of data do I have for each? 4 possible combinations: 2 variables 2 data types

44. X = input Y = outcome Cat. Cont. T-test Rank sum ROC 22 Correlation Analysis for Table 2

45. Analysis for Table 3

46. X = input Y = outcome Cat. Cont. Logistic regression Linear regression T-test Rank sum ROC 22 Correlation Analysis for table 3

47. Subset analysis Consistency of findings Generalizability

50. “This is not research anymore”

51. “That guy”