2. DESIGN FEATURES OF NCHS SURVEYS By Iris Shimizu Mathematical Statistician Office of Research and Methodology, NCHS Disclaimer: The opinions in this presentation are those of the presenter and not necessarily those of NCHS.

3. OUTLINE DESIGN FEATURES OF ESTABLISHMENT SURVEYS oSAMPLE DESIGN oDATA WEIGHTS FEATURES COMMON TO ALL COMPLEX SAMPLE SURVEYS (both establishment and population surveys) 3

4. ESTABLISHMENT SURVEYS TARGETED ANALYSIS UNITS: CLIENTS OF ESTABLISHMENT EVENTS OCCURRING AT/WITH ESTABLISHMENT ESTABLISHMENTS THEMSELVES 4

5. National Health Care Survey National Hospital Discharge Survey National Survey of Ambulatory Surgery National Ambulatory Medical Care Survey National Hospital Ambulatory Medical Care Survey National Nursing Home Survey National Home and Hospice Care Survey 5

6. FEATURES MULTI-STAGE SAMPLING PRIMARY SAMPLING UNITS (PSUs) oESTABLISHMENTS oAREAS (USED TO SAVE COSTS) 6

7. FEATURES (CONTINUED) STRATIFICATION oGEOGRAPHY oPROVIDER SPECIALTY oSIZE (INPATIENT BEDS, VISIT VOLUME) oESTABLISHMENT TYPE oOWNERSHIP TYPE SELECTION WITH PROBABILITY PROPORTIONAL TO SIZE (PPS) 7

8. FEATURES (CONTINUED) SAMPLING FREQUENCY oEVERY YEAR FOR PHYSICIANS oPERIODICALLY FOR OTHER ESTABLISHMENTS  BASIC SAMPLE –NEW DESIGN  UPDATES PERIODICALLY 8

9. FEATURES (CONTINUED) WITHIN ESTABLISHMENT SAMPLING oTIME SAMPLE oVISIT SAMPLE –FROM FRAME PROVIDED BY ESTABLISHMENT oSTRATIFICATION oSYSTEMATIC RANDOM SAMPLING oPPS FOR SELECTING SERVICE AREAS 9

10. OVERALL PROBABILITY PRODUCT OF PROBABILITIES AT EACH SAMPLING STAGE ACCOUNTS FOR SAMPLING DESIGN FEATURES 10

11. DATA WEIGHTS INVERSE OF SELECTION PROBABILITIES ADJUSTMENT FOR UNIT NON-RESPONSE CALIBRATION – USES DATA FROM NON- SAMPLE SOURCE FOR UNIVERSE 11

12. VARIANCES USING PUBLIC USE FILES REFER TO DATA FILE DOCUMENTATION FOR RECENT YEARS AND BARRING RISKS, NEEDED DESIGN VARIABLES ARE IN FILES RESEARCH DATA CENTER 12

13. SUMMARY FOR ESTABLISHMENT SURVEY DESIGN DESIGNS USE MULTI-STAGE STRATIFIED SAMPLES WEIGHTS AND VARANCES REFLECT THE COMPLEX SAMPLES 13

14. TURNING ATTENTION TO FEATURES COMMON TO ALL SURVEYS (BOTH POPULATION & ESTABLISHMENT 14

15. DANGERS OF NOT USING SAMPLE WEIGHTS UNWEIGHTED ESTIMATES: OF TOTALS WILL BE TOO SMALL OF RATES AND OTHER RATIOS COULD BE DISTORTED. I.E., UNWEIGHTED SAMPLE PROPORTIONS COULD DIFFER FROM THE CORRESPONDING CENSUS PROPORTIONS 15

16. VARIABILITY OF SURVEY ESTIMATES ESTIMATES BASED ON SAMPLES ARE SUBJECT TO SAMPLING VARIABILITY ESTIMATES OF SAMPLING VARIANCES MUST ACCOUNT FOR SAMPLE DESIGNS FOR VALIDITY 16

17. COMPLEX SURVEY FEATURES AFFECTING VARIANCE ESTIMATION CLUSTERING ANALYTIC UNITS WITHIN PRIMARY SAMPLING UNITS (PSUs) STRATIFICATION OF PSUs 17

18. DANGER OF USING SAMPLE SUBSETS TO ESTIMATE VARIANCES VARIANCE ESTIMATES BASED ONLY ON SUBSETS OF SAMPLE MAY NOT CORRECTLY REFLECT SAMPLE DESIGN COULD UNDERSTATE SAMPLING VARIANCE 18

19. DANGERS OF IGNORING SAMPLE DESIGN IN VARIANCE ESTIMATION VARIANCE ESTIMATES PROBABLY TOO SMALL “DEGREES OF FREEDOM” WOULD BE TOO LARGE 19

20. GENERAL REFERENCE FOR SURVEY ANALYSIS SOFTWARE http://www.hcp.med.harvard.edu/statistics/ survey-soft/ Provides descriptions and links to software packages that do variance estimation with complex sample data. 20

21. SUMMARY FOR ALL COMPLEX SAMPLE SURVEYS SAMPLING WEIGHTS SHOULD BE USED SHOULD USE COMPLEX SAMPLE ANALYSIS SOFTWARE TO ESTIMATE VARIANCES SHOULD USE WHOLE SAMPLE TO ESTIMATE VARIANCES 21