1. William M. van der Veld w.vanderveld@bsi.ru.nl Rites & Rituals in statistics Reporting stepwise analysis or reconstructed logic Rites & Rituals in statistics

2. Introduction Stepwise analysis (not stepwise variable selection) Students, PhDs, Supervisors. Main theme: If you don’t now why, just don’t do it. Any reported result which is not necessary to draw a conclusion will lead to confusion. (Do not report intermediate results that are irrelevant for your conclusion). Examples: -Regression (Competing theories, Interaction) -Path analysis (Reciprocal causality)

3. Regression: Competing theories Generalized Social Trust (GST) is assumed to be fundamental for a healthy society (Putnam, 1993; Fukuyama, 1995). Several theories have been suggest that explain GST (Delhey & Newton, 2003). -Success and Well-being theory (SWB) -Social network theory (SN) -Community theory (CO) -Voluntary organisations theory (VO) Research question: How much does each theory contribute to the explanation of GST? (Who is the winner?) Operationalizations (ESS, 2005): -SWB = Happy, MeetEnds -SN = Social -CO = FeelSafe, CitySize -VO = Formal Participation Country: Greece (n=2483) There is some arbitrariness in the operationalization, which is not of key importance here.

4. Regression: Competing theories Model12 34 (Constant)1.84 * 1.69 * 1.22 * FeelSafe.42 *.41 *.40 *.33 * CitySize.12 *.11 *.10 * FormPart.13 *.12 *.11 * Social.05 *.01 Happy.14 * MeetEnds -.03 R 2.035 *.039 *.041 *.065 * R 2 -change.004 *.002 *.024 * OCD behaviour: add the groups of variables (CO, VO, SN, SWB) one by one. This is reported as follows: Added value of reporting the analysis in a stepwise manner? Model 1, 2, and 3, support for the research question? How? What happens if the order of the steps is changed?

5. Regression: Competing theories Model12 34 (Constant) 2.70 * 2.59 * 1.25 * 1.22 * Happy.16 *.14 * MeetEnds -.12 * -.11 * -.04-.03 Social.03.02.01 FeelSafe.33 * CitySize.11 *.10 * FormPart.11 * R 2.041 *.062 *.065 * R 2 -change.001.021 *.002 * Add the groups of variables (SWB, SN, CO, VO) one by one (vs. CO, VO, SN, SWB). Not a significant increase in model 2. Social not important? One cannot draw a conclusion until the complete model. With model 4 it is difficult to conclude about the contribution of groups of variables. Reporting in-between results like this, leads to confusion.

6. Regression: Competing theories ModelSWBSN COVOAll (Constant) 2.70 * 3.08 * 1.84 * 3.34 * 1.22 * Happy.16 *.14 * MeetEnds -.12 * -.03 Social.09 *.01 FeelSafe.42 *.33 * CitySize.12 *.10 * FormPart.17 *.11 * R 2.041 *.006 *.035 *.006 *.065 * Analyse groups of variables (SWB, SN, CO, VO). Analyse them all at once, complete model. Research question: The literature was right, all theories – separately – explain GST. When considered simultaneously, things change.

7. Regression: Interaction Generalized Social Trust (GST) is assumed to be fundamental for a healthy society (Putnam, 1993; Fukuyama, 1995). Voluntary organizations (Putnam, 1993) Research question: Is the theory correct and does it work in the same way in different cultures/countries. Operationalizations (ESS, 2005): -VO = Formal Participation -Country = Greece (n=2483), Denmark (n=1459) -Interaction = product of centered variables.

8. Regression: Interaction Model12 3 (Constant) 3.54 * 1.96 * 2.03 * Income.39 *.05 * Gender -.19 *.09 * FormPart.34 *.33 * Country 3.16 * 3.20 * Interaction -.20 R 2.087 *.450 *.451 * R 2 -change.364 *.001 * OCD behaviour: Add the interactions in a separate step. This is reported as follows: Added value of reporting the analysis in a stepwise manner? Model 1 and 2, support for the research question? How?

9. Path analysis: Reciprocal causality If people work together, the will learn to trust each other (VO) Why not: If people trust each other they will work together. Why not both? Reciprocal causality (conceptual model). Translates into this path model, longitudinal design. y1y2 y1 t1 y1 t2 y2 t1 y2 t2

10. Path analysis: Reciprocal causality y1y2

11. Path analysis: Reciprocal causality This leads to the following hypothesis: H1: work-related stress (y1 t1 ) is negatively affected by sleep quality one year later (y2 t2 ). H2: sleep quality (y2 t1 )is negatively affected work-related stress one year later (y1 t2 ). Obviously, -work-related stress at t1 predicts work-related stress at t2 (autoregression). -sleep quality at t1 predicts sleep quality at t2 (autoregression). Originally, there was no research question in this study, but this would do (I guess). Research question: Is there a reciprocal causal relationship between work-related stress and sleep quality? y1 t1 y1 t2 y2 t1 y2 t2

12. Path analysis: Reciprocal causality OCD behaviour: add and delete effects and compare models. y1 t1 y1 t2 y2 t1 y2 t2 y1 t1 y1 t2 y2 t1 y2 t2 y1 t1 y1 t2 y2 t1 y2 t2 y1 t1 y1 t2 y2 t1 y2 t2

13. Path analysis: Reciprocal causality ModelCHI2df RMSEACFINNFIAGFI 1 18.002.10.98.95 2A 5.621.07 1.00.97 2B 11.431.11.99.94 30 0 - - -- 1 - 2A12.381 2A is a better model 1 - 2B6.571 2B is a better model 2A - 35.621 3 is a better model 2B - 311.4313 is a better model Results of 4 analyses are reported similar to the table below. Added value of reporting the analysis in a stepwise manner? Model 1, 2A, and 2B, support for the research question? How?

14. Conclusion Research question determines what you report. Research question does not determine how you analyse (stepwise is alright). There is nothing wrong with doing stepwise analyses. It often leads to confusion if you ritualistically report intermediate results of the stepwise analyses. Use reconstructed logic to formulate how you report your analyses, and make sure the research question is answered.