Mediation Models Laura Stapleton UMBC

Mediation Models Laura Stapleton UMBC
I said some opening stuff…. Then…. Everything that I will be talking about today is about a two-level context… so get that framework in your head… we have students at level 1 and schools at level 2.

Tasha Beretvas University of Texas at Austin
Mediation Models Tasha Beretvas University of Texas at Austin Laura is my Geppetto and MacKinnon, Barron, Kenny, Pituch, etc. are her puppeteer assistants. I said some opening stuff…. Then…. Everything that I will be talking about today is about a two-level context… so get that framework in your head… we have students at level 1 and schools at level 2.

Session outline What is mediation? Basic single mediator model
Short comment on causality Tests of the hypothesized mediation effect Mediation models for cluster randomized trials Brief mention of advanced issues First, we will discuss the basic mediation model, described in the classic Baron and Kenny article (which you all had on your required reading list)… Then, I have a short comment on causality with experimental designs and examining mediation… what you can say and what you cannot say. And then, about half of the time today, we will talk about TESTs of the mediation effect. There are different ways to test that effect. In grant proposal note which you’re using and JUSTIFY choice. In the next section, another ½ of the session, We will go over mediation models FOR cluster randomized situations instead of assuming you have single level data. What are the types of models you are going to need to run to test those paths?

What is mediation? A mediator explains how or why two variables are related. In the context of interventions, a mediator explains how or why a Tx effect occurs A mediator is thought of as the mechanism or processes through which a Tx influences an outcome (Barron & Kenny, 1986). If X  M and M  Y, then M is a mediator X causes proximal variable, M, to vary which itself causes distal, variable,Y, to vary A mediator is the mechanism by which an effect occurs.

What is mediation? Mediational process can be Who cares?!
Observed or latent Internal or external At the individual or cluster level Based on multiple or sequential processes Who cares?! Mediation analyses can identify important processes/mechanisms underlying effective (or ineffective!) treatments thereby providing important focal points for future interventions.

Mediation Examples Bacterial exposure  Disease
Bacterial exposure  Infection  Disease Stimulus  Response Might work for simple organisms (amoebae!), however, for more complex creatures: Stimulus  Organism  Response Stimulus  Expectancy  Response Monkey and lettuce example Maze-bright, maze-dull rats and maze performance example Expectancy theory: Showed lettuce on a plate to one random sample of monkeys. Put a screen in front, lifted screen and monkeys devoured lettuce. To second monkeys’ sample, showed BANANA on a plate. Put screen in front, substituted lettuce, lifted screen and monkeys got angry and yelled and refused to eat lettuce. Told researchers that rats were maze-bright or maze-dull (although rats were simply randomly assigned) before giving researchers rats to train. Maze-bright rats did better on maze performance. Self-fulfilling prophecy hypothesis

Mediation Examples Intervention  Outcome
Intervention  Receptivity  Outcome Intervention  Tx Fidelity  Outcome Intervention  Tch Confid Outcome Intervention  Soc Comp Achievement Intervention  Phon Aware  Reading Intervention  Peer Affil  Delinq Beh Emphasize again that mediator can be function of researcher, teacher, individual, contexts, etc.

Mediation  Moderation
A moderator explains when an effect occurs Relationship between X and Y changes for different values of M More in later session of workshop…

Basic (single-level) mediation model
Treatment Outcome Mediator a b Treatment Outcome c’ First presenting single-level context… assume that we have randomly sampled children and randomly assigned individuals to treatment or control We have a treatment and we have an outcome. We hope that the treatment affects the outcome in some way = total effect Picture and equations. If you like the pictures, look here… if you like the Greek, look here. If you like neither, you’re probably at the wrong workshop So, here we have our total treatment effect. The total effect of the treatment on the outcome. This is typically referred to as PATH C in writings on mediation Beta1 is the same as our “C” PATH value. If you are interested in mediation though, what you are interested in is “does my treatment effect some proximal variable (we’ll call that the “A PATH” and then does my mediator (that proximal variable) have an effect on the distal outcome that I am interested in (the “B PATH”). And then, finally, is there a remaining direct effect from the treatment to the outcome? In order to do this, we have 2 dependent variables… (mediator and outcome) .. So we are going to run 2 regression analyses (unless we use PATH ANALYTIC software and then we can do it all in one analysis). Mediator is regressed on some intercept plus treatment (Beta1 is the effect of the treatment on the mediator) And then we also have the regression of Y, the outcome, on both the mediator and the treatment. Note that each time you see a prime sign that refers to the regression eqn including the outcome as the dependent variable. No prime indicates that the mediator is the outcome (regressing mediator on exogenous variable). So, our b = beta1-prime path is the effect of the mediator on the outcome c-prime = beta2-prime is the effect of treatment on the outcome, AFTER controlling for whatever happens between the mediator and outcome. Notice that the total effect, that “C” up there… the total effect of treatment on outcome, for example, students in treatment had 5 points higher on outcome on average. That total effect, C, that 5? It can be split into 2 pieces, the indirect effect, calculated by multiplying path A times path B (product), and the direct effect. So, if my A path was a 2 and my B path was a 1, then the indirect effect would be 2 and the C-prime would be 3. In single level models, your indirect effect and direct effect added together will always be your total effect. In multilevel models, it could be slightly different, but it usually isn’t. I have Jumped right into modeling…. But I haven’t really talked about WHY you want to look at mediators… total effect = indirect effect + direct effect c = ab c’

Causality concerns Just because you estimate the model X  M  Y
does not mean that the relationships are causal Unless you manipulate M, causal inferences are limited. Mediation model differs from Mediation design

Causality concerns – mediation model
Remember, if the mediator is not typically manipulated, causal interpretations are limited Z Mediator M a b  Treatment T Ok! Outcome Y Possible misspecification Because we have not manipulated the mediator, your causality interpretations are somewhat limited: haven’t ruled out other possible explanations Treatment affects the mediator. You’re actually okay with a causal interpretation of that because you have manipulated treatment. But it’s this path. The path form the mediator to the outcome that you really cannot use a causal interpretation for. You might have a misspecification in your model. You might have a feedback loop (simultaneity). If you have feedback loops, maybe treatment affects the teacher’s ratings of importance, which maybe affects the child’s behavior, But hey, the child is behaving really well now, I think this is even more important … and then the child behaves even better. There is a way you can use instrumental variables to parse out that feedback. It is difficult to do But also we could have this situation… The treatment directly affects the children’s behavior, children get better, and hey, teacher decides this is really important. And another one… there could be a 3rd variable, termed a confounding variable, Z, out here that is causing both of them. All sorts of things could be happening. I encourage you to do not make causal statements. Do no say THIS mediates the relation between X and Y. Say “I’m assuming… or I’m hypothesizing… that M is causing Y. And given that assumption, this is the strength of that indirect effect. This is the strength of the mediation.” But you need to lay out to the reader that “I’m making that causal assumption that the arrow is going from the mediator to the outcome.” Judea Pearl has written some fabulous stuff on making that assumption and clarifying to people that that’s an assumption that you’re working with. MacKinnon suggests that, if you can, in future research manipulate the mediator directly. In fact, this gets back to that single subject design where they go back in and do components analysis and directly manipulate teacher importance. I’m not sure… you take some teachers and get them into a room and brain wash them (you WILL change your attitude!) The other people you leave along… and you see if there is a change in child outcome. That was just a quick review of causality and I won’t practice what I preach here but be careful For now, be sure to substantively justify the causal direction and “assume or hypothesize that M causes Y and assuming that, here’s the strength of that effect…” In future research, manipulate mediator

Tests of the hypothesized mediation effect
Mediator M a b Treatment T Outcome Y c’ The estimate of the indirect effect, ab, is based on the sample To infer that a non-zero αβ exists in the population, a test of the statistical significance of ab is needed Several approaches have been suggested and differ in their ability to “see” a true effect (power) Tests of the hypothesized mediation effect. This is going to be a big chunk of stuff. When you run your model and you get that ab indirect effect value, that’s the value for your sample. And everything you’ve done in statistics is “okay, I’ve got an estimate of what the population parameter is…” So you ab will be your ESTIMATE of the population paramenter alpha-beta but to infer that a non-zero alpha-beta exists in the population, you’ve actually got to test it somehow. A significance test. And there have been several approaches that have been developed to test whether that ab product is significantly different from zero. And, this is something that… well, I have graduate students who are not sort of stat-geek graduate studnets… this is a very uncomfortable thing to think about… there are many different methods of doing it and no one has decided which one is right. Everytone think… you know… stats… There’s a formula! That must be right. Or there’s one way to do things… But there are several ways that people test indirect effects and basically anytime someone wants a dissertation topic in my field, they say “oh, let me test that one!” Simulation studies. So, we’re going to learn about five Different types of tests. I have my own opinoin (which I will share with you) about which one will be better to choose if you were going to tell a funding agency which you would use…

Tests of the hypothesized mediation effect
Causal steps approach (Baron & Kenny) Test of joint significance z test of ab (with normal theory confidence interval) Asymmetric confidence interval (Empirical M or distribution of the product) Bootstrap resampling There’s a causal steps approach…. And you folks have read about that in your required reading. There’s a test of joint significance…. Oh, and I’m going to go through each one of these in detail…this is just an overview. The test of joint significance… it sounds really fancy, but it’s not.. .very easy. The Z-test of that ab product…. You’ve actually seen this all the time… you don’t yet know it … but you’ve seen it. And then you’ve got the asymmetric confidence interval… these next two things you may not have heard about… but the asymmetric confidence interval…we’ll build a confidence interval around the ab product and see if it contains zero or not. And bootstrap resampling… there are several different ways resampling can be done…with the ab product and I’ll cover some of those. So, that’s where we’re headed. We are going to cover these five different types of tests. Once we do that, we will then consider some models for field trials (instead of the single level model we discussed).

Causal steps approach Step 2: test the effect of T on M (path a)
Step 1: test the effect of T on Y (path c) c Treatment Outcome Step 2: test the effect of T on M (path a) Mediator a The causal steps approach. In step 1, all we are going to do is test that C path. Is C, the path from the treatment to the outcome, significantly different from zero? I run my analysis, I get a C value of “5”… is that significantly different from zero? Yes? Okay, I go on. If it’s no, I stop. I say “nope, there is no significant mediation.” Baron and Kenny assume that you’ve got to have that first step. In step two, you test the effect of the treatment on the mediator. And that’s our path A. Is that significantly different from zero? Yes? Okay, you move on. No? you stop. Treatment

Causal steps approach Step 3: test the effect of M on Y, controlling for T (path b) Mediator b Treatment Outcome c’ Step 3. You probably know where I am going…. You’re going to test the effect of the mediator on the outcome… if it is significantly different from zero, we’re golden… we can now say “hey, there is a significant indirect effect… there is significant mediation of the effect of treatment on the outcome via this mediator that I’m interested in. (assuming my hypothesized model is correct). There’s a 4th step that people tend to do which is to determine if it is partial mediation or complete mediation. If I have that mediator in there…is c-prime so close to zero now that it’s not significant. If it is just about zero, then you have complete mediation. The entire effect of the treatment is explained by what’s going on with that mediator. I have rarely seen complete mediation. Step 4: to decide on partial or complete mediation, test the effect of T on Y, controlling for M (path c’)

Causal steps approach: performance
Step 1 may be non-significant when true mediation exists Mediator FdF +2 +3 What if… Treatment T Outcome Dep -6 Problems with this approach… Suppression could lead to direct effect = zero when there is true mediation exists. There was a school-based clinical trial for girls with depression… and the treatment consists of girls doing cognitive behavior therapy but with parent training as well… so the parents came and had to talk with the girls. Suppose my mediator was “Family dysFunction” measured as amount of arguing,etc. Well, in the treatment group, the girls are now talking (arguing) with their parents… where in the control group they rarely talk with them… So, it could be that the indirect effect is positive… being in treatment, leads to higher scores on family dysfunction, which leads to higher scores on dep symptoms. However there is a very strong negative direct effect of treatment. Those in treatment had 6 points lower on dep symptoms. (So, girls with equal amount of family dysfunction would benefit greatly from treatment). If we had just done Step 1, we would have concluded that overall effect of treatment was 0 so we would stop. If we had done this analysis, we would have realized that perhaps we have a component in our treatment that we need to change… need to work with families so that when they do talk, they need to not argue… This situation is called suppression. Another example of the problem of step 1 is if there are off-setting mediators. Suppose our treatment increases the girls’ social support because they learn to talk about their feelings so they talk with their friends, etc. So, the short story is, This Baron and Kenny approach is problematic because if you have a non-significant finding at step one then you stop. Mediator FdF +2 +3 or… Treatment T Outcome Dep +3 -2 Mediator SS

Causal steps approach: performance
Lacks power Power is a function of the product of the power to test each of the three paths Power discrepancy worsens for smaller n and smaller effects Lower Type I error rate than expected i.e., too conservative Another problem is that it lack power when true mediation exists. And the way that this has been demonstrated in simulation studies.. What they’ve design is they have a population of data, where the a path is not zero, and the b path is non-zero, there’s a mediation in the population, and I randomly sample data from that population…and I run the analysis… and then I randomly sample from the population… and I run the analysis…I do that thousands of times…I’ll find out that a very low percentage of the time I’ve rejected the null hypothesis of no mediation. My power was low to find mediation. There have been several simulation studies… just a few in the randomized cluster arena…and I’m going to summarize in a very broad way what the results are with regard to each of these tests. And I feel that I am doing them a disservice because I’m saying “oh, it lacks power”….but it’s hard to summarize because all these different simulation studies using different conditions…. Suppose you have this many sites…and this many people and your effect size is the big…and you have non-normal data or normal data… but in general, this one lacks power. The smaller the effect size, the smaller the number of people, the bigger the discrepancy between this method and another method of testing the indirect effect. The bigger the effect size, the bigger your sample size, the more the power sort of evens out, but in all the simulation studies I’ve seen, this one never gets up to the power of the other methods of testing the indirect effect. The power is basically a function of testing three different paths. It is a function of the power to test each of the paths. And also, mirror image… at the same issue… you’ve got low power, so you’ll have a lower Type I error rate than you expected. Usually, you go into an analysis, you say “I’m going to use an alpha value of .05…that .05 says “5 percent of the time I want to reject the null hypothesis when I shouldn’t.” This is the only profession where you are aiming to be wrong 5% of the time! And, with this test, you’re wrong less than 5% of the time. And that is very dissatisfying…at least to statisticians. I want it to be wrong 5%. And with this method, it might be 1% of the time.

Test of joint significance
Very similar to causal steps approach Mediator a b Treatment Outcome c’ 1st: test the effect of T on M (path a) 2nd: test the effect of M on Y, controlling for T (path b) If both significant, then infer significant mediation The test of joint significance. Sounds fancy… not at all. It’s very similar to the causal steps approach. The first thing you are going to do is test that A path from the treatment to the mediator. “is that significantly different from zero?” Next, you test that B path. “Is that significantly different from zero?” If the answer is yes to both of those, then you claim that you have a signfiicant indirect effect. So, we’re just looking at two pieces of that causal steps approach. The performance of this test…. (next slide)

Test of joint significance: performance
Better power than causal steps approach Type I error rate slightly lower than expected Power nearly as good as newer methods in single- level models Power lower than other methods in multilevel models No confidence interval around the indirect effect is available Actually, the power is better than the causal steps (only testing 2 paths intead of 3). Type I error rate is slightly lower than expeceted… instead of .05, maybe .03, .035…. Power, surprisingly was almost as good with this method as some newer methods I’ll be talking about today FOR THE SINGLE LEVEL CASE. When we get to multilevel models that we’ll be using for cluster randomized designs, it doesn’t quite reach as good a power as those other methods. And one thing that has gained importance lately is to provide confidence intervals around your indirect effect estimate. If I say, well, I think the indirect effect is 3… somewere aroudn 3…. Between 2.5 an 3.5… that’s a reasonably narrow actually CI around my indirect effect. I feel good about that… if someone tried to replicate my study, they would probably get a number around 3. But if I report a confidence interval that goes from .2 to 5.8…. Eh… that’s pretty wide. If someone replicated my study, I’m not sure where they are going to end up. They could have a pretty different indirect effect from the size that I saw. So, it’s really nice to provide a confidence interval for your reader…when you’re writing up your results…after you’ve been funded. So, with a test of joint significance, you don’t get that confidence interval. So that’s one downside.

z test of ab product Sobel’s seab =
Calculate z = Sobel’s seab = Compare z test value to critical values from the standard normal distribution Can also calculate confidence interval around ab CI = The Z-test of the ab product. The first thing you need to do is calculate your Z-value. And that is your product (ab) over the standard error of ab. And, how do you get the SE for a times b? When you run your regression analysis for M and Y, SPSS or whatever we run will spit out the a parameter estimate, the b parameter esitmate, and the st error of the a estimate and the SE of the b parameter estimate, but it will not spit out the SE for the ab product. There are a couple of software parackages that will do that. But, you are probably going to have to calculate your SE for the ab product and this is called the Sobel standard error (Sobel, 1982…). There are two other equations for the SE, one called unbiased and one called exact – you either add or subtract the product of the 2 sampling variances but Sobel SE has been found to perform the best of the 3. Once you have the Z test value, you compare it to your critical values on the Z distribution. So, if your z-value is above 1.96 then “woo hoo” I’ve got significant mediation.(Or below -1.96). And then you can also calculate a confidence interval using that SE and ab +/- Z (Sobel SE) Wonderful Z-test. You see it all the time (test of the indirect effect). Some people call it the Sobel TEST but it is really the Sobel SE with Z-test.

z test of ab product: performance
One of the least powerful approaches Type I error rate much lower than expected .05. Single-level models, it approaches the power of other methods when sample size are 500 or greater, or effect sizes are large Multilevel models, it never reaches the levels of other models although it does get closer although still lower Problem is that the ab product is not normally distributed, so critical values are inappropriate How is the ab product distributed? One of the least powerful approaches you can use. If your goal is power, I strongly suggest you not use this test. Type I error is much lower than expected (close to zero). In single level analyses, approaches power of other method when you have sample size of 500 or more. In multilevel model simulations, I have yet to see it get up to power levels of other tests. ..it gets closer (differential of .02 or .03 if you have 60 schools with 30 kids in each)..power almost gets there. But why give away that .02 power? The problem is that the ab product is not normally distributed. But we are using critical values (-1.96 and 1.96) that assume the sampling distribution of our estimate IS normally distributed. And to show you what this problem is, I did a little simulation (If I get the chance to make up data, I will….)

Sampled 1,000 a ~ N(0,1) and of b ~ N(0,1)
Distribution of path a Distribution of path b I simulated 1,000 estimates of a and 1,000 estimates of b…. What I did was just randomly geernate a paths when I knew in the poulation it should be zero. There is no mediation. A path should be zero and a SD of 1, so really I’m just randomly drawing numbers from a normal distribution. I simulated gettings 1000 As and I simulated getting 1000 Bs. And here was the distribution of my A paths. I have 1,000 estimates… and around this bin of 0, there are about 100 of the estimates. And 60 fell in a bin around 1… So, you can see, those 1,000 estimates seem to be normally distributed. Very rarely did I get a random draw of A out here… I did the same for B. The distribution looks a little bit different, but its basically a normal distribution… Then what I did,,, I’ve got 1,000 As and 1,000 Bs and used Excell… I multiplied each A times the B to create AB values. I have a column of AB values in my spreadsheet. What do you think the distribution of the AB product looks like? (Wait and have them guess). It is very bunched up in the middle… Anyone know what this distribution is called? What does superman do over tall buildings? He leaps… It is a Leptokurtic distribution…. (animation)…. So those critical values of and 1.96 are not relevant. They will not separate out the more extreme 2.5% of sample estimates. When ALPHA and BETA are zero, the distribution is symmetric, but that’s not the case when ALPHA and BETA are not zero. Distribution of product of axb

Empirical M-test (asymmetric CI)
Determines empirical (more leptokurtic) distribution of z of the ab product (not assuming normality) αβ=0: dist’n is leptokurtic and symmetric αβ>0: dist’n is less leptokurtic and +ly skewed αβ<0: dist’n is less leptokurtic and -ly skewed Due to asymmetry, different upper and lower critical values needed to calculate asymmetric confidence intervals (CIs). Meeker derived tables for various combinations of Za and Zb values (increments of 0.4) that could be used to calculate asymmetric CIs. This test, easier to use, has gone by a few different names. David MacKinnon at ASU has done most of the recent work on this test. This is referred to as the asymmetric CI. This approach is, instead of using normal theory, to develop a CI around your ab product., and see if that CI contains zero. We’re going to create an empirical distribution that is more leptokurtic and use that empirical distribution to figure out where that 2.5% of sample estimates fall below and above. It is leptokurtic and symmetric when alpha-beta = 0 but is skewed if alpha-beta is greater than 0 (tail will be out in the positive range) and if alpha-beta isn the poulation is neative, then you will have a neagtively skewed sampling distribution. As the population alpha-beta moves away from 0, it becomes less leptokutic. Now, the problem is, when you create a confidence interval with normal theory, assumptions, you can get a number out here oh, I”ve estimated 20… oh, +/- 1.96… oh, I’ve estimated -10, then +/- 1.96… That 1.96 is relevant where you go. With this leptokurtic distribution that is changing in its symmetry and changing in its lepto-kurticness-ity, it is very difficult to draw a CI. There is no one formula that you can use. And so there were some difficult approahces that some people tried to figure out how do we draw this confidence interval. And a guy named Meeker developed a table that you could use if your z-values was this for A and Z-value was this for B, then here is what your critical values are. But it was so labor intensive and he used values that incremented every .4. So, if your z value =0 and if you z-value = .4 and if your z-value =.8… it was very imprecise. And then MacKinnon provided a more sensitive table of values (incremented by .2s)…. Then thankfully, he created a program (with others) that uses numerical integration to use info provided…. All you have to give it is your A value, your B value and the SEs for those 2 things and it calculates that asymmetric CI for you. It is wonderful… we’ll be using ti with our example later on. You now have no excuses to use the z-test, or the causal steps approach. You can easily use what is called PRODCLIN. PRODCLIN stands for the PRODuct of the coefficient Confidence Limits for the INdirect effect. One of the readings on the recommended list introduces that software. It will provide us with CL, calculated as (shown on slide)…. Sofwtaer provides us with two nubmers. If the CI does not include zero, then you conclude that the mediation effect is significant.

Empirical M-test (asymmetric CI)
MacKinnon et al created PRODCLIN that, given a, b, and their SEs, determines the distribution of ab and relevant critical values for calculating asymmetric CI. (MacKinnon & Fritz, 2007, ). Confidence interval limits: If CI does not include zero, then significant

Empirical M-test: performance
Good balance of power while maintaining nominal Type I error rate Performed well in both single-level and multi-level tests of mediation Only bootstrap resampling methods had (very slightly) better power than this method PRODCLIN software is easy to use It has worked well in both single-level analsyes (by MacKinnon) and mutlilevel analyses (not MacKinnon – Pituch et al). Only bootstrap resampling methods have performed better than this one…and actually not that much better…. Power could be gained .03 or so with bootstrapping…. PRODCLIN is very easy to use. Okay, now we’re on to our last set of methods (next slide)

Bootstrap resampling methods
Determines empirical distribution of the ab product Several variations Parametric percentile Non-parametric percentile Bias-corrected versions of both Can bootstrap cases or bootstrap residuals. It is typical in multilevel designs to bootstrap residuals. HAVE NOT TRANSCRIBED.

Parametric percentile bootstrap
With original sample, run the analysis and obtain estimates of variance(s) of residuals New residuals are then resampled from a distribution ~N(0,σ2) (thus, the “parametric”). New values of M are created by using the fixed effects estimates from the original analysis, T and the resampled residual(s). New values of Y are created using the fixed effects, and T and M values and residual(s). Then, the analysis is run and the ab product is estimated

Parametric percentile bootstrap
The process of resampling and estimating ab is repeated many times (commonly 1,000 times) The ab estimates are then ordered With 1,000 estimates, the 25th and the 975th are taken as the lower and upper limits of the 95% (empirically derived) CI. Note that the CI limits may not be symmetric around the original ab estimate If CI does not include zero, then significant mediation

Non-parametric percentile bootstrap
The parametric bootstrap involves the assumption that the residuals are normally distributed Instead, residuals can be resampled with replacement from the empirical distribution of actual residuals (saved from the original sample’s analysis) The remaining process is the same as for the parametric version

Bias-corrected bootstrap
With both the parametric and non-parametric bootstrap, the initial ab product may not be at the median of the bootstrap ab distribution Thus, the initial ab estimate is biased BC-bootstrap procedures “shift” the confidence interval to adjust for the difference in the initial estimate and the median

Bootstrap resampling methods: performance
Resampling methods provide the most power and accurate Type I error rates of all methods Parametric has best confidence interval coverage BC-parametric had best power, especially with low effect sizes with normal and non-normally distributed residuals; Type I error rate was slightly high for multilevel analyses Non-parametric had the most accurate Type I error rates; good overall power BC Non-parametric had good power But … complicated to program Finding indirect effects is very important and need most power then use these methods! Note however that differences were not substantial.

Summary: tests of the hypothesized mediation effect
Causal steps approach Test of joint significance z test of ab Empirical M Bootstrap resampling   OK for single level…   Yes! Easy!  Yes! Not quite as easy… but does have the most power Z-test is under-powered: no reason to lose power Use bootstrapping if available in your analytic software…

Example for today Social-emotional curriculum = Tx
Child social competence = outcome Randomly selected classrooms (one per school) Why would Tx affect outcome? Teacher attitude about importance? Child understanding of others’ behaviors? Puppet show down-time soothes child? Researcher should think in advance of possible mediators to measure Curriculum includes Puppet show with ducks, monkeys Kid less hyper after relaxation of puppet show – not content related Suppose the treatment changes the way a teacher thinks about the importance of soc/emot curriculum… and that changes the teachers’ level of empathy with the children… and that changes how the teacher behaves with the children… and that changes how the child behaves… you could go into extremely detailed levels and so that would be referred to as molecular medaition

Mediation models for cluster randomized trials
Extend basic model to situations when treatment is administered at cluster level Model depends on whether mediator is measured at cluster or individual level Definition (Krull & MacKinnon, 2001) depends on level at which each variable is measured: T → M →Y Upper-level mediation [2→2→1] Cross-level mediation [2→1→1] Cross-level and upper-level mediation [2→(1 & 2) →1] That was the end of big section 1… now we’re on to big section 2. From here, we will walk through examples – and examine models.. The models will differ depending on what level your mediator is measured. So, we will put some context to this. We will extend that basic T to M to Y model to data from a cluster randomized trial. And we will go through 3 different models today. Model depends on whether you have measured your mediator at the individual level or at the group level. One type of mediation is called upper level mediation. Krull and MacKinnon have “named” these models as 2->2->1.. So, we have treatment is measured at level 2, our mediator in this upper level design is measured at level 2 and our outcome is measured at level 1. So, in the context we talked about before, we have a socio-emotional curriculum assigned at the group level… I’m measuring how teachers find soc-emotional curriculum important, and then I’m measuring child social competence at level 1. The next type of model is called cross-level mediation. So I have treatment at level 2, my mediator is measured at level 1 – maybe I’m measuring how much the child understands about soc-emotional things or how they understand other peoples’ reactions, and social competence at level 1. And then, --- cross level and upper level mediation. This is a contextual mediation. You have a mediator that is measured at level 1, but you think that the cluster average on that mediator is also an important player in this. Maybe there is a synergistic effect. That if that school has kids who really understand socio-emot things then that may have an added impact on your outcome, even more so than just individual differences in the kids’ individual degree of understanding. So that could be called cross-level and upper-level mediation or contextual mediation.

Measured variable partitioning
Cluster uoj First, consider that any variable may be partitioned into individual level components and cluster level components Yij And before we get into data, I just want to share with you how some of these models are depicted. Borrowed from ML-SEM pictures. Note, don’t depict intercepts. First consider any variable could be particioned into individual level components and cluster level components. Any of our variables in our set of 3 could be of that kind. Individual rij Note: No intercepts depicted

Mediation model possibilities
Tx Cluster M Cluster Y Cluster Tx M Y So as soon as you have cluster randomized trial data, you have to think: for my treatment… is there within- and between- variance? For my mediator, is there within variability and between variability? For my outcome, is there within and between? For each of the models today, we will figure out where the circles go and then you have to decide where to draw in arrows (direct effects). Tx Individual M Individual Y Individual

Data Example Context Cluster randomized trial (hierarchical design)
14 preschools: ½ treatment, ½ control 6 kids per school (/classroom) Socio-emotional curriculum Outcome is child social competence behavior Possible mediators: teacher attitude about importance of including this kind of training in classroom, child socio-emotional knowledge Sample data are on handout I’ve got a cluster randomized design (hierarchical design is another name)… 14 pre schools, one classroom in each pre-school has been tapped to participate in the study. ½ of the classrooms are assigned to treatment and ½ to control For ease, I have just 6 kids in each school (ease was for showing on 2 pages of paper!) My treatment is some sort of socio-emotional curriculum. The outcome is child behavior – I will have observers go into the classroom and watch the child for a week; maybe I’ll have 2 raters and average for better reliability. My possible mediators that I am thinking about are teacher’s attitudes toward the importance of including socioemotional training in the classroom and the actual socio-emotional knowledge of the child. I have an assessment where the child watches video vignettes and then answers a semi-structured interview about whether things happen on accident/purpose/etc. (not my area, I apologize for any misstatements) The handout is posted (and you have it in front of you?) – has all the data – I’ll show examples next. All analyses in this presentation use HLM, but the handout has SAS syntax as well.

Total effect of treatment
Before we examine mediation, let’s examine the total effect of treatment on the outcome… Tx Cluster 01 Y Cluster Tx Y I wanted to look at Step 1 of Baron and Kenny’s causal steps. Is there a total effect of treatment on outcome? Pictorially, my outcome variable is meausred at the inidivuudal level, but I know that there is clustering . We have kids within classrooms, so my measured variable of outcome is a function of individual variability and between group variability. And my treatment is at the between level. So, I’m going to regress that between-cluster outcome on that treatment variable. Here is the equation.. Yij is a function of between and within. And that Between effect is a function of an intercept (which I never put in my diagrams) and that gamma01prime coefficient times the treatment value plus some random error. And that gamma 01prime is our estimate of the treatment effect. So…. (next slide) Y Cluster

Total effect of treatment: FE Results
Final estimation of fixed effects: Standard Approx. Fixed Effect Coefficient Error T-ratio d.f. P-value For INTRCPT1, B0 INTRCPT2, G T, G I ran the HLM model as shown here. LISTEN TO TALK>>> DO NOT HAVE TRANSCRIPTION The results… all I’m sharing today is the fixed effects, but you can check them out at home running it on your own computers. For the fixed effects, on average, children in the control group (T=0) have a score of 34 on this behavioral measure. That’s the intercept. And the effect of the treatment… is 4.238, so on average, children in the treatment groups had socially competent behaviors that were about 4 points higher than the control group. Is that a significant effect? Yep. Probability of obtaining that large an estimate if there were no effect is only .014. Univariately, Ys standard deviation was 4.381, so the effect size is rather large. Don’t expect this in real life! Note the df; when I created these data I had to make this effect size quite large to obtain a significant effect. c

Upper-level mediation model (2→2→1)
Cluster 01 ’01 Tx Cluster Y Cluster ’02 Tx M Y Y Cluster Now we’ll look at our first mediation model…. The model. Our mediator, teacher rating of importance, is only measured at level 2. We do not have any within school variability on this measure, so the only place it has variability is up here. So to get that “A” path, we’re just going to have to run a simple regression. The Mj mediator value for school j is equal to some intercept plus the effect of treatment (gamma 01) and a residual at the school level. Notice that there are no little subscript “I”s. I will not run this analysis on 84 children. I will run this analysis on 14 observations – for the 14 teachers. I’ll regression their mediator values on the treatment status of their school. I’ll get a gamma 01 value and that’s my A path. Then, to get my B path, I’m going to have to regress Y, the outcome variable, will have components of between and within, and the between component is a function of some overall intercept, plus a function of the mediator, gamma01-prime, plus some function of treatment, gamma02-prime. In this second equation, I’ll get my B path from that gamma01-prime , and if I care, I can get c-prime would come from gamma02-prime. I will run two regressions; one of them will only be a single level regression on the data for 14 schools.

Upper-level mediation model: Results
To estimate the a path, I ran an OLS regression in SPSS using the Level 2 file I ran a regression in SPSS with J=14 schools. Our path coefficient (effect of T on M) is The treatment has .714 points higher on M than those in the control. Is that significantly different from zero? No. A= SEa = .628 We will use these later when we TEST the indirect effect. Now, let’s run a model to get that b path. What is the estimate of a and its SE?

To estimate the b path, I ran a model in HLM Final estimation of fixed effects: Standard Approx. Fixed Effect Coefficient Error T-ratio d.f. P-value For INTRCPT1, B0 INTRCPT2, G M1, G T, G LISTEN TO TALK TO HEAR HLM STUFF (didn’t transcribe) A student who is in the control group who has a teacher importance rating of 0 is expected to have a socio-emotion al score of (Note, however, that 0 does not exist on the teacher importance rating…. I recommend that you center your predictors, that way, intercepts are more meaningful – they would be for the child with a teacher at the average). Here we see that the unique direct effect of treatment is 3.67 and the B path from the mediator to the outcome is .795 and its SE is Not significnt from zero. Notice too that although we’re using 84 children for this analysis, the df are 11. Because all of this analysis is happening on the school effect size because all of our predictors are at the school level. 14 schools – 3 coefficient estimates =11 degress of freedom. Our B path = .795, SE – we’ll use these later. What is the estimate of b and its SE? What is the estimate of c’ and its SE?

Cluster .714 .795 Tx Cluster Y Cluster 3.671 Tx M Y Y Cluster Here is our picture. I’ve got our coefficients listed up there. Our unique direct effect is and our ab indirect path is the product of .714 and .795 which is .568. And so, the total effect is = 4.239 The same as we saw before. (well, very close… With multilevel it might not be exact, but pretty close). We can consider effect sizes….this is a new area… not well developed… You could create a ratio of the IE over the TE… so, .568/4.238 = 13% % of the total effect may be mediated through teacher importance. But, this ratio ES can be a problem because, in suppression, total effect may be zero and you have a large indirect effect. There are other ratios that people have proposed but none so far suggested for multilevel models. Now our estimate of the indirect effect = .568, we want to know if that estimate is significantly different from zero… and now is where YOU CHOOSE your method to test that effect. Direct effect = 3.671 Indirect effect = (.714)(.795) = .568 Total effect = DE + IE = = 4.239

Causal steps approach Test of joint significance z test of ab product Empirical-M test BC parametric bootstrap No. Step 1 significant, but not Steps 2 and 3 No. Neither path a nor path b are significant No. se=.68, z=.83, p= % CI = -.78 to 1.91 LISTEN TO TALK (did not transcribe). ASK FOLKS WHAT THEY WOULD HAVE DECIDED IF THEY HAD USED THE CAUSAL STEPS APPROACH…. ASK IF ANYONE CALCULATED THE SOBEL SE TO HELP WITH THE Z-TEST… CHIDE THE “SMART” FOLKS IN THE CLASS TO, NEXT TIME, CALCULATE IT IF THEY ARE BORED! EMPIRICAL M – PRODCLIN IS SHOWN ON THE NEXT PAGE…. BOOTSTRAP WAS DONE BY PROGRAM KEENAN AND I WROTE. Perhaps small sample size led to some bad draws for bootstrapping. No. 95% CI = -.47 to 2.26 95% CI = -.42 to 3.68 No.

PRODCLIN Prodclin/ I downloaded PRODCLIN to my computer… there are several versions (R, SAS).. But I don’t see why you’d need to use those (and embed them in another program)…. You just need to thype in 4 numbers, so it seems more trouble to open another rcomputer program to run (unless you were doing your multilevel analysis in R or SAS). So, I downloaded the PRODCNL2 executabel file. It lists “Enter values…” and I did (.714 then a space then .628) and hit enter. Then “Enter values… for B” and I did (.795 then a space then .656). Then “enter correlation…” You’re pretty much always going to say 0. This is the correlation between the A path and the B path – the correlation of those etsimates across sampling distributions. There are a few cases when you’d have something other than a zero, but it is beyond the realm of this talk. And then enter your alpha level for a 90% CI, .05 for a 95% CI.. Hit enter… You have your confidence interval of the indirect effect. Our CI, -.47 to 2.26 contains zero, so we would conclude we have a non-significant indirect effect. Note – is the CI symmetric around the estimate of the indirect effect? Is .568 directly in the center of -.47 and 2.26? No.

Cross-level mediation model (2→1→1)
Model A Model B Mediator CLUSTER γ01 Treatment CLUSTER Treatment CLUSTER γ’01 Outcome CLUSTER Mediator Mediator Treatment Treatment Outcome Mediator INDIVIDUAL Mediator INDIVIDUAL γ’10 Our next model…. Now, I’m examining a mediator that is not measured at level 2, it is measured at level 1. In this case, I’m using child knowledge of socio-emotional “stuff” as my mediator. In this one, I’ll have to have 2 models that describe this. I’ll have a model for the mediator (Med is a function of within and between variability) and that between intercept is a function of the treatment and random error. And then, in Model B, for our outcome, we have between variability and within variability and our between variability is a function of treatment but because our mediator is measured at level 1 , we could hypothesize that I think that there is an individual level mediation going on here. That kids differ in how much soc-emot they know and that kid differences will result in the difference in their behavior in the classroom Outcome INDIVIDUAL

Cross-level mediation model: Results
To estimate the a path: Final estimation of fixed effects: Standard Approx. Fixed Effect Coefficient Error T-ratio d.f. P-value For INTRCPT1, B0 INTRCPT2, G T, G To estimate the a path for this model,,, LISTEN TO TALK.. DID NOT TRANSCRIBE HLM SYNTAX. Children in the control group have child knowledge score on average of (remember this is a grand-mean centered mediator). And childen in the treatment group have 2.64 more points on average. The SE is 1.195… p=.047… signif different from zero. This is our A path. What is a and its SE?

To estimate the b path: Final estimation of fixed effects: Standard Approx. Fixed Effect Coefficient Error T-ratio d.f. P-value For INTRCPT1, B0 INTRCPT2, G T, G For M2_GRAND slope, B1 INTRCPT2, G For our B path… LISTEN TO TALK FOR HLM STUFF…. Y is regressed on the mediator at level 1 and at level 2, it is a function of treatment. A children at average knowledge and in control group has a behavior score of on average for each unit increase in child knowledge, expected behavior score increases This is our B path. SE is .143… p<.001. Notice our degrees of freedom… 81 for individual effects… 12 for group effects. What is b and its SE? And for c’?

Model A Model B Mediator CLUSTER 2.643 Treatment CLUSTER Treatment CLUSTER 2.675 Outcome CLUSTER Mediator Mediator Treatment Treatment Outcome Mediator INDIVIDUAL Mediator INDIVIDUAL .592 Outcome INDIVIDUAL The effect of treatment above and beyond mediation is (c-prime). The indirect effect is x (this is why it is called cross-level – need to jump down here). So, IE is If I add together… get total effect of ES = / = approx 35% of effect could be attributed to mediation through child knowledge. Is this IE significantly different from zero? Direct effect = 2.675 Indirect effect = (2.643)(.592) = 1.564 Total effect = = 4.239

Causal steps approach Test of joint significance z test of ab product Empirical-M test BC parametric bootstrap Steps 1, 2 and 3 significant Yes Paths a and b significant Yes se=.802, z=1.95, p= % CI = -.01 to 3.13 No Truth is that teacher mediator was generated from a population where there was no indirect effect and child mediator (here) WAS existent. So insufficient power of Z test noted here. 95% CI = .19 to 3.32 Yes 95% CI = .31 to 3.57 Yes

Cross-level and upper-level mediation model [2→(1 & 2) →1]
Model A Model B Mediator CLUSTER γ’02 γ01 Mediator CLUSTER Treatment CLUSTER Treatment CLUSTER γ’01 Outcome CLUSTER Mediator Avg M Mediator Treatment Treatment Outcome Mediator INDIVIDUAL Mediator INDIVIDUAL γ’10 Our last model. The prior model, we were looking at a child level measure as a mediator. We saw it as totally at the individual component level – kids vary in knowledge and that affects their behavior. But, we might also think that there might be a synergistic effect – that if you have a bunch of kids who are all unknowledgable, then that’s a total mess… or all knowledgable, then maybe its like Stepford wives (with 3 year olds)… It is the same model as before on the A side. Slightly different model on the B side. For the B paths, I’ve added an average medaitor at the SCHOOL elevel – you have an important decision to make about level 1 centering of the mediator. Could use group or grand mean centering… we argue using grand mean centering is more useful for interpretation… We will have two b paths, 2 indirect effects, parsing the IE into two pieces: individual and contextual effects. Outcome INDIVIDUAL

Cross-level and upper-level mediation model: Results
Path a is the same as in the prior model. For the b and c’ paths: Final estimation of fixed effects: Standard Approx. Fixed Effect Coefficient Error T-ratio d.f. P-value For INTRCPT1, B0 INTRCPT2, G T, G M2_AVE, G For M2 slope, B1 INTRCPT2, G Path a was the same, so am not showing it. For Path b…. LISTEN TO TALK FOR HLM. B path at the individual level is .600 (similar to last time with model was .592) B path at the cluster level is avareage mediator value related to average outcome, controlling for individual effects. Value is Therefore classroom “knowledge” doesn’t mediate treatment’s effect over and above individual’s.

Cross-level and upper-level mediation model [2→(1 & 2) →1]
Model A Model B Mediator CLUSTER -.041 Mediator CLUSTER 2.643 Treatment CLUSTER Treatment CLUSTER 2.761 Outcome CLUSTER Mediator Avg M Mediator Treatment Treatment Outcome Mediator INDIVIDUAL Mediator INDIVIDUAL .600 Outcome INDIVIDUAL Direct effect WAS – went up a little with better specified path included (that, or had to maintain same total effect value). If I had used group mean centering of the mediator in Model B, note that path b at the individual level does not change (nor does its SE), but the path B at the cluster level changes… from to Path B at the cluster level includes BOTH indivudal and cluster effects on the outcome variable. This value (.559) is the sum of the indivudal (.600 and the contextual effect -.041)… = .559. They are equivalent models, just different in interpretation. Much more straightfoward to use grand mean centering so you get a test of the contextual effect. abind = (2.643)(.600) = 1.586 abcluster = (2.643)(-.041) = -.109 Total indirect effect = – = 1.477 Total effect = = 4.238

Cross-level and upper-level mediation model [2→(1 & 2) →1] Group-mean centered M
Model A Model B Mediator CLUSTER 0.559 Mediator CLUSTER 2.643 Treatment CLUSTER Treatment CLUSTER 2.761 Outcome CLUSTER Mediator Avg M Mediator Treatment Treatment Outcome Mediator INDIVIDUAL Mediator INDIVIDUAL .600 Outcome INDIVIDUAL Direct effect WAS – went up a little with better specified path included (that, or had to maintain same total effect value). If I had used group mean centering of the mediator in Model B, note that path b at the individual level does not change (nor does its SE), but the path B at the cluster level changes… from to Path B at the cluster level includes BOTH indivudal and cluster effects on the outcome variable. This value (.559) is the sum of the indivudal (.600 and the contextual effect -.041)… = .559. They are equivalent models, just different in interpretation. Much more straightfoward to use grand mean centering so you get a test of the contextual effect. If the level one predictor had been group-mean centered, then the L2 path would have been not This path would be interpreted as the sum of the average individual and contextual effects of M. Under grand-mean centering, the path represents the unique contextual effect.

Cross- and upper-level mediation model: Results at the individual level
Causal steps approach Test of joint significance z test of ab product Empirical-M test BC parametric bootstrap Steps 1, 2 and 3 significant Yes Paths a and b significant Yes se=.886, z=1.79, p= % CI = -.15 to 3.32 No Only did it at the individual level since the cluster-level effect was so small it didn’t seem worth it but do do it to be rigorous 95% CI = .19 to 3.44 Yes ? Not yet programmed

Brief review of advanced issues
Multisite / randomized blocks (1→1 →1) More complicated! Testing mediation in 3-level models Including multiple mediators Examining moderated mediation Dichotomous or polytomous outcomes Measurement error in mediation models Reading list for each of these topics See pituch, murphy & tate (2010) article on three-level models. Turn circles into latent variable models.

Notes on software HLM,SPSS Plug results into PRODCLIN SAS (PROC MIXED)
See handout Can use Stapleton’s macros for bootstrapping MLwiN, MPlus Have limited bootstrapping capacity but still have to summarize results SEM software Provide test of  but using Sobel. (If sig with Sobel then will be sig with others)

Mediation Models Laura Stapleton UMBC

Similar presentations

Presentation on theme: "Mediation Models Laura Stapleton UMBC"— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

Mediation Models Laura Stapleton UMBC

Similar presentations

Presentation on theme: "Mediation Models Laura Stapleton UMBC"— Presentation transcript:

Similar presentations

About project

Feedback