Paper presented at the American Evaluation Association 2010 conference Evaluating potential impact of intervention in community settings when no comparison data is available: mixture latent growth modeling for exploring differential change in female condom use Paper presented at the American Evaluation Association 2010 conference Maryann Abbot1, MA; Emil Coman2A Ph.D., & Peg Weeks3 Ph.D. 1: Project Manager; 2: Senior Research Scientist-Evaluator,& 3: Executive Director Institute for Community Research, Hartford, CT USA A: acknowledged indirect effect from Dave Kenny, his SEM mentor. DISCLAIMER: All error found here were totally mediated by 2nd author!

Statement of problem Evaluation of intervention ‘effects’ in one-group pre-post test design (no controls) is challenging Structural equation modeling (SEM) or covariance structure analysis (CSA) with latent grouping variable (mixture ‘regression’1, 2) is an option The question of causality cannot be answered definitely however BUT we can describe ‘what happened’ and try to understand why Jo, B., & Muthén, B. (2000). Intervention studies with noncompliance: Complier average causal effect estimation in growth mixture modeling. In N. Duan & S. Reise (Eds.), Multilevel Modeling: Methodological advances, issues, and applications (pp. 112-139). Mahwah, NJ: Lawrence Erlbaum Associates. Tarpey, T., & Petkova, E. (2010). Latent regression analysis. Statistical Modelling, 10(2), 133.

‘what happened/does it seem to work’? The challenge Lack of a comparable group change data Few studies ongoing on Female Condom (FC) use Host of historical/context effects Primarily new marketing campaign recently launched for FC 2 Other HIV prevention local interventions, not necessarily focused on FC however We want answer to the question: ‘what happened/does it seem to work’? i.e. has the intervention changed FC use and attitudes in any of the participants; who are those who have not changed, what seem to make the difference?

The Hartford FC study Intervention conducted in Hartford, CT aimed at increasing awareness and use of the female condom (FC) as a women-initiated HIV and sexually transmitted infections (STI) prevention method Prospective, longitudinal cohort study of 461 women (baseline, 1 & 10 months) We provided a brief knowledge/skills demonstration to all women at baseline and wave 2: included demonstrating proper FC use (using their hands as a model to show proper insertion and removal) explaining common problems with FC insertion/use and possible solutions answered questions

The Hartford FC study Designed to to reduce the primary initial barriers to FC adoption, namely, the lack of information, demonstration, community availability, and troubleshooting support for FC use Baseline (wave 1: n = 461), wave 2 (n = 392), & wave 3 (n = 245); common sample across time points of 228 participants 55% African/Black American, 30% Puerto Rican/Other Latino, 14% White/Other 71%, 22%, and 20% at waves 1, 2, and 3 respectively have not used FC in the past 30 days

SEM reminder Models are specified by replacing correlations with causal paths (one-directional), or deleting them (forcing them equal to zero). For mixture analyses - specify a latent categorical variable: can be both cause and effect of other variables, or of certain parameters (like slopes of growth curves). MODEL: %OVERALL% i s | fctimes1@0 fctimes2@1 fctimes3@2 ; Fit for mixture models is relative, based on AIC, BIC, entropy.

Methods Mixture growth modeling1 method of identifying latent classes (unobserved groupings) in the sample analyzed Purpose: to identify latent classes of potential and actual FC users who exhibit different growth trajectories in terms of their past 30 days FC (female condom) use Similar to say Segawa et al. (2005) but with no control group Li, F., Duncan, T. E., Duncan, S. C., & Acock, A. (2001). Latent growth modeling of longitudinal data: A finite growth mixture modeling approach. Structural Equation Modeling: A Multidisciplinary Journal, 8(4), 493-530. Segawa, E., Ngwe, J., Li, Y., & Flay, B. (2005). Aban Aya Coinvestigators (2005). Evaluation of the effects of the Aban Aya Youth Project in reducing violence among African American Adolescent males using latent class growth mixture modeling techniques. Evaluation review, 29(2), 128-148.

Growth Mixture Latent Classes Intercept Slope Times FC use 1 2 1 Latent Classes Intercept Slope ?

Overall results FC use - percentages

Overall results FC use - percentages

SEM results Based on classification entropy and AIC and BIC fit indices, the best solution was deemed a three latent classes grouping for the no-covariate times used FC GMM mixture model It divided up the sample into 109 fast increasing use participants, 93 moderate increase Ps, and 26 consistent (stable) Ps. The intervention seems to have been very successful for 48% of the sample, somewhat successful for another 41%, while not impacting the rest of 11% participants.

More results They had slopes of .399 (p<.001), .103 (p=.021), and .008 (p=.831). The intercepts for the groups were -1.145, 1.375, and 2.122. The groups differ significantly in terms of Times used FC at waves 2 and 3 FC stage of use at all three waves FC beliefs at all three waves FC knowledge at wave 3

Growth Mixed Modeling results

Mixture GLM results ‘Stage of FC use 1 = never thought about using; 2 = thought about or got but didn’t use ; 3 = tried but decided not to use again ; 4 = use FC occasionally 5 = use FC regularly or as primary

Mixture GLM results FC beliefs 4= strongly favorable ; 3= favorable; 2= unfavorable ; 1= strongly unfavorable

Mixture GLM results FC knowledge Average knowledge score 0 = incorrect; 1 = correct

Additional problems Mixture modeling with vs. without covariates have advantages/disadvantages; covariates may separation between classes1. Growth models have alternative specifications possible: correlation between slope and intercept growth parameters zero/not. Slopes set to common 0,2,3, etc may need to follow real time lags (baseline, 1 month, 10 month interview) “Recovered’ classes may not always represent meaningful groups 1. Lubke, G. (2010). Latent Variable Mixture Models. In Hancock, G., & Mueller, R. The Reviewer's Guide to Quantitative Methods in the Social Sciences: Routledge.(p. 209-219).

Conclusions There seems to be in our sample of female participants three distinct groups of actual and potential FC users. Their FC use, FC beliefs, FC knowledge, and stage of FC use differ, so we might conclude these are distinct groups that need to targeted by differential (tailored) intervention strategies for FC adoption/increased use. Additional explorations should be done on other demographic but more importantly relationship variables.

Thank you! http://evaluationhelp.wordpress.com www.icrweb.com ---