Confirmatory Bifactor modelling Philip Hyland philipehyland@gmail.com University of Ulster

Lecture Outline Describe the nature of bifactor modelling Describe issues that arise in conceptualizing and modelling multidimensionality, Outline confirmatory bifactor modelling, Differentiate between bifactor and second-order models, and Describe strengths and limitations associated with bifactor modelling.

Dimensionality Many psychological measures are constructed to capture a single latent variable of interest (e.g. depression, anxiety, self-esteem). Factor analytic studies often reveal evidence of both a general factor and multidimensionality. When subjected to CFA, a scale almost always displays a multidimensional structure.

Dimensionality Unsurprising given the challenges inherent in writing a set of scale items that attempt to; Measure a single latent variable of interest but are not entirely redundant (i.e., the same question asked over and over), Are heterogeneous enough to validly represent the diverse manifestations of the construct, and Provide acceptable reliability.

Dimensionality Need to create a heterogeneous set of items to appropriately capture the entire breath of a given construct (e.g. psychological and psychological components of anxiety). Puts researchers in a difficult position: Measure one thing while simultaneously measuring diverse aspects of this same thing.

Dimensionality Not surprising to find conflicting evidence of unidemsionality and multidimensionality for certain psychological measures. Leads to debates - measuring a single construct or a variety of distinct, yet related, constructs? Bifactor modelling in its simplest form offers an appealing solution to these kinds of problems.

Unidimensional MODEL Underlying structure of a measure are usually assessed according to three general forms. The unidemsional model Each item is influenced by a single common factor and a uniqueness term that reflects both systematic and random error components.

Unidimensional MODEL

Unidimensional MODEL Often the hoped for structure. Summed scores on the individual indicators provide a clear indication of individual differences on the latent variable of interest (e.g. depression, self- esteem etc.). Variation in each item on the scale is determined by levels of the general latent variable of interest.

Unidimensional MODEL Even when scales are constructed to capture a single psychological construct unidimensional solutions are rarely identified. Necessary then to find alternative multidimensional model structures. The second type of structural model is the correlated traits model.

Correlated traits model

Correlated traits model Latent variable of interest is separated into its component parts. Components usually considered to be related to varying degrees. No attempt to measure a single common construct (e.g. PTSD) – rather the various components of PTSD are being measured. Multiple latent variables are correlated - no attempt made to place a measurement structure on the correlated latent factors.

Higher order model The third type of structural model normally considered is a higher order model

Higher order model Includes a single common source of variation – unlike correlated traits model Correlations between the first order factors are explained in terms of a higher order factor Latent variable of interest is being modelled but at a more abstract or distal level then in the unidimensional model. No direct relationship between the latent variable of interest (PTSD) and each of the indicators of PTSD. Rather there is an indirect pathway mediated by the first order traits.

BIFACTOR MODEL An alternative model conceptualisation exists but is rarely applied in the personality and psychopathology domains. This type of model is the bifactor model.

BIFACTOR MODEL Variation among individual indicators is assumed to arise for a number of sources. In a simple form - all indicators are modelled to load onto a single common latent factor – the latent variable of interest (PTSD). But, item covariation can also arise due to what are called ‘group factors’ or in some models ‘nuisance factors’.

BIFACTOR MODEL Bifactor models include two or more uncorrelated (orthogonal) group/nuisance factors. Each indicator is allowed to load onto two distinct latent variables: A general factor (e.g. PTSD) and one (and only one!) group/nuisance factor (e.g. Intrusions). Covariation is caused by a group of items tapping similar aspects of the trait (e.g. items measuring symptoms of Hyperarousal).

Factor Analysis Appearances of multidimensionality can be the result of heterogeneous item content A given set of indicator are primarily attempting to measure the latent variable of interest but they are also purposefully measuring another factor (this is termed a ‘grouping factor’), or A given set of indicators are primarily attempting to measure the latent variable of interest but they inadvertently measuring another factor (this is termed a ‘nuisance factor),

BIFACTOR or higher order Bifactor model allows researchers to retain the idea of a single common construct (PTSD) while also recognising multidimensionality (Intrusions etc.) A useful alternative to the traditionally employed higher order model. In many ways the bifactor model and the higher-order model are very similar but there are a few important differences that are worth recognising.

BIFACTOR or higher order Higher-order models are frequently used - bifactor models are not. Major difference lies in how multidimensionality is conceptualised. In “correlated traits” models assumption that the observed item covariation is the result of two or more correlated primary factors. Higher order model - latent variable of interest (PTSD) is determined by what a set of primary latent variables have in common not what observable indicators have in common.

BIFACTOR MODEL Bifactor models - latent variable(s) of interest explains some proportion of covariation among observable indicators Other group/nuisance factors explain additional covariation among observable indicators. The latent variable of interest and group/nuisance factors are therefore on equal conceptual footing and compete for explaining item covariance—neither factor “higher” or “lower” than the other.

BIFACTOR MODEL In contrast to the higher-order model conceptualisation, in a bifactor model conceptualisation the latent variable(s) of interest is measured by what is common among the observable indicators. This means that variations in the intensity of the specific latent variable of interest will directly influence levels of the observable indicators rather than indirectly, as in the case with the higher-order model.

Applications A bifactor modelling approach can be used to address important issues that routinely arise in psychometric analysis of personality and psychopathology measures. Bifactor modelling is useful for: Evaluating the plausibility of subscales, Determining the degree to which sum scores reflect a single factor, and Evaluating the feasibility of applying a unidimensional model structure to a measure with heterogeneous indicators.

Subscales Researchers often argue about the usefulness of creating subscales. In many cases where a single general factor is broken down into subscales these traits are highly correlated - multicollinearity becomes a problem. Our ability to determine the unique contribution of each of the subscales in predicting some important outcome is severely compromised.

Subscales A second reason often advanced for creating subscales is that the subscales may have differential correlations with external variables. Technically true but weak justification for “cutting up” a measure. Any two items not perfectly correlated potentially have different correlations with external variables. However it makes little sense to argue that one should investigate external correlates for each item separately.

Subscales A third and very seldom recognised problem with creating subscales emerges from a bifactor modelling perspective. Subscale scores reflect variation on both a general factor and a more specific group/nuisance factor. The effect of this is that the subscale may appear to be reliable, but in fact, that reliability is a function of the general trait, not the specific group factor.

Subscales Finally, it has also been noted that subscales are often so unreliable compared to the composite score that the composite score is actually a better predictor of an individual’s true score on a subscale than is the subscale score itself. Sinharay and Puhan (2007) have therefore argued that subscale scores are seldom, if ever, empirically justified. How can bifactor models help?

Subscales Provides a method for determining the usefulness of creating subscales. Because the general and group/nuisance factors are uncorrelated in a bifactor model, inspection of the factor loadings on the general and group/nuisance factors is informative. If factor loadings are high on the general factor and low on the group/nuisance factors - makes little sense to create subscales. If factor loadings are high for both the general factor and the group/nuisance factors - subscale creation should be considered.

Applications Reise, Moore, and Haviland (2010) - main problem in traditional psychometric evaluation of scales is that the wrong default model is used. A unidimensional model is normally used as the default model and is also frequently a researcher’s ideal solution. Yet, item response data drawn from complex measures are rarely, if ever, strictly unidimensional.

Applications In a way it’s also not desirable to achieve such a solution To achieve unidimensionality one essentially has to write a set of items with very narrow conceptual bandwidth (i.e., the same item written over and over in slightly different ways) Results in poor predictive power or theoretical usefulness. Reise et al. (2010) argue that a bifactor model, which contains a single common trait but also allows for multidimensionality due to item content diversity, provides a better foundational model for conceptualizing dimensionality.

Strengths Ability to maintain a unidimensional conceptualisation while also recognising issues of multidimensionality. What else? So far we have focused on the role on grouping factors. Factors that are present within a scale that arise from intentionally attempting to capture complexities of a particular psychological construct (e.g. the cognitive-affective and somatic-performance components of depression). But, what about nuisance factors?

Strengths Factors that are present within a scale which arise unintentionally when attempting to capture the primary latent variable(s) of interest. When unwanted nuisance factors are present within a scale the fit of a theoretically derived model may be affected. Bifactor models allow researchers to model (control for) these unwanted nuisance factors Allows for a clearer assessment of the adequacy of theoretically derived models.

Limitations The general and group/nuisance factor must be specified to be uncorrelated. What does it mean to propose the existence of a group factor such as Intrusions that partially reflect PTSD but which also has a specific component that is independent of PTSD? Many researchers are therefore sceptical that the model itself makes any sense. Highly restrictive assumptions which need to satisfied in order that group/nuisance factors will be identified. Data must be multidimensional but also that the multidimensionality be well structured - each item measures a general factor and one, and only one, group/nuisance factor.

Conclusion Many psychological scales are constructed to measure a single latent variable of interest (e.g. depression, self- esteem). However due to item heterogeneity necessary to capture the subtleties and complexities of a psych construct scales invariably display signs of multidimensionality. Correlated trait models and higher-order models are traditionally applied in order to model the underlying structure of such scales.

Conclusion Bifactor models provide a plausible and useful alternative to traditionally employed higher-order model in order to maintain a unidimensional structure consistent with the theoretical foundation upon which the measure was constructed while also taking into account important item grouping factors. Bifactor modelling also allows researchers to model (control for) unwanted/unintended/meaningless nuisance factors which are preventing theoretically plausible models from achieving appropriate model fit. A number of important strengths and limitations should be considered when deciding whether or not to conduct a bifactor analysis.

Confirmatory Bifactor modelling HOW TO CARRY OUT IN Mplus

Example We will be further examining the factor structure of the Measure of Criminal Social Identity (Boduszek et al., 2012). 8 Items designed to measure Criminal Social Identity – Includes three grouping factors. Cognitive Centrality In-Group Affect In Group Ties

Saving data for use in mplus We will be using the data set entitled ‘Criminal Identity’ Unlike SPSS, Mplus does not allow you to use drop-down commands to estimate the model - you must write the syntax yourself (don’t panic!). It is a good idea to create a shorter data set yourself for your specific analysis in Mplus.

Saving data for use in mplus Mplus cannot directly read an SPSS file. Mplus can easily read Tab delimited data, so we can save our dataset as a .dat file. This can be done by choosing File, Save as. Be sure to untick the box “Write Variable Names to Spreadsheet” We will save the variable names quickly from SPSS by copying them from the Variable View window and pasting them into a new text editor or directly into an Mplus input file. Ready to open a new Mplus window and start writing syntax.

Mplus syntax for bifactor

Saving data for use in mplus First we have to provide a TITLE for our analysis (Criminal Social Identity Bifactor Model) To read our DATA we indicate the location of the .dat file we saved. Under the VARIABLE heading after ‘names are’ you paste in your variable names from your SPSS data set. In the next line, we indicate which values should be considered missing in each variable. In our example missing are all (-9).

Saving data for use in mplus In USEVAR enter those variables which are to be used for the current analysis. The CATEGORICAL option is used to specify which variables are treated as binary or ordered categorical (ordinal) variables in the model and its estimation. Under the ANALYSIS heading we must indicate what ESTIMATOR we will be using.

Saving data for use in mplus Because our observed variables are measured on 5- point Likert scale we will use Robust Maximum Likelihood (MLR) estimation. If your observed variables are categorical use Estimator = WLSMV

Saving data for use in mplus Because our observed variables are measured on 5-point Likert scale we will use Robust Maximum Likelihood (MLR) estimation. If your observed variables are categorical use Estimator = WLSMV Under the MODEL command we must specify the bifactor model This is the place where you have to create your latent variables (seven factors in this example – four general factors and three nuisance factors). In bifactor modelling, like CFA, we use the command “by” to create latent variables.

Saving data for use in mplus The latent variable “CSI” is measured by all 8 items CI1C* CI2C CI3C CI4IA CI5IA CI6IT CI7IT CI8IT For each latent variable created the first indicator must be followed by an astrix (*) This sets the variance of each Latent Variable to be 1 – Allows for model identification and latent variable scaling. We then must indicate.... CSI@1; C@1; A@1; T@1;

Saving data for use in mplus The group factors must all be uncorrelated with the general factors. We need to provide Mplus with this command CSI with C-T@0; The grouping factors must also be uncorrelated with each other C with A-T@0; A with T@0; If you have more than 1 latent variable of interest these are allowed to correlate.

Saving data for use in mplus Once you have created syntax for confirmatory factor analysis press to run the model. Save this as an input file under some name e.g., CSI bifactor model.inp in the same folder as the criminal identity.dat file. This produces a text output (.out) file stored in the working directory with the results. For this model the output file looks like the following:

Mplus output bifactor

Mplus output bifactor The first part of the output provides a summary of the analysis including: The number of groups (1) The number of observations (participants included in the analysis, N = 303) The number of items included in the confirmatory model (number of dependent variables = 8) The number of latent variables (4). Furthermore, Mplus gives more info which you do not need to report except what Estimator was used (in this example it was MLR= robust maximum likelihood).

Mplus output bifactor The next step is to investigate how well the model fits the data. This model of CSI was specified and estimated in Mplus as a restricted bifactor solution. Before we look at the factor structure we have to assess the fit between the data and pre-established theoretical model. Goodness-of-fit indices are used to assess model fit – Same as used in EFA and CFA.

Mplus output bifactor

Mplus output bifactor This bifactor model represents a superior fitting model compared to all other tested models We are interested in more details about this model. Mplus output provides lots of information however you are interested only in few of them. Unstandardized Factor Loadings and Standard Errors Standardized Factor Loadings and Significance Levels Factor Correlations.

Unstandardized/S.E.

Standardized loadings 1

Mplus output bifactor You should inspect the strength of the factor loadings for the psychological factors relative to the grouping factors. If strong on grouping factors – consider creating subscales. If weak or weaker than the general factor – consider unidimensionality. Also you would be interested in the factor correlations for the psychological factors – if 2 or more.