Measurement and Modeling of Change: Just some of the issues Todd D. Little Yale University (for now…) Auburn University, May 8, 2001

Outline Multilevel analyses –(aka HLM; special case of SEM) Selecting Indicators –Parceling –Finding optimal sets Selecting Reporters –The case of aggression Missingness, Dropout, and Selectivity –What’s the diff? Simple Longitudinal Modeling

Multilevel Structures Observations at one level are nested within observations at another Number of levels theoretically limitless, bounded by practicality Examples: –Students within classrooms, grade-level, gender Between subjects designs –Persons within time of measurement Within subjects designs

"Once you know that hierarchies exist, you see them everywhere." "Once you know that hierarchies exist, you see them everywhere." -Kreft and de Leeuw ( 1998 )

Multilevel Approaches Distinguish HLM (a specific program) from hierarchical linear modeling, the technique –A generic term for a type of analysis Probably best to discuss MRC(M) Modeling –Multilevel Random Coefficient Modeling Different program implementations –HLM, MLn, SAS, BMDP, LISREL, and others

Logic of MRCM Coefficients describing level 1 phenomena are estimated within each level 2 unit (e.g., individual- level effects) –Intercepts—means –Slopes—covariance/regression coefficients Level 1 coefficients are also analyzed at level 2 (e.g., dyad-level effects) –Intercepts: mean effect of dyad –Slopes: effects of dyad-level predictors

Features of Multilevel Analyses Parameter estimates incorporate effects across the hierarchies Analyze all basic phenomena (means, variances and covariances) at multiple levels simultaneously –Relationships (covariances) can differ across levels of analysis Psuedo-Variants include MACS models in SEM when Level 2 N is small and two-stage modeling

Negative Individual, Positive Group

Positive Individual, Negative Group

No Individual, Positive Group

No Group, Mixed Individual

A Contrived Example Y ij = Friendship Closeness ratings of each individual i within each dyad j. Level 1 Measures: Age & Social Skill of the individual participants Level 2 Measures: Length of Friendship & Gender Composition of Friendship dyad

The Equations y ij =  0j +  1j Age +  2j SocSkill +  3j Age*Skill + r ij The Level 1 Equation:  0j =  00 +  01 (Time) +  02 (Gnd) +  03 (Time*Gnd) + u 0j  1j =  10 +  11 (Time) +  12 (Gnd) +  13 (Time*Gnd) + u 1j  2j =  20 +  21 (Time) +  22 (Gnd) +  23 (Time*Gnd) + u 2j  3j =  30 +  31 (Time) +  32 (Gnd) +  33 (Time*Gnd) + u 3j The Level 2 Equations: =

Types of Constructs Multidimensionality Dirtyness Degree of Complexity

Construct Space

… and a centroid

Selecting Indicators

Elements of the Simulation Communality Axis of the Construct's Centroid Maximum Reliability of an indicator (1.0) Selection Diversity Selection Planes

Indicators of the Construct

Selecting Three (at random)

Latent Construct Centroid

Construct Trueness TrTr TaTa

… and another

… yet another

Selecting Six (Three Pairs)

… take the mean

… and find the centroid

How about 3 pairs of 3?

Centroid of Yellow

Centroid of Red

Centroid of Orange

… taking the means

… yields reliable & valid indictors

Construct Specific (but reliable) Random Error Variance Components

+= Aggregate the 2 Indicators

+= Standardizing the Scale

A Word (picture?) of Caution +=+ +=+ +=+ } 50% What is this Construct?

Absolute Trueness Values Indicators 95% Max 95% Max 95% Max Low DiversityMedium DiversityHigh Diversity Note. These values are symmetric. The degree of possible bias between any two constructs would be 2 times the tabled values

Finding Optimally Efficient Indicators

Implications of Selection Confirmatory Analyses are very good. – Little or no evidence of bias – Doesn't overcorrect for measurement error You can have validity without reliability – Hard to argue, but possible – Need very good theory Don't "just do it," think about it first… – Facilitates making better selections – Avoids the allure of the 'bloated specific'

How To Find Them? Focus on Construct Space! Not item space. Focus on Construct Space! Not item space. – 1) Select a broad set of constructs Some with Small, Medium, & Large Positive Correlations Some with Small, Medium, & Large Positive Correlations Some with Small, Medium, & Large Negative Correlations Some with Small, Medium, & Large Negative Correlations Some that are zero Correlated Some that are zero Correlated – 2) Calculate Latent Correlations on whole sample (save) – 3) Split sample into two random halves – 4) Find Optimal Set on 1 st Half of Sample Systematically select all possible combinations of n items from the original item pool Systematically select all possible combinations of n items from the original item pool Determine the best set that reproduces whole sample correlations Determine the best set that reproduces whole sample correlations – 5) Cross-validate on Second Half – 6) Repeat by generating on 2 nd half and cross-validating.

Inter-Reporter Relations Self Friend Peer Teacher Parent O R O R O R O R O R

Overt Reactive Overt Instrumental Relational Reactive Relational Instrumental Overt (Dispositional) Relational (Dispositional) A Unifying Model of Aggression

ReactiveInstrumentalReactiveInstrumental Overt (Dispositional) Relational (Dispositional) A Unifying Model of Aggression

ReactiveInstrumentalReactiveInstrumental Overt (Dispositional) Relational (Dispositional) ReactiveInstrumental A Unifying Model of Aggression

Overt (Dispositional) Relational (Dispositional) ReactiveInstrumental A Unifying Model of Aggression

Reactively Aggressive Instrumentally Aggressive Neither Both Primarily Instrumental Primarily Reactive ‘Typical’ range Sub-types of Aggression Based on Function

Dropout: Random Process Time 1 Time 2 Time 3 Time 2 Time 1 Time 3 Time 2 Time 1

Time 2 Dropout: Functionally Random Time 1 Time 2 Time 3 Time 1 Time 3 SelectiveInfluence R = 0

Dropout: Selective Process(es) Time 1 Time 2 Time 3 Time 1 Time 2 Time 3 Survival Analysis Dropout Analysis SelectiveInfluence R = ? Time 3 Time 2

Time and Intervals Age in years, months, days. Experiential time: Amount of time something is experienced – Years of schooling, Length of relationship, Amount of practice – Calibrate on beginning of event, measure time experienced Episodic time: Time of onset of a life event – Toilet trained, driver license, puberty, birth of child, retirement – Early onset, on-time, late onset: used to classify or calibrate. – Time since onset or time from normative or expected occurance. Measurement Intervals – How fast is the developmental process? – Intervals must be equal to or less than expected processes of change – Cyclical processes E.g., schooling studies at yearly intervals vs half-year intervals

Some Measurement Features Operationally define Constructs as precisely as possible – Reduces alternative outcome interpretations – Increases translate-ability – Uni- vs Multidimensional constructs Use multiple indicators of each construct – Triangulates measurement to assess validity and correct for unreliability Use large samples – Increases power – Allows sophisticated and powerful analyses Screen data – Estimate missing values, transform non-normal distributions, identify and fix outliers – Increases power and reduces spurious conclusions

Some Design Features Use Multiple Constructs – Assists in validity assessment – Can show what a construct is, as well as what it is not Specify Competing Hypotheses – Strengthens a theoretical position by demonstrating that some hypotheses are rejectable while others are not Emphasize Confirmatory Designs – Encourages careful theorizing – Minimizes capitalization on chance – Strengthens theoretical position When using Exploratory Approaches – Use cross-validation techniques – Interpret borderline effects carefully – Focus on effect size rather than significance

Some Developmental Truisms Homotypic vs heterotypic expressions – E.g., Aggression Surface-structure vs deep-structure roots of behavior – E.g., resource-directed behavior Different paths can lead to same outcome Same path can lead to different outcomes Development is both Qualitative & Quantitative – Light is both wave and particle…

One construct -- Four Occasions Time 1 Time 2 Time 3 Time 4 e1e1 e2e2 e3e3 e1e1 e2e2 e3e3 e1e1 e2e2 e3e3 e1e1 e2e2 e3e3 Equating the reliable measurement parameters e 4, 5, 6 e 7, 8, 9 e 10, 11, 12

A Longitudinal Simplex Structure T1 T2 T3 T4 Time 1 Time 2 Time 3 Time 4

Longitudinal Structures Model Time 1 Time 2 Time 3 Time 4 1111      21 =  21,  32 =  32,  43 =  43  31 =  21  32  41 =  21  32  43 In standardized solution, the correlations are reproduced by tracing the paths:  42 =  32  43 e1e1 e2e2 e3e3 e1e1 e2e2 e3e3 e1e1 e2e2 e3e3 e1e1 e2e2 e3e3 e 4, 5, 6 e 7, 8, 9 e 10, 11, 12.8

Identification & Scale Setting 1 Indicator 0.0* e 1.0* 2 Indicators ee e = 1.0* 3 Indicators eee e e e 1.0* Var1 Corr1,2 Var2 Var1 C12 Var2 C13 C23 Var3 Useable Information:

Figural Conventions AB en Circles and Ovals represent Latent Constructs Boxes & Rectangles represent manifest (measured) variables Single-headed lines are directional (causal) relationships represented as regression estimates Double-headed lines are non-directional (co- occurring) relationships represented as covariance or variance estimates

LISREL Conventions Mn(row, column); mn(end, start) e.g. 3 from 2 When drawing, number top to bottom then left to right

1.0* e e e e e e e e e e e e e DA NG=1 NI=6 ME=ML NO=100 KM FI=OUT_THERE.DAT SD FI=OUT_THERE.DAT ME FI=OUT_THERE.DAT MO ny=6 ne=2 ly=fu,fi te=sy,fi ps=sy,fi !note ly=ny,ne ly(indicator,construct) Fr ly(1,1) ly(2,1) ly(3,1) Fr te(1,1) te(2,2) te(3,3) !note: te=ny,ny ty(indicator,indicator) Fr ly(4,2) ly(5,2) ly(6,2) Fr te(4,4) te(5,5) te(6,6) !note: ps=ne,ne ps(construct,construct) VA 1.0 ps(1,1) ps(2,2) Fr ps(2,1) OU so ad=off rs sc Simple Confirmatory Factor Model

* e e e e e e e e e e e e e * e e e e e e e e e e e e e A Longitudinal Confirmatory Factor Model e e e e

DA NG=1 NI=12 ME=ML NO=100 KM FI=OUT_THERE_long.DAT SD FI=OUT_THERE_long.DAT ME FI=OUT_THERE_long.DAT MO ny=12 ne=4 ly=fu,fi te=sy,fi ps=sy,fi be=ze Fr ly(1,1) ly(2,1) ly(3,1) ly(4,2) ly(5,2) ly(6,2) Fr ly(7,3) ly(8,3) ly(9,3) ly(10,4) ly(11,4) ly(12,4) Fr te(7,7) te(8,8) te(9,9) te(10,10) te(11,11) te(12,12) Fr te(1,1) te(2,2) te(3,3) te(4,4) te(5,5) te(6,6) FR te(7,1) te(8,2) te(9,3) te(10,4) te(11,5) te(12,6) VA 1.0 ps(1,1) ps(2,2) ps(3,3) ps(4,4) fr ps(2,1) Fr ps(4,3) Fr ps(3,1) ps(4,1) ps(3,2) ps(4,2) Ou so ad=off rs sc LISREL Source Code

e e e e= e e e e e e e= * e e e e e e e e e e e e e A Longitudinal Confirmatory Factor Model e e e e

DA NG=1 NI=12 ME=ML NO=100 KM FI=OUT_THERE_long.DAT SD FI=OUT_THERE_long.DAT ME FI=OUT_THERE_long.DAT MO ny=12 ne=4 ly=fu,fi te=sy,fi ps=sy,fi Fr ly(1,1) ly(2,1) ly(3,1) ly(4,2) ly(5,2) ly(6,2) Fr ly(7,3) ly(8,3) ly(9,3) ly(10,4) ly(11,4) ly(12,4) Fr te(1,1) te(2,2) te(3,3) te(4,4) te(5,5) te(6,6) Fr te(7,7) te(8,8) te(9,9) te(10,10) te(11,11) te(12,12) VA 1.0 ps(1,1) ps(2,2) !ps(3,4) ps(4,4) fr ps(2,1) Fr ps(4,3) Fr ps(3,1) ps(4,1) ps(3,2) ps(4,2) LISREL Source Code: Part 1

EQ ly(1,1) ly(7,3) Eq ly(2,1) ly(8,3) Eq ly(3,1) ly(9,3) Eq ly(4,2) ly(10,4) Eq ly(5,2) ly(11,4) Eq ly(6,2) ly(12,4) Fr ps(3,3) ps(4,4) Ou so ad=off sc rs LISREL Source Code: Part 2

e e e e= e e e e e e e= * e e e e e e e e e e e e e A Longitudinal Cross-lag Structural Model e e e e

DA NG=1 NI=12 ME=ML NO=100 KM FI=OUT_THERE_long.DAT SD FI=OUT_THERE_long.DAT ME FI=OUT_THERE_long.DAT MO ny=12 ne=4 ly=fu,fi te=sy,fi ps=sy,fi be=fu,fi Fr ly(1,1) ly(2,1) ly(3,1) ly(4,2) ly(5,2) ly(6,2) Fr ly(7,3) ly(8,3) ly(9,3) ly(10,4) ly(11,4) ly(12,4) Fr te(1,1) te(2,2) te(3,3) te(4,4) te(5,5) te(6,6) Fr te(7,7) te(8,8) te(9,9) te(10,10) te(11,11) te(12,12) VA 1.0 ps(1,1) ps(2,2) fr ps(2,1) Fr ps(3,3) ps(4,4) Fr ps(4,3) !note: be=ne,ne be(construct,construct) (to,from) (row,column) Fr be(3,1) be(4,1) be(3,2) be(4,2) LISREL Source Code: Part 1