2. Measurement and Modeling of Change: Just some of the issues Todd D. Little Yale University (for now…) Auburn University, May 8, 2001

3. Outline Multilevel analyses –(aka HLM; special case of SEM) Selecting Indicators –Parceling –Finding optimal sets Selecting Reporters –The case of aggression Missingness, Dropout, and Selectivity –What’s the diff? Simple Longitudinal Modeling

4. Multilevel Structures Observations at one level are nested within observations at another Number of levels theoretically limitless, bounded by practicality Examples: –Students within classrooms, grade-level, gender Between subjects designs –Persons within time of measurement Within subjects designs

5. "Once you know that hierarchies exist, you see them everywhere." "Once you know that hierarchies exist, you see them everywhere." -Kreft and de Leeuw ( 1998 )

6. Multilevel Approaches Distinguish HLM (a specific program) from hierarchical linear modeling, the technique –A generic term for a type of analysis Probably best to discuss MRC(M) Modeling –Multilevel Random Coefficient Modeling Different program implementations –HLM, MLn, SAS, BMDP, LISREL, and others

7. Logic of MRCM Coefficients describing level 1 phenomena are estimated within each level 2 unit (e.g., individual- level effects) –Intercepts—means –Slopes—covariance/regression coefficients Level 1 coefficients are also analyzed at level 2 (e.g., dyad-level effects) –Intercepts: mean effect of dyad –Slopes: effects of dyad-level predictors

8. Features of Multilevel Analyses Parameter estimates incorporate effects across the hierarchies Analyze all basic phenomena (means, variances and covariances) at multiple levels simultaneously –Relationships (covariances) can differ across levels of analysis Psuedo-Variants include MACS models in SEM when Level 2 N is small and two-stage modeling

9. Negative Individual, Positive Group

11. Positive Individual, Negative Group

13. No Individual, Positive Group

15. No Group, Mixed Individual

16. A Contrived Example Y ij = Friendship Closeness ratings of each individual i within each dyad j. Level 1 Measures: Age & Social Skill of the individual participants Level 2 Measures: Length of Friendship & Gender Composition of Friendship dyad

17. The Equations y ij =  0j +  1j Age +  2j SocSkill +  3j Age*Skill + r ij The Level 1 Equation:  0j =  00 +  01 (Time) +  02 (Gnd) +  03 (Time*Gnd) + u 0j  1j =  10 +  11 (Time) +  12 (Gnd) +  13 (Time*Gnd) + u 1j  2j =  20 +  21 (Time) +  22 (Gnd) +  23 (Time*Gnd) + u 2j  3j =  30 +  31 (Time) +  32 (Gnd) +  33 (Time*Gnd) + u 3j The Level 2 Equations: =

18. Types of Constructs Multidimensionality Dirtyness Degree of Complexity

19. Construct Space

20. … and a centroid

21. Selecting Indicators

22. Elements of the Simulation Communality Axis of the Construct's Centroid Maximum Reliability of an indicator (1.0) Selection Diversity.4.8.1.3.5 Selection Planes

23. Indicators of the Construct

24. Selecting Three (at random)

25. Latent Construct Centroid

26. Construct Trueness TrTr TaTa

27. … and another

28. … yet another

29. Selecting Six (Three Pairs)

30. … take the mean

31. … and find the centroid

32. How about 3 pairs of 3?

33. Centroid of Yellow

34. Centroid of Red

35. Centroid of Orange

36. … taking the means

37. … yields reliable & valid indictors

38. Construct Specific (but reliable) Random Error Variance Components

39. += Aggregate the 2 Indicators

40. += Standardizing the Scale

41. A Word (picture?) of Caution +=+ +=+ +=+ } 50% What is this Construct?

42. Absolute Trueness Values Indicators 95% Max 95% Max 95% Max Low DiversityMedium DiversityHigh Diversity Note. These values are symmetric. The degree of possible bias between any two constructs would be 2 times the tabled values

43. Finding Optimally Efficient Indicators

45. Implications of Selection Confirmatory Analyses are very good. – Little or no evidence of bias – Doesn't overcorrect for measurement error You can have validity without reliability – Hard to argue, but possible – Need very good theory Don't "just do it," think about it first… – Facilitates making better selections – Avoids the allure of the 'bloated specific'

46. How To Find Them? Focus on Construct Space! Not item space. Focus on Construct Space! Not item space. – 1) Select a broad set of constructs Some with Small, Medium, & Large Positive Correlations Some with Small, Medium, & Large Positive Correlations Some with Small, Medium, & Large Negative Correlations Some with Small, Medium, & Large Negative Correlations Some that are zero Correlated Some that are zero Correlated – 2) Calculate Latent Correlations on whole sample (save) – 3) Split sample into two random halves – 4) Find Optimal Set on 1 st Half of Sample Systematically select all possible combinations of n items from the original item pool Systematically select all possible combinations of n items from the original item pool Determine the best set that reproduces whole sample correlations Determine the best set that reproduces whole sample correlations – 5) Cross-validate on Second Half – 6) Repeat by generating on 2 nd half and cross-validating.

47. Inter-Reporter Relations Self Friend Peer Teacher Parent 1.0.841.0.19.171.0.18.24.951.0.31.20.25.211.0.21.15.20.721.0.24.19.24.17.52.391.0.15.13.16.09.33.32.861.0.33.28.06.11.26.27.231.0.28.26.00.07.14.24.18.871.0 O R O R O R O R O R

48. Overt Reactive Overt Instrumental Relational Reactive Relational Instrumental Overt (Dispositional) Relational (Dispositional) A Unifying Model of Aggression

49. ReactiveInstrumentalReactiveInstrumental Overt (Dispositional) Relational (Dispositional) A Unifying Model of Aggression

50. ReactiveInstrumentalReactiveInstrumental Overt (Dispositional) Relational (Dispositional) ReactiveInstrumental A Unifying Model of Aggression

51. Overt (Dispositional) Relational (Dispositional) ReactiveInstrumental -.07.83 A Unifying Model of Aggression

52. Reactively Aggressive Instrumentally Aggressive Neither Both Primarily Instrumental Primarily Reactive ‘Typical’ range Sub-types of Aggression Based on Function

53. Dropout: Random Process Time 1 Time 2 Time 3 Time 2 Time 1 Time 3 Time 2 Time 1

54. Time 2 Dropout: Functionally Random Time 1 Time 2 Time 3 Time 1 Time 3 SelectiveInfluence R = 0

55. Dropout: Selective Process(es) Time 1 Time 2 Time 3 Time 1 Time 2 Time 3 Survival Analysis Dropout Analysis SelectiveInfluence R = ? Time 3 Time 2

56. Time and Intervals Age in years, months, days. Experiential time: Amount of time something is experienced – Years of schooling, Length of relationship, Amount of practice – Calibrate on beginning of event, measure time experienced Episodic time: Time of onset of a life event – Toilet trained, driver license, puberty, birth of child, retirement – Early onset, on-time, late onset: used to classify or calibrate. – Time since onset or time from normative or expected occurance. Measurement Intervals – How fast is the developmental process? – Intervals must be equal to or less than expected processes of change – Cyclical processes E.g., schooling studies at yearly intervals vs half-year intervals

57. Some Measurement Features Operationally define Constructs as precisely as possible – Reduces alternative outcome interpretations – Increases translate-ability – Uni- vs Multidimensional constructs Use multiple indicators of each construct – Triangulates measurement to assess validity and correct for unreliability Use large samples – Increases power – Allows sophisticated and powerful analyses Screen data – Estimate missing values, transform non-normal distributions, identify and fix outliers – Increases power and reduces spurious conclusions

58. Some Design Features Use Multiple Constructs – Assists in validity assessment – Can show what a construct is, as well as what it is not Specify Competing Hypotheses – Strengthens a theoretical position by demonstrating that some hypotheses are rejectable while others are not Emphasize Confirmatory Designs – Encourages careful theorizing – Minimizes capitalization on chance – Strengthens theoretical position When using Exploratory Approaches – Use cross-validation techniques – Interpret borderline effects carefully – Focus on effect size rather than significance

59. Some Developmental Truisms Homotypic vs heterotypic expressions – E.g., Aggression Surface-structure vs deep-structure roots of behavior – E.g., resource-directed behavior Different paths can lead to same outcome Same path can lead to different outcomes Development is both Qualitative & Quantitative – Light is both wave and particle…

60. One construct -- Four Occasions Time 1 Time 2 Time 3 Time 4 e1e1 e2e2 e3e3 e1e1 e2e2 e3e3 e1e1 e2e2 e3e3 e1e1 e2e2 e3e3 Equating the reliable measurement parameters e 4, 5, 6 e 7, 8, 9 e 10, 11, 12

61. A Longitudinal Simplex Structure 1.0.801.0.64.801.0.51.64.801.0 T1 T2 T3 T4 Time 1 Time 2 Time 3 Time 4

62. Longitudinal Structures Model Time 1 Time 2 Time 3 Time 4 1111      21 =  21,  32 =  32,  43 =  43  31 =  21  32  41 =  21  32  43 In standardized solution, the correlations are reproduced by tracing the paths:  42 =  32  43 e1e1 e2e2 e3e3 e1e1 e2e2 e3e3 e1e1 e2e2 e3e3 e1e1 e2e2 e3e3 e 4, 5, 6 e 7, 8, 9 e 10, 11, 12.8

63. Identification & Scale Setting 1 Indicator 0.0* e 1.0* 2 Indicators ee e = 1.0* 3 Indicators eee e e e 1.0* Var1 Corr1,2 Var2 Var1 C12 Var2 C13 C23 Var3 Useable Information:

64. Figural Conventions AB en Circles and Ovals represent Latent Constructs Boxes & Rectangles represent manifest (measured) variables Single-headed lines are directional (causal) relationships represented as regression estimates Double-headed lines are non-directional (co- occurring) relationships represented as covariance or variance estimates

65. LISREL Conventions Mn(row, column); mn(end, start) e.g. 3 from 2 When drawing, number top to bottom then left to right

66. 1.0* 1 2 1 2 3 4 5 6 e e e e e e e e e e e e e DA NG=1 NI=6 ME=ML NO=100 KM FI=OUT_THERE.DAT SD FI=OUT_THERE.DAT ME FI=OUT_THERE.DAT MO ny=6 ne=2 ly=fu,fi te=sy,fi ps=sy,fi !note ly=ny,ne ly(indicator,construct) Fr ly(1,1) ly(2,1) ly(3,1) Fr te(1,1) te(2,2) te(3,3) !note: te=ny,ny ty(indicator,indicator) Fr ly(4,2) ly(5,2) ly(6,2) Fr te(4,4) te(5,5) te(6,6) !note: ps=ne,ne ps(construct,construct) VA 1.0 ps(1,1) ps(2,2) Fr ps(2,1) OU so ad=off rs sc Simple Confirmatory Factor Model

67. 3 4 7 8 9 1.0* e e e e e 10 11 12 e e e e e e e e 1 2 1 2 3 1.0* e e e e e e e e e e 4 5 6 e e e A Longitudinal Confirmatory Factor Model e e e e

68. DA NG=1 NI=12 ME=ML NO=100 KM FI=OUT_THERE_long.DAT SD FI=OUT_THERE_long.DAT ME FI=OUT_THERE_long.DAT MO ny=12 ne=4 ly=fu,fi te=sy,fi ps=sy,fi be=ze Fr ly(1,1) ly(2,1) ly(3,1) ly(4,2) ly(5,2) ly(6,2) Fr ly(7,3) ly(8,3) ly(9,3) ly(10,4) ly(11,4) ly(12,4) Fr te(7,7) te(8,8) te(9,9) te(10,10) te(11,11) te(12,12) Fr te(1,1) te(2,2) te(3,3) te(4,4) te(5,5) te(6,6) FR te(7,1) te(8,2) te(9,3) te(10,4) te(11,5) te(12,6) VA 1.0 ps(1,1) ps(2,2) ps(3,3) ps(4,4) fr ps(2,1) Fr ps(4,3) Fr ps(3,1) ps(4,1) ps(3,2) ps(4,2) Ou so ad=off rs sc LISREL Source Code

69. 3 4 7 8 9 e e e e= e e e 10 11 12 e e e e= 1 2 1 2 3 1.0* e e e e e e e e e e 4 5 6 e e e A Longitudinal Confirmatory Factor Model e e e e

70. DA NG=1 NI=12 ME=ML NO=100 KM FI=OUT_THERE_long.DAT SD FI=OUT_THERE_long.DAT ME FI=OUT_THERE_long.DAT MO ny=12 ne=4 ly=fu,fi te=sy,fi ps=sy,fi Fr ly(1,1) ly(2,1) ly(3,1) ly(4,2) ly(5,2) ly(6,2) Fr ly(7,3) ly(8,3) ly(9,3) ly(10,4) ly(11,4) ly(12,4) Fr te(1,1) te(2,2) te(3,3) te(4,4) te(5,5) te(6,6) Fr te(7,7) te(8,8) te(9,9) te(10,10) te(11,11) te(12,12) VA 1.0 ps(1,1) ps(2,2) !ps(3,4) ps(4,4) fr ps(2,1) Fr ps(4,3) Fr ps(3,1) ps(4,1) ps(3,2) ps(4,2) LISREL Source Code: Part 1

71. EQ ly(1,1) ly(7,3) Eq ly(2,1) ly(8,3) Eq ly(3,1) ly(9,3) Eq ly(4,2) ly(10,4) Eq ly(5,2) ly(11,4) Eq ly(6,2) ly(12,4) Fr ps(3,3) ps(4,4) Ou so ad=off sc rs LISREL Source Code: Part 2

72. 3 4 7 8 9 e e e e= e e e 10 11 12 e e e e= 1 2 1 2 3 1.0* e e e e e e e e e e 4 5 6 e e e A Longitudinal Cross-lag Structural Model e e e e

73. DA NG=1 NI=12 ME=ML NO=100 KM FI=OUT_THERE_long.DAT SD FI=OUT_THERE_long.DAT ME FI=OUT_THERE_long.DAT MO ny=12 ne=4 ly=fu,fi te=sy,fi ps=sy,fi be=fu,fi Fr ly(1,1) ly(2,1) ly(3,1) ly(4,2) ly(5,2) ly(6,2) Fr ly(7,3) ly(8,3) ly(9,3) ly(10,4) ly(11,4) ly(12,4) Fr te(1,1) te(2,2) te(3,3) te(4,4) te(5,5) te(6,6) Fr te(7,7) te(8,8) te(9,9) te(10,10) te(11,11) te(12,12) VA 1.0 ps(1,1) ps(2,2) fr ps(2,1) Fr ps(3,3) ps(4,4) Fr ps(4,3) !note: be=ne,ne be(construct,construct) (to,from) (row,column) Fr be(3,1) be(4,1) be(3,2) be(4,2) LISREL Source Code: Part 1