1. Learning Environment for Multilevel Methodology and Applications NCRM node based at University of Bristol Staff Kelvyn Jones Harvey Goldstein Edmond NG Fiona Steele Sally Thomas Jon Rasbash

2. Talk content Talk about one of the substantive research projects, the Geography of School Effects Briefly about the structure of our proposed Virtual Learning Environment - training Talk briefly about methodology developments Outline other major substantive research projects under LEMMA Review some general concepts and diagrams which help with modelling complex populations

3. Complex population structures – general concepts and diagrams In a multilevel model the population is defined as being made up of a number of levels or classifications. For example, students, households, teachers, schools. The relationship between classifications can take one of three forms Nested : for example students within schools Cross-classified: for example students are lie within a cross- classification of schools and area of residence Multiple membership: for example students attend more than one school These three elemental types of relationship between classifications can be combined to model very complex population structures.

4. Schematic unit diagram for a nested structure Pupils P1 P2 P3 P4 P5 P6 P7 P8 P9 P10 P11 P12 Area A1 A 2 A3 School S1 S2 S3 S4 Pupils nested within schools, schools nested within area With unit diagrams we draw the connections between individual units of each classification

5. School S1 S2 S3 S4 Pupils P1 P2 P3 P4 P5 P6 P7 P8 P9 P10 P11 P12 Area A1 A2 A3 School S1 S2 S3 S4 Pupils P1 P2 P3 P4 P5 P6 P7 P8 P9 P10 P11 P12 Area A1 A2 A3 School S1 S2 S3 S4 Pupils P1 P2 P3 P4 P5 P6 P7 P8 P9 P10 P11 P12 Area A1 A2 A3 In this structure schools are not nested within areas. For example Pupils 1 and 2 attend school 1 but come from different areas Pupils 6 and 10 come from the same area but attend different schools Schematic unit diagram cross-classified structure Schools are not nested within areas and areas are not nested within schools. School and area are are cross-classified

6. Schematic unit diagram for a multiple membership model School S1 S2 S3 S4 Pupils P1 P2 P3 P4 P5 P6 P7 P8 P9 P10 P11 P12 Area A1 A2 A3 P1 Lets take the cross-classified model of the last slide but suppose Pupil 1 moves in the course of the study from residential area 1 to 2 and from school 1 to 2 Now in addition to schools being crossed with residential areas pupils are multiple members of both areas and schools.

7. Classification diagrams Unit diagrams, with one node per unit, are useful but breakdown when we have very many classifications, each with their own set of units. A more useful and abstract diagrammatic tool for thinking about complex population structure is the classification diagram with one node per classification. Classifications connected by arrows are nested, the arrow indicates the direction of nesting Classifications not connected by arrows are crossed Classifications connected by double arrows indicate a multiple membership relation

8. Examples of classification diagrams pupil area school areaschool pupil areaschool pupil Nested Pupils nested within schools Schools nested within areas Crossed and nested Pupils nested within schools Pupils nested within areas Schools crossed with areas Multiple membership and crossed Pupils MM of schools Pupils MM of areas Schools crossed with areas

9. Geography of school effects We will be using the Pupil Level Annual School Census(PLASC) and Avon Longitudinal Study of Parents and Children(ALSPAC) data sets. Questions involve how do we properly model students attainment over time, taking account of complex structure including pupil mobility? School effectiveness models have assumed that schools are independent units. That is a child’s attainment,say, is not effected by what is happening in surrounding schools. Given we now have a “quasi-market” in education with schools in some sense competing is this assumption of independent school effects realistic? How might we model the non-independence? In the slides that follow I will refer to the ALSPAC data but similar issues and structures arise with the PLASC data. The slides are not a definitive statement of the research we plan to do-rather an exemplar showing some of the issues and structures that occur across all the LEMMA substantive research projects.

10. All the children born in the Avon area in 1990 followed up longitudinally Many measurements made including educational attainment measures Children span 3 school year cohorts(say 1994,1995,1996) Suppose we wish to model development of numeracy over the schooling period. We may have the following attainment measures on a child : m1 m2 m3 m4 m5 m6 m7 m8 primary school secondary school Alspac data

11. Measurement occasions within pupils M. Occasion PupilP. Teacher At each occasion there may be a different teacher P School Cohort Pupils are nested within primary school cohorts Primary schoolArea All this structure is nested within primary school Pupils are nested within residential areas Structure for primary schools

12. M. occasions PupilP. Teacher P School Cohort Primary school Area Nodes directly connected by a single arrow are nested, otherwise nodes are cross- classified. For example, measurement occasions are nested within pupils. However, cohort are cross-classified with primary teachers, that is teachers teach more than one cohort and a cohort is taught by more than one teacher. T1T2T3 Cohort 1959697 Cohort 2969798 Cohort 3989900 A mixture of nested and crossed relationships

13. It is reasonable to suppose the attainment of a child in a particular year is influenced not only by the current teacher, but also by teachers in previous years. That is measurements occasions are “multiple members” of teachers. m1 m2 m3 m4 t1 t2 t3 t4 M. occasions PupilP. Teacher P School Cohort Primary school Area We represent this in the classification diagram by using a double arrow. Multiple membership

14. If pupils move area, then pupils are no longer nested within areas. Pupils and areas are cross- classified. Also it is reasonable to suppose that pupils measured attainments are effected by the areas they have previously lived in. So measurement occasions are multiple members of areas M. occasions Pupil P. Teacher P School Cohort Primary school Area M. occasions Pupil P. Teacher P School Cohort Primary school Area Classification diagram without pupils moving residential areas Classification diagram where pupils move between residential areas BUT… What happens if pupils move area?

15. Classification diagram where pupils move between areas but not schools If pupils move schools they are no longer nested within primary school or primary school cohort. Also we can expect, for the mobile pupils, both their previous and current cohort and school to effect measured attainments M. occasions Pupil P. Teacher P School Cohort Primary school Area M. occasions Pupil P. TeacherP School Cohort Primary school Area Classification diagram where pupils move between schools and areas If pupils move area they will also move schools

16. Non-independence of school effects Schools competing in a quasi-market and increased mobility and priority on educational attainment of middle-class parents can lead to strongly differentiated school intakes. This has been modelled in terms of value added and compositional effects in school effectiveness studies. The question arises are there competition effects after compositional and value added effects have been modelled.

17. Differential school intake –value added effects The model for child attainment is conditional on baseline measures prior to school entry. The school effects therefore relate to progress during schooling not gross effects. Prior ability attainment School A School B Gross school effects Prior ability attainment School A School B Value added school effect taking account of differential school intake

18. Differential school intake - Compositional effects We often find a strong peer ability group effects. School means for intake ability were ranked and then categorised into low(bottom 25%), mid(25%-75%) and high(top 25%). The graph below shows the relationship between pupil level prior ability and predicted pupil attainment for low ability peer groups and high ability peer groups. A very able student in terms of intake ability, will on average do 1sd better in their final attainment when educated in high ability peer group rather than a low ability peer group

19. Modelling non-independence of school effects After adjusting for the value added and compositional effects of differential school intake how would we model any residual competitive effects between schools. One place to start would be to allow a covariance between schools with overlapping catchments. cov(u j, u k ) =  catchment (j,k) where u j is the school effect for school j and u k is the school effect for school k. catchment (j,k) is 1 if schools j and k have overlapping catchments, 0 otherwise If  is estimated as positive this means schools with overlapping catchments show a positive correlation. If  estimated as negative this means schools with overlapping catchments show a positive correlation.

20. Interpretation of spatially correlated school effects An overall positive correlation between schools with overlapping catchments means that if school j has a positive school effect and it shares a catchment with school k, then school k will tend to have a positive effect. Likewise if school j has a negative effect, school k will tend to have a negative effect. school rank school effect 0 An overall negative correlation means between pairs of schools with overlapping catchments means if one school has a positive school effect then the other school will tend to have a negative school effect. Negative correlation: two schools sharing a catchment. Positive correlation: two schools sharing a catchment.

21. An over simplistic model? Fitting a single parameter to schools the correlation between schools with overlapping catchments may be over simplistic. There may be structural attributes that vary across school pairs with overlapping catchments that cause some pairs to be negatively correlated and some pairs positively correlated resulting in an average correlation of zero. In which case we would wrongly conclude, with the single parameter model, that there were no “competitive” effects. For example, school pairs with overlapping catchments with similar intake compositions may be positively correlated. While pairs with different intake compositions may be negatively correlated, due to the lower intake school being demoralised due to its proximity to its “affluent” neighbour.

22. Modelling the effect of differential school intake on spatially correlated school effects cov(u j, u k ) =  1 catchment (j,k) +  2 intake (j,k) catchment (j,k) where u j is the school effect for school j and u k is the school effect for school k. catchment (j,k) is 1 if schools j and k have overlapping catchments, 0 ortherwise intake (j,k) = abs([mean intake for school j]-[mean intake for school k]) abs([mean intake for school j]-[mean intake for school k]) cov(u j, u k ) 0 Only allowing correlation between schools with overlapping catchments may also be too simplistic and more complex distance functions may be needed.

23. Other LEMMA substantive projects Modelling the duration of episodes in hospital (Steele, Jones) Data set :Hospital Episodes Statistics Multilevel, multiple membership event duration models for length of stay in hospital Voting choice (Johnston, Jones, Rasbash) Data set:BHPS Individual, household and neighbourhood determinants of voting abstention and party choice. Multilevel binary and multinomial repeated measures with latent categorical random effects at the individual and household levels. Mental health and psychosocial development (Rasbash, Lewis, Propper, Jones) Data sets:BHPS and ABSS Comparing continuous and discrete random effects for models of psychosocial development. Modelling group diversity (Goldstein, Burgess, Gordon) Data set PLASC Obtaining less biased estimates of diversity indexes, which can then be used as predictor variables.

24. Methodological developments There has recently been a lot of interest in latent categorical random effects. Where variation between units is modelled by unit membership of a set of discrete categories rather than continuous Normally distributed random effects. The main application has been in developmental trajectories, where individual’s growth curves are classified into a set of discrete patterns. MPLUS(Muthen), PROC TRAJ (Nagin), Latent Gold(Vermeuth), GLAMM(Rabe-Heskith et al) We will develop estimation procedures for these models and allow mixtures of continuous and discrete latent random effects for populations structured by multiple classifications To be implemented in the MlwiN software developed by the team.

25. Related methodological and substantive input from other ESRC projects Developing Multilevel Models for Realistically Complex Social Science Data(Goldstein, Rasbash, Steele, Plewis) Missing Data for Multilevel Models(Carpenter, Kenwood,Goldstein) Multilevel Multiprocess Models for Partnership and Childbearing Event Histories(Steele, Goldstein, Joshi) Methodologies for Studying Families and Family Effects(O’Connor, Rasbash) The Impact of Local Areas Influences on Individual Outcomes (Burgess, Jones, Propper, Johnson) Sample size, MCMC efficiency and identifiability(Browne)

26. Multilevel virtual learning environment Built collaboratively by cross-disciplinary team of statisticians, social scientists, ICT professionals and software engineers. How do we provide support to the social science community to deliver the NCRM’s required “step change” in methodological capacity? The team has an extensive track record in running training workshops. The problem is how to provide the follow up to allow workshop participants to convert awareness gained at workshops to improved methodological skills that they routinely use in their research. Not enough expert mentors to go around Can the LEMMA MVLE provide a platform to foster self-sustaining research communities? The LEMMA MVLE aims to facilitate both solo and group learning

27. MVLE Level 1 Training repository Materials for solo learning or group work at levels 2+3  Data sets and exemplar analyses  FAQs  Self-tests and quizzes  Conceptual modules  Check list for ethics  Links to other online resources  Reviews of software packages for multilevel modelling  Annotated archived multilevel modelling discussions from levels 2 and 3 of the MVLE  Annotated bibliographies MVLE Level 2 Online workshops Course management system  Notice boards  Asynchronous discussion groups  Synchronous chat with archiving  Calendar/diary-web log  Task lists  Work groups  Content management and version control  Collaborative writing tools  Learning log  Screen sharing  On-line questionnaire/survey tools  Videoconferencing MVLE Level 3 Research network building Research collaboration tools  Notice boards  Asynchronous discussion groups  Synchronous chat with archiving  Calendar  Task lists  Content management  Learning log  Screen sharing  Videoconferencing LEMMA MVLE – Learning Architecture