1. On-line Learning Environment for Multilevel Modelling Fiona Steele and Sacha Brostoff Centre for Multilevel Modelling University of Bristol

2. 2 The LEMMA Project A node of the ESRC-funded National Centre for Research Methods LEMMA – Learning Environment for Multilevel Methodology and Applications – www.cmm.bris.ac.uk Research and training components

3. 3 LEMMA Training Capacity building in the analysis of data with complex structure – Ultimate goal is to move learners to take-off, i.e. conducting and publishing multilevel analyses Different modes of delivery – Face-to-face workshops (3-day + 5-day with time allocated to analysis of participants data) – Face-to-face workshop followed by on-line mentoring – Training for established networks (e.g. university departments) – Web-based materials in a virtual learning environment

4. 4 Lessons Learnt Participants need to be motivated and have time to learn – Best motivated are those with data and research questions that can be addressed through MLM Experience of on-line follow-up and targeting established groups disappointing Participants often do not possess pre-requisites for MLM (good understanding of multiple regression) In practical sessions, tendency to focus on mechanics of using software rather than interpretation

5. 5 Basic Principles Accessible to anyone with basic statistics training (up to simple regression) Modules to have 2 integrated components: concepts and practice Facility for learners self-evaluation – Pre-requisite quiz, and regular quizzes throughout materials Collect data to evaluate materials and inform future training initiatives – Basic user profile information collected on registration – Quiz responses, webstats on patterns of use Design materials so they can be easily modified by other trainers

6. 6 Types of Material 5-minute video giving overview of content Prerequisites with links to other online resources 2 linked documents: (i) Concepts and methods, and (ii) practice (MLwiN instructions with interpretation of output) Quiz questions Further reading (published research and other online resources) Glossary For each module:

7. 7 Structure of Linked Documents Concepts – 30-40 pages, split into lessons – Illustrative examples from mix of disciplines – Draw links between fitted model equations, graphs of predictions and verbal interpretation – No reference to software – Expect other trainers to use with little change Practice – Each Concepts lesson followed by exercises in MLwiN – Thorough analysis and interpretation of one dataset – Trainers can rewrite for other datasets and software

8. 8 Core Materials 1. Types of variable 2. Introduction to statistical modelling 3. Multiple regression (single-level) 4. Data structures 5. Multilevel modelling of continuous data 6. Logistic regression (single-level) 7. Multilevel logistic regression

9. 9 Future Materials Substantive examples linked to core materials Models for other types of outcome – Nominal, ordinal, counts, duration Models for non-hierarchical structures

10. 10 The LEMMA VLE Moodle Open source, widely used – Open University is involved in its development Has a good licensing model – Avoids annual fees – No restrictions on how many users – Sustainable Does everything we need

11. 11 Demonstrating the prototype Encouraging engagement – Video overviews – Quizzes – Straw polls – ML Drivers Licence

12. 12 Further work Improving navigation Pilot testing Link in to the follow up to Athens Link administration with MLwiN UK free version More question types Certainty based marking