Performance on In-class vs Performance on In-class vs. Online Administration of Low-Stakes Assessments Xochith Herrera1, Manher Jariwala2, Jayson Nissen1, Eleanor Close3, Ben Van Dusen1 1California State University, Chico, 2Boston University, 3Texas State University, xherrera@mail.csuchico.edu Introduction Data Analysis with Hierarchical Linear Models Research-based assessments (RBA) are powerful tools for transforming physics courses. But using them in the classroom can be a real pain. Level 2 (N=25) Course Type: Algebra-Based Mech. Calculus-Based Mech. Calculus-Based E&M Missing Data Course Section Student Test Score Imputed with Hierarchical Multiple Imputations. Missing data imputed m times to create m complete data sets. Each complete data set is analyzed independently. Results from the m analyses are combined. Preferable to only using matched data as it maximizes statistical power and has the same basic assumptions1. Level 1 (N=1,310) Testing Condition: Completed Concept Inventory as a PPT – paper and pencil test CBT – computer-based test Designed to address the needs of educators and researchers, the LASSO platform hosts, administers, scores, and aggregates a variety of RBAs for researchers and educators. Thereby, LASSO seeks to transform We built two-level Hierarchical Linear Models for both the pretest and posttest scores with the HLM 7.01 software. Outcome variable of test score Level 2 variables for the course types Level 1 variable for testing condition RBAs from a research tool used at elite institutions into an easy to use tool that drives course transformations across STEM disciplines at a wide variety of institutions. 1. Y. Dong and C.-Y. J. Peng, “Principled missing data methods for researchers,” Springerplus 2, 0 (2013). HLM Results Conclusion Collecting data with LASSO provides equivalent data to that collected with paper and pencil tests. This can free up in class time and reduce the barriers to instructors using RBAs. One limitation is that we do not know the extent to which missing data may skew results. Our use of HMI may have mitigated this potential effect. Research Question To what extent were scores on concept inventories collected online outside of class with the LASSO platform similar to those collected in class using paper and pencil tests? Instructors and researchers use our recommended practices to maximize student participation. Design and Methods Student Participation An experimental design that divided 1,310 students from 25 sections in 3 different courses into 2 groups using random stratified sampling. For more information see Poster XXXXXXXX In conjunction with this research we investigated how instructor practices influenced student participation rates using Hierarchical Generalized Linear Models. We found that when instructors use all four recommended practices participation rates with LASSO were similar to those with paper and pencil tests Differences in scores between conditions for both pre and post test Were small and inconsistent Were unreliable The addition of testing condition to the models increased the variance, indicating it was a very poor predictor of test scores2. Posttests Paper CLASS Online FCI or CSEM Paper FCI or CSEM Online CLASS Pretests Group 1 Group 2 Class split in two The LASSO Platform Predicted scores with 95% CIs. Create account at: https://learningassistantalliance.org Creating course within LASSO Assigning course assessments Uploading class roster and launching the pretests Launching posttests at the end of the course Download summary report The predicted test scores for PPT and CBT conditions fell well within the 95% confidence interval for the other testing condition in the same course. Recommended Practices The assessments used in the study were: FCI - Force Concept Inventory CSEM - Conceptual Survey on Electricity and Magnetism CLASS - Colorado Learning Attitudes about Science Survey 2. S.W. Raudenbush and A.S. Bryk, Hierarchical Linear Models: Applications and Data Analysis Methods (SAGE Publications, 2002). Email reminders Pretest participation credit In class reminders Posttest participation credit This work funded in part by the NSF under grant nos. DUE-1525354, DUE-1525338 DUE-808790 & DUE-1431578. Back ground photo and image of rake and legs is sand art by Tony Plant, images courtesy of the Daily Mail © London Media. Cartoon of man stepping on rake: artist unknown.