Confidential and Proprietary. Copyright © 2011 Educational Testing Service. All rights reserved. 2/3/2016 Detection of Unusual Administrations Using a.

Slides:

Advertisements

Similar presentations

11-1 Empirical Models Many problems in engineering and science involve exploring the relationships between two or more variables. Regression analysis.

Advertisements

Qualitative predictor variables

Kin 304 Regression Linear Regression Least Sum of Squares

FACTORIAL ANOVA Overview of Factorial ANOVA Factorial Designs Types of Effects Assumptions Analyzing the Variance Regression Equation Fixed and Random.

Irwin/McGraw-Hill © Andrew F. Siegel, 1997 and l Chapter 12 l Multiple Regression: Predicting One Factor from Several Others.

6-1 Introduction To Empirical Models 6-1 Introduction To Empirical Models.

11 Simple Linear Regression and Correlation CHAPTER OUTLINE

Linear regression models

Ch11 Curve Fitting Dr. Deshi Ye

1 Copyright © 2005 Brooks/Cole, a division of Thomson Learning, Inc. Simple Linear Regression Estimates for single and mean responses.

LINEAR REGRESSION: Evaluating Regression Models Overview Assumptions for Linear Regression Evaluating a Regression Model.

LINEAR REGRESSION: Evaluating Regression Models. Overview Assumptions for Linear Regression Evaluating a Regression Model.

LINEAR REGRESSION: What it Is and How it Works. Overview What is Bivariate Linear Regression? The Regression Equation How It’s Based on r Assumptions.

REGRESSION Want to predict one variable (say Y) using the other variable (say X) GOAL: Set up an equation connecting X and Y. Linear regression linear.

Multivariate Data Analysis Chapter 4 – Multiple Regression.

Simple Linear Regression Analysis

Cumulative Geographic Residual Test Example: Taiwan Petrochemical Study Andrea Cook.

Quantitative Business Analysis for Decision Making Simple Linear Regression.

11-1 Empirical Models Many problems in engineering and science involve exploring the relationships between two or more variables. Regression analysis.

Business Statistics - QBM117 Statistical inference for regression.

Simple Linear Regression and Correlation

Week 14 Chapter 16 – Partial Correlation and Multiple Regression and Correlation.

Simple Linear Regression Analysis

Variance and covariance Sums of squares General linear models.

Forecasting Revenue: An Example of Regression Model Building Setting: Possibly a large set of predictor variables used to predict future quarterly revenues.

1 1 Slide © 2008 Thomson South-Western. All Rights Reserved Slides by JOHN LOUCKS & Updated by SPIROS VELIANITIS.

Correlation & Regression

Quantitative Business Analysis for Decision Making Multiple Linear RegressionAnalysis.

Marketing Research Aaker, Kumar, Day and Leone Tenth Edition

Chapter 11 Simple Regression

Correlation and Linear Regression

Linear Regression: Test scores vs. HW scores. Positive/Negative Correlation.

Forecasting Revenue: An Example of Regression Model Building Setting: Possibly a large set of predictor variables used to predict future quarterly revenues.

● Final exam Wednesday, 6/10, 11:30-2:30. ● Bring your own blue books ● Closed book. Calculators and 2-page cheat sheet allowed. No cell phone/computer.

Correlation & Regression

Multiple Regression and Model Building Chapter 15 Copyright © 2014 by The McGraw-Hill Companies, Inc. All rights reserved.McGraw-Hill/Irwin.

Chapter 14 Inference for Regression AP Statistics 14.1 – Inference about the Model 14.2 – Predictions and Conditions.

Lesson Multiple Regression Models. Objectives Obtain the correlation matrix Use technology to find a multiple regression equation Interpret the.

Copyright ©2011 Nelson Education Limited Linear Regression and Correlation CHAPTER 12.

SW388R6 Data Analysis and Computers I Slide 1 Multiple Regression Key Points about Multiple Regression Sample Homework Problem Solving the Problem with.

1 11 Simple Linear Regression and Correlation 11-1 Empirical Models 11-2 Simple Linear Regression 11-3 Properties of the Least Squares Estimators 11-4.

Yesterday Correlation Regression -Definition

Simple Linear Regression (SLR)

Simple Linear Regression (OLS). Types of Correlation Positive correlationNegative correlationNo correlation.

Overview of our study of the multiple linear regression model Regression models with more than one slope parameter.

General Linear Model.

A first order model with one binary and one quantitative predictor variable.

Recapitulation! Statistics 515. What Have We Covered? Elements Variables and Populations Parameters Samples Sample Statistics Population Distributions.

Copyright © 2004 by The McGraw-Hill Companies, Inc. All rights reserved.

Copyright © 2011 by The McGraw-Hill Companies, Inc. All rights reserved. McGraw-Hill/Irwin Simple Linear Regression Analysis Chapter 13.

Beginning Statistics Table of Contents HAWKES LEARNING SYSTEMS math courseware specialists Copyright © 2008 by Hawkes Learning Systems/Quant Systems, Inc.

1 1 Slide The Simple Linear Regression Model n Simple Linear Regression Model y =  0 +  1 x +  n Simple Linear Regression Equation E( y ) =  0 + 

Copyright © 2011 by The McGraw-Hill Companies, Inc. All rights reserved. McGraw-Hill/Irwin Multiple Regression Chapter 14.

Chapter 8 Relationships Among Variables. Chapter Outline What correlational research investigates Understanding the nature of correlation What the coefficient.

732G21/732G28/732A35 Lecture 3. Properties of the model errors ε 4. ε are assumed to be normally distributed

Copyright © 2015 McGraw-Hill Education. All rights reserved. No reproduction or distribution without the prior written consent of McGraw-Hill Education.

The “Big Picture” (from Heath 1995). Simple Linear Regression.

11-1 Empirical Models Many problems in engineering and science involve exploring the relationships between two or more variables. Regression analysis.

Kin 304 Regression Linear Regression Least Sum of Squares

Week 14 Chapter 16 – Partial Correlation and Multiple Regression and Correlation.

BPK 304W Regression Linear Regression Least Sum of Squares

BIVARIATE REGRESSION AND CORRELATION

LESSON 4.4. MULTIPLE LINEAR REGRESSION. Residual Analysis

Simple Linear Regression

Ch11 Curve Fitting II.

Regression III.

Checking the data and assumptions before the final analysis.

Regression Forecasting and Model Building

Chapter 14 Inference for Regression

Chapter 13 Multiple Regression

Presentation transcript:

Confidential and Proprietary. Copyright © 2011 Educational Testing Service. All rights reserved. 2/3/2016 Detection of Unusual Administrations Using a Linear Mixed Effects Model Minzhao Liu, University of Florida Mentors: Yi-Hsuan Lee Alina von Davier

Confidential and Proprietary. Copyright © 2011 Educational Testing Service. All rights reserved. Outline 2/3/2016 Design of Experiment Model and Method Results and Interpretation Prediction Discussion 2

Confidential and Proprietary. Copyright © 2011 Educational Testing Service. All rights reserved. Objectives Detect unusual administrations based on the model we fit Use regression model to fit the data Hypothesis: individual scores can be explained by background predictors 2/3/2016 3

Confidential and Proprietary. Copyright © 2011 Educational Testing Service. All rights reserved. Illustration of the Hypothesis 2/3/ Gender Country Repeater Years of study Daily study time Level of education Major Yrs abroad (Eng) Status Individual score Hypothetical Distribution of individual score Mean score Of subgroup Distribution of subgroup mean score

Confidential and Proprietary. Copyright © 2011 Educational Testing Service. All rights reserved. Data Description 2/3/2016 TOEIC data 15 administrations Random sample of 18,000 test- takers per administration 16 background questions 5

Confidential and Proprietary. Copyright © 2011 Educational Testing Service. All rights reserved. 2/3/ Design of Experiment TOEIC data 15 admin 18,000 test-takers per admin 16 background variables 3 groups Country A Country B Equated Univariate responses: Reading scores Listening scores An equating plan

Confidential and Proprietary. Copyright © 2011 Educational Testing Service. All rights reserved. Linear Mixed Effects Model 2/3/2016 Normality assumption (checked through residual diagnostics) Random effects can explain correlation among individuals In general: 7

Confidential and Proprietary. Copyright © 2011 Educational Testing Service. All rights reserved. Why Mixed Model 2/3/ Group random effect Admin random effect Individual error Fixed effects (background variables)

Confidential and Proprietary. Copyright © 2011 Educational Testing Service. All rights reserved. Why Mixed Model (cont’d) 2/3/2016 9

Confidential and Proprietary. Copyright © 2011 Educational Testing Service. All rights reserved. Variable Selection Forward selection 3 steps: 1.Random effects 2.Main effects 3.Interactions criteria 2/3/

Confidential and Proprietary. Copyright © 2011 Educational Testing Service. All rights reserved. Final Models Reading: 1.Years of study 2.Yrs abroad (Eng) 3.Repeater/First time 4.Level of Education 5.Major Random: admin 2/3/ Listening: 1.Years of study 2.Yrs abroad (Eng) 3.Repeater/First time 4.Level of Education 5.Major 6.Daily study time Random: admin Note: ‘group’ and ‘country’ variables did NOT turn out to be significant.

Confidential and Proprietary. Copyright © 2011 Educational Testing Service. All rights reserved. Prediction Definition of ‘Unusual’ Distribution of mean score of certain admin Approximate 95% prediction interval Example: for administration 1, the observed listening mean score is 317.3, and the prediction interval is (301.7, 350.0) 2/3/

Confidential and Proprietary. Copyright © 2011 Educational Testing Service. All rights reserved. Discussion Good news: 1.No significant effect for ‘country’ 2.No significant effect for ‘group’ 3.Prediction interval is narrow Limitations: – Nonlinearity – Few groups and administrations – Seasonality – Country and admin are confounded in the design 2/3/

Confidential and Proprietary. Copyright © 2011 Educational Testing Service. All rights reserved. 2/3/ Thank You for Your Attention~ Go Gators !