Presentation is loading. Please wait.

Presentation is loading. Please wait.

UCLA Graduate School of Education & Information Studies National Center for Research on Evaluation, Standards, and Student Testing Growth Models for Monitoring.

Published byHugh Douglas Modified over 9 years ago

Similar presentations

Presentation on theme: "UCLA Graduate School of Education & Information Studies National Center for Research on Evaluation, Standards, and Student Testing Growth Models for Monitoring."— Presentation transcript:

1 UCLA Graduate School of Education & Information Studies National Center for Research on Evaluation, Standards, and Student Testing Growth Models for Monitoring Schools, Programs and Students Pete Goldschmidt, Ph.D. Washington Educational Research Association Seminar on Growth Modeling Renton, WA – June 2, 2006 If you choose to use this title slide, simply delete the previous slide (the one-line title version). This will be slide 1 of your presentation.

2 2/28 Evaluation and Monitoring - Overview Purpose Accountability and Evaluation are not equal Effects and Replicablilty Interpretations Taking the data structure into account Representations

3 3/28 Purpose of Growth Models Parents are not concerned with why their child does or does not do well, or why their school does or does not do well Parents are willing to pay a premium for school quality (even inaccurate measures of quality) Policy makers need to move beyond simple static aggregate measures and examine the process of schooling Rely on multiple observations to address issues of design Examine mediating and moderating factors Examine effects of implementation

4 4/28 Interpretation: Gain Model – Two cohorts

5 5/28 What Can Growth Models Tell Us About Students and Schools? Growth Models can examine the role of individual student characteristics as well as school context Question and Model 1) What difference does the school a child attends make in a child’s achievement growth? a. Student variables grand mean centered – means adjusted for differing school enrollments. b. How would the average student do in the school? 2) Do the effects of student background vary among schools? a. Group mean center variables – means are school means. 3) Do schools play a mediating role in terms of student factors that affect achievement growth? a. Group mean center student variables and add mean student level variables into school level model

6 6/28 The Basic model (again) Y ti =  0i +  1i T ti + e ti {within subject model}  0i =  00 + u oi {between subject model}  1i =  10 + u 1i Where Y ti = outcome for subject i at time t.  0i = status of subject i when T =0.  1i = average change per unit of T for subject i.  00 = grand mean of status when T =0.  10 = grand mean change per unit of T. e ti = within person error. u oi = unique increment to grand mean of status for subject i u 1i = unique increment to grand mean of change for subject i

7 7/28 Basic Model Results - Focusing on Individual Students Fixed Effects Estimates.e. approx p Initial Statusb 00 414.825.46.00 Growthb 10 13.782.14.00 Random Effectss.d Initial Statusu o 158.10.00 Growthu 1 42.78.00 Residuale it 70.92 When pasting text from another document, do the following: 1.Highlight the text you want to replace 2.Go to the EDIT menu and select PASTE SPECIAL 3.Select “Paste as: UNFORMATTED TEXT”

8 8/28 Results for 20 Students STUIDUoiU1ib00b10 010191M052-120.02422.523414.81813.779 010192F014107.456-4.792414.81813.779 010192F019-114.52916.18414.81813.779 010292F070165.611-61.27414.81813.779 010292M066-206.226-2.184414.81813.779 010293M01861.9228.184414.81813.779 010391F069-97.07-29.021414.81813.779 010391M05833.025-67.302414.81813.779 010391M066-164.066-26.971414.81813.779 010391M090-137.015-30.408414.81813.779 010392M02146.936-34.622414.81813.779 010591M106186.2692.205414.81813.779 010592F015-151.8818.034414.81813.779 010592M083249.405-38.667414.81813.779 010592M097-45.7714.336414.81813.779 010691M07550.14238.178414.81813.779 010692M053-137.538-16.268414.81813.779 010692M094-24.384-1.962414.81813.779 010693M009-76.7719.13414.81813.779

9 9/28 Distribution of initial Status

10 10/28

11 11/28 Results of Three Occasion Growth Model with Student SES Fixed EffectsEstimates.e. approx p I.S. not Low SESb 00 643.1020.50.00 I.S. Low SES gapb 01 -242.321.30.00 Growth not Low SESb 10 10.97.30.14 Growth Low SES gapb 01 2.97.70.70 Random Effectss.d Initial Statusuo147.66.00 Growthu142.85.00 Residualeti70.91

12 12/28 Individual Growth Trajectories accounting for SES

13 13/28 Evaluating Professional development Using Individual Teacher Data Using LPGM with Piecewise Growth Can examine longevity of effects Within Student (teacher) model becomes Y ti =  0i +  1i T ti +  2i PostInst ti + e ti

14 14/28 Teacher Knowledge (pedagogic decisions) Fixed EffectsCoeff.SEp-value Mean Pre-Institute0.030.020.06 Difference between: Female and Male0.170.050.00 Asian and White-0.070.080.41 Decline and White-0.060.050.23 Other and White-0.180.070.02 Hispanic and White-0.180.040.00 Black and White0.090.080.24 Cred. Ins. Private CA and CSU0.040.060.54 Cred. Ins. UC and CSU0.260.060.00 Cred. Ins. Out of State and CSU0.060.040.16 Emergency-Waiver and Full-Clear Preliminary-Intern and Full-Clear Teach ELL Fall and Not Teach ELL Fall 1-2 Yrs Teaching Exp. And 0 Yrs Teaching-0.280.070.00 3+ Yrs Teaching Exp. And 0 Yrs Teaching-0.170.060.00

15 15/28 Teacher Knowledge (pedagogic decisions)

16 16/28 Evaluating Professional development Using Individual Student Data (LGPM) and Piecewise Growth Use teacher Assessment results (reading levels) Some issues regarding linearity in early grades but could be handled in model as well Within Student (teacher) model becomes Y ti =  0i +  1i T ti +  2i Summer ti + e ti

17 17/28 Evaluating Professional development Using Individual Student Data (LGPM)

18 18/28 Evaluating Professional development Using Individual Student Data (LGPM)

19 19/28 Evaluating Professional development Using Individual Student Data (LGPM) RSRC Fixed EffectEstimateS.E. EstimateS.E. Summer Drop-0.90.1*-0.890.08* Grade-0.010.04-- Gender0.040.060.020.06 American Indian-0.060.59-0.010.3 Alaskan Native0.020.110.050.1 Hispanic0.040.150.020.14 Only other language, no English0.590.1*-0.260.06* Mostly other language, some English0.110.080.150.1 Enrolled in the EL/Gifted Program 00 or 010.290.150.360.17* Special Education0.310.13*0.340.11* Attended BLS0.640.18*0.820.25* * p<.05

20 20/28 School reform evaluation Change-from-baseline with posttest effect for program students Ytij = 0ij + 1ij(TIME)tij+ 2ij(TEST)tij + 3ij(PREVLATE)tij + 4ij(POSTVOTH)tij + etij, Between Students 0ij = 00j + 01j(PROGRAM)ij + r0ij, 1ij = 10j + 11j(PROGRAM)ij + r1ij, 2ij = 20j, 3ij = 30j + 31j(PROGRAM)ij + r3ij, 4ij = 40j + r4ij, Between Schools 00j = 000 + 010 +u00j, 10j = 100 + 110 +u10j + u11j, 20j = 200, 40j = 400 + u40j.

21 21/28 LBPM Results for Schools (round 2)

22 22/28 LBPM Results for Subgroups

23 23/28 LBPM Results for Schools by Subgroups

24 24/28 Growth Model Using Latent Growth to Predict distal Outcome (MLGC) Questions this type of growth model addresses How well do estimated initial status and growth trajectories predict student end-status; How does varying the number of test occasions affect the accuracy of end-status prediction; Do the data need to be contiguous; and What are the effects of covariates included at various levels in the models? A simpler approximation of this model would be a two stage growth model

25 25/28 Observed Initial Status and Growth trajectories by CAHSEE status

26 26/28 MLGC Results for Comparison Student

27 27/28 MLGC Results for Subgroups

28 28/28 Conclusions Longitudinal (Growth) Models attempt to examine factors affecting change Longitudinal Models can address some design issues Allow one to consider causal relationships more strongly Value Added Models focus “unexplained” differences between expected and actual growth Value Added Models are useful in examining whether schools produce gains/growth greater or less than expected (VAMs differ on how calculate expectations) Should not ignore data simply to base decisions on unexplained part of model Different parameterizations of longitudinal models can address many issues Longitudinal program evaluations address longevity of effects and effects of time-varying elements Different models (non-linear) can be used to place students or schools on a continuum and examine whether trajectories are sufficient to meet distal standards When pasting text from another document, do the following: 1.Highlight the text you want to replace 2.Go to the EDIT menu and select PASTE SPECIAL 3.Select “Paste as: UNFORMATTED TEXT”

29 29/28 next presentation Pete Goldschmidt voice email 310.794.4395 Goldschmidt@cse.ucla.edu If you choose to use this end slide, simply delete the previous slide (with no contact information). ©2006 Regents of the University of California

Download ppt "UCLA Graduate School of Education & Information Studies National Center for Research on Evaluation, Standards, and Student Testing Growth Models for Monitoring."

Similar presentations

Value Added in CPS. What is value added? A measure of the contribution of schooling to student performance Uses statistical techniques to isolate the.

Value Added in CPS. What is value added? A measure of the contribution of schooling to student performance Uses statistical techniques to isolate the.
Jamesville-DeWitt School Report Card 2008-9 Presented to the Board of Education May 10, 2010.

Jamesville-DeWitt School Report Card Presented to the Board of Education May 10, 2010.
Working with the Teachers’ Standards in the context of ITE. Some key issues for ITE Partnerships to explore.

Working with the Teachers’ Standards in the context of ITE. Some key issues for ITE Partnerships to explore.
Random effects as latent variables: SEM for repeated measures data Dr Patrick Sturgis University of Surrey.

Random effects as latent variables: SEM for repeated measures data Dr Patrick Sturgis University of Surrey.
Data Analysis State Accountability. Data Analysis (What) Needs Assessment (Why ) Improvement Plan (How) Implement and Monitor.

Data Analysis State Accountability. Data Analysis (What) Needs Assessment (Why ) Improvement Plan (How) Implement and Monitor.
Explaining Race Differences in Student Behavior: The Relative Contribution of Student, Peer, and School Characteristics Clara G. Muschkin* and Audrey N.

Explaining Race Differences in Student Behavior: The Relative Contribution of Student, Peer, and School Characteristics Clara G. Muschkin* and Audrey N.
UCLA Graduate School of Education & Information Studies National Center for Research on Evaluation, Standards, and Student Testing Moving Beyond Status:

UCLA Graduate School of Education & Information Studies National Center for Research on Evaluation, Standards, and Student Testing Moving Beyond Status:
Communicating through Data Displays October 10, 2006 © 2006 Public Consulting Group, Inc.

Communicating through Data Displays October 10, 2006 © 2006 Public Consulting Group, Inc.
English Language Arts Performance Assessment: Its Fairness and Predictive Validity Jia Wang, David Niemi, Pete Goldschmidt, and Haiwen Wang UCLA Graduate.

English Language Arts Performance Assessment: Its Fairness and Predictive Validity Jia Wang, David Niemi, Pete Goldschmidt, and Haiwen Wang UCLA Graduate.
UCLA Graduate School of Education & Information Studies National Center for Research on Evaluation, Standards, and Student Testing Keeping Kids in School:

UCLA Graduate School of Education & Information Studies National Center for Research on Evaluation, Standards, and Student Testing Keeping Kids in School:
Using Growth Models for Accountability Pete Goldschmidt, Ph.D. Assistant Professor California State University Northridge Senior Researcher National Center.

Using Growth Models for Accountability Pete Goldschmidt, Ph.D. Assistant Professor California State University Northridge Senior Researcher National Center.
Using Hierarchical Growth Models to Monitor School Performance: The effects of the model, metric and time on the validity of inferences THE 34TH ANNUAL.

Using Hierarchical Growth Models to Monitor School Performance: The effects of the model, metric and time on the validity of inferences THE 34TH ANNUAL.
Multilevel Modeling Soc 543 Fall 2004. Presentation overview What is multilevel modeling? Problems with not using multilevel models Benefits of using.

Multilevel Modeling Soc 543 Fall Presentation overview What is multilevel modeling? Problems with not using multilevel models Benefits of using.
What Makes For a Good Teacher and Who Can Tell? Douglas N. Harris Tim R. Sass Dept. of Ed. Policy Studies Dept. of Economics Univ. of Wisconsin Florida.

What Makes For a Good Teacher and Who Can Tell? Douglas N. Harris Tim R. Sass Dept. of Ed. Policy Studies Dept. of Economics Univ. of Wisconsin Florida.
Mark DeCandia Kentucky NAEP State Coordinator

Mark DeCandia Kentucky NAEP State Coordinator
UCLA Graduate School of Education & Information Studies National Center for Research on Evaluation, Standards, and Student Testing David Niemi August 10,

UCLA Graduate School of Education & Information Studies National Center for Research on Evaluation, Standards, and Student Testing David Niemi August 10,
March, 2011. What does the new law require?  20% State student growth data (increases to 25% upon implementation of value0added growth model)  20%

March, What does the new law require?  20% State student growth data (increases to 25% upon implementation of value0added growth model)  20%
Digital Media and Writing in Upper Elementary Schools: A Mixed Methods Study Mark Warschauer Binbin Zheng.

Digital Media and Writing in Upper Elementary Schools: A Mixed Methods Study Mark Warschauer Binbin Zheng.
-- Preliminary, Do Not Quote Without Permission -- VALUE-ADDED MODELS AND THE MEASUREMENT OF TEACHER QUALITY Douglas HarrisTim R. Sass Dept. of Ed. LeadershipDept.

-- Preliminary, Do Not Quote Without Permission -- VALUE-ADDED MODELS AND THE MEASUREMENT OF TEACHER QUALITY Douglas HarrisTim R. Sass Dept. of Ed. LeadershipDept.
Archived Information. MPR Associates 1 Effective Performance Measurement Systems  Define valid and reliable measures of student performance  Use appropriate.

Archived Information. MPR Associates 1 Effective Performance Measurement Systems  Define valid and reliable measures of student performance  Use appropriate.

Similar presentations

Ads by Google