1. 35th Annual National Conference on Large-Scale Assessment June 18, 2005 How to compare NAEP and State Assessment Results NAEP State Analysis Project Don McLaughlin Victor Bandeira de Mello

2.  how do NAEP and state assessment trend results compare to each other?  how do NAEP and state assessment gap results compare to each other?  do NAEP and state assessments identify the same schools as high-performing and low- performing?  how do NAEP and state assessment trend results compare to each other?  how do NAEP and state assessment gap results compare to each other?  do NAEP and state assessments identify the same schools as high-performing and low- performing? overview: the questions

3.  results are different because  standards are different  students are different  time of testing is different  motivation is different  manner of administration is different  item formats are different  test content is different  tests have measurement error  results are different because  standards are different  students are different  time of testing is different  motivation is different  manner of administration is different  item formats are different  test content is different  tests have measurement error overview: the differences

4.  results are different because  standards are different  students are different  time of testing is different  motivation is different  manner of administration is different  item formats are different  test content is different  tests have measurement error  results are different because  standards are different  students are different  time of testing is different  motivation is different  manner of administration is different  item formats are different  test content is different  tests have measurement error overview: the differences

5.  the problem of different standards  how we addressed it  the problem of different students  how we addressed it  factors that affect validation  the problem of different standards  how we addressed it  the problem of different students  how we addressed it  factors that affect validation overview: the focus

6.  the problem of different standards

7.  trends and gaps are being reported in terms of percentages of students meeting standards  the standards are different in every state and in NAEP  comparisons of percentages meeting different standards are not valid  trends and gaps are being reported in terms of percentages of students meeting standards  the standards are different in every state and in NAEP  comparisons of percentages meeting different standards are not valid the different standards

8.  concept of population profile  a population profile is a graph of the achievement of each percentile of a population  concept of population profile  a population profile is a graph of the achievement of each percentile of a population the different standards

9.  a population achievement profile the different standards

10.  a population achievement profile the different standards 76% 32% 5%

11.  a population trend profile the different standards

12.  a population trend profile the different standards +9% +13% +5% gains

13.  a population gap profile the different standards gaps

14.  a population gap profile the different standards gaps after a 20-point gain

15.  a population gap profile the different standards 8 points smaller the same 6 points larger gap changes

16.  trends and gaps are being reported in terms of percentages of students meeting standards  the standards are different in every state and in NAEP  comparisons of percentages meeting different standards are not valid  trends and gaps are being reported in terms of percentages of students meeting standards  the standards are different in every state and in NAEP  comparisons of percentages meeting different standards are not valid the different standards

17.  the solution to the problem is to compare results at comparable standards  for comparing NAEP and state assessment gains and gaps in a state, score NAEP at the state’s standard  the solution to the problem is to compare results at comparable standards  for comparing NAEP and state assessment gains and gaps in a state, score NAEP at the state’s standard the different standards

18.  NAEP  individual plausible values for 4th and 8th grade reading in 1998, 2002, and 2003 and mathematics in 2000 and 2003  state assessment scores  school percentages meeting standards linked to NCES school codes, in 2003 and some earlier years  www.schooldata.org  NAEP  individual plausible values for 4th and 8th grade reading in 1998, 2002, and 2003 and mathematics in 2000 and 2003  state assessment scores  school percentages meeting standards linked to NCES school codes, in 2003 and some earlier years  www.schooldata.org the different standards

19.  a school-level population gap profile the different standards

20.  comparing school-level population gap profiles the different standards

21.  comparing school-level population gap profiles the different standards

22.  scoring NAEP at the state assessment standard  determine the cutpoint on the NAEP scale that best matches the percentages of students meeting the state’s standard  compute the percentage of the NAEP plausible value distribution that is above that cutpoint  scoring NAEP at the state assessment standard  determine the cutpoint on the NAEP scale that best matches the percentages of students meeting the state’s standard  compute the percentage of the NAEP plausible value distribution that is above that cutpoint the different standards

23.  equipercentile equating the different standards

24.  equipercentile equating the different standards A B C D average 205 215 225 235 220 average NAEP scale score hypothetical NAEP results in four schools in a state (actual samples have about 100 schools)

25.  equipercentile equating the different standards A B C D average 205 215 225 235 220 10%20%40%50%30% average NAEP scale score percent meeting state standard in school A, the state reported that 10% of the students met the standards

26.  equipercentile equating the different standards A B C D average 205 215 225 235 220 10%20%40%50%30% 225 235 230 average NAEP scale score percent meeting state standard NAEP scale score corresponding to percent meeting state standard in school A, 10% of the NAEP plausible value distribution was above 225

27.  equipercentile equating the different standards A B C D average 205 215 225 235 220 10%20%40%50%30% 225 235 230 5%10%45%60%30% average NAEP scale score percent meeting state standard NAEP scale score corresponding to percent meeting state standard percent above 230 on NAEP If the equating is accurate, we should be able to reproduce the percentages meeting the state’s standard from the NAEP sample

28.  equipercentile equating the different standards A B C D average 205 215 225 235 220 10%20%40%50%30% 225 235 230 5%10%45%60%30% error -5%-10%+5%+10% average NAEP scale score percent meeting state standard NAEP scale score corresponding to percent meeting state standard percent above 230 on NAEP

29.  relative error  in estimating cutpoints for state standards, relative error is the ratio of the observed error in reproducing school-level percentages meeting standards to that expected due to sampling and measurement error  relative error  in estimating cutpoints for state standards, relative error is the ratio of the observed error in reproducing school-level percentages meeting standards to that expected due to sampling and measurement error the different standards

30.  mapping of primary state standards on the NAEP scale: math grade 8 in 2003 the different standards

31.  mapping of primary state standards on the NAEP scale: math grade 8 in 2003 the different standards

32.  mapping of primary state standards on the NAEP scale: math grade 8 in 2003 the different standards

33.  national percentile ranks corresponding to state grade 4 reading standards in 2003 the different standards

34.  states have set widely varying standards  does it matter?  standards should be set where they will motivate increased achievement  surely some are too high and some are too low  states have set widely varying standards  does it matter?  standards should be set where they will motivate increased achievement  surely some are too high and some are too low the different standards

35.  states with higher standards have lower percentages of students meeting them the different standards

36.  on NAEP, states with higher standards do about the as well as other states the different standards

37.  the problem of different students

38.  different school coverage  different grade tested  absent students  excluded SD/ELLs  the problem of different students  different school coverage  different grade tested  absent students  excluded SD/ELLs the different students

39.  different school coverage  our comparisons between NAEP and state assessment results are for the same schools. NAEP weights these schools to represent the public school population in each state  we matched schools serving more than 99 percent of the public school population. However, especially for gap comparisons, we were missing state assessment results for small groups whose scores were suppressed for confidentiality reasons  the median percentage of the NAEP student population included in the analyses was about 96 percent  different school coverage  our comparisons between NAEP and state assessment results are for the same schools. NAEP weights these schools to represent the public school population in each state  we matched schools serving more than 99 percent of the public school population. However, especially for gap comparisons, we were missing state assessment results for small groups whose scores were suppressed for confidentiality reasons  the median percentage of the NAEP student population included in the analyses was about 96 percent the different students

40.  different grades tested  in some states, assessments were administered in grades 3, 5, or 7, and we compared these results to NAEP results in grades 4 and 8  the difference in grades involved a different cohort of students, as well as a difference in curriculum content. These effects combined to reduce NAEP-state assessment correlations in some states by about 0.05 to 0.10  different grades tested  in some states, assessments were administered in grades 3, 5, or 7, and we compared these results to NAEP results in grades 4 and 8  the difference in grades involved a different cohort of students, as well as a difference in curriculum content. These effects combined to reduce NAEP-state assessment correlations in some states by about 0.05 to 0.10 the different students

41.  absent students  some students are absent from NAEP sessions, and some of these are not made-up in extra sessions. NAEP imputes the achievement of the absent students to be similar to that of similar students who were not absent  a study by the NAEP Validity Studies Panel has found that these imputations leave negligible (if any) bias in NAEP results due to absences that study compared the state assessment scores of students absent from NAEP to the scores of students not absent  absent students  some students are absent from NAEP sessions, and some of these are not made-up in extra sessions. NAEP imputes the achievement of the absent students to be similar to that of similar students who were not absent  a study by the NAEP Validity Studies Panel has found that these imputations leave negligible (if any) bias in NAEP results due to absences that study compared the state assessment scores of students absent from NAEP to the scores of students not absent the different students

42.  excluded SD/ELLs  some students with disabilities and English language learners are excluded from NAEP and others are included. A teacher questionnaire is completed for each SD/ELL selected for NAEP  in the past, NAEP has ignored this exclusion, and there is clear evidence that as a result, states in which NAEP exclusions increased had corresponding reports of larger NAEP achievement gains (and vice versa)  excluded SD/ELLs  some students with disabilities and English language learners are excluded from NAEP and others are included. A teacher questionnaire is completed for each SD/ELL selected for NAEP  in the past, NAEP has ignored this exclusion, and there is clear evidence that as a result, states in which NAEP exclusions increased had corresponding reports of larger NAEP achievement gains (and vice versa) the different students

43.  full population estimates  the trend distortions caused by changing exclusion rates can be minimized by imputing the achievement of excluded students. in this project, comparisons between NAEP and state assessment results are based on the NAEP full population estimates [1]  imputations for excluded SD/ELLs are based on the achievement of included SD/ELLs with similar questionnaire and demographic profiles in the same state [1] an appendix includes comparisons using standard NAEP estimates  full population estimates  the trend distortions caused by changing exclusion rates can be minimized by imputing the achievement of excluded students. in this project, comparisons between NAEP and state assessment results are based on the NAEP full population estimates [1]  imputations for excluded SD/ELLs are based on the achievement of included SD/ELLs with similar questionnaire and demographic profiles in the same state [1] an appendix includes comparisons using standard NAEP estimates the different students

44.  statistically significant state NAEP gains from 1996 to 2000 grade 4 grade 8 17 of 37 16 of 35 12 of 377 of 35 ignoring excluded students full population estimates the different students

45.  statistically significant state NAEP gains and losses from 1998 to 2002 grade 4grade 8 gainslossesgainslosses 18186 23072 ignoring excluded students full population estimates the different students

46.  factors that affect validation

47.  the question  do state assessments and NAEP agree on which schools are doing better than others?  the measure  correlation between state assessment and NAEP school-level results  the question  do state assessments and NAEP agree on which schools are doing better than others?  the measure  correlation between state assessment and NAEP school-level results validation

48.  factors that specifically affect NAEP-state assessment correlations of school-level statistics  size of school NAEP samples  grade level the same or different  extremeness of the standard  factors that specifically affect NAEP-state assessment correlations of school-level statistics  size of school NAEP samples  grade level the same or different  extremeness of the standard validation

49.  median school-level correlations between NAEP and state assessment results grade 4grade 8 mathreadingmathreading original 0.760.720.810.73 adjusted 0.840.820.860.81 validation

50.  NAEP and state assessment school means validation

51.  two reports have been produced on 2003 NAEP-state assessment comparisons, one for mathematics and one for reading  each report has an appendix with multi-page comparison profiles for all of the states the following are examples of the kinds of information included  two reports have been produced on 2003 NAEP-state assessment comparisons, one for mathematics and one for reading  each report has an appendix with multi-page comparison profiles for all of the states the following are examples of the kinds of information included summary

52.  state profiles of NAEP-state assessment comparisons  test score descriptions and results summary  standards relative to NAEP  correlations with NAEP  changes in NAEP exclusion/accommodation rates  trends (NAEP vs. state assessment)  gaps (NAEP vs. state assessment)  gap trends (NAEP vs. state assessment)  state profiles of NAEP-state assessment comparisons  test score descriptions and results summary  standards relative to NAEP  correlations with NAEP  changes in NAEP exclusion/accommodation rates  trends (NAEP vs. state assessment)  gaps (NAEP vs. state assessment)  gap trends (NAEP vs. state assessment) state profiles

53.  a state’s standards relative to its achievement distribution state profiles

54.  a state’s math trends comparison state profiles

55.  poverty gap comparison state profiles

56.  poverty gap comparison  state assessment results  poverty gap comparison  state assessment results state profiles

57.  poverty gap comparison  NAEP results  poverty gap comparison  NAEP results state profiles

58.  poverty gap comparison  NAEP - state assessment  poverty gap comparison  NAEP - state assessment state profiles

59.  a state’s poverty gap comparison state profiles

60.  trends  gaps  overall coverage  subpopulation coverage  school analyses sample  trends  gaps  overall coverage  subpopulation coverage  school analyses sample other results

61.  comparison of trends reported by NAEP and state assessments (number of states) other results: trends grade 4grade 8 math 00-03read 98-03math 00-03read 98-03 3555 103111 060 state assessment reported greater gains no significant difference NAEP reported greater gains

62.  reading 2003  NAEP and state assessments tended to find similar achievement gaps  math 2003  NAEP tended to find slightly larger gaps than state assessments did  reading 2003  NAEP and state assessments tended to find similar achievement gaps  math 2003  NAEP tended to find slightly larger gaps than state assessments did other results: gaps

63.  median state percentages of NAEP schools and student population matched and included in analyses other results: coverage grade 4grade 8 mathreadmathread 99.1 99.2 99.599.699.8 94.994.495.394.2 95.895.496.896.1 percent of schools matched percent of student population matched percent of schools included in analyses percent of students included in analyses

64.  number of states and percent minority students included in the 2003 reading gap analyses other results: coverage grade 4grade 8 number of states2620 students included (%) 88.099.2 number of states 1413 students included (%) 84.591.7 number of states 3128 students included (%) 87.990.1 black hispanic disadvantage

65.  percent meeting standards from state tests in NAEP schools and from state reports, 2003 other results: school sample

66.  producing the report

67. SAS programs  the process  find state scores for NAEP sample  score NAEP in terms of state standards  compute inverse CDF pair for subpopulation profiles  compute mean NAEP-state gap differences and standard errors  compute trends and gains  compute smoothed frequency distribution of plausible values  compute NAEP-state correlations  the process  find state scores for NAEP sample  score NAEP in terms of state standards  compute inverse CDF pair for subpopulation profiles  compute mean NAEP-state gap differences and standard errors  compute trends and gains  compute smoothed frequency distribution of plausible values  compute NAEP-state correlations

68. SAS programs  the process  find state scores for NAEP sample  score NAEP in terms of state standards  compute inverse CDF pair for subpopulation profiles  compute mean NAEP-state gap differences and standard errors  compute trends and gains  compute smoothed frequency distribution of plausible values  compute NAEP-state correlations  the process  find state scores for NAEP sample  score NAEP in terms of state standards  compute inverse CDF pair for subpopulation profiles  compute mean NAEP-state gap differences and standard errors  compute trends and gains  compute smoothed frequency distribution of plausible values  compute NAEP-state correlations data setup

69. SAS programs  the process  find state scores for NAEP sample  score NAEP in terms of state standards  compute inverse CDF pair for subpopulation profiles  compute mean NAEP-state gap differences and standard errors  compute trends and gains  compute smoothed frequency distribution of plausible values  compute NAEP-state correlations  the process  find state scores for NAEP sample  score NAEP in terms of state standards  compute inverse CDF pair for subpopulation profiles  compute mean NAEP-state gap differences and standard errors  compute trends and gains  compute smoothed frequency distribution of plausible values  compute NAEP-state correlations population profiles

70. SAS programs  the process  find state scores for NAEP sample  score NAEP in terms of state standards  compute inverse CDF pair for subpopulation profiles  compute mean NAEP-state gap differences and standard errors  compute trends and gains  compute smoothed frequency distribution of plausible values  compute NAEP-state correlations  the process  find state scores for NAEP sample  score NAEP in terms of state standards  compute inverse CDF pair for subpopulation profiles  compute mean NAEP-state gap differences and standard errors  compute trends and gains  compute smoothed frequency distribution of plausible values  compute NAEP-state correlations

71. SAS programs  the process  find state scores for NAEP sample  score NAEP in terms of state standards  compute inverse CDF pair for subpopulation profiles  compute mean NAEP-state gap differences and standard errors  compute trends and gains  compute smoothed frequency distribution of plausible values  compute NAEP-state correlations  the process  find state scores for NAEP sample  score NAEP in terms of state standards  compute inverse CDF pair for subpopulation profiles  compute mean NAEP-state gap differences and standard errors  compute trends and gains  compute smoothed frequency distribution of plausible values  compute NAEP-state correlations

72. SAS programs  the process  find state scores for NAEP sample  score NAEP in terms of state standards  compute inverse CDF pair for subpopulation profiles  compute mean NAEP-state gap differences and standard errors  compute trends and gains  compute smoothed frequency distribution of plausible values  compute NAEP-state correlations  the process  find state scores for NAEP sample  score NAEP in terms of state standards  compute inverse CDF pair for subpopulation profiles  compute mean NAEP-state gap differences and standard errors  compute trends and gains  compute smoothed frequency distribution of plausible values  compute NAEP-state correlations

73. SAS programs  programs  makefiles.sas  standards.sas  gaps.sas  gaps_g.sas  trends.sas  trends_r.sas  trends_g.sas  distribution.sas  correlation.sas  programs  makefiles.sas  standards.sas  gaps.sas  gaps_g.sas  trends.sas  trends_r.sas  trends_g.sas  distribution.sas  correlation.sas

74. SAS programs  programs  makefiles.sas  standards.sas  gaps.sas  gaps_g.sas  trends.sas  trends_r.sas  trends_g.sas  distribution.sas  correlation.sas  programs  makefiles.sas  standards.sas  gaps.sas  gaps_g.sas  trends.sas  trends_r.sas  trends_g.sas  distribution.sas  correlation.sas http://www.schooldata.org/reports.asp

75. SAS programs: setup  makefiles.sas  for state st get NAEP plausible values for subject s, grade g, and year y  get state assessment data for NAEP schools ( from NLSLASD )  merge files to getexample02.sas7bdat and example03.sas7bdat  makefiles.sas  for state st get NAEP plausible values for subject s, grade g, and year y  get state assessment data for NAEP schools ( from NLSLASD )  merge files to getexample02.sas7bdat and example03.sas7bdat

76. SAS programs: setup  makefiles.sas *******************************************************************************; * Project : NAEP State Analysis *; * Program : MakeFiles.SAS *; * Purpose : make source file for workshop at LSAC 2005 *; * *; * input : naep_r403 NAEP Reading grade 4 2003 data *; * naep_r402 NAEP Reading grade 4 2002 data *; * XX state XX assessment data *; * YY state YY assessment data *; * *; * output : example02 - 2002 data *; * example03 - 2003 data *; * *; * Author : NAEP State Analysis Project Staff *; * American Institutes for Research *; * *; *******************************************************************************;  makefiles.sas *******************************************************************************; * Project : NAEP State Analysis *; * Program : MakeFiles.SAS *; * Purpose : make source file for workshop at LSAC 2005 *; * *; * input : naep_r403 NAEP Reading grade 4 2003 data *; * naep_r402 NAEP Reading grade 4 2002 data *; * XX state XX assessment data *; * YY state YY assessment data *; * *; * output : example02 - 2002 data *; * example03 - 2003 data *; * *; * Author : NAEP State Analysis Project Staff *; * American Institutes for Research *; * *; *******************************************************************************;

77. SAS programs: setup  standards.sas  compute NAEP equivalents of state standards based on school level state assessment scores in NAEP schools  macro %stan(s,g,y,nlevs) output Stansgy file with state standard cutpoints on NAEP sample  standards.sas  compute NAEP equivalents of state standards based on school level state assessment scores in NAEP schools  macro %stan(s,g,y,nlevs) output Stansgy file with state standard cutpoints on NAEP sample sgyvarnamelevelcutstderrorpercent R403Rs5t04032164.23.291.9 R403Rs5t04033205.81.168.9 R403Rs5t04034264.91.89.4 StanR403

78. SAS programs: setup  standards.sas  generate school level file with percentages meeting levels by reporting category, with jackknife statistics  macro %StateLev(file,s,g,y)  macro %NAEP_State_Pcts(s,g,y,group)  macro %Sch_State_Pcts(standard,s,g,y) output StPcts_standard_sgy with school stats for first/recent standard, by category  macro %Criterion(standard,s,g,y) output criterion_ standard_sgy with criterion values for cutpoints  standards.sas  generate school level file with percentages meeting levels by reporting category, with jackknife statistics  macro %StateLev(file,s,g,y)  macro %NAEP_State_Pcts(s,g,y,group)  macro %Sch_State_Pcts(standard,s,g,y) output StPcts_standard_sgy with school stats for first/recent standard, by category  macro %Criterion(standard,s,g,y) output criterion_ standard_sgy with criterion values for cutpoints

79. SAS programs: gaps  gaps.sas  compute and plot subpopulation profiles (inverse CDF) and compute mean NAEP-state gap differences and respective standard errors, by regions of the percentile distribution  macro %gap(s,g,lev,y1,y2,group1,group2) where y1is the earlier years (need not be present) y2is the later year levis the standard for which the gap is being compared group1is the 5-char name of the focal group group2is the 5-char name of the comparison group  gaps.sas  compute and plot subpopulation profiles (inverse CDF) and compute mean NAEP-state gap differences and respective standard errors, by regions of the percentile distribution  macro %gap(s,g,lev,y1,y2,group1,group2) where y1is the earlier years (need not be present) y2is the later year levis the standard for which the gap is being compared group1is the 5-char name of the focal group group2is the 5-char name of the comparison group

80. SAS programs: gaps  gaps.sas  output:inverse CDF for comparison pairs ICDFr4__03group1group2  mean NAEP-State gap differences and SEs by regions of the percentile distribution DiffGapsMINtoMAXgroup1group2R4__03.XLS DiffGapsMINtoMEDgroup1group2R4__03.XLS DiffGapsMEDtoMAXgroup1group2R4__03.XLS DiffGapsMINtoQ1_group1group2R4__03.XLS DiffGapsQ1_toQ3_group1group2R4__03.XLS DiffGapsQ3_toMAXgroup1group2R4__03.XLS  gaps.sas  output:inverse CDF for comparison pairs ICDFr4__03group1group2  mean NAEP-State gap differences and SEs by regions of the percentile distribution DiffGapsMINtoMAXgroup1group2R4__03.XLS DiffGapsMINtoMEDgroup1group2R4__03.XLS DiffGapsMEDtoMAXgroup1group2R4__03.XLS DiffGapsMINtoQ1_group1group2R4__03.XLS DiffGapsQ1_toQ3_group1group2R4__03.XLS DiffGapsQ3_toMAXgroup1group2R4__03.XLS

81. SAS programs: gaps  gaps.sas  output:population profiles STATE_PV_03.gif state achievement profile STATE_BW_03.gif state achievement profile NAEP_PV_03.gif NAEP achievement profile NAEP_BW_03.gif NAEP achievement profile NAEP_STATE_PV_03.gif NAEP/state gap profile NAEP_STATE_BW_03.gif NAEP/state gap profile d  gaps.sas  output:population profiles STATE_PV_03.gif state achievement profile STATE_BW_03.gif state achievement profile NAEP_PV_03.gif NAEP achievement profile NAEP_BW_03.gif NAEP achievement profile NAEP_STATE_PV_03.gif NAEP/state gap profile NAEP_STATE_BW_03.gif NAEP/state gap profile d

82. SAS programs: gaps  gaps.sas  output:population profiles NAEP_PV_03.gif NAEP achievement profile  gaps.sas  output:population profiles NAEP_PV_03.gif NAEP achievement profile

83. SAS programs: gaps  gaps_g.sas  plot subpopulation profiles and place them on a four-panel template to include in report  macro %pop_profileset SAS/Graph options  macro %plot_gapsplot graphs using options  macro %createtemplatecreate four-panel template  macro %replaygapsplace graphs in template  gaps_g.sas  plot subpopulation profiles and place them on a four-panel template to include in report  macro %pop_profileset SAS/Graph options  macro %plot_gapsplot graphs using options  macro %createtemplatecreate four-panel template  macro %replaygapsplace graphs in template

84. SAS programs: gaps  gaps_g.sas

85. SAS programs: trends  trends.sas  compute difference between state and NAEP and respective standard errors  output data file trends_sy including both NAEP and NAEP state standard measures  trends_r.sas  compute gains and respective standard errors  output data file summary_s  trends.sas  compute difference between state and NAEP and respective standard errors  output data file trends_sy including both NAEP and NAEP state standard measures  trends_r.sas  compute gains and respective standard errors  output data file summary_s

86. SAS programs: trends  trends_g.sas  plot NAEP and state assessment trends by grade and place them on a two-panel template to include in report  compute t for testing significance of differences in gains between NAEP and state assessment  trends_g.sas  plot NAEP and state assessment trends by grade and place them on a two-panel template to include in report  compute t for testing significance of differences in gains between NAEP and state assessment

87. SAS programs: correlations  correlation.sas  compute NAEP-state correlations and standard errors  macro %corrs(standard,s,g,y,group) output CorrsY_standard_groupsgy file with state standard  correlation.sas  compute NAEP-state correlations and standard errors  macro %corrs(standard,s,g,y,group) output CorrsY_standard_groupsgy file with state standard RtR4032 correlation0.600.730.43 standard error0.110.060.10

88. SAS programs: distribution  distribution.sas  create file with plausible value frequency distribution output distribution_sgy file  distribution.sas  create file with plausible value frequency distribution output distribution_sgy file

89. SAS programs  all programs and data files are available for download at http://www.schooldata.org/reports.asp including files with the imputed scale scores for excluded students we used in the report  all programs and data files are available for download at http://www.schooldata.org/reports.asp including files with the imputed scale scores for excluded students we used in the report

90. NAEP State Analysis Project  American Institutes for Research  Victor Bandeira de MelloVictor@air.org  Don McLaughlinDMcLaughlin@air.org  National Center for Education Statistics  Taslima RahmanTaslima.Rahman@ed.gov  American Institutes for Research  Victor Bandeira de MelloVictor@air.org  Don McLaughlinDMcLaughlin@air.org  National Center for Education Statistics  Taslima RahmanTaslima.Rahman@ed.gov