Getting the Most Value for Your Assessment Dollar: Cost Analysis for the New Kentucky Assessment System John F. Olson Assessment Solutions Group CCSSO National Conference on Student Assessment Detroit, MI June 23, 2010

Overview The ASG Cost Model was used to develop the cost of the new Kentucky assessment system components. ASG’s report provided the KDE with a realistic estimate of the cost of the key components of its new 2012 assessment system that can be used for budgeting purposes. Modeling the costs of the new system using the actual design of the new assessment program will provide an accurate estimate of the costs KDE should expect to see when it bids the program in 2010 Assessment Solutions Group Confidential 2

Overview (cont.) ASG model outputs also allowed the KDE to gain a deeper understanding of the costs of the various components of each assessment This provides KDE the ability to make trade-offs in assessment program design prior to competitive bidding in order to develop the highest quality and cost-effective assessment for the budgeted dollars. The ASG Cost Model is a variable input, metric-based calculation output model. Specific assessment program variables are input to the model and applied against cost factors to derive assessment cost Several hundred variables are contained in the model. The cost for the entire assessment is built from the ground up, element by element, function by function. Costs are provided for total assessment, by year, by content area, by grade, and by functional activity. Assessment Solutions Group Confidential 3

Key Program Elements & Assumptions The new 2012 Kentucky assessment system consists of the following elements: Combined CRT/NRT assessments in Math grades 3-8, 11 Reading grades 3-8, 10 Science grades 4, 7, 11 Social Studies grades 5, 8, 11 A writing test grades 4, 6, 8, 10, 12 A writing editing & mechanics exam grades 4, 8, 12 Two forms of each assessment will be developed for year 1 (2012) with a new form developed each subsequent year. Assessment Solutions Group Confidential 4

Key Program Elements & Assumptions Assessments will be given to students during the last two weeks of the school year, as specified in the new state law (May – early June). Test results will be mailed to parents and posted online in August. For this study ASG calculated: Incremental year zero (2011) costs required to develop and field test the items to be used in the operational, year one (2012) administration Ongoing total assessment costs for the years 2012, 2013, and 2014 Assessment Solutions Group Confidential 5

Year 0 Costs Cost by functional area for Year 0 (2011) Note that ASG’s base case assumes year 0 work is priced at a 50% margin, which is typical of “transition year” programs. Year 0 work also is shown at a 30% margin. September 28, 2009 Assessment Solutions Group Confidential 6

2012 Costs Cost for each functional area and cost per student – Year 1 (2012) September 28, 2009 Assessment Solutions Group Confidential 7

2012 Cost by Content Area Total cost by content area – Year 1 (2012) September 28, 2009 Assessment Solutions Group Confidential 8

Cost by Grade Total cost for the Kentucky assessment program by grade (2012 – 2014) Note: grades 4 and 8 are tested more frequently; consumable test books are used for grade 3 September 28, 2009 Assessment Solutions Group Confidential 9

Cost by Content Area & Grade Costs for the Kentucky assessment program by content area & grade for 2012 – September 28, 2009 Assessment Solutions Group Confidential 10

Total Program Costs Total program costs for September 28, 2009 Assessment Solutions Group Confidential 11

Total Costs and Key Metrics by Year Total program costs and key metrics by year for September 28, 2009 Assessment Solutions Group Confidential 12

Summary of Impact of Options/Changes in Costs September 28, Assessment Solutions Group Confidential Chart shows cost savings or increases related to various options/changes to the program for

Summary, Conclusions, & Recommendations ASG performed a special study analyzing the costs of the new Kentucky 2012 assessment system Year 0 (2011) activities – item development and field testing Years 1 – 3 ( ) – full administration of new assessment A vendor performing efficiently is expected to price the year 0 activities at $3.2M and the total program costs for years at $12.5M, $12.7M, and $13.2M respectively Consistent with other NCLB state programs, the key costs are in the item development, scoring, and printing areas ASG performed a number of scenarios to consider what could change the cost picture significantly September 28, 2009 Assessment Solutions Group Confidential 14

Summary, Conclusions, & Recommendations ASG’s analysis finds the total cost and cost per student of the new Kentucky assessment system appear to be in a reasonable range for a high stakes, NCLB-type state assessment. Metrics and benchmarks used in the analysis represent those of an efficient contractor realistically bidding on the program The proposed item release rate and use of NRT components require only one form per year which significantly controls costs. September 28, 2009 Assessment Solutions Group Confidential 15

Summary, Conclusions, & Recommendations ASG recommends that the KDE consider ways to improve the cost-effectiveness and efficiency of the new assessment system in the future. When developing the RFP for the new Kentucky assessment program it will be critical to have a clearly written proposal that calls for the vendor to submit the metrics and specifications outlining their understanding of the program requirements. September 28, Assessment Solutions Group Confidential