1. Getting the Most Value for Your Assessment Dollar: Cost Analysis for the New Kentucky Assessment System
John F. Olson
Assessment Solutions Group
CCSSO National Conference on Student Assessment
Detroit, MI
June 23, 2010

2. Assessment Solutions Group www.assessmentgroup.org Confidential
Overview
The ASG Cost Model was used to develop the cost of the new Kentucky assessment system components.
ASG’s report provided the KDE with a realistic estimate of the cost of the key components of its new 2012 assessment system that can be used for budgeting purposes.
Modeling the costs of the new system using the actual design of the new assessment program will provide an accurate estimate of the costs KDE should expect to see when it bids the program in 2010
Assessment Solutions Group Confidential

3. Assessment Solutions Group www.assessmentgroup.org
Overview (cont.)
ASG model outputs also allowed the KDE to gain a deeper understanding of the costs of the various components of each assessment
This provides KDE the ability to make trade-offs in assessment program design prior to competitive bidding in order to develop the highest quality and cost-effective assessment for the budgeted dollars.
The ASG Cost Model is a variable input, metric-based calculation output model.
Specific assessment program variables are input to the model and applied against cost factors to derive assessment cost
Several hundred variables are contained in the model.
The cost for the entire assessment is built from the ground up, element by element, function by function.
Costs are provided for total assessment, by year, by content area, by grade, and by functional activity.
Assessment Solutions Group
Confidential

4. Key Program Elements & Assumptions
The new 2012 Kentucky assessment system consists of the following elements:
Combined CRT/NRT assessments in
Math grades 3-8, 11
Reading grades 3-8, 10
Science grades 4, 7, 11
Social Studies grades 5, 8, 11
A writing test grades 4, 6, 8, 10, 12
A writing editing & mechanics exam grades 4, 8, 12
Two forms of each assessment will be developed for year 1 (2012) with a new form developed each subsequent year.
Assessment Solutions Group
Confidential

5. Key Program Elements & Assumptions
Assessments will be given to students during the last two weeks of the school year, as specified in the new state law (May – early June).
Test results will be mailed to parents and posted online in August.
For this study ASG calculated:
Incremental year zero (2011) costs required to develop and field test the items to be used in the operational, year one (2012) administration
Ongoing total assessment costs for the years 2012, 2013, and 2014
Assessment Solutions Group
Confidential

6. Assessment Solutions Group www.assessmentgroup.org Confidential
Year 0 Costs
Cost by functional area for Year 0 (2011)
Note that ASG’s base case assumes year 0 work is priced at a 50% margin, which is typical of “transition year” programs. Year 0 work also is shown at a 30% margin.
September 28, 2009
Assessment Solutions Group Confidential

7. Assessment Solutions Group www.assessmentgroup.org Confidential
2012 Costs
Cost for each functional area and cost per student – Year 1 (2012)
September 28, 2009
Assessment Solutions Group Confidential

8. Assessment Solutions Group www.assessmentgroup.org Confidential
2012 Cost by Content Area
Total cost by content area – Year 1 (2012)
September 28, 2009
Assessment Solutions Group Confidential

9. Assessment Solutions Group www.assessmentgroup.org Confidential
Cost by Grade
Total cost for the Kentucky assessment program by grade (2012 – 2014) Note: grades 4 and 8 are tested more frequently; consumable test books are used for grade 3
September 28, 2009
Assessment Solutions Group Confidential

10. Cost by Content Area & Grade
Costs for the Kentucky assessment program by content area & grade for 2012 – 2014.
September 28, 2009
Assessment Solutions Group Confidential

11. Total program costs for 2011 - 2014
September 28, 2009
Assessment Solutions Group Confidential

12. Total Costs and Key Metrics by Year
Total program costs and key metrics by year for
September 28, 2009
Assessment Solutions Group Confidential

13. Summary of Impact of Options/Changes in Costs
Chart shows cost savings or increases related to various options/changes to the program for
September 28, 2009
Assessment Solutions Group
Confidential

14. Summary, Conclusions, & Recommendations
ASG performed a special study analyzing the costs of the new Kentucky 2012 assessment system
Year 0 (2011) activities – item development and field testing
Years 1 – 3 ( ) – full administration of new assessment
A vendor performing efficiently is expected to price the year 0 activities at $3.2M and the total program costs for years at $12.5M, $12.7M, and $13.2M respectively
Consistent with other NCLB state programs, the key costs are in the item development, scoring, and printing areas
ASG performed a number of scenarios to consider what could change the cost picture significantly
September 28, 2009
Assessment Solutions Group Confidential

15. Summary, Conclusions, & Recommendations
ASG’s analysis finds the total cost and cost per student of the new Kentucky assessment system appear to be in a reasonable range for a high stakes, NCLB-type state assessment.
Metrics and benchmarks used in the analysis represent those of an efficient contractor realistically bidding on the program
The proposed item release rate and use of NRT components require only one form per year which significantly controls costs.
September 28, 2009
Assessment Solutions Group Confidential

16. Summary, Conclusions, & Recommendations
ASG recommends that the KDE consider ways to improve the cost-effectiveness and efficiency of the new assessment system in the future.
When developing the RFP for the new Kentucky assessment program it will be critical to have a clearly written proposal that calls for the vendor to submit the metrics and specifications outlining their understanding of the program requirements.
September 28, 2009
Assessment Solutions Group
Confidential