Evaluation Data Compiler Better, Faster, Cheaper Selection Lawrence F. Alwin, CPA Texas State Auditor NSAA Annual Meeting 2004

Evaluation Data Compiler Automated decision support tool Collects, organizes, analyzes, reports quantitative and qualitative data Reduces evaluation time Ensures objectivity Enhances decision quality Minimizes probability of award protest

Service Impact and Financial Outcomes Saved $41,000 - $80,000 per evaluation by: l Standardizing evaluation process l Accelerating evaluation schedules l Eliminating all paper Additional savings from avoiding protests

Usefulness and Success Simplifies evaluation process Rated very useful by users l Cleanest, fastest way to evaluate responses l If success scale is 1 to 5, the Compiler is a procurements totaling $516 million – no award protests

Innovative Features Use for any criteria-based evaluation Processes up to 360,000 scores and 360,000 comments References and cross-references criteria Uses any scoring or weighting design Eliminates need for evaluators to be in same location (virtual evaluation tool)

Built-in Controls Eliminates subjectivity of using only ranking Automates and locks out all calculations Validates score entry Statistically tests scores Examines evaluator consistency/collusion Flags evaluator disagreements Creates fully automated audit trail

Sample Summary Report

Built-in Controls Eliminates subjectivity of using only ranking Automates and locks out all calculations Validates score entry Statistically tests scores Examines evaluator consistency/collusion Flags evaluator disagreement Creates fully automated audit trail

Reporting and Documentation Reports on up to 1200 criteria Creates easy-to-use summary reports Sorts and reports scores and comments by respondent, evaluator, team or criterion Permanently links scores and comments Permanently locks and stores data for archive

Data Compiler Future Web-based version Ready for alpha testing Beta test site selected Available late 2004