1. http://pdstools.arc.nasa.gov Ames Validator Analytics March 26, 2010 Joshua Ganderson http://pdstools.arc.nasa.gov/pdsWeb/ViewStats.action

2. http://pdstools.arc.nasa.gov Purpose Evaluate Usage – Confirm people are using the tool Determine Benefit – Confirm that the tool is having an impact Provide Metrics – Surface available metrics to MC and HQ Draw Conclusions and Identify Issues or Opportunities

3. http://pdstools.arc.nasa.gov Scope – Collected Data, Surfaced Reports Collected Data What Was Validated And When What Errors Were Found In Validations Reports Validations Per Day – Graphical representation of validations over time General Stats – Total Validations, Max/Min Duration, Bounces, etc Improvement By Error Type – Errors fixed by type across volume validations Error Statistics by Validation and Unique Volumes – num errors, in validations or volumes General Volume Stats - General Stats for last validation Improvement By Volume - Max and min errors and error types across validations Volume Details – All validation info for a given volume across validations

4. http://pdstools.arc.nasa.gov Recently Used as Much as 56 Times in a Day ~1,500 Validations ~80% Validated More than Once Increasing Rate of Validation Evaluate Usage

5. http://pdstools.arc.nasa.gov Millions of Problems Identified Errors Fixed Across Validations Identification of Issues Not Being Fixed Determine Benefit

6. http://pdstools.arc.nasa.gov Certain Problems More Prevalent – ex. Unknown Value, Unknown File, Label Not Indexed. Mostly common errors types expect to see in volumes being assembled. Some Problems Fixed Often – Above, parse errors, errors that occur to infrequently to make a definitive call on. Some Problems Rarely Fixed – ex. Value out of range, Illegal Line Ending, Type Mismatch. While some issues are difficult to fix, it’s unclear why easily fixable problems ignored. No Volume Completely Fixed – Some get close but none get to zero. Disagreement over rules? New values not yet ingested? Node specific housekeeping? Need contact users for more info. Actionable Data – Enough data gathered that worth contacting people doing validation to follow up. Conclusions, Issues And Opportunities

7. http://pdstools.arc.nasa.gov Associate Volumes with Data Producer and Curating Node – Either requires manual association or EN services. Follow Up With Users – Contact users of the tool to determine why some problems not fixed. Track Cancellations – Due to logistical issues, cancellations tracking has not yet been implemented. Surface Problem Severity and Details – Surface more detail about problems. Sorting and Filtering – Allow sorting of all tables. Allow filtering by date range and, if implemented, data producer and curating node. Address Performance – Currently slow and going to get slower. Can address issue by rolling up statistics after each validation in a background process. Next Steps

8. http://pdstools.arc.nasa.gov What else do you want to see? What else is useful to surface to HQ? Alternate visualizations for available data? Willing to log in for better data tracking? Way to establish time saved and benefits? Alternate usage for data? Downloads useful? Questions