1. Ryan Kardell WFO Springfield

2.  Purpose of the Database  Data Sources  User Guide  Formulas Used for Scoring

3.  Track aviation performance metrics for the purpose of determining the forecaster of the quarter  Provide a focal point tool that… can weigh forecast elements based on highest impact events is user friendly and easy to maintain is secure ensuring that integrity is preserved allows for office scores to be displayed in a way that will ensure privacy among forecasters

4.  Statistics used are from the first 6 hours of the TAF period. All elements that can potentially have an adverse effect on flight operations are scored.  IFR statistics are taken from “Stats on Demand” website http://140.90.19.120/services/public/login.aspx?Ret urnUrl=%2fcontent%2fpm%2findex.aspx  All other statistics are obtained from “Aviation Verify”

5. This program uses macros to perform many of it’s functions. Follow these steps to enable macros.

6. This button allows authorized users to edit the database. This button will display the raw forecast performance values. This button will display the performance values as percentage scores. This button will show the final scores with the weighting factors set by the focal point. This button will output the scores to the specified webpage.

7. Start by clicking the Edit Database button.

8. If you see this, you need to add your login name (format - john.doe) to the list in the Admin tab.

9. Enter the password to unlock the list for editing. The default password is “metrics”.

10. Use these steps to protect the worksheet with a new password once you have updated the list.

11. Now try the Edit Database button again.

12. Notice that the columns expanded to allow more forecasters to be added. Also, data in underlined italics are automatically calculated for you. This button will clear all the performance values. This is the path where the web report will be saved. These are the weight values assigned to each performance factor. These are secret ID numbers known only to you and the individual forecasters. Click the Save Database button when finished editing the data.

13. On the Raw Performance tab, the data has not yet been scored.

14. On the Percentage Score tab, the data has been scored (more detail in the next section).

15. Scores with weighting factors included Scores with opportunity factored in

16. Notice that only the Secret ID numbers are showed on the web report to preserve confidentiality.

17.  Absolute Scores Score simply equals the raw performance  Critical Success Index  Probability of Detection  Inverse Scores Score = 1 – Raw Performance  Ex: 0% False Alarm Rate = 100% score  Ex: 30% Bad Tempo Groups = 70% score  Peer-Based Scores All “Percentage Over” scores  Worst performer receives 0%, best receives 100%, the rest fall in between. Raw-47%-7%28%-14%-12%-64% -1%10% Score18%62%100%54%57%0% 68%80%

18.  Weighted Scores Certain performance factors, selected by the focal point, can be weighted higher with more possible points and therefore more importance. IFR has been weighted heavier in the previous examples.  Opportunity Based Score These scores take the amount of forecast opportunity into account.

19. Notice that Bob did very well with his scores, but had far less weather than the other forecasters. Therefore, his opportunity based rank was much lower.

20.  Your input will be greatly appreciated with regards to… Suggested improvements Bug reports  Please send comments to ryan.kardell@noaa.gov ryan.kardell@noaa.gov