Ryan KardellWFO Springfield

Purpose of the Database

Data Sources

User Guide

Formulas Used for Scoring

Track aviation performance metrics for the purpose of determining the forecaster of the quarter

Provide a focal point tool that…

• can weigh forecast elements based on highest impact events

• is user friendly and easy to maintain• is secure ensuring that integrity is preserved• allows for office scores to be displayed in a way

that will ensure privacy among forecasters

Statistics used are from the first 6 hours of the TAF period. All elements that can potentially have an adverse effect on flight operations are scored.

IFR statistics are taken from “Stats on Demand” website• http://140.90.19.120/services/public/login.aspx?

ReturnUrl=%2fcontent%2fpm%2findex.aspx

All other statistics are obtained from “Aviation Verify”

This program uses macros to perform many of it’s functions. Follow these steps to enable macros.

This button allows authorized users to edit the database.This button will display the raw forecast performance values.This button will display the performance values as percentage scores.

This button will show the final scores with the weighting factors set by the focal point.

This button will output the scores to the specified webpage.

Start by clicking the Edit Database button.

If you see this, you need to add your login name (format - john.doe) to the list in the Admin tab.

Enter the password to unlock the list for editing. The default password is “metrics”.

Use these steps to protect the worksheet with a new password once you have updated the list.

Now try the Edit Database button again.

Notice that the columns expanded to allow more forecasters to be added. Also, data in underlined italics are automatically calculated for you.

This button will clear all the performance values.This is the path where the web report will be saved.These are the weight values assigned to each performance factor.

These are secret ID numbers known only to you and the individual forecasters.

Click the Save Database button when finished editing the data.

On the Raw Performance tab, the data has not yet been scored.

On the Percentage Score tab, the data has been scored (more detail in the next section).

Scores with weighting factors included

Scores with opportunity factored in

Notice that only the Secret ID numbers are showed on the web report to preserve confidentiality.

Absolute Scores• Score simply equals the raw performance

Critical Success Index Probability of Detection

Inverse Scores• Score = 1 – Raw Performance

Ex: 0% False Alarm Rate = 100% score Ex: 30% Bad Tempo Groups = 70% score

Peer-Based Scores• All “Percentage Over” scores

Worst performer receives 0%, best receives 100%, the rest fall in between.

Raw -47% -7% 28% -14% -12% -64% -64% -1% 10%

Score 18% 62% 100% 54% 57% 0% 0% 68% 80%

Weighted Scores• Certain performance factors, selected by the

focal point, can be weighted higher with more possible points and therefore more importance. IFR has been weighted heavier in the previous examples.

Opportunity Based Score• These scores take the amount of forecast

opportunity into account.

Notice that Bob did very well with his scores, but had far less weather than the other forecasters. Therefore, his opportunity based rank was much lower.

Your input will be greatly appreciated with regards to…

• Suggested improvements• Bug reports

Please send comments to ryan.kardell@noaa.gov