2. 2013 EFOA TPE Program

3. 4 Topics to Address
Evaluation of last year’s program and specific recommendations for improvement.
Board decisions this past off-season on changes to the program.
Sample 2013 TPE Official’s Evaluation.
2013 TPE Program Summary.
Please hold your questions until the end of the presentation

4. 2012 Evaluation and Recommendations
In general, it was felt that the TPE evaluation system was an overall improvement from the 2011 to the 2012 seasons. With the additional tools available to the Association (HD cameras and the HUDL program), the evaluation system can be much improved and a valuable tool for all officials working to become the best officials that they can become. Some areas for improving the system are…..

5. 1. EVALUATORS
2012 evaluations were often lacking as to providing information to officials as to how they “graded out” during specific games. If the overall goal of the TPE evaluation system is to give officials direct feedback as to their on-field performance, we did not achieve the goal of providing specific areas where officials could improve or positive comments on what they were doing well.

6. 2. EVALUATION FORMAT
it is suggested that EFOA’s format be changed so as to mirror the state’s evaluation program (RTO program). By closely matching the state, we will be able to better meet the state’s criteria.

7. 3. VIDEO
With access to the HUDL program, we will have excellent videos for both the TPE evaluators to use to complete their assigned evaluations and for officials to access to better be able to see areas where improvement can be made and also the areas that were done well. All members would have access to their individual games within days of their individual game dates. Videos would also be used in training.

8. 4. NUMBER OF EVALUATIONS
While up to 6 games could be taped per week, their would be no need to have the TPE evaluate all those games. 3 or 4 games a week evaluated by TPE evaluators would allow the association to evaluate all varsity and some intermediate officials during the season. Officials would be able to view all (evaluated and non-evaluated) games.

9. 5. VIDEO QUALITY
During this past season, the general quality of the tapes were poor as the videographers were learning “their trade.” It is also noted that there was a change to using the HUDL program during the season-another learning curve. For the 2013 season, videographers will need to be trained prior to the start of the season. Scrimmages would be perfect for this activity. Each videographer was paid $30 per game during the 2012 season, and it is recommended that they continue to paid $30 per game but only $20 for the second game of any double-header that they are assigned to do. They do not incur any additional travel for the second game, thus the $20 figure.

10. 6. VIDEO EVALUATIONS
We will need to change the way evaluations by the TPEs are done. It will no longer be mandated that they attend the games they are assigned to evaluate as they will be able to do evaluations from home via HUDL on their computers. Their specific comments are the areas where improvement is needed(lesser scores) and the areas where they did well and met our criteria. Officials will be able to see exactly what the TPEs found as areas of strength and areas needing improvement-video evidence!

11. 7. EVALUATOR INDEPENDENCE
This past year, it was an expectation that the TPEs attend post game conferences and possibly pre game conferences. Under HUDL, the TPEs will NOT attend either conference and thus crews will generally not know who the evaluator is. The evaluators need to be and must be totally independent and free from any bias from the crews.

12. 8. SCORES AND RANKINGS
True rankings should be published for all members to view as this is the only way that they can see specifically where they rank among the other officials. The excuse that some officials may become upset once they see the rankings is NOT a valid argument. How else will officials be able to improve themselves and “move up the ladder”? The new HUDL and evaluation program will make it much easier when it comes to assigning state playoff games. It also creates a completely transparent system for all members when it comes time to select state playoff officials. Thus, it is suggested that the EFOA board post the rankings for each position in the order where officials stand from #1 to last. Publishing the rankings is mandatory feedback for the membership.

13. 9. ACCOUNTABILITY
A weakness of past evaluation programs was the total lack of any kind of feedback to the membership. How can you improve as an official when you do not know in what areas you need to improve as per the specific mechanics for their individual positions. Also, officials need to receive positive feedback for the things that they are doing well. The HUDL program will specifically point out to officials where improvement is needed and also where they were proficient as it will be pointed out with specific play references on the tape. It is difficult if not impossible to refute video evidence of the “good, bad and ugly”. If specific areas of concern are noted through the HUDL evaluation process, and those areas are either not improved or are marginally improved, then the board should address those areas of concern with the official involved and take appropriate action to remedy the deficiency.

14. 2. Board Decisions for the TPE 2013 Program
10 specific decisions the board made this past off-season to improve the TPE program for 2013.

15. 1. Who will normally have access to view game videos in HUDL?
Should it be just the crew of the game plus the assigner and Evaluation board member? Or should everyone see every game? (It goes without saying that if there is an issue in a game that was videoed and this issue needs board attention, then the board would have access to that game.)
Given the past experience with some of our officials and the way they compare their results to those of other officials; it would be easy to imagine a situation where some individuals would use the game videos inappropriately. E.g.: “Did you see what Joe Blow did in his game last week on clip #33? “
We could also let everyone see every video.
DECISION: B - Everyone

16. 2. What Categories should be on the evaluation?
The WOA has adopted a rubric or matrix for officials that will be used across the state. This will be part of the training component of the RTO Program. Since this rubric will be the baseline for the state, it would make sense to have our TPE results for officials follow the rubric. The categories on the rubric are:
Professionalism [15]: Rule application, flag & whistle use, communication on Penalties
Communication [20]: Assignor, game management, coaches, officiating crew, during the game, signals, whistle/signals
Game management [15]: Decisiveness, Tempo, Works with chain crew & timers, Sideline control
Positioning and mechanics [30]: dead ball officiating, positioning, mechanics, forward progress spots, special positioning (field goals, punts, kickoffs), hustle
Rule Application/enforcement[20]: Rule application, Flag & whistle, Communication on penalties.
Maximum is 100 points – scored automatically by computer.
DECISION: Use the above 5 categories with the further breakouts as specified in the WOA rubric to be the TPE categories for comment and scoring.

17. 3. How will the results be used?
In past years, the TPE results had three uses for the association: First, for state playoff and other special games, it helped determine the assignments (Prior and current year). Second, at the end of the year, the board could and did, make changes to the ranking for officials (100, 125, 150, etc) based on that year’s performance. And third, at the end of the year, local playoff assignments were made based on the TPE scores. There may be other uses or the board may decide not to use the TPE scores for one or more of these. The scoring system for TPEs must be designed to meet the projected uses for the TPE scores.

18. 3. How will the results be used?
DECISION: Use TPE results for state tournament and other special game selection.
DECISION: Use the TPE scores for adjusting the rankings for officials:
Average score
=> 100/200 level
=> 125/225 level
78-84 => 150/250 level
0-77 => board review required.
DECISION: Use TPE scores for making local playoff assignments.
DECISION: Other planned uses for the TPE Scores include training assignments and breakout group leaders.

19. 4. What do the officials see from the TPE program?
Current practice has been to give them the comments only as that contains the things they need to work on to improve. With the use of HUDL this year, the TPE output for the officials will also contain the clip numbers and comments which should be even more helpful to officials. In the past, TPE actual scores were never provided to officials as the scores alone meant nothing (you have no context to compare to). The possible options are:
Show the officials their scores. This could cause major problems with officials comparing their results with others. Some officials would become argumentative about their score compared to others.
At the end of the year, show the officials their score and the highest and lowest score from their position. This would give them a good idea of where they fall in the actual rankings. However, officials could still abuse the scores when comparing them with others.
Publish the final scores for each varsity official by position. This would let everyone know where they stand, but the results could still be abused.
Just publish the comments and notes for each play as has been done in the past.
DECISION: A - Show officials their total score for each evaluation.

20. 5. Pay for Evaluators
In the past, evaluators attended the game and were paid a normal game fee and travel fee. Now with the use of HUDL, evaluators will need to do the evaluation in front of their computer. It is recommended that the TPE pay be $50 per game.
DECISION: Pay evaluators $50 per game.

21. 6. Pay for Cameramen
In the past, the camera men were paid $30 per game to pick up the camera before the game, video the game, and then return the camera. With the additional video equipment, the cameras will be assigned to cameramen. They will retain the camera during the season and only need to exchange the game media for blank media the day after their games. We will also be able to video both games of the double headers. It is recommended that the cameramen be paid $30 for a game and if they are shooting a second game at the same location, that they receive $20 for that second game.
DECISION $30 video game fee and the $20 second video game fee is approved.

22. 7. Evaluations used in ranking
In the past, officials may receive multiple evaluations. Excluding blocked officials, the number of evaluations in the season for a varsity official ranged from 1 to 6. This was due to the problems trying to get everyone scheduled into the games where they were needed. The assignor’s first priority has to be to get the games properly covered and only secondarily, to see that the TPE assignments were level. Because of the imbalance in the number of evaluations an official could receive, not all evaluations were taken in to account for the scoring. The officials however, could view all of their evaluations.
DECISION: Use the highest three evaluations per official.

23. 8. Assignments of Evaluators
It is difficult for anyone to try to even out the TPE assignments. Varsity officials needed two, those candidates for state usually got three, intermediates got one and those intermediates nearing promotion to varsity usually got two. In the past, this was done by close coordination between the assignor and the TPE program administrator using a computer program to try to balance the assignments.
DECISION: Assignments will have priority, the TPE board representative will coordinate with the Assigner on getting the correct number of assignments as much as possible.

24. 9. Anonymity
In the past, officials knew who was evaluating them and evaluators knew who the officials were. There were some concerns of personal biases maybe creeping into the evaluations. Now, since evaluators will use HUDL and will not attend the games, there is no need for the evaluators to know who they are evaluating nor is there any need for the officials to know who evaluated them. In fact, if there is a substandard evaluation given, other evaluators may be asked to review it prior to it being finalized.
DECISION: The board supports anonymity for both the evaluators and the officials.

25. 10. Complaints
In the past, any complaints by officials about evaluations did not have a clear cut process to go through. A process of handling questions and concerns of officials is needed to make the system better resolve any issues. It is recommended that:
Officials with questions or concerns be directed to submit them to the board member for evaluations.
The board member for evaluations will investigate and get an answer to any questions and concerns and forward that to the official.
If the official is not satisfied with the answer given, he may then appeal to the board.
The board will hear the concern and also the response from the board member for evaluations and then make a final decision.
DECISION: The above conflict resolution process is approved.

26. 3. Sample 2013 TPE Evaluation
This is only a sample. The clip numbers refer to an actual 2nd round playoff game that was RTO’d by the state last year. While Tom Vaughn’s name is in this sample, it is not actually his evaluation. The comments with the clip numbers are the actual comments from that state RTO’d game.

30. 4. 2013 TPE Program Summary 3 TPE Evaluators for 2013
Bob Ollikainen
Bill Obley
Jim Schaefer
We will use the same 5 categories that the WOA uses for the RTO program. Scores will be tallied by the computer based on the clip comments of the evaluators.

31. TPE Program Summary
We have 4 HD video camera systems that we plan to use at:
McKenzie Stadium
Kiggins Bowl
Doc Harris Stadium
District Stadium
Columbia River HS
Ridgefield HS
Washougal HS

32. TPE Program Summary
We will try to get all varsity officials a minimum of three evaluations per year. If they get more, we will use only the highest three for ranking. We will try to get intermediates as many evaluations as can be fit into the assigning schedule. Priority in scheduling will always be to meet the Assigner’s needs first.

33. 4. 2013 TPE Program Summary TPE scores will be used for the following:
To move an official up or down in the rankings (200, 125, 150, 200, 225, 250).
To qualify for state championship tournament games and any special out of area games.
All locally assigned playoff games.

34. Questions?????