Awards and Judging: Developing a Systematic Approach BEST National Conference Auburn University July 21, 2011

Awards and Judging – why? Purpose – to reward students/teams for quality work and effort Provide positive, constructive feedback Outcomes Students understand expectations Students will rise to the level of your expectations Teams improve The level of competition at your hub will improve Students learn more!

A & J operations Judges and the Judging process Training (teachers and judges) Policies and compliance Staffing roles and responsibilities Resources

Resources Awards and Judging Guidelines Published each year in late summer (posted on bestinc website by August 1)Published each year in late summer (posted on bestinc website by August 1) Includes:Includes: Policy Reference DocumentPolicy Reference Document Awards and Judging Policies – Part I (not editable)Awards and Judging Policies – Part I (not editable) Awards and Judging Policies – Part II (editable)Awards and Judging Policies – Part II (editable) Score Sheets (limited editing)Score Sheets (limited editing) Team Demographics FormTeam Demographics Form Awards SpecificationsAwards Specifications Scoring databaseScoring database

Resources Released in Kick Off Packets Awards and Judging Policies – Part IAwards and Judging Policies – Part I Awards and Judging Policies – Part IIAwards and Judging Policies – Part II Score SheetsScore Sheets Team Demographics FormTeam Demographics Form Recommended – list of awards your hub will giveRecommended – list of awards your hub will give

Resources New Electronic Scoring System for Notebook Judging Piloted in 2010 at Souths BEST Regional and Nationals Presentation by Dr. Bob Ashurst on system later this session Will be available this year for hubs to test/use

Staffing Judging Administrator Head Judge Judges Team Captains Scoring Leader 1-2 scoring assts. to double check scores

Staffing Judging AdministratorJudging Administrator Recruits judges Develops judging schedule and secures facilities Distributes appropriate docs to teams Orders trophies/plaques/medals Prepares score sheets Answers judging related questions from teams

Staffing Head JudgeHead Judge May help recruit judges Trains judges Oversees all judging processes during Game week Handles questions and disputes “Bad Cop” Chases down judges who do not complete score sheet

Staffing JudgesJudges Score teams on BEST Award categories and special competition awards Provide constructive feedback for teams

Staffing Team CaptainsTeam Captains Category specific training Follows up with their team throughout the day to answer questions Stays until all score sheets from that category have been submitted and checked Works with Head Judge to resolve problems

Staffing Scoring LeaderScoring Leader Excellent computer skills Enters scores and alerts the Head Judge of problems Scoring AssistantsScoring Assistants Double check math on all score sheets before they are entered into scoring database

Policies Policy Reference Document Addresses Mandatory and Optional Awards Trophy, plaque and medal guidelines BEST Award Guidelines Engineering Notebook Guidelines

Compliance Sizes of trophies and plaques Mandatory Awards Follow A & J Policies Advancement of teams to Regional Collect Team Demographic forms on Game Day and submit compiled data on website following competition

Training Teachers/studentsTeachers/students Summer workshop Kick Off Day break out JudgesJudges Orientation prior to each judged event

Judges & Judging Process Recruiting Training Organizing Game Day logistics

Judges & Judging Process Recruiting (for what?) BEST Award Categories Best Website Award Special Awards on Game Day  Robot Awards (Founders, Photogenic, Robust, Elegant)  Best Team T-shirt  Best CAD Design  Sponsor Awards  Etc.

Judges & Judging Process Recruiting Recruiting – Who? Advocates for program Not directly tied to a team College faculty: STEM, Science/Math Ed, Communications College students (Sr. level undergrad and grad) STEM teachers from non-participating schools Industry Community organizations (Lion’s, Rotary) State Dept. of Ed employees (Tech Ed) Local retired engineers Local/regional workforce development council

Judges & Judging Process Recruiting Head Judge Engineer/Scientist/Tech Professional who understands technical side of BEST Level-headed, calm, long fuse Detail oriented – willing to learn A&J rules and Game Rules Someone you trust to oversee judging process Respected by other judges Team Captains Willing to learn A&J rules and Game Rules Respected by other judges Experience judging

Judges & Judging Process How Many?  As many as possible! Rule of thumb is 2+ times the number of BA teams  Don’t overburden judges with too many assignments  Never turn a judge down based on being full – add more!  Don’t feel obligated to take every judge who submits a form – find out who they are!

Judges & Judging Process How Many?  4 to 5 judges per notebook (drop the lowest score)  each judge scores 6 total  3 judges viewing each presentation  3 judges minimum per table  each judge scores 6-8  2 judges minimum per team on spirit/sportsmanship (“3 rd judge” is ballot)  each judge scores 8-10 teams

Judges & Judging Process How many judges do you need? # BA teams NB Mark. Pres. TableS & SSpecialTotal (min) For each additional non-BA team, add 0.6 judges to Notebooks Assumes NB scored 4 times, 6 NB per judge; Tables scored 3 times, 6 Tables per judge; S&S scored 2 times, 8 teams per judge; only 2 Special Awards to judge (Founders, Robust) Not included: judges for extra awards (Web Page, CAD Design, etc)

Judges & Judging Process Organizing and Training Develop org charts for each judged category and overall – send to judges 2 weeks prior for input Important for teams to be judged by equal # of judges Have 2-3 alternate judges ‘at the ready’ in case of no-shows

Judges & Judging Process Head Judge - Dr. Peter Jones, AU Mechanical Engineering BEST Award Categories NotebooksOral PresentationsTable Displ./Int.Spirit/Sports.Special Awards 8-OctOcotber 910-Oct Thursday (5:30-9)Friday (1-4:30)Sat. (9-3) Shelly AonoMatthew BaccusPaul Cobine (TL)Martha Anne AllisonShelly Aono Bob AshurstPatrick BassJames DickeyChris McDuffie (TL)Bob Ashurst Ashley BeardKirby BrownNicholas KlannGentry SniderBrian Bowman Taha Ben BrahimNancy CappsJonathan McCoryBeth ThompsonMike Fogle Kirby BrownLaRonda Conley (TL)Dion PeoplesEmily VenglarikKeith Phillips James DickeyWilliam JosephsonLauren Salomm Karl Ward David EltonSteve KempfSamantha Sylvest Ginger Ware Katie HutchinsonEmily McGindyTeresa Wall James Westmoreland Trafina JadhavShawn Newman Crystal JohnsonBen Scheierman (TL) William JosephsonKathryn Smith Elisabeth PresslerBeth Thompson Amy RutherfordKarl Ward (TL) Lauren SalommSaturday 9-11:30 Virginia StephensPatrick Bass Beth ThompsonKirby Brown Bonnie WilsonLaRonda Conley (TL) Katherin Glynn Trafina Jadhav (TL) Shawn Newman see specific Amy Rutherford assignments below Bonnie Wilson TL = Team Leader Special Competition Awards Most ElegantMost CreativeMost RobustMost PhotogenicBest T-shirt design Keith PhillipsShelly AonoKeith PhillipsShelly Aono (TL)All Special Ginger WareBob Ashurst (TL)Ginger WareBob AshurstAward Judges Karl Ward (TL)Brian BowmanKarl WardBrian Bowman James WestmorlandMike Fogle James Westmorland (TL)Mike Fogle Table Display Design and Construction Award will be judged by the Table Display Team, with Paul Cobine as acting Team Leader on this award. Blood, Sweat, and Duct Tape Award will be judged by the Most Robust Team, with Karl Ward as acting Team Leader on this award.

Judges & Judging Process Room # 1Room # 2Room # 3Judges Room Team Captain:Ben ScheiermanKarl WardPeter Jones 2:15-2:45 Loachapoka High School Nancy Capps William Josephson Steve Kempf Beth Thompson 2:45-3:15 BrewtechEpiscopal Day School Break Patrick BassMatthew Baccus Nancy Capps LaRonda ConleyEmily MdGindy William Josephson Shawn NewmanArchana Reddy Steve Kempf Kathryn SmithTeresa Wall Beth Thompson 3:15-3:30 Judges Break 3:30-4:00 Dothan Technology Break Matthew Baccus Patrick BassSteve Kempf Shawn Newman Nancy CappsEmily MdGindy Kathryn Smith LaRonda ConleyArchana Reddy Teresa Wall William JosephsonBeth Thompson 4:00-4:30 Beverlye MagnetChambers CountyBreak Nancy CappsPatrick BassMatthew Baccus LaRonda ConleyWilliam JosephsonShawn Newman Steve KempfEmily MdGindyKathryn Smith Archana ReddyBeth ThompsonTeresa Wall 4:30-5:00 LAMP High School Stanhope Elmore High Wetumpka High School Matthew BaccusPatrick BassArchana Reddy LaRonda ConleyShawn NewmanNancy Capps Emily MdGindyTeresa WallSteve Kempf Kathryn SmithWilliam JosephsonBeth Thompson

Judges Training Why? Ensures consistency across hubs Organization with judging process is key to cultivating program advocates When? Prior to each judged event How? Go through training manual with entire group Assign experienced Team Captains to each sub-group and let them break off for more detailed orientation session Stress importance of comments on Score Sheets – promotes student learning

Judges & Judging Process Game Day Logistics Need a designated judges room for training and consultations Color code and label score sheets Create a folder for each judge that contains the score sheets for the teams they were assigned to judge Create one folder per team to place score sheets after they have been processed

Judges & Judging Process Game Day Logistics Submit compiled Notebook scores to Head Game Scorekeeper (wildcard match) Remind Game Scorekeeper that you need rankings immediately following seeding matches for Robot Performance category Give out awards throughout the day Communicate with Game Scorekeeper throughout the day to stay on pace with Game Consult with Judges at lunch to make sure they are pacing correctly Invite judges to game floor after lunch to be recognized as a group

Invite lead teachers to floor and distribute team participation plaques

Invite reps from sponsoring companies to help give out awards during closing ceremony

Judges & Judging Process Game Day Logistics Advancing teams to Regionals (make sure you know prior to Game Day how many teams your hub can advance) The sequence of advancing teams: 1. BEST Award 1 st Place 2. Game 1 st Place 3. BEST Award 2 nd Place 4. Game 2 nd Place 5. BEST Award 3 rd Place 6. Game 3 rd Place 7. BEST Award 4 th Place 8. BEST Award 5 th Place 9. BEST Award 6 th Place

Judges & Judging Process Game Day Logistics Host a brief meeting following Awards Ceremony with Advancing Teams and supply them with information supplied by your Regional Rep Post all award winners on your website asap for media purposes Copy team score sheets (delete judge name) and mail to teams the following week judges on Monday after with results of competition and a heart-felt thank you for their time

Care and Feeding of Judges Provide meals Thurs. night dinner for Notebook judges Sat. breakfast and lunch for Game Day judges Supplies for Judges Room – ear plugs, ibuprofen, hard candy, pens, pencils, calculators, name tags Small gifts of appreciation (with your logo) E.g. portfolios, pens, jump drives, shirts, hats Local Chamber of Commerce or Visitors Bureau may provide gifts Print judge’s name and company/college/school affiliation in program guide and make sure they get a copy (great to show the boss!)

Timeline Feb – Oct Recruit judges April-May Save the Date to potential judges June/July Teacher Training workshop(s) Organize times and locations for judged events August Download A & J docs from BEST website Prepare and copy docs for Kick Off packets/CDs Determine what awards you will give and include in KO packet

Timeline September Kick Off Day – distribute A&J materials; host an A&J break out session for teachers/students; confirm which teams will compete for BEST Award Determine total # of judges needed sign up form (or web link) to potential judges – give a deadline for submission Confirm facilities for judged events Order tables for Team Exhibits Order trophies/plaques/medals Order judges gifts Team sign-up for Presentation time slot Invite judges to Mall Day Determine judging staff Recruit judging alternates

Timeline October Order food for judged events Create org charts with judging assignments judges with assignments, schedule of events, A&J rules, Game summary (2 weeks out) Copy color-coded score sheets and label with team name/# Signage Game Week Finalize assignments Prepare Judges Handbooks Prepare team folders Prepare judging folders with assigned score sheets Pick up trophies/plaques/medals/judges gifts Prepare judging supply box Post signage

Tips for Success Ask ALL judges on Game Day to help with S&S judging – provide a place for them to post their comments Post robot photos labeled by team in Judge’s Room on Game Day Spread out awards – give as many optional awards as you can Provide judges with a handbook (rules, schedules, team stat sheet, sample score sheets, etc.) Give individual medals if your budget allows

Tips for Success Create 1-2 page summary of Game for judges Give “Teacher of the Year” and “Mentor of the Year” plaques if possible Spread judging over 3 days (Thursday- Notebooks, Friday- Presentations, Saturday – everything else) to recruit judges for multiple events During Awards Ceremony, explain that results are preliminary and final results will be confirmed the following week

A & J Don’ts Don’t forget about wildcard match (scorekeeper needs notebook scores) Don’t accuse teacher of wrong-doing without knowing all the facts Don’t give larger trophies than the Regional! Don’t let team participate until they have submitted Team Demographics Form Don’t forget to compile Team Demographic info from your hub and post results at bestinc.org

Team Advancement BRI Board of Directors determines which hubs fall under each regional on an annual basis. Each regional determines the number of teams (T) that can advance from the region. Regionals will determine the number of teams advancing for each hub based on the following formula: N = (H  R)  T N = Number of teams that can advance for a specific hub H = Number of teams in the specific hub R = Number of teams in the region (sum of all teams in all hubs) T = Total number of teams that can advance to regional Each regional announces this figure to hubs in its region no later than the Monday following the last hub’s season kickoff. Hubs may send fewer than the total number of teams allotted for advancement. Each regional may reallocate an unused allotment to another hub.

Questions? Mary Lou Ewald Chair, Awards and Judging Committee (cell) (office)

Next Up Dr. Bob Ashurst Chemical Engineering, Auburn University Electronic Notebook Scoring System