Talking Stats with Firemen Highlights from StatCom's first year

● What is StatCom? Why should you get involved? ● Project #1: Shoreline Fire Department ● Project #2: ACLU of Washington State ● Q & A Outline

What is StatCom? ● Student-run organization which provides statistical consulting services to non-profit community and governmental groups ● Serves clients external to the university community who do not have the resources to hire professional statistical consultants

A brief history ● 2001: StatCom founded at Purdue University ● 2005: Purdue seeks to expand StatCom, contacts other universities. Assaf Oron decides to create StatCom UW ● 2006: StatCom featured as Amstat News cover story. First joint meeting of StatCom chapters at JSM, Seattle. ● 2007: 5-6 active chapters (Purdue, UW, Cornell, Michigan, Ohio State, UHasselt (Netherlands))

Services ● Study design ● Questionnaire design ● Data analysis ● Statistical literature review ● etc

UW ● Members: – Students from Stat/Biostat, Genome Sciences – Recent graduates (Epi, Math) currently working at UW and Microsoft Research ● Meetings: – Roughly monthly, depending on member availability and workload ● Structure: – Non-hierarchical, decisions reached by consensus

UW (cont'd) ● Project teams – Usually involve 3-4 StatCom members – Often led by a more senior/experienced student – Autonomous, report on progress at general StatCom meetings

What's in it for me? ● Real-world consulting experience ● Potential for publications ● Good karma

Example Projects

Project: Shoreline Fire Dept. ● Serves a 13-square mile area with a population of 50,000 ● 100 employees ● Handles 8,000 calls annually for – fire suppression – technical rescue – emergency medical and advanced life support – fire prevention, education, and inspection – etc

SFD Project: Initial problem ● How to compare peer evaluation ratings of candidates for promotion to paramedic from different station houses? Paramedics Station House #1Station House #2Station House #3

SFD Project: Initial solution ● Fit a linear mixed effects model to account for an individual-specific “rater” effect ● OK, but... the peer evaluation system as a whole needed an overhaul ● So... StatCom took on a larger challenge: Design and help implement an improved peer evaluation system for the Shoreline Fire Department

SFD Project: Problems to fix ● Questionnaire: Peer eval questionnaire much too long ● Rater assignment: Employees volunteer to rate candidates of their choosing ● Analysis: Possibility for “halo” effect by station

SFD Project: Solutions ● Questionnaire: Run psychometric analysis tools on data from last year's evaluations to identify/eliminate questions which – are redundant – correlate poorly with total score – have small variance

SFD Project: Solutions (cont'd) ● Rater assignment: Introduce two-stage rating process: – Stage 1: Raters identify candidates whose abilities they are familiar with – Stage 2: Raters are matched to candidates via semi-random algorithm and asked to complete full questionnaire ● Analysis: Linear mixed effects model as described above

SFD Project: Current status ● Two-stage design proposal adopted ● Shortened questionnaire adopted in principle, still a work in progress ● Design being rolled out for current round of evaluations of Drivers ● First-stage (matching) data in within two weeks. Second-stage (questionnaire) data in for analysis at end of October