Transcript Talking Stats with Firemen: Highlights from StatCom's first

Talking Stats with Firemen
Highlights from StatCom's first year
Outline
●
What is StatCom? Why should you get
involved?
●
Project #1: Shoreline Fire Department
●
Project #2: ACLU of Washington State
●
Q&A
What is StatCom?
●
●
Student-run organization which provides
statistical consulting services to non-profit
community and governmental groups
Serves clients external to the university
community who do not have the resources to
hire professional statistical consultants
A brief history
●
●
●
●
2001: StatCom founded at Purdue University
2005: Purdue seeks to expand StatCom,
contacts other universities. Assaf Oron decides
to create StatCom chapter @ UW
2006: StatCom featured as Amstat News cover
story. First joint meeting of StatCom chapters
at JSM, Seattle.
2007: 5-6 active chapters (Purdue, UW, Cornell,
Michigan, Ohio State, UHasselt (Netherlands))
Services
●
Study design
●
Questionnaire design
●
Data analysis
●
Statistical literature review
●
etc
StatCom @ UW
●
●
Members:
–
Students from Stat/Biostat, Genome Sciences
–
Recent graduates (Epi, Math) currently working at
UW and Microsoft Research
Meetings:
–
●
Roughly monthly, depending on member availability
and workload
Structure:
–
Non-hierarchical, decisions reached by consensus
StatCom @ UW (cont'd)
●
Project teams
–
Usually involve 3-4 StatCom members
–
Often led by a more senior/experienced student
–
Autonomous, report on progress at general StatCom
meetings
What's in it for me?
●
Real-world consulting experience
●
Potential for publications
●
Good karma
Example Projects
Project: Shoreline Fire Dept.
●
Serves a 13-square mile area with a population
of 50,000
●
100 employees
●
Handles 8,000 calls annually for
–
fire suppression
–
technical rescue
–
emergency medical and advanced life support
–
fire prevention, education, and inspection
–
etc
SFD Project: Initial problem
●
How to compare peer evaluation ratings of
candidates for promotion to paramedic from
different station houses?
Paramedics
Station House #1
Station House #2
Station House #3
SFD Project: Initial solution
●
●
●
Fit a linear mixed effects model to account
for an individual-specific “rater” effect
OK, but... the peer evaluation system as a
whole needed an overhaul
So... StatCom took on a larger challenge:
Design and help implement an improved
peer evaluation system for the Shoreline
Fire Department
SFD Project: Problems to fix
●
●
●
Questionnaire: Peer eval questionnaire
much too long
Rater assignment: Employees volunteer to
rate candidates of their choosing
Analysis: Possibility for “halo” effect by
station
SFD Project: Solutions
●
Questionnaire: Run psychometric analysis
tools on data from last year's evaluations
to identify/eliminate questions which
–
are redundant
–
correlate poorly with total score
–
have small variance
SFD Project: Solutions (cont'd)
●
●
Rater assignment: Introduce two-stage
rating process:
–
Stage 1: Raters identify candidates whose
abilities they are familiar with
–
Stage 2: Raters are matched to candidates via
semi-random algorithm and asked to complete
full questionnaire
Analysis: Linear mixed effects model as
described above
SFD Project: Current status
●
●
●
●
Two-stage design proposal adopted
Shortened questionnaire adopted in principle,
still a work in progress
Design being rolled out for current round of
evaluations of Drivers
First-stage (matching) data in within two
weeks. Second-stage (questionnaire) data in
for analysis at end of October