Transcript Addressing the Limited Data Dilemma

Addressing the Limited Data Dilemma
Non-Traditional Sources of Safety Data
Presented by: Terecia Wilson
Director of Safety
March 2006
Traditional Data Collection
 Crash Data
 Driver Records
 Vehicle Records
 Roadway Inventory Data
 Citation Data / Adjudication Data
 EMS Run Reports
March 2006
Alternative Data Collection






Road Safety Audits
Opinion Surveys
Observational Surveys
Program Assessments
Interviews
SCP Forum (Data Guide)





CODES Data
Data Cube
Complaint Files
Program Evaluations
Professional Judgment
March 2006
What is a Road Safety Audit?
A formal examination of an existing or
future road or traffic project, or any project
which interacts with road users, in which
an independent, qualified examiner
reports on the project’s crash potential
and safety performance.
March 2006
Why Do RSAs?
 Proactive approach to highway safety.
 Widely used in other countries - highly
effective.
 Possible even with limited resources.
 Supports Strategic Plan Goal of improving
safety.
March 2006
When Can an Audit Be Done?
 Future Roads
 Stage 1 – Planning
 Stage 2 – Preliminary Design
 Roads Under Construction
 Stage 3 – Final Design
 Stage 4 – Pre-opening
 Existing Roadways
 Stage 5 – Operations Review
March 2006
Types of Audit Data
Information collected for use in audit
 Traffic counts
 Public hearing information
 Detailed designs
 Crash Data
March 2006
Types of Audit Data
Information collected from audit report
 Prioritized findings and recommendations
 Multi-disciplinary report
 Comments from special road user groups
March 2006
RSA Team Participants
 Geographic Representation from across state.
 Representation from various disciplines: Traffic
Engineering, Planning, Engineering, Construction,
Pre Construction, Special Interest groups.
 Special Interest groups (ie, local law enforcement,
EMS, Disabilities and Special Needs, AARP, etc.)
March 2006
RSA Follow Up
 Conduct follow up study to determine impact on
traffic safety 3 years after final audit report.
 Examine traffic collision data 3 years before and 3
years after audit.
 Include RSA team to assist in evaluation as
needed.
March 2006
Method
Advantages
Disadvantages
Surveys
Mail
· Can survey many people
· Not time-consuming
· Difficult to get much detail
· Sometimes difficult to get correct addresses
· Relatively inexpensive
· May be problems with interpreting questions
· Everyone gets the same instrument
· Sometimes a problem getting surveys completed & returned
· Objective Interpretation
Telephone
Interview
· Able to ask for more detail when needed
· Sometimes difficult reaching people
· Everyone gets the same instrument
· Lack of anonymity
· Researcher knows how people are interpreting questions
· Time-consuming time, can limit sample size
· Able to ask for more detail when needed
· Subjective interpretation
· Provide detailed data
· Can be expensive
· Can be difficult to analyze
Focus Group
· Researcher knows how people are interpreting questions
· Group setting may inhibit some individuals from providing
information
· Able to ask for more detail when needed
· Sometimes hard to coordinate multiple schedules
· Able to interview multiple people at one time, thus, more
cost-effective
· Responses from one person provide stimulus for other people
· Responses from one person provide stimulus for other
people
Observation
· Objective interpretation
· Time-consuming
· Low burden for people providing data
· Some items are not observable
· Can be expensive
· Participant behavior may be affected by observer presence
March 2006
Opinion Surveys
 Telephone Surveys
 Random Digit Dialing
 Unbiased
 Varied Demographics (Age, Race, Education,
Income)
 Representative sample of licensed drivers
March 2006
Opinion Surveys
 Examples of Telephone Surveys
 Follow-up evaluation of public information
campaigns
 Determine public opinion on safety issues
• Primary Safety Belt Law
• .08 BAC
• Motorcycle Helmet Legislation
 Determine changes in public opinion on safety
issues (Baseline/Ongoing)
March 2006
Opinion Surveys
Focus Groups
 Used for more qualitative information
 Used in a wide range of applications in nearly
every field of market research.
 Encourages participants to express their feelings
freely and without inhibitions
 Probes more deeply into issues in a relaxed,
uncontrolled atmosphere.
March 2006
Opinion Surveys
Focus Groups
 Discussion leaves participants feeling, justifiably,
that their opinion is important
 Gives clients a clearer perspective on how their
customers feel and why they feel that way.
 Information gained may lead to changes in
program and product development and
implementation.
 Include demographic representation of target
population.
March 2006
Opinion Surveys
 Used as part of dispute resolution, or to solicit
opinions before making significant changes.
 Useful in developing consensus when
community opinion is not immediately
obvious in normal discussion.
 Especially useful during legislative debate on
controversial safety issues.
 Also, useful in developing funding priorities.
March 2006
Observational Surveys
 Sometimes the best way to collect information about
people's behavior is to watch them.
 Observation allows the researcher to collect
information without being a burden on the person
providing the information.
 Typically evaluators develop guides that structure the
observation process.
March 2006
Observational Surveys
 Drawbacks:
 Measures only what you can see. Other types of
data (e.g., opinions, reasons behind behavior)
cannot be collected in this fashion.
 Time-consuming as multiple observations are
often required.
 Presence of collectors may influence behavior.
 Safety of observers.
March 2006
Program Assessments
 Help determine ways to improve
effectiveness and efficiency of programs.
 Provide tools and documentation by which
additional steps can be taken to make
programs better and/or safer.
 Identify gaps in services.
 Provide support for additional financial and
human resources.
March 2006
Program Assessments
Peer reviews of programs. NHTSA
currently offers program assessments
for the following:
 emergency medical services
 impaired driving
 traffic records
 motorcycle safety
 occupant protection
March 2006
Program Assessments
Examples :
 Studies on the effectiveness of program activities
 Identifies steps to be taken to enhance existing programs
 Evaluates the implementation of new programs
 Assists in justifying additional funding and program
support
 Considers current legislation and the direction for
legislative action
 Provides documentation to be used as National and
State input for policy, training and program development.
March 2006
How Do You Arrange for a
Program Assessment?
 The State Highway Safety Offices obtain
program assessments by writing and
requesting an assessment from one of the
NHTSA Regional Offices.
 If information or assistance is needed
regarding the Highway Safety Program
Assessments, please contact the NHTSA
Regional office for your state.
March 2006
Interviews
 Interviews with local agencies personnel (Police, EMS
Responders, Local Engineers, Coroners) provide
invaluable data.
 Insight on perceived needs (more enforcement,
engineering improvements, etc.)
 Opportunity to speak with someone that may have been
first on the scene at a particular incident or fatality
 The benefit in some cases of a play by play account of
what happened at a particular crash
March 2006
Interviews
 Provide insight on local uses for roads (cut through,
alternate route to avoid traffic, racing, truck route, etc.)
 Offer insight as to how areas surrounding road may
change with the various seasons
 Is there anything planted that might impede sight distance at an
intersection?
 Are there any streams that deer gravitate toward which might
increase the need for them to cross roads?
 Become aware of planned projects that may impact the
road
March 2006
Interviews
 When do you conduct interviews?
 Site visits
 Monitoring visits
 Roundtable discussions
 Development meetings
 Public Hearings / Town Hall Meetings
 Annual Professional Conferences
 Individual Interviews
March 2006
Interviews
 Questions asked in interviews
 What is the most common type of crashes?
 What happens when it rains? Does the road flood in
particular area?
 Did something change in the landscape recently that
might affect why crashes increased (i.e., cutting
down trees that block the sun)?
March 2006
Safety Conscious Planning
“...a proactive approach for the
prevention of motor vehicle crashes and
unsafe transportation conditions.”
Improving Safety on Our Highways
March 2006
Safety Conscious Planning
... a comprehensive, system wide,
multi-modal, proactive process that
better integrates safety into surface
transportation decision making.
March 2006
Safety Conscious Planning
 Considers all aspects of highway safety –
engineering, education, awareness,
enforcement & emergency response
 Uses a system-wide approach including sites,
corridors & entire state, regional & local
transportation systems
March 2006
Safety Conscious Planning
Multi-modal including transit, pedestrian
& bicycle safety needs
Proactive - addresses current safety
problems & looks for opportunities to
prevent them in the future
March 2006
Forum Participants
 Broad cross section of planning and safety
communities
 Statewide representation
 Multidisciplinary group (including MPO’s,
COG’s)
 Federal partners (FHWA, NHTSA, FMCSA)
March 2006
Plans Provided to Participants
 AASHTO Strategic Highway Safety Plan
 Emergency Medical Services State Plan
 Federal Railroad Administration Action Plan
 402 Highway Safety Plan
 Injury Control Plan
 SCDOT Strategic Plan
 SC Long Range Transportation Plan
 Others
March 2006
Data Guide
 SCDOT (Road Inventory, Traffic Counts, Mileage
Reports)
 SCDPS (Collision File)
 SCDMV (Driver & Vehicle Files)
 EMS (Run Reports, Trauma Registry)
 DAODAS (School Age & Adult Surveys)
 Office of Research & Statistics (CODES, Census,
Hospital Discharge)
March 2006
Forum Accomplishments
 Brought over 200 partners together, many 1st
time
 Adopted several goals & strategies to improve
safety – all willing to support in their plan
 Improved communications among partners (Email group)
 Enlightened participants on available data
sources
March 2006
CODES Project
 Crash Outcome Data Evaluation System
 Multi-agency effort which includes:
 SC Department of Public Safety
 SC Department of Transportation
 SC Department of Health and Environmental Control
 SC Emergency Medical Services
 SC Budget and Control Board Office of Research and
Statistics
March 2006
CODES: Goal
Provide a comprehensive view of motor
vehicle crashes and their resultant impact
on morbidity, mortality, health care
services and associated costs.
March 2006
CODES Project
 Collaborative approach to obtain medical and
financial outcome information related to motor
vehicle crashes for highway safety and injury
control decision making.
 Evolved as the result of the Intermodal Surface
Transportation Efficiency Act (ISTEA) of 1991
 Report to Congress about the benefits of safety belts
and motorcycle helmets for persons involved in motor
vehicle crashes.
March 2006
CODES Project
 Measure benefits in terms of reducing death, disability,
and medical costs
 Includes statewide data for all persons involved in policereported crashes
 Includes those who were injured or who died as well as those who
were not injured.
 Allows comparisons between those using and not using safety
belts or motorcycle helmets
 Identifies and contrasts characteristics of injured and uninjured
persons within each of the restraint use groups.
March 2006
CODES: Linked Data Sets
 Crash (DPS)
 Emergency Medical Services (DHEC EMS)*
 Hospital (ORS)
*Prior to 2001 only
March 2006
Data Collected: Crash
 Demographic Data
 Driver / Pedestrian / Pedalcyclist
 Passengers
 Restraint Usage
 Crash Location / Type of Crash
 Contributing Factors
 Injuries / Fatalities / Transported
 Alcohol or Drug Involvement
March 2006
Health Care Utilization Databases
Hospital Inpatient Discharges
Ambulatory Surgery Episodes
Emergency Room Visits
March 2006
Data Collected:
Health Care Utilization Databases
 ABOUT THE PATIENT: age, race, gender, geographic
location codes
 ABOUT THE EPISODE: Hospital & Physician
Characteristics
 primary diagnosis and nine related diagnoses
 primary and nine secondary procedures with dates,
admission and discharge dates, length of episode
 destination at discharge (home, home health referral, death,
etc.)
March 2006
Data Collected:
Health Care Utilization Databases
 COST OF CARE:
 Detailed charges by revenue center (e.g., pharmacy, lab,
respiratory therapy, etc.)
 Primary and secondary payer class (e.g., Medicare,
Medicaid, Private Insurance, Self-pay/Indigent)
*Hospital charges used as proxy
March 2006
CODES Project
 Links databases containing information about
individual persons collected from police crash
reports, emergency medical transports, emergency
room visits and inpatient hospitalization records.
 Uses probabilistic linkage methods
 SC has been a CODES state for 7 years.
March 2006
CODES Project
 System helps in analyzing crash patterns.
 Includes a mapping system to locate crashes based
on such characteristics as crash severity, volume of
crashes, age of driver and involvement of alcohol
and/or drugs.
 Developed CODES Internet site to facilitate the
dissemination of information from the project.
March 2006
CODES Project
 Statewide data collected on all persons:
 Involved in police-reported crashes
 Transported by emergency medical services due to crashes
 Visits to emergency room due to injuries
 Hospitalized due to injuries.
 Analyses of data used to measure impact of crashes by
communities in terms of reducing injuries, deaths, and medical
costs.
 Comparisons can be made between the characteristics of those
using and not using safety belts, helmets, and other restraints.
 PDO crashes were not included in the linkage or analysis.
March 2006
Uses of CODES Data
 Provides economic argument for safety legislation
 Primary Belt Law
 .08
 .10 Per Se
 Automated Enforcement at Red Lights
 Develops profiles for Safe Communities Programs
 Provides demographic data to develop educational /
enforcement programs.
March 2006
Impact of ED Data on CODES
 Policy decisions can be made based on better
estimates of medical cost data
 Population of crash victims injured and treated in a
hospital setting more accurate
 Surrogate SC Trauma Registry
 Complete look at crash and injury patterns for
Community Needs Assessments (high volume vs.
high injury)
March 2006
Data Request / Fact Sheet
 Restraint use by pay source and treatment
type (ED and inpatient)
 Number and rate of injuries
 Total and average charges
 Length of stay (inpatient only)
 Also included the total numbers of injured, number
and percent linked from crash to hospital data.
March 2006
What is a Data Cube?
 While ORS answers requests now using
information from the Data Warehouse, our
dream was to create a
 “WEB-Accessible User-Driven Query Based
System that agencies can access and explore their
own questions”
 Cubes would be for statistical / aggregate analyses
March 2006
SC Data Warehouse
 Build off of existing systems (legacy systems from
state agencies and private sector)
 Create a Unique ID (not related to any other number)
 Identifiers are pulled off of the statistical data. Use
only the statistical data
 Data is always “owned” by the originating agency.
Must have permissions to use and/or link any data
March 2006
LEGEND
Elder
Services &
Assessments
Disabilities &
Special Needs
Vocational
Rehabilitation
Law
Enforcement
Juvenile
Justice
Legal/Safety
Services
Disease
Registries
Public
Safety
Social Services
Education
Probation
Parole &
Pardon
Health
Department
Environmental
Conditions
Alcohol & Drug
Services
Claims Systems
All Payer Health
Care Databases
Corrections*
Integrated
Data
System
Behavioral Health
Child
Care
Social
Services
Mental
Health
Health
Department
Education
Medicare
Hospitalizations
Medicaid
Services
Free Clinic
Visits
Emergency Room Visits
Outpatient
Surgeries
Home
Health
Care
Community
Health Centers*
State
Employee
Health
Services
Other State
Support Agencies
Disease
Registries
*Still in contract negotiations.
March 2006
SC Data Warehouse
 Data Warehouse Allows Agencies & Other Entities to:
 Evaluate their programs
 Look at Outcomes
 Understand better how their programs interact with other
agency & other entity programs
 Study Health, Human Service, Education, and Law
Enforcement Issues
 Analyze Statistical – Aggregate Information
 Access Analytic Data Cubes
 Partner in the Development of Customized Software
Applications
March 2006
LEGEND
Elder
Services &
Assessments
Disabilities &
Special Needs
Vocational
Rehabilitation
Law
Enforcement
Juvenile
Justice
Legal/Safety
Services
Disease
Registries
Public
Safety
Social Services
Education
Probation
Parole &
Pardon
Health
Department
Environmental
Conditions
Alcohol & Drug
Services
Claims Systems
All Payer Health
Care Databases
Corrections*
Injury and
Violence
Cube
Behavioral Health
Child
Care
Social
Services
Mental
Health
Health
Department
Education
Medicare
Hospitalizations
Medicaid
Services
Free Clinic
Visits
Emergency Room Visits
Outpatient
Surgeries
Home
Health
Care
Community
Health Centers*
State
Employee
Health
Services
Other State
Support Agencies
Disease
Registries
*Still in contract negotiations.
March 2006
Linking Data Sets
 Records are linked for the same individual
using a unique tracking number
 Tracking number is random so cannot be
unencrypted to identify the individual
 An individual is assigned the same number
over time
March 2006
What Is an Analytic Cube?
 A way to “slice and dice” through large
amounts of data
 Define “slicers,” characteristics that are
important to analyzing the subject population
 Pre-aggregate the linked data by all possible
combinations of “slicers”
March 2006
Data shown as an example - not for
analytical use
March 2006
Data shown as an example - not for
analytical use
March 2006
Data shown as an example - not for
analytical use
59
March 2006
Data shown as an example - not for
analytical use
March 2006
Cubes Under Development
 Injury Cube (funded through a CDC grant)
 Mental Health/Alcohol Cube funded through
DMH funds
 Medicaid Cube (all ages)
 Elderly Cube (just received funding to
development this cube!)
March 2006
Complaint File
 Track all complaints
 News stories, editorials, news articles
 Be Responsive to public concerns
 Is it a real problem or just perceived?
 Coordinate site visit
 Prepare response
 Include in future planning
 Compile crash data for complaint site
Do not ignore the public!
March 2006
Program Evaluation Reports
 Assess how well the program has been
implemented.
 Assess the extent to which the activities have
achieved the project’s goals.
 Identify gaps in services.
 Identify spin-off efforts.
March 2006
Establish an Evaluation Plan
 How will you know you’re achieving goals?
 What will you measure?
 How will you evaluate?
 Who measures? When?
 What documentation will you maintain?
 When and what evaluation report?
March 2006
Process Evaluation
 Why a program succeeds or fails.
 Compares program design with
implementation.
 Describes and documents life of program.
March 2006
Outcome Evaluation
 Deals with short-term, direct effects of program.
 Identifies the results of a program's/initiative's
effort.
 It seeks to answer the question, "What difference
did the program make?“
 It provides information about effects of a program
after a specified period of operation.
March 2006
Impact Evaluation
 Deals with long-term, ultimate effects of program.
 Assess program effectiveness in terms of end
results, including intended and unintended results.
 Also assess the net effect of a program by
comparing impacts with an estimate of what would
have happened in the absence of the program
March 2006
Professional Judgment
 Allows for a multi-disciplinary approach.
 Assimilates data from variety of sources to determine
strategies.
 Provides benefits from years of knowledge and
experience.
 Essential to any planning process.
March 2006
For More Information Contact:
Terecia Wilson
Director of Safety
803-737-1161
[email protected]
March 2006