Addressing the Limited Data Dilemma
Download
Report
Transcript Addressing the Limited Data Dilemma
Addressing the Limited Data Dilemma
Non-Traditional Sources of Safety Data
Presented by: Terecia Wilson
Director of Safety
March 2006
Traditional Data Collection
Crash Data
Driver Records
Vehicle Records
Roadway Inventory Data
Citation Data / Adjudication Data
EMS Run Reports
March 2006
Alternative Data Collection
Road Safety Audits
Opinion Surveys
Observational Surveys
Program Assessments
Interviews
SCP Forum (Data Guide)
CODES Data
Data Cube
Complaint Files
Program Evaluations
Professional Judgment
March 2006
What is a Road Safety Audit?
A formal examination of an existing or
future road or traffic project, or any project
which interacts with road users, in which
an independent, qualified examiner
reports on the project’s crash potential
and safety performance.
March 2006
Why Do RSAs?
Proactive approach to highway safety.
Widely used in other countries - highly
effective.
Possible even with limited resources.
Supports Strategic Plan Goal of improving
safety.
March 2006
When Can an Audit Be Done?
Future Roads
Stage 1 – Planning
Stage 2 – Preliminary Design
Roads Under Construction
Stage 3 – Final Design
Stage 4 – Pre-opening
Existing Roadways
Stage 5 – Operations Review
March 2006
Types of Audit Data
Information collected for use in audit
Traffic counts
Public hearing information
Detailed designs
Crash Data
March 2006
Types of Audit Data
Information collected from audit report
Prioritized findings and recommendations
Multi-disciplinary report
Comments from special road user groups
March 2006
RSA Team Participants
Geographic Representation from across state.
Representation from various disciplines: Traffic
Engineering, Planning, Engineering, Construction,
Pre Construction, Special Interest groups.
Special Interest groups (ie, local law enforcement,
EMS, Disabilities and Special Needs, AARP, etc.)
March 2006
RSA Follow Up
Conduct follow up study to determine impact on
traffic safety 3 years after final audit report.
Examine traffic collision data 3 years before and 3
years after audit.
Include RSA team to assist in evaluation as
needed.
March 2006
Method
Advantages
Disadvantages
Surveys
Mail
· Can survey many people
· Not time-consuming
· Difficult to get much detail
· Sometimes difficult to get correct addresses
· Relatively inexpensive
· May be problems with interpreting questions
· Everyone gets the same instrument
· Sometimes a problem getting surveys completed & returned
· Objective Interpretation
Telephone
Interview
· Able to ask for more detail when needed
· Sometimes difficult reaching people
· Everyone gets the same instrument
· Lack of anonymity
· Researcher knows how people are interpreting questions
· Time-consuming time, can limit sample size
· Able to ask for more detail when needed
· Subjective interpretation
· Provide detailed data
· Can be expensive
· Can be difficult to analyze
Focus Group
· Researcher knows how people are interpreting questions
· Group setting may inhibit some individuals from providing
information
· Able to ask for more detail when needed
· Sometimes hard to coordinate multiple schedules
· Able to interview multiple people at one time, thus, more
cost-effective
· Responses from one person provide stimulus for other people
· Responses from one person provide stimulus for other
people
Observation
· Objective interpretation
· Time-consuming
· Low burden for people providing data
· Some items are not observable
· Can be expensive
· Participant behavior may be affected by observer presence
March 2006
Opinion Surveys
Telephone Surveys
Random Digit Dialing
Unbiased
Varied Demographics (Age, Race, Education,
Income)
Representative sample of licensed drivers
March 2006
Opinion Surveys
Examples of Telephone Surveys
Follow-up evaluation of public information
campaigns
Determine public opinion on safety issues
• Primary Safety Belt Law
• .08 BAC
• Motorcycle Helmet Legislation
Determine changes in public opinion on safety
issues (Baseline/Ongoing)
March 2006
Opinion Surveys
Focus Groups
Used for more qualitative information
Used in a wide range of applications in nearly
every field of market research.
Encourages participants to express their feelings
freely and without inhibitions
Probes more deeply into issues in a relaxed,
uncontrolled atmosphere.
March 2006
Opinion Surveys
Focus Groups
Discussion leaves participants feeling, justifiably,
that their opinion is important
Gives clients a clearer perspective on how their
customers feel and why they feel that way.
Information gained may lead to changes in
program and product development and
implementation.
Include demographic representation of target
population.
March 2006
Opinion Surveys
Used as part of dispute resolution, or to solicit
opinions before making significant changes.
Useful in developing consensus when
community opinion is not immediately
obvious in normal discussion.
Especially useful during legislative debate on
controversial safety issues.
Also, useful in developing funding priorities.
March 2006
Observational Surveys
Sometimes the best way to collect information about
people's behavior is to watch them.
Observation allows the researcher to collect
information without being a burden on the person
providing the information.
Typically evaluators develop guides that structure the
observation process.
March 2006
Observational Surveys
Drawbacks:
Measures only what you can see. Other types of
data (e.g., opinions, reasons behind behavior)
cannot be collected in this fashion.
Time-consuming as multiple observations are
often required.
Presence of collectors may influence behavior.
Safety of observers.
March 2006
Program Assessments
Help determine ways to improve
effectiveness and efficiency of programs.
Provide tools and documentation by which
additional steps can be taken to make
programs better and/or safer.
Identify gaps in services.
Provide support for additional financial and
human resources.
March 2006
Program Assessments
Peer reviews of programs. NHTSA
currently offers program assessments
for the following:
emergency medical services
impaired driving
traffic records
motorcycle safety
occupant protection
March 2006
Program Assessments
Examples :
Studies on the effectiveness of program activities
Identifies steps to be taken to enhance existing programs
Evaluates the implementation of new programs
Assists in justifying additional funding and program
support
Considers current legislation and the direction for
legislative action
Provides documentation to be used as National and
State input for policy, training and program development.
March 2006
How Do You Arrange for a
Program Assessment?
The State Highway Safety Offices obtain
program assessments by writing and
requesting an assessment from one of the
NHTSA Regional Offices.
If information or assistance is needed
regarding the Highway Safety Program
Assessments, please contact the NHTSA
Regional office for your state.
March 2006
Interviews
Interviews with local agencies personnel (Police, EMS
Responders, Local Engineers, Coroners) provide
invaluable data.
Insight on perceived needs (more enforcement,
engineering improvements, etc.)
Opportunity to speak with someone that may have been
first on the scene at a particular incident or fatality
The benefit in some cases of a play by play account of
what happened at a particular crash
March 2006
Interviews
Provide insight on local uses for roads (cut through,
alternate route to avoid traffic, racing, truck route, etc.)
Offer insight as to how areas surrounding road may
change with the various seasons
Is there anything planted that might impede sight distance at an
intersection?
Are there any streams that deer gravitate toward which might
increase the need for them to cross roads?
Become aware of planned projects that may impact the
road
March 2006
Interviews
When do you conduct interviews?
Site visits
Monitoring visits
Roundtable discussions
Development meetings
Public Hearings / Town Hall Meetings
Annual Professional Conferences
Individual Interviews
March 2006
Interviews
Questions asked in interviews
What is the most common type of crashes?
What happens when it rains? Does the road flood in
particular area?
Did something change in the landscape recently that
might affect why crashes increased (i.e., cutting
down trees that block the sun)?
March 2006
Safety Conscious Planning
“...a proactive approach for the
prevention of motor vehicle crashes and
unsafe transportation conditions.”
Improving Safety on Our Highways
March 2006
Safety Conscious Planning
... a comprehensive, system wide,
multi-modal, proactive process that
better integrates safety into surface
transportation decision making.
March 2006
Safety Conscious Planning
Considers all aspects of highway safety –
engineering, education, awareness,
enforcement & emergency response
Uses a system-wide approach including sites,
corridors & entire state, regional & local
transportation systems
March 2006
Safety Conscious Planning
Multi-modal including transit, pedestrian
& bicycle safety needs
Proactive - addresses current safety
problems & looks for opportunities to
prevent them in the future
March 2006
Forum Participants
Broad cross section of planning and safety
communities
Statewide representation
Multidisciplinary group (including MPO’s,
COG’s)
Federal partners (FHWA, NHTSA, FMCSA)
March 2006
Plans Provided to Participants
AASHTO Strategic Highway Safety Plan
Emergency Medical Services State Plan
Federal Railroad Administration Action Plan
402 Highway Safety Plan
Injury Control Plan
SCDOT Strategic Plan
SC Long Range Transportation Plan
Others
March 2006
Data Guide
SCDOT (Road Inventory, Traffic Counts, Mileage
Reports)
SCDPS (Collision File)
SCDMV (Driver & Vehicle Files)
EMS (Run Reports, Trauma Registry)
DAODAS (School Age & Adult Surveys)
Office of Research & Statistics (CODES, Census,
Hospital Discharge)
March 2006
Forum Accomplishments
Brought over 200 partners together, many 1st
time
Adopted several goals & strategies to improve
safety – all willing to support in their plan
Improved communications among partners (Email group)
Enlightened participants on available data
sources
March 2006
CODES Project
Crash Outcome Data Evaluation System
Multi-agency effort which includes:
SC Department of Public Safety
SC Department of Transportation
SC Department of Health and Environmental Control
SC Emergency Medical Services
SC Budget and Control Board Office of Research and
Statistics
March 2006
CODES: Goal
Provide a comprehensive view of motor
vehicle crashes and their resultant impact
on morbidity, mortality, health care
services and associated costs.
March 2006
CODES Project
Collaborative approach to obtain medical and
financial outcome information related to motor
vehicle crashes for highway safety and injury
control decision making.
Evolved as the result of the Intermodal Surface
Transportation Efficiency Act (ISTEA) of 1991
Report to Congress about the benefits of safety belts
and motorcycle helmets for persons involved in motor
vehicle crashes.
March 2006
CODES Project
Measure benefits in terms of reducing death, disability,
and medical costs
Includes statewide data for all persons involved in policereported crashes
Includes those who were injured or who died as well as those who
were not injured.
Allows comparisons between those using and not using safety
belts or motorcycle helmets
Identifies and contrasts characteristics of injured and uninjured
persons within each of the restraint use groups.
March 2006
CODES: Linked Data Sets
Crash (DPS)
Emergency Medical Services (DHEC EMS)*
Hospital (ORS)
*Prior to 2001 only
March 2006
Data Collected: Crash
Demographic Data
Driver / Pedestrian / Pedalcyclist
Passengers
Restraint Usage
Crash Location / Type of Crash
Contributing Factors
Injuries / Fatalities / Transported
Alcohol or Drug Involvement
March 2006
Health Care Utilization Databases
Hospital Inpatient Discharges
Ambulatory Surgery Episodes
Emergency Room Visits
March 2006
Data Collected:
Health Care Utilization Databases
ABOUT THE PATIENT: age, race, gender, geographic
location codes
ABOUT THE EPISODE: Hospital & Physician
Characteristics
primary diagnosis and nine related diagnoses
primary and nine secondary procedures with dates,
admission and discharge dates, length of episode
destination at discharge (home, home health referral, death,
etc.)
March 2006
Data Collected:
Health Care Utilization Databases
COST OF CARE:
Detailed charges by revenue center (e.g., pharmacy, lab,
respiratory therapy, etc.)
Primary and secondary payer class (e.g., Medicare,
Medicaid, Private Insurance, Self-pay/Indigent)
*Hospital charges used as proxy
March 2006
CODES Project
Links databases containing information about
individual persons collected from police crash
reports, emergency medical transports, emergency
room visits and inpatient hospitalization records.
Uses probabilistic linkage methods
SC has been a CODES state for 7 years.
March 2006
CODES Project
System helps in analyzing crash patterns.
Includes a mapping system to locate crashes based
on such characteristics as crash severity, volume of
crashes, age of driver and involvement of alcohol
and/or drugs.
Developed CODES Internet site to facilitate the
dissemination of information from the project.
March 2006
CODES Project
Statewide data collected on all persons:
Involved in police-reported crashes
Transported by emergency medical services due to crashes
Visits to emergency room due to injuries
Hospitalized due to injuries.
Analyses of data used to measure impact of crashes by
communities in terms of reducing injuries, deaths, and medical
costs.
Comparisons can be made between the characteristics of those
using and not using safety belts, helmets, and other restraints.
PDO crashes were not included in the linkage or analysis.
March 2006
Uses of CODES Data
Provides economic argument for safety legislation
Primary Belt Law
.08
.10 Per Se
Automated Enforcement at Red Lights
Develops profiles for Safe Communities Programs
Provides demographic data to develop educational /
enforcement programs.
March 2006
Impact of ED Data on CODES
Policy decisions can be made based on better
estimates of medical cost data
Population of crash victims injured and treated in a
hospital setting more accurate
Surrogate SC Trauma Registry
Complete look at crash and injury patterns for
Community Needs Assessments (high volume vs.
high injury)
March 2006
Data Request / Fact Sheet
Restraint use by pay source and treatment
type (ED and inpatient)
Number and rate of injuries
Total and average charges
Length of stay (inpatient only)
Also included the total numbers of injured, number
and percent linked from crash to hospital data.
March 2006
What is a Data Cube?
While ORS answers requests now using
information from the Data Warehouse, our
dream was to create a
“WEB-Accessible User-Driven Query Based
System that agencies can access and explore their
own questions”
Cubes would be for statistical / aggregate analyses
March 2006
SC Data Warehouse
Build off of existing systems (legacy systems from
state agencies and private sector)
Create a Unique ID (not related to any other number)
Identifiers are pulled off of the statistical data. Use
only the statistical data
Data is always “owned” by the originating agency.
Must have permissions to use and/or link any data
March 2006
LEGEND
Elder
Services &
Assessments
Disabilities &
Special Needs
Vocational
Rehabilitation
Law
Enforcement
Juvenile
Justice
Legal/Safety
Services
Disease
Registries
Public
Safety
Social Services
Education
Probation
Parole &
Pardon
Health
Department
Environmental
Conditions
Alcohol & Drug
Services
Claims Systems
All Payer Health
Care Databases
Corrections*
Integrated
Data
System
Behavioral Health
Child
Care
Social
Services
Mental
Health
Health
Department
Education
Medicare
Hospitalizations
Medicaid
Services
Free Clinic
Visits
Emergency Room Visits
Outpatient
Surgeries
Home
Health
Care
Community
Health Centers*
State
Employee
Health
Services
Other State
Support Agencies
Disease
Registries
*Still in contract negotiations.
March 2006
SC Data Warehouse
Data Warehouse Allows Agencies & Other Entities to:
Evaluate their programs
Look at Outcomes
Understand better how their programs interact with other
agency & other entity programs
Study Health, Human Service, Education, and Law
Enforcement Issues
Analyze Statistical – Aggregate Information
Access Analytic Data Cubes
Partner in the Development of Customized Software
Applications
March 2006
LEGEND
Elder
Services &
Assessments
Disabilities &
Special Needs
Vocational
Rehabilitation
Law
Enforcement
Juvenile
Justice
Legal/Safety
Services
Disease
Registries
Public
Safety
Social Services
Education
Probation
Parole &
Pardon
Health
Department
Environmental
Conditions
Alcohol & Drug
Services
Claims Systems
All Payer Health
Care Databases
Corrections*
Injury and
Violence
Cube
Behavioral Health
Child
Care
Social
Services
Mental
Health
Health
Department
Education
Medicare
Hospitalizations
Medicaid
Services
Free Clinic
Visits
Emergency Room Visits
Outpatient
Surgeries
Home
Health
Care
Community
Health Centers*
State
Employee
Health
Services
Other State
Support Agencies
Disease
Registries
*Still in contract negotiations.
March 2006
Linking Data Sets
Records are linked for the same individual
using a unique tracking number
Tracking number is random so cannot be
unencrypted to identify the individual
An individual is assigned the same number
over time
March 2006
What Is an Analytic Cube?
A way to “slice and dice” through large
amounts of data
Define “slicers,” characteristics that are
important to analyzing the subject population
Pre-aggregate the linked data by all possible
combinations of “slicers”
March 2006
Data shown as an example - not for
analytical use
March 2006
Data shown as an example - not for
analytical use
March 2006
Data shown as an example - not for
analytical use
59
March 2006
Data shown as an example - not for
analytical use
March 2006
Cubes Under Development
Injury Cube (funded through a CDC grant)
Mental Health/Alcohol Cube funded through
DMH funds
Medicaid Cube (all ages)
Elderly Cube (just received funding to
development this cube!)
March 2006
Complaint File
Track all complaints
News stories, editorials, news articles
Be Responsive to public concerns
Is it a real problem or just perceived?
Coordinate site visit
Prepare response
Include in future planning
Compile crash data for complaint site
Do not ignore the public!
March 2006
Program Evaluation Reports
Assess how well the program has been
implemented.
Assess the extent to which the activities have
achieved the project’s goals.
Identify gaps in services.
Identify spin-off efforts.
March 2006
Establish an Evaluation Plan
How will you know you’re achieving goals?
What will you measure?
How will you evaluate?
Who measures? When?
What documentation will you maintain?
When and what evaluation report?
March 2006
Process Evaluation
Why a program succeeds or fails.
Compares program design with
implementation.
Describes and documents life of program.
March 2006
Outcome Evaluation
Deals with short-term, direct effects of program.
Identifies the results of a program's/initiative's
effort.
It seeks to answer the question, "What difference
did the program make?“
It provides information about effects of a program
after a specified period of operation.
March 2006
Impact Evaluation
Deals with long-term, ultimate effects of program.
Assess program effectiveness in terms of end
results, including intended and unintended results.
Also assess the net effect of a program by
comparing impacts with an estimate of what would
have happened in the absence of the program
March 2006
Professional Judgment
Allows for a multi-disciplinary approach.
Assimilates data from variety of sources to determine
strategies.
Provides benefits from years of knowledge and
experience.
Essential to any planning process.
March 2006
For More Information Contact:
Terecia Wilson
Director of Safety
803-737-1161
[email protected]
March 2006