Transcript National Weather Service Marine/Tropical Survey Presentation of Customer Satisfaction Results

National Weather Service
Marine/Tropical Survey
Presentation of
Customer Satisfaction Results
March 16, 2004
Federal Consulting Group
© CFI Group
1
Introduction
© CFI Group
2
Snapshot of the American Customer
Satisfaction Index (ACSI)
• The #1 national economic indicator of
customer satisfaction
• Compiled by the National Quality Research Center
at University of Michigan since 1994 using
methodology licensed from CFI Group
• Measures 30 industries, 180+ organizations
covering 75% of the U.S. economy
– Over 70 U.S. Federal Government agencies have used ACSI to
measure more than 120 programs/services
• Advanced methodology quantifiably measures and links
satisfaction levels to performance and prioritizes actions for
improvement
• Results from all surveys are published quarterly in The Wall
Street Journal
© CFI Group
3
Scope of the 2003 NWS
Customer Satisfaction Program
 Four segments measured in 2003
– Aviation (pilots and dispatchers) – completed
– Emergency managers – completed
– Media – completed
– Marine/Tropical – completed
© CFI Group
4
Marine
Customer Satisfaction Results
© CFI Group
5
Project Background
© CFI Group
•
Target segment: Users of Marine and Coastal, Tropical, and/or
Great Lakes products and services, including recreational boaters,
coastal residents, beachgoers, and professionals.
•
Survey content: Survey designed over July and August 2003, with
feedback from process owners in NWS (Silver Spring and regions).
•
Survey method: The survey was conducted online and hosted on
CFI Group’s secure server.
•
Customers contacted: NWS publicized the survey to interested
parties and provided links to the survey on multiple web sites,
including the National Hurricane Center.
•
Response totals: The online survey accepted responses from
September 9 through October 6, 2003. Completed a total of 6298
interviews.
6
Completed Surveys, by Date
1000
900
800
700
600
500
Survey link
removed from NHC
site
Isabel
400
300
200
100
© CFI Group
5-Oct
3-Oct
1-Oct
29-Sep
27-Sep
25-Sep
23-Sep
21-Sep
19-Sep
17-Sep
15-Sep
13-Sep
11-Sep
9-Sep
0
7
Who Are the Marine Respondents?
Services Used
Tropical Services
71%
Marine and
Coastal Services
60%
Other Services
13%
Great Lakes
Services
9%
0%
© CFI Group
10%
20%
30%
40%
50%
60%
70%
80%
8
Who Are the Marine Respondents?
Primary Operating Area
Atlantic
42%
Gulf of Mexico
20%
Tropical Atlantic/
Caribbean
16%
Other
9%
Great Lakes
7%
West Coast
3%
Pacific
Alaska
2%
0%
0%
© CFI Group
10%
20%
30%
40%
50%
9
Who Are the Marine Respondents?
Primary Marine Activity
Recreational Boater
34%
Coastal resident
29%
Other
15%
Beachgoer
12%
Science/research
4%
Search and rescue
2%
Law enforcement
1%
Commercial fishing
1%
Passenger transport
1%
Freight transport
1%
Cruise ship
0%
0%
© CFI Group
5%
10%
15%
20%
25%
30%
35%
40%
10
Who Are the Marine Respondents?
Most-Used Products: Marine and Coastal
Coastal Waters Forecasts
91%
Hazardous Weather Outlooks
91%
Short Term Forecasts
85%
Coastal Flood Watch/Warnings
79%
Area Forecast Discusssions
78%
Special Marine Warnings
74%
Marine Weather Statements
70%
Offshore Waters Forecasts
62%
High Seas Forecasts
51%
Surf Zone Forecasts
0%
© CFI Group
43%
20%
40%
60%
80%
100%
11
Usefulness of Dissemination Methods
69
NOAA weather radio
83
NOAA weather radio
83
Internet access
40
40
42
Internet access
U.S. Coast Guard broadcasts
74
U.S. Coast Guard broadcasts
Local or cable TV/radio
68 74
31
Local or cable TV/radio
31
EMWIN
45
35
EMWIN
35
NOAA Weather Wire, FOS, etc.
32
32
31
30
30
32
NOAA Weather Wire, FOS, etc.
Phone
Phone
Private
Private vendor
vendor
19
19
20
Radiofacsimile
Radiofacsimile
NAVTEX
NAVTEX
18
Inmarsat-C
Inmarsat-C SafetyNET
SafetyNET
18
00
10
10
20
41
31
31
29
26
30
40
50
Sea
Sea
60
60
70
70
80
90
90
100
100
Land
Land
97% of respondents rate land methods
58% of respondents rate sea methods
© CFI Group
90
90
12
CSI Comparison across Government
Agencies
ACSI Overall
74
Federal Gov't Aggregate
70
NWS Emergency Managers
80
NWS Marine
78
NWS Aviation
77
NWS Media Personnel
76
U.S. Mint - Collectors
84
GSA FCIC
80
Bureau of the Census
73
Export - Import Bank
70
FDA
67
FAA
64
IRS
62
0
10
20
30
40
50
60
70
80
90
Customer Satisfaction Index (CSI) scores are based on three questions:
© CFI Group
• Overall satisfaction with products and services
• Products and services compared to expectations
• Products and services compared to ideal
13
Satisfaction Scores
Services Used
Other Services
13% of respondents
Tropical Services
71% of respondents
Marine and Coastal
Services
60% of respondents
Great Lakes Services
© CFI Group
9% of respondents
81
80
77
76
14
Satisfaction Scores
Primary Operating Area
© CFI Group
Tropical Atlantic/Caribbean
16% of resp.
80
Other
9% of resp.
Atlantic
42% of resp.
78
Gulf of Mexico
20% of resp.
78
Pacific
2% of resp.
Great Lakes
7% of resp.
West Coast
3% of resp.
79
77
75
74
15
Satisfaction Scores
Primary Marine Activity
© CFI Group
Law enforcement
1% of resp.
Passenger transport
1% of resp.
81
Coastal resident
29% of resp.
81
Beachgoer
12% of resp.
81
Science/research
4% of resp.
79
Search and rescue
2% of resp.
79
Freight transport
1% of resp.
78
Other
15% of resp.
78
Recreational boater
34% of resp.
Commercial fishing
1% of resp.
82
75
73
16
NWS Marine CSI Model
INPUTS
Quality of Forecasts
Timelines of Forecasts
Great Lakes Short-Fused
And Other Forecasts
Quality of Ice Freeze-up Outlooks
Timelines of Ice Break-up Outlooks
Great Lakes Ice Forecasts
Clarity and utility of hurricane and tropical storm
watches and warnings
OUTPUTS
Great Lakes Marine Forecasts
Quality of Short Term Forecasts
Timelines of Short Term Forecasts
Usefulness of tropical forecast products
Ease of understanding tropical
forecast products
SATISFACTION
NHC Forecasts
Tropical Warnings and Watches
Contact NWS
How well HLSs support mission needs in
geographic area of interest
Quality of Forecasts
Timelines of Forecasts
Quality of Short Term Forecasts
Timelines of Short Term Forecasts
Quality of Area Forecast Discussions
Timelines of Special Marine Warnings
Hurricane Local Statements
Coastal /Offshore/High Seas Forecasts
Marine/Coastal Short-Fused and
Other Forecasts
Coastal Flood and Surf Zone Information
Contacted NWS to report problem or
make suggestion
Customer
Satisfaction
NWS overall
NWS vs. expectations
NWS vs. ideal
Take Action
Likelihood to take action based on
NWS information
Confidence in
NWS
Confidence in NWS
Accuracy of Marine Warnings
Overall accuracy of the Marine Forecasts
Marine Forecast Accuracy
Ease of understanding the data provided
Ease of reading charts and graphs
General Forecast Performance
Timelines of warnings broadcast over NOAA
Weather Radio
Frequency of forecast updates
General Forecast Availability
Frequency NWS forecast events occur within six
hours of predicted time frame
General Forecast Verification
© CFI Group
17
NWS Marine Model
INPUTS
Score:
Measures
performance
on a 0-100
scale.
76
Great Lakes Marine Forecasts
0.9
78
Great Lakes Short-Fused
And Other Forecasts
0.3
63
Great Lakes Ice Forecasts
0.0
85
NHC Forecasts
0.9
85
Tropical Warnings and Watches
0.5
80
Hurricane Local Statements
0.1
76 Coastal /Offshore/High Seas Forecasts 0.6
79
Marine/Coastal Short-Fused and
Other Forecasts
OUTPUTS
SATISFACTION
0.4
Impact: Measures leverage
of component on target
variable. If NHC Forecasts
improves 5 points, CSI will
improve 0.9 points.
13%
Contact NWS
Customer
Satisfaction
78
90
Take Action
Confidence in
NWS
87
78 Coastal Flood and Surf Zone Information0.0
© CFI Group
74
Marine Forecast Accuracy
0.9
78
General Forecast Performance
0.7
76
General Forecast Availability
0.4
69
General Forecast Verification
0.5
18
Priority Matrix: Where to Focus
Attention
Building on Success
HIGH
Maintain
NHC Forecasts 85/0.9
Great Lakes Marine Forecasts 76/0.9
General Forecast Performance 78/0.7
Coastal/Offshore/High Seas Forecasts 76/0.6
Tropical Warnings and Watches 85/0.5
Score
Marine/Coastal Short-Fused
and Other Forecasts 79/0.4
General Forecast Availability 76/0.4
Great Lakes Short-Fused
and Other Forecasts 78/0.3
Hurricane Local Statements 80/0.1
Coastal Flood
and Surf Zone Information 78/0.0
Monitor
Top Priority
Great Lakes Ice Forecasts 63/0.0
LOW
Marine Forecast Accuracy 74/0.9
General Forecast Verification 69/0.5
© CFI Group
LOW
Impact on Satisfaction (CSI)
HIGH
19
Marine Forecast Accuracy
Impact on CSI: 0.9
Marine Forecast
Accuracy
74
Accuracy of Marine
Warnings
76
Overall accuracy of the
Marine forecasts
75
Accuracy of the wind
forecast
Accuracy of the wave
forecast
© CFI Group
72
67
20
NHC Forecasts
Impact on CSI: 0.9
NHC Forecasts
85
• The usefulness of additional probabilistic information is rated an 86 – very useful
• When asked which area needs improvement most, 51% of respondents say ‘Track’. No other
feature is mentioned more than 11% of the time.
Usefulness of the
NWS/NHC tropical
forecast products
87
Overall quality of
NWS/NHC tropical
forecast products and
services
87
Improvements in
NWS/NHC tropical
forecasting over the past
five years
Ease of understanding
the NWS/NHC tropical
forecast products
© CFI Group
86
82
21
General Forecast Performance
Impact on CSI: 0.7
General Forecast
Performance
78
The ease of reading
charts and graphs
80
Ease of understanding
the data provided
Timeliness of responses
to your queries
Availability of
information to explain
the data
© CFI Group
79
76
75
22
Coastal/Offshore/High Seas Forecasts
Impact on CSI: 0.6
Coastal/Offshore/High
Seas Forecasts
76
Quality of Coastal
Waters Forecasts
77
Timeliness of Coastal
Waters Forecasts
© CFI Group
76
Quality of High Seas
Forecasts
75
Timeliness of High Seas
Forecasts
75
Quality of Offshore
Waters Forecasts
74
Timeliness of Offshore
Waters Forecasts
74
23
Tropical Warnings and Watches
Impact on CSI: 0.5
Tropical Warnings and
Watches
85
• Most-mentioned minimum desired probability associated with a watch: 30%
• Most-mentioned minimum desired probability associated with a warning: 70%
• 50% of respondents prefer break points over county boundaries, and 28% are indifferent
Clarity and utility of NWS
hurricane and tropical
storm watches and
warnings
© CFI Group
85
24
General Forecast Availability
Impact on CSI: 0.4
General Forecast
Availability
76
The timeliness of
warnings broadcast over
NOAA Weather Radio
The frequency of
forecast updates
© CFI Group
81
72
25
Marine and Coastal Short-Fused/Other
Impact on CSI: 0.4
79
Marine/Coastal Short-Fused and Other Forecasts
82
Quality of Special Marine Warnings
Quality of Hazardous Weather Outlooks
Quality of Marine Weather Statements
79
Timeliness of Special Marine Warnings
79
Quality of Area Forecast Discussions
78
Quality of Short Term Forecasts
78
Timeliness of Marine Weather Statements
78
Timeliness of Hazardous Weather Outlooks
78
Timeliness of Short Term Forecasts
Timeliness of Area Forecast Discussions
© CFI Group
81
77
76
26
Coastal Flood and Surf Zone Info
Impact on CSI: 0.0
Coastal Flood and Surf
Zone Information
78
Quality of Coastal Flood
Watch/Warnings
80
Timeliness of Coastal
Flood Watch/Warnings
Timeliness of Surf Zone
Forecasts
Quality of Surf Zone
Forecasts
© CFI Group
79
72
71
27
Other Aspects of NWS Performance
The use of English
versus metric units
82
The availability of data
in languages other than
English
48
Usefulness of 500
milibar forecasts
Usefulness of wave
period forecasts
© CFI Group
53
45
28
What Customers Want
Prefer graphical representation of
forecasts and warnings services
78%
Prefer text-based forecasts and
warnings services
60%
Prefer audio forecasts
36%
Prefer gridded and/or vector forecasts
Prefer other
0%
30%
4%
20%
40%
60%
80% 100%
• Customers express a clear preference for graphical forecasts.
• A constantly-evolving forecast is preferred over a regularly-scheduled
forecast, 79% to 21%.
© CFI Group
29
What Customers Want (cont.)
Graphical Hazardous Weather Outlooks
72%
Graphical Hurricane Local Statements
70%
Graphical Probability-based Wave
Height Forecasts
46%
Graphical Rip Current Outlooks
26%
Graphical Waterspout Outlooks
25%
Other
0%
4%
20%
40%
60%
80%
• When asked if they want any weather information not currently provided by
NWS, 82% say no.
• Needs are varied, from better barometric readings to more accurate lightning
forecasts to better wave forecasts.
© CFI Group
30
Recommendations
•
•
•
•
•
•
•
•
•
Keep up the good work! (e.g., Isabel forecasting)
Improve perceived forecast accuracy by increasing the number of
official forecasts
Give customers more information to explain the forecast data
Educate customer regarding reasonable expectations, especially
with wave height and period forecasts
As in the Media survey, improve the consistency of Area Forecast
Discussions and terminology use
Ensure storm track is a top priority for tropical/hurricane forecasts
Focus on graphical products, but keep in mind that radio is still a
dominant method of information acquisition
Examine possible causes of lower scores for Great Lakes Ice and
Coastal Surf Zone forecasts
Consider how different regions/users may respond to the survey
differently
– These overall findings will have varying applicability to specific subpopulations
© CFI Group
31
Discussion
© CFI Group
32
For More Information
Brian LaMarre, NOAA
Marine Program Manager
(301) 713-1677 x108
[email protected]
Doug Helmreich, CFI Group
Senior Consultant
(734) 623-1321
[email protected]
Bernie Lubran, Federal Consulting Group
Project Manager
(202) 906-5642
[email protected]
© CFI Group
33