Transcript SBAS and GBAS Integrity for Non-Aviation Users: Moving Away from
SBAS and GBAS Integrity for Non Aviation Users: Moving Away from "Specific Risk"
Sam Pullen, Todd Walter, and Per Enge
Stanford University
ION ITM 2011 San Diego, CA.
25 January 2011
Motivation (1): SBAS and GBAS for Non-
Aviation Users
•
Where augmentation signals can be received, SBAS and GBAS benefits are available to all users.
•
However, integrity algorithms in airborne MOPS are designed to support specific aviation applications.
–
Resulting integrity protection levels are not well suited for other classes of users
•
Correcting this would increase the attractiveness of SBAS and GBAS to non-aviation transport users (auto, rail, marine) and others.
25 January 2011
Integrity for Non-Aviation Users: Moving Away from "Specific Risk"
2
Motivation (2): Accuracy and Integrity
HPL (non aviation application) 95% HPE
Illustrative example – not to scale or direction
HPL (per MOPS)
•
Accuracy bounds (e.g., 95% vertical position error, or VPE) can be measured and modeled with high precision
•
Integrity bounds (e.g., 10 -7 vertical protection level, or VPL) cannot be
– – – –
Lack of sufficient measurements Flaws in Gaussian extrapolations to low probabilities Dependence on details of failure models and assumptions
Too little is known; too much is uncertain…
25 January 2011
Integrity for Non-Aviation Users: Moving Away from "Specific Risk"
3
WAAS VPE vs. VPL from FAA PAN Data
(3 rd Qtr 2010: July – Sept.) Source: WAAS PAN Report #34, Oct. 2010.
http://www.nstb.tc.faa.gov/ DisplayArchive.htm
Max. VPE
7 m (at Barrow, AK)
25 January 2011
95% VPE
1.2 m 99% VPE
VPE (m)
4
WAAS Reference Station Classifications
(for this study only)
Figure source: FAA GNSS Press Kit http://preview.tinyurl.com/4ofdzz4
18 Remote Stations 13 Outer Stations 7 Inner Stations
25 January 2011
Integrity for Non-Aviation Users: Moving Away from "Specific Risk"
5
Max. VPE and VPL from FAA PAN Data
(1 Jan. 2004 – 30 Sept. 2010) Worst Case Between “Inner” and “Outer” WAAS Stations
->
“InOut” Set
25 January 2011 50 45 40 35 30 25 20 15 10 5 0
Max. VPL Max. VPE 95% VPE
10 15 20 25 30
Quarterly PAN Report Number (8 – 34)
Integrity for Non-Aviation Users: Moving Away from "Specific Risk"
35 6
Max. HPE and HPL from FAA PAN Data
(1 Jan. 2004 – 30 Sept. 2010) Worst Case Between “Inner” and “Outer” WAAS Stations
->
“InOut” Set
45 40 35
As expected, both HPE and HPL are significantly lower than VPE and VPL.
30 25
Max. HPL
20 15
One unusual
result: 12 m error at Cleveland in Spring 2005 (correct number?)
25 January 2011 10 5 0
Max. HPE 95% HPE
10 15 20 25
Quarterly PAN Report Number (8 – 34)
30 35 7
Ratio of Max. VPL and Max. VPE from FAA PAN Data (“InOut” Station Set)
9
Less error reduction after PAN #20 (March 2007).
8 2 1 4 3 7 6
Mean Ratio
=
5.38
5
Noticeable improving trend
likely due to error reduction at individual WAAS reference stations.
0 10 15 20 25
Quarterly PAN Report Number (8 – 34)
30
Integrity for Non-Aviation Users: Moving Away from "Specific Risk"
35 25 January 2011 8
Ratio of Max. HPL and Max. HPE from FAA PAN Data (“InOut” Station Set)
10 9 2
Unusual error at Cleveland (if correct) just barely exceeded by HPL.
25 January 2011 1 0 4 3 6 5 8 7
Mean Ratio
=
5.21
10
Weaker but visible improving trend – more variability.
15 20 25
Quarterly PAN Report Number (8 – 34)
30
Integrity for Non-Aviation Users: Moving Away from "Specific Risk"
35 9
How Many Samples Were Collected?
All validated PAN data from 1 Jan. 2004 to 30 Sept. 2010 Assume data correlated over 600 sec (10 min) Assume data correlated over 150 sec (~ one CAT I approach) Assume data correlated over 30 sec
7.1
10 -6 independent samples
25 January 2011
2.8
10 -7 independent samples 1.4
10 -8 independent samples
Integrity for Non-Aviation Users: Moving Away from "Specific Risk"
4.25
10 -9 sec (49,324.6 days) (105.04 years)
10
Average vs. Specific Risk Assessment
•
Average Risk (my definition): the probability of unsafe conditions based upon the convolved (“averaged”) estimated probabilities of all unknown events.
–
Probabilistic Risk Analysis (PRA) is based on this procedure
–
Risk aversion and value of information (VOI) are applied to the outputs of PRA
integrity risk requirements, alert limits
•
Specific Risk (my definition): the probability of unsafe conditions subject to the assumption that all (negative
but credible) unknown events that could be known
occur with a probability of one.
– –
Evolved from pre-existing FAA and ICAO safety standards Risk aversion and VOI and buried inside specific risk analysis
–
Results (risk and protection levels) are inconsistent with PRA
25 January 2011
Integrity for Non-Aviation Users: Moving Away from "Specific Risk"
11
Simplified Example: Ionospheric Spatial Decorrelation (1)
Severe Ionospheric Storm Observed over CONUS on 20 November 2003
20:15 UT 21:00 UT
25 January 2011
Integrity for Non-Aviation Users: Moving Away from "Specific Risk"
12
Simplified Example: Ionospheric Spatial Decorrelation (2)
•
Using PRA, estimated “prior” probabilities of severe decorrelation are combined with the likelihood of SBAS or GBAS mitigation to derive resulting user risk.
– –
Prior probabilities need not be known precisely Benefits of improved mitigation (“better information”) appear naturally as lower integrity risk.
•
Under FAA interpretation of Specific Risk, worst-case iono. delay gradient is “credible” and thus is assigned a probability of one.
–
Worst-case for GBAS (CAT I): an extremely large gradient that escapes detection by “matching speed” with ground station
»
This differs in real time for each site and GNSS geometry
–
Worst-case for SBAS (LPV): a very large gradient that is just small enough to avoid detection by master station
25 January 2011
Integrity for Non-Aviation Users: Moving Away from "Specific Risk"
13
Simplified Example: Ionospheric Spatial Decorrelation (3)
Simulated results for Memphis GBAS impacted by severe ionospheric gradient (RTCA 24-SV GPS, 6-km, User-to-ground separation, 1 and 2-SV impacts)
0.14
0.12
Most errors are exactly zero due to ground detection and exclusion, but all zero errors have been removed from the histogram.
0.1
0.08
Most plotted (non-zero) errors are below 10 m even under severe conditions.
0.06
0.04
Worst-case error, or “MIEV”, is
41 m
0.02
0 0 5 10 15 20 25 30 35
User Vertical Position Error (meters)
Integrity for Non-Aviation Users: Moving Away from "Specific Risk"
40 45 25 January 2011 14
Benefits of an “Average Risk” Approach
(Potential SBAS PL Reduction)
40 35
95% VPL
30 25
95% HPL Max. 95% PLs among stations in CONUS (“InOut” set)
20 15 10
Adjusted VPL
Conservative reduction factors from PAN data: VPL / 4.0
5
Adjusted HPL
HPL / 2.5
From reports since Jan. 2008
• 0 24 25 26 27 28 29 30
PAN Report Number
31 32 33 34
“Average risk” approach supports large reductions in HPL and VPL implied by WAAS PAN data, pending more complete database analysis.
•
Use “full-scale” PRA to re-assess “rare-normal” and faulted errors.
25 January 2011
Integrity for Non-Aviation Users: Moving Away from "Specific Risk"
15
A Combined “Average/Specific” Risk Approach
•
Depending on user and decision maker risk aversion, separate “average risk” and “specific risk” integrity requirements could be issued.
–
Both apply at all times
one or the other will tend to dominate for a particular application.
•
For example: 10 -7 integrity risk per operation (“ average ”) plus requirement that a worst-case undetected condition cannot increase the total vehicle loss risk by more than a factor of 10 .
–
For aircraft case, factor of 10 increase in total risk equates to specific risk requirement of 10 -5 (more strictly, 9
10 -6 ) per operation for nav. system
– –
Specific factors for each vehicle and application would vary.
There is no “correct” degree of risk aversion.
25 January 2011
Integrity for Non-Aviation Users: Moving Away from "Specific Risk"
16
Summary
•
Existing integrity assurance procedures for SBAS and GBAS are unique to aviation and its history and may not be suitable for other users.
•
SBAS (and GBAS) data analysis suggests that 10 -7 HPL and VPL can be greatly reduced if “average risk” approach is taken.
–
Examination of past data is useful, but more thorough PRA analysis should be conducted.
•
If worst-case elements of risk assessment are still desired, an average/specific risk mixture can be used.
–
This flexible “mixture” capability should satisfy almost any level of user and decision maker risk aversion.
25 January 2011
Integrity for Non-Aviation Users: Moving Away from "Specific Risk"
17
Backup Slides follow…
25 January 2011
Integrity for Non-Aviation Users: Moving Away from "Specific Risk"
18
25 January 2011
WAAS VPE from FAA PAN Data
(3 rd Qtr 2010: July – Sept.) Source: WAAS PAN Report #34, Oct. 2010.
http://www.nstb.tc.faa.gov/ DisplayArchive.htm
Max. VPE
7 m at Barrow, AK Meas. from 37 WAAS stations
19
(from PAN #34)
Example Error Table from PAN #34
25 January 2011
Integrity for Non-Aviation Users: Moving Away from "Specific Risk"
20
Max. VPE and VPL from WAAS PAN Data
(1 Jan. 2004 – 30 Sept. 2010) PAN Report Inner WRS Inner VPE_95% Inner VPE_Max
8 9 10 11 12 13 14 15 16 17 18 19 29 30 31 32 33 34 20 21 22 23 24 25 26 27 28
Ave
Chicago Dallas Dallas Dallas Dallas Dallas Dallas Dallas Albuquerque Dallas Dallas Dallas Dallas Dallas Denver Kansas City Memphis Denver Denver Denver Chicago Cleveland Dallas Denver Cleveland Denver Memphis 1.086
1.442
1.388
1.371
1.298
1.504
1.141
1.469
0.934
1.202
1.210
1.281
1.184
1.028
1.281
0.945
0.889
0.800
1.100
1.022
0.852
1.041
1.001
1.108
1.001
0.938
1.048
1.132
7.541
8.191
8.722
8.280
9.301
9.457
6.426
6.719
8.195
7.893
6.888
6.879
6.040
5.064
3.975
5.016
4.800
3.401
5.025
4.571
4.046
4.664
4.459
5.045
4.143
4.754
4.070
6.058
Inner VPL
(all numbers are in meters)
Outer WRS Outer VPE_95% Outer VPE_Max
49.612
39.956
43.829
31.969
33.699
28.399
26.887
24.612
24.246
34.771
37.435
35.097
30.050
26.238
34.868
24.232
24.742
27.877
28.390
25.254
21.989
24.292
50.101
25.872
28.377
36.569
13.567
Minneapolis Minneapolis Minneapolis Minneapolis Salt Lake City Minneapolis Oakland Minneapolis Minneapolis Oakland Oakland Miami Seattle Miami Oakland Minneapolis Seattle Seattle Oakland Seattle Miami Miami Wash DC Miami Miami Miami Seattle
30.849
1.710
1.695
1.790
1.501
1.155
1.765
1.706
1.956
1.157
1.273
1.228
1.657
0.886
1.231
1.043
1.067
0.801
0.766
1.061
0.915
2.041
1.537
1.124
1.612
2.005
1.298
0.849
1.364
9.133
7.794
7.376
8.034
8.581
12.756
7.931
7.439
8.002
6.385
7.296
6.913
5.858
5.160
4.119
5.029
4.273
4.553
4.808
4.972
4.462
4.384
4.589
4.240
4.738
4.516
4.920
6.232
Outer VPL Remote WRS Remote VPE_95% Remote VPE_Max Remote VPL
37.430
40.806
32.210
37.367
47.939
44.758
37.235
28.722
31.380
47.296
46.769
46.396
22.705
37.664
30.970
32.445
20.643
23.230
23.802
20.294
28.787
29.033
33.014
24.229
26.618
30.514
37.557
N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A Fairbanks Fairbanks Kotzebue Fairbanks Puerto Vallarta Tapachula San Juan S.J. Del Cabo Iqaluit Fairbanks Barrow Iqaluit Iqaluit Iqaluit Barrow Barrow
33.326
N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A 1.080
1.062
1.183
1.118
1.466
1.917
1.300
1.138
2.087
0.997
1.128
1.731
1.766
1.869
1.245
1.165
1.391
N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A 7.395
22.492
37.308
9.255
5.854
7.347
5.859
5.566
6.977
8.018
6.733
9.768
7.556
8.106
7.700
6.975
10.182
N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A 40.632
33.620
39.902
34.793
40.937
44.259
31.842
31.806
28.362
35.478
26.198
42.103
27.882
45.033
38.500
44.427
36.611
Max
25 January 2011
1.504
9.457
50.101
2.041
12.756
47.939
Integrity for Non-Aviation Users: Moving Away from "Specific Risk"
2.087
37.308
45.033
21
95% and Max. VPE from FAA PAN Data
(1 Jan. 2004 – 30 Sept. 2010)
50 45 40 35
Severe iono. scintillation in Alaska in March and May 2007 (user receiver should prevent) VAL for LPV Note: VPL always bounds VPE.
30 25
Remote Stations
20 15
Outer Stations Max. VPE 95 % VPE
10 5 0
Inner Stations
10 15 20 25
Quarterly PAN Report Number (8 – 34)
Integrity for Non-Aviation Users: Moving Away from "Specific Risk"
30 35 25 January 2011 22
WAAS VPE vs. VPL in CONUS (2003 – 2006) ( from Wanner, et al, 2006)
Ratios
: VPL 99.99
%
VPE
6.9
6.4
6.8
6.5
6.0
6.8
6.8
99.99% VPE 99.9% VPE 99% VPE 95% VPE Mean VPE 1
s
VPE
Integrity for Non-Aviation Users: Moving Away from "Specific Risk"
25 January 2011 23
WAAS Max. VPE in CONUS (2003 – 2006)
(from Wanner, et al, 2008)
25 January 2011
Integrity for Non-Aviation Users: Moving Away from "Specific Risk"
24
•
An “Average Risk” Approach to SBAS (and GBAS) – word version
Data imply an “average risk” equivalent VPL for WAAS ~ 4 – 5 times lower than current value.
•
Re assess “rare-normal” and faulted error models and data to build a “certifiable” safety case.
–
Multiple rare normal (“fault-free”) models built from existing data to incorporate remaining uncertainty
– –
Faults whose impact is driven by worst-case scenarios (ionosphere, signal deformation) will become less important .
–
All fault-mode analyses follow the same approach:
• •
Estimate prior fault probabilities and probability uncertainties.
Simulate all significant variations of each fault type rather than “worst case” focus
convolve with prior dist. to estimate risk.
Multiple-fault scenarios neglected as too improbable may become more important , as probabilistic weighting of risk may show that fault-combination cases are non-negligible.
25 January 2011
Integrity for Non-Aviation Users: Moving Away from "Specific Risk"
25
A Combined “Average/Specific” Risk Approach (1)
Derived from FAA “Hazard Risk Model” (1) and Simplified Aircraft Accident Risk Breakdown (2)
Major
(Slight risk of aircraft loss/pilot challenged) 10 -5
10 -6 Hazardous
(Risk of a/c loss; Severe loss of safety margin) 10 -7
10 -7
~ 1%
Catastrophic
(Likely a/c hull loss) 10 -9
10 -9
Overall a/c loss prob.
~ 10%
Loss prob. due to equipment failure
~ 1% (~ 100 systems)
Loss prob. due to GNSS nav. failure (1) (2) FAA System Safety Handbook, 2008. http://www.faa.gov/library/manuals/aviation/risk_management/ss_handbook/ R. Kelly and J. Davis, “Required Navigation Performance (RNP),” Navigation, Spring 1994.
25 January 2011
Integrity for Non-Aviation Users: Moving Away from "Specific Risk"
26