WAAS Test Program Overview & In-service Monitoring

Tom McHugh, ATO-P (ACB-430) DOT/FAA W.J.H. Technical Center WAAS Technical Director [email protected]

(609) 485-5183

Test Program Structure

• •

WAAS Test Program

–

Contractor Testing

• • •

Design Qualification Production Acceptance Analysis & Simulation

–

FAA Testing

• •

Operational Test & Evaluation Shakedown Testing

–

Testing of Upgrades In-service Monitoring

– – –

Performance Evaluation WIPP Off-line Monitoring Anomaly Detection & Resolution

Test Challenges

• • • •

The WAAS test program was structured on historic FAA processes used for developing and fielding new systems

–

Some of those processes are not applicable outside of the FAA Those processes required tailoring because they were not designed for SBASs SBASs have unique features that require innovation in the test program

– –

Integrity performance numbers that cannot be proven by test or demonstration Performance is a function of the constellation and the ionosphere which are outside the control of the SBAS

–

SBASs are single systems with geographically distributed components

•

The navigation service in the immediate vicinity of the component is not directly tied to that component

– –

Difficult (nearly impossible) to make corrections to the DO-229 Access to GEOs SBASs are new

– –

The FAA was still learning how to build and test WAAS while we were doing it How SBASs work is generally not well understood by all stakeholders

System Test and Monitoring Facilities

• • • •

WAAS reference stations (WRS) IP multicast their data onto the WAAS network Real time one-way access to that data is provided to:

– –

The WAAS prime contractor's facility in Fullerton CA The FAA Technical Center in Atlantic City NJ

•

FAATC also forwards the measurement data to NOAA/NGS for incorporation into the CORS network

–

The Mike Monroney Aeronautical Center in OKC Access to the real time measurement data allows WAAS master station equipment in the prime contractor's facility to function as if it were operational

– – –

WAAS Shadow System Extensively used for system level testing Currently WAAS has 5 shadow systems in the Fullerton facility Independent measurement data from the FAA National Satellite Test Bed is also fed from FAATC to the Raytheon facility in Fullerton CA

Contractor Testing

•

Contractor testing

–

Hardware

•

Subsystem

–

Factory acceptance

–

Site acceptance

•

COTS

–

OEM vendor data and testing

–

Software

•

FAA required processes (FAA-STD-026)

•

RTCA DO-178B

–

System

•

Algorithm validation and tuning

•

Comparison runs

•

Build verification testing

• •

System integration and stability testing Design qualification tests

–

60 Day Stability & Performance Test

• • • •

Regression tests Service volume modeling Safety analyses Reliability analyses

Test Capabilities / Tools

• • • •

All measurement data is always recorded Simulations can be generated from that recorded data

– –

Modified or un-modified Test Support Software (TSS) used for unit and configuration item level software testing Sim C&V

–

Off-line version of WAAS corrections and verification subsystem master station software

•

Much quicker than real time

–

Created from the operational software code using compile time switches

–

Can repeatedly test WAAS performance on any recorded data set

•

Algorithm implementation validation

• •

Anomaly resolution Regression testing Traceability maintained in a DORS database

Service Volume Modeling (SVM) Tools

• • • • • •

MITRE SVM Stanford University SVM Raytheon SVM Volpe Service Prediction NOTAMS Tool FAATC Coverage and Airport Availability Tools Raytheon Service Monitoring System (SMS)

Upgrade Testing

•

Scope of testing is dependent on the nature of the upgrade

–

Standard testing

• • • • •

DO-178B objective satisfaction Requirements change verification Shadow system stability testing Build validation testing Cutover testing

–

Unique testing

•

Algorithm validation

• • •

HMI analysis Comparison run tests Integration and checkout of new hardware

FAA Testing (in support of IOC)

• •

Operational Test and Evaluation

–

Flight testing

• •

8 locations spread over the LPV service area 40 approaches per location

–

Static testing

•

Simulate aircraft at each of the WAAS and NSTB reference stations by applying the WAAS corrections to the receiver measurements

•

24/7 1 Hz for the entire 60 day test

–

FAATC has been monitoring WAAS performance 24/7 since November 1998

–

Degraded operations

•

simulated major system failures Shakedown Testing

–

Demonstrates that the operators and maintainers were sufficiently proficient to operate the system

FAA In-service Monitoring (continuous and upgrades)

•

Three types of in-service monitoring

–

WAAS PAN Report (Quarterly)

• • • • •

WAAS user service performance Availability and coverage Accuracy Outages Anomaly reporting

–

GPS PAN Report (Quarterly)

•

Civilian GPS performance

–

WIPP off-line monitoring

• • • • •

Broadcast ephemeris accuracy statistics Antenna multipath environment Signal Quality Monitoring (SQM) Periodic antenna position survey updates GPS failure rate statistics