Transcript Anomaly Detection for Prognostic and Health Management System Development Tom Brotherton

Anomaly Detection for Prognostic
and Health Management System
Development
Tom Brotherton
TB 6/22/06
1
New Stealth Technology
TB 6/22/06
2
Outline
• What is Anomaly Detection
– Different types of anomaly detectors
• Radial Basis Function Neural Net Anomaly Detector
–
–
–
–
The basics
Comparison with other neural net approaches
Feature ‘off-nominal’ distance measures
Training
• Implementations
– Continuous = Gas turbine engine monitoring
– Snap shot = Web server helicopter vibration condition indicators
• RBF NN & Boxplots
• Application to detection of helicopter bearing fault
• Application to monitoring fish behavior for water quality monitoring
TB 6/22/06
3
What is Anomaly Detection?
• Anomaly Detection = The Detection of Any
Off-Nominal Event Data
– Known fault conditions
– Novel event = New - never seen before data
• New type of fault
• New variation of ‘known’ nominal or fault data
• What is ‘Nominal’
– Sets of parameters that behave as expected
• Physics models
• Statistical models
TB 6/22/06
4
Approaches
•Ex: Gas Turbine Engine Deck: Component
level physics model
Physics
•State Variable Models (derived from physics)
Parametric
•Hybrid Model: Combine Physics + Empirical
- Estimate of physics
•JPL: BEAM (coherence = model of
linear relationships)
•Neural nets (non-linear relationships)
Empirical
- Derived from collected data
•Fused empirical: BEAM + NN
•Academic: Support Vector
•Simple statistics
Applicability
TB 6/22/06
5
Empirical Modeling
An anomaly
Idea: Theoretical boundary (multidimensional ‘tube’) that data should lie
within:
- Nominal data is inside the boundary
- Anomaly data is outside
Problem: How to estimate /
approximate the boundary?
Collected ‘Nominal’ Data
Problem: What
measurement(s) caused the
anomaly?
Problem: How far off-nominal
is the anomaly / feature?
TB 6/22/06
6
RBF Neural Net Anomaly Detection: The Idea
Radial Basis Function
(RBF) Neural Net Model
NN = Model for Nominal Data
•
Dynamic data = Lots of NN basis
units to model
•
•
•
?
‘Distance’ from
Nominal Model
= Sample of nominal data
= Sample of anomalous data
Distance measure = Function of
the signal set
Individual signal distances from
nominal = distance from
“closest” basis unit
–
Yes
•
•
Piecewise stationary
approximation
Detection can be for set of
signals when no single signal is
anomalous
The model can be adaptively
updated to include additional
data / known fault classes
Trajectories of features relative
to basis unit = Prognosis
TB 6/22/06
7
Why Use Radial Basis Function Neural Nets?
• Radial Basis Function Neural Net
– Nearest neighbor classifier
– Distance metric : Measure “nominal”
– Multi-layer perceptron (MLP) does not have these
properties
MLP NN
RBF NN
?
?
TB 6/22/06
8
Support Vector Machine
Training data
Support Vector
Machine Model
RBF
Model
• In some sense, much better model
of ‘truth’ …. but
- Automated selection of number of
basis units
• Lots!
• Trade off between fidelity vs
smoothness
• Not practical for on-wing
• How to compute individual signal
distances
• Loss of intuition
TB 6/22/06
9
Feature Distance Calculation
NN = Model for Nominal Data
Mahalanobis
Distance s2
Mahalanobis
Distance s1
?
 Nearest Neighbor Distance
TB 6/22/06
10
Alternative Distance Calculation
NN = Model for Nominal Data
Closest
Basis Unit
Truth
- Truth: Single Feature X = ‘Bad’
-Report: Feature X = ‘OK’ & Feature Y = ‘Bad’
-Alternative Distance = Which Basis Unit gives the smallest number of
individual off-nominal features
-> Hamming Distance (from digital communications decoding)
TB 6/22/06
11
‘RBF’ NN Architectures
Input features
Weights

•
•
•
Detector
Output
Is output for Nominal?
=1
 Yes
> 1-  Likely
< 1-  ?
< 1-  No
0<  <  <1
Basis Units
Gaussian elliptical basis function :
= Gaussian Mixture Model
Rayleigh basis function :
Fuzzy membership basis function :
Good for magnitude spectral data
* Basis function is ‘matched’ to the data distribution
For those who like things fuzzy
TB 6/22/06
12
Training : Neural Net Architectures – How to
select parameters
- Small number of clusters
 Small number of basis units
 Low False Alarms
Very general
 Missed detections

Too General ?
- Large number of clusters
 Good ‘tracking’ of data dynamics
 Large number of basis units
More sensitive to outliers
 More false alarms

Over Trained ?
Don’t know a-priori what are the ‘best’ settings
TB 6/22/06
13
M of N Detection
Idea: M of N detection allows one sample high false
alarm rate – Then integrate over time to remove
Only 2 points
= false alarm
False alarms?
Large scale factor
Small
scale
factor
4 points persist over
time = detection
•
Trade off single point detection
capability vs false alarm rate
 Large Scale Factor / Small N
-
Short – high SNR anomalies
 Small Scale Factor / Large N
-
Long – persistent – low SNR
anomalies
Detection?
False alarm?
TB 6/22/06
15
Alternatives
• This technique works well
– Demonstrated by Pratt & Whitney for C-17 F117
applications
• Transient engine operations
– Long time to train – lots of different types of transients
– Model can become very complex
• Engine control system
• On-wing memory and timing constraints
• Alternative
– Combine equipment operating regime recognition with
anomaly detector
– Ex: Identify steady operation and then take a snapshot of
the data
• Simple statistics may suffice
TB 6/22/06
16
Example Gas Turbine Operations
Input Signal
Vector
Scale Signal
Break the big problem in to a set
of small problems
Regime recognition
Regime
Recognition
-
Regimes:
•
•
•
•
Neural Net
Select
Neural Net
Neural
NeuralNet
Net
Detection
Detection
Detection
Transient Throttle up
Transient Throttle down
Steady state – B14 open
Steady state – B14 closed
Median Filter
Trained NNs
Off-Nominal
Signal
Distance
Detection
Flag
TB 6/22/06
17
Anomaly Detection of Stationary Regime
Detected Data
• Web Server Implementation for Helicopter
Vibration Data
– Condition Indicators (CIs) = Features derived
from on-board vibration measurements
• Two types of problems:
– Single CI for a component
• Simple statistics solution = Boxplot
– Intuitive = Army user’s like it
• RBF neural net implementation as well
– Multi-CIs for a component
• RBF neural net implementation
TB 6/22/06
18
On Board System
FWDLAT
FWDVRT
FWDSP
CPITVRT
CPITLAT
FWDXMSNVRT
FWDXMSNLAT
Advanced Rotor
Smoothing / Engine
Diagnostics
HB2
HB3
HB4
HB5
HB6
HB7
Engines
ENG1COMP
ENG1NOSE
ENG1AXIAL
ENG1LAT
ENG2COMP
ENG2NOSE
ENG2AXIAL
ENG2LAT
Transmissions
XSHAFT1
XSHAFT2
AFTLAT
AFTVRT
AFTSP
Tail Gearbox
CBOXOCFA
CBOXOCLAT
APU
AFTFANLAT
AFTXMSNVRT
AFTXMSNLAT
Intermediate
Gearbox
Configuration
• 36 Vibration Sensors
• 2 Speed Sensors
• 1553 connection to HUD
Main
Rotor
Cockpit
VMU
Main
D/S
Cockpit Control Head
USB Memory Drive
Parameter Data
Absorbers
CVR-FDR
Hanger
Bearings
USB Download
IAC-1209
• 18 Sensors Installed – Vibration
Ethernet
Modern Signal
• Automated Exceedance Monitoring
using HUD data
Processing Unit
• Automated engine HIT, Max Power
Check and exceedances
+28VDC Power
(MSPU)
• Complete aircraft vibration survey in under 30 seconds
Accelerometer
Tach Sensor
Other Connections
TB 6/22/06
19
Aircraft / Server Physical Connectivity
SCARNG
USB Memory Stick
Data Download
AIRCRAFT
OEMs
VMEP
PARTNER
Browser
PC-GBS Remote
PC-GBS Facility
AARNG
INTERNET
Wireless link
PC-GBS Remote
PC-GBS Facility
Deployed Unit
PC-GBS Remote
TB 6/22/06
20
Aircraft / Server Logical Connectivity
Facility Systems
Portable System
-Army P-GBS
Aircraft Maintenance
-Electronic help desk
- Automated data archive
- Automated s/w upgrades
Support Team
- e-mail notification
- Fleet level reports
- Automated s/w upgrades
Web Client
- Army F-GBS
Browser
MDS Server
Help Training Base
Electronic Manuals
FAQs
Network
Security
Help Desk
Diagnostics
Prognostics
Anomaly
Anomaly
Detection
Detection
Automated
Data Archive
Fleet Statistics
& Reports
Data Archive
A/C config files
TB 6/22/06
21
Advanced Engineering on the Web
The role of anomaly detection on the
website is to detect and bring to
engineering’s attention the MOST
INTERESTING data = Something that has
NOT been encountered before
- More normal data not really of interest
TB 6/22/06
22
Single Feature Anomaly Detection
Boxplots = Simple statistics - single feature
anomaly detector. No Gaussian assumption, just
counting points. They seem to work very well!
Default based on
boxplot statistics
User set
TB 6/22/06
23
Threshold Setting
TB 6/22/06
24
Anomaly Analysis
Summary of all
aircraft
TB 6/22/06
25
The Raw Data
TB 6/22/06
26
Gaussian Transformation Data
• Problem: How to select a “matched” basis function
– Gaussian assumption? Usually violated!
• Statistical Model Fit
– Transform data to be Gaussian
• Transformation stored and is part of the model
– Almost always only a single basis unit is required!
• Works on single feature data
• All processing “behind the scenes” done on transformed data
Original
Transformed
TB 6/22/06
27
RBF Anomaly Detection
TB 6/22/06
28
RBF Anomaly Detection
TB 6/22/06
29
Case Study: Apache Swashplate Bearing Spectral
Server Data
•
Anomalous data identified with RBF NN AD running on the Server
– Aircraft was in Iraq
– Automatic email alert sent to users
• “Evidence” sent as well
– Data reviewed by AED-Aeromechanics and IAC via iMDS website
• Large peak in spectral data at 1250 Hz for tail #460
• Sidebands spaced at intervals corresponding to bearing fault frequencies
•
Suspected bad swashplate bearing
Main SP Spectra
5
Tail 460
Tail 460
4
Magnitude (g)
Other
A/C
3
2
Tail 460
1
Tail 986
Other A/C
0
0
2000
4000
Frequency (Hz)
6000
TB 6/22/06
30
Case Study
Apache Swashplate Bearing
•
AED-Aeromechanics acquired raw vibe data Apr 04 and received
swashplate May 04 before aircraft was turned-in for D model conversion
•
Swashplate disassembled by PIF per DMWR Aug 04
•
Minor spalling, corrosion and broken cage discovered
•
Additional algorithms developed from raw data and implemented into
VMEP for release Sep 04
Broken Cage
Spalling/Corrosion
TB 6/22/06
31
Follow Up
• Specific algorithms to identify this fault now
included with the on-board system
• US Army now uses ‘on-condition’ information
from the system to perform maintenance
– True condition-based maintenance (CBM)
TB 6/22/06
32
Other Applications
Water Quality Bio-Monitor
IAC 1090 is a mobile, web-enabled automated
biomonitoring system that utilizing the ventilatory and
body movement patterns of the bluegill fish as a biosensor, much like a canary in a coal mine.
Sixteen Bluegills are placed in individual flowthrough Plexiglas chambers. Each chamber is
equipped with an individual water input and drainage
system. By utilizing sixteen different Bluegills, the
IAC 1090 samples more biosensors than any other
system on the market resulting in lower false alarm
rates.
All fish generate a micro volt level electric field.
Each individual fish is monitored by non-contact
electrodes suspended above and below each fish in
a Plexiglas chamber.
The electrical signals generated by the fish’s
normal movement is amplified, filtered and passed
on via the internet to IAC’s Bio-Monitoring Expert
(BME) software system for automated analysis.
TB 6/22/06
33
Water Quality Bio-Monitor
BME is a neural network based expert system that
provides for rapid, real time assessment of water
toxicity based on the ventilatory behavior of fish. BME
has shown excellent detection capabilities for toxic
compounds with a low false alarm rate. False alarms,
common in other similar systems, are typically
generated by normal, non-toxic variations in the
environment.
Automated data collection and management tools,
user interfaces, and real-time data interpretation
employing advanced (artificial intelligence) models of
fish ventilatory behavior make BME easy to use.
Remote (Internet) access to IAC 1090 is provided
through an easy-to-use graphical user interface. BME’s
modular design provides users with the ability to
reconfigure the system for different biomonitoring
applications and biosensors
TB 6/22/06
34
Questions?
Conference papers / case studies
available at:
www.iac-online.com
TB 6/22/06
35