Transcript Slide 1

Formal Methods in
Safety-Critical Systems
Dr. Steven P. Miller
Advanced Computing Systems
Rockwell Collins
400 Collins Road NE, MS 108-206
Cedar Rapids, Iowa 52498
[email protected]
Advanced Technology Center
Slide 1
What Problem are We Solving?
 Safety-Critical Software Is Too Expensive
Cut Development Costs/Cycle Time in Half
 Safety-Critical Software Is Often Wrong
Find 10x More Errors than Current Methods
 DO-178B Certification Is Too Expensive
Already Applying This to DO-178B Developments
Advanced Technology Center
Slide 2
Are We Making Progress?
 Model-Based Development Spreading Rapidly
Several projects at Rockwell Collins
 Prove Properties of Simulink & SCADE Models
In Seconds on Models with Over 10**100 States
 Finding Errors Early in the Lifecycle
On Real Products!
Advanced Technology Center
Slide 3
Outline of Presentation
Introduction
Overview of Our Approach
An Example – FGS Mode Logic
Some Recent Accomplishments
The Underlying Technology
What’s Next?
Summary
Advanced Technology Center
Slide 4
Who Are We?
A World Leader In Aviation Electronics And Airborne/ Mobile
Communications Systems For Commercial And Military Applications
Communications
Navigation
Automated Flight Control
Displays / Surveillance
Aviation Services
In-Flight Entertainment
Integrated Aviation Electronics
Information Management Systems
Advanced Technology Center
Slide 5
Rockwell Collins
Headquartered in Cedar Rapids, Iowa
14,500 Employees Worldwide
Advanced Technology Center
Slide 6
RCI Advanced Technology Center
Government Systems
Commercial Systems
Advanced Technology Center
 The Advanced Technology Center (ATC) identifies, acquires, develops and
transitions value-driven technologies to support the continued growth of
Rockwell Collins.
 The Automated Analysis group applies mathematical tools and reasoning to
the problem of producing high assurance systems.
Advanced Technology Center
Slide 7
Automated Analysis Group

Participants in the MCC Formal Methods Transition Study
1991

Formal Specification of the μReal Time Executive in RAISE
1992

Formal Specification of the GE1 Graphics Processor
1996

Formal Verification of Microprocessors
– AAMP5 Microcode Using PVS
– AAMP-FV Microcode Using PVS
– JEM Java Virtual Machine Microprocessor Using PVS
– FCP2002 Microcode Using ACL2
– FCP 2002-2000 Microcode Equivalence Using ACL2
– AAMP7 Security Separation Kernel Using ACL2
1993 - 2005
Formal Validation of Embedded System Requirements
– FGS Mode Logic using SPC’s CoRE Method
– FGS Mode Logic using NRL’s SCR* Tools
– FGS Mode Logic Using PVS
– FGS Mode Logic Using Matrix-X and T-VEC
– FGS Mode Logic Using RMSL-e, PVS, and NuSMV
– FGS/FMS/AT Logic Using SCADE and Simulink
1995 - 2005

Advanced Technology Center
Slide 8
1994
1995
1998
1999
2001
2003
1995
1996
1997
1998
2002
2004
Methods and Tools for
Flight Critical Systems Project
 Five Year Project Started in 2001
 Part of NASA’s Aviation Safety Program
(Contract NCC-01001)
 Funded by the NASA Langley Research Center
and Rockwell Collins
 Practical Application of Formal Methods To
Modern Avionics Systems
Advanced Technology Center
Slide 9
Outline of Presentation
Introduction
Overview of Our Approach
An Example – FGS Mode Logic
Some Recent Accomplishments
The Underlying Technology
What’s Next?
Summary
Advanced Technology Center
Slide 10
Convergence of Two Trends
Model-Based
Development
Automated
Analysis
A Revolutionary Change in How
We Design and Build Systems
Advanced Technology Center
Slide 11
Model-Based Development Examples
Company
Product
Tools
Specified & Autocoded
Benefits Claimed
Airbus
A340
SCADE
With Code
Generator
 20X Reduction in Errors
 Reduced Time to Market
Eurocopter
EC-155/135
Autopilot
GE &
Lockheed
Martin
Schneider
Electric
FADEDC Engine
Controls
SCADE
With Code
Generator
ADI Beacon





US
Spaceware
PSA
CSEE
Transport
Honeywell
Commercial
Aviation
Systems
70% Fly-by-wire Controls
70% Automatic Flight Controls
50% Display Computer
40% Warning & Maint Computer
90 % of Autopilot
 Not Stated
Nuclear Power
Plant Safety
Control
DCX Rocket
SCADE
With Code
Generator
MATRIXx
 200,000 SLOC Auto Generated
from 1,200 Design Views
Electrical
Management
System
Subway
Signaling System
SCADE
With Code
Generator
SCADE
With Code
Generator
MATLAB
Simulink
 50% SLOC Auto Generated
Primus Epic
Flight Control
System
Advanced Technology Center
 Not Stated
 50% Reduction in Cycle Time




Reduction in Errors
50% Reduction in Cycle Time
Decreased Cost
8X Reduction in Errors while
Complexity Increased 4x




50-75% Reduction in Cost
Reduced Schedule & Risk
60% Reduction in Cycle Time
5X Reduction in Errors
 80,000 C SLOC Auto Generated
 Improved Productivity from
20 to 300 SLOC/day
 60% Automatic Flight Controls
 5X Increase in Productivity
 No Coding Errors
 Received FAA Certification
Slide 12
Does Model-Based Development Scale?
Airbus A380
Systems Developed Using MBD
• Flight Control
• Auto Pilot
• Flight Warning
• Cockpit Display
• Fuel Management
Length
239 ft 6 in
• Landing Gear
Wingspan
261 ft 10 in
• Braking
Maximum Takeoff Weight 1,235,000 lbs
Passengers
Up to 840
Range
9,383 miles
Advanced Technology Center
Slide 13
• Steering
• Anti-Icing
• Electrical Load Management
How Do We Reduce Costs
and Improve Quality?
Requirements
Elicitation
Reuse
15% Autotest
Modeling 10%
Reduces Cost of Testing
Clear Specifications
Improves Communication
Enables More Testing
5% Autocode
Simulation 10%
Eliminates Manual Coding
Easy Validation
Makes Model Primary Artifact
Finds Errors Early
Automated
Analysis 10% - 20%
Cheaper Than Manual Analysis
Advanced Technology Center
Slide 14
Finds the Really Hard Errors
Outline of Presentation
Introduction
Overview of Our Approach
An Example – FGS Mode Logic
Some Recent Accomplishments
The Underlying Technology
What’s Next?
Summary
Advanced Technology Center
Slide 15
Flight Guidance System Mode Logic
Requirements
Elicitation
Reuse
Modeling
Autotest
Simulation
Autocode
Automated
Analysis
Advanced Technology Center
Slide 16
Captured Requirements as Shalls
Advanced Technology Center
Slide 17
Modeling
Requirements
Elicitation
Reuse
Modeling
Autotest
Simulation
Autocode
Automated
Analysis
Advanced Technology Center
Slide 18
Modeling Notations
Textual (Lustre, PVS, SAL, …)
Tabular (RSML-e, SCR)
node Thrust_Required(
FG_Mode : FG_Mode_Type ;
Airborne : bool ;
In_Flare : bool ;
Emergency_Descent : bool;
Windshear_Warning : bool ;
In_Eng_Accel_Zone : bool ;
On_Ground : bool)
returns (IsTrue : bool) ;
let
IsTrue =
(FG_Thrust_Mode(FG_Mode) and
Airborne)
or
(Airborne and Emergency_Descent)
or
Windshear_Warning
or
((FG_Mode = ThrottleRetard) and
In_Flare)
or
(In_Eng_Accel_Zone and On_Ground) ;
tel ;
Advanced Technology Center
Slide 19
Graphical (Simulink, SCADE)
Simulation
Requirements
Elicitation
Reuse
Modeling
Autotest
Simulation
Autocode
Automated
Analysis
Advanced Technology Center
Slide 20
Simulation
Advanced Technology Center
Slide 21
Automated Analysis
Reuse
Requirements
Elicitation
Modeling
Autotest
Simulation
Autocode
Automated
Analysis
Model Checkers
Advanced Technology Center
Theorem Provers
Slide 22
What Are Model Checkers?
 Breakthrough Technology of the 1990’s
 Widely Used in Hardware Verification (Intel, Motorola, IBM, …)
 Several Different Types of Model Checkers
– Explicit, Symbolic, Bounded, Infinite Bounded, …
 Exhaustive Search of the Global State Space
– Consider All Combinations of Inputs and States
– Equivalent to Exhaustive Testing of the Model
– Produces a Counter Example if a Property is Not True
 Easy to Use
– “Push Button” Formal Methods
– Very Little Human Effort Unless You’re at the Tool’s Limits
 Limitations
– State Space Explosion (1020 – 10300 States)
Advanced Technology Center
Slide 23
Advantage of Model Checking
Testing Checks Only the Values We Select
Even Small Systems Have Trillions (of Trillions) of Possible Tests!
System
Advanced Technology Center
Slide 24
Advantage of Model Checking
Model Checker Tries Every Possible Input and State!
Model
Advanced Technology Center
Slide 25
Model Checking Process
SMV
Spec.
Model
Automatic Translation
Does the system
have property X?
Counter Example
Yes!
SMV
Automatic Translation
Engineer
Advanced Technology Center
SMV Properties
Properties
Slide 26
Translated Shalls into SMV Properties
Advanced Technology Center
Slide 27
Validate Requirements
through Model Checking





Proved Over 280 Properties in Less Than an Hour
Found Several Errors
Some Were Errors in the Model
Most Were Incorrect Shalls
Revised the Shalls to Improve the Requirements
Advanced Technology Center
Slide 28
Translator Optimizations
CPU Time
Model
(To Compute Reachable States)
Improvement
Mode1
Before
> 2 hours
After
11 sec
Mode2
> 6 hours
169 sec

Mode3
> 2 hours
14 sec

Mode4
Arch
8 minutes
34 sec
< 1 sec
< 1 sec
480x
34x
WBS
29+ hours
1 sec
Advanced Technology Center
Slide 29

105,240x
What are Theorem Provers?
 Available Since Late 1980’s
– Widely Used on Security and Safety-Critical Systems
 Use Rules of Inference to Prove New Properties
– Also Consider All Combinations of Inputs and States
– Also Equivalent to Testing with an Infinite Set of Test Cases
– Generate An Unprovable Proof Obligation if a Property is False
 Not Limited by State Space
– Applicable to Almost Any Formal Specification
 Limitations
– Require Experience - About Six Months to Become Proficient
– Constructing Proofs is Labor Intensive
Advanced Technology Center
Slide 30
Theorem Proving Using PVS
Model
PVS
Spec.
Automatic Translation
Why not?
Does the system
have property X?
Guru
PVS
Automatic Translation
Engineer
Advanced Technology Center
Properties
PVS Properties
Slide 31
Validate Requirements
Using Theorem Proving
 Proved Several Hundred Properties Using PVS
 More Time Consuming that Model-Checking
 Use When Models are Stable and Model-Checking Won’t Work
Advanced Technology Center
Slide 32
Outline of Presentation
Introduction
Overview of Our Approach
An Example – FGS Mode Logic
Some Recent Accomplishments
The Underlying Technology
What’s Next?
Summary
Advanced Technology Center
Slide 33
Example 1 – Mode Logic
Mode Controller A
6.8 x 1021 Reachable States
Mode Controller B
Requirement
Mode A1 => Mode B1
Counterexample Found in
Less than Two Minutes!
Found 27 Errors to Date
Advanced Technology Center
Slide 34
Example 2 – Displays Logic
883 Subsystems
9,772 Simulink Blocks
2.9 x 1052 Reachable States
Requirement
Drive the Maximum Number of Display Units
Given the Available Graphics Processors
Counterexample Found in 5 Seconds!
Checked 178 Properties – Found Several Errors
Advanced Technology Center
Slide 35
Outline of Presentation
Introduction
Overview of Our Approach
An Example – FGS Mode Logic
Some Recent Accomplishments
The Underlying Technology
What’s Next?
Summary
Advanced Technology Center
Slide 36
Original Tool Chain
RSML-e to NuSMV
Translator
NuSMV Model Checker
RSML-e
RSML-e to PVS
Translator
Rockwell Collins/U of Minnesota
SRI International
Advanced Technology Center
Slide 37
PVS Theorem Prover
Conversion to SCADE
NuSMV
SCADE
Lustre
Safe State
Machines
Design
Verfier
Rockwell Collins
Esterel Technologies
SRI International
Advanced Technology Center
PVS
Slide 38
Extension to MATLAB Simulink
Simulink
NuSMV
Simulink
Gateway
SCADE
Lustre
StateFlow
Safe State
Machines
Design
Verfier
Rockwell Collins
Esterel Technologies
SRI International
MathWorks
Advanced Technology Center
PVS
Slide 39
Adding SRI Tools to the Chain
Simulink
NuSMV
Simulink
Gateway
SCADE
PVS
Lustre
StateFlow
Safe State
Machines
ACL2
Design
Verfier
ICS
Rockwell Collins
SAL
Esterel Technologies
SRI International
MathWorks
Advanced Technology Center
Symbolic
Model Checker
Bounded
Model Checker
Infinite
Model Checker
Slide 40
Current Tool Chain
Simulink
NuSMV
Simulink
Gateway
SCADE
PVS
Reactis
StateFlow
Lustre
Safe State
Machines
ACL2
Design
Verfier
ICS
Rockwell Collins
SAL
Esterel Technologies
SRI International
MathWorks
Bounded
Model Checker
Infinite
Model Checker
Reactive Systems
Advanced Technology Center
Symbolic
Model Checker
Slide 41
Outline of Presentation
Introduction
Overview of Our Approach
An Example – FGS Mode Logic
Some Recent Accomplishments
The Underlying Technology
What’s Next?
Summary
Advanced Technology Center
Slide 42
Extending the Verification Domain
Theorem Provers
SAT-Based
BDD-Based
Arbitrary
Systems
Model
Checkers
Model Checkers
(Real
Numbers,
Large
Integers,
Infinite
State…)
Complex
Boolean
& Enumerated
Types
+ Integers
& Reals
Boolean
& Enumerated
Types
Very Large State Spaces
Infinite State Spaces
Advanced Technology Center
Slide 43
Verification of Adaptive Systems
Advanced Technology Center
Slide 44
Requirements Based Test Case Generation
Requirements
Properties
Requirements Based Testing

State Requirements as Properties

Automatically Generate Tests

Goal is to Cover the Requirement
Create
Model
Create
Test
Case
Requirements
Generator
Based Tests
Model
Conformance Testing

Autogenerate Test Cases From Model


Commercial Tools Available
– (T-VEC, REACTIS)
Show Code Conforms to the Model

Goal is Structural Coverage (MC/DC)
Advanced Technology Center
Slide 45
Code
Generator
Code
Create
Test
Case
Additional
Generator
Structural
Tests
Model-Based Safety Analysis
Green Pump
Blue Pump
Loss All
Braking
Isolation Valve Isolation Valve
Power A
Pedal 1
System
A
Shut
Normal
System
Plant
Feed back
Pedal 2
Power B
System
B
A Accumulator
L
Valve
T
E
Accumulator
R
Pump
N
Meter
A Valve
Mechanical
T
Pedal
E
N
O
R
M
A
L
AntiSkid
Command
Fault Tolerant
Braking System
Control Unit
( BSCU )
Normal Sys
Loss
Selector Valve
Braking+
AntiSkid
Command
Meter
Valve
Green Pump
Loss
Meter Valve
Loss
Alt Sys
Loss
BSCU Loss
of Command
Power
Supplies
Fail
Both Pumps
Fail
Acc/AS/Mech
Meter Fails
BSCU Select
Signal
Inverted
SelValve
Stuck
Blue Fails
Acc Fails
Meter
Valve
Plant
Model

Model the Physical System and the Digital Controller Architecture

Add Fault Model for Physical System and Digital Controller Architecture

Integrates System and Safety Engineering About a Common Model
 Automation Enables “What-If” Consideration of System Designs
Advanced Technology Center
Slide 46
Outline of Presentation
Introduction
Overview of Our Approach
An Example – FGS Mode Logic
Some Recent Accomplishments
The Underlying Technology
What’s Next?
Summary
Advanced Technology Center
Slide 47
Summary
 Formal Verification is Becoming Practical
– Availability of Accurate Models Early in the Lifecycle
– Growing Power of Automated Analysis Tools
 Benefits
– Find Errors Early
– Avoid Rework Late in the Lifecycle
– Cheaper and Easier than Traditional Methods
– Orders of Magnitude Better at Finding Errors
Advanced Technology Center
Slide 48
Summary
Rockwell Collins is a World Leader in the
Industrial Use of Formal Methods
 Almost 15 Years of Experience
 Thriving Automated Analysis Group
 Doing Extensive Work for NASA and the NSA
 Broad Tool Expertise
– PVS, ACL2, NuSMV, Prover, SAL, Simulink, SCADE, SCR, …
 Focus on “Application to Real Systems”
Advanced Technology Center
Slide 49