Transcript DO-17B Training, developed by Vance Hilderman 2005 & 2006

Avionics Development : Best & Worst Habits
By Vance Hilderman, President Atego HighRely – [email protected]
via BILDES Group, Ankara, Turkiye
1
Three Avionics Facts for 2012
 Simple Facts:

The average avionics development project is 55% over budget
and 30% behind schedule
 DO-178 increases first-time development costs by 25% - 40%
(Best Case)
 Re-use is rare, unless planned/paid up-front
 “DO-178 is the worst standard in the world … except for all the
others …” (Vance Hilderman, 1992)
2
“The Good, Bad, and the Ugly” (Clint Eastwood)
 Like Life, Avionics Development acquires “Good”, “Bad”, and
“Ugly” habits
 From 400+ projects for 95 of America and Europe’s top 100
avionics companies, the habits are surprising …
3
Bad Habits: Answers …
 Will bad habits fix themselves?
 Does FAA expect perfection?
 Is Safety more important than Money?
4
Bad Habits: Answers …
 Will bad habits fix themselves?
 Absolutely not
 Does FAA expect perfection?
 Absolutely not
 Is Safety more important than Money?
 Not necessarily …
5
Avionics Development Worst Habits - 2012
7
• Weak Plans & Ignoring
6
• Starting Over
5
• Not Understanding DO-297
4
3
• Not Understanding AC 20148
• Missing “A” in ARP-4754A
2
• Too many, or too few, open
problems
1
• Weak/Soft Requirements
6
Avionics Development Best Habits - 2012
6
• CMMI 3-4 – “Truly”
5
• Applying CAST-27
4
• Automated Testing
3
• Strong QA / Cert Liaison
2
• Strong Checklists
1
• Preventing Mistakes
instead of Fixing Misteaks
7
About Atego HighRely
 North America / Europe’s Largest Avionics Certification/Services
Company
⁻
30% Avionics Software Engineering
⁻
20% Avionics Systems Engineering
⁻
20% Avionics Software/Hardware Testing
⁻
10% Project Management, Strategy, Gap Analysis, JumpCert
⁻
10% DER’s/Certification
 Products for modelling, code-generation, managing
processes, plans, checklists
 Largest repository of DO-178 & DO-254
White Papers
8
Some “Mistakes” …
Safety
Assessment
ARP 4761
9
2012 Worst Habit #7: Weak Plans, or not Following Good Plans
Plans must be “Good” (not “Perfect”)
5 plans, 3 Standards
Must cover 71 Objectives
Medium Detail in Plans
“What”, not “How”
100+ Item checklist for
each plan
Good Plans - Ignored
Prove you follow plans
1) Engineering Checklist
2) QA Audit Checklist
10
2012 Worst Habit #6: Starting Over …
Rarely must you really “Start Over”
Perform “Gap Analysis”
Analyze “Gaps”,
Delta Close
Reverse Engineering can be Good
Reverse Engineering is
common in avionics
projects:
60% of hardware and
30% of software
11
2012 Worst Habit #5: Not Understanding DO-297
DO-297: “Integrated Modular Avionics” (IMA)
Basis of future avionics
Especially helicopters
(Why?)
IMA has certification challenges
Progressive
functionality roll-up
Extrapolated interface
usage & ARP-4754A
Even if not required, UNDERSTAND IMA
Apply advanced techniques
Improved re-use, fewer bugs
12
2012 Worst Habit #4: Not Understanding AC 20-148
AC 20-148
(Reusable
Software
Components)
Reduced
Cost,
Risk, &
Schedule.
DO-178C
Slide 13
13
What is AC 20-148?
AC 20-148
• FAA Advisory Circular AC 20-148:
• “Reusable Software Components”
(Reusable
Software
Components)
• Guidelines for building and deploying reusable software:
• Reduces re-certification effort & cost for “components”
• Promotes modern software principles & tools
• Promotes software’s Holy Grail: “Reuse”
Slide 14
14
Why Reuse?

DO-178 increases development costs by 25-40%

Long-term, DO-178 can be cost-effective

With reuse, DO-178 is cost effective on the 2nd project:
Typical
DO-178 Project
• Added 60% - 100% Cost
Successful
DO-178 Project
• Initial 25-40% $ increase
• 20-50% $ decrease 2nd time
Technical Project without • Solid processes
DO-178
• Experienced Team

Source: “DO-178 Costs versus Benefits” Whitepaper: HighRely – Vance Hilderman
Slide 15
15
How does AC 20-148 Help?

AC 20-148 is rigorous, and not required: “guidance” only

When formally used, can greatly simplify reuse and ‘recert”

But AC 20-148 has ample info, even when not formally used:
Establishes Reuse Framework
• Safety considerations
• Interface requirements
• Defined Tool usage
Defines DO-178 Reuse Activities • Risk analysis/mitigation
• Define/validate reuse
assumptions
• Subset for approval
submission
• Post-delivery support
Specifies documentation
submittals & follow-up
Slide 16
16
What is an “RSC”?

“Reusable Software Component” :
What is an RSC?
Previously Developed Software Component Intended for re-use
What is AC 20-148?
Means for full or partial cert credit to reduce future re-cert
Motivation?
Modern software trends in reuse
Economic: build once, use often
Slide 17
17
2012 Worst Habit #3: Missing “A” in ARP-4754A
ARP-4754 was Good
Avionics Systems
Development
ARP-4754 often
ignored
In 2012: FAA Mandates ARP-4754A
Thou Shall Use It
Thou Shall Prove It
Aircraft is more than sum of parts
“1 + 1 = 11 “
Consider Integrated Aircraft
18
2012 Worst Habit #2: Too many, or too few, open Problems
Identifying problems is Good
Preventing problems is better ;)
Too many open problems?
Too difficult to manage
Inefficient
Too few problems?
Dishonest
Too rigorous (OK to Flight Test with
open problems; state Impact)
19
2012 Worst Habit #1: Weak or Soft Requirements
#1 Cause of Avionics Defects?
Wrong Assumptions
Best way to minimize Assumptions?
Detailed, verifiable
Requirements
Istanbul versus
Cappadocia
Requirements “Contract” with “User”
Build a house without
knowing desired result?
Same for Avionics
20
Checklist, Checklist
2012 Best Habit #6: CMMI Level 3-4
 Carnegie Mellon University, Software Engineering Institute (SEI)
⁻
Devise a method of assessing and grading software organizations
⁻
“Capability Maturity Model” (CMM)
⁻
“Capability Maturity Model Integration” (CMMI)
⁻
Why that acronym?
21
SEI CMMI Maturity Levels
Level 1
 SEI CMMI’s 5 Levels:
Level 2
1. Initial
Level 3
2. Repeatable (disciplined)
Level 4
3. Defined (consistent))
Level 5
4. Managed (predictable)
5. Optimizing (continuous improvement)
 Each level is a perfect superset of the
preceding level
22
Typical Avionics Organization Questions
Level 1
 What is the average U.S. avionics
team’s level?
Level 2
 What Level should an avionics
team be?
Level 4
Level 3
Level 5
 What is your team’s level?
23
2012 Best Habit #5: Applying CAST-27
 DO-254 is “confusing” but increasingly required
 Different interpretation between USA and Europe
 How to resolve? Where are the Answers?
 CAST-27!!!
24
CAST-27 Purpose
 Clarify common misunderstandings when applying DO-254 to custom
micro-coded components:
⁻
ASICs, PLD’s, FPGAs
⁻
Addresses fact that Europe does not officially recognize AC 20-152
⁻
Try to clarify scope and application of DO-254 while “harmonizing” different
worldwide certification opinions
25
CAST-27 Focus
Addresses specific areas of DO-254 the especially need clarification:
 Modifiable devices (Section 4)
 Device level assurance (Section 5)
 Certification Plan (Section 6)
 Validation Processes (Section 7)
 Verification Processes (Section 8)
 Traceability (Section 9)
 Configuration Management (Section 10)
 Tool assessment and qualification (Section 11)
 COTS IP (Section 12)
26
Need a copy of CAST-27 or a Summary?
 Request from Atego HighRely via Bildes Group (Ankara)
27
2012 Best Habit #4: Automated Testing
Testing in DO-178/254 requires more time than development
Every change must be
regression tested
Huge expense over
product life
The Answer?
Automated testing
The Boeing Thrust
Reverser …
But which type of test tool?
System Testing
Low level Testing
28
Most popular?
VectorCast
2012 Best Habit #3: Strong QA / Cert Liaison
Who is the most important person on Avionics project?
Chief
Systems?
Chief
Software?
Chief
Tester?
QA?
QA manages Certification
FAA/Military Focal
Single Point Failure
QA Needs?
Proactive QA
Strong QA
checklists/audits
Metrics & Proof
29
Remember
Concorde
2012 Best Habit #2: Strong Checklist/Practices
Innocent until proven guilty?
Opposite …
No!
How to prove innocence?
Checklists, checklists
Beg, borrow, steal, or buy …
Ensure they cover all 71
Objectives
Customize for YOUR project
30
2012 Best Habit #1: Prevent Mistakes, don’t simply Fix Misteaks
Fixing bugs is FUN!
It’s FUN to spend money
It’s FUN to work weekends
Let’s have less fun …
Prevent Mistakes
700% more cost effective
to Prevent vs. Fix
How to prevent bugs?
Detailed Rqmts
Solid
reviews/checklists
31
Write tests before
code
For More information

Vance Hilderman, [email protected]
 Bildes Group, Ankara Turkey
(Atego HighRely’s Representative)

Mr. Nezih Usta

Mr. Ergun Kaleli
Slide 32
32