Transcript Software in Practice a series of four lectures on why

Software in Practice
a series of four lectures on why software
projects fail, and what you can do about it
Martyn Thomas
Founder: Praxis High Integrity Systems Ltd
Visiting Professor of Software Engineering,
Oxford University Computing Laboratory
Lecture 4: Principles of
Software Engineering
Dependability
Theory and Practice: the 30-year gap
What we Know
Strong Software Engineering
2/34
Software Dependability:
our highest priority problem
Dependability: umbrella term for safety, reliability,
security, trustworthiness, availability …
A dependable system is one known to have all
its required properties.
We need dependable processes as well as
dependable systems …
… to build systems on time and budget, and to
be able to rely on what we deliver.
3/34
World dependability initiatives
-evidence that dependability is now high priority
 Microsoft:
trustworthiness initiative (11000 engineers);
Palladium/Longhorn/Next Generation Secure Computing
Base.
 Trusted Computing Platform Alliance
 Intel’s LaGrande technology
 IBM’s Autonomic Computing initiative
 Critical Infrastructure Protection initiatives
 Dependable Systems Evolution identified as a computer
science Grand Challenge
 DIRC
4/34
In a 50 year old discipline, there is a 30year gap between theory and practice!
“A study of program structure has revealed that
programs—even alternative programs for the
same task—can differ tremendously in their
intellectual manageability. A number of rules
have been discovered, violation of which will
either seriously impair or totally destroy the
intellectual manageability of the program.”
Dijkstra, 1972
5/34
Theory and Practice
Theory
The inherent difficulty of software design has
been well understood for more than 30 years,
and the problems and solutions have been
taught to a generation of graduates.
Practice
The insight has been largely ignored by
industry “not because it has been tried and
failed, but because it has been considered
hard, and not tried.” after G. K. Chesterton
6/34
Practice in Industry
Despite much progress in software development:
most software development staff have:
 little formal education in software engineering
 no good grounding in either the processes or the theories that
underpin their professional work.
A fascination with the latest fad or fashion
managers who have even less understanding than their junior staff
of the great complexity of the engineering tasks they are
undertaking, and of the risks inherent in departing from what is
known to work.
… most software staff are recruited for their knowledge of
a programming language or a software package, not for
any engineering knowledge.
7/34
Software 2005, Santa
Clara
 "The quality I get from you people is
abysmal.” David Watson, CTO, Kaiser Permanente
"We're at too much risk to have the level of
problems with bugs we have today."
Ed Meehan, Lockheed
 "You're not all doomed. But many of you
are.” Neal Cameron, CIO, Unilever
8/34
Managers and engineering
knowledge ...
9/34
Software Engineering
Computer based applications are increasingly
complex
We choose to put that complexity into the
software
More and more software is business critical
Quite often, software is safety critical
Constructing complex and important systems is
engineering
10/34
Engineering is ...
Developing complex systems or objects ...
using scientific principles ...
and experience of successful systems and
subsystems ..
embedded in a mature process ...
with effective quality and risk management
Many of our current concerns would be familiar
to the builders of the Great Pyramids
11/34
7 Lessons from the Past
Complexity kills
The cost of correcting an error rises steeply with
time
Change is inevitable
Most development is documentation, not
invention
Reuse is better than originality
Engineering is a process of risk management
Optimisation is the last thing you should do
12/34
Complexity kills
 Writing simple programs for simple problems is easy
 Rigorous abstraction masters complexity
 Mathematical rigour has always been essential: from LL1
grammars; strong typing; concurrency primitives; and
E/R models; to formal specifications and proof
 Structural simplicity is key: modularity, information
hiding (Simula 67!), low coupling / high cohesion,
problem-oriented structures (eg JSP/JSD), Object
Orientation
13/34
The cost of correcting an
error rises steeply with time
Possibly 10-fold with each lifecycle phase
Specification errors are 100 times more
costly than coding errors if both are found
in unit testing
A good development method provides
strong V&V at each phase - this is why
formal methods are so cost-effective
Costs increase with time, even when the
product is in service
14/34
Change is inevitable
 Every business system encapsulates a business process
- which has to change to stay competitive
 Every successful product needs new versions with new
features
 The lifetime cost of most software is 5-10 times the
development cost
 So maintenance and upgrade are the most costly steps
in the lifecycle
 We therefore need to focus on methods, tools and
system architectures that
preserve structural integrity
limit the growth in complexity
15/34
Most development is
documentation, not invention
 In all engineering, the main output is documentation
aeronautical engineers say that an aircraft is not ready to fly
until the documentation weighs more than the aircraft
 For software, the documentation must support the
maintenance/update phase
and vice-versa!
 Updating code before updating specification or design
documents is vandalism - it reduces (and ultimately
destroys) the value of the product
 Writing good code from good documentation is easy the reverse can be impossible
16/34
Reuse is better than
originality
 Engineers build systems from reliable subsystems
They use successful designs from the past
 A new design is almost always wrong
even if it isn’t, building justified confidence is very expensive
 Studying successful systems of the past is the basis for
building successful systems in the future
 Design for reuse
components that provide a single function or closely related
group of functions
clear, precise specification
secure version management
17/34
Engineering requires the
management of risk
 We cannot know everything we need to, when we start
 Unacceptable risks must be identified and eliminated early
 The residual risks have to be managed.
plan to do work to avoid or accommodate them
the cost multiplied by the probability must be planned in the budget
for the project
revised costs and probabilities are needed for every project review
 The forecast cost to complete is a three-valued function
if all risks materialise
engineering judgement
if no risks materialise
18/34
Optimisation is the last
thing you should do
Jackson’s law
Jackson’s second law (for experts only)
This doesn’t mean being wasteful of
resources
a good engineer should know how to work
economically
“an engineer is someone who can do for
twopence what any fool can do for a shilling”
19/34
The Professional software
engineer….
... is someone who
has a good grounding in computer science, mathematics,
and human factors
understands and can work with mature engineering
processes involving teamwork, risk management, and
quality assurance
understands measurement as a basis for quality
improvement
has studied successful systems of the past, and
understands their design principles
understands a the limits of their own competence
keeps up to date by reading the journals and CPD 20/34
What we know:
Specifications
Form: must mirror the structure of the
real world requirement. Principles of Program Design,
Jackson, AP 1975
Notation: must support unambiguous
statement of requirements and reasoning
about properties. Only then can you
understand the system, or the impact of
changes.
Mathematical foundations are essential 21/34
Alan Turing knew this in the 1940s.
Abstraction
 The two most important characteristics of a specification
notation are (1) that it should permit problem-oriented
abstractions to be expressed …
 … and (2) that it should have rigorous semantics so that
specifications can be analysed for anomalies and to
explore system properties.
“In this connection it might be worthwhile to
point out that the purpose of abstracting is not to
be vague, but to create a new semantic level in
which one can be absolutely precise”. Dijkstra 1972
22/34
What we know:
Engineering
Engineering is the application of science, within
mature processes, to build practical systems,
cost-effectively.
For software engineers, this means applying
computer science, within mature processes for
planning, risk and quality management, formal
reviews, configuration management, and the
rest of ISO 9001/CMM level 3+ …
 All engineers use mathematics to model and
analyse their systems.
23/34
What we know: Testing
 “Testing shows the presence, not the absence,
of bugs.” Dijkstra 1972

“One can construct convincing proofs quite readily
of the ultimate futility of exhaustive testing of a program
and even of testing by sampling. So how can one
proceed? The role of testing, in theory, is to establish
the base propositions of an inductive proof. You should
convince yourself, or other people, as firmly as possible,
that if the program works a certain number of times on
specified data, then it will always work on any data. This
can be done by an inductive approach to the proof.”
Hoare 1969
24/34
What we know: Change can
destroy dependability
If you cannot reason about the impact
of a change, the change must invalidate
all pre-existing evidence of
dependability.
25/34
Software Death
Software dies when a typical fix
introduces more errors than it corrects
If your average error rate is 1 error/50
LoC, your software dies when the size of
your average bug fix exceeds 50 LoC.
Debugging maximises the number of bugs
remaining, for a given reliability.
IBM data showed that typical bugs only
recurred every 100+ user-years! Chapman 26/34
The State of the Art in
Software
Annual cost to US economy of poor quality
software: $60B source: US NIST Report 7007.011, May
2002.
Typical industrial / commercial software
development:
6-30 faults delivered / 1000 lines of software
1M lines: 6000-30,000 faults on delivery
source: Pfleeger& Hatton, IEEE Computer, pp33-42, February
1997.
27/34
What we know: Languages
 “I will start with a strong statement of opinion. I think
that any significant advance in the programming art is
sure to involve very extensive automated analyses of
programs. … … Doing thorough analyses of programs is
a big job. … It requires a programming language which
is susceptible to analysis. I think other programming
languages will head either to the junk pile or to the
repair shop for overhaul, or they will not be effective
tools for the production of large programs.” E S Lowry
(IBM) Nato Conference, Rome, October 1969.
 Such languages and tools exist:
http://www.sparkada.com
28/34
Experience with Strong
Software Engineering
 Software development based on formal specification,
strong static analysis, ISO 9001 mature processes
0.1 - 1 faults / 1000 lines of software
1M lines: 100 - 1000 faults on delivery instead of 6000 30000
100-fold improvement at no extra development cost!
sources:
Pfleeger& Hatton, IEEE Computer, pp33-42, February 1997;
Amey, http://www.sparkada.com/downloads/Mar2002Amey.pdf
29/34
Formal methods can reduce
risk, save money, and deliver
successful projects
“Z is more efficient at finding faults than
the most efficient test phase”
http://www.sparkada.com/downloads/sholis.pdf
“compelling evidence that development
methods that focus on bug prevention
rather than bug detection can both raise
quality and save time and money.”
http://www.sparkada.com/downloads/Mar2002Amey.pdf
30/34
But: get the basics right first
First … ISO 9001 and/or CMM Level 3+
now you are under control
Then…Formal methods for requirements
now you really understand what you are
trying to do
Then ...Implementation using well-defined
languages with formal annotations
for example, SPARK
31/34
Two reasons for using
Formal Methods
Formal methods reduce development
risks and save development budget
FMs are cheaper
Formal methods lead to far fewer
delivered bugs
delivered software is much higher quality
the extra quality is free
support costs are much reduced
you can even guarantee the software
32/34
Summary
We can build complex, dependable systems - but
only if we always use strong software
engineering.
System and software engineering must be
rigorous, disciplined, conservative and
evolutionary, learning from what has worked
dependably in the past. Formal methods are
fundamental to strong software engineering.
It will take numerate, motivated graduates
to change things for the better: you.
33/34
Questions?
34/34