Transcript Document

Read’n
Writ’n
‘Rithmetic
… & Cod’n
Grady Booch
IBM Fellow
© 2007 Grady Booch
NUMB3RS @ http://www.cbs.com/primetime/numb3rs/
2
© 2007 Grady Booch
QuickTime™ and a
H.264 decompressor
are needed to see this picture.
“One Hour,” NUMB3RS @ http://www.cbs.com/primetime/numb3rs/
3
© 2007 Grady Booch
Software in the Mainstream
 An entire generation has grown up believing that
the Internet has always existed
 Software-intensive systems pervade the interstitial
spaces of society
4
© 2007 Grady Booch
Our civilization runs on software.
Bjarne Stroustrup
5
© 2007 Grady Booch
You have riches and freedom here but I feel
no sense of faith or direction. You have so
many computers, why don’t you use them in
the search for love?
Lech Walesa
Epstein, J., Yale Book of Quotations, New Haven, Connecticut: Yale University Press, 2006.
6
© 2007 Grady Booch
The Promise
Software-intensive
Software-intensivesystems
systemscan
canamplify
amplifyhuman
human
intelligence
– They cannot replace human judgment
Software-intensive
Software-intensivesystems
systemscan
canfuse,
fuse,coordinate,
coordinate,
classify, and analyze information
– They cannot create knowledge
•
7
© 2007 Grady Booch
The Limits
Not
 Noteverything
everythingwe
wewant
wanttotobuild
buildcan
canbebebuilt
built
– There exist pragmatic theoretical and technical limits
Not
 Noteverything
everythingwe
wewant
wanttotobuild
buildshould
shouldbebebuilt
built
– There exist moral, economic, social, and political limits
•
8
© 2007 Grady Booch
The Beauty
Software-intensive
Software-intensivesystems
systemsare
areperhaps
perhapsthe
themost
most
intellectuallycomplex
complexartifacts
artifacts created
createdby
byhumans
humans
intellectually
– Software is invisible to most of the world
– The best software is simple, elegant, and full of drama as
manifest in the cunning patterns that form its structure
and command its behavior
•
9
© 2007 Grady Booch
1994 Chaos Report
 16.2% of software projects are successful
 31.1% of software projects are outright failures
 52.7% of software projects are challenged
http://www.standishgroup.com
10
© 2007 Grady Booch
2006 Chaos Report
 35% of software projects are successful
 19% of software projects are outright failures
 46% of software projects are challenged
http://www.standishgroup.com
11
© 2007 Grady Booch
Reasons for Improvement
 Better project management
 Iterative development
 Emerging Web infrastructure
http://www.standishgroup.com
12
© 2007 Grady Booch
Forces in Software
13
© 2007 Grady Booch
Software Engineering Body of Knowledge
 Project of IEEE, released in 2004
 Knowledge areas
– Requirements
- Configuration management
– Design
- Management
– Construction
- Process
– Testing
- Tools and methods
– Maintenance
- Quality
http://www.swebok.org
14
© 2007 Grady Booch
Computing Curricula
 Project of ACM, first released in 1968
 Major subjects
– Information structures and processes
– Information processing systems
– Methodologies
– Mathematical sciences
– Physical and engineering sciences
Curriculum 68, Communications of the ACM, Vol. 11 (3), March 1968.
15
© 2007 Grady Booch
Computing Curricula (Computing Topics)

Data structures

Theoretical numerical analysis

Programming languages

Methods of applied mathematics

Models of computation

Optimization theory

Computer design and organization

Combinatorial mathematics

Translators and interpreters

Mathematical logic

Computer and operating systems

Number theory

Special purpose systems

Probability and statistics

Numerical mathematics

Operations analysis

Data processing and file management

General physics

Text processing

Basic electronics

Computer graphics

Circuit analysis and design

Simulation

Thermodynamics and statistical mechanics

Information retrieval

Field theory

Artificial intelligence

Digital and pulse circuits

Process control

Coding and information theory

Instructional systems

Communication and control theory

Elementary analysis

Quantum mechanics

Linear algebra

Differential equations

Algebraic structures
Curriculum 68, Communications of the ACM, Vol. 11 (3), March 1968.
16
© 2007 Grady Booch
Historical Perspectives
 Computing as mathematics
 Computing as engineering and design
 Computing as art
 Computing as science
 Computing as a social science
 Computing as interdisciplinary
Goldweber, M., et al, Historical Perspectives On The Computing Curriculum, ACM SIGCUE Outlook, Vol. 25 (4), October 1997.
17
© 2007 Grady Booch
Computing Curricula
 Project of ACM and IEEE, released in 2005
 Major disciplines
– Computer engineering
– Computer science
– Information systems
– Information technology
– Software engineering
http://www.acm.org/education/curricula.html
18
© 2007 Grady Booch
Computing Curricula (Computing Topics)
 Algorithms and complexity
 Architecture
 Discrete structures
 Graphics and visual computing
 Human-computer interaction
 Information management
 Intelligent systems
 Net-centric computing
 Operating systems
 Programming fundamentals
 Social and professional issues
 Software engineering
http://www.acm.org/education/curricula.html
19
© 2007 Grady Booch
Computing Curricula (Computing Topics)

Programming fundamentals

Analysis of technical requirements

Iterative programming

Engineering foundations for software

Algorithms and complexity

Engineering economics for software

Computer architecture and organization

Software modeling and analysis

Operating system principles and design

Software design

Operating systems configuration and use

Software verification and validation

Net Centric principles and design

Software evolution

Net Centric use and configuration

Software process

Platform technologies

Software quality

Theory of programming languages

Computer systems engineering

Human-computer interaction

Digital logic

Graphics and visualization

Embedded systems

Intelligent systems

Distributed systems

Information management theory

Security issues and principles

Information management practice

Security implementation and management

Scientific methods

Systems administration

Legal/professional/ethics/society

Management of information systems organization

Information systems development

Systems integration

Analysis of business requirements

Digital media development

E-business

Technical support
http://www.acm.org/education/curricula.html
20
© 2007 Grady Booch
K-12 Model Curriculum
 Project of ACM, released in 2003
 Sructure
– Foundations of computer science (K-8)
– Computer science in the modern world (9-10)
– Computer science as analysis and design (10-11)
– Topics in computer science (11-12)
http://csta.acm.org/Curriculum/sub/ACMK12CSModel.html
21
© 2007 Grady Booch
Missing Curriculum
 Estimation, procrastination, and motivation
 Programming and testing
 Tools
 Documentation
 Users and User Interface
 Delegation and interpersonal skills
 Meetings
 Health and appearance
 Finance and future
http:engtech.wordpress.com/2007/03/01/missing_programmer_curriculum/
22
© 2007 Grady Booch
Estimation, Procrastination, and Motivation
 Personal estimation: why it is going to take more
than two weeks to hit your deadline
 Why cramming for deadline is effective at school
but not in the workplace
 The Internet: do you really need it for your job?
 The passion of the code: loving what you do
 Metrics: what isn’t measured, isn’t done
 Focus on the important, not the immediate
http:engtech.wordpress.com/2007/03/01/missing_programmer_curriculum/
23
© 2007 Grady Booch
Programming and Testing
 Why we don’t’ teach scripting languages even though you’ll
use them more often than Fortran, Pascal, C, or Java
 All programming is maintenance programming
 Why re-writing from scratch is always the worst choice
 Verification: if it wasn’t tested it doesn’t work
 Automation vs doing it by hand: what is more efficient?
 Hardening: the only bug-free code is old code
 The turtle and the rabbit: fast and lazy vs slow and diligent
http:engtech.wordpress.com/2007/03/01/missing_programmer_curriculum/
24
© 2007 Grady Booch
Tools
 Don’t blame the tools (even though the tools suck)
 Don’t reinvent the wheel: leveraging open source
 How version control will save your life
 Text editors: why notepad isn’t enough
 Bug tracking: more than Post-It notes
 Back up strategy: painful but necessary
http:engtech.wordpress.com/2007/03/01/missing_programmer_curriculum/
25
© 2007 Grady Booch
Documentation
 Diagrams, diagrams, diagrams: a picture is worth a
thousand words
 English: not just for managers
 Automation document generation: why documentation
should be tied to code
 Abbreviation: why abbreviating four to five helps no one
http:engtech.wordpress.com/2007/03/01/missing_programmer_curriculum/
26
© 2007 Grady Booch
Users and User Interface
 Keep it simple, stupid
 Cognitive overload
 Users are people, not problems
 Support isn’t a four letter word (it’s seven)
 Servicing interrupts: getting work done while supporting coworkers
http:engtech.wordpress.com/2007/03/01/missing_programmer_curriculum/
27
© 2007 Grady Booch
Delegation and Interpersonal Skills
 Passing the buck and shifting the blame
 Making small talk when you have nothing in common
 Why co-workers are companions, not competition
 Trust issues: don’t assume that what your co-worker gave
you works
 Bribing IT: getting your computer problems fixed in a
reasonable time
http:engtech.wordpress.com/2007/03/01/missing_programmer_curriculum/
28
© 2007 Grady Booch
Meetings
 How to avoid meetings
 Who to invite to meetings
 How to stay awake in meetings when they are unavoidable
 Shutting up: keeping meetings productive when you have
nothing useful to say
 Why design by committee doesn’t work
http:engtech.wordpress.com/2007/03/01/missing_programmer_curriculum/
29
© 2007 Grady Booch
Health and Appearance
 Staying in shape even though you’re sedentary
 Eating healthy when you have fast food every day
 Hygiene: friend of foe?
 World Wide Wassup: meeting your mate on the
Internet
 I am pretty: why do into consulting or marketing
http:engtech.wordpress.com/2007/03/01/missing_programmer_curriculum/
30
© 2007 Grady Booch
Finance and Future
 Planning for retirement
 Public companies vs start ups: pros and cons
 How much of the company do VCs own?
 Stock options dos and don’ts
 How to evaluate benefits compared to salary
http:engtech.wordpress.com/2007/03/01/missing_programmer_curriculum/
31
© 2007 Grady Booch
Handbook Of Software Architecture (Genres)

Artificial intelligence

Commercial and non-profit

Communications

Content authoring

Devices

Entertainment and sports

Financial

Games

Government

Industrial

Legal

Medical

Military

Operating systems

Platforms

Scientific

Tools

Transportation

Utilities
http://www.booch.com/architecture
32
© 2007 Grady Booch
Lessons Learned
 Essential skills
– Abstraction
– Collaborative development
 Metaskills
– Fundamentals
– The ability to learn
– Passion, beauty, joy, and awe
33
© 2007 Grady Booch
The Limits of Software
 Laws of physics
Fundamental
 Laws of software
 Challenge of algorithms
 Difficulty of distribution & concurrency
 Problems of design
 Importance of organization
 Impact of economics
 Influence of politics
 Limits of human imagination
34
Human
•
© 2007 Grady Booch
Laws of Physics
 The speed of light
– Has pragmatic implications for distributed systems
 Relativistic effects
– There is no such thing as absolute time
 Quantum effects
– There are theoretical as well as practical limits to
information density
 Thermodynamic effects
– Software is weightless/containers for software are not
– Computation dissipates heat
35
© 2007 Grady Booch
Laws of Software
 Sometimes we can’t do it
– Halting problem
– Gödel's theorem/highly non-computable problems
 Sometimes we can’t afford to do it
– Sorting problem/Towers of Hanoi/chess
– Exponential time
 Sometimes we just don’t know
– Traveling salesman/scheduling/bin packing
– NP complete
Harel, D., Computers, Ltd: What they Really Can’t Do, Oxford, England: Oxford University Press, 2003
36
© 2007 Grady Booch
Challenge of Algorithms
 Compression
 Photorealistic rendering
QuickTime™ and a
H.264 decompressor
are needed to see this picture.
 Speech recognition
 Simulation
 Knowledge representation
 Intimate/massive parallelism
•
Halo 3 @ http://www.bungie.net
37
© 2007 Grady Booch
Difficulty of Distribution/Concurrency
 A distributed system is one in which the failure of
a computer you didn’t even know existed can
render your computer unusable (Leslie Lamport)
 The average developer does not have, as a core
competency, the ability to develop secure,
concurrent, and distributed systems
38
© 2007 Grady Booch
Problems of Design
 The entire history of software engineering is that
of the rise in levels of abstraction
 The limitations of human understandability
 Building for resilience
 Sometimes worse is better
 The discovery of patterns
39
© 2007 Grady Booch
Importance of Organization
 All meaningful development is formed by the
resonance of activities that beat at different
rhythms
– The activities of the individual developer
– Social dynamics among small sets of developers
– Dynamics among teams of teams
 Work products and work flows
 Parallel development
 The cacophony of stakeholders
 The tension of high and low ceremony processes
40
© 2007 Grady Booch
Impact of Economics
Performance = (Complexity) (Process) * (Team) * (Tools)
Performance =
Complexity
Process
Team
Tools
41
=
=
=
=
Effort or time
Volume of human-generated code
Methods, notations, maturity
Skill set, experience, motivation
Software process automation
© 2007 Grady Booch
Influence of Politics
 Success as defined by the software development
team is sometimes misaligned with success as
defined by the management team
 Software as a strategic weapon
 Software as a pawn
42
© 2007 Grady Booch
Limits of Human Imagination
 The visionaries
– Alan Kay, Danny Hillis, Marvin Minsky, Rodney Brooks,
Bill Joy, Bill Gates, Adele Goldburg, Tim Berners-Lee,
Nicholas Negroponte…
 The dreamers
– Arthur C. Clarke, Neal Stephenson
43
© 2007 Grady Booch
Software development
has been, is, and will remain
fundamentally hard
44
© 2007 Grady Booch
What We Know
 Fundamentals
– Craft crisp and resilient abstractions
– Maintain a good separation of concerns
– Create a balance distribution of responsibilities
 Process
– Grow a system’s architecture through the incremental
and iterative release of testable executables
45
© 2007 Grady Booch
What We Know
 Reuse
 Patterns
 Languages
 Architectural codification of certain domains
46
© 2007 Grady Booch
Representing Software Architecture
Logical View
End-user
Functionality
Process View
Implementation View
Use Case View
Programmers
Configuration management
Deployment View
System integrators
Performance
Scalability
Throughput
Conceptual
System engineering
System topology
Communication
Provisioning
Physical
Kruchen, P. The 4+1 Model View, IEEE Computer, vol. 12 (6), November 1995
47
© 2007 Grady Booch
Gallery of Software Architecture: Air Traffic Control
Gallery of Software Architecture @ http://www.booch.com/architecture/architecture.jsp?part=Gallery
48
© 2007 Grady Booch
Gallery of Software Architecture: C3I
Gallery of Software Architecture @ http://www.booch.com/architecture/architecture.jsp?part=Gallery
49
© 2007 Grady Booch
Gallery of Software Architecture: Games
Gallery of Software Architecture @ http://www.booch.com/architecture/architecture.jsp?part=Gallery
50
© 2007 Grady Booch
Gallery of Architecture: Games
Gallery of Software Architecture @ http://www.booch.com/architecture/architecture.jsp?part=Gallery
51
© 2007 Grady Booch
Gallery of Software Architecture: Google
Gallery of Software Architecture @ http://www.booch.com/architecture/architecture.jsp?part=Gallery
52
© 2007 Grady Booch
Gallery of Software Architecture: Pathfinder
Gallery of Software Architecture @ http://www.booch.com/architecture/architecture.jsp?part=Gallery
53
© 2007 Grady Booch
Gallery of Software Architecture: Speech Recognition
Gallery of Software Architecture @ http://www.booch.com/architecture/architecture.jsp?part=Gallery
54
© 2007 Grady Booch
Movements in Web-centric Architectures
 Simple documents
 Colorful clients
 Simple scripting
 Rise of middleware
 Rise of simple frameworks
 Emergence of dynamic frameworks
 Semantic web
55
© 2007 Grady Booch
Handbook of Software Architecture
 No architectural reference exists for software-intensive
systems
 Goals of the handbook
– Codify the architecture of a large collection of interesting softwareintensive systems
– Study these architectural patterns in the context of the engineering
forces that shaped them
– Satisfy my curiosity
Handbook of Software Architecture @ http://www.booch.com/architecture
56
© 2007 Grady Booch
Preservation Of Classic Software
 No comprehensive and intentional activity has yet been
undertaken to preserve the industry’s seminal software
artifacts
 There are a number of reasons to act now
– Many of the authors of such systems are still alive
– Many others may have the source code or design
documents for these systems collecting dust in their
offices or garages
– Time is our enemy
Computer History Museum @ http://www.computerhistory.org
57
© 2007 Grady Booch
How We Got Here
1910s beginning of automation
1920s beginning of expansion
1930s beginning of dependence
1940s beginning of von Neuman machines
1950s rise of the machines
1960s rise of the languages and methods
1970s death of the mainframe
1980s age of the personal computer
1990s age of the Internet and new methods
2000s retrenchment
58
© 2007 Grady Booch
The Current State
 The typical software-intensive system is
– Continuously evolving
– Connected, distributed & concurrent
– Multilingual and multiplatform
– Secure
– Autonomic
 Most systems are actually systems of systems
– Services and other messaging mechanisms dominate
59
© 2007 Grady Booch
Where We Are Going
2010s age of transparency
2020s total dependence
2030s rise of the machines
60
© 2007 Grady Booch
The Future State
 Every advance leading to the future state of the
world requires the presence of software yetunwritten as of today
61
© 2007 Grady Booch
Languages & Algorithms
 Most programmers still write algorithmic snippets
in the context of a sea of objects
 Legacy XML, Java, C++, and UML persist
 Some algorithmic breakthroughs have emerged
– Searching massive quantities of information is still a bit
of a struggle
 Domain-specific frameworks are mainstream
•
62
© 2007 Grady Booch
Platforms
 Moore’s law has died
 The typical personal computer contains multiple
processors, a petabyte of main memory, an
exabyte of external memory, and untethered
terabit connectivity
 Virtual high resolution displays dominate; 3D
windows, mice, gestures, and voice are the usual
mechanisms for interaction
 Form factors will change such that most personal
computers will be wearable or embedded; most
software is embedded in devices
63
© 2007 Grady Booch
•
Operating Systems & Middleware
 Operating systems have largely been
commoditized
 Middleware that does transaction isolation, load
balancing, resource management, and data
access still dominates
– but it too has largely been commoditized, forcing the
platform vendors to keep growing the value pile
•
64
© 2007 Grady Booch
Connection
 More than ever, the network is the computer
– Monolithic -> client/server -> Web -> grid
 Network access is a global utility
 Not everything is an enterprise system, but most
applications are connected to several
•
65
© 2007 Grady Booch
Security
 New kinds of cybercrime have arisen
– Unlimited piles of money still do not yield secure systems
– Air gaps are still not enough
 Rolling failures still plague some systems
•
66
© 2007 Grady Booch
Autonomics
 No computer has yet passed the Turing Test (but
we have come close)
 Most interesting systems exhibit signs of agency
and self-repair
QuickTime™ and a
H.264 decompressor
are needed to see this picture.
•
Star Trek@ http://www.startrek.com
67
© 2007 Grady Booch
Developer Experience
 Most developers have grown up believing that the
Internet has always existed
 Most programming occurs on the edge of a
system and in the interstitial spaces among
systems
 Most programming is now done by domainspecific developers who only incidentally know
how to program
 There have been only incremental improvements
in programmer productivity and the programming
model
– CLI-> IDE -> XDE -> CDE
68
•
© 2007 Grady Booch
The Contradiction
systems
arethat
perhaps
the most
Software-intensive
The Internet changes
the way
individuals
intellectually complex
artifactscollaborate
created by humans
communicate
and businesses
– The Internet provides an environment where terrorists
and other criminal enterprises can operate with little fear
of detection
•
Internet Mapping Project @ http://www.cheswick.com/ches/map/index.html
69
© 2007 Grady Booch
The Contradiction
are perhaps
the mostfor
Software-intensive
The Web provides systems
unprecedented
mechanisms
intellectually
complex artifacts created by humans
social
networking
– The Web creates a greater opportunity for the
exploitation of children as well as fraud and theft directed
against individuals and organizations
•
Britney Spears @ http://www.myspace.com/britneyspears
70
© 2007 Grady Booch
The Contradiction
perhaps
the most
Software-intensive
Software-intensivesystems
systemsare
permit
real time
and
intellectually
complex
artifacts created by humans
distributed
access
to information
– Software-intensive systems can erode personal privacy
and other basic human rights
•
WeatherLink @ http://www.davisnet.com
71
© 2007 Grady Booch
The Contradiction
systems are perhaps
the most
Software-intensive
Email and other software-intensive
mechanisms
intellectually
complex
created by humans
increase
the velocity
ofartifacts
communication
– Email and the aging of digital archives threatens the
preservation of history
•
Project Gutenberg @ http://www.gutenberg.org
72
© 2007 Grady Booch
The Contradiction
perhaps
the most
Software-intensive
Software-intensivesystems
systemsare
create
new forms
of
intellectually
complex artifacts created by humans
artistic
expression
– Piracy disrupts the economic underpinnings of traditional
media companies and can dilute the intellectual property
of artists
•
Halion @ http://www.steinberg.net
73
© 2007 Grady Booch
The Contradiction
perhaps
most
Software-intensive
Software-intensivesystems
systemsare
enable
and the
accelerate
intellectually
complex artifacts created by humans
scientific
research
– Software-intensive systems are at the center of a new
generation of offensive and defensive weapons
•
Entrez Genome Project @ http://www.ncbi.nlm.nih.gov/entrez
74
© 2007 Grady Booch
The Contradiction
systems
are
perhaps
the most
Software-intensive
Software is a part of
the very
fabric
of civilization,
intellectually
complexspaces
artifacts created by humans
living
in its interstitial
– The complexity of software-intensive systems continues
to grow; this complexity impacts its users as well as the
stakeholders who develop, deploy, operate, and evolve
them
Annual U.S. Patent Applications/Awards
New or modified source lines of code per year per developer & cumulative
800,000,000,000
350
300
700,000,000,000
600,000,000,000
250
500,000,000,000
400,000,000,000
Applications
Awards
New or modified source lines of code per
year
Cumulative source lines of code
200
300,000,000,000
200,000,000,000
100,000,000,000
19
4
5
19
4
8
19
5
1
19
5
4
19
5
7
19
6
0
19
6
3
19
6
6
19
6
9
19
7
2
19
7
5
19
7
8
19
8
1
19
8
4
19
8
7
19
9
0
19
9
3
19
9
6
19
9
9
20
0
2
20
0
5
0
150
100
50
0
1910 1920 1930 1940 1950 1960 1970 1980 1990 2000
•
75
© 2007 Grady Booch
It is a tremendous privilege to be a
software professional
it is also a tremendous responsibility
•
76
© 2007 Grady Booch
Read’n
Writ’n
‘Rithmetic
… & Cod’n
Grady Booch
IBM Fellow
77
© 2007 Grady Booch