Transcript Software Development in an Academic Environment: Lessons

Project Management and
Embedded Systems
Christopher Brooks
Center for Hybrid and Embedded
Software Systems (CHESS)
Executive Director
UC Berkeley EECS124
March 6 & 10, 2008
Christopher Brooks
• I’m a release engineer, training electrical
engineers in the art of software
engineering.
• I’ve worked with Professor Edward A. Lee
since 1992, first on Ptolemy Classic (C++)
and now on Ptolemy II (Java).
• I’ve taken undergrad CS classes at
Berkeley
• I’m taking classes from UC Extension in
Project Management
– Software Project Management Sequence
– 7 Classes – mostly in the Business Division
Christopher Brooks: Project Management and Embedded Systems
Mar. 6 & 10, 2008
2
Project Management
• Project management is?
• The art of managing projects to a
successful completion.
• The Project Management Triple Constraint:
Scope
Quality
Cost
Christopher Brooks: Project Management and Embedded Systems
Time
Mar. 6 & 10, 2008
3
PMI: Project Management
Institute
• “A Guide to the Project Management
Body of Knowledge “ (aka PMBOK)
• PMI Certification
– Certified Associate in Project Management
(CAPM)
– Project Management Professional (PMP)
– Program Management Professional (PgMP)
Christopher Brooks: Project Management and Embedded Systems
Mar. 6 & 10, 2008
4
How to have an unsuccessful project
•
•
•
•
Copyright © Garry Trudeau
What happened?
Team dynamics: team mate is crashed out
Scheduling: only an hour left?
Overambitious: dual drive? AI Chip?
Christopher Brooks: Project Management and Embedded Systems
Mar. 6 & 10, 2008
5
Solving Alex’s Problems
• Team dynamics: team mate is crashed out
– Scheduling Problem
• Scheduling: only an hour left?
– Scheduling Problem
• Overambitious: dual drive? AI Chip?
– Vague requirements
– Goldplating
Christopher Brooks: Project Management and Embedded Systems
Mar. 6 & 10, 2008
6
• What’s the problem?
Copyright © Garry Trudeau
– Gold plating
Christopher Brooks: Project Management and Embedded Systems
Mar. 6 & 10, 2008
7
How I’ve messed up
•
•
•
•
LED T-Shirt: 8x10 led’s
Tansy sewed the 80 surface mount LEDs
I worked on the Ptolemy Software
Problems
– Missed deadline – bad planning
– Data size too large for chip!
• Could have avoided with
– Better planning: avoid surprises
Christopher Brooks: Project Management and Embedded Systems
Mar. 6 & 10, 2008
8
Project Management/Embedded Systems
Problems
• Hardware and software designs affect
each other
• But you don’t know about problems until too
late.
• Solutions include
– Software Simulation
– Hardware Prototyping
Christopher Brooks: Project Management and Embedded Systems
Mar. 6 & 10, 2008
9
How to successfully manage a project
• Develop a one page project charter
– Get your sponsor to sign off
• Develop a time line with milestones
– Work backwards
– Describe deliverables
• Monitor progress
– If you don’t look at the plan, then why bother?
– Status reports can be email messages or
meetings
Christopher Brooks: Project Management and Embedded Systems
Mar. 6 & 10, 2008
10
EECS 124 Deliverables
• Due one week from today
• One page Project Charter
• One page Schedule
Christopher Brooks: Project Management and Embedded Systems
Mar. 6 & 10, 2008
11
Project Charter
• Formalize the project with the sponsor
• Sections:
–
–
–
–
–
–
Project Overview
Project Approach
Project Objectives
Major Deliverables
Constraints
Risks and Feasibility
• In one page
– Most people, especially busy people, will not
read more. In fact, it has been shown that
most people don’t even read (hi mom!) slides.
Christopher Brooks: Project Management and Embedded Systems
Mar. 6 & 10, 2008
12
Project Charter Example: Web Site
www-cad Project Overview
This project is to create a new web site for the CAD group faculty. The current website at wwwcad.eecs has a very old look, it needs an update so that we can attract new students.
Project Approach
The project is a fairly small website based partly on a preexisting site, so we will use a classic
waterfall approach with milestones. The project team will consist of the following people. I’ve
estimated the maximum amount of time we can get from each person over the life of the project.
Kurt Keutzer (2 hrs week for 6 weeks)
Ken Lutz
(2 hrs/week for 6 weeks)
Brad Krebs
(10 hrs/week for 6 weeks)
Christopher Brooks (10 hrs/week for 6 weeks)
Allen Hopkins (5 hrs/week for 6 weeks)
Carol Sitea
(1 hr/week for 6 weeks)
The project sponsor is Professor Keutzer. Professor Keutzer is on sabbatical this semester, but we
hope to get feedback from him on a continuing basis.
Project Objectives
One page!
•
•
•
•
•
Update the look and feel of the website to a modern standard
Provide access to student and faculty pages
Provide access to active projects
Provide access to summaries, downloads and key papers of inactive projects. The old pages
of inactive projects should be archived.
Provide a simple static listing of seminars. A more complex calendar and a search engine are
deferred due to schedule constraints.
Major Deliverables
•
•
•
•
•
A schedule along with time estimates.
A prioritized list of features.
An example of the main page so we can review look and feel.
An archive of the old website
The final website.
Constraints
Professor Keutzer would like to see the web site completed by mid-March: that is when students
start looking at graduate schools. Developers might not have much time to work on this project.
The project requires timely feedback from the faculty.
Risk and Feasibility
The primary risk is that the project takes too long to complete and we miss the mid-March opportunity. Another risk is that we complete the project too quickly and quality suffers. A third risk is
that there are only so many resources available. By fast tracking, we can handle some of the
tasks in parallel and avoid these risks. The project is definitely feasible if we roll out the website
in stages.
Christopher Brooks: Project Management and Embedded Systems
Mar. 6 & 10, 2008
13
Project Charter: Overview
• In a sentence, describe the project:
This project is to create a new web site for the CAD
group faculty.
• Possibly include the business reason:
The current website at www-cad.eecs has a very old
look, it needs an update so that we can attract new
students.
Christopher Brooks: Project Management and Embedded Systems
Mar. 6 & 10, 2008
14
Project Charter: Approach
• Describe your software development life
cycle (SDLC):
• What’s that?
– Waterfall
– Iterative aka Spiral
– Extreme Programming
• List the team
• List the time commitments
Christopher Brooks: Project Management and Embedded Systems
Mar. 6 & 10, 2008
15
Waterfall Method
• Complete one phase before going on to the
next [Royce, 1970]
Requirements
Design
Implementation
Validation
Maintenance
Time
Christopher Brooks: Project Management and Embedded Systems
Mar. 6 & 10, 2008
16
Microsoft Project Gantt Chart
Christopher Brooks: Project Management and Embedded Systems
Mar. 6 & 10, 2008
17
Spiral Model [Boehm, ’88]
•
Christopher Brooks: Project Management and Embedded Systems
Source: B.W Boehm,
“A spiral model of
software development
and enhancement,”
Mar. 6 & 10, 2008
18
Extreme Programming
• Planning
– Release planning
– Frequent Releases
• Coding
– Standards
– Unit tests first
– Pair Programming
– Late Optimization
• Designing
– Simplicity
– System Metaphor
– Refactoring
• Testing
– Unit Tests
– Bugs need Tests
– Acceptance Tests
Based on http://www.extremeprogramming.org/rules.html
Christopher Brooks: Project Management and Embedded Systems
Mar. 6 & 10, 2008
19
Project Charter: Approach (again)
• Describe your software development life
cycle (SDLC):
• What’s that?
– Waterfall
– Iterative aka Spiral
– Extreme Programming
• List the team
• List the time commitments
Christopher Brooks: Project Management and Embedded Systems
Mar. 6 & 10, 2008
20
Project Charter: Approach Example
The project is a fairly small website based partly on a
preexisting site, so we will use a classic waterfall approach
with milestones. The project team will consist of the
following people. I’ve estimated the maximum amount of
time we can get from each person over the life of the
project.
Kurt Keutzer (2 hrs week for 6 weeks)
Ken Lutz
(2 hrs/week for 6 weeks)
Brad Krebs
(10 hrs/week for 6 weeks)
Christopher Brooks (10 hrs/week for 6 weeks)
Allen Hopkins (5 hrs/week for 6 weeks)
Carol Sitea
(1 hr/week for 6 weeks)
The project sponsor is Professor Keutzer. Professor Keutzer
is on sabbatical this semester, but we hope to get feedback
from him on a continuing basis.
Christopher Brooks: Project Management and Embedded Systems
Mar. 6 & 10, 2008
21
Project Charter: Objectives
• List your objectives
• Objectives are one way to measure success
• Don’t get too specific, there is time for
that later
Christopher Brooks: Project Management and Embedded Systems
Mar. 6 & 10, 2008
22
Project Charter: Objectives Example
• Update the look and feel of the website to a
modern standard
• Provide access to student and faculty pages
• Provide access to active projects
• Provide access to summaries, downloads and key
papers of inactive projects. The old pages of
inactive projects should be archived.
• Provide a simple static listing of seminars. A more
complex calendar and a search engine are deferred
due to schedule constraints.
Christopher Brooks: Project Management and Embedded Systems
Mar. 6 & 10, 2008
23
Project Charter: Deliverables
• What’s a deliverable?
• Physical artifacts which describe your
progress and include your product
• Deliverables should include:
– A description (Scope)
– A date (Time)
– A person or persons who are responsible (Cost)
Christopher Brooks: Project Management and Embedded Systems
Mar. 6 & 10, 2008
24
Project Charter: Deliverables Example
• A schedule along with time estimates.
• A prioritized list of features.
• An example of the main page so we can review
look and feel.
• An archive of the old website
• The final website.
Christopher Brooks: Project Management and Embedded Systems
Mar. 6 & 10, 2008
25
Project Charter: Constraints
• Schedule, Budget and Resource problems
– What’s a common constraint for eecs124?
– Hardware availability
• Example:
Professor Keutzer would like to see the web site completed by
mid-March: that is when students start looking at graduate
schools. Developers might not have much time to work on this
project.
The project requires timely feedback from the faculty.
Christopher Brooks: Project Management and Embedded Systems
Mar. 6 & 10, 2008
26
Project Charter: Risks
• List things that could go wrong and how you
will avoid them
• Don’t skip the risks.
• Example:
The primary risk is that the project takes too long to complete
and we miss the mid-March opportunity. Another risk is that
we complete the project too quickly and quality suffers. A
third risk is that there are only so many resources available.
By fast tracking, we can handle some of the tasks in parallel
and avoid these risks. The project is definitely feasible if we
roll out the website in stages.
Christopher Brooks: Project Management and Embedded Systems
Mar. 6 & 10, 2008
27
Why Milestones
• How will you measure success?
• How will you divide up the work?
• How will you make sure everyone
participates?
• Milestones
Christopher Brooks: Project Management and Embedded Systems
Mar. 6 & 10, 2008
28
What is a milestone?
• A milestone is a checkpoint in the project
– A milestone has
• A description of the point
• A date
• A person or persons who are responsible
• A milestone may or may not have a
deliverable associated with it.
– A deliverable has
• A description of a deliverable (scope)
• A date (time)
• A person or persons who are responsible (cost)
Christopher Brooks: Project Management and Embedded Systems
Mar. 6 & 10, 2008
29
Work Breakdown Structure (WBS)
• A work breakdown structure is an outline
that describes the deliverables.
• Each level of the outline describes 100% of
the work below it (the 100% Rule)
Scope
• A WBS should include:
What (Scope)
When (Time)
Who (Cost)
Cost
Christopher Brooks: Project Management and Embedded Systems
Time
Mar. 6 & 10, 2008
30
Metro II Software
Development
and Release
Resouce: team
Cost: ? > 8d, ~ 31d?
Start: 1/3/07
Finish: 7/31/07
2.0 Alpha
Release
1.0 Planning
Resource: cxh
Cost: 10hr
Finish: 2/6/07
Resource: team
Cost: ?>5d, ~10d?
Finish: 3/13/07
1.1 Charter
Resource: cxh
Cost: 4hr
Finish: 1/23/07
3.1 Features
Development
Finish: 5/29/07
1.1 WBS
Resource: cxh
Cost: 3hr
Finish: 1/30/07
1.1 Schedule
Resource: cxh
Cost: 3hr
Finish: 2/6/07
2.1 Build Env
Resource: cxh
Cost: 10hr
Finish: 2/15/07
2.1.1 Directory
Structure
Resouce: cxh
Cost: 2hr
Finish: 1/30/07
2.1.2 Doc Sys
Resource: cxh
Cost: 3hr
Finish: 2/15/07
2.1.3 Testing
System
Resource: cxh
Cost: 5hr
Finish: 2/15/07
2.1 Feature
Development
Resource:
team
Cost: ?>3d
Finish: 3/13/07
2.2.1 IP
Wrapping
Resource: tcm
Cost: 8r?
Finish: 1/31/07
A
4.0 Final Release
Resource: team
Cost: ?> 1d, ~10d??
Finish: 6/4/07
Resource: team
Cost: ?>2d ~ 10d??
Finish: 7/31/07
3.2 Feature
Refinement
Finish: 6/4/07
3.1.3 Mappers
& Adapters
Resource: ??
Cost: ?
Finish: 5/29/07
3.2.1 IP
Wrapping
Resource: ?
Cost: ?
Finish: 1/31/07
C
3.2.2 Three
Phase
Execution
Semantics
Resource: ?
Cost: ?
Finish: 3/13/07
2.3 Installer
Resource: cxh
Cost: 10hr
Finish: 3/13/07
3.0 Beta Release
3.3
Integration
Testing
Resource: ?
Cost: ?
Finish: 6/4/07
3.4 Installer
Finish: 6/4/07
3.4.1 Installer Testing
Resource: cxh + team
Cost: 8 (cxh)
0.5/team member.
Finish: 6/4/07
3.4.2 Publish to Web
Resource: cxh
Cost: 1hr
Finish: 6/4/07
4.1 Integration
Testing
Resource: cxh + team
Cost: ?
Finish: 7/20/07
4.2 Docs
Resource: team
Cost: 8d
Finish: 7/20/07
4.3 Installer
Finish: 7/31/07
4.2.1 Installer Testing
Resource: cxh + team
Cost: 8hr (cxh) +
0.5hr/team member
Finish: 7/31/07
2.3.1 Installer
Testing
Resource: cxh
Cost: 8 hr
Finish: 3/13/07
4.2.2 Publish to Web
Resource: cxh
Cost: 1hr
Finish: 7/31/07
2.3.2 Publish
to Web
Resource: cxh
Cost: 2hr
Finish: 3/13/07
2.2.2 Three
Phase
Execution
Semantics
Resource: ?
Cost: ??
Finish: 3/13/07
B
Metro II Work Breakdown Structure
3/6/2008
Page 1 of
2
Christopher Brooks: Project Management and Embedded
Systems
Christopher
Brooks
Mar. 6 & 10, 2008
31
WBS for the LED T-Shirt
• LED T-Shirt (Tansy and Christopher) due on 5/10
– Hardware (Tansy)
•
•
•
•
•
•
Obtain Electronics (Christopher) (3/15)
Obtain Fabric Materials (Tansy) (3/15)
Design Schematic (Christopher) (3/10)
Solder LEDs (Christopher) (3/20)
Sew Schematic (Tansy) (4/15)
Test Circuit (Christopher) (4/20)
– Software (Christopher)
•
•
•
•
Build Code Generation Environment (Christopher) (3/15)
Develop algorithm in simulation (Christopher) (3/15)
Breadboard Circuit (Christopher) (3/20)
Provide Software requirements for Schematic
(Christopher) (3/10)
• Download Software onto Chip (Christopher) (4/20)
– Integration (Tansy and Christopher) (4/25)
– Testing (Tansy and Christopher) (5/1)
Christopher Brooks: Project Management and Embedded Systems
Mar. 6 & 10, 2008
32
The Critical Path
• What’s the Critical Path?
• The longest path through the project,
which determines the shortest time to
completion.
• Solutions during Planning:
– Do things in parallel
– Add more resources
– Prune deliverables
• Solutions during Operation:
– Replan
– Panic
Christopher Brooks: Project Management and Embedded Systems
Mar. 6 & 10, 2008
33
11 Steps to successfully
completing a software project
1. Create a one page charter
2. Separation of concerns: MVC: gui vs backend
3. Start writing tests early, use a code coverage tool
4. Use a nightly build
5. Use a consistent coding style and use a tool to enforce the style
6. Use tools: memory leaks, warnings, spelling errors, performance
problems, other compilers, other operating systems.
7. Document your code. Writing documentation first can prevent
hours of wasted time.
8. Don’t debug for more than an hour by yourself – get help.
9. Design Review and Code Review (or at least desk check)
10. Expect the unexpected: wacky user input, wacky user
interaction, problems with threads.
11. Don’t be afraid to throw away code and start over.
Christopher Brooks: Project Management and Embedded Systems
Mar. 6 & 10, 2008
34
Conclusions
• Your Deliverables (in <7 calendar days)
– One page project charter (example on website)
– 1-2 page schedule:
• Milestones (what, when, who)
• Work backwards: what can you realistically accomplish?
Deliver the good stuff first.
• Avoid common mistakes
–
–
–
–
–
–
–
Plan
Allow for testing and for mistakes
Order parts now: risk for project charter?
Integrate early: simulate, prototype
Partition the work: work in teams
Keep track of your status: update the plan
Give credit to your sources
Christopher Brooks: Project Management and Embedded Systems
Mar. 6 & 10, 2008
35
Questions?
Christopher Brooks: Project Management and Embedded Systems
Mar. 6 & 10, 2008
36
Ptolemy Software Practice
•
•
•
•
•
•
•
•
•
Study Groups
Common Off the Shelf Tools
Version Control
Coding Style
Nightly Builds
Testing
Code Cleaning
Design and code reviews, Code Rating
Regular releases with excellent docs
Christopher Brooks: Project Management and Embedded Systems
Mar. 6 & 10, 2008
37
Tools
• Emacs, make – older tools, available
everywhere – know how to use make
• autoconf – older configuration system.
- know how to run configure
• Cygwin under Windows
• Ant – a build system,
– but not a configuration system.
• Eclipse – Common for Java
• JUnit – Well integrated with Ant and
Eclipse
• Microsoft Visual Studio
– better than it was
Christopher Brooks: Project Management and Embedded Systems
Mar. 6 & 10, 2008
38
Automatic Testing Tools
• Static code checkers
– gcc –Wall
– Build and test under different chip
architectures (x86, Sparc) or OS’s
– Coverity – not Free
– Fortify – not Free
– Java tools like FindBugs,PMD
• Memory checkers
– Electric Fence (for C)
– Purify – not Free
– Valgrind – Linux only
Christopher Brooks: Project Management and Embedded Systems
Mar. 6 & 10, 2008
39
Version Control
• Used to ship tar files around
• Provide a way to back out changes
and look at history
• SCCS and RCS
•
•
•
•
– older and non-networked
CVS is commonly used
Clearcase costs $$
SourceSafe is MS Only
Subversion is the tool of choice
Christopher Brooks: Project Management and Embedded Systems
Mar. 6 & 10, 2008
40
CVS
• Concurrent development by multiple users on
the same file
• Read only and read/write access over the
network.
• CVS tree is backed up and replicated.
• Platform independence
• Freely available and fairly common
• Graphical and web interfaces available
Christopher Brooks: Project Management and Embedded Systems
Mar. 6 & 10, 2008
41
Coding Style
• Common style helps when
looking at the code of others.
• It does not matter what the style is, as
long as it is consistent and documented.
• Everyone but Chief Scientists and
Professors will need to follow a coding
standard – so get used to it.
• The style must be mandatory
• Automated tools are a big help
• The style can slowly evolve.
Christopher Brooks: Project Management and Embedded Systems
Mar. 6 & 10, 2008
42
Coding Style Features
• Document using complete sentences with
good grammar
– Be nice to yourself and others that use your
code.
• Identifiers use complete words
(CamelCase)
– This aids in readability and accessibility – the
developer knows that the variable or method is
numberOfEspressos, not numEsp or n.
Christopher Brooks: Project Management and Embedded Systems
Mar. 6 & 10, 2008
43
Code Cleaning
• If source code is shipped, it should be
treated like any other publication
• Spell check your source code – adding a
custom dictionary can help
• Run an indenter – All the code should have
the same style
• Create a script to find common problems
Christopher Brooks: Project Management and Embedded Systems
Mar. 6 & 10, 2008
44
Copyrights
©
• Get buy in from the faculty or manager
• Ptolemy has a liberal copyright that
permits commercial use.
• UC Office of Technology Licensing (OTL)
copyright policy is different.
• Corporate sponsors may flee if subjected
to copyleft.
• Each file really should have the copyright.
Christopher Brooks: Project Management and Embedded Systems
Mar. 6 & 10, 2008
45
If you copy code into a project
• Make sure it is permitted
– Copyright violation
– Plagiarism for class work
– Work rules
• Be sure to credit the source
• Be aware of different Open Source
copyrights
– BSD (Commercial use ok, credit us, don’t sue us)
– GPL
Christopher Brooks: Project Management and Embedded Systems
Mar. 6 & 10, 2008
46
Build Culture
• McCarthy
- “If you build it, it will ship”
• “If you don't build it, it will never ship”
• “Don't break the build”
- prompts developers to test changes
before checking them in
• Developers see problems immediately . . .
And sometimes wear a silly hat
Christopher Brooks: Project Management and Embedded Systems
Mar. 6 & 10, 2008
47
Build Culture II
• Integrate early and often
Don't go dark
• If it is not in the build, it does not exist
But that's ok for experimental work.
• Programmer's Progress
Over time, code gets reviewed
and code coverage increases,
programmers see the results
in the nightly build.
Christopher Brooks: Project Management and Embedded Systems
Mar. 6 & 10, 2008
48
Testing
• Code that is untested might as
well not exist
– Untested code can hurt you.
• Regression testing – running tests against
a known good results
• Unit tests test a single module
• Integration tests – combine multiple
modules
• Code coverage is a big help
• Keep the tests close to the code
Christopher Brooks: Project Management and Embedded Systems
Mar. 6 & 10, 2008
49
Testing Ptolemy
• Unit Tests: Scripting (Ptolemy: ~10k)
• Integration/System tests (Ptolemy: ~1.1k)
– Run test models with known good results
• Built in sanity tests (Ptolemy: ~10)
– Open all the models
– Check all the links
• UI Testing (Ptolemy: Users )
– not automated (hard/expensive)
Christopher Brooks: Project Management and Embedded Systems
Mar. 6 & 10, 2008
50
Ptolemy Unit Testing
• Scripting languages: well suited for test
suite development because of the iterative
nature of test suites.
• We use Jacl, a 100% Java implementation
of a subset of Tcl that permits access to
Java.
• Jython might be a better choice today.
• JUnit, xUnit
– Common, but not scriptable
– If you are using Java, use JUnit.
Christopher Brooks: Project Management and Embedded Systems
Mar. 6 & 10, 2008
51
Jacl
• Instantiate a new Foo:
set a [java::new ptolemy.kernel.Foo]
• Call a method:
$a myMethod
• Simple Test:
test Foo-1.1 {Test myMethod} {
set a [java::new ptolemy.kernel.Foo]
$a myMethod
} {You invoked myMethod!}
Christopher Brooks: Project Management and Embedded Systems
Mar. 6 & 10, 2008
52
JUnit
•
•
•
•
•
Compiled Java Code
Well integrated with Ant and Eclipse
Subclass junit.framework.TestCase
If necessary, override setup()
Create test methods that begin with test:
testFoo(), testBar()
• If necessary, override tearDown()
• Groups of tests go into suite()
Christopher Brooks: Project Management and Embedded Systems
Mar. 6 & 10, 2008
53
Design and Code Reviews
• Objective is “publishable software”
• Defined roles for participants
– Author has the last word
• Mechanism for new group members to
learn to differentiate good from bad
software.
All technical reviews are based on the
idea that developers are blind to some
of the trouble spots in their work...
Steve McConnell
Christopher Brooks: Project Management and Embedded Systems
Mar. 6 & 10, 2008
54
Code rating
•
A simple framework for
–
–
•
quality improvement by
peer review
change control by
improved visibility
• What is this about
Four confidence levels
–
–
–
–
Red. No confidence at all.
Yellow. Passed design
review. Soundness of the
APIs.
Green. Passed code review.
Quality of implementation.
Blue. Passed final review.
Backwards-compatibility
assurance.
really?
– Confidence in
quality
– Commitment to
stability
Source: John Reekie
Christopher Brooks: Project Management and Embedded Systems
Mar. 6 & 10, 2008
55
How we do a review
•
Top level
–
–
–
–
•
The author announces that the package is ready for review
The moderator organizes and moderates the review
The author responds to the issues raised in the review,
redesigning or reworking as necessary
The author announces the new rating.
In the review
–
–
–
–
–
The moderator runs the meeting and keeps the discussion
on track; and acts as reader (in our process).
The reviewers raise issues and defects
The author answers questions
Roles define and
The scribe notes raised issues and defects
clarify responsibility
Nobody attempts to find solutions!
Source: John Reekie
Christopher Brooks: Project Management and Embedded Systems
Mar. 6 & 10, 2008
56
What were the review benefits?
•
Students
–
–
–
–
–
–
–
•
better design and more confidence.
good feedback about documentation and naming issues
revealed quite a few flaws
an affirmation that your architecture is sound
encourage other people in the group to reuse code
forcing function to get documentation in order
my coding style changed
Staff
–
–
exposed quite a few design flaws
caught lots of minor errors, and quite a few insidious
errors
Source: John Reekie
Christopher Brooks: Project Management and Embedded Systems
Mar. 6 & 10, 2008
57
Small Released: Nightly Build
•
Build and test the system regularly
– Every night
•
Why? Because it is easier to fix problems earlier than later
– Easier to find the cause after one change than after 1,000 changes
– Avoids new code from building on the buggy code
•
Aiken: Test is usually subset of full regression test
– “smoke test”
– Just make sure there is nothing horribly wrong
Keutzer: I disagree with this point. Typical case should be to run
entire regression test
Jim McCarthy (Director of MSVC++ Group): “If you build it, it will
ship”
Build a release every night, run tests – makes integration easier.
•
•
•
Source: Alex Aiken
Christopher Brooks: Project Management and Embedded Systems
Mar. 6 & 10, 2008
58
Ptolemy II Nightly Build
– Ptolemy II has ~11,000 tests for ~2300 Java files
contain 675,000 lines of code.
Christopher Brooks: Project Management and Embedded Systems
Mar. 6 & 10, 2008
59
Code Coverage
• The fireOneRound() method is not covered
Christopher Brooks: Project Management and Embedded Systems
Mar. 6 & 10, 2008
60
Testing Documentation: doccheck
Doccheck is a javadoc plug-in from Sun that
points out common problems.
Christopher Brooks: Project Management and Embedded Systems
Mar. 6 & 10, 2008
61
Orca and testing
• Orca “an open-source framework for developing
component-based robotic systems “
(http://orca-robotics.sourceforge.net/)
• Orca uses CMake (http://www.cmake.org)
to configure the system
• Orca uses the Dart2 Dashboard to show nightly
build output
(http://129.78.210.237:8081/orca2/Dashboard/)
• Orca uses CTest, part of CMake for testing.
http://wiki2.cas.edu.au/orca/index.php/Orca:faq:general:tes
ting:write:tests
• CMake is useful for complex, multiplatform
systems
Christopher Brooks: Project Management and Embedded Systems
Mar. 6 & 10, 2008
62
Orca Dashboard http://129.78.210.237:8081/orca2/Dashboard/
Christopher Brooks: Project Management and Embedded Systems
Mar. 6 & 10, 2008
63
Orca Dashboard Coverage
Christopher Brooks: Project Management and Embedded Systems
Mar. 6 & 10, 2008
64
Orca Dashboard Coverage Detail
Christopher Brooks: Project Management and Embedded Systems
Mar. 6 & 10, 2008
65
Orca Dashboard Coverage Detail
Christopher Brooks: Project Management and Embedded Systems
Mar. 6 & 10, 2008
66
How is Ptolemy II released
•
Decide a rough ship date
•
Decide on features
•
Look over
•
•
•
•
•
•
•
•
Decide on a schedule, review it with the group
Nag
Let the date slip
“Showstopper” (sic) bugs appear
New features are inserted despite everyone knowing better
Stay up late, work hard
Ship
Drink beer
– Conferences, Contracts, Holidays
– Who is graduating?
– What’s ready or not ready?
– Features are worked on and some are reviewed
– Bug system reports
– Code Coverage
– Code Ratings from Reviews
Christopher Brooks: Project Management and Embedded Systems
Mar. 6 & 10, 2008
67
Developer Problems
• A Ph.D. student refused to check their source into
the tree
• “It’s not ready”
• Q: What sort of design methodology is this?
• A: Waterfall: no feedback from customer
• When the code went in, a serious fundamental
design flaw was found.
• The flaw could have been easily fixed earlier
• Results: Lower impact Thesis.
• Correction: Change the culture
Christopher Brooks: Project Management and Embedded Systems
Mar. 6 & 10, 2008
68
Release Problems
• Last minute changes causes most of the
bugs and most of the incremental releases
• Release does not include all files
– Ship only what is needed is overapplied
– Testing did not catch it
• Platform issues
– We don’t have that platform in house.
• Releases are late
– No specification
– Research software
Christopher Brooks: Project Management and Embedded Systems
Mar. 6 & 10, 2008
69
Lessons not learned:
Review Process Problems
• What sort of problems can happen with the
review process?
• Review process decayed
• No read-ahead
– This is a walkthrough, not a review
• No follow up
Christopher Brooks: Project Management and Embedded Systems
Mar. 6 & 10, 2008
70
How Should Ptolemy II Be Released?
• What software methodology should be
used?
– Waterfall – no – this is research
– Spiral – would be nice
– Modified XP works ok
• What changes could we make?
– Requirements docs
– Walkthroughs are separate
– More thourough design reviews
Christopher Brooks: Project Management and Embedded Systems
Mar. 6 & 10, 2008
71