Transcript Software Estimation Methods

Software Estimation Methods
Fundamental Questions
Survey of Methods
Fidelity of Estimates
Thoughts to Ponder
Fundamental estimation questions
Fundamental Estimation Questions:
1. How much effort is required to complete an activity?
2. How much calendar time is needed to complete an
activity?
3. What is the total cost of an activity?
Costing and Pricing
 Estimates are made to discover the cost, to the developer,
of producing a software system
 There is not a simple relationship between the development
cost and the price charged to the customer
 Broader organisational, economic, political and business
considerations influence the price charged
 Be aware of the difference between an estimate and a price
Estimation Methods
1. Decomposition methods
2. Algorithmic/Parametric: COCOMO and others
3. Estimate by analogy/experience – based on what you
have done before
4. Expert: Wideband Delphi – consensus building
5. Agile methods
1. Planning Poker – Agile version of consensus building
2. XP Planning Game – relative estimation and guidance
6. Parkinson’s Law - estimate to your resources
7. Pricing to Win - estimate what will win
Decomposition-based Estimating
Base estimate on solution “parts” and size
•
•
•
•
Decompose the problem during analysis & design
Estimate each part
Sum the result (and add an integration fudge factor)
Perhaps the most straightforward/commonsense
• But you still need quality estimates for each part
Examples:
• LOC or FP sizing (remember metrics in 316?)
• Change sizing – look at existing project and the delta
that needs to be applied for reuse
• RUP’s Use Case Point Analysis – sizes based on
problem space decomposition, not solution space
• Many branch off into algorithmic/analogy methods
Example: WBS and Estimation
• “Top-down” Decomposition-based Estimation:
 Work breakdown structure allows estimation of
costs/time required, by “rolling up” such quantities:
 (children resources) = parent resource
 A WBS that is too coarse makes it difficult to assign
tasks and utilize resources.
• Stellman and Greene suggest “10 to 20” tasks as a rule of
thumb
Algorithmic/Parametric Models
“COCOMO style”, Conversion tables (McConnell)
 These rely on a large historical base of project data
Cost is estimated as a mathematical function of
product, project and process attributes whose
values are estimated by project managers
 Effort = A  SizeB  M
 A is an organisation-dependent constant, B reflects the
disproportionate effort for large projects and M is a
multiplier reflecting product, process & people attributes
 Most commonly used product attribute for estimation is
code size (Lines of Code [LOC] or Function Points [FP])
 Most models similar but use different values for A, B & M
Algorithmic Case Study: COCOMO
The COnstructive COst MOdel
 Well-established model by Barry Boehm (1981)
 “Basic COCOMO” used formula from previous slide
• Simple (“organic”) projects: 2.4 (KDSI)1.05 x M
• Moderate (“semi-detached”): 3.0 (KDSI)1.12 x M
• Embedded (mission-critical): 3.6 (KDSI)1.20 x M
 Evolved today into COCOMO II 3-level model:
• Early prototyping - Prototyping projects with much reuse
• Early design - Estimates after requirements agreed to
• Post-architecture - After analysis, a tuned version of ED
 Formulas constructed and tuned to 161 data points
on large projects as collected by Boehm
Estimating by analogy/experience
What you are building next most likely resembles
something you have built before
• Especially if you work in a specialized domain
• Another technique relying on historical data
Example: PROBE
• By Watts Humphrey - remember the PSP guy?
• “Proxy-based Estimating”
• Based on idea that if an engineer is building a
component similar to one s/he already built, then it
will take about the same effort as it did in the past.
• Individual engineers use a database to maintain a history of
the effort they have put into their past projects.
• A formula based on linear regression is used to calculate the
estimate for each task from this history.
Expert Consensus: Wideband Delphi
“Secret” estimates by experts, averaged & presented,
revised & reviewed until consensus
• Boehm’s 1981 book Software Engineering Economics had 6 steps:
1.
2.
3.
4.
5.
6.
Coordinator presents each expert with a specification and an estimation form.
Coordinator calls a group meeting in which the experts discuss estimation
issues with the coordinator and each other.
Experts fill out forms anonymously.
Coordinator prepares and distributes a summary of the estimates
Coordinator calls a group meeting, specifically focusing on having the experts
discuss points where their estimates vary widely
Experts fill out forms, again anonymously, and steps 4 to 6 are iterated for as
many rounds as appropriate.
• Advantages
 Easy, inexpensive, utilizes expertise of several people
 Does not require historical data
• Disadvantages
 Difficult to repeat
 May fail to get consensus, reach wrong one, or all may have same bias
Agile Methods: Planning Poker
•
•
•
•
Agile version of Wideband Delphi
Every developer participates, & maybe other team members
Product Owner conducts the session.
Script:
1.
2.
3.
4.
5.
Each estimator is given some cards with estimates on them
Product Owner selects a story card and reads aloud
Each estimator selects an estimate card and puts it face down
When ready, all estimators “roll-up” their cards at the same time
High and low estimators then conduct a short discussion of their
varying estimates
6. Return to step 3 and continue until consensus is reached
• Issues:
 Having the “right amount of discussion”
 Getting assumptions out in the open
Agile Methods: The Planning Game
XP Requirements
Methodology
• 3 phases
 Exploration – find out what
the system should do
 Commitment – decide what
subset of all possible
requirements to do next
 Steering – Guide
development as reality
molds the plan
The XP Approach
Exploration Phase
1. Business people write a story
2. Development estimates, or asks for a clarification or
split of the story
1. If a clarification or split required, return to step 1
2. Provide an estimate in “Ideal Engineering Time” (IET)
Exploration
Commitment
Steering
The XP Approach
Commitment Phase

Business chooses scope and release date
1. Sort by value: Put cards into 3 piles
1. Those that the system cannot function without
2. Less essential but provide significant business value
3. Nice to have
2. Sort by risk: 3 piles
1. Those Development can estimate precisely
2. Those Development can estimate reasonably well
3. Those Development cannot estimate at all
3. Set velocity: Development tells Business how fast (in IET)
per calendar month it will take to implement
4. Choose scope: Business chooses the set of cards for a
release
– Set date first, then choose cards, - OR –
– Choose functionality, and then set date
XP: More on Velocity
Estimating Using Project Velocity
 Velocity is a measure of your team’s capacity, not of
the “speed” of the project.
 Developers’ contribute velocity points per iteration
• Example:
– Joe is an expert guru programmer: 5 velocity pts/iteration
– Sue is an average programmer: 3 velocity pts/iteration
– Jimmy is a junior programmer: 1 point per iteration
 User Stories are then estimated via “points”
• Includes all efforts, including “spikes”
 These measures are relative!
XP does not allow velocity to be increased
midstream, or estimates to be revised downward
Two Other Ways to Estimate
Parkinson’s Law
 The project costs whatever resources are available
 Advantages: No overspend
 Disadvantages: System is usually unfinished
Pricing to Win
 project costs whatever the customer has to spend on it!!!
 Advantages: You get the contract
 Disadvantages:
• The probability that the customer gets the system he or she wants
is small.
• Costs do not accurately reflect the work required
 Sometimes, this is the only method available!
Estimating Problems
Course Technology, 1999
Complex tasks requiring a significant amount of effort.
 Revisit/revise estimates at various stages of the project
Many people have little experience doing them.
 Garbage-in, Garbage-out
 Try to provide training and mentoring
Sponsors want a number for a bid, not a real estimate.
 PMs must negotiate with sponsors to create real cost estimates
A bias toward underestimation in developers
 Developer “fudge-factor” to avoid poor reviews
 Review and ask questions to ensure estimates are not biased
A bias toward overestimation in PMs
 Rewarded for early delivery
 One successful “rush-job” can lead to many
Fidelity of Estimates
DeMarco (1982):
“An estimate is the most optimistic prediction that has a nonzero probability of coming true…or…’what’s-the- earliestdate-by-which-you-can’t-prove-you-won’t-be-finished’”
Cone Of Uncertainty (Boehm 1995)
Moral: Your estimates get better as you get closer to the deadline
Estimating Thoughts to Ponder
• “…any disagreement is generally about what is required to
perform the task itself, not about the effort required to
complete it.” (p.35)
• “The most accurate estimates are those that rely on prior
experience.” (p. 34)
• “For the estimates to be most effective, the assumptions
must be written down” (p.35)
• “The assumption allows the team to reach a temporary
decision, knowing the final decision is still open.” (p. 36)
• “Estimates are most accurate when everyone on the project
team feels that he was actively part of the estimation
process.” (p. 37)
From Ch. 3 Applied Software Project Management, Stellmen & Greene
Programmers vs. PMs
• “It is common for nontechnical people to assume that programmers pad
their estimates.” (p. 37)
• “Software engineers are often overoptimistic by nature.” (p.50)
• “Some project managers respond to an unrealistic situation by creating
‘estimates’ that are too low but meet it.” (p. 51)
• “…if her estimates come up short, she will be penalized at her next
review…Her manager will catch on and eventually catch on and start
cutting down any estimate she provides.” (p. 50)
• “There is a basic tug-of-war going on here. Engineers prefer higher
estimates, giving them as much time and as little pressure as
possible to do their work. Managers prefer to deliver things more
quickly, in order to please stakeholders.” (p. 50)
From Ch. 3 Applied Software Project Management, Stellmen & Greene
Final Thoughts
Summary: 4 general approaches
1.
2.
3.
4.
“Predictions are always hard,
especially about the future”
-- Yogi Berra
Based on size (Decomposition, XP)
Use of expert opinion (Delphi, analogy/experience)
Based on historical data (algorithmic, analogy/experience)
Estimate to what it needs to be (Parkison’s Law, Priced To Win)
Estimating Tips:








Estimating is hard.
Estimating can be personal and political
Productivity is not proportional to the number of people working
on a task
Adding people to a late project makes it later.
The unexpected always happens. Anticipate change.
Get group buy-in via participation and transparency
Revisit and revise your estimates often
Set stakeholder expectations