Transcript Value Based Software Engineering
Value Based Software Engineering
What, Why, How & The Mysterious
Nupul Kukreja SoCal SPIN May 6 ‘11
About Me
• • • • Pursuing PhD in Computer Science under Dr. Barry Boehm. Focus Area: Value Based Software Engineering (VBSE) MS in CS from USC and BE in CS from Mumbai University, India Worked as a Software Developer at Capgemini Consulting India Pvt. Ltd.
Lecturer for CS Undergraduates at Watumull Institute (Mumbai University) teaching Computer Graphics and Operating Systems 10/6/2010 © USC-CSSE 2
• •
Agenda
Section 1* : Overview – Understanding “Theory W” – VBSE’s 4+1 Theory – – VBSE Agenda VBSE: 7 Key Elements (Overview) Section 2: Applying 7 Key Elements in practice – Identifying company’s strategic goals and their overall value – Benefits Realization Analysis – Measurements/Estimation and Value of Information – – Creating a Value Based Business Case Elicitation/Reconciliation of Stakeholder Win Conditions – Prioritization of Win Conditions/Requirements using Kano Analysis – Tracking Value Earned
*Content courtesy: Dr. Barry Boehm
10/6/2010 © USC-CSSE 3
Tools & Techniques Involved
• • Application of VBSE in practice involves understanding practices from various domains: – Decision Theory – Statistics – Management/People Engineering – Software Engineering Principles We shall see how to effectively combine them to help practice VBSE with minimal overhead 10/6/2010 © USC-CSSE 4
• Theory W*: Enterprise Success Theorem – An informal proof Theorem: “Your enterprise will succeed if and only if it makes winners of your success-critical stakeholders” • • Proof of “if”: Everyone that counts is a winner.
Nobody significant is left to complain.
Proof of “only if”: Nobody wants to lose.
Prospective losers will refuse to participate, or will counterattack.
The usual result is lose-lose.
*An Initial Theory of VBSE – Barry Boehm & Apurva Jain 10/6/2010 © USC-CSSE 5
Theory W: WinWin Achievement Theorem
Making winners of your success-critical • • • • stakeholders (SCSs) requires: Identifying all of the SCSs Understanding how the SCSs want to win Having the SCSs negotiate a win-win set of product and process plans Controlling progress toward SCS win-win realization, including adaptation to change.
10/6/2010 © USC-CSSE 6
Initial VBSE Theory: 4+1
• • Engine: Theory W (stakeholder win-win): What values are important?
– Enterprise Success Theorem – Theory of Justice – Win-Win Equilibrium and Negotiation Four Supporting Theories – Dependency Theory: How do dependencies affect value realization?
• Results chains; value chains; cost/schedule/performance tradeoffs – Utility Theory: How important are the values?
• Multi-attribute utility; Maslow’s need hierarchy – Decision Theory: How do values determine decisions?
• Investment theory; game theory; statistical decision theory – Control Theory: How to monitor and control value realization?
• Feedback control; adaptive control; spiral risk control 10/6/2010 7
VBSE Theory 4+1 Structure
Dependency Theory How do dependencies affect value realization?
How to adapt to change and control value realization?
Control Theory Utility Theory What values are important?
How is success assured?
Theory W: SCS Win-Win How important are the values?
How do values determine decision choices?
Decision Theory
10/6/2010 8
Initial VBSE Theory: 4+1 Process – With a great deal of concurrency and backtracking
10/6/2010 9
VBSE Agenda
• • Objective: Integrating value considerations into the full range of existing & emerging Software Engineering principles in a manner so that they ‘compatibly’ reinforce one another Major Elements: – VB* Requirements Engineering – – – – – – – VB Architecting VB Design and Development VB Verification And Validation VB Planning and Control VB Risk Management VB Quality Management VB People Management *VB = Value-Based 10/6/2010 10 © USC-CSSE
VBSE: Seven Key Elements
• • • • • • • Benefits Realization Analysis Stakeholder Value Proposition Elicitation & Reconciliation Business Case Analysis Continuous Risk and Opportunity Management Concurrent System & Software Engineering Value-Based Monitoring & Control Change as Opportunity 10/6/2010 11 © USC-CSSE
1. Benefits Realization Analysis
DMR/BRA* Results Chain Assumption(s): -Order to delivery time is an important buying criterion
Accountable Stakeholder(s)
INITIATIVE Contribution OUTCOME Contribution OUTCOME Implement a new order entry system Reduced order processing cycle (intermediate outcome) Reduce time to process order Increased sales Reduce time to deliver product * DMR Consulting Group’s Benefits Realization Approach
12 © USC-CSSE
Key ‘Benefits’ of Doing BRA/Results Chain
• • Explicitly maps out the intended benefits of the system to be acquired (or any initiative to be undertaken) Identifies what all initiatives need to be taken to actually realize the benefits (A.K.A. Work Breakdown Structure (WBS) at times i.e., “within” the initiatives) • Identifies the ‘key stakeholders’ accountable for the above initiatives • • • NOTE: Must be refined as more is learnt about the initiatives or an unforeseen benefit is uncovered Done at multiple levels of granularity and stabilizes earlier into the SDLC than most artifacts Subtle but valuable: Lays foundation for the relevant metrics required to ‘track’ the benefits 10/6/2010 13 © USC-CSSE
• • 2. Stakeholder Value Proposition
Elicitation & Reconciliation
Myth: ALL SCSs have readily expressible and compatible value propositions that can be easily turned into a set of objects for each initiative and the overall ‘portfolio’ of initiatives
The following figure is also known as a “Model Clash” 10/6/2010 14 © USC-CSSE
Techniques
• There are a myriad techniques that can be applied for stakeholder value proposition reconciliation: – Expectations Management: Just awareness of potential conflicts could help SCSs relax their less critical levels of desire – Visualization & Trade-off Analysis Techniques: Prototyping, scenarios, estimation models help SCSs to mutually understand most important & achievable aspects of the system – Prioritization: Helps identify which combination of capabilities best satisfies the stakeholders’ most critical needs within available resource constraints – Groupware: Some groupware tools have support for collaborative brainstorming, discussions, prioritization etc., 10/6/2010 15 © USC-CSSE
3. Business Case Analysis (BCA)
• • • • Another extremely valuable technique for stakeholder value proposition reconciliation Helps determine where is the best bang for the buck – i.e., capabilities providing the best ROI (We’ll see how to create a VB-Business Case later) Can directly influence capability prioritization TWO Major factors influencing BCA – Unquantifiable benefits (a.k.a. intangibles) – Uncertainties & risk 10/6/2010 16 © USC-CSSE
Techniques
• • • • • ROI – most commonly used to justify a business case. Difficult to apply for intangibles Deriving variables/attributes from an objectives hierarchy – closer to value based but we can derive them straight from the benefits chain!
Each of the variables need to be ‘estimated’ either using subjective judgment, past data, expert advice… …OR we could ‘calibrate’ the estimator to provide better estimates! We’ll see how to do this in Section 2 10/6/2010 © USC-CSSE 17
4. Continuous Risk & Opportunity Management • Understanding & Addressing People’s Utility Functions – Risk Averse
Utility
1,1
It is possible to have
– – Risk Neutral Risk Seeking
negative utilities too
losses
• • • Implication(s): 0,0 Very people oriented process Continuous and Iterative manage and mitigate risks
Benefit
Must have plans/processes in place to identify, 10/6/2010 18 © USC-CSSE
Central Tenet of Risk Management
• Metric Risk Exposure:
RE = Probability (Loss) * Size (Loss)
– “Loss” of stakeholders’ value –
Ex.: profits, reputation, quality of life etc.,
• Prioritizing Risks using Risk Exposure (may be misleading!) High Check Utility Loss Estimate Major Risk Low 10/6/2010 Low Little Risk
Loss of Utility
Check Probability Estimate High
• • •
5. Concurrent System & Software Engineering
Increasing pace of change in IT marketplace traditional sequential approach to software development (i.e., Waterfall model) is increasingly risky to use!
Preferable: Concurrently engineer the product’s operational concept, requirements, architecture, life cycle plans and key sections of code Concurrent engineering further preferable if: – Software system requirements emergent from usage and/or prototyping rather than prespecifiable – Costs, benefits & risks of COTS software and/or outsourcing decisions affects requirements, architectures, cost, schedules etc., 10/6/2010 20 © USC-CSSE
Lifecycle Objective (LCO)
For at least one architecture, a system built to that architecture will: • Support core Operational concept • Satisfy core requirements • Be faithful to prototype(s) • Buildable in budget/schedule in the plan • Show a viable Business case • Have key SCSs committed to support the ‘next phase’
Lifecycle Architecture (LCA)
For a specific detailed architecture a system built to that architecture will: • Support elaborated operational concept • Satisfy elaborated requirements • Be faithful to prototype(s) • Buildable in budget/schedules in the plan • Show a viable Business case • Have key SCSs committed to support the full lifecycle • All Major risks resolved/covered by a risk management plan
Initial Operational Capability (IOC)
An implemented architecture, an operational system that has: • Realized the operational concept • Implemented the initial operational requirements • Prepared a system operation & support plan • Identified the initial site(s) of deployment for initial transition • Identified users, operators & maintainers to assume operational roles 21
Techniques
• • There are a myriad software process models – choose the one that fits best to your organization Feasibility evidence MUST be provided at each milestone to support the claims as shown on the previous slide 10/6/2010 © USC-CSSE 22
6. Value-Based Monitoring & Control
• Use project’s business case for monitoring the actual business value of the capabilities to be delivered YES Develop/update business case; time phased cost; benefit flows; plans 10/6/2010 Perform to plans Value being realized?
YES Assumptions still valid?
NO Determine Corrective Actions 23 NO © USC-CSSE
Techniques
• • • • Forms the bulk of the ‘detailed’ section on VBSE in Practice What metrics to use? Are those the right metrics?
Are we gaining benefits? Are we doing the right things and doing them correctly?
We shall answer these questions in Section 2.
10/6/2010 © USC-CSSE 24
7. Change as Opportunity
• • • • • Today’s world – a not so good Present Expending resources to adapt to change is often stated as “rework costs” Changes are treated as defects in change tracking systems The real world: Continuous ongoing changes in – Technology – Marketplace – Organizational –
Opportunities can become risks if nothing is done about it!
Stakeholders’ value propositions and priorities 10/6/2010 25 © USC-CSSE
• • • •
A Brave New World
Organizations that can adapt to change more rapidly will succeed better in the their mission Developing change anticipatory modular design can be considered a good investment to enhance system value in the future Two techniques for enhancing adaptability to change: – Architecture based – Refactoring based Example of “Change as Opportunity” Internet and the Web and their effect on ecommerce 26
10/6/2010
Section 2: Applying VBSE in Practice
© USC-CSSE 27
Agenda
• • • • • Identify the need Identify the motive/goal/objective of the need to be satisfied Check alignment with company/organization’s goals and objectives: • Find/Derive the ‘value’ of the initiative • Identify initiatives to realize value • Identify validate/assumptions when articulating value • Identify success critical stakeholders (SCSs) Elicit value propositions (win conditions) of SCSs Create a value-based business case with information from above and check for business alignment • ‘Percolate’ the value to all phases of the SDLC (software • development lifecycle) Track/adjust the business case over time to ascertain value gained/created and validate assumptions 28
Company Objectives
• • • • Before we start – we MUST know on what basis does the organization decide to pursue any endeavor Profit/Revenue is not the only variable… …Competitive advantage, Brand projection, Enhanced customer loyalty etc., to name a few The current undertaking must align with the strategic goals of the company 10/6/2010 © USC-CSSE 29
Ranking of Strategic Goals
Say you have two programs each satisfying (or satisficing) the organization’s goals in varying degrees, which program should you choose?
1. Layout the full ‘results chain’ and see what benefits can be gained – disadvantage? There is no inherent ranking 2. Use Multi-Attribute Utility Theory MAUT (could also use Analytical Hierarchy Process (AHP) but we’ll focus on the former) 10/6/2010 © USC-CSSE 30
WHY MAUT??
• • • • Mathematically sound Easier to capture the risk attitude in the form of utility functions Strategic goals don’t tend to have too many attributes and thus attribute space is well managed and easily understood Easier to lay coarse grained benchmarks for various attributes 10/6/2010 © USC-CSSE 31
MAUT Example
• • Let’s capture a single utility function for Revenue The more revenue the better – increasing utility
50%
Gamble Best: $10M
50%
Worst: $0.5M
Sure thing: Best: $10M_ _____ _____ _____ _____ _____ Worst: $0.5M
For which value of the sure thing are you indifferent between the sure thing and the gamble? _______ 10/6/2010 © USC-CSSE 32
Risk Averse
Graphing Utility
(1, $10M) Utility 0.75
0.5
0.25
(0, $0.5M) Revenue $3M $1.5M
$4.7M
$7.5M
$8.1M
$8.6M
Risk Seeking © USC-CSSE 33 10/6/2010
Finding Utility
• • • Q: Given a value of the ‘attribute’ what is its utility? How do we find that from the graph?
This second step involves understanding the ranking of the attributes: –
“If you can change one attribute from its worst state to the best state which one would you choose? Ties are allowed.”
– Repeat this question till all attributes are ranked For each of the ranked attributes repeat the Gamble vs Sure thing lottery to find the ‘P’ i.e., probability at which you’ll be indifferent!
10/6/2010 © USC-CSSE 34
Calculating the Attribute Tradeoff Scaling Constant (k
i
)
P 1-P
Gamble Best System All attributes at best state Worst System All attributes at worst state Sure thing: Reference System: -All attributes at worst state -Chosen attribute at best state P = 100% _____ _____ _____ _____ _____ 0% For what value of ‘P’ are you indifferent between the sure thing and the gamble? ________ 10/6/2010 © USC-CSSE 35
Finding Utility (Finally!)
10/6/2010 © USC-CSSE 36
MAUT (Cont’d)
• • • • Advantages: Simple question answering and choosing the outcomes of a lottery (sure thing vs. gamble) Can easily be ‘computerized’ Understanding a lottery comes naturally to most people and the risk attitude is hidden within the choices Difficult to ‘game’ the system with false attitudes • • • Disadvantages: May be non-intuitive to start with Time consuming to get 5 point utility functions and attribute tradeoff scaling constants Sometimes it’s just difficult to get utility functions (Arrow’s impossibility theorem) 10/6/2010 © USC-CSSE 37
10/6/2010
Next: Benefits Analysis
Understanding the Value of ‘what you are doing’ © USC-CSSE 38
Before We Begin…
• • • The previous application of MAUT is optional if your organization already has prioritized it’s strategic goals and objectives (continually updated) Still may be valuable to ‘mathematically’ affirm the choices!
There are a myriad of methods available to help you with this: Balanced Score Cards, AHP etc., They are more ‘management’ i.e., no mathematical basis 10/6/2010 © USC-CSSE 39
• •
Approach
Use a live example from anyone in the audience (could be any example even something you are currently working on) or use a case study example of Sierra Mountain Bikes (SMB) (Former preferable) Walk through the steps of applying VBSE – – ONE such way of applying VBSE Has research viewpoints – – Open to discussion Combines techniques from Statistical Decision Theory, Management and Software Engineering *Taken from: An Initial Theory of VBSE – Barry Boehm & Apurva Jain 10/6/2010 © USC-CSSE 40
• • •
SMB – Problem Description
Susan Swanson, CEO of SMB when developing a shared constructive vision of SMB’s primary opportunities and problems, determined a significant opportunity of growth, since they produced top quality bikes at a competitive price. Major problem area: old manual order processing system. Distributors, retailers and customers were dissatisfied
with high rates of late/wrong deliveries, poor synchronization between order entry, confirmation & fulfillment; and disorganized responses to problem
solutions. It was decided to outsource the development of the new system, since it wasn’t their area of expertise.
10/6/2010 © USC-CSSE 41
The initiative identified by SMB is to Assumptions: - Increasing market size with product ‘implement a new order entry system’ to get rid of the manual order processing - Relatively stable e-commerce New Order Fulfillment System infrastructure - Continued high staff performance Safety, fair ness of inputs New Order Fulfillment processes, outreach, training Inter - operability inputs New Order Entry System Lesser time, fewer errors/order fulfillment system Less time, fewer errors in order processing Faster, better order fulfillment system Developers Increased customer satisfaction, decreased operation costs On-time assembly
Under what SO WHAT?
are these ‘contributions’ of the initiatives?
Faster order entry steps, fewer errors New order entry processes, training and outreach Sales personnel, distributors Improved supplier coordination Suppliers 10/6/2010 © USC-CSSE Increased sales, profitability, customer satisfaction Increased profit, growth
What all to do to realize the benefits/ outcomes??
Who are the accountable stakeholders to help realize the benefits?
42
Benefits Analysis
• • • • • The previous slide lays out the expected benefits of having a new order entry/fulfillment system The ‘initiative’ of having a new system is linked to the final outcome ‘increased profitability/growth’ (strategic goal?) through intermediate outcomes!
Each initiative must be followed by an outcome or to realize the benefits of an outcome some initiative(s) need to be taken (may not be so for the final outcome(s)) Key question to ask at each stage: “So what?” to get the subsequent outcome/initative.
Stakeholders who are/would be accountable for realizing/reporting the benefits should be identified 10/6/2010 © USC-CSSE 43
Key Questions for performing Benefits Analysis Q1: Why do you want to know the value of “<
initiative >”?
A: You must know the why to know the purpose of performing the benefits analysis i.e., the value of knowing the value of the initiative It’ll help understand the alignment with the organization’s strategic goals (will also help with identifying the need)
Ex: “Why do you want to know the value of having a new order entry processing system?” A: So that the company has a viable case to know/believe if and whether a new system would/could increase productivity, growth, profit and customer satisfaction and by how much
10/6/2010 © USC-CSSE 44
Key Questions (Cont’d)
Q2: What IS the value of “< initiative >”?
A: Perform the prior benefits analysis to see the value!!
Q2.1: How do you define “value”?
A: You just did! Questions 1 and 2 help answer this question itself. The “value” in this case is alignment with company goals and ‘how much’ does the said initiative align with it!!
Q3: All this is good but ‘what exactly is the value’? What do I track? There are tons of intangibles to complicate the problem further?
10/6/2010 © USC-CSSE 45
Epiphany: Measurement!
• • • • • WHAT to measure? We already have standard metrics
that we track for every project
Are those ‘standard metrics’ really tracking overall project value or just some numbers?
What about intangibles? They are almost impossible to
track at best. We have a 1-N rating system to help us with that
They may be hard, but not impossible! Douglas Hubbard has even published a book “How to Measure Anything – Finding the Value of Intangibles in Business”
…Let’s see how we can use some of that wisdom to
alleviate our measurement problem. But first… 10/6/2010 © USC-CSSE 46
WHAT to track/measure?
• • • • This is where we’ll build a value-based business case Directly derivable from our prior benefits analysis!
Yes, we’ll also track/measure intangibles – probably won’t be a 1-N scale but actual numbers Risks! We didn’t forget that. We’ll also see how to better identify ‘strategic risks’ and measure them too!
10/6/2010 © USC-CSSE 47
Assumptions: - Increasing market size Distributors, retailers, customers - Continuing consumer satisfaction with product - Relatively stable e-commerce infrastructure New Order Fulfillment System New Order Fulfillment processes,
What all measurable items can we find from this diagram?
- Continued high staff performance Safety, fair ness of inputs Inter - operability inputs New Order Entry System Lesser time, fewer errors/order fulfillment system Less time, fewer errors in order processing Faster, better order fulfillment system Developers Increased customer satisfaction, decreased operation costs
Associated costs of having a new system
Faster order entry steps, fewer errors New order entry processes, training and outreach
(make/buy/customize…)
Sales personnel, distributors On-time assembly Improved supplier coordination Suppliers Increased sales, profitability, customer satisfaction Increased profit, growth
Something is missing… Something is still missing…
• • • • • •
But still…What exactly should be measured?
The previous exercise should promote the discussion of what is concretely meant by the terms – i.e., how can it be quantified?
But this is exactly the problem with measuring coarse grained criteria!
And that’s where we start to think
REALLY HARD!
If you can observe a difference then it can probably be measured!
In fact the same benefits chain diagram can be used to simulate the discussion!!
Let’s see how… 10/6/2010 © USC-CSSE 49
Example: Growth
Increased sales Increase profits Increase growth Cost savings Increase market share
SO WHAT?
This approach probably looks familiar – something from classical decision theory?
10/6/2010 © USC-CSSE 50
Objectives Hierarchy
• • • • • You could replace the previous analysis with that of ‘objectives’ hierarchy In fact you could replace the entire benefits chain analysis with an objectives hierarchy (You already know the top most goal “Why do you want to know the value of “_____”?) You will however lose the connections between outcomes, stakeholders accountable, initiatives to be taken etc., of the benefits chain approach Suggestion: You can use it at this stage when ‘drilling down’ the ambiguous metrics to more specific attributes! OR Transform the benefits chain into an objectives hierarchy if you are comfortable with that!
10/6/2010 © USC-CSSE 51
How to Measure/Estimate
• • • • Once you’ve drilled down both, tangibles and intangibles to concretely observable (measurable) facts we can proceed with how best to estimate/measure them The prior analysis from benefits chain all the way to measurable attributes should provide enough dimensions on which to capture the value of the project But aren’t the estimates just guesstimates for the ‘value based business case’?
Yes, to some extent. Let’s see how to solidify these estimates… 10/6/2010 © USC-CSSE 52
Are you 90% Confident?
• • • • • When was the last time you were 100% confident with your estimates?
Were you confident 100% of the time ?
100% confident estimates 100% of the time would make you a priceless asset!!
Can you at least be 90% confident 90% of the time?
Wait a minute! How can you measure this confidence level??
That’s where we take
’s help. Remember him?
10/6/2010 © USC-CSSE 53
Getting a 90% Confidence Interval (CI)
• • • Remember CIs from your statistics class?
A 90% CI is a range that has a 90% chance of containing the ‘correct answer’ Let’s try a simple test: (No need to Google ) Question What is the circumference of Mars?
Lower Bound _____________ Upper Bound _____________ Provide a range wide enough so that you believe there is a 90% chance that the value will be within this range (Units don’t matter. We are more interested in the range) 10/6/2010 © USC-CSSE 54
Would you Gamble?
• • Let’s say we gave you the following proposition: You win $1000 if the actual answer is within your range You win $1000 by spinning the wheel below:
$0 10% 90% Win $1000
• What would you NOW CHOOSE?
10/6/2010 © USC-CSSE 55
Choices & Inferences
Choice
Gamble Your initial estimate Indifference
Inference
You think the gamble has a higher chance of payoff! Implies your 90% CI is NOT really your 90% CI but lower – 50%, 60%, 80%. Basically an overconfident estimate This means that you think there is more than a 90% chance your range contains the answer! An underconfident estimate You are indifferent between your initial estimate and the gamble an equivalent 90% chance 90% CI 10/6/2010 © USC-CSSE 56
Sure Thing vs Gamble
• • • If you are still with me you’ll realize that the
previous exercise is similar to the one
we did for capturing stakeholder utility! We just replace the ‘sure thing’ by your estimate and the gamble is a 90% Win and 10% loss! You ARE using utility theory concepts to make a 90% confident estimate that factors in uncertainty and probable prior knowledge of experts!
10/6/2010 © USC-CSSE 57
• • • •
The Value Based Business Case
The prior analyses of benefits, initiatives, stakeholder identification, metrics can provide a great first cut value based business case For quantities that need to be estimated you could apply the 90% CI calibration exercise to get ‘better’/more reliable estimates However, one question is not enough to help you calibrate. You may have to spend some time practicing calibrating yourself* You may need some sample information to help you get started… *Please refer “How to Measure Anything…” by Douglas Hubbard for more details/exercises 10/6/2010 © USC-CSSE 58
• • • • • •
Value of Information
How much time/money should you spend (if at all) gathering information ‘about’ the metric before estimating?
That depends on the expected value (EV) of information – Commonly applied in statistical decision theory but uncommonly so in software engineering!
Defined as Expected Value of Perfect Information (EVPI): EVPI = EV (perfect info) – EV (no info) EVPI is the ‘theoretical maximum’ you should be willing to pay for procuring further information Since perfect information is not obtainable, EVSI (expected value of sample information) is usually employed. Thus the net expected value of information is gained by ‘subtracting’ cost of sampling/procuring information!
This was elucidated way back in 1981 by Dr. Boehm when he published the Blue Book*: “Software Engineering Economics”!
*Chapter 20: Statistical Decision Theory: Value of Information
10/6/2010 © USC-CSSE 59
10/6/2010
Next: Eliciting Stakeholders’ Win Conditions
(a.k.a., Value Propositions) © USC-CSSE 60
Stakeholders’ Value Propositions
• • • • We started our discussion with Theory-W… …we now build on how to ‘implement’ it in action Value Propositions are also known as Win
Conditions
Theory-W involves eliciting, understanding and reconciling SCS (Success Critical
Stakeholders) win conditions with achievable solutions
10/6/2010 © USC-CSSE 61
Win-Win Negotiation Process
* • • • • • • • Collect SCS win conditions (WC) – basically brainstorming what the ‘expect’ from the system Converge on WCs – since it’s captured in natural language effort is taken to converge on a concisely worded, non-redundant, unambiguous list of Win Conditions Define glossary of key terms – basically the domain glossary used within a project Prioritize WCs – Business Value vs. Ease of Realization Reveal Issues/Constraints – the prioritization helps facilitate discussion about issues that may arise with the stakeholder’s expectations Identify Issues and Options – the team posts issues (if any) for each of the identified win conditions and identifies the options to deal with the issues Negotiate Agreements – negotiate/agree to the above, using oral discussion and entering it into the ‘online tool’. We say WinWin Equilibrium is reached when all WCs and Options have been agreed to.
*Stakeholder Value Proposition Elicitation and Reconciliation – Grunbacher et. al.
10/6/2010 © USC-CSSE 62
How/Where to Capture Win Conditions
• • I’m currently working on an online collaboration tool, similar to Facebook to help capture stakeholder’s win conditions, raise issues, supply options, prioritize win conditions and indicate the current statue of equilibrium Attached are the screenshots of the tool. The tool will be deployed on USC’s servers for the upcoming class of Software Engineering CS577a in Fall ’11 (The tool will also be available for download for use by anyone) 10/6/2010 © USC-CSSE 63
Winbook 64
• •
Prioritization
These are two primary approaches that I’ve narrowed down from a value based perspective
1.Incremental Funded Methodology (IFM)
Advocated by the book “Software by Numbers” – Mark Denne, Jane Cleland-Huang.
Deals with grouping features into minimum marketable features and performing financial analysis using net present values and using that to predict ROI
2.Kano Analysis
Commonly applied in Quality Function Deployment for prioritizing voice of customer data. Also advocated by Agile Mentor Mike Cohn for use in agile software development.
We’ll go with the latter since the former is mathematically involved. Both could be applied in tandem, but we’ll go with the latter 10/6/2010 © USC-CSSE 65
Kano Analysis
Linear: The more of the feature the better Exciter: If absent customer wouldn’t care but would be delighted to have them Must be: Absence implies severe dissatisfaction. Expected to be present
So, how do we know which features fall in which category?
Image From: http://en.wikipedia.org/wiki/Kano_model
10/6/2010 © USC-CSSE 66
Kano Questionnaire
* You ask the following question(s) to the prospective users: Functional Form of Question If feature X were present how would you feel?
I like it that way I expect it to be that way I am neutral I can live with it that way I dislike it that way
X
Dysfunctional Form of Question If feature X were absent how would you feel?
I like it that way I expect it to be that way I am neutral I can live with it that way I dislike it that way
X
*Further details can be found in Agile Estimating and Planning – Mike Cohn.
10/6/2010 © USC-CSSE 67
Cross-referencing the Responses
Functional Question 10/6/2010 Customer Requirements Dysfunctional Question Like Expect Neutral Live with Dislike Like
Q R R R R
Expect Neutral Live with Dislike
I I I E R I I I E R
E Exciter
I
M Must Have Indifferent © USC-CSSE L Linear Q Questionable R Reverse
I I I E R L M M M Q
68
• • • • • • •
Percolation of Value into the SDLC
The prioritized win conditions give an idea of how the customers values the features on a scale of desirability The win conditions are the set of value propositions of the stakeholders – their expectation(s) from the system Win conditions can be ‘mapped’ to benefits – either in the form of a matrix or by visual inspection You can even ‘rank’ the benefits in order of importance using MAUT too!
Decisions about various win conditions are then taken with respect to the benefits to be achieved Decisions at the project level must factor in the impact on strategic goals This ‘framework’ of expected benefits should always be visible to the development team!
10/6/2010 © USC-CSSE 69
Value Based Tracking
• • • • • I haven’t explored this area in it’s totality but what seems to hold a lot of promise is “Multivariate Statistical Analysis” especially Multivariate Regression Analysis We can pick out key ‘independent variables’ from the intermediate outcomes of the benefits chain and let the ‘dependent variable’ be that from the final intended goal/outcome We can run a regression analysis to know if we really gained the benefits. Statisticians have been doing it all along!
This implies that if we DON’T have relevant data we may need to perform sampling to have enough data points to warrant this analysis.
Is the sampling effort worth it? It depends on the value of information!
10/6/2010 © USC-CSSE 70
10/6/2010
Wrapping Up!
…finally © USC-CSSE 71
Conclusion
• • • • • Presented ONE way of applying VBSE in practice using ONE set of tools and techniques.
There are myriads of possible combinations of tools/techniques that could be used. Use the one best suited to your organization… …The ideas in this presentation should help you get started We’ve covered a lot of ground but it’s only possible to present a limited amount. I’ve attached a set of references for further reading!
Hope you gained some ‘value’ out of this presentation!
10/6/2010 © USC-CSSE 72
References
• • • • • • • • • • Value Based Software Engineering – Stefan Biffl et. al.
How to Measure Anything - Douglas Hubbard The Information Paradox – John Thorp Software Engineering Economics – Barry Boehm Making the Software Business Case – Donald Reifer Decision with multiple objectives – Keeney, Raiffa Agile Estimation and Planning – Mike Cohn Software by Numbers – Mark Denne, Jane Cleland Huang Lean Software Development - Poppendieck Agile Software Requirements – Dean Leffingwell 10/6/2010 © USC-CSSE 73