Risk and Risk Management (Theory and Practice) Todd Little and Chris Matts “It’s tough to make predictions, especially about the future.” Yogi Berra, Niels Bohr.
Download ReportTranscript Risk and Risk Management (Theory and Practice) Todd Little and Chris Matts “It’s tough to make predictions, especially about the future.” Yogi Berra, Niels Bohr.
Risk and Risk Management (Theory and Practice) Todd Little and Chris Matts “It’s tough to make predictions, especially about the future.” Yogi Berra, Niels Bohr Exercise What are the types of risk? How do “Risky Businesses” work Financial Markets A severe depression like that of 1920-21 is outside the range of probability. Harvard Economic Society, Weekly Letter, November 16, 1929. New Product Development I think there is a world market for about five computers. Thomas J. Watson, chairman of IBM, 1943. War They couldn't hit an elephant at this dist… General John B. Sedgwick, Union Army Civil War officer's last words, uttered during the Battle of Spotsylvania, 1864 Oil & Gas Exploration Which Risks Are Important Oil and Gas Drilling Well 2 Well 1 Surface Seismic reflection A Reservoir Water B Oil Oil and Gas Drilling Well 2 Well 1 Surface Seismic reflection A Reservoir Water B Oil Geosteering for Uncertainty Well 2 Well 1 Surface A Planned Reservoir Actual Reservoir B Well 2’ Figure 2 Hurricane Rita Getting Better Movies Books Venture Capital Categories of risk Delivery Failure Business Case Failure Collateral Damage Delivery Challenges/Failures Succesful 35% Failed 19% Challenged 46% Standish Group 2006, reported by CEO Jim Johnson, CIO.com, ‘How to Spot a Failing Project’ Business Case Failure Collateral Damage Wrong Priorities Collateral Damage An effective roll-back strategy Delivery Failure results in Collateral Damage I’m beginning to think it wasn’t such a good idea to turn off those unit tests Business Case Failure Features and Functions Always or Often Used: 20% Always 7% Often 13% Sometimes 16% Never Used 45% Rarely Used 19% Never or Rarely Used: 64% Standish Group Study, reported by CEO Jim Johnson, XP2002 Purpose Alignment Model High Partner Differentiating Who Cares? Parity Market Differentiating Low High Low Mission Critical Purpose Does Not Equal Priority Graphically - Before High Project Tracking Document Mgmt Document Edit Document Library Search EDGAR Integration Market Differentiating Low High Low Mission Critical Graphically - Before High Portal Market Differentiating Document Edit Project Tracking Document Mgmt Document Library Search EDGAR Integration Low High Low Mission Critical Result: Better product in half the time and 60% of the original cost Example: Apple High New Product Design User Experience Content Distribution ATT Market Differentiating Peripherals Other Software MS Office Intel Hardware Low High Low Mission Critical Considerations Risks Assumptions Constraints Delivery Failure Managing the Coming Storm Inside the Tornado Project Kickoff When will we get the requirements? All in good time, my little pretty, all in good time But I guess it doesn't matter anyway Just give me your estimates by this afternoon Team Unity Not so fast! Not so fast! ... I'll have to give the matter a little thought. Go away and come back tomorrow No, we need something today! Ok then, it will take 2 years. No, we need it sooner. Doesn't anybody believe me? I already promised the customer it will be out in 6 months You're a very bad man! We’re not in Kansas Anymore Developer Hero I may not come out alive, but I'm goin' in there! Reorg The Great and Powerful Oz has got matters well in hand. My! People come and go so quickly here! Testing "Hee hee hee ha ha! Going so soon? I wouldn't hear of it! Why, my little party's just beginning! IEEE Software, May/June 2006 Data from LGC Initial Estimate vs. Actual Project Duration (from LGC Portfolio Database) 1200 1000 y = 1.6886x Actual 800 600 LGC Data Ideal 400 Linear (LGC Data) 200 0 0 100 200 300 400 500 Initial Estimate 600 700 800 900 1000 CDF Distribution Curve (LGC) Landmark Graphics Cumulative Distribution of Portfolio Projects 1 0.9 Cummulative Probability 0.8 p(10) p(50) p(90) 0.7 0.6 0.5 0.4 0.96 1.76 3.23 0.3 0.2 0.1 0 0.1 1 Ratio of Actual to Estimate 10 How does Estimation Accuracy Improve Over Time? (Boehm) 10 Minimum Maximum 2 1 0.5 0.1 Feasibility Concept of Operation Requirements Product Detail Design Spec Design Spec Spec Accepted Software Landmark Cone of Uncertainty Another look? 1200 1000 800 Ideal From Start Post Env Post Plan Post Dev 600 400 200 0 0 100 200 300 400 500 600 But is Uncertainty Really Reduced? “Take away an ordinary person’s illusions and you take away happiness at the same time.” Henrik Ibsen--Villanden Cumulative Distribution (CDF) Curve 1 0.9 0.8 0.7 0.6 Initial Post Env Post Plan Post Dev Log Normal 0.5 0.4 0.3 0.2 0.1 0 0.1 1 10 100 Remaining Uncertainty The Pipe of Uncertainty 10 Minimum Maximum 2 1 0.5 0.1 Envisioning Planning Developing Stabilizing Why is Software Late? From the home office in Duncan, Oklahoma Dubai, UAE Top Ten reasons why we are late in 2008 Top Ten reasons why we are late in 2008 10: Requirements, what Requirements? What you want, baby I got it R-E-Q-U-I-R-E Find out what it means to me Top Ten reasons why we are late in 2008 9: Dependencies on other groups that were late Top Ten reasons why we are late in 2008 8: Over-optimistic Schedule Estimation Always look on the bright side of code ....... Always look on the bright side of code ....... The code’s a piece of $#!^, when we look at it We can always overlook a minor kink . . . . It probably compiles, it might even link . . Surely that must mean it doesn’t stink Top Ten reasons why we are late in 2008 7: Those weren’t MY estimates Scheduling Ritual How low can you go! Top Ten reasons why we are late in 2008 6: Not enough testers or documentation resources. Who needs them anyway? We put those bugs--I mean features-in there on purpose. Besides, it was difficult to program, it should be difficult to use. Top Ten reasons why we are late in 2008 5: Offshore and Outsourcing issues My source code lies over the ocean, My source code lies over the sea . My source code lies over the ocean, Oh bring back my source code to me ..... Bring Back, Bring Back, oh bring back my source code to me, to me Bring Back, Bring Back, oh bring back my source code to me Top Ten reasons why we are late in 2008 4: One word, Ch-ch-ch-changes Top Ten reasons why we are late in 2008 3: I can’t get no, System Admin – I can’t get no, CM action – ‘cause I try, – ..and I try, – ….and I try, – ……and I try…. Top Ten reasons why we are late in 2008 2: You didn’t give me the headcount that you promised Top Ten reasons why we are late in 2008 1: Weren’t you doing the backups!? Why is Software Late? Genuchten 1991 IEEE General Manager Project Manager 1 10 Insufficient front end planning 2 3 Unrealistic project plan 3 8 Project scope underestimated 4 1 Customer/management changes 5 14 Insufficient contingency planning 6 13 Inability to track progress 7 5 Inability to track problems early 8 9 Insufficient Number of checkpoints 9 4 Staffing problems 10 2 Technical complexity 11 6 Priority Shifts 12 11 No commitment by personnel to plan 13 12 Uncooperative support groups 14 7 Sinking team spirit 15 15 Unqualified project personnel Item Why is Software Late? Genuchten 1991 IEEE General Manager Project Manager 4 1 Customer/management changes 10 2 Technical complexity 2 3 Unrealistic project plan 9 4 Staffing problems 7 5 Inability to track problems early 11 6 Priority Shifts 14 7 Sinking team spirit 3 8 Project scope underestimated 8 9 Insufficient Number of checkpoints 1 10 Insufficient front end planning 12 11 No commitment by personnel to plan 13 12 Uncooperative support groups 6 13 Inability to track progress 5 14 Insufficient contingency planning 15 15 Unqualified project personnel Item Defending an Unpopular Schedule • Developers tend to be temperamentally opposed to the use of negotiating tricks. Such tricks offend their sense of technical accuracy and fair play. Developers don't want to offer lopsidedly high initial estimates even when they know that customers, marketers, or bosses will start with lopsidedly low bargaining positions. – Steve McConnell http://www.stevemcconnell.com/ieeesoftware/bp03.htm We want this Negotiation Bias • "It is difficult to get a man to understand something when his salary depends upon his not understanding it.“ » Upton Sinclair: Test 1 (Jørgensen) Group Guidance A 800 B 40 C 4 D None Result 160 Test 1 Group Guidance Result A 800 300 B 40 100 C 4 60 D None 160 Test 2 Group Guidance A Minor Extension B New Functionality C Extension Result 50 Test 2 Group Guidance Result A Minor Extension 40 B New Functionality 80 C Extension 50 Test 3 Group Guidance A Future work at stake, efficiency will be measured B Control Result 100 Test 3 Group Guidance Result A Future work at stake, efficiency will be measured 40 B Control 100 Understand Bias • "What gets us into trouble is not what we don't know. It's what we know for sure that just ain't so.“ » Mark Twain Uncertainty Know that we know Know that we don’t know Don’t know that we know Knowable Don’t know that we don’t know Unknowable Uncertainty Know that Planning we know Know that Risk Management we don’t know p10 p50 Don’t know that we know Knowable Don’t Uncertainty know that we Management don’t know Unknowable p90 The Goal Time To Spec Within Budget On Da Plan, Boss – Da Plan The Cone of Uncertainty We expect uncertainty and manage for it through iterations, anticipation, and adaptation. Delivery Failure. staff liquidity Context Leadership Model Bulls Sheep Dogs Cows Uncertainty Colts Project Complexity Context Leadership Model High Colts Uncertainty Simple, young projects. Need agility Tight Teams Sheep Dogs laissez faire Bulls Agility to handle uncertainty Process definition to cope with complexity Cows Complex, mature market Need defined interfaces Low Low High Project Complexity Bull Product Release Reduce Uncertainty or Complexity Uncertainty Complexity Attribute Score Attribute Score Market ███ Team Size █████████ Technical ███ Mission Critical █████████ # Customers █████████ Team Location █████████ Duration █████████ Team Maturity ███ Change ███ Domain Gaps ███ Dependencies █████████ Opportunities to Reduce Uncertainty: Opportunities to Reduce Complexity: • Use proven technologies • Reduce project duration • Collocate the team • Break project into sub-projects Partitioning Colt Project Bull Program Dog Project Cow Project Remember: Loose Coupling and Strong Cohesion First Integration Release High Colts Bulls The Integration Release Uncertainty New acquisitions SheepDogs Existing Products Cows Integration data model Low Low High Project Complexity Integrating Software by Integrating People Creating the Future Friday@4 Weekly PMM Quarterly Developers’ Conference Yearly Y2K Release High Colts Bulls None Uncertainty None Dogs All Products Cows The overall Program Low Low High Project Complexity Project Leadership Guide Create Change High Deploy Market Differentiating Invent Ad Hoc Eliminate Change Offload Low Embrace Change Outsource Low Agile Control Change Manage Structured Mission Critical High Portfolio Management RAPID Quadrant Assessment 12.0 Uncertainty Uncertainty 10.0 8.0 6.0 4.0 2.0 0.0 0.0 5.0 10.0 15.0 20.0 Project Complexity Project Complexity 25.0 30.0 Not all dogs are the same Successful Projects? Contact Todd Little [email protected] www.toddlittleweb.com www.accelinnova.com Extra Slides Project Selection High Probability of Success Consider Caution Extreme Caution Avoid Low Low Risked Cost of Failure High Decisions, Decisions, Decisions What do we do? When do we do it? When do we decide? Real Options Deciding Is Not a Once in a Lifetime Event • Knowledge Improves • Business Conditions Change • Project Conditions Change • Do You Know Why You Are Deciding Early? Risks – Types of Uncertainty Cost/Time Uncertainty General Market Uncertainty Feature Acceptance Uncertainty Target Best possible scenario if everything went perfectly. Uncertainty Value Plan Planned scope for the release at the optimal time that it can be released. Contract Minimum scope for the release at the latest date that it can be released. Tim e The A/B/C List sets proper expectations A MUST be completed in order to ship the product. SHOULD be completed in order to ship the product. MAY be completed prior to shipping the C product if time allows. Only “A” features may be committed to customers. B “A” features must fit in a p90 confidence schedule. No more than 50% of the planned effort can be allocated to “A” items A/B/C List 50% 25% 25% B C D A Typical Delivery 1.2 1 0.8 Backlog Plan 0.6 A B C 0.4 0.2 50% 100% Target Delivery Date em be r r D ec em be N ov ct O m be te ob er r t us Se p Au g ly Ju ne Ju ay M il Ap r ar ch M ry ua Fe br Ja nu ar y 0 A/B/C List 50% 25% 25% A B Uncertainty Risk C D 1.2 1 0.8 Backlog Plan 0.6 A B C 0.4 0.2 50% 100% Target Delivery Date em be r r D ec em be N ov ct O m be te ob er r t us Se p Au g ly Ju ne Ju ay M il Ap r ar ch M ry ua Fe br Ja nu ar y 0 Products Lifecycle Paths Product Lifecycle Uncertainty High C Colts A Low Low Bulls Skunks B Dog s Complexity Cows High