Transcript Pragmatic Uses for Risk Management Practices

Pragmatic Uses for Risk Management
Practices
A work in progress !!
Presented by Douglas Brown, PMP
PMO Head
SEC Office of Information Technology
202-551-8176
[email protected]
PMI Silver Spring Chapter
12 April 2006
Disclaimer
The Securities and Exchange Commission, as
a matter of policy, disclaims responsibility for
any private publication or statement by any of
its employees. The views expressed herein are
those of the author and do not necessarily
reflect the views of the Commission or of the
author’s colleagues upon the staff of the
Commission.
Perspectives
SEC IT

Approximately $95 million annually
 Went from $40 M to over $100 M 2002-2004
 Seismic culture shift, aftershocks still settling

103 internal IT employees, 300+ contractors
supporting






4000 agency employees at HQ and 10 regional offices
1000-3000 contractors and other users
Several thousand “regulated entities”
Millions of individual investors (EDGAR)
40-60 new projects or new phases annually
60-100 ongoing projects at any one time
Risk Management – by the PMBoK
Risk Identification
Risk Analysis
Risk Response Planning
Risk Monitoring and Control
Generally results in



A Risk Management Plan
That sits on the shelf
But meets a compliance requirement
Risk Basics
A good word for OMB-300


E-300 specifies that project schedules and budgets must
be specifically risk-loaded
This is the genesis of SEC’s approach
Differentiate between Risk and Issue



Risk = EVENT or CONDITION that MAY occur and, if it did,
would lead to project failing to meet baseline
ISSUE = something that is happening (or, usually, NOT
happening) now that will result in failing to meet baseline if
not resolved by X date.
ACTION ITEM = something that someone needs to do to
carry out their part of the plan. If they do not, it would be a
problem but we do not have any reason to believe they will
not (or if we do so believe, then we have an ISSUE).
SEC OIT Uses for Risk Management
Initial project concept


ROM estimation – FY and TCO
ID most obvious risks
Selection of SDLC/PM style
Pre-Acquisition Review




Account for the 19 OMB risk elements
Derive risk score
Assign risk strategy
Allocate cost and schedule buffers
Control
Evaluate
Ranges of Estimates
PMBoK range estimates at
various points in SDLC
Budgeting process does not
recognize ranges
Conflicting interests:
80
60
40


20
Maintain
Deploy
Test & Train
Execution
-40
Planning
-20
Concept
0
Pad to avoid failure
Understate to avoid project
disapproval
Uncertainty = risk.
Recognition is 90% of the
battle
Buffers
Program-level unallocated
funds
Pre-Select: ROM Estimation
Need a number 24-48 months
out
Estimating cost of 18 servers
is easy – but why 18?
Software:




SLOC meaningless
nowadays
Function points imply design
work largely done
Can’t estimate from user
requirements – or can we?
What about re-use
SEC Directions



Establishing repository to
permit development of
parameters
Establishing EA maps to
functional components to
permit identification of re-use
opportunities
Provide a ROM estimate tool
for use in those “not a clue”
situations
 Will be refined over time
based on actuals

Seeking to work with other
agency estimation processes
ROM estimator
Update the cells highlighted in green. All sub-totals will be automatically calculated in the grey cells. The Rough Order of Magnitude total
will be calculated in the purple cell.
Rough Order of Magnitude (ROM) Cost Estimate
Investment:
Prepared By:
Test Project
As of:
J. Kluger
Select One
This project affects only one small Program
This project affects a large Program Office, but only
This project affects multiple Program
This project affects Regional/District Offices as well as the SEC
This project affects the SEC Enterprise (all users, all locations)
Storage Impact
Amount of Storage
Yes
N/A
Facility - Data Center
Size Required
N/A
Hardware ROM Cost
Hardware ROM
ROM + Complexity Factor
Hardware Buffer
+ Buffer
Software ROM Total
$73,750
$73,750
20%
$14,750
$88,500
Software ROM Cost
Software ROM
ROM + Complexity Factor
Software Buffer
+ Buffer
Software ROM Total
$6,500
$6,500
20%
$1,300
$7,800
Services ROM Cost
Services ROM
ROM + Complexity Factor
Services Buffer
+ Buffer
Services ROM Total
$200,040
$200,040
20%
$40,008
$240,048
Security
10%
Maintenance
28%
Steady State
25%
ROM Breakdown
Category
Hardware
Software
Services
Storage Impact
Facility
Security
Maintenance
Steady State
ROM Cost Estimate
Cost
$88,500
$7,800
$240,048
$0
$0
$33,635
$94,177
$84,087
$548,247
Hardware
Complexity
C
o
s
t
B
r
e
a
k
d
o
w
n
Yes
Easy
Yes
No
No
No
No
Software
Complexity
Yes
Easy
Qualitative data entry in green cells
- complexity
- scope
2/2/2005
Services
Complexity
Yes
Easy
Project Cost Breakdown (w/o Buffer & Steady State Costs)
Project Category
Cost
Hardware $73,750
Software
$6,500
Services $200,040
Storage Impact
$0
Facility
$0
Security
$8,025
Total
$280,290
Buffer Total
$56,058
Lifecycle Breakdown
Phase
%
Planning
0%
Analysis
25%
Solution
45%
Test/Training
10%
Deployment
20%
ROM Cost Est.
100%
Cost
$0
$70,073
$126,131
$28,029
$56,058
$280,290
Buffer
$0
$10,002
$31,165
$6,890
$8,002
$56,058
Total Cost through Deployment by Lifecycle Phase (Cost + Buffer)
Phase
Total
Cumulative
Planning
$0
$0
Analysis $80,075
$80,075
Decide how much
Solution $157,295
$237,370
you can accomplish
Test/Training $34,919
$272,288
and/or afford in this FY
Deployment $64,060
$336,348
Steady State Total (Security / Maintenance / Steady State)
$211,899
ROM Cost Estimate
Project Category
Buffer
$280,290
$56,058
Yes
No
Steady State
Total
$211,899
$548,247
N/A
N/A
Easy
Small
Moderate
Medium
Difficult
Large
Cost through deployment
Buffer
TCO ROM
Pre-Select: Concept Approval
ROM process identifies buffer (30-150% of
base)
PM can request less (usually acquisition-only)
Concept request identifies “most likely reasons
why project might fail” – bullet list
SDLC and PM Style as Risk Tools
3 SDLCs



Structured (waterfall)
Iterative (releases of functionality)
Acquisition-only (straight purchases) - assigned at time of preselect decision
3 PM Styles


PM-Lite
Custom PM: as needed, based on risk and complexity; Level 1
or Level 2 PM assigned
PM Levels – conform to Acquisition Workforce




Collateral duty
Level 1 – system supporting single SEC office
Level 2 – enterprise or complex functional system
Level 3 – multi-agency (no such project yet)
Select: Pre-Acquisition Review
19 Risk Elements assigned in OMB Circular A11





Some overlap but consistency has value
Each area assigned High, Medium or Low for
probability and impact
Positive outcomes also assessed, treated as HIGH
to protect them
Identify one or more risk statements per area to
explain
Explain AVOIDANCE plan for HIGH-HIGH, HIGHMED risks
Risk Assessment
Risk Category
Risk Level
Schedule: Risk that some projects in the program will
overrun current timeline
Score
Risk Category
L
1
L
1
overrun cost estimates for current request
will overrun future-year costs
L
1
Obsolescence: Risk that technology being implemented
1
L
Feasibility: Risk that some projects will run into
1
L
1
too much downtime or other discontinuity of service to fulfill
requirements
L
Interoperability: Risk that some projects will not integrate
1
well with rest of SEC technical environment
L
Asset Protection: Risk that some program products
(including data generated) will be lost or stolen.
1
L
Procurement Monopoly: Risk that approach will result in
vendors being “locked in” and able to raise prices or degrade
support over time with impunity
program requirements will substantially change during
implementation period
some projects will change during the course of the program
(including operational phase)
L
manage this program
L
+
=4
program will run into implementation issues due to
excessive complexity, or technical difficulty or failure
+ve: Project success = large benefits
–ve: Project failure = damage agency’s ability to do its
mission.
L
1
L
1
1
L
1
1
L
1
L
Security: Risk that some projects in the program could
1
create or run into significant security issues
Data Privacy: Risk that some projects may result in
L
exposure of protected personal data
Program Resources: Risk that most projects in the
1
SEC Oversight: Risk that the SEC will not effectively
=1
because affected divisions, offices, or external parties resist
the required behavioral change
Strategic Impact:
Reliability: Risk that delivered technology will experience
L
1
Technical Delivery: Risk that some projects in the
implementation issues due to unknown technology OR
business-side realities (right idea, wrong time)
=2
L
Data Stability: Risk that the nature of data used by the
will create future cost, integration, or support issues because
of aging/obsolescence
M
fail to achieve scope, cost or schedule
Fluid Requirements: Risk that some projects in the
Life Cycle Cost: Risk that some projects in the program
=4
Overall Failure Risk: Risk that the some projects will
Score
Organizational Change: Risk of program failure
Initial Cost: Risk that some projects in the program will
H
Risk Level
L
1
program will not receive the funding or staff participation
required for success
Total Risk Score
1
L
19
Assigning risk buffers
Pretty simple: risk score = expected buffer
Review board questions buffers that deviate from the
risk score
Last year, tried 1-4 as scale (minimum 19, maximum
76)
For 2006, adjustments:



Accommodating acquisition-only projects BUT experience that
most projects WERE under-estimated = revise buffer range to
5% to over 100%
Minimum score 4.75, maximum could be 161 – but highly
unlikely to approve project with 4-6 high risk elements
Introduction of pre-acquisition review (more detail available)
Future directions in measurement
Specify risk management approaches



HIGH = Avoid
Moderate (<9) = Mitigate, Transfer
Low (0.25) = Accept
Refine buffer calculations as data gained
Narrow the total size of buffer assigned by ROM
Reward PMs for declaring and returning buffer –
without encouraging padding to get reward
Reward contractors for early delivery under budget with
incentive-based contracts
Risk Management in Control Phase
Risk log for regular risks
Project dashboard



Performance light based on schedule and cost buffer
consumption
“Management attention” light for PM to declare need
for help (risk has become issue)
Customer satisfaction light for customer to sound
alarm
Re-evaluation at SDLC phase gates
Buffer Consumption
100%
0%
0%
100%
% complete
SDLC Phases and Associated Reviews
Initiation
Planning
Analysis
Design
Solution
Test
Train/Deploy
Steady State
Opns
Acquisitiononly
Structured
Release 1
Release 2
Iterative
Release N
Output
Business case,
Approval
Contract
Detailed
Technical
award
Business & Design
High-level
Technical
reqts
Reqt’s
Diamonds are mandatory go-forward milestones.
GOLD = Formal review; BLUE = sign-offs
Solution
OIT
Acceptance
Business
Value
Retirement
Evaluation Phase
Project Close-out Reports



Review issues
Compare to initial risk assessments
Gather actual cost and schedule data
Conduct 90-120 Day Operational Assessments
Recap: Uses for Risk Management
Initial project concept


ROM estimation – FY and TCO
ID most obvious risks
Selection of SDLC/PM style
Pre-Acquisition Review




Account for the 19 OMB risk elements
Derive risk score
Assign risk strategy
Allocate cost and schedule buffers
Control
Evaluate
?
?
Conclusions
Knowing the enemy and yourself
Those who are ignorant of history are doomed
to repeat it
Pride goeth before a fall
Pragmatic Uses for
Risk Management Practices
Douglas M. Brown, Ph.D., PMP
PMO Head
U.S. Securities and Exchange Commission
Office of Information Technology
202-551-8176
[email protected]
PMI Silver Spring Chapter
12 April 2006