Transcript EVM: A Simple Concept

Earned Value
Management
Expectations
Presentation to the NDIA/ICPM
November 18, 2008
David G. Ahern
Director, Portfolio Systems Acquisition
Deputy Under Secretary of Defense (A&T)
Agenda
• Improving Contract Performance
• Program & Integrated Project
Management
• EVM Roles & Responsibilities
• EVM Issues
• Data-Related Concerns
• Current/Future Initiatives
How are DoD Contracts Performing?
Notional Data: For illustrative Purposes only
Improving Program Performance
lessons learned
REQUIREMENTS DEFINITION
M/S A
DAB
JCIDS
JROC
JROC
FSA
ICD
ICD
++
CONOPS
CONOPS
Draft
Draft CDD
CDD ++
CONOPS
CONOPS
Rev
Rev 00
1
Draft
Draft CDD
CDD ++
CONOPS
CONOPS
Rev
Rev 1..n
1..n
Final
Final CDD
CDD ++
CONOPS
CONOPS
3
2
4
5
6
SSA
TMA
TMA
AoA
AoA
Preferred
Preferred
System
System
Concept
Concept
System
System
Definition
Definition
(SDS)
(SDS)
System
System
Spec
Spec
Technology Development &
System Requirements Development
Maturity Assessment
& Technology Demonstration
Concept Refinement
Production
Production
Spec
Spec
RFP
RFP
System Development and Demonstration
Models &/or
Prototypes
Defense
Acq
System
ASR
ITR
Phase 0
SRR I
Phase I
AoA: Analysis of Alternatives
CDD: Capability Development Document
FSA: Functional Solution Analysis
RFP: Request For Proposal
SDS: System Design Specification
SRR II
SFR
Test
Articles
PDR
CDR
Phase II
Phase III
KEY:
TMA: Technology Maturity Assessment
DAB: Defense Acquisition Board
JROC: Joint Requirements Oversight Council
ICD: Initial Capabilities Document
SSA: Source Selection Authority
Technical
Underpinnings
Cost Risk
Assessment
Schedule Risk
Assessment
Technical Risk
Assessment
Solid Understanding of Requirements
Realistic Baseline
PARTNERING WITH INDUSTRY
INFORMATION VISIBILITY
$60,000
LRE Variance $
$50,000
$40,000
$30,000
UNFAVORABLE
$20,000
FAVORABLE
$10,000
$0
($10,000)
($20,000)
Overhead
Management Reserve
General & Administrative
Flight Control
Field Services
Cost of Money
Product Assurance
Configuration Mgmt
Undistributed Budget
Furnishings / Equipment
Program / Project Management
Loading / Payloads / Equipment
Services
Air Vehicle
Technology
Survivability
Information Systems
Testing & Evaluation
Crew Station / Systems
Maintenance Instructions
Logistical Support Analysis
Design to Cost / Life Cycle Cost
Wing
Other
Supportability / Support Equipment
Training
Avionics
Airframe
ECP / LLP
In Service Engineering
Data / Software - Handling & Management
Armament
Propulsion
Landing Gear
Instructions / Publications
Electrical & Electrical Systems
Fuselage
Flight Test
Repairability
Environmental
Penetration Aids
Targeting & Sensors
Communication / Navigation
Supplier / Supply Chain Mgmt
Systems Engineering / Integration
Collaborative
Process
Cockpit Mgmt, Controls, & Display
($30,000)
Hydraulics / Pneumatics / Mechanical
EXECUTION
Boeing Common WBS Elements
LRE in $,000
Improvement Activity
“Lean Six Sigma”
Continuous Improvement
lessons learned
BRIEF DATE: August 12, 2008
CONFIG. MGR: Steven Corum, AIR-4.2.3, (301) 757-7718
FILE NAME: ahern_ndia_brief_20080812.ppt
Burning
Platform
Cockpit
Chart
Leading
Indicators
Transform Data into Information
lessons learned
Assessment
Assessment
lessons learned
PLANNING
Draft
Draft
ICD
ICD
AoA
AoA
Plan
Plan
PROGRAM BASELINE
M/S B
DAB
M/S A’
DoD Program Management
TWO DISTINCT BUT RELATED AREAS OF FOCUS
Acquisition Program
Management
• Acquisition Planning
• APB
• Source Selection Plan
• Award Fee
• SAR/DAES
Integrated Project
Management (IPM)
• Project Status/CPI & SPI
• Estimate at Completion
• Staffing
• Tasks Completed
• Management Processes
• Risk Mitigation
• Technical Reviews
IPM/EVM Is Not Just CPI & SPI
BRIEF DATE: August 12, 2008
CONFIG. MGR: Steven Corum, AIR-4.2.3, (301) 757-7718
FILE NAME: ahern_ndia_brief_20080812.ppt
IPM
Cost
Schedule
Technical
Integrated Project Management
Earned Value Management
PLANNING
•
•
•
•
•
•
•
•
•
•
•
Contract Requirements
Source Selections
Award Fee Criteria
Program Item Descriptions (PID)
Management System
Assessments (MSAs)
Integrated Baseline Reviews
(IBRs)
Integrated Master Schedule
(IMS)
Cost Analysis Requirements
Description (CARD)
Risk Identification
Key Performance Indicators
(KPPs)
Resource Forecasts
EXECUTION
IPM
Cost
Schedule
Technical
• Technical Reviews (PDR, CDR,
etc.)
• Management System
Assessments (MSA)
• Integrated Baseline Reviews
(IBRs)
• Award Fee Determination
• Schedule Risk Assessments
(SRAs)
• Program Management Baseline
(PMB)
• Estimates at Completion (EACs)
• Risk Mitigation
• Program Performance
Measurement
• Burn Rate Trends
• Management Reserve Usage
• Staffing Plans
• Baseline Execution Evaluations
IPM = Cost, Schedule And Technical Integration
“The Application of Basic Project Management Principles”
BRIEF DATE: August 12, 2008
CONFIG. MGR: Steven Corum, AIR-4.2.3, (301) 757-7718
FILE NAME: ahern_ndia_brief_20080812.ppt
EVM Roles and Responsibilities
• Program Offices
–
–
–
–
Get the correct EVM requirements on contract
Use the EVM information to manage
Take action based on EVM information
Hold suppliers accountable for providing good information
• Defense Contract Management Agency (DCMA)
– Review EVMS plans; verify initial and continuing system compliance
• Conduct EVMS reviews
• Conduct ongoing EVMS surveillance
– Work with government program offices and suppliers
• Suppliers
–
–
–
–
Use EVM as a management tool
Implement and maintain a compliant EVMS
Provide good EVM information to the government
Ensure proper flow down to subcontractors
EVM roles and responsibilities clearly delineate authority
and accountability for monitoring use of EVM
EVM Requirements
• Issue: EVM requirements incorrectly placed on contract
– EVM not being used as management tool; perception EVM is just a financial
report
– Failure to include applicable EVM requirements
– Inappropriately modifying EVM requirements; incorrectly tailoring DIDs for CPR
and IMS
– Specifying contract requirements not consistent with EVM policy and EVMS
guidelines
– Use of EVM not considered in contract type selection decisions
– Disallowing use of contractor’s EVMS on contracts not requiring reporting
• Action: Ensure EVM requirements are identified and incorporated
into solicitations and contracts, and are then properly executed
– Work closely with program manager and EVM community throughout acquisition
process
– Involve EVM subject matter experts in establishing and implementing contract
requirements (statement of work (SOW), request for proposals (RFP), CDRLs)
– Consider EVM implications when selecting contract type
– Discuss requirements in pre- and post-award conferences or similar forums
– Include EVM on agenda at contracting meetings and conferences
– Ensure the “right” EVM training is provided to contracting community
It’s important to get the EVM requirements right up front –
fixing problems later is more painful … and costly
Contract Incentives
• Issue: Contract incentives that counter EVM’s objectives
– Contracting policies can work at cross purpose with EVM
– Use of award fee incentives that drive undesired behavior –
incentivizing achievement of good numbers rather than good
performance
– Result is PMs are denied objective performance information on their
programs
• Action: Find better ways to incentivize contractors (better use of
fees)
– Avoid use of cost and schedule indices (CPI and SPI) as only EVMrelated award fee criteria
– Work with EVM community to devise objective, yet effective, contract
incentives
– Consult EVMIG for ideas on appropriate EVM-related incentives (e.g.,
realistic and current cost and schedule forecasts)
Paying contractors to maintain cost and schedule indices
within predetermined boundaries encourages undesirable behavior
Contractor EVM Systems
• Issue: Deficiencies in contractor EVM systems
–
–
–
–
–
–
–
–
–
Lack of knowledge by control account managers
Span of control issues
Inadequate schedules
Data integration and integrity problems
Undisciplined work authorization and change control processes
Poor variance analyses
No formal corrective action processes
Inadequate estimate at completion procedures
EVM used to report data, not to manage
• Action: Contact DCMA for help with contractor EVMS problems
– Work with local DCMA EVM specialist and/or the EVM Center
– Implement appropriate remedial actions for EVMS non-compliance
Based on findings from recent DCMA EVMS compliance reviews
Baseline Planning/Execution
• Issue: Lack of timely, realistic, and executable Performance
Measurement Baseline (PMB)
– Conduct of IBRs impacted by delays in defining work scope
– Contractual direction to defer baseline planning until definitization
– Incongruities between SOW and PMB (e.g., undefinitized contract
actions)
– Lag time in incorporating changes into baseline
• Action: Know, understand, and implement ways to improve
timeliness and quality of baseline planning and execution
– Ensure IBRs are conducted within 180 days after contract award, and
after exercise of significant options or major modifications
– Stop direction to delay baseline planning until after contract
definitization
– Incorporate authorized unpriced work into PMB – detail plan near-term
effort
– Work with EVM community to ensure integrity of work scope and PMB –
achievable contract schedule and proposal estimate
Establishing an executable baseline is a critical
prerequisite to the successful implementation of EVM
Things To Watch Out For
•
Direction in SOW or H-clauses that contradict policy (DFARS)
or conflict with supplier’s validated system (EVMS guidelines)
•
Contract changes by individuals other than warranted
contracting officers
•
Use of EVM cost and schedule indices as fee incentives
•
Retroactive changes to baseline except to correct errors
•
Single point adjustments to baseline that eliminate cost and/or
schedule variances
•
Improper contract direction on use of management reserve
•
Reallocation of budgets to mask overruns
•
Failure to apply EVM to “discrete” work on LOE contracts
Contracting officers can help improve EVM compliance
Leading Indicators –
Contractor Staffing And Task Completion
MACRO ANALYSIS
MICRO ANALYSIS
Month 48
1400
Full-Tim e Equivalents (FTE s)
1200
1000
800
600
INCOMPLETE WORK
400
Year 2
Year 3
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36
Aw ard
Feb-06
M ar-06
Apr-06
M ay-06
Jun-06
Jul-06
Aug-06
Sep-06
O ct-06
N ov-06
D ec-06
Jan-07
Feb-07
M ar-07
Apr-07
M ay-07
Jun-07
Jul-07
Aug-07
Sep-07
O ct-07
N ov-07
D ec-07
Jan-08
Feb-08
M ar-08
Apr-08
M ay-08
Jun-08
Jul-08
A ug-08
Sep-08
O ct-08
N ov-08
D ec-08
Jan-09
Month 47
Month 46
Month 45
Year 1
Month 44
Month 43
Month 42
Month 41
Month 40
Month 39
Month 38
Month 37
Month 36
Month 35
Month 34
Month 33
Year 4
Month 32
Month 31
Month 30
Month 29
Month 28
Month 27
Month 26
Month 25
Month 24
Month 23
Month 22
Month 21
Year 3
Month 20
Month 19
Month 18
Month 17
Month 16
Month 15
Month 14
Month 13
Month 12
Month 11
Month 10
Month 9
Year 2
Month 8
Month 7
Month 6
Month 5
Month 4
Month 3
Month 2
Award
Year 1
800
700
600
500
400
300
200
100
0
-100
-200
-300
-400
-500
-600
-700
-800
-900
-1000
-1100
-1200
UNDERSTAFFED
* equivalent of 3 man-months
per person
Planned Staff
Actual S taff
C ontracted S taff
N ext six m onths staffing plan
Last M onth's six m onth staffing plan
Actual m an-m onths over or under plan
C um ulative m an-m onths over or under plan
200
0
B a s e lin e E x e c u tio n
400
S t ill In c o m p le te
F in is h O ve r O n e M o n t h L a te
F in is h le s s t h a n O n e M o n t h L a t e
F in is h O n Tim e
F u t u re t a s k p la n n e d
F in is h E a rly
350
-200
STAFFING SHORTFALL
300
-400
INCOMPLETE WORK
Nu m b er o f T asks
250
-600
200
150
-800
* >40% of delinquent
tasks not yet started
-1000
-1200
100
50
LAUNCHED THREE ANALYSIS
ACTIVITIES
–
–
–
SCHEDULE CAPABILITY
SKILLS STAFFING
ENVIRONMENT
SUBCONTRACTING
ENVIRONMENT
8
8
-0
b
e
F
7
7
-0
an
J
7
0
-0
-0
v
ec
o
N
D
7
7
-0
tc
O
7
0
-0
l-
p
g
u
e
S
A
7
7
-0
y -0
n
u
a
u
p
INITIAL RESULTS INDICATE FEW CONTRACTORS HAVE THE CAPABILITY TO DEVELOP AND
MAINTAIN A HEALTHY INTEGRATED MASTER SCHEDULE.
PRIMES WHO SUBCONTRACT LARGE PORTIONS OF WORK EXPERIENCE GREATER SCHEDULE
DEVELOPMENT & MAINTENANCE DIFFICULTY, PARTICULARLY IF WORK IS SUBBED USING FFP.
PRIMES ARE EXPERIENCING SKILLS SHORTAGES AND OVERAGES – MOST ARE INCAPABLE OF
ACCURATELY PREDICTING OR FULFILLING SKILLS DEMANDS FOR +1 MONTH.
SUBCONTRACTED WORK IS INCREASING; UP TO 7-FOLD INCREASES IN BUY DECISIONS OVER
PRIOR EFFORTS, UP TO 3-FOLD INCREASES IN SKILLS SUBCONTRACTING FOR DESIGN EFFORTS.
Skills Staffing And Task Completion Inextricably Linked
BRIEF DATE: August 12, 2008
CONFIG. MGR: Steven Corum, AIR-4.2.3, (301) 757-7718
FILE NAME: ahern_ndia_brief_20080812.ppt
J
J
M
7
7
7
0
r -0
-0
a
e
b
r-
A
F
M
6
6
6
7
-0
J
an
-0
v
o
ec
D
6
-0
0
O
c
t-
-0
p
g
u
Program C Cumulative Understaffed
e
Program C Incomplete Tasks
Program B Cumulative Understaffed
S
Program B Incomplete Tasks
Program A Cumulative Understaffed
A
Program A Incomplete Tasks
N
6
0
-0
-1400
Enterprise Data Quality Dimensions
EV / Funding Data Quality
Funding Data Quality: Acceptable for Critical Measurements and Decision-making
EVM Data Quality: Unacceptable for Critical Measurements and Decision-making
EVM Data Integrity
Current/Future Initiatives
•
Outreach to contracting and program management communities
•
Contractor EVMS reviews; new DCMA EVMS surveillance
process
•
EVM “trip wire” metrics; other data analysis efforts
•
Implementation of EVM Central Repository on all major
programs
•
Assessments of program compliance with EVM reporting
requirements
•
Review of EVM training across the curriculum
•
New program start-up workshops/expanded post-award
conferences
•
Review of EVM policy and practices
•
Over Target Baseline/Over Target Schedule Handbook
Many initiatives are ongoing to address the issues
and challenges, but we need your help