Transcript FEA SRM CBA

Meeting the NDAA 2010 Mandate
to establish an Agile IT Acquisition
Process
John Weiler
Technical Director, Interop. Clearinghouse www.ICHnet.org
Vice Chair, IT Acquisition Advisory Council www.IT-AAC.org
[email protected] 703.768.0400
Agenda
Understanding the IT Acquisition Challenge
Implementation Challenges
Business Drivers & Benefits
Enablers & Critical Success Factors
Recommendations for Transformation
Discussion
www.IT-AAC.org
2
The IT Acquisition Challenge
Wave 3 Technologies can’t be acquired using Wave 2 processes…
 We are in early stages of Wave 3 information technology
Information Driven Capability
 Mainframe and Client-Server waves remain in place
3. Internet - Cloud
• Virtualized compute; global
network enabled, plug & play
• IT Infrastructure decoupled
from Applications
• COTS & OSS Integration,
Software as a Service
 Waves represent many co-dependent
technologies, matured over time
 Adding functional capability has
become easier with each new wave
 But enterprise infrastructure 2.
gaps & vulnerabilities have
become more critical
Client/Server - Decentralized
• PC enabled and network
• Software distributed in both server and client
computers
• Heavy focus on software development
1. Centralized - Mainframe
• Central computer center, slow turn around
• One size fits all
• Limited reuse of application modules
1950
1960
1970
1980
1990
2000
2010
2020
Information Technology Evolution
DoD is using Wave 2 acquisition & budget processes; to acquire Wave 3 capability
www.IT-AAC.org
3
DoD IT Acquisition Challenge
Assuring Acquisition Outcomes
PM Challenges:
• How do requirements fold into Capabilities and align with IT
Services?
• Early identification of high requirements and over
specification?
• Making architecture relevant to PMs. How does architecture
align with the acquisition strategy?
• To what extent can current technologies and IT services satisfy
my capability gaps (service orientation)?
• How do I move from requirement to deployment in 360 days
without cutting corners?
DoD IT Acquisition Challenges:
• DoD 5000, DODAF, TRL & JCIDS designed for WS development
& not tuned for non-developmental items or fast paced IT
market
Acquisition Warriors circa post 2003
Making Sound Investment Decisions in a Complex, Fast
• Analytical Methods inconsistent and often produce
Paced Environment.
unpredictable errors
• AQ Risk aversion inhibits tailoring or speedy acquisition
• Time, cost of duplicative redundant decision making.
• Failure to define realm of the possible results in over
7/7/2015
www.IT-AAC.org
4
specification
and costly program lifecycles
New SOA/Cloud Computing Paradigm
S/W Development gives way to COTS/OSS Application Assembly
Yesterday
Design, Code & Test
• Focus is Software Development
• Code everything to spec
• Timeframes 12-24 months
SOA & Agile Acquisition
focuses on COTS
integration over
development
• Complexity and rate of
change manageable (CMM)
• Technology base Stable
• Driven by data model &
structured methods
Today
Architect, Acquire, Integrate
• Focus on Component Assembly &
Integration
• Model, Evaluate, & Acquire
• Timeframes are 12-24 weeks!
• Reliance on industry standards
• Rate of change is high and
accelerating
• Increased Agility & Adaptability of
Enterprise Systems
www.IT-AAC.org
5
Traditional DoD EA and 5000.2 Processes
Ineffective in coping with the fast paced IT market
DODAF 2.0
Traditional 5000.2 Process
Based on technology and
Typically waterfall – not iterative
standards (IDEF, UML)
Produces monolithic apps – not
Poor alignment of stakeholder
modular
views
No consistent COTS evaluation
No cross-agency or crossand acquisition process
application business process
Inhibits use of commercial best
refactoring
practices & SW artifacts
Assume custom S/W
Drives stove-pipe development
development w/ little COTS
Assumes 20 year lifecycle, and
No consistency enforcement of
control of all “components”
EA artifacts (inter- and intraJCIDS does not have a bottom up
agency)
Does not produce actionable or
comparable output
www.IT-AAC.org
6
IT Acquisition Reform Way Forward
NDAA 2010 Policy Guidance for IT Acquisition




Stream Lines the cost, schedule and performance reporting for major automated
information systems (MAIS), by providing guidance to DOD to designate MAISs that
also qualify as major defense acquisition programs as one or the other;
Authorizes the department to enter into agreements with private sector
organizations to arrange for the temporary assignment of DOD IT professionals to
the private sector, or for private sector IT professionals to be assigned to DOD
organizations. This capability reduces skill gaps in mission critical occupations by
accelerating learning of industry best practices through direct interactions;
Establishes more meaningful metrics for the acquisition of software-intensive
programs by amending the reporting requirement in law through the replacement of
references to “initial operational capability” and “full operational capability” with the
term “full deployment decision” in order to bring terminology more in line with
updated acquisition regulations; and
Requires the DOD to undertake business process re-engineering efforts before
approving the acquisition of new business systems. DOD would also be required to
undertake such business process re engineering analyses for business systems
already approved for acquisition and deployment.
www.IT-AAC.org
8
Agile Acquisitions CSFs and Enablers
Services Oriented Architecture




Way of thinking about systems as set of modular services:
business, data, infrastructure
A business driven, services oriented style
Focused on shared infrastructure services
Partitions capabilities and standardizes interfaces
COTS & Capability-based Acquisitions (CBA)






Approach to structuring enterprise
solutions that increases modularity and adaptability
Focus on outcomes and COTS
Drives use of Service Level Agreements
Required by Clinger Cohen Act
Facilitates alignment of business and technology
Consistent with Industry Best Practices
www.IT-AAC.org
9
Enablers and Critical Success Factors
for Agile Acquisition
Enablers




Technologies Exist to Enable SOA
Commercial components encourages by ATL
Standards & Best Practices Exist - Adopt them
Innovators and COTS providers provide best value
Critical Success Factors




Business Driven EA and AQ Approach (OMB)
Independent Architecture Development (DSB)
Revised Solution Development Lifecycle focused on
COTS acquisition/integration (AF SAB)
Mechanism for leverages existing COTS and
associated implementation best practices (CCA)
www.IT-AAC.org
10
Acquisition Assurance Method (AAM)
Decision Analytics for the Agile Acquisition
Streamlining and Assuring the IT Acquisition Processes through SOA
and Cloud Computing
Mitre Assessment of ICH Method “... the concept of the Interoperability Clearinghouse is sound and vital. Its developing
role as an honest broker of all interoperability technologies, no matter what the source, is especially needed. Such efforts
should be supported by any organization that wants to stop putting all of its money into maintaining archaic software and
obtuse data formats, and instead start focusing on bottom-line issues of productivity and cost-effective use of information
technology.”
www.IT-AAC.org
11
Agility requires Clear Touch points in the
Federal Acquisition Lifecycle
 Performance Measures, Objectives, Outcomes (PRM)
 Business Objectives (BRM)
 Funding, Partnering Strategies
Strategy
 Identify Best Practices, technology Enablers, and Components
 Existing Stake Holders, Business Processes, and Workflows
 Existing Delivery and Access Channels (Portfolio)
Market Rearch
 Must Have Functions, Features, and Info Exchanges
 Short and Long-Term Requirements
 Assessment of As-is state: Gap analysis
Requirements
 Define Component Relationships to BRM
 Wiring & Activity Diagrams, Component Arch,
Data Arch
 To-Be architecture ‘blueprints’
Architecture
Knowing What’s
Possible
Model the Business
Define the Gaps
Develop the
“Blueprints”
 Define/Align Service Components
Obtain Components
 Component Common Criteria, SLA
 Select COTS based on normalized EA vendor
submissions.
Acquisition
Integration
Iterative Development
Value-Based Releases
Understanding the
Business
Execution
 Prototype Solution
Architecture
 Verify ROI, business fit
 Validate Sequencing Plan
 Deploy
 Manage
 re-Baseline
Artifacts and Activities
www.IT-AAC.org
Assemble the
Components
Execute &
Deploy
12
AAM Aligns and Validates
AAM
Service
Component
Vendor
Solution
Templates
Reference Models
BRM
Business
Drivers &
Metrics
(BRM/OV)
Core Business
Mission Objectives
Business
Processes &
Infrastructure
BRM
Service
Appl Service
ComponentsComponents
& Metrics Layer 1
(SRM/SV) Infrastructure
Service Components
Layer N
BRM
Technical
Solution &
Metrics
(TV, TRM)
Application
Layer 1
Common
Infrastructure
Layer M
www.IT-AAC.org
Associated Metrics
Performance Metrics
Security Profiles
Effectiveness/Efficiency
Common Criteria
Interoperability,
Fit, Finish
Secure Solutions
SAIL Solution Frameworks
Aligns with business needs
User/Integrator
Best Practices
Business Driven Top Down
Business Needs with Technical Solutions
13
Acquisition Assurance Method (AAM)
Outcomes: Improved Analysis, Transparency & Timely Decision Making
The Result DBSAE will Achieve w/ CAM
How DBSAE will make it Work
• Reduce IT Delivery Time to the Warfighter
CAM
Guidebooks
• Reduce Non-essential Requirements
Acquisition
Assurance
Method
Standardized for:
•
•
•
Efficiency
Sustainability
Measurability
• Reduce Total Lifecycle Costs
PRODUCT NAME
Portal 1
Portal 2
Portal 4
Portal 10
Portal 11
Portal 5
Portal 15
Portal 8
Portal 6
Portal 9
Portal 12
Portal 3
Portal 13
Portal 7
Portal 14
Portal 16
Weight
0.10
2
1
2
1
3
4
2
3
1
3
3
3
3
3
3
5
0.03
1
2
5
2
3
2
2
3
3
3
4
4
3
1
5
5
0.16
1
1
1
2
1
2
3
2
4
3
4
3
4
5
4
5
0.08
1
5
2
1
4
2
3
3
2
3
5
4
4
5
5
5
0.08
1
1
3
2
4
2
3
3
4
3
4
4
4
5
5
5
0.16
1
1
2
2
1
3
1
3
3
3
4
4
5
4
5
5
0.08
1
2
2
3
3
2
3
2
4
3
3
4
5
4
4
5
0.16
2
1
1
3
1
2
3
3
2
3
3
5
5
5
5
5
0.10
1
2
3
1
4
2
3
3
3
3
3
4
5
5
5
5
0.05
2
2
3
3
3
4
3
3
3
3
3
4
3
4
4
5
1.31
1.58
2.00
2.01
2.30
2.46
2.55
2.76
2.88
3.00
3.59
3.90
4.32
4.39
4.51
5.00
Blue = Essential
Green = Desirable
Yellow = Less Desirable
Red = Undesirable
Decision-Quality
Results
Streamline the Business
System Acquisition Process
7/7/2015
Weighted Risk
Customer Service
System Monitoring and Reporting
Document/Records Management
Workflow
Content Management and Delivery
Search and Indexing
Collaboration
Knowledge Management
Scale: 1= lowest risk 5= highest risk
Personalization
Standardized
Mentored
How-to’s
Traceable
Transparent
ARCHITECTURE ASSESSMENT -Sorted
•
•
•
•
•
Compliance with Industry
Standards
A Proven Method:
• Reduce Time to Market
12 mo.
www.IT-AAC.org
24mo.
36 mo.
14
AAM Streamlines Acquisition Supply Chain
A Services Oriented Approach
www.IT-AAC.org
15
AAM Predictable Outcomes
Conflict free Mechanisms for SOA Common Services & Solution Assessment
A Standardized Architecture Method for:





Aligning DODAF and FEA-PMO structure
Capturing Business Process and Information Sharing
Driving SOA Enabled IT Acquisition Lifecycle
Assuring Services Integration and Contractor Mgt
Capability Prioritization w/ SLAs and metrics
Assessment Framework for:








Mitigate deployment risk.
CAM’s structure provide
decision quality data earlier
Concentrates on
Vetted Business Processes and Requirements
Capabilities and their
Assessing Vendor Service Components (COTS, GOTS, Open Source)
Importance to Mission
Discerning Technology Feasibility/Risk Assessment. Verify market
Formalizes the Prioritization
ability to perform
Process
Service Level Agreements
Access to Critical Expertise and Knowledge
Sources:

Biz Value to PMs
Architecture and Acquisition Processes that work
Healthcare Informatics Domain Expertise
Emerging technologies, innovations and open source markets
Industry best practices in IT Infrastructure and SOA
Stake Holder facilitation and outreach
www.IT-AAC.org
Greater Access to
Innovative Solutions
Innovative approach for
assuring successful
IT Acquisitions
16
Predictable Results
Repeatable, measurable, sustainable success
Navy: Assessment of AFLOAT Program
– CANES SOA & Security Strategy
Eliminated hi-risk Requirements
by 23%, $100Ms in potential
savings
USAF: Streamlined COTS Acquisition
Process. Applied to Server
Virtualization.
Established optimal arch with ROI
of 450% & $458 million savings
USAF: Procurement of E-FOIA
System using AAM
Completed AoA, BCA, AQ
Selection in just 4 months
USMC: AoA and BusCase for Cross
Domain, Thin Client Solutions
GSA: Financial Mgt System
consolidation using AAM.
BTA: Assessment of External DoD
Hosting Options using AAM
Greatly Exceeded Forecasted
Saving in both analysis and
acquisition
Moved FMS from OMB “red” to
“green”. Eliminated duplicative
investments that saved $200M
$300 million in potential savings
with minimal investment
BTA: Apply AAM to complete AoA and
BCA for Hosting and SOA
GPO: Developed Acquisition Strategy
for Future Digital System
JFCOM: MNIS Evaluation of
Alternatives for Cross Domain Solutions
Evaluated 100’s of Options in 90
Led to successful acquisition and
days, enabling stake holder buy in
implementation on time, on budget
and source selection.
and 80% cheaper than NARA
RMS
“We have put to practice the AF Solution Assessment Process (ASAP) at the Air Force
Communications Agency (AFCA) with some well documented success. It was
Reduced average cycle time and
cost of Analysis by 80%
developed with Interoperability Clearinghouse (ICH) and provides a structured and measurable IT assessment process with the agility to provide decision-quality assessments
ranging from quick-looks to more in-depth capability-focused technology assessments and lightweight business case analysis.”
General Mike Peterson, AF CIO
www.IT-AAC.org
17
Transformation Roadmap
Consolidated Recommendations from DSB,
IAC and AF SAB
www.IT-AAC.org
18
18
AAM Quality Controls
Project
Strategy
Capability
Analysis
Capability
Determination
Capability
Prioritization
Activities







Feasibility
Assessment
Economic
Analysis
Artifacts/Deliverables
Determine Sponsor and Stake Holder
representatives
Codify Business Problem statement
Validate Project Scope, Timeline, Outcomes
Collect and evaluate existing data from RFI
responses and other sources
Deliver Project POAM
Establish Stake Holder Agreement and
Success Criteria
Establish Measures of Effectiveness





Clear Problem Statement, Capability
Gaps
RFI Assessment
Realm of the Possible
Measures of Effectiveness
Other data as Price lists
Critical Success Factors
Entry Criteria



Exit Criteria - Outcomes
Initial Data collection
Initial identification of Capabilities
Business Needs & Gaps



0 months
1 month
Approval of Project Plan
Approval of Business Problem and
Outcome
Criteria: Adequacy of Capabilities or Plan
for correction
2 months
www.IT-AAC.org
3 months
4 months
19
19
AAM Quality Controls
Project
Strategy
Capability
Analysis
Capability
Determination
Capability
Prioritization
Activities
Economic
Analysis
Artifacts/Deliverables
Capture Problem Statement w/Sponsor
Establish Performance Measurements
Document Agency Services Baseline
Determine industry capabilities and metrics
Capture Function Capabilities
Determine level of granularity needed
Hold Requirements WGs w/Sponsor's Key
Stakeholders







Feasibility
Assessment


Publish Capability Analysis Report (CAR)
(Requirements and their Justification)
Work papers on:
– Justification of Requirements, &
Capabilities
– Problem Statement Validation
Critical Success Factors
Entry Criteria

0 months
Exit Criteria
Approved Project Plan and POAM
1 month

Approval of the CAR by the Functional Sponsor
Criteria: Adequacy of Capabilities or Plan for
correction
2 months
www.IT-AAC.org
3 months
4 months
20
20
AAM Quality Controls
Project
Strategy
Capability
Analysis
Capability
Determination
Capability
Prioritization
Activities







Feasibility
Assessment
Economic
Analysis
Artifacts/Deliverables

Refine Capabilities into Service Component
solution models (per OMB FEA-PMO)
Conduct Market Survey

Establish Service Component & Groupings
Review RFQ for adequacy of detail
If RFI responses lack depth or breadth, ICH
will conduct Industry Outreach and
Benchmarking
Construct Service Component Analysis Groups
PMO review
Analysis Group; Service Components reference
model mapping to capabilities/requirements
Work papers on:
– Results of the Market Survey
– Standards of Practice
– Industry Benchmarking Data
– Standardized Vocabulary for describing
service components and basis for
establishing SLAs (not in scope)
Critical Success Factors
Entry Criteria

0 months
Approved CAR (Validated Capabilities)
1 month
Exit Criteria

Approval of Service Component by the PM
Criteria: Adequate industry metrics or plan for
correction
2 months
www.IT-AAC.org
3 months
4 months
21
21
AAM Quality Controls
Project
Strategy
Capability
Analysis
Capability
Determination
Capability
Prioritization
Activities




Feasibility
Assessment
Economic
Analysis
Artifacts/Deliverables
Hold Functional WG w/Sponsor's Key
Stakeholders
Develop Prioritization Weighting Scale
Team Normalized weighting of the Service
Components
Document each weights rationale


Capability Prioritization Matrix
Work papers on:
– Service Component Prioritization Scale
– Rationale for each weight given
(traceability)
Critical Success Factors
Entry Criteria

0 months
Approved Analysis Groups, Service
Components and Standards of Practice
1 month
Exit Criteria

Approved Capability Prioritization Matrix
Criteria: Functional Sponsor Approval
2 months
www.IT-AAC.org
3 months
4 months
22
22
AAM Quality Controls
Project
Strategy
Capability
Analysis
Capability
Determination
Capability
Prioritization
Activities









Feasibility
Assessment
Economic
Analysis
Artifacts/Deliverables
Evaluate RFI Responses
Establish alternatives for the assessment
Establish Scoring WG team
Develop Scoring Plan
Score Alternatives + +
Perform Sensitivity Analysis on Scoring
Results
Analyze results
Review AoA date points
Present Results to Functional Sponsor May included Functional WG team


Analysis of Alternative (Compare New/Existing
Solutions against Prioritized Capability)
Work papers on:
– Scoring Plan
– Scoring Rationale
– Sensitivity analyses performed
– Technology Maturity Assessment
Critical Success Factors
Entry Criteria

0 months
Approved Capability Prioritization Matrix
1 month
Exit Criteria

Approval of Feasibility Assessment Report by DBSAE/ PMO
Criteria: (1) Assessment Team agreement on the scores.
(2) Reference material justifying scores
2 months
www.IT-AAC.org
3 months
4 months
23
23
AAM Quality Controls
3
Project
Strategy
Capability
Analysis
Capability
Determination
Capability
Prioritization
Activities
Setup Main Cost Model
Determine the quantities and time frame to
be Evaluate
 "Setup Sub-Models for direct, indirect &
migration cost + savings"
 Determine Model's elements related to ROI
 Determine the models for each alternative
 Collect Data industry data and assumptions
 Conduct TCO
 Review of Economic Analysis Results
Present Results to Functional Sponsor - May
included Functional WG team
 Develop Economic Analysis Report
Entry Criteria
0 months
Economic
Analysis
Artifacts/Deliverables



Feasibility
Assessment
Economic Analysis Report
Solution Architecture
Documented CCA compliance
Work papers on:
– Model Documentation
– Documentation of each Alternative
– Documentation on costs developed for
the Mode;
– Documentation of Industry Metric
determined
Critical Success Factors
Exit Criteria
Approved Feasibility Assessment, AoA
1 month





Approval of Economic Analysis Report by Functional Sponsor
Criteria: (1) Functional Sponsor Agreement
(2) Reference material justifying cost models
2 months
www.IT-AAC.org
3 months
4 months
DBSAE
24
Assessment