Transcript Slide 1

Manufacturing Readiness Assessment
Acquisition Training
Gary Stanley
Manufacturing Technology Division
AFRL/RXMT Phone # 937-904-4398
[email protected]
Public Release Case # 88ABW-2008-0329
1
Defense Policy Insertion Plans:
Policy Targets
• DoD documents
1) Defense Acquisition Guidebook
2) DoDI 5000.2
3) DoDD 5000.1
• Air Force documents
– AFI 63-1201, “Disciplined Systems Engineering”
– Secy of the Air Force for Acquisition Policy Memo
– AFRL Advanced Technology Demonstrations
• Program Baseline Development
• AFMC Instruction 61-102, ATD Tech Transition Planning
2
What Will The Policy Look Like?
• MRLs are linked very closely with TRLs
• MRAs will be performed prior to each Milestone
Decision
–
–
–
–
MRL 1
M/S A – MRL 4
M/S B – MRL 6
M/S C – MRL 8
Full Rate Production – MRL 9
MRL 2
Basic mfg
Mfg
implications concepts
identified identified
MRL 3
MRL 4
MRL 5
MRL 6
MRL 7
Mfg proof Capability to Capability to Capability to
produce a
of concept produce the
produce
prototype
developed technology in
prototype
system or
a laboratory components
subsystem in
environment
in a
production a production
relevant
relevant
environment environment
A
Capability to
produce
systems,
subsystems or
components
in a
production
representative
environment
B
MRL 8
MRL 9
MRL 10
Pilot line
Low rate
Full rate
capability
production
production
demonstrated. demonstrated. demonstrated
Ready to
Capability in
and lean
begin low rate place to begin
production
production
full rate
practices in
production
place
C
3
PRR and MRA
• The areas evaluated in a PRR and MRA process are
nearly the same
• Major differences
– Timing - PRRs usually occur in the SDD phase versus MRAs
that occur throughout the Acquisition and S&T Phases
– PRR will evaluate the total program's readiness to proceed into
production vs MRA focusing on MRL ratings and MMPs
– PRR focuses on a total program risk assessment versus MRA
providing an objective score on the manufacturing maturity of the
program and how to achieve required MRLs
• Bottom Line – if you have done a PRR, you can do an
MRA
– Requires same skill base
4
Manufacturing Readiness Assessment
Acquisition Training
Jim Morgan
Manufacturing Technology Division
AFRL/RXMT Phone # 937-904-4600
[email protected]
Public Release Case # 88ABW-2008-0329
5
Session Outline
• What is a Manufacturing Readiness Assessment
(MRA)?
• Why Manufacturing Readiness?
• What are Manufacturing Readiness Levels (MRLs) and
how do they pertain to the Acquisition Life Cycle?
• How to do an MRA
• Sample Outputs and Deliverables
• Finding and Conclusions
• Additional Information
6
What Is A Manufacturing Readiness
Assessment?
• An MRA is
– An Assessment of a Program’s Readiness to Manufacture and
Produce Its Intended Design
– A Tool to Develop and Implement • Manufacturing Risk Mitigation Plans
• Business Strategies
– Effects of Design Changes (Planned Upgrades, Spiral)
– Pricing Agreements (Long Term vs. Single Lot)
– Capital Investment Plans (Contractor and/or Government)
• An MRA
– Assigns Manufacturing Readiness Levels (MRLs) to Key System
Components
– Analogous to Technology Readiness Levels (TRLs)
7
Session Outline
• What is a Manufacturing Readiness Assessment (MRA)?
• Why Manufacturing Readiness?
• What are Manufacturing Readiness Levels (MRLs) and
how do they pertain to the Acquisition Life Cycle?
• How to do an MRA
• Sample Outputs and Deliverables
• Finding and Conclusions
• Additional Information
8
Why Manufacturing Readiness?
Manufacturing & Industrial Base Challenge
• Consensus among Congress, OSD, CSAF, GAO:
“Advanced weapon systems cost too much, take too long to field, and are
too expensive to sustain”
• GAO study of 54 weapons programs:
– Core set of 26 programs: RDT&E costs up by 42% and schedule
slipped by 20%
• $42.7B total cost growth
• 2.5 years slip on average
– Characteristics of successful programs (GAO):
• Mature technologies, stable designs, production processes in control
• S&T organization responsible for maturing technologies, rather than
program or product development manager
• To mitigate impact of diminishing manufacturing infrastructure
– People, policy, programs gutted
– Lost recipe on how to manage manufacturing risk
– Won’t get infrastructure back but still need to manage
manufacturing risk
9
Why Manufacturing Readiness?
Acquisition Health and Manufacturing Readiness
Manufacturing risk/maturity is not the only
cost/schedule/performance driver, but we need
to manage manufacturing readiness integral to
the overall acquisition process
 Products made by immature manufacturing
processes generally:
-
Cost more
Are prone to quality problems
May not all perform the same
Are less reliable in service
Have a hard time delivering on schedule
10
Technology Readiness Levels
(TRLs)
Provide a common language and widely-understood
standard for:
• Assessing the performance maturity of a technology and
plans for its future maturation
• Understanding the level of performance risk in trying to
transition the technology into a weapon system application
TRLs leave major transition questions unanswered:
•
•
•
•
Is the technology producible?
What will these cost in production?
Can these be made in a production environment?
Are key materials and components available?
11
Session Outline
• What is a Manufacturing Readiness Assessment (MRA)?
• Why Manufacturing Readiness?
• What are Manufacturing Readiness Levels (MRLs) and
how do they pertain to the Acquisition Life Cycle?
• How to do an MRA
• Sample Outputs and Deliverables
• Finding and Conclusions
• Additional Information
12
Manufacturing Readiness Levels
(MRL)
• Common language and standard for
– Assessing the manufacturing maturity of a technology or
product and plans for its future maturation
– Understanding the level of manufacturing risk in trying to
produce a weapon system or transition the technology into a
weapon system application
• Designed to help set the agenda for manufacturing risk
mitigation
13
MRL Relationships
Relationship to System Acquisition Milestones
Pre-Concept Refinement
Concept
Refinement
System
Development &
Demonstration
Technology
Development
A
B
Production &
Deployment
C
MRL 1
MRL 2
MRL 3
MRL 4
MRL 5
MRL 6
MRL 7
MRL 8
MRL 9
MRL 10
Basic Mfg
Implications
Identified
Mfg
Concepts
Identified
Mfg
Proof of
Concept
Developed
Manufacturing
Processes
In Lab
Environment
Components
In Production
Relevant
Environment
System or
Subsystem
In Production
Relevant
Environment
System or
Subsystem
In Production
Representative
Environment
Pilot Line
Demonstrated
Ready for
LRIP
LRIP
Demonstrated
Ready for
FRP
FRP
Demonstrated
Lean Production
Practices in
place
TRL 1
Basic
Principles
Observed
TRL 2
TRL 3
Proof
of
Concept
TRL 4
TRL 5
TRL 6
Concept
Formulation
Breadboard
in
Lab
Breadboard
in Rep
Environment
Prototype
in Rep
Environment
TRL 8
System
Qual
TRL 9
Mission
Proven
TRL 7
Prototype
in Ops
Environment
Relationship to Technology Readiness Levels
14
MRL Definitions
MRL 1
MRL 2
Basic mfg
Mfg
implications concepts
identified identified
MRL 3
MRL 4
MRL 5
MRL 6
MRL 7
Mfg proof Capability to Capability to Capability to
produce a
of concept produce the
produce
prototype
developed technology in
prototype
system or
a laboratory components
subsystem in
environment
in a
production a production
relevant
relevant
environment environment
A
Capability to
produce
systems,
subsystems or
components
in a
production
representative
environment
B
MRL 8
MRL 9
MRL 10
Pilot line
Low rate
Full rate
capability
production
production
demonstrated. demonstrated. demonstrated
Ready to
Capability in
and lean
begin low rate place to begin
production
production
full rate
practices in
production
place
C
• Production relevant environment – An environment normally found during MRL 5 and 6 that contains key
elements of production realism not normally found in the laboratory environment (e.g. uses production
personnel, materials or equipment or tooling, or process steps, or work instructions, stated cycle time, etc.).
May occur in a laboratory or model shop if key elements or production realism are added.
• Production representative environment – An environment normally found during MRL 7 (probably on the
manufacturing floor) that contains most of the key elements (tooling, equipment, temperature, cleanliness,
lighting, personnel skill levels, materials, work instructions, etc) that will be present in the shop floor production
areas where low rate production will eventually take place.
• Pilot line environment – An environment normally found during MRL 8 in a manufacturing floor production area
that incorporates all of the key elements (equipment, personnel skill levels, materials, components, work
instructions, tooling, etc.) required to produce production configuration items, subsystems or systems that meet
design requirements in low rate production. To the maximum extent practical, the pilot line should utilize rate
production processes.
15
9 MRL Evaluation Criteria
(“Threads”)
1. Technology and Industrial Base
• Technology maturity, technology transition to production, ManTech development
2. Design
• Producibility program, design maturity
3. Cost and Funding
• Production cost knowledge (cost modeling), cost analysis, mfg investment budget
4. Materials (raw matls, components, subassys, subsystems)
• Maturity, availability, supply chain management, special handling
5. Process Capability and Control
• Modeling & Simulation (product & process), mfg process maturity, process yields/rates
6. Quality Management, to include supplier quality
7. Manufacturing Personnel, to include specialization, training, &
certification
8. Facilities, to include capacity and plant layout & design
9. Manufacturing Management
• Manufacturing planning and scheduling
• Materials planning
• Tooling and special test equipment
Milestone “ BA ”
Key Manufacturing Considerations
• Industrial Base capabilities identified for key technologies and key
processes
• Producibility & Manufacturability assessment of design concepts
completed
• Establishment/validation of manufacturing capability and management of
manufacturing risk for the product lifecycle
• Initial Key Performance Parameters (KPPs) identified
• Producibility cost risks assessed
• Survey completed to determine if materials have been used before
• Lead times identified for all materials
• Survey completed for potential supply chain sources
• Special handling requirements identified
• Survey completed to determine the current state of proposed processes
• Yield and Rates assessed on proposed processes
• Quality strategy developed
• Manufacturing skill sets identified
• Specialized facility requirements/needs identified
• Special Tooling/Special Test Equipment (STE) requirements are
17
considered
Milestone “ BB ”
Key Manufacturing Considerations
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
Industrial capability in place to support manufacturing of development articles
Required manufacturing technology development solutions demonstrated in a production relevant
environment
Producibility assessments of key technologies/components and producibility trade studies completed
Key Characteristics and tolerances established
Lead times have been identified for all materials
Cost model inputs include design requirements, material specifications, tolerances, integrated master
schedule, results of system/subsystem simulations and production relevant demonstrations
Material maturity verified through technology demonstration articles
Availability issues addressed to meet technology demonstration articles
Supply chain plans in place
Plans to address special handling requirements complete
Initial simulation models developed at the technology, sub-system or system level
Manufacturing processes demonstrated in production relevant environment
Yields and Rates evaluated in production relevant environment
Initial Quality Plan and Quality Management System is in place
Manufacturing workforce skills available for production in a relevant environment
Manufacturing facility and facility development plans adequate to support SDD or Technology insertion
Manufacturing risk mitigation approach for SDD or Technology insertion Programs defined.
Most material decisions made (make/buy), material risk identified and plans made to mitigate
Prototype tooling concepts demonstrated in relevant manufacturing environment.
18
Milestone “ CC ”
Key Manufacturing Considerations
•
•
•
•
•
•
•
•
•
•
•
•
Industrial Capability Assessment (ICA) for MS C has been completed. Industrial capability
is in place to support LRIP.
Required manufacturing technology solutions validated on a pilot line
Known producibility issues have been resolved and pose no significant risk for LRIP
Detailed design of product features and interfaces is complete
Major product design features are sufficiently stable such that key LRIP manufacturing
processes will be representative of those used in FRP
Engineering cost model driven by detailed design and validated with data from relevant
environment
Cost analysis of proposed changes to requirements or configuration
Yields and Rates evaluated in production relevant environment
Materials proven and validated on System Demonstration and Development (SDD)
production as adequate to support LRIP
Long Lead procurement initiated for LRIP. Availability issues pose no significant risk for
LRIP
Most material decisions made (make/buy), material risk identified and plans made to
mitigate
Prototype tooling concepts demonstrated in relevant mfg environment
19
Manufacturing Considerations for
Full Rate Production Decision (MRL 9)
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
Industrial capability is in place to support start of FRP
Producibility issues/risks discovered in LRIP have been mitigated and pose no significant
risk for FRP
Known producibility issues have been resolved and pose no significant risk for LRIP
Major product design features are stable and LRIP produced items are proven in product
testing
Major product design features are sufficiently stable such that key LRIP manufacturing
processes will be representative of those used in FRP
Variability experiments conducted to show FRP impact and potential for continuous
improvement
Program has budget estimate for lean implementation during FRP
Special handling procedures demonstrated in LRIP
Manufacturing processes & procedures are established and controlled in production to 3sigma or other appropriate quality level
Yield and rate targets achieved, yield improvements on-going
Quality targets verified on production line
FRP personnel requirements identified
Capacity plans adequate to support FRP decision
All manufacturing risks have been validated and mitigated using LRIP articles
All tooling, test and inspection equipment proven on LRIP
20
Session Outline
• What is a Manufacturing Readiness Assessment (MRA)?
• Why Manufacturing Readiness?
• What are Manufacturing Readiness Levels (MRLs) and
how do they pertain to the Acquisition Life Cycle?
• How to do an MRA
• Sample Outputs and Deliverables
• Finding and Conclusions
• Additional Information
21
MR Assessment Process
• Assessment Lead Briefs PM on Manufacturing
Assessment Efforts/Expectations
• Works with PM to
• Determine appropriate level for Manufacturing Readiness
Assessment(s) (MRAs) -- System may contain several critical
technologies/components/manufacturing cells
• Schedule on-site MRA with contractor(s)
• Send Orientation Package to contractor(s)
• Define Assessment Team Membership
• Define Deliverables of Assessment Results
• Conduct on-site assessment with contractor(s)
• Deliver final report/briefing
22
Manufacturing Readiness Implementation
Approach (ACATs)
Meet with Wing/Program
Management Team
And Other Stakeholders
INTRODUCE
TRAIN
Define Objectives
- Yield Improvement
- New Variant (e.g. Spiral)
- Increased Capacity (Surge)
OBJECTIVE
STATEMENT
DEFINED
ASSESS
INCORPORATE
MANAGE
Decompose the Problem Space
- By Technology (i.e. Component)
- By Supplier
- Handle Assembly & Test
Wing/PM Team owns
the plan
23
Preparations
• Contact Defense Contract Management Agency (DCMA) to
gather information on the contractor’s current and past
performance.
• Notify companies and send orientation package
–
–
–
–
Purpose, approach, questions, strawman agenda
MRL definitions/threads
Address contract issues if any
Self-Assessment
• Select Assessment team(s)
– Typically 2-6 members per team
– Appropriate members (include Gov’t customer)
• Specialists for key technologies (if needed)
• Schedule On-site assessments
– Months prior to key milestone decisions to establish a baseline and allow
time to develop/implement risk mitigation plans
• Team Orientation–Meet prior to on-site assessment
24
On-site MRA Process Review
• Contractor welcome, review of agenda and orientation to facility
• Introduction of assessment team and contractor personnel
• Government team lead briefing to contractor describing objectives
and expectations for the on-site visit
• Contractor overview and discussion of the results of their selfassessment
• Shop-floor visits to key areas by individuals or small groups
• One-on-one or small group discussions between assessment team
members and contractor subject matter experts focused on key
areas
• Private meeting of Government assessment team to:
–
–
–
–
–
Prepare feedback and identify any action items
Initial assessment of current MRL (their area or overall)
Key strengths/risks/issues
Key missing data (if any)
Proposed action items
• Outbriefing by Government team to contractor
25
Example Process Flow
Generic Aircraft
• Large programs can require multiple MRAs
Deliver to Facility B
Composite
Fabrication
Install
Cables
Composite
Assembly
Cable
Fabrication
Install
Avionics
INS
Delivery
Install
Oil System
Install
Fuselage
Fuel System
Assemble
Tails
Engine
Delivery
Landing Gear
Build-up
Engine Build-up
Deliver to Facility C
Initial
Power-up
Checks
System
Tests
Install
Wings/Tails
Final
Inspection
Move A/C
To Test
Install
Engine Cowls
Assemble
Wings
Install
Power plant
Install
Landing Gear
Install
Brake System
Colors represent supplier/facility
location
26
Supplier MRA Plan
• Identify and prioritize critical suppliers
• Develop common SOW for distribution to suppliers
– Scope of MRA detailed
– Method of MRA detailed
– Output defined
• Developed detailed MRA execution plan with each
supplier (Schedule, format, personnel)
• Execute MRA
• Define/Plan/Execute MRL mitigations
• Measure mitigation effectiveness, update assessment
27
Follow-on Activities
• Gather any key missing data
• Convene team meeting -- Typically within 2 weeks of
on-site assessment
– Discuss and finalize assessment
– Examine current program and manufacturing risk reduction
plans
– Agree on likely MRL at completion of milestone if current
plan is followed
• Share results with contractor
• Identify the specific risk reduction activities necessary
to reach the next milestone
• Identify the funding, time-phasing and approach to
carrying out each activity
• Prepare and submit final report
28
Session Outline
• What is a Manufacturing Readiness Assessment (MRA)?
• Why Manufacturing Readiness?
• What are Manufacturing Readiness Levels (MRLs) and
how do they pertain to the Acquisition Life Cycle?
• How to do an MRA
• Sample Outputs and Deliverables
• Finding and Conclusions
• Additional Information
29
MRA Process Outputs
• Baseline Manufacturing Readiness Levels (MRL)
• Key Factors Where Manufacturing Readiness Falls Short Of Target
– Define MRL Driving Issues
• Identify Programs And Plans To Reach Target MRL
– Identify Existing Investments and Additional Needs
– Summarize Improvement In Manufacturing Plan
• Assess Risk to Manufacturing Cost, Schedule, and Performance
• Implement and Execute the Manufacturing Plan
• Assess Effectiveness Of the Manufacturing Plan
–
–
–
–
Address Right Issues?
Timely? Adequately Funded?
Probability Of Success?
Options For Increased Effectiveness
30
SAMPLE SUMMARY ROLL-UP
OF COMPONENTS
SubSystem
Top Level
MRL
Observations
Most Critical
Guidance
3
- Lacking detailed process information
- Key suppliers identified; Need key
performance parameters
- Need detailed process plans
Detector from supplier A
- Design & production issues
- No alternate source
Data
Processor
3
- New processor architecture
-Immature design tools
- New attachment processes needed
Board Supplier can’t test at their
site
Low yields on initial run
Propulsion
6
- Same as other systems in use
- New component scheme
Re-validate manufacturing
process
Supplier handle increased rate
Air
Vehicle
7
- Same supplier as system X
- Need to test new mating and assembly
processes at the prime
No critical items
Test Plan
6
Several instances of re-design work and
new test processes
- New test strategy and plan
- What will new design
incorporate?
- Manufacturing experience vital
31
SAMPLE SUMMARY
(Drill down)
Guidance
Sub
systems
Top Level
MRL
Observations
Most Critical
Front End
Sensor
3
- Lacking details on builds
- Process procedures need more work
- Test and assembly procedures have not been
verified in manufacturing environment
Detector from supplier A
- Design & production issues
- No alternate source
Data
Processing
PWB
3
- New processor architecture
-Awaiting Design for Manufacturing and
Assembly (DFMA) results
Low yields on initial build
Working process controls
Looking at re-design for easier
fabrication
Cables For:
Power
Data
3
- Using same suppliers other weapon systems
- Have not received prototypes, awaiting
supplier delivery
Re-validate manufacturing process
as seen on past programs
Need new process plan
Housing
4
- New supplier: limited experience
- Need new assembly processes at the prime
Need supplier management
process; need new process plans
Cooling
3
- Form, fit factors for new cooling design not
in place
Initial process plan for build in place
Final cooling plan will be defined
after front end is stable
Integration
Process that
includes
assembly
and test
3
- Several new test processes need
development for new components
New test strategy and plan
New special test equipment must
be ordered
32
MRA Risk Management
• Assessing Risk is independent of the MRL value
assigned
– Higher MRL value may be highest risk
• Eg. Requires new equipment, high cost process
• Risk Assessment should consider
– Time needed to reach target MRL
– Require new personnel, training, capital, or more POM samples
to flush out the process
– Leverage other programs
– Captive or Merchant Supplier Dependency??
– Part of a company’s core business
• Leads into an industrial base assessment
• Effective of use of Design for Manufacturing Tools and
other simulation techniques.
33
Session Outline
• What is a Manufacturing Readiness Assessment (MRA)?
• Why Manufacturing Readiness?
• What are Manufacturing Readiness Levels (MRLs) and
how do they pertain to the Acquisition Life Cycle?
• How to do an MRA
• Sample Outputs and Deliverables
• Finding and Conclusions
• Additional Information
34
AFRL/RXM MRA Deskbook
• The “how-to” of MRAs
• First draft completed in March 07
• Modeled after TRA Deskbook
– Similarities
• Achieving levels of readiness for risk reduction
• Selection process for assessment areas
– Differences
• Readiness in S&T and Acquisition world
• Rigorous assessment process
• Draft revised based on lessons learned from Reaper
MRA
– Dec 07, Public releasable, on DAU website
35
Some MRL Thoughts
• MRLs are not a report card
– MRL 7 might not be good
– MRL 3 might not be bad
• MRLs are a tool to manage and mitigate
manufacturing risk
– A common language used to assess manufacturing
maturity
– Provide insight not oversight
36
Some MRA Lessons Learned
• Process is more effective if company is actively
engaged in the assessment
• System integration and test operations are often
ripe for maturation efforts
• Resources required to conduct an MRA will vary
significantly
• Subject matter expertise is needed to “do it right”
37
Findings and Conclusions
• Looking at transitioning technology to production
– Must incentivize good decision processes;
– Unlike TRLs, going backwards on MRLs might be a good thing
• A low MRL number may be ok
– Is there time to raise the level?
– Is there a new manufacturing process being pursued?
– Replacing a manual process with an automated process
• Encouraging repeatability, faster cycle time, etc.
• Identify opportunities to validate manufacturing processes
– Avoid accepting analogous process claims during the design
phase and claiming fabrication is maturing
• May never build enough units to reach MRL 10
– Achieve a six sigma or equivalent process
– Stable Line, may require a multi-product factory
38
MRA Thoughts
• MRA process highlights areas needing attention
to lower production risk
• Easy Tracking for Prime contractor and
Government as manufacturing proceeds
• Detailed analysis rank ordered; Can be an
investment strategy
• Accomplished the goal as an acquisition test
case
Manufacturing maturity through the MRA process
enables efficient, cost effective manufacturing
39
Session Outline
• What is a Manufacturing Readiness Assessment (MRA)?
• Why Manufacturing Readiness?
• What are Manufacturing Readiness Levels (MRLs) and
how do they pertain to the Acquisition Life Cycle?
• How to do an MRA
• Sample Outputs and Deliverables
• Finding and Conclusions
• Additional Information
40
Additional Information
DoD MRL web site
http://www.dodmrl.com
- MRA link to DAU CoP
- Contains MRL definitions, MRL criteria matrix, MRA Deskbook
and more
TRA Deskbook
http://www.dod.mil/ddre/doc/tra_deskbook.pdf
- MRLs contained in Appendix I
DAU PQM Community of Practice
https://acc.dau.mil/pqm
- Manufacturing Readiness folder
•
•
•
•
Look for MR definitions
Look for MR matrix
Look for MRL tutorial
Look for MRA Deskbook
41