Transcript Title

Life Cycle Cost Minimization & the Impact of
Early Implementation of Human Systems
Integration Requirements
Liana Algarín, Ph.D. Candidate, The George Washington University
Thomas A. Mazzuchi, D.Sc., The George Washington University
Shahram Sarkani, Ph.D., P.E., The George Washington University
Presented at Systems Engineering in DC
Chantilly, VA
April 3-5, 2014
Overview
1.
2.
3.
4.
5.
6.
7.
8.
9.
Problem Statement
Motivation
Life Cycle of an Acquisition Program
Human Systems Integration Requirements
Case Study: WIN-T
Training Burden
Methodology
Pilot Study
Conclusion & Future Research
Problem Statement

Incorporating Human Systems Integration (HSI) 
requirements earlier in the acquisition life cycle
may minimize cost. One way to predict the effect
that HSI requirements will have on life cycle cost
would be to identify HSI language within
acquisition documents. Regression analysis using
specific variables identified from the text of HSI
language is a potential method for making a
prediction about life cycle cost. The predictor

variables will be generally categorized by time
(when HSI language appeared in documentation),
training recommendation and component, and
specific HSI issue.



HSI has nine domains:
1.
2.
3.
4.
5.
6.
7.
8.
9.
Human Factors Engineering,
Environment,
Safety,
Occupational Health,
Habitability,
Survivability,
Manpower,
Personnel, &
Training
Project Manager (PM) Warfighter Information
Network-Tactical (WIN-T) is a tactical
communications network
“WIN-T is like the ‘Verizon’ of the Army.” Dr. Bev
Knapp, Army G-1, MANPRINT
WIN-T is an ubiquitous tactical communications
network that the Army will continue to implement
and improve during the coming years
The Army is committed to making WIN-T work
efficiently
Problem Statement
4
Motivation
 Primary Stakeholders
•
•
US Army’s Manpower Personnel Integration (MANPRINT) program
Human Research and Engineering Directorate (HRED), which is one of six directorates in
the US Army Research Laboratory (ARL)
 MANPRINT program mission is to cut costs and prevent soldier loss during
the system development life cycle
 MANPRINT’s goal is for Army’s Soldiers to experience less of a burden in
both training and long-term Operations and Maintenance (O&M)
 My goal is to deliver new analytics that would assess the benefits (or
costs) of implementing HSI early in the life cycle
 Stakeholders have been wanting to examine this idea, and they would like
to have insight on how WIN-T can/could have been improved
Acquisition Program Life Cycle






Major decision points occur during Milestones A, B, and C; some requirements are not
included at Milestone A but are later introduced during Milestone C
Schedule slippage and cost overruns are affected by long-term manpower planning
In addition to poor planning, poor project management strategy can cause a program to fail
to meet its objectives
When Systems Engineering is applied early in the life cycle, the Technology Development
Phase and Knowledge Points are benefited
Function allocation early in the life cycle helps a complex work system, such as WIN-T, to be
more effective
It is also beneficial to create a system value from stakeholders’ opinions and apply this system
value to the life cycle
HSI Requirements







Human Systems Integration (HSI) involves incorporating people with the systems
that they use to complete various tasks
When an acquisition program begins, there may not be a clear idea of what the
end user will need in the long-term
However, this is not a justification for entirely leaving out HSI requirements at
Milestone A
Usability defines how well a human can use a system; usability issues can be
anticipated early in the life cycle
Including HSI up-front at the life cycle planning stage helps to reduce long-term
costs by means of reducing the burden of training the workers
Early incorporation of HSI requirements and HSI methods can benefit a system or
program later in its life cycle
HSI requirements can be successfully incorporated when the stakeholders are
open to including HSI requirements and following HSI methods and when HSI
happens early in the life cycle
Case Study: WIN-T
 A case study with the Army’s tactical communications network, known as
Warfighter Information Network-Tactical (WIN-T), will illustrate how
there have been cost overruns as a result of integrating HSI requirements
late in the life cycle
 WIN-T will be used at locations where Soldiers must communicate with
other units
 There is no common interface across all of WIN-T’s components
•
•
•
WIN-T combines user interfaces from several individual systems
Combining these interfaces has created some confusion for the Soldier end user
Confusion has created a heavy training burden on Soldiers
 There are 4 WIN-T increments being completed/scheduled since 2004
onward
•
•
Each increment of WIN-T is expected to show improvements for usability
WIN-T Increment 2’s team is currently conducting usability analysis of WIN-T’s
components
Training Burden
 As more time is spent training soldiers to interact with and use WIN-T, cost
increases
 Some Military Occupational Specialties (MOS) have the appropriate
experience for working with the types of interfaces that are incorporated
with WIN-T
 However, Soldiers with different MOS may sometimes use WIN-T and find
it difficult to use, particularly due to the several user interfaces
Methodology
 Objective is to create new analytics to measure the extent to which early
implementation of HSI requirements will affect an acquisition program
 Data collection will be taken from source documents provided by
stakeholders at the MANPRINT and HRED offices
 Potential WIN-T Source Documents
•
•
•
•
•
•
•
•
•
•
•
Assessments (including MANPRINT Assessments)
Statements of Work
Analysis of Alternatives
Engineering Change Proposals
Emails
Test Reports
Meeting Minutes
Initial Capability Document
Capabilities Production Document
Capability Development Document
etc.
Pilot Study
 Using existing data from WIN-T Increments 1 & 2, we can make predictions
for WIN-T Increments 3 & 4
 Regression analysis is a statistical method that can be used for making a
general prediction based on previously made observations
 The planned analysis will be a regression analysis with one Dependent
Variable and several Predictor Variables
Pilot Study
 Dependent Variable
• Will be defined using MANPRINT assessment scores
 Predictor Variables
• Time: Date in month/year (number of months into WIN-T’s acquisition life
cycle)
• Time: If a document is released sometime after the events to which the
document refers, such as a report about a test from a few months ago, then
the date in month/year will indicate closest approximation to the appropriate
time in WIN-T’s life cycle
• Training: Recommendations for training improvement (by each WIN-T
component)
• HSI-Related Language: Previously selected HSI words/phrases
– Selected by Expert Elicitation – Asking stakeholders what words or phrases they
think are relevant
– Selected by Cluster Analysis (CA) – Do CA on a primary word that is synonymous
with HSI, such as a unique word related to any of the nine HSI domains
Conclusion & Future Research
 New analytics will be created to demonstrate that applying HSI
requirements earlier in the acquisition life cycle would be beneficial
 The method of looking for HSI language within documents and noting
when certain language was used can be repeated with any program
 Milestones and assessments are commonly used within systems
acquisition programs, particularly for military and weapons systems
 These new analytics could potentially be used to:
•
•
•
increase accuracy of schedule predictions,
minimize Life Cycle Costs for hours spent on training, and
minimize component usability issues related to HSI
Questions
References
Ahram, Tareq Z., and Waldemar Karwowski. "A Framework for Human Total Ownership Cost Based on
Universal Human Performance Cost Components." Paper presented at the Proceedings of the
Human Factors and Ergonomics Society 56th Annual Meeting, HFES 2012, October 22, 2012 October 26, 2012, Boston, MA, United states, 2012.
Boehm, Barry, Ricardo Valerdi, and Eric Honour. "The Roi of Systems Engineering: Some Quantitative
Results for Software-Intensive Systems." Systems Engineering 11, no. 3 (2008): 221-34.
Brinkman, W. P., R. Haakma, and D. G. Bouwhuis. "Component-Specific Usability Testing." IEEE
Transactions on Systems Man and Cybernetics Part a-Systems and Humans 38, no. 5 (Sep 2008):
1143-55.
Browning, Tyson R., and Eric C. Honour. "Measuring the Life-Cycle Value of Enduring Systems." Systems
Engineering 11, no. 3 (2008): 187-202.
Challenger, R., C. W. Clegg, and C. Shepherd. "Function Allocation in Complex Systems: Reframing an Old
Problem." Ergonomics 56, no. 7 (Jul 2013): 1051-69.
Chatterjee, Samprit, and Ali S. Hadi. Regression Analysis by Example. Wiley Series in Probability and
Statistics. Fifth ed. Hoboken, NJ: John Wiley & Sons, Inc., 2012.
Chung, Lawrence, and Kendra Cooper. "Defining Goals in a Cots-Aware Requirements Engineering
Approach." Systems Engineering 7, no. 1 (2004): 61-83.
Colombi, John M., Michael E. Miller, Michael Schneider, Major Jason McGrogan, Colonel David S. Long,
and John Plaga. "Predictive Mental Workload Modeling for Semiautonomous System Design:
Implications for Systems of Systems." Systems Engineering 15, no. 4 (2012): 448-60.
Dacus, Chad L. "Improving Acquisition Outcomes through Simple System Technology Readiness Metrics."
Defense ARJ 19, no. 4 (October 2012): 444–61.
Deitz, Daniel, Timothy J. Eveleigh, Thomas H. Holzer, and Shahryar Sarkani. "Improving Program Success
through Systems Engineering Tools in Pre-Milestone B Acquisition Phase." Defense ARJ 20, no. 3
(October 2013): 283–308.
Eigbe, Arekhandia Patrick, Brian J. Sauser, and John Boardman. "Soft Systems Analysis of the Unification
of Test and Evaluation and Program Management: A Study of a Federal Aviation
Administration's Strategy." Systems Engineering 13, no. 3 (2010): 298-310.
Hardman, Nicholas, and John Colombi. "An Empirical Methodology for Human Integration in the Se
Technical Processes." Systems Engineering 15, no. 2 (2012): 172-90.
Hart, C. G., Z. He, R. Sbragio, and N. Vlahopoulos. "An Advanced Cost Estimation Methodology for
Engineering Systems." Systems Engineering 15, no. 1 (2012): 28-40.
References
Horowitz, B. M., and J. H. Lambert. "Assembling Off-the-Shelf Components: 'Learn as You Go' Systems
Engineering." Systems, Man and Cybernetics, Part A: Systems and Humans, IEEE Transactions on
36, no. 2 (2006): 286-97.
Karwowski, W. "A Review of Human Factors Challenges of Complex Adaptive Systems: Discovering and
Understanding Chaos in Human Performance." Human Factors 54, no. 6 (Dec 2012): 983-95.
Knapp, Dr. Bev. "Army Manprint Perspective." Paper presented at the NDIA HS Workshop, Washington,
DC, 2012.
———. "Moving Manprint (Manpower Personnel Integration) Left (Mml)." 2013.
———. "Moving Manprint to the Left (Mml)." 41. Washington, DC, 2013.
Lattin, James M., J. Douglas Carroll, and Paul E. Green. Analyzing Multivariate Data. Belmont, CA:
Brooks/Cole, 2003.
Liu, Kevin K., Ricardo Valerdi, Donna H. Rhodes, Larry Kimm, and Alvis Headen. "The F119 Engine: A
Success Story of Human Systems Integration in Acquisition." Defense ARJ (April 2010).
Madni, Azad M. "Integrating Humans with Software and Systems: Technical Challenges and a Research
Agenda." Systems Engineering 13, no. 3 (2010): 232-45.
Marchant, Alan B. "Obstacles to the Flow of Requirements Verification." Systems Engineering 13, no. 1
(2010): 1-13.
Patanakul, Peerasit, and Aaron Shenhar. "Exploring the Concept of Value Creation in Program Planning
and Systems Engineering Processes." Systems Engineering 13, no. 4 (2010): 340-52.
Pflanz, Mark, Chris Yunker, Friedrich N. Wehrli, and Douglas Edwards. "Applying Early Systems
Engineering: Injecting Knowledge into the Capability Development Process." Defense ARJ 19, no.
4 (October 2012): 422–43.
Pharmer, James A. . "The Challenges and Opportunities of Implementing Human Systems Integration
into the Navy Acquisition Process." Defense ARJ (February 2007).
Sohoni, Milind G., Ellis L. Johnson, and T. Glenn Bailey. "Long-Range Reserve Crew Manpower Planning."
Manage. Sci. 50, no. 6 (2004): 724-39.
Tan, W. P., B. J. Sauser, J. E. Ramirez-Marquez, and R. B. Magnaye. "Multiobjective Optimization in
Multifunction Multicapability System Development Planning." IEEE Transactions on Systems
Man Cybernetics-Systems 43, no. 4 (Jul 2013): 785-800.
U. S. A. R. L. "Improved Performance Research Integration Tool." U. S. Army Research Laboratory,
http://www.arl.army.mil/www/default.cfm?page=445.
U. S. Army. "Manprint Program Mission." U. S. Army,
http://www.manprint.army.mil/manprint/about.html.
———. "Our Mission." U. S. Army, http://www.manprint.army.mil/directorate/about.html.
Biographies

Liana Algarín is working toward a Ph.D. in Engineering Management and Systems Engineering
at The George Washington University. She is also a Human Factors Engineer at Alion Science
and Technology, where she has designed user interfaces, training manuals, and user guides to
support the Army and NASA. She has also conducted a critical task and usability analyses of
Coast Guard cutters.

Thomas A. Mazzuchi, D.Sc., is Professor of Engineering Management and Systems
Engineering at The George Washington University. His research interests include reliability,
life testing design and inference, maintenance inspection policy analysis, and expert
judgment in risk analysis. He served as Research Mathematician at Royal Dutch Shell, and has
conducted research for the U.S. Air Force, Army, and Postal Service; for NASA, and for the
Port Authority of New Orleans, among others.

Shahram Sarkani, Ph.D., P.E., is Professor of Engineering Management and Systems
Engineering, and Director of EMSE Off-Campus Programs, at The George Washington
University. He designs and administers graduate programs that enroll over 1,000 students
across the U.S. and abroad. In over 150 technical publications and in sponsored research with
NASA, NIST, NSF, AID, and Departments of Interior, Navy, and Transportation, his research has
application to risk analysis, system safety, and reliability.