Transcript Slide 1

M&S Based System Development and
Testing in a Joint Net-Centric Environment
Bernard P. Zeigler, Ph.D.,
Co-Director,
Arizona Center for Integrative Modeling and Simulation
www.acims.arizona.edu
and
Joint Interoperability Test Command
Fort Huachuca, AZ 85613-7051
There is an acute need for a new Net-centric testing paradigm
at the enterprise level where joint and coalition operations are
conducted (Editorial, ITEA Jnl, Sept, 2005)
Net-Centric T&E places an increased emphasis on standards
conformance self-testing over the Global Information Grid
Service Oriented Architecture (GIG-SOA ) and emphasizes
services rather than systems
Simulation-based acquisition requires the use of M&S in all phases of the
system development life-cycle
Concept
Refinement
Modeling
and
Simulation
Technology
Development
System
Development
&
Demonstration
Systems
of
Systems
Production &
Deployment
especially starting early in system development
when defects can be caught and corrected with low
cost
Operations &
Support
Simulation-based development implies the need for simulation-based
testing
Specified
as
abstract
model,
e.g. UML
Connecting middle
ware, e.g. HLA
System Under
Test (SUT)
send/receive
messages
Test Device
send/receive
messages
Network
-- an infrastructure to test a system that is first formulated as an abstract
model is offered by distributed simulation
-- both the System under Test (SUT) and the test device are coupled by
exchanging data packets on a network
Raises the question: How is the Test Device developed in an authoritative manner?
Problem: Testing Polar Opposite Requirements
 To deal with the increasing complexity and advanced decision
capabilities of C4ISR systems
=>testing methodology has to become more
rigorous, in-depth and thorough
To keep up with the rapid change and short development life
cycles expected from the system builders
=> tests have to be ready to conduct in time
scales compatible with the agile development strategies of
new systems.
Solution: employ formal M&S
• to increase capabilities for simulation-based testing
and
•as a basis to increase the automation of testing processes.
Testing of interface standards is a focus area for automated
simulation-based testing.
Link-16 is required in all Joint and multi-national operations.
Theater
Warning
DSP/SBIRS
ABL
AWACS
JLENS
F-15
THAAD
TEL
PATRIOT
MEADS
SIS(MSCS)
SIS(MSCS)
AVENGER
ATACMS
AEGIS (CEP)
The Joint Interoperability Test Command (JITC) has developed
an automated test generation methodology as its core
technology for testing conformance of systems to Link-16
This methodology is fundamentally enabled by the DEVS
formalized modeling and simulation approach
Link-16
specification
Recent Successful Application
The JITC employed M&S-based testing for the initial major
milestone evaluation of the Integrated Architecture Behavior
Model (IABM) developed by the Joint Single Integrated Air
Picture (SIAP) System Engineering Organization (JSSEO) in
2005.
The test exercise produced significant results that uncovered
flaws in the model design and added acknowledged value to
the model development.
The ACTGen Development Team, NGIT & ACIMS was selected as
the winner in the Cross-Function category for the 2004/2005 M&S
Awards presented by the National Training Systems Association
(NTSA).
Types of Distributed Simulation Testing
Testing
Description
Example
standards
conformance
test whether system
conforms to
standard
supports
interoperability
Tactical Data Link
Standards,
Link-16, VMF,
USMTF,
interoperability
test whether
systems can
interoperate (at the
syntactic, higher,
levels)
Joint Translator
Forwarder (JXF) – air
to/from land exchange
of tactical data
mission/capabilities
test whether system Joint Close Air
of systems have
Support (JCAS)
capabilities required
for mission
Levels of Distributed Simulation Testing
Level/Variation
Description
Example
One-on-One
Stimulus/
Response
- managed time
- real time
Test Driver sends message stimuli to
which SUT responds
IABM SIAP
One-on-Many Federation
------------------------- managed time
- real time
Test Driver sends message stimuli
and controls supporting federates
including sensors
IABM Peer to Peer
Architecture
Multiplatform
(Many-to-Many
interaction)
------------------------- managed time
- real time
Active mode (test driver controls
scenario interaction)
JIT 06-03 live C2 systems
from USAF, USMC, Navy in
remote locations
Passive Mode (test manager
opportunistically observes
interaction
Joint Combined Hardwarein-the-Loop (HWIL) Event
(JCHE) Phase 3 CNI pilot
event “Runs for Record”
(~Aug 06)
(CNI = Navy Common
Network Interface)
JXF tests
Multiplatform Distributed Simulation
– controlled testing
Platform
(System,
Component)
Platform
(System,
Component)
Platform
(System,
Component)
Test Driver
Test Driver controls the scenario
Multiplatform Distributed Simulation
- uncontrolled testing
Platform
(System,
Component)
Observer
Platform
(System,
Component)
Observer
Platform
(System,
Component)
Observer
Test
Coordinator
Distributed Observers look for opportunities to test
Joint Test Enterprise Requirements
• Must respond to requirements for increasing complexity, evolving
from standards conformance, to interoperability, to joint operations
capabilities testing
• Top down test development methodology – aim for maximum reuse
of test assets by supporting configuration of assets to meet
objectives of each new exercise
• Requires support of appropriate concepts and levels of abstraction –
e.g., Objectives, capabilities, functionalities, rule sets, test model
federates
• Requires development of software at levels of abstraction with
appropriate flexibility to interface with configurable middleware and
hardware to meet conditions of tests, e.G., High speed/accuracy
implies real-time operating system replaces default windows OS
• Develop new assets when no configuration of existing assets can
satisfy objectives; then develop with reusability and fit within
methodology in mind
Joint Test Enterprise Methodology
• Determine objectives up front and drive test
configuration process
• Determine capabilities to be tested
• Develop scenarios/mission threads to involve SUT use
of identified capabilities
• Identify lower level functions supporting capabilities
• Decompose functions into relevant sub sets of rules
• Develop testable sequences of rules from relevant rule
sets
• Apply automated development of test sequences as test
model federates
Integrated Development and Testing Process
Reference Master Model
Standard Provides
Behavior
Requirements
at one or more
levels of System
Specification
Formalization by
mapping into
DEVS representations
Corresponding levels of
System Specification
Simulation based
testing in net-centric
environment
Semi-automated test
suite design based on
Experimental Frames
Real-time
implementation and
execution
Summary
• Interoperability is mandated throughout the system
development process to achieve mission effectiveness
• The emerging challenges of testing in a joint
environment require a new paradigm
• A test methodology at the enterprise level that scales to
net-centric services on the GIG is needed
• Automation of the generation of test environments is
needed – illustrated for tactical data link standards
conformance testing