Transcript Developing a data-driven Perception Inference Engine (PIE) for Test & Evaluation of autonomous systems DoD 2015 “Taking the Pentagon to the People” HBCU/MI Technical.

Developing a data-driven
Perception Inference Engine (PIE)
for Test & Evaluation of autonomous systems
DoD 2015 “Taking the Pentagon to the People”
HBCU/MI Technical Assistance Training
Greensboro, NC
8 June 2015
Ali Karimoddini, PhD
Autonomous Cooperative Control of Emergent Systems of Systems (ACCESS) Lab, Director
TECHLAV Center, Deputy Director and leader of Research Thrust 2
Department of Electrical and Computer Engineering
North Carolina A&T State University
1601 E. Market Street/524 McNair Hall
Greensboro, NC 27411
Email: [email protected]
Website: http://eceserver.ncat.edu/akarimod/
Office: 336-285-3847
Fax: 336-334-7716North Carolina Agricultural and Technical State University
Explore. Discover. Become.
Focused on the mission of the Test Resource Management Center (TRMC) to address
T&E needs of Department of Defense (DoD), ACIT Institute has developed a novel datadriven technique for test and evaluation of autonomous systems using an advance fuzzy
expert system.
Remark: The views and conclusions being discussed here are those of the panelist and
should not be interpreted as necessarily representing the official policies or endorsements,
either expressed or implied, of DoD, TRMC, or the U.S. Government.
North Carolina
Agricultural and Technical State University
A. Karimoddini , Developing a data-driven Perception Inference Engine (PIE) for T&E of autonomous systems
2
A sad moment …
On October 28, 2014, the first stage of an Antares rocket on an unmanned resupply
mission carrying Cygnus CRS Orb-3 failed catastrophically six seconds after liftoff from
Mid-Atlantic Regional Spaceport at Wallops Flight Facility, Virginia. The flight termination
system was activated just before the rocket hit the ground, but an explosion and fire
destroyed the vehicle and cargo. There were no casualties, and Launch Pad 0A
escaped significant damage.
North Carolina
Agricultural and Technical State University
A. Karimoddini , Developing a data-driven Perception Inference Engine (PIE) for T&E of autonomous systems
3
Motivations
• Testing of Unmanned Systems is
required
for
the
Military
Departments to be able to certify
compliance with regulations and
demonstrate safe operations.
• Unmanned Systems must meet
the same requirements of a
manned systems that is
intended to be put into service.
Challenge: Testing unmanned
systems in general is a significant
challenge and can be very costly.
North Carolina
Agricultural and Technical State University
A. Karimoddini , Developing a data-driven Perception Inference Engine (PIE) for T&E of autonomous systems
4
What is a right key?
Problem
Testing
Software
Testing simple
systems
Solution
Set of
experiments
Software model
checking
Pros
Simplicity
Powerful for
testing software
Cons
Not scalable and
not expandable
for complex
systems
Specific to software
and difficult to be
used for hardware
testing
Model based
algorithmic testing
Testing complex
systems
Formal
verifcation
Data driven
techniques
Guarantee the
performance
Not applicable to
complex systems
with unmodelled
behaviors
Capture complexity
and unmodelled
behaviors
Only valid for the
trained range
North Carolina
Agricultural and Technical State University
A. Karimoddini , Developing a data-driven Perception Inference Engine (PIE) for T&E of autonomous systems
5
Sources of complexity:
Cyber-Physical nature
Cyber-physical systems (CPS) are engineered systems with tight combination
of (large number of) interacting computational systems and physical processes.
Control
North Carolina
Agricultural and Technical State University
A. Karimoddini , Developing a data-driven Perception Inference Engine (PIE) for T&E of autonomous systems
6
Project goals:
Project goals: Developing a Data-driven Perception Inference
Engine (PIE) tool to
1- Infer the internal states of the system from external
observations only
2- Evaluate intelligent systems from a cognitive
perspective
3- Predict behavior and evaluate the performance of
increasingly intelligent systems
4- Capture the dynamic, non-deterministic, uncertain
behavior of intelligent, autonomous systems
North Carolina
Agricultural and Technical State University
A. Karimoddini , Developing a data-driven Perception Inference Engine (PIE) for T&E of autonomous systems
7
Assumptions / Challenges
Assumptions: Testers may have only limited knowledge of
the internal states of the system under test, but can
externally observe the behavior of the system
Challenges: How to infer the internal states and dynamics
of the system from only external observations and how to
use this information to evaluate the performance of the
system.
North Carolina
Agricultural and Technical State University
A. Karimoddini , Developing a data-driven Perception Inference Engine (PIE) for T&E of autonomous systems
8
Our approach
Approach: Creating a tool to enable users to predict the
system’s perception, decision-making, and behaviors by
integrating
• Type 2 Fuzzy Logic System (FLS)
We use Type 2 FLS due to its unique capabilities in handling
uncertainty and capturing unmodelled emerging behaviors of the
system and environment.
• Learning Classifier Systems (LCS)
We use LCS as a capable machine learning technique to
synthesize the data base to form the knwoledge base.
North Carolina
Agricultural and Technical State University
A. Karimoddini , Developing a data-driven Perception Inference Engine (PIE) for T&E of autonomous systems
9
General Structure of the Fuzzy Type-2 Based PIE
PIE
Decision making
and planning Computer
LCS
T2FLS
Actuators
Fuzzifier
Inference
System
Adjustment
Sensing unit
Rule Generator
(Knowledge base)
Rule Base
Fuzzifier
Teleop Unit
Defuzzifier
Output
Process
Type
Reducer
Command
Center
Matrix Translation of
Fuzzy Rules
North Carolina
Agricultural and Technical State University
A. Karimoddini , Developing a data-driven Perception Inference Engine (PIE) for T&E of autonomous systems
10
Our T&E Team at ACIT Institute
Principle Investigators:
Dr. A. Karimoddini
Dr. A. Homaifar
Research Associates:
Daniel Opoku
Graduate Students
Nnamdi J. Enyinna
Alejandro White
Muhammad Sohail
Undergraduate Students
Evan Olney
Billy Whitehead Emmanuel Arzate Nicholas Donald Michael Lowe
Vin K
North Carolina
Agricultural and Technical State University
A. Karimoddini , Developing a data-driven Perception Inference Engine (PIE) for T&E of autonomous systems
11
Acknowledgment
Thanks to
• Test Resources Management Center (TRMC)
•
Scientific Research Corporation (SRC)
for supporting the NC A&T project on developing a T&E tool for testing and evaluation
of unmanned systems.
North Carolina
Agricultural and Technical State University
A. Karimoddini , Developing a data-driven Perception Inference Engine (PIE) for T&E of autonomous systems
12
Q&A
North Carolina
Agricultural and Technical State University
A. Karimoddini , Developing a data-driven Perception Inference Engine (PIE) for T&E of autonomous systems
13