Transcript SSC SD Briefing Template

Distributed Multimedia Systems 07
The Future of Visual and Iconic Language
Applications and Inscription Technology:
A New Way to Chat
LorRaine Tauchi Duffy, PhD
SPAWARSYSCEN San Diego 5361
[email protected]
Sept 6 2007
SSC San Diego … on Point and at the Center of C4ISR
SSC San Diego TEAM
• LorRaine Duffy, PhD. Code 5361, [email protected]
• Emily Wilson Medina, Code 2734, [email protected]
• Sunny Fugate. Code 2725,, [email protected]
• Omar Amezcua, Code 536204, [email protected]
• Gary Rogers, Code 536204, [email protected]
• Nghia Tran, Code 23711, [email protected]
• Hoa Phan, Code 2856, [email protected]
• Vincent Dinh, PhD. Code 2856, [email protected]
• Marion Ceruti, PhD. Code 536, [email protected]
• Erin O’Conner, San Diego State Student Foundation
SSC SD S&T Grand Challenge: A 5 year
commitment to technology development
2
DMS 07
Sept 06, 2007
SSC San Diego…on Point and at the Center of C4ISR
© SSC SD Duffy, LorRaine 09/01/2007
Tactical Situation Assessment (SA)
Background
• Based on text chat: Primary and often singular means
of communicating situation updates on intermittent/
discontinuous networks:
– Strikegroups: Establish hundreds of chatrooms, 2-4000 users in
each “channel, ”based on functional roles; establish at beginning
of mission, continuing for >18 months
– Joint/Coalition Forces: Establish chatrooms with joint/coalition
service participation, based on mission objectives; allows joint
access to service-specific chatrooms to maintain non-intrusive
situation awareness of service-specific activities
• Functions:
– Preserves interaction history
– Supports edge users (on low/intermittent b/w)
– Immediate COP (common operational picture) change and
context updates
– General information sharing updates on regular basis (across
months) to establish battle rhythm management;
– Supports Cross-Domain operations; those without the COP
3
DMS 07
Sept 06, 2007
SSC San Diego…on Point and at the Center of C4ISR
© SSC SD Duffy, LorRaine 09/01/2007
What Chat Looks Like Today
The 50’s Defense
Messaging (and Email):
R xxxxxxZ OCT 05
FM COMCARSTRKGRU TWELVE
TO USJFCOM
USPACOM
Produced Time-late
Example Chat Log:
ALLIED COMMAND TRANSFORMATION
AC2C4ISR LANGLEY VIRGINIA
Difficult to read
DISA WASHINGTON DC
Produced in real time
SPAWAR SYSTEMS CENTER SAN DIEGO C
NAVAL POSTGRADUATE SCHOOL MONTERY CA
Not used for real-time
ops, but as formality
USS ENTERPRISE
USS ANZIO
Still difficult to read
INFO COMMANDER NAVAL NETWORK WARFARE COMMAND
BT
MSGID/GENADMIN/COMCARSTKGRU TWELVE/-/OCT//
SUBJ/WELL DONE//
RMKS/1. I WISH TO SEND NOTE OF PERSONAL THANKS FOR YOUR
SUPPORT OF OUR JOINT/COALITION OPEN STANDARDS TACTICAL
CHAT DEMONSTRATION ON 19 OCT 05. YOUR ORGANIZATIONS
PROVIDED EXCEPTIONAL SUPPORT IN TERMS OF ENGINEERING,
EQUIPMENT TO HOST THE TEST, AND PERSONNEL TO PARTICPATE
AND PROVIDE FEEDBACK. THE TEST WENT EXCEEDINGLY WELL
AND WOULD NOT HAVE BEEN POSSIBLE WITHOUT YOUR ASSISTANCE
AND DEDICATION. SPECIAL APPRECIATION GOES TO:
4
DMS 07
Sept 06, 2007
SSC San Diego…on Point and at the Center of C4ISR
© SSC SD Duffy, LorRaine 09/01/2007
Challenge: Creating the S&T
•
•
Objective: Enable Rapid Situation Assessment Updates through mixture
(not engineering integration) of technologies (multi-disciplinary effort) in an
area that is relatively new, but ubiquitous (and not very well respected…)
What is chat going to look like in 10 years? A New “’leet speak”
– Must not be solely text-driven (too slow; too ambiguous); focus on visual and
iconic images and gestures, integrated with text, I-18-N (Internationalization)
– Must be able to accommodate edge (tactical) users
• Chemical-Biological Warfighters in MOPP (Mission Oriented Protective Posture) gear
• Special Operations in Inhospitable Settings: oil spills, fire, tsunamis, oxygen-deprived
environments
– Must be able to integrate with geographic land-based terrain maps AND nongeographic computer-based “terrain” (global network ops-network topology)
– Must be able to incorporate object relationships and time in providing more
precise context, mor complex information in smaller “space”
– Must be small (in bandwidth) and robust (survive network drops)
– Not necessarily keyboard-driven, point and click, nor restricted to PDAs
•
•
5
Embody the challenge of an NBIC (Nanotechnology/Biotechnology/
Information Systems/Cognitive Sciences) approach to development
Need to stay independent of commercial developments
DMS 07
Sept 06, 2007
SSC San Diego…on Point and at the Center of C4ISR
© SSC SD Duffy, LorRaine 09/01/2007
Technical Approach: 3 Development Areas
• Linguistic Research & Analysis
– Improve the ability to mathematically derive the content of
current chat messages, by defining candidate linguistic
“themes/words” of military language, for transition to an
augmented icon-based language, to improve efficiency of
knowledge sharing
• Visual and Iconic Language
– Develop an iconic/visual (gesture) language to augment text,
that can be used to more quickly communicate complex
relationships and the evolution of relationships of objects of
interest in geographic and non geographic environments; a true
situation assessment “leet speak”
• Technology of Inscription
– Prototype a keyboard-independent technology of inscription for
this new language, for use in a Net-Centric environments
6
DMS 07
Sept 06, 2007
SSC San Diego…on Point and at the Center of C4ISR
© SSC SD Duffy, LorRaine 09/01/2007
Linguistic Research & Analysis
• Analyzing chatroom text represents a subset of a
larger body of research : unstructured text analysis
– Internet documents, blogs, emails, chatrooms
• Military usage of chat is widespread, but inadequate
for typical information retrieval and management
• Statistical analysis of chat--its structure, topical
organization, and user trends--must first be applied,
few chatroom metrics exist
• Acquire sizeable corpus of chat content for chat user
database (domain specific/general; millions of lines of
chat are required)
• Statistically reduce content to significant words,
phrases, concepts (within domain and in general)
• Produce candidate list for iconic/visual representation
7
DMS 07
Sept 06, 2007
SSC San Diego…on Point and at the Center of C4ISR
© SSC SD Duffy, LorRaine 09/01/2007
Visual And Iconic Language
• Goal is to build a mixed-media situation assessment-specific chatbased lexicon (integrating words, pictures, gestures, with hardware),
based on linguistic analysis
• In order to:
– Increase information carrying capacity
– Increase expressiveness of the chat medium
– Decrease ambiguity by using a formal iconography and
shared context
– Increase the ability to reliably share the same context
• How can we best augment and disambiguate textual content using
visual representations of language, objects, and information?
8
DMS 07
Sept 06, 2007
SSC San Diego…on Point and at the Center of C4ISR
© SSC SD Duffy, LorRaine 09/01/2007
Visual and Iconic Language (cont)
Method:
• Gather knowledge of existing iconic language research,
development, & design, both military and non-military
• Develop software to support visual language
communication
• Develop icon-set based on linguistics results
• Expand visual language dictionary and insert standard
icon sets and acronyms
• Choose gesture set and develop a visual mapping to
items in the dictionary
• Test icon suitability
• Visual And Icon Language (VAIL) Conference, Aug 2-3,
2007: : http://cs.unm.edu/~vail
9
DMS 07
Sept 06, 2007
SSC San Diego…on Point and at the Center of C4ISR
© SSC SD Duffy, LorRaine 09/01/2007
Visual And Iconic Language
10
DMS 07
Sept 06, 2007
SSC San Diego…on Point and at the Center of C4ISR
© SSC SD Duffy, LorRaine 09/01/2007
Technology of Inscription
Prototype a keyboard-independent language-entry
technology for net-centric warfare
– Phase 1: Wireless data glove
– Phase 2 : Visual & Icon-based language
integrated/gesture-smart glove with 3-D modeling
(requiring knowledge of biomechanics)
– Phase 3: Miniaturization via nanotechnology-based
circuitry, novel data entry technology
• Keeping in mind that Nintendo Wii and Microsoft’s
“Table Interface” will dominate commercial
market
11
DMS 07
Sept 06, 2007
SSC San Diego…on Point and at the Center of C4ISR
© SSC SD Duffy, LorRaine 09/01/2007
Many Parallel Developments to Inscription
Technologies
Microsoft Surface
Table
© SSC SD 2006
Nintendo Wii
RONI GALGANO / Union-Tribune
Evan Morikawa, a senior at High Tech High School, was one of
four winners in the Greater San Diego Science & Engineering
Fair for creating a chording keyboard glove that sends wireless
signals to his personal digital assistant.
12
DMS 07
Sept 06, 2007
Minority Report © 20TH Century Fox
SSC San Diego…on Point and at the Center of C4ISR
© SSC SD Duffy, LorRaine 09/01/2007
Summary
• 3 Areas of Research Encompassing NBIC principles
• Multidisciplinary Team:
– computational linguist, software architect, computer
programming/science, electrical engineers, physicist,
psychologist, 3-D modeler, chemist, iconic artist…
• Long Range Applied Goals Drive Integrated Knowledge
for unique direction and outcome
– Not commercially driven, like business-based research
– Not long-range, single-focus science, as in academics
– Not a trivial problem, but a unique one…
13
DMS 07
Sept 06, 2007
SSC San Diego…on Point and at the Center of C4ISR
© SSC SD Duffy, LorRaine 09/01/2007