Transcript ВИРТУАЛЬНЫЕ МИРЫ В

Interactive and immersive technologies:
research and experience of application
Saint-Petersburg State University of Aerospace Instrumentation
Dept. of Computing Systems and Networks
Laboratory of computer graphics and virtual reality
A.A.Nikitin, A.V.Nikitin,
A.A.Nikitina, N.N.Reshetnikova
Research and custom software
development projects
Fields of application
• Culture
• Cultural tourism
• Art
• Education
• Science
• Industry
• Health and Medicine
• Fitness
• Entertainment
• Advertising
• Smart City
• and other
Directions
 Interactive 3D models of objects
and events with contextual
information.
 Interactive and immersive 3/4/5/6 D
representations of objects and
events.
Technology
• Interactive and Immersive 3D
• Virtual Reality/Worlds
• Augmented reality
•
Augmented virtuality
•
Multiuser VW/MR
 Modern guides (navigators)
presenting historical cultural
information about surrounding
objects.
•
Collaborative VW/MR
•
Multimodal interfaces
•
Tangible interface
 Serious massively multiplayer
online games with elements of
geopositioning and augmented
(mixed) reality.
• Internet of Things
•
Simulators
 Interactive electronic technical
manuals, 3/4D passports.
•
Desktop, mobile, wearable and
surface projection systems
•
and other
• Electronic educational resources.
• and other
• Adaptive interface
Areas of research and experiments
•
•
•
•
•
•
•
•
•
•
•
•
Interactive and immersive 3/4/5/6 D representation of events and objects
Augmented reality
o Addition and combination of different sensations - visual, auditory, tactile, etc
o Use markers, images, geo-location, in- and outdoor, single- and multi-user
Augmented virtuality
Collaborative VW/MR
o Visual, voice, text and nonverbal communications in collaborative work
o Autonomous characters with intelligent behavior
Multimodal interfaces
o Combination of different modalities
o Use of human physiological parameters - pulse, blood pressure and others.
Tangible interface - person interacts with the VW/ MR through real-world objects
Internet of Things
Adaptive interface
o Changes in the properties of the VW/ MR (presentation, navigation), depending on the user's behavior
o Personalized interface
Simulators - bicycle, car, plane, etc.
Serious games, massively multiplayer online games with elements of geopositioning and augmented reality
Desktop, mobile, wearable and surface projection systems
Devices - 3D Vision, HMD, trackers, gloves, multiscreen, bicycle, etc.
Basic Interactive and Immersive Technologies
•
Virtual reality/worlds (VR/VW)
o immersion into artificial environment
o movement in a world with 6DOF and interaction with its objects in real time
o modeling of human behavior based on real-world physics and artificial
intelligence
o usage of various characteristics of a person - physical, sensorimotor, etc.
•
Mixed reality (MR) - augmented reality/virtuality (AR/AV)
o addition of human perception of real or virtual world respectively by virtual or
real objects
•
Multiuser VW/MR (Collaborative VW/MR)
o each visitor is represented by special network object-avatar and see other users
o actions performed by users in virtual world are synchronized and visible to
everyone
•
Multimodal interfaces
o support of communication with VW/ MR using more than one modality such as
voice, gesture, touch, movement of the head and body, etc.
Interactive and Immersive Theatre
Purpose
Creation of models’ database for theatrical
stages and Russian spectacles to support
preservation and promotion of Theatre Arts
Team
Alexandrinsky Theater,
State Theatre Arts Academy, ITMO, SUAI
•
•
•
•
•
•
•
•
•
Solutions
i3D model of stage and auditorium
i3/4D model of "Masquerade“ spectacle
Interaction with the model based on
gestures
Augmented reality component shows
information about recognizable points of
interest (single/multi-image)
Usage
Previsualization of stage and spectacle
Scientific historical reconstruction
Performance and / or games in mixed
reality
Teaching and the creative process
Printed materials based on augmented
reality, an appearance of artists and an
audience.
Historical reconstructions of Yam Fortress
Purpose
• Preservation and promotion of historical
information about the lost object of cultural
heritage
• Resource for economic and social
development of the region (cultural tourism)
Team
Museum Agency of the Leningrad Region,
Historical museum of Kingisepp , SUAI,
Institute of History of Material Culture RAS,
IBRD + FIBP
Solutions
• i3D models of fortresses and warriors
(XIY- XYIII centuries)
• Interaction with the model based on
gestures
• Augmented reality component shows
information about recognizable points of
interest (single/multi-image)
• 2D multimedia information system
Usage
• Virtual audio-visual excursions
• Extension of museum exhibitions
• Historical game in mixed reality
• Historical research, educational and artistic
reconstruction
• i3D Theatre for 20 persons
Personality and History of Yam Fortress
State Museum of the History of Religion
Purpose
• Demonstration of religious buildings
of cultural heritage
• Cultural tourism
Solutions
• i3D model of the Temple of Apollo
• i3D Church of the Holy Sepulcher
• Multimedia information system
Usage
• Extension of museum exhibitions
o Religion of Ancient Greece
(Delphi)
o Christian relics on the Holy Land
• Virtual audio-visual excursions
• Historical research and educational
reconstruction
• Free walk
• Quiz
• Versions - electronic kiosk and CD
ROM
Russian Ilizarov Scientific Center
for Restorative Traumatology and Orthopaedics
Purpose
• Advertisement of Center
• Innovation in medicine
• Conservation of Ilizarovov‘s heritage
• Entertainment for hospital patients
• Guide for the Center (Medical tourism)
Solutions
• i3D model of buildings (exterior and interior)
and surrounding area
• i3D Ilizarov apparatus, human organs, etc.
• Augmented reality component shows
information about recognizable points of
interest (single/multi-marker)
• Multimedia information systems
Usage
• Virtual audio-visual excursions and free walk
• Extension of Center’s museum exhibitions
• “Augmented Reality in Medicine” application
o Image-Guided surgery interventions
o Books based on augmented reality
• Promotional and educational materials
Industrial i3D Vizualisation,
Interactive Electronic Technical Manuals
Purpose
• Virtual prototyping and simulation
• Information resources for training and
maintenance
Solutions
• i3D model of the buildings (exterior and
interior) and the surrounding area
• I3/4D model of complex technical objects
• Augmented reality component shows
information about recognizable points of
interest (single/multi-image)
• Multimedia information systems
Possibilities of usage
• Advertising campaigns
• After-sales service - cars, ships, planes,
buildings, etc
o Maintenance
o Repair
o Assembly Work Instructions
o Operating Procedures
• Training
• Extension of corporate museums exhibitions
• Promotional and educational materials
• Corporate museums
Russian as foreign language
Cybernetic Bicycle
for training, entertainment, medicine and sport
i3D Panorama Battle
for Leningrad
Games