Transcript HCI lecture 29- augmented reality and data visualization.ppt

Augmented Reality and
Visualization
Lecture 29
By
Prof. Dr. Sajjad Mohsin
Today we shall cover

Augmented Reality
 History,

Applications, Limitations, Future
Visualization
 Definitions
 History
 Types
 Domains
of Applications
2
What is Augmented Reality?

Augmented reality is a
combination of a real life scene
from a person, and a virtual
scene from a computer that
augments the scene with
additional information

Specifically, augmented
reality superimposes audio,
graphics and other virtual
affects upon the surrounding
environment.
3
The Goal

Augmented reality
enhances one's
perception. The real
world is simply the
foundation. The goal is to
make a system that will
be so efficient that the
user will not be able to tell
the difference between
the real world and the
virtual augmentation.
4
Virtual vs. Augmented



Virtual Reality is defined as, "a computer generated, interactive,
three-dimensional environment in which a person is immersed.”
In other words, virtual reality completely immerses the use in the
virtual environment
Augmented reality virtually changes the real world while allowing the
user to keep a sense of presence about his surroundings
5
Performance Criteria

The mechanism that augments reality can be judged based on two things
 Update rate for generating the augmented image
 Accuracy of the registration of the real and virtual image
So in order for AR to be perceived as “real,” the rate at which the system renders
the image must be no fewer than 10 images a second. It also means that the
system must be able to register its environment accurately enough so that a
virtual change can be super imposed
6
The Hardware


In order for the AR
system to work, certain
hardware is needed,
including: display,
tracking, input devices,
sensors and a processor.
Currently, components
such as a CPU, camera,
display, accelerometer,
GPS and solid state
compass.
7
Display

One can perceive augmented reality in
generally three different ways:
 Head mounted: the most immersive
experience, HMD puts the virtual
scene over the user’s view of the
world. The device is either optical seethrough or video see-through
 Handheld: the most commercially
successful AR system, handheld
display is generally video see-through
with a gyroscope in order to orient the
image
 Spatial: any AR device separated from
the user. Can be used by multiple
people. In some cases, users are able
to touch virtual physical objects giving
a sensation of realness.
8
Software

AR vision methods are
based on visual
odometry, or determining
position using camera
images


This generally includes
feature detection and other
image processing methods
It also requires a real world
coordinate system to be
made once the image is
processed, created by
perhaps fiduciary marks on
the image, simultaneous
localization and mapping of
the image, or other
mapping methods
9
Applications: Medical

Augmented Reality
has the ability to
enhance the
efficiency of surgeries
and diagnostics
 Virtual
x-rays
 Real time ultra-sound
images
 Nuclear magnetic
resonance images
 MRI image
10
Applications: Military








Numerous applications:
Detect objects not discernable to the naked eye,
such as toxins
Examine sound of gunfire to determine location
of shooter and type of weapon
Enhanced visual feedback of surroundings
Represent physical objects that aren’t yet
visible, such as an airplane that has yet to land
Displays in cockpits of military vehicles
War game simulation technology
Etc.
11
Applications: Navigation





Building navigation; in order to analyze and explore
buildings effectively
Military operations
Disaster management
Heads up displays that cover the displays of automobiles,
planes, etc.
Individual navigation through new urban areas (street to
street, place to place (GPS))
12
Applications: Entertainment,
Performance and Education




Augmented Reality systems
can create virtual objects and
environments that enhance the
experience of entertainment,
and the quality of education
Musical performances can be
augmented to add to the
experience or create certain
sound effects
Interaction with educational
tools
Virtual guides that can even
activate based on location on
GPS or proximity to marker
13
Applications: Digital Applications

Since smart phones come
with most of the basic
components for an AR
system, applications soon
began to flood the smart
phone market, making AR
commercially successful.
14
Smart Phone App -- Wikitude

Gives the user data about their
surroundings by overlaying information on
the real-time scene taken by the phone’s
camera
15
Smart Phone App -- Augmented ID

Gives the user the ability to identify
someone by looking at them through the
camera of the phone
16
Smart Phone App -- Image Space

Lets the user record messages, photos and video and
tag them with both place and time. Then, when
someone else goes to that same location they can see
the images that were left in the area
17
Smart Phone App -- Layer

Positioning the camera over your environment, the
phone can overlay images from web pages and
information about businesses in the area
18
Smart Phone App -- Video Games

A game is generally overlayed on to the user’s surroundings, then
the user may interact with the scene through the phone
19
Everyday Augmented Reality





Powers immersive video games
Very common to use it to navigate one’s
surroundings
Interact and understand digital content more
interestingly
Clear benefits recognized in using AR technology
Now found on most smart phones, opening up a
whole new way to interact with the digital and
physical environment



There is now an entire market for AR apps, including
one’s that possibly give schematics for a building,
the constellations in the sky or the information about
nearby restaurants
AR now available in Adobe Flash, making AR
accessible through web browsing
Recently there has been a new form of crude 2D
bar code technology, which can be used to quickly
transfer information
20
Limitations and Cons




Does not work as well indoors
GPS only accurate within 9 meters
Danger of ‘too much information,” where a
disconnect forms between the user and the real
world
Privacy will be harder to keep due to virtual
interaction
21
Limitations and Cons
Over reliance on AR technology
 Image recognition technology not
developed to potential yet
 Inaccurate depth perception
 Calibration and latency still poor

22
The Future of Augmented Reality


Next generation of smart
phones will be able to
handle augmented reality
to a much greater extent
due to improved
batteries, compasses,
graphics chips and
processors.
The mobility of the smart
phone and the location
based services make AR
a very popular
investment.
23
Future….
The line between virtual and physical
begins to blur as augmented technology
grows
 Greater access to information based on
AR will mean greater awareness of
surrounding
 Virtual graffiti and art could be developed
 Desire for increased AR technology in the
field of medicine, as well as other fields.

24
Information and
Data Visualization
Visualization

“A picture is worth more than a thousand
words”. – a Chinese proverb

“A picture is worth more than a thousand
numbers”.
26
It looks like a swirl. There are smaller swirls
at the edges. It has different shades of red
at the outside, and is mostly green at the
inside. The smaller swirls have purple
highlights. The green has also different
shades. Each small swirl is composed of even
smaller ones. The swirls go clockwise. Inside
the object, there are also red highlights.
Those have different shades of red also. The
green shades vary in a fan, while the purple
ones are more uni-color. The green shades
get darker towards the outside of the fan......
27
Terrain geometry:
Terrain Texture:
(10,20,21), (12,13,14), (13,32,12),....,
(1,2,3), (2,4,5),(3,5,6),.....
(23,34,54), (23,34,23), (45,26,78),....
Time 0:
Volumetric cloud cover:
0, 0, 12, 14, 15, 15, 17, 12, 23, 45,.....
Wind vectors:
(0.2, 0.3, 0.93,5), (0.4,0.5,0.76,12),...,
Time 1:
Volumetric cloud cover:
0, 0, 11, 12, 13, 16, 20, 12, 32, 45,.....
Wind vectors:
(0.4,0.5,0.76,12),(0.5,0.5,0.7,6),...
28
What Is Visualization?
•
•
•
•
“seeing is believing”
we observe and draw conclusions
seeing is also understanding
beware of ‘illusions’ (magicians)
29
What Is Visualization?
• Transformation of data or
information into pictures
• engages primary human sensory
apparatus - vision
30
What Is Visualization?
• Is a Tool for:
– Aid For Learning/Understanding
– Compact Representation Of Information
(e.g. Numbers)
– “Carrier” of Information
31
Visualization Flavors?
• Scientific Vis. - User Interfaces, data
representation/processing Algorithms,
Visual Representations
• Data Visualization - Include financial
data and statistical methods
• Information Visualization - Abstract
Data: WWW documents, file structures,
arbitrary relationships
32
History
1137 - earliest known map (China)
 1603 - first star charts by Johann
Beyer
 1637 - cartesian coordinate system
(Descartes)

33
History (2) - Statistical

1686 - first meteorological chart
(Halley)
• 1693 - mortality
tables of city of
Breslau (Halley)
-> first attempt
to correlate two
variables
34
History (3) - 2D
Approx. 1750 - contour lines (height)
 1817 - isotherms (temperature)
 1829 - isochromatic lines (color)
 1864 - isobars (pressure)

35
History (4) - 3D Imaging
1895 - X rays by W. Röntgen
 1898 - stereo X rays (mackenziedavidson) - locating foreign bodies in
humans
 1938 - x-ray sections or slices (3D!)
 1912 - x-ray crystallography (Laue) position of atoms in a crystal

36
History (5) - Computer Graphics
1949 - SAGE air defense - tracked
position of aircraft by radar, analyzed
results and display on CRT
 1965 - sketchpad (Sutherland) interactive graphical drawing system
 Used to be BIG and EXPENSIVE

37
History (6) - Scientific Visualiz.
1987 - NSF report [McCormick87]




Personal/exploratory graphics - to enable a scientist
to gain more knowledge (interact with data)
Peer graphics - enable scientist to show information
to their colleagues and to collaborate
Presentation graphics - communicate information and
results (high quality, fully annotated)
Publication of visualization - enable others to use the
data (replicable)
38
History (7) - Augmented Reality.

1983 - responsive environments
(Myron Krueger)
39
Visualization Domains
Volumetric data sources are usually produced by:



Scanning devices
Computation (mathematical), or
Simple measuring
40
Applications - Vis. As a Toolkit
Application tools usually coupled with



Haptic feedback devices
Stereo output (glasses)
Interactivity
demanding of the rendering algorithm
41
Scanning - Domains

Medical scanners (MRI, CT, SPECT, PET, ultrasound)
42
Scanning - Applications



Primary education
Medical education for surgery, anesthesia
Illustration of medical procedures to the patient
43
Scanning - Applications





Surgical simulation for treatment planning
Tele-medicine
Inter-operative visualization in brain surgery,
biopsies, etc.
Industrial purposes (quality control, security)
Games with realistic 3D effects?
44
Scanning (2)


Domain - biological scanners, electronic microscopes,
confocal microscopes
Apps - paleontology, microscopic analysis
45
Scientific Computation - Domain

Mathematical analysis

ODE/PDE (ordinary and partial differential equations)

Finite element analysis (FE),

Supercomputer simulations,
46
Scientific Computation - Apps

Computational fluid dynamics (CFD),

Computational field simulations (CFS),
47
Vector Field Viz Applications
Computational Fluid Dynamics
Weather modeling
48
Vector Field Visualization Challenges
General Goal:
Display the field’s directional information
Domain Specific: Detect certain features
Vortex cores, Swirl
49
Streamlines
A curves that connect all the particle positions
50
Streamlines (cont’d)
- Displaying streamlines is a local technique because you can
only visualize the flow directions initiated from one or a few
particles
- When the number of streamlines is increased, the scene
becomes cluttered
- You need to know where
to drop the particle seeds
- Streamline computation is
expensive
51
Measuring - Domains
•
Orbiting satellites
•
Spacecraft
•
Seismic devices
•
Statistical Data
52
Measuring - Applications
•
for military intelligence,
•
weather and atmospheric studies
•
planetary and interplanetary exploration
•
oil, precious metal exploitation, and
•
earth quake studies
•
Statistical Analysis - Info Vis (Financial Data …)
53
Taxonomy
Volumes
Surfaces
visualization
Data
Geometric
model
sampling
CT, MRI, Scanners,
Ultrasound
scanning
sensors,
Seismic
cameras
Image
processing
Image
(signal)
Display
Supercomputers
Numerical Simulations
Video
Recording
54
Viz vs. Graphics vs.. Imaging
• Imaging - Enhance, analyze, manipulate
and store 2D/3D images
• Graphics - Make pictures! Digital Image
Synthesis: sampling + illumination
• Visualization - Exploration,
transformation, viewing data as images
55
Relation To Other Fields
Illumination
Engineering
Optics
Signal/Image
Processing
Vision
Computational
Geometry
Visualization
Applied
Mathematics
Psychology
Cognition
Hardware
User
Interfaces
56
Famous topics of Visualization










Data representation on various types of grids
Rendering of scalar and multi-modal data
sets
Rendering of vector fields and diffusion data
sets
Efficient iso-surfacing algorithms
Distance fields and voxelization
Parallel graphics and visualization
Point-based graphics
Image-based graphics
Information visualization
Basic geometrical modeling concepts
57
Summary
By interacting with the visualization, the
user may obtain a deeper understanding
of the data and discover features that
would
be
impossible
with
static
representations.
 Visualization research is a growing field in
computer science.

58