Transcript Human Physiology and Perception in Virtual Environment

Human Physiology and
Perception in Virtual
Environment
[email protected]
1
Content
2.1 Physiology of visual perception
2.2 Physiology of auditory perception
2.3 Physiology of haptic and kinaesthetic
perception
2.4 Virtual presence
2.5 Summary
[email protected]
2
2.1


Physiology of visual perception
Most important channel to the Virtual
Environment
human very sensitive to any anomalies
of images
[email protected]
3
2.1.1 The Eye


Protected by the orbit (bony cavities in skull)
and fatty material in the surrounding
supported by 6 extraocular muscles, allows:
o
 movements 50 to left an right
o
o
 40 above and 60 below
[email protected]
4

Others:
 eyelid: provide protection from bright lights
 cornea
 iris
 pupil: 2 - 8mm in diameter, dilated in
accordance with amount of light
 cystalline lens: provides variable focusing
 retina: photosensitive, convert
electromagnetic radiation into impulses
[email protected]
5
Rod system

Unevenly distributed in the retina





Fovea area - absent
outside - 15 000 and 170 000mm-2
Several rods linked to a single nerve cell
React to light of lower intensity
Contain rhodopsin: maximum
sensitivity, dark-adpted for about 35
minutes
[email protected]
6
Cone system





Responsible for bright light, colour and visual
acuity (sharpness)
most densely distributed in the fovea
each cone receptor is connected to a single
nerve cell
Contain photochemical substance called
iodopsin
the blindspots a palce where there is lack
photoreceptor
[email protected]
7
Optic nerve


Connection to the brain (visual cortex of
the brain)
the nerves form the left and the right
eyes intersect at “chiasma”
[email protected]
8
Adaption



Human sensitive to extremely small variation
in light over a wide range ~ 1013 (from levels
of just about perceptible to level of max.
energy - can burn retina)
light sensing device must be in the above
range
So far no man-made sensor within this range
[email protected]
9
Accomodation




Resting eye’s depth: 6m - infinity
pupil diameter will effect the depth fill
the process of altering the curvature of
the crystalline lens by means of the
ciliary muscles is call “Accomodation”
expressed in dioptres (the unit for
specifying the refractive power of a
lens)
[email protected]
10
2.1.2 Visual field

The ability to see an object:



the object in the image of the retina
where on the retina it appears
charts constructed to show the visual
field
[email protected]
11
Visual angle

Dimensions of objects expressed in
terms of visual angle - angle
substended by the eye
[email protected]
12
2.1.3 Stereopsis (binocular
vision)






Both eyes share large portion of visual field
relates to neural and physiological interaction
of the two eyes in the overlap region
if 2 very different images presented, the visual
system often suppresses one of the images,
occasionally alternates - binocular rivalry (br)
br effect depends on size, brightness and hue
user’s performance will be effected by br
stereopsis in not always necessary for depth
perception
[email protected]
13
Partial binocular overlap




Alternative to binocular display
display monocular image inwards or outwards to
create partial binocular overlap
can produce smaller and lighter optical system
also an improvement in resolution
Average human has a binocular overlap (between
two eyes) of ~120o with 35o monocular vision
either side of the overlap region
 also consider the optics of the eye and the
bridge of the nose
[email protected]
14

binocular overlap can be convergent and
divergent


depending on the optical system being mounted
inward or outward
outward >> divergent system, inward convergent
system
[email protected]
15
Binocular rivalry effects



All observer have a dominant eye, brain suppresses the
disparate image on one eye
 dominant eye, eye whose image is perceived for a
greater period of time
occurs in helmet-mounted display (HMD)
 one channel is brighter than the other
 display scene representation and complexity
 easier to control rivalry effects in closed helmet
display (no direct view of the real world)
 harder to control rivalry in normal helmet-mounted
displays that overlay outside world
substantial amount of information can be ignored
[email protected]
16
Stereopsis: Limit of depth cue

4 cues are responsible for giving depth
perception
 lateral retinal image disparity
 motion parallax
 differential image size
 texture gradients
[email protected]
17
Lateral retinal image disparity


The difference in relative position of the
image of an object on the observer’s retinas
as a function of the IPD(interpupillary
distance) and vergence position
depth perception:
 lateral retinal image disparity in the range
of 0o-10o
o
o
 > 0 -10 results in double images
(diplopia) and the sensation of depth is lost
[email protected]
18
Vertical retinal image disparity




Occurs when vertical displacement of the
retinal image in one of the eyes
limits the horizontal movement of the eye
vertical disparities do not convey any depth
information to the observer
small amount can lead to diplopia (most
users can adapt to this after 15-20 mins)
 the user need to readapt to the real world
 accounts for some criticisms of the VE
system
[email protected]
19
Image rotation misalignment


A gradual increase of rotation misalignment
between left and right eye channels quickly
reaches the point where double images occur
Can cause
 visual fatigue
 image blurring
 tears
 queasiness
[email protected]
20
Image magnification induced
retinal disparity

Slightly magnified image in one eye can
cause several disparities depending on the
nature of the magnificatin
 known as aniseikonia
[email protected]
21
Stereoacuity

The ability to resolve small differences
in depth between two object and is
expressed in terms of visual angle

must eliminate other cues such as
monocular depth cues, inter-position and
perspective
[email protected]
22
2.1.4 Visual motion perception


Not properly understood
can be convincing if the observer is
static
[email protected]
23
Motion parallax


Refers to the relationship between
objects in an observer’s field of view as
the observer moves in relation to the
objects
does not matter if the observer is
moving relative to the scene or vice
versa; the visual effects do not alter
[email protected]
24
2.1.5 Temporal resolution

The eye respond to changing or varying
light levels in order to give dynamic
representations of a scene
[email protected]
25
Perception of flicker in display
device

Majority of displays produce images
that are derived sequentially in rasterlike fashion

so observer may perceive the display to be
flickering on and off
[email protected]
26
2.1.6 Spatial resolution



Display structure can effect the perceived eye level
Spatial frequency response of a display system
 an optical system can increase the amount of blurring
produced by a display system
 blurring can be caused by a small point source in the
image plane being reproduced as a spot of finite size
Spatial frequencies relationship to frame rate and
bandwidth
 the video bandwidth of a CRT(Cathode Ray Tube) or
matrix display is proportional to the frame rate and the
total number of pixels.
 Slow frame rates can increase the perception of flicker
[email protected]
27
2.1.7 Visual Space perception


Surroundings provides a framework for
the perception of spatial relationships
between object
best define axis set more centred on
the observer
[email protected]
28
Constancy scaling


Misleading effect on distance perception
examples Delboeuf, Judd, Lipps, MullerLyer, ponzo...
[email protected]
29
2.1.8 Colour perception

3 attributes applied to colour perception



Hue: the correct term to describe a colour
by name
Saturation: describe the amount of purity
or proportion of pure chromatic colour in
the perception
Brightness/Intensity: the degree of a
luminous light source
[email protected]
30
Intensity


The appearance of a coloured object
depends greatly upon a few conditions
a complex subject
[email protected]
31
Test for colour perception


Test includes:
 simple matching of lightness or hue of one test
objects to another
 comparisons involving coloured objects on
coloured background
 others
achromatic response- exhibits a certain behaviour for
a wide range of wavelength usually from blue to red
(white light response)
[email protected]
32


monochromatic - a narrower spectral response over a
few tens of wavelength (result in a single colour
display - incorrect to describe a black and white
display)
a coloured object when viewed under incandescent
light will appear to have different hue, lightness and
chrome, compared to natural daylight
[email protected]
33
Colour specification



Definition subjective
more accurate to plot the radiance of
the light source as a function of
wavelength
for transparent objects such as coloured
filters, the spectral transmittance
distribution is usually specified
[email protected]
34
2.2 Physiology of auditory perception




Extremely impressive sensory system
responsive to a wide range with a sensitivity of 0.2%
 the ears of an average young person are sensitive
to all sounds from about 15 Hz to 20,000 Hz
 the frequencies to which the ear is most sensitive
(about 1,000 to 2,000 Hz)
upper capability of around 110dB(unit to measure
the intensity/loudness of sound)
a sensitivity of a fraction of decibel at all intensity
levels is something that no current electronic system
can easily emulate
[email protected]
35
TO synthesize realistic auditory environment

take into account






acoustic environ and deal with aspects such as echo or
reverberation
position and orientation of the listener’s head, the source position
(head related transfer function, HRTF)
phase differences
overtones
2 pathways for the reception of sound: normal air conduction
route via ears, the bone conduction route in the head
total auditory perception = the routes of sound + the masking
effects caused by the head + inter-aural time delay between
two ears
[email protected]
36
2.2.1 Hearing


Divided into 3:
 the external
 middle
 internal
external collects sound waves with pinna (auricle) >> into
external auditory canal >> tympanic membrane (eardrum)
[email protected]
37
External ear



The pinna: elastic-like cartilage coverd by a thick layer of skin
 acts as a linear filter whose transfer function is dependent
upon the direction and distance of the sound source
 code the spatial characteristics of the sound field into
temporal and spectral attributes
 experiences properties such as diffraction, dispersion,
interference, masking, reflection and resonance
the external auditory canal: 2.5 cm long, wall composed of bone
lined with the same cartilage material of pinna, surface covered
with thin, highly sensitive skin
the tympanic membrane: semi transparent layer of connective
tissue that separates the external auditory canal and the middle
ear
 vibrates according to sound, external ear teminates here
[email protected]
38
Middle ear




Small air-filled cavity (epithelial lined)
one end is the tympanic membrane and the other end are two
openings - oval and round windows
also contain auditory tube that connects with the throat
also contain small auditory assicles known as the malleus, incus
and stapes
 malleus connected to tympanic membrane and >>
vibrations to incus >> stapes >> oval window
[email protected]
39
Inner ear

Complicated series of canals

an outer bony labyrinth



consists of series of cavities, the vestibule, the cochlea and semicircular
canals
an inner membranous labyrinth
tympanic membrane as an amplifier, increasing the auditory
signals to the oval window >> into the perilymph fluid of inner
ear

vibration in perilymph cause the oval window to vibrate accordingly
with the auditory signal >>causes pressure to vary in the
endolymph of cochlea >> causing hairs of the basilar membrane to
move against the tectorial membrane >> causes the generation of
nerve impulses >> travel to midbrain, the thalamus >> finally to
auditory region of the temporal lobe of the cerebral cortex
[email protected]
40
2.2.2 Auditory localization


The perception of where the sound is coming from - the pinnae
palys an important role
2 mechanism involve in our localization of sound
 Intensity differences
 operate for audio tones whose frequency is above 1.5kHz
 the higher the frequency, the greater the effect of head
masking
 relative intensity can approach 20 dB attenuation in one
ear compared to the other
 Temporal differences:
 an interaural delay as small as 10 μs can be detected by
some individual
 an interaural delay in excess of 650 μs can be sufficient
to localize a sound
[email protected]
41
Cont.



Audio localization is a function of:

The difference of sound reaching the two ears

the differences in sound relative to the directions and range relative
to the observer

monaural cues derived from pinna

Head movement
localization understood in the horizontal plane (left to right) but
very little is known for median plane and front to back
possible for sound to arrive 700 μs earlier in one ear, can be
attenuated as much as 40 dB, the interaural differences is
negligible at distances grater than 1m
[email protected]
42
Cont.





Interaural differences = 0, if sound source exists directly in front of
(or behind) the listener
 even if vertical position is raised or lowere from the central position
if head is moved the sound is put outside th emedian plane, resulting
in interaural differences
familiar sound distinguished because of the differential effects of the
pinna
A sound is generally softer when originating from the back compared
to the front
timbre is the name given to distinguish 2 auditory signals that have the
same pitch and intensity
 spectral composition of the sound source
 brightness, mellowness and richness
 depends on the envolope of the sound signal and the rate of
amplitude [email protected]
43
2.2.3 Frequency analysis




Signals mixed into complex audio signal
in order to hear the individual components, the frequency must
be sufficiently separated from one another - frequency
selectivity
When the component frquencies are too close, the signal is
perceived as a single component: the Ohm’s acoustic law
the ear can be said to act as a series of narrowly tuned filters
(oversimplification)
[email protected]
44
2.2.4 Pitch discrimination






Human sensitive to sound pitch variation in the range of 1kHz
to 3 kHz
perceptible frequency difference by average human is 0.3 %
(change fromm 1000Hz to 1003Hz)
 this figure can be improved >> musicians
At lower frequency on range of 32 Hz to 64 Hz, the perceptible
frequency difference is in the order of 1 %
At higher frequencies 16kHz to 20 kHz pitch discrimination is
extremely poor
average human can resolve about 2000 pitch variations
musician>> pitch depends on intensity and frquency

pitch is not a simple function of frequency
[email protected]
45
2.3 Physiology of haptic and
kinaesthetic perception


Haptic- concerned with the sense of
touch or force on the body)
Kinaesthetic - awareness of where body
parts an limbs are in space, both
statically and dynamically
[email protected]
46
2.3.1 Physiology of touch
(cutaneous sensitivity)





Mechanical contact with skin
 contact, vibration, sharp, pressure
touch more acurately calledcutaneous sensitivity
 complex
 must consider other stimuli such as heat
skin ~1-2mm below the eperdermis (outer layer)
local disturbance distributed, thereby attenuating the sensation
of touch
certain region of skin more sensitive to touch than others
[email protected]
47
Anatomy of the skin

Three mechanical stimuli that produce the sensation of touch




skin has certain thresholds for touch (vibrotactile thresholds)>>
depends on:




step function: a displacement of the skin for an extended period of
time, for example a small point resting on the surface of the skin
Impulse function: a transitory displacement of the skin that lasts
for a few milliseconds
periodic functions: transitory displacement of the skin that is
repeated regularly at constant or variable frequency
the type and position of the stimuli (where on the body)
temporal summation (the frequency of stimuli)
Rapidly adapting receptor (phasic): pressure, touch and smell
Slowly adapting receptor (tonic): pain and body position
[email protected]
48
Cont.





Some sensation have after-images, which persist when the
stimulus has been removed
Some sensation may disappear, even when the stimulus is still
being applied
Cutaneous receptors consists of dendrites of sensory neurons
that can be enclosed in a capsule of epithelial or connective
tissues
The nerve impulses generated by the cutaneous receptors >>
somatic afferent neurons in the spinal and cranial nerves >>
thalamus >> the general sensory area of the parietal lobe of
the cortex
neural pathway for touch, light and pressure : anterior (ventral)
spinothalamic pathway >> ventral posterolateral nucleus of the
thalamus (some sensation of light touch and pressure) >>
cerebral cortex (fully localized)
[email protected]
49
Cont.



Light touch: ability to perceive that something has touched the skin
discriminative touch: ability to localize exactly the point of touch
skin:

tactile receptors:






hair root plexuses: detect movement on the surface of body, hair as simple
lever
free nerve endings:detection of pain, object continuous contact
tactile discs : assist in discriminative touch
corpuscles of touch(numerous in the fingertips and palms of the hand):
perception of discriminative touch
type II cutaneous mechanoreceptors: heavy and continous touch sensation
Cutaneous sensitive nerve fibres:


SA1, SA2: slowly adapting nerve fibres - responsive to an object that comes
into contact with the skin
RA1, RA2: fast adapting nerve fibres - respond on the movement of the skin
[email protected]
50
Cont.

To construct device to communicate the sensation of touch to a
user


must be aware of the dynamic range of touch receptors and
adaption to certain stimuli
too easy to disregard the fundamental characteristics of the
human body
[email protected]
51
Proprioception/kinaesthesia


Awareness of the movements and relativepositions of various
part of the body
takes into account:







the rate and movement of our limbs
the static position when movement ceases
so can estimate the weight supported by the limbs / the force
exerted by the contaction of our muscles
human can remember a particular joint position even after 24
hours of an event
visual cues help in kinaesthetic judgement
muscles receptors: awareness of static limb position
muscles receptors + vitaneous receptors : awareness of limb
dynamics
[email protected]
52
Interstingly:

lack of sense of static position of the finger joints 



joints closer to body insensitive to small angles compared to
distal joints
also consider the velocity of individual joints


this comes from cues of movement signals from the muscles and
the skin receptors and
non-kinaesthetic cues
small rate movement might be too small for perception
subject not fully understood for example tnesing one’s muscles
improve movement sense
[email protected]
53
Cont.

Human can gauge or measure the force produced by the
contraction of certain muscles by 2 mechanisms:




sensory inputs from tension receptors provide an awareness of
mucscle tension
command signals generated in the brain are monitored and provide
a sense of effort
term ‘light’ or ‘heavy’: perception of the weight of an object
depends on weight of object and the condition of the muscles
effect known as: “weight expectancy illusion” when heavy
object is lifted prior to lifting a lighter object >>
underestimation of the weight of the lighter object
[email protected]
54
Cont.


Human detection of joint angle in range of 0.20o - 6.10o
order of decreasing sensitivity of the joints:



hip, shoulder, knee, ankles, elbow, knuckles, wrist, the last toe
metacarpophalangeal joints, toes
human also possess an internal image of the positions of our
limbs/joints that does not depend on sensing information +
memory capability for limb position
mental capability can be fooled by the effect of surroundings
[email protected]
55
Mechanoreceptors

Cutaneous mechanoreceptors partly responsible kinaesthesia
>>




does not convey a sense of joint position
joint mechanoreceptors found in ligaments and capsules of joint
and adapt slowly to stretching of ligaments and compression of
capsules


skin stretches and contracts as we move
differentiation in cutaneous mechanoreceptors over different
regions of body :hands, feet and face more sensitive than skin
around elbows
exception when hyperextension of a joint occurs
Muscles mechanoreceptors most dominant mechanism for
kinaesthesia
[email protected]
56
Muscles spindles and Golgi tendon

Muscles spindles Sensitive to limb position and movement



lie in parallel with the contractile elements of muscle tissue
can determine muscles stretch and rate of increase in muscle
length
Golgi tendon organs: detecting tension


lie in series with the tension producing muscle elements
produce an indication of tension developed in the muscles
[email protected]
57
2.4 Virtual presence

Two types of feeling in virtual presence

Feeling of being part of a synthetic experience




feels immersed to some extend but also aware of the outside world
not a complete immersion
such as some of the virtual environment games
other factors also contribute


poor graphics, cartoon type environment, hand-held button ...
Feeling of being immersed in the synthetic experience



felt part of the actual environment
can take palce slowly
visual cue important to gain sense of presence
[email protected]
58
Breakdown of virtual presence

When:






person feels tired
head-mounted display becomes uncomfortable
unnatural movements or lags in VE
visual or auditory information not realistic
this subject not fully understood
need to be aware of the way human being interacts with a real
world and the adaptation that takes place when things change
in the real or external environment. Ellis (1991) states that:


knowledge constantly being updated by behavioral plasticity of
visual-motor coordination and vestibular reflexes
large part of our sense of physical reality >> internal processing
NOT from immediate sensory of info we receive
[email protected]
59
Cont.

Important to



Sheridan (1992) states that:



Match our virtual environment peripherals to the operator
perceptual system
how we present information to the operator
presence is subjective sensation like mental workload and mental
model
likely to be multidimensional
Presence - subjective or objective measures? Have not been
determined!
[email protected]
60
Seeing parts of one’s own
body

Seeing parts of one’s body reinforce the feeling of presence
 strong feeling of presence if visual system allows actual
parts of operator to come into field of view at the
appropriate time.
[email protected]
61
High resolution and large field
of view


Must no have the framework of display visible >> can cause
conflict in the sense of presence
operator/user should be able to move / see things in the
environment as in the real world

restrictive view can reduce the feeling of presence
[email protected]
62
Familiarity of virtual
environment of scene

time taken to adapt to VE shorter if VE relates to real world


depends on the VE system as a whole
Zeltzer (1991 and 1992) states that, VE has 3 components



a set of models/objects or processes
a means of modyfying the states of these model
a range of sensory modalities to allow the participant to experience
the VE
[email protected]
63
Zeltzer’s cube



The cube: (scaling 0 - 1)
Autonomy: refers to qualitative measure of the virtual object’s
ability to react to events and stimuli.
Interaction: refers to the degree of access to the parameters or
variables of an object.



0: non real time control of variables
1: variables that can be manipulated in real time during program
execution
Presence: crude measure of the fidelity of the sensory input and
output channels.

Dependen upon the task requirements: application has a bearing
[email protected]
64
Determinants of presence

Sheridan (1992) proposed 3 determinants + another one:





Extent of sensory information
Ability of the observer to modify theor viewpoint for visual parallax
or visual field. Includes the ability to reposition the head to
maintain binaural hearing
Ability to modify the spatial relationships of objects in the virtual
environment
The closed loop performance due to an opertor-induced motor
movement. Also includes the dynamic behaviour of movable
objects in the virtual environment
task to be performed important not just the visual displayed

matching devices to performance of human sensory >> difficult
[email protected]
65
Summary



Human factors principles in VE
Key factor is to develop a series of metrics to measure human
performance
subject of human perception is vast
[email protected]
66