Transcript Lecture #5: The Role of Sensors

The Role of
Sensors in
Robotics
Review: Why is robotics hard?
 sensors are:
 limited
 inaccurate
 noisy
 effectors are:
 limited
 crude
 the state (internal and external, but mostly external) of the
robot is partially-observable, at best
 the environment:
 often dynamic (changing over time)
 full of potentially-needed information
• Sensors are one of the key
elements as well as limitations in
robotics.
Sensors
 Sensors constitute the perceptual
system of a robot.
 magnetism -> compasses
 Sensors do not deliver state!
 smell -> chemical
 Sensors are physical devices that
measure physical quantities, such
as:
 temperature -> thermal, infra
red
 physical property ->
technology
 contact -> bump, switch
 distance -> ultrasound,
radar, infra red
 light level -> photo cells,
cameras
 sound level -> microphones
 strain -> strain gauges
 rotation -> encoders
 inclination -> inclinometers,
gyroscopes
 pressure -> pressure gauges
 altitude -> altimeters
 and others...
 Note: the same property can be
measured with different
sensors
Mobile Robotics Sensors that we
used in the past
 magnetism ->
compasses (PSUBOT)
• contact -> bump, switch
 distance -> ultrasound, sonar,
infrared
 light level -> photo cells,
cameras
 sound level -> microphones
 strain -> strain gauges
 rotation -> encoders
 smell -> chemical (fire
detector)
 temperature -> thermal,
infra red
 inclination ->
inclinometers,
gyroscopes
 pressure -> pressure
gauges
Simple and Complex Sensors
• Sensors range from simple to complex in the amount of information
they provide:
 a switch is a simple on/off sensor
 a human retina is a complex sensor consisting of more than a hundred
million photosensitive elements (rods and cones)
• Sensors provide raw information, which can be treaded in various
ways,
– i.e., can can be processed to various levels.
• For example, we can simply react to the sensor output:
– if the switch is open, stop, if the switch is closed, go.
• More complex sensors both require and allows to do more complex
processing.
Simple and Complex Sensors
• We can ask the following question:
"given the sensory reading I am getting, what was the
world like to make the sensor give me this reading."
• This is what is done in computer vision, for
example, where:
– the sensor (a camera lens) provides a great deal of
information (for example, 512 x 512 pixels = 262,144
pixels of black & white, or gray levels, or color), and
– we need to compute what those pixels correspond to in
the real world (i.e., a chair, a phone?).
Signals -> Symbols(States)
Sensors do not provide state/symbols,
just signals
A great deal of computation may be
required to convert the signal from a
sensor into useful state for the robot.
This process bridges the areas of:
 electronics,
signal processing, and
computation.
Levels of Processing
• Example 1. just to figure out if a switch is open or closed, you need to
measure voltage going through the circuit; that's electronics
 Example 2. now suppose you have a microphone and you want to
recognize a voice and separate it from noise; that's signal processing
 Example 3. now suppose you have a camera, and you want to take the
pre-processed image
 (suppose by some miracle somebody has provided you with all the edges in the
image, so you have an "outline" of the objects),
 and now you need to figure out what those objects are,
 perhaps by comparing them to a large library of drawings;
 that's computation
Levels of Processing
• As you can see, sensory data processing is
challenging and can be computationally
intensive and time consuming.
• Why does that matter?
• Because it means your robot needs a brain
to do this processing.
What does the brain have to have to
process sensors:
• analog or digital processing capabilities (i.e., a
computer)
• wires to connect everything
• support electronics to go with the computer
batteries
– to provide power for the whole thing
• Thus perception requires:
 sensors (power and electronics)
 computation (more power and electronics)
 connectors (to connect it all)
What does the brain have to have to
process sensors:
• It is not a good idea to separate:
– what the robot senses,
– how it senses it,
– how it processes it, and
– how it uses it.
• If we do that, we end up with a large, bulky, and ineffective robot.
• Historically, perception has been treated poorly:
– perception in isolation;
– perception as "king";
– perception as reconstruction.
• Traditionally these approaches came from computer vision, which
provides the most complex data.
The best is Sensor Integration
Approach
• Instead, it is best to think about these as a single
complete design:
 the task the robot has to perform
 the best sensors for the task
 the best mechanical design that will allow the robot to get
the necessary sensory information to perform the task
(e.g., the body shape of the robot, the placement of the
sensors, etc.)
New and Better Approaches to
Perception
• Perception in the context of action and the task
 Action-oriented perception
 Expectation-based perception uses knowledge
about the world as constraints on sensor
interpretation
 Focus-of-attention methods provide constraints on
where to look
 Perceptual classes partition the world into useful
categories
A New
and Better
Way
New and
Better
Approaches
to
Perception
• Nature is very clever in the way it solves perception/sensing
problem;
– it evolves special sensors with special geometric and mechanical
properties.
• Facetted eyes of flies, or
• polarized light sensors of birds have, or
• horizon/line sensors of bugs have, or
• the shape of the ear, etc.
• All biological sensors are examples of clever mechanical
designs that maximize the sensor's properties, i.e., it's range
and correctness.
Proprioception - internal state
• Origin of received sensory information divides
perception into
 Proprioception: sensing internal state (e.g., muscle tension,
limb position)
 Exteroception: sensing external state (e.g., vision, audition,
smell, etc.)
 Examples of proprioception :
 path integration (dead-reconning)
 balancing
 all movements...
Affordances
• Affordances are "potentialities for action
inherent in an object or scene" (Gibson 1979,
psychology)
 The focus is the interaction between the robot
and its environment
 Perception is biased by what needs to be done
(the task)
 E.g.: a chair can be something to sit in, avoid,
throw, etc.
Affordances
• As a robot designer, you may not get the chance to
make up new sensors, but you will always have the
chance (and the need) to design interesting ways of
using the available sensors to get the job done.
• Utilize the interaction with the world and always
keep in mind the task.
• Food for thought:
– how would you detect people in an
environment?
How to detect people?
• For example, how would you detect people? Some options include:
 temperature: pyroelectric sensors detect special temperature ranges
 movement: if everything else is static
 shape: now you need to do complex vision processing
 color: if people are unique colored in your environment
• Let's think about something even more simple: how would you
measure distance:
 ultrasound sensors give you distance directly (time of flight)
 infra red through return signal intensity
 two cameras (i.e., stereo) can give you distance/depth
 a camera can compute it from perspective
 use a laser and a fixed camera, triangulate
 structured light; overlying grid patterns on the world
 frequency and phase modulation
 interferometry
Sensor Fusion
• Another clever thing to do is to combine multiple sensors on a robot
to get better information about the world.
• This is called sensor fusion.
• Sensor fusion is not simple:
– Different sensors give different types, accuracy and complexity of information;
– processing is necessary to put them together in an intelligent and useful way,
– and in real-time.
• The brain processes information from many sensors (vision, touch,
smell, hearing, sound).
• The processing areas are distinct in the brain (and for vision, they are
further subdivided into the "what" and "where" pathways).
• Much complex processing is involved in combining the information.
Information Filters
• Sensory organs act as information filters.
– Extract only part of the total information available
– form a representation or physical encoding which
facilitates the answers to some questions while making
others impossible to answer
• Simple light sensors function like a set of goaloriented detectors, e.g. frog eyes
– are designed to detect motion not interpret static images.
Vision
• Vision is the process of converting sensory
information into the knowledge of shape, identity or
configuration of objects.
• Other sensors besides light sensors can also provide
similar information:
– bat sonar
– pit viper heat detector
– touch
Vision (more)
Vision
• Previous input and its interpretation and pre-wired
processing can greatly affect current processing of sensory
data.
• Seeing is the physical recording of the pattern of light
energy received from the environment.
• It consists of:
– selective gathering in of light
– projection or focusing of light on a photoreceptive surface
– conversion of light energy into a pattern of chemical or electrical
activity
Costs and Benefits
• A cost of sensing of a system in terms of:
– 1. energy,
– 2. organizational complexity and
– 3. the possibility of malfunction.
• The nature of useful information is related to
organism’s needs and goals.
– For example, plants only need information on light
direction.
• Their system compares the light energy received on differently
oriented surfaces.
Receptors in biological organisms
• Sensitivity to environmental influences is a general
characteristic of living cells.
• In addition to general sensitivity, most animals
develop a range of specialized receptor cells
– These often form parts of multi cellular sense organs.
• Types of senses are called sensory modalities.
Sensory Modality
• Classifications of sensors
– 1. Exteroceptors - sensitive to external influences
– 2. Interoceptors - respond to internal factors
– 3. Proprioceptors - signal movements or positions of muscles,
joints, etc.
• Classification can be based on the physical characteristic of
the stimulus concerned, e.g. light, mechanical, chemical.
• Phasic receptors respond to changes in the environment.
• Tonic receptors relate to the absolute level of stimulation.
• Some receptors are a combination of phasic and tonic.
• Sensitivity to one modality can be exploited to provide
information about another.
Sensory
Modality
Sensory Modality
• Classifications (more)
• Receptors sensitive to gravity are called statocysts.
– These receptors function by using sensory cilia in a
vesicle which contains one or more dense bodies to sense
the position of these bodies.
• These organs can also sense acceleration.
• Note:
– insects lack these specialized organs,
– instead, they depend on the information from many
sense organs associated with their joints to provide
relevant information.
Specialist and Generalist Receptors
• 1. Receptors which are specialists respond only to a
restricted range of whatever they are sensing.
– For example, olfactory specialists have a restricted
spectrum of response to odors
• with an acute sensitivity to only a single compound such as a
pheromone.
• 2. Generalist receptors respond to a wide variety of
stimuli within the modality.
– But each generalist has its own pattern of sensitivity, so a
substance can be recognized by the unique combination
of receptors activated.
Intensity Coding in biological
sensors
• Information from sensors is usually not just ON or
OFF, but also includes ``how much''.
• The range of stimulation intensity to which an
organism is sensitive is often a controllable factor.
• Also different cells can operate across different parts
of a wide range.
Sensory Processing Example
• In the locust, simple light sensing organs on the top
of the head produce a poorly focused image.
• A massive amount of receptor information (about
1000 receptors) in each organ is funneled through a
small number of second-order neurons (25).
• During flight, the ocelli provide a rapid, overall
assessment of the position of the horizon.
Another
Example
• When a male hoverfly has a possible
mate in its field of vision, it sets a course
to intercept.
• To plot a course, it needs distance,
velocity and course information of target
– probably not determined from observation.
• The fly ``assumes'' that the object in the
visual field is
– 1.the size of one of its own kind
– 2.travelling at approximately the same
velocity
• The size assumption leads to a
determination of distance.
• The direction and speed at which the
object moves across the visual field
indicate then its course and the intercept
can begin!
Convergence
• Convergence occurs when multiple sources of
information are compressed into a much smaller
domain.
• A sensory field is an array of receptors which
provide sensory input to a cell or centre in a nervous
pathway.
Divergence
• Divergence is the conveying of information from a single receptor
cell, or group of cells, into the nervous system via multiple or parallel
pathways.
• These pathways can be used to extract and segregate different types of
information.
• Divergence also covers the concept of a system responding to a single
sensory modality, but providing out to different centers and thus
influencing different types of behavior.
Labeled Lines
• This principle works on the premise that similiar signals from different receptors are
handled as if they were ``labeled'' by their origin.
• An example is the escape response of the cockroach.
• The lunging attack of a toad creates a current of air which is detected by sensory
hairs on the anal cerci of the insect,
• The hairs are arranged in a number of columns which are sensitive to wind from
different directions.
• The different columns form distinct combinations of connections with processing
neurons so that the insect is aware of the location of the threat.
• The combinations of sensory input trigger appropriate movements.
The Photopine
• Sensors
distributed over
vehicle body
• As the sensor is
touched, the
reflex response
is immediate
and it
determines the
area of contact.
Sources
• A. Ferworn
• Maja Mataric
• Fred Martin