Lect.002.Sensors Their Role New Approchoaches Perception

8/8/2019 Lect.002.Sensors Their Role New Approchoaches Perception

1/36

The Role ofThe Role of

Sensors inSensors in

RoboticsRobotics


2/36

Review: Why is robotics hard?Review: Why is robotics hard?

y sensors are:

y limited

y inaccurate

y noisy

y effectors are:

y limited

y crude

y the state (internal and external, but mostly external) of the

robot is partially-observable, at best

y the environment:

y often dynamic (changing over time)

y

full of potentially-needed information


3/36

SensorsSensors Sensors are one of the key

elements as well as limitations in

robotics.

y Sensors constitute the perceptualsystem of a robot.

y Sensors do not deliver state!

y Sensors are physical devices that

measure physical quantities, such

as:

y physical property ->

technology

y contact -> bump, switch

y distance -> ultrasound,

radar, infra red

y light level -> photo cells,

cameras

y sound level -> microphones

y strain -> strain gauges

y rotation -> encoders

y magnetism -> compasses

y smell -> chemical

y temperature -> thermal, infra

red

y inclination -> inclinometers,gyroscopes

y pressure -> pressure gauges

y altitude -> altimeters

y and others...

y Note: the same property can be

measured with different

sensors


4/36

Mobile Robotics Sensors that weMobile Robotics Sensors that we

used in the pastused in the past

contact -> bump, switchy distance -> ultrasound, sonar,

infrared

y light level -> photo cells,

camerasy sound level -> microphones

y strain -> strain gauges

y rotation -> encoders

y magnetism ->

compasses (PSUBOT)

y

smell -> chemical (firedetector)

y temperature -> thermal,

infra red

y

inclination ->inclinometers,

gyroscopes

y pressure -> pressure

gauges


5/36

Simple and Complex SensorsSimple and Complex Sensors

Sensors range from simple to complex in the amount ofinformationthey provide:

y a switch is a simple on/off sensor

y a human retina is a complex sensor consisting of more than a hundred

million photosensitive elements (rods and cones)

Sensors provide raw information, which can be treaded in various

ways,

i.e., can can beprocessedto various levels.

For example, we can simply reactto the sensor output: if the switch is open, stop, if the switch is closed, go.

More complex sensors both require and allows to do more complex

processing.


6/36

We can ask the following question:"given the sensory reading I am getting, what was the

world like to make the sensor give me this reading."

This is what is done in computer vision, forexample, where:

the sensor (a camera lens) provides a great deal of

information (for example, 512 x 512 pixels = 262,144

pixels of black & white, or gray levels, or color), and

we need to compute what those pixels correspond to in

the real world (i.e., a chair, a phone?).

Simple and Complex SensorsSimple and Complex Sensors


7/36

Signals -> Symbols(States)

ySensors do not providestate/symbols,just signals

yA great deal of computation may

be required to convert the signalfrom a sensorinto useful state for

the robot.

yThis process bridges the areas of:

y electronics,

ysignal processing, and

ycomputation.

Real state

measurement

Model of realState in robot memory


8/36

Levels of Processing

Example 1.just to figure out if a switch is open or closed, you need tomeasure voltage going through the circuit; that's electronics

y Example 2. now suppose you have a microphone and you want to

recognize a voice and separate it from noise; that's signal processing

y

Example 3. now suppose you have a camera, and you want to take thepre-processed image

y (suppose by some miracle somebody has provided you with all the edges in the

image, so you have an "outline" of the objects),

y and now you need to figure out what those objects are,

y perhaps by comparing them to a large library of drawings;

y that's computation


9/36

Processing sensory data needs brain

As you can see, sensory data processing ischallenging and can be computationally

intensive and time consuming.

Why does that matter?

Because it means that your robot needs aneeds a brainbrainto do this processing.


10/36

What does the brain have to have to

process sensors:

analog or digitalprocessing capabilities (i.e., a

computer)

wires to connect everything

support electronics to go with the computer batteries

to provide power for the whole thing

Thus perception requires:

y sensors (power and electronics)

y computation (more power and electronics)

y connectors (to connect it all)


11/36

What does the brain have to have to

process sensors: It is not a good idea to separate:

what the robot senses,

how it senses it,

how it processes it, and

how it uses it.

If we do that, we end up with a large, bulky, and ineffective robot.

Historically, perception has been treated poorly:

perception in isolation;

perception as "king";

perception as reconstruction.

Traditionally these approaches came from computer vision, whichprovides the most complex data.

Not a good idea to separate

and reconstruct


12/36

The best isThe best is Sensor IntegrationSensor Integration

ApproachApproach Instead, it is best to think about these as a single

completecomplete design:

y the task the robot has to perform

y the best sensors for the task

y the best mechanical design that will allow the robot to get

the necessary sensory information toperform the taskperform the task

y the body shape of the robot,

y theplacementof the sensors,

y

etc.


13/36

New and BetterNew and Better Approaches toApproaches to

PerceptionPerception Perception in the context of action and the task

y Action-orientedperception

y Expectation-basedperception uses knowledge about theworld as constraints on sensor interpretation

y Focus-of-attention methods provide constraints on whereto look

y Perceptual classes partition the world into useful

categoriescategories


14/36

A New and Better Way

Nature is very clever in the way it solves perception/sensing problem;

it evolves special sensors with special geometric and mechanical properties.

Facetted eyes of flies, or

polarized light sensors of birds have, or

horizon/line sensors of bugs have, or

the shape of the ear, etc.

All biological sensors are examples of clevermechanical designs

These designs maximize the sensor's properties:

it's range

correctness.

Use ideas from Nature forUse ideas from Nature for

PerceptionPerception


15/36

ProprioceptionProprioception - internal state

Origin of received sensory information dividesperception into

y Proprioception: sensing internalinternal state (e.g., muscle tension,

limb position)y Exteroception: sensing externalexternal state (e.g., vision, audition,

smell, etc.)

y Examples of proprioception :ypath integration (dead-reconning)

ybalancing

y all movements...


16/36

Affordances

Affordances are "potentialities for action inherent in anobject or scene"(Gibson 1979, psychology)

y The focus of affordances is the interactionbetween the

robot and its environment

y Perception is biased by what needs to be done.

y Robot thinks what is my task?

y I see a chairbecause I want to sit on it

y I see a chair as something to avoid.

y I see a chair as something to throw at my enemy.


17/36

Affordances

As a robot designer, you may not get the chance to make upnew sensors.

But you will always have the chance to design interesting

ways of using the available sensors to get the job done.

It is often not only a chance but a necessity!

Utilize the interaction with the world and always keep in mind the task.

Food for thought:

how would you detect people in an environment?


18/36

How to detect people? For example, how would you detect people? Some options

include:y temperature: pyroelectric sensors detect special temperature ranges

y movement: if everything else is static

y shape: now you need to do complex vision processing

y color: if people are unique colored in your environment

Let's think about something even more simple: how would you

measure distance:

y ultrasound sensors give you distance directly (time of flight)

y infra red through return signal intensity

y two cameras (i.e., stereo) can give you distance/depth

y a camera can compute it from perspective

y use a laser and a fixed camera, triangulate

y structured light; overlying grid patterns on the world

y frequency and phase modulation

y interferometry


19/36

Sensor FusionSensor Fusion Another clever thing to do is to combine multiple sensors on a robotto get better information about the world.

This is called sensor fusion.

Sensor fusion is not simple:

Different sensors give different types, accuracy and complexity of information;

processing is necessary to put them together in an intelligent and useful way,

and in real-time.

The brainprocesses information from many sensors (vision, touch,

smell, hearing, sound). The processing areas are distinct in the brain

For vision, they are further subdivided into the "what" and "where" pathways.

Much complex processing is involved in combining the information.


20/36

Information FiltersInformation Filters

Sensory organs act as information filters. Extract only part of the total information available

form a representation or physical encoding which facilitates the

answers to some questions while making others impossible to

answer Simple light sensorsSimple light sensors function like a set ofgoal-oriented

detectors.

For instance, frog eyesfrog eyes

are designed to detect motion notnot interpret static images.


21/36

VisionVision Vision is the process of converting sensory information into

the knowledge of:

shape, identity or

configuration of objects.

Other sensorsbesides light sensors can also provide similarinformation:

bat sonarsonar

pit viperheat detector

touchtouch


22/36

Vision (more)

Currentprocessing of sensory data can be greatly affectedby:

previous input and its interpretation and

pre-wired processing

Seeing is the physical recording of thepattern of lightenergy received from the environment.

It consists of: selective gathering in of light

projection or focusing of light on a photoreceptive surface

conversion of light energy into a pattern of chemical or electrical

activity

Characteristics ofVision sensorCharacteristics ofVision sensor


23/36

Costs and Benefits of SensingCosts and Benefits of Sensing

A cost of sensing of a system in terms of:

1. energy,

2. organizational complexity and

3. the possibility ofmalfunction.

The nature of useful information is related toorganisms needs and goals.

For example, plants only need information on lightdirection.

Their system compares the light energy received on differentlyoriented surfaces.


24/36

Receptors in biological organismsReceptors in biological organisms

Sensitivity to environmental influences is a general

characteristic of living cells.

In addition to general sensitivity, most animals

develop a range ofspecialized receptor cells

These often formparts of multi cellularsense organs.

Types of senses are called sensory modalities.

Tell the story of bacteria foraging


25/36

Sensory ModalitySensory Modality

C

lassifications of sensorsC

lassifications of sensors 1. Exteroceptors - sensitive to external influences 2. Interoceptors - respond to internal factors

3. Proprioceptors - signal movements or positions of muscles, joints, etc.

Classification can be based on the physical characteristic of the stimulus concerned,

e.g.: light,

mechanical,

chemical.

Phasic receptors respond to changes in the environment.

Tonic receptors relate to the absolute level of stimulation.

Some receptors are a combination of phasic and tonic.

Sensitivity to one modality can be exploited to provide information about another.


26/36

Sensory Modality

Classifications (more)Classifications (more) Receptors sensitive to gravity are called statocysts.

These receptors function by using sensory cilia in a vesicle which

contains one or more dense bodies to sense the position of these

bodies.

These organs can also sense acceleration.

Note:

insects lack these specialized organs,

instead, they depend on the information from many sense organs

associated with their joints to provide relevant information.

Sensory ModalitySensory Modality


27/36

Specialist and Generalist ReceptorsSpecialist and Generalist Receptors

1.1. Receptors which are specialists respond only to arestricted range of whatever they are sensing. For example, olfactory specialists have a restricted

spectrum of response to odors

with an acute sensitivity to only a single compound such as apheromone.

2.2. Generalist receptors respond to a wide variety ofstimuli within the modality:

but each generalist has its own pattern of sensitivity

so a substance can be recognized by the unique

combination of receptors activated.


28/36

Intensity Coding in biologicalIntensity Coding in biological

sensorssensors Information from sensors is usually not just ON or

OFF, but also includes ``how much''.

The range of stimulation intensity to which an

organism is sensitive is often a controllable factor. Also different cells can operate across different parts of a

wide range.


29/36

Sensory Processing ExampleSensory Processing Example

In the locust, simple light sensing organs on the top

of the head produce a poorly focused image.

A massive amount ofreceptor information (about

1000 receptors) in each organ is funneled through a

small number ofsecond-order neurons (25).

During flight, the ocelliocelliprovide a rapid, overall

assessment of the position of the horizon.


30/36

Another

Example

When amale hoverfly has a possible

mate in its field of vision, it sets a course

to intercept.

To plot a course, it needs distance,velocity and course information of target

probably not determined from observation.

The fly ``assumes'' that the object in the

visual field is

1.the size of one of its own kind

2.travelling at approximately the same

velocity

The size assumption leads to a

determination of distance. The direction and speed at which the

object moves across the visual field

indicate then its course and the intercept

can begin!


31/36

ConvergenceConvergence

Convergence occurs when multiple sources of

information are compressed into a much smaller

domain.

A sensory field is an array of receptors which

provide sensory input to a cell or centre in a nervous

pathway.


32/36

DivergenceDivergence Divergence is the conveying of information from a single receptor

cell, or group of cells, into the nervous system via multiple or parallelpathways.

These pathways can be used to extract and segregate different types of

information.

Divergence also covers the concept of a system responding to a single

sensory modality,but providing out to different centers and thus

influencing different types of behavior.


33/36

Labeled LinesLabeled Lines This principle works on the premise that similiar signals from different receptors are

handled as if they were ``labeled'' by their origin.

An example is the escape response of the cockroach.

The lunging attack of a toad creates a current of air which is detected by sensory hairs on

the anal cerciof the insect,

The hairs are arranged in a number of columns which are sensitive to wind from differentdirections.

The different columns form distinct combinations of connections with processing

neurons so that the insect is aware of the location of the threat.

The combinations of sensory input trigger appropriate movements.


34/36

The PhotopineThe Photopine

Sensors

distributed over

vehicle body

As the sensor is

touched, thereflex response

is immediate

and it

determines thearea of contact.


35/36

Sources

A. Ferworn Maja Mataric

Fred Martin


36/36

Questions for students1. List all robotics sensors that you are familiar with.

2. List some inexpensive sensors that one can purchase in Radio Shack or HomeDepot and how they can be used in robotics.

3. What is the concept of sensor fusion. Write a simple program to fuse 2 typesof sensors.

4. What is proprioception and exteroception?

5. What are affordances, give examples for our robot theatre. Why affordancesare so important.

6. Discuss simple vision system for a humanoid robot, what can it do?

7. Give examples of complex and simple sensors and their role.

8. Give examples of perception system of some simple animal and how thissystem can be modeled in our robots, both stationary and mobile robots.What kind of experiments would you implement with this system?

9. Give examples of divergence and convergence of sensors.

10. How would you implement a vision system for a simple mobile robot thatwould act according to three principles: (1) if a moving item is small, it is apotential food, attack it. (2) if a moving item is large, it is a potential enemy,escape. (3) if a moving target is your size, it is a potential mating partner,mate.

Documents

Lect.002.Sensors Their Role New Approchoaches Perception