Haptics Technologies:Bringing Touch to Multimedia

C4: Machine Haptics

Outline Haptic Interfaces

Robotics Perspective Haptic Interface System

HAVE Sensors Electro-Mechanical Sensors Optical Sensors Capacitive Sensors Resistive Sensors Force Sensors Strain Gauge sensors Magnetic sensors

HAVE Actuators Magnetic Levitation Devices Non-holonomic Devices Cable and Linkage Devices Parallel Mechanisms

Performance Specifications Physical Attributes Spatial Attributes Temporal Attributes

State-of-the-Art Haptic Interfaces Tactile Interfaces: Commercial Development Tactile Interfaces Research Prototypes Kinesthetic Interfaces Research and Development Efforts in Kinesthetic haptic displays

Closing Remarks

Introduction Haptic interface delivers higher sense of

immersion in a HAVE application via the sense of touch.

Haptic interfaces distinguished characteristic is the bi-directional flow of energy and information

This chapter covers: Components that make a haptic interface Attributes that define the quality of a haptic interface Survey existing haptic interfaces

Robotics Perspective Robotics refer to the science of designing and

developing forms of artificial life to replace human operators.

A major issue in the development of robotics is force control (integration of task goals, trajectory generation, force and position feedback, and the modification of trajectories)

Haptic interfaces stem from the well established robotics field and technologies.

Haptic Interface System Sensors measure interactions between a contact surface and the

environment Actuators provide mechanical motion in response to an electrical

stimulus ADC (analog-to-digital converter) and DAC (digital-to-analog converter Communication Module: communicates information with the computer

system

Human

Actuator(s)

DAC

Communication Module

Computer

Sensor(s)

ACD

Mechatronic System

HAVE Sensors A sensor is a device that measures a physical quantity

and converts it into a signal that can be read by an observer or an instrument

The output signal of a sensor is a linear or is linearly proportional (algorithmic) to the value of the measured physical property

Haptic sensor read either the interaction forces or the haptic interface handler position (position, force, or pressure information)

HAVE Sensors

Physical properties of HAVE Sensor Sensitivity error: when the measured value differs from the real

value. The lower the sensitivity error, the more precise and expensive the sensor is

Offset or sensor bias: this error occurs when the measured property is zero but the output signal is not zero

Dynamic error: caused by a rapid change of the measured property over time

Noise: represents a random deviation of the signal over a given period of time

Digitization error: an error that occurs with digital sensors when the input is converted to a digital signal.

Examples of Array Sensor Implementations and their Densities

Category Sensing Strategy Implementation Example

Density mm-2 Size

Electrical

Capacitive devices

(Boie, 1984) 0.18 8 x 8

(Siegel et al., 1987) 0.27 8 x 8

(Fearing, 1990) 0.07 8 x 8

Piezoresistors (Speeter, 1988) 1.58 16 x 16Polysilicon Piezoresistors (Sugiyama et al., 1990) 4.00 32 x 32

Mechanical

Conductive Plastic (Raibert and Tanner, 1982) 1.00 6 x 3

Conductive Silicone Rubber

(Hillis, 1982) 2.56 16 x 16

(Reynaerts and Brussel, 1993) 0.69 16 x 16

(Shimojo et al., 1991) 1.00 64 x 64

Conductor Rubber Strain Gauge (Russel, 1987) 0.007 5 x 5

Electrorheological (Monkman, 1993) 0.25

Magnetic Magnetic Dipole (Hackwood et al., 1983) 0.25 7 x 7

ElectromagneticOptical Waveguide (Maekawa et al., 1992) 0.08 10 x 10

Ultrasonic (Hutchings et al., 1994) 0.31 16 x 16

Electro-Mechanical Sensors Enables the conversion of mechanical information into

equivalent electrical signals. Are characterized by high reliability and sensitivity and

are mainly used to build tactile sensing arrays However, they suffer from being fragile and vulnerable to

overpressure due to mechanical compliances, as well as being too expensive and bulky for use in wearable applications

Are vulnerable to electromagnetic interference (EMI) and corrosion

Optical Sensors

Optical sensors are immune to external electromagnetic interference

Intrinsic sensors: Sensing takes place within the fiber itself moving an obstruction into the light path causes a modulation of the

intensity of light more sensitive but are more expensive Optical

receiver

DeformabletubeOptical

source (a)

Optical Sensors Extrinsic Sensors: Sensing takes place in a region outside the fiber The intensity of the received light is a function of the

distance between the reflector and the plane of the source and represents the applied force

Are less expensive, easy to use, but are less sensitive

Combined opticalreceiver and source Spring steel

Reflective surface

(b)

Capacitive Sensors Utilize the change of capacitance between two electrodes covering

a deformable dielectric The two electrodes are separated by a dielectric medium, which is

used as an elastomer There is an effective limit on the resolution of the capacitive array

and have complex signal conditioning

Resistive Sensors Based on the measurement of the resistance of a

conductive elastomer between two points Are very robust when withstanding overpressure, shock,

and vibration the deformation of the elastomer might permanently alters

the elastomer material density

Force Sensors Piezoresistivity-conductive polymer that changes

resistance following the application of force to its surface A polymer sheet with a sensing film Applying a force to the surface of the sensing film

causes particles to touch the conducting electrodes, which changes the resistance of the film

Strain Gauge sensors

Detects the change in length of the material attached to it when external forces are applied

Can be used as a load cell where the stress is measured directly at the point of contact

Magnetic sensors Uses magnetoresistive material is a material whose

magnetic characteristics are modified when the material is subjected to changes in externally applied physical forces

The magnetoresistive (or magnetoelastic) sensor has a number of advantages like high sensitivity and dynamic range, a lack of measurable mechanical hysteresis, a linear response, and physical robustness.

HAVE Actuators An actuator is a force and/or position source that exerts

forces on the human body/skin to simulate a desired sensation

Important factors in actuator design are: response time safety, mechanical transparency, workspace, number of degrees of freedom, maximum applicable forces and stiffness range, compactness, control bandwidth

Electrical actuators Include motor-based actuators with many different types

of motors, such as Direct Current (DC), brushed, Permanent Magnet (PM), stepper motors, and rotary, linear, and latching solenoid actuators

Do not require significant amounts of space to operate and are easy to install, and produce only negligible levels of electromagnetic noise oscillation

However, the small torques they generate (compared to their size and weight), their low bandwidth, and rigidity are their main limitations.

Pneumatic actuators Utilize compressed air pressure to transfer energy from

the power source to the haptic interface Simple, lightweight, and provide higher power-to-weight

ratios Limitations are low bandwidth and stiffness due to the

compression of air Lubrication is an issue because static friction is not

handled well

Hydraulic actuators

Based on a fluid, which is in most cases oil High bandwidth devices and do not suffer

from a friction problem The disadvantages are their bulkiness and

weight Need for more maintenance since the oil

must be filtered and cleaned

Magnetic Levitation Devices Use the Lorentz force principle to suspend an object with

the support of magnetic fields The approach is simple, compact, and offers high

bandwidths but has limited workspace Example:

the Butterfly Haptics Meglev 200 device

Other Actuation Technologies

Non-holonomic Devices: uses non-holonomic joints to alleviate problems of performance, stability, and safety.

Cable and Linkage Devices: the user grasps a handle that is controlled and supported from all directions by several actuated cables or springs

Parallel Mechanisms: actuators are kept at the base of the device. Therefore, they are mostly grounded, which leads to lower device inertia and greater strength and rigidity

Performance Specifications Physical Attributes

Inertia: This attribute depends on the mass of the haptic device. Backdrivability: the ability to move the end-effector of the device

within the workspace without op Friction/Damping: comes in two forms: Coulomb friction, and

viscous or damping friction position/resistance Exertable Force Attributes: characterize the ability and flexibility

of the device to generate force feedback (such as maximum exertable force, continuous force, minimum displayed force)

Stiffness: the ability of a device to mimic a solid virtual object Size/Weight: The size and weight of a haptic interface has a

direct impact on the comfort level of the user

Spatial Attributes Workspace: The area or volume in real world space that the end-

effector of a haptic device can reach Position Resolution: Defined as the smallest amount of movement

over which the position sensors can detect a change in the position of the end-effector

Degree of Freedom: number of independent directions along which the haptic interface is able to display motion, sensing, or actuation capabilities

Precision and Repeatability: Precision refers to how accurately the position sensor can refer to its position. Repeatability represents how accurately the haptic device can sense the identical physical position as being the same virtual position

Grounding Location: The grounding location is the base reference that the device is attached to

Temporal Attributes Device Latency: is the time measured from the instant of sending a

command to the device to the instant of receiving a response Bandwidth: is defined as the range of frequencies over which the

hand-controller provides force feedback Haptic Refresh Rate: is the speed at which the feedback loop can be

completed, and it is usually expressed in Hertz Maximum Acceleration: reflects the ability of a haptic device to

simulate stiffness of virtual objects like walls Haptics Update Rate / System Latency: includes sensing the

position of the haptic device, computing the force feedback in the simulation, sending the force to the device, and reading the next position

Tactile Interfaces The sense of natural physical contact with the ambient

environment. Tactile interfaces are capable of reproducing: tactile sensations such as pressure, texture, puncture, thermal

properties, softness, and wetness friction-induced phenomena such as slippage, adhesion, and

micro failures local features of objects such as shape, edges, embossing and

recessed features

Commercial and Development Tactile-haptic Interfaces

Product Description Sensation Vendor

Com

mer

cial

Inte

rfac

es

CyberTouch Vibro-tactile Stimulators: Six (one on each finger, one on the palm)

Pulses or sustained vibration

Immersion Corporation

Touch Master 4 Vibrotactile Stimulators (each finger)

Vibration EXOS, Inc.( Microsoft)

Tactile Mouse Vibro tactile vibrations LogitechTactool System 2 Fingers Impulsive vibration Xtensory, IncDisplaced Temperature System

Via Thimble Temperature Change CM Research, Inc.

Res

earc

h &

Dev

elop

men

t

HAPTAC Tactile feedback Electric pulses Shape Memory Alloy (SMA)

Armstrong Laboratory

Prototype Tactile Shape Display

Two-fingered hand with 2 DOFs in each finger.

Electric pulses (SMA)

Temperature Display Fingertip bed Temperature feedback

Electrorheological fluids for tactile displays

Colloidal dispersion of malleable oil and dielectric solid particulate

Oil malleability ,

Tactile display with flexible endoscopic forceps

Distal shaft of the forceps

Contact pressure sensations

at

Tactile Display Thumb, index finger, middle finger, and palm simultaneously

Tactile stimulus Sandia National Laboratories

The Tactile Mouse The Tactile mouse helps users to haptically distinguish

graphic elements such as menu options, icons, or virtual objects by making them feel different when overlapped by the mouse cursor

An example is the iFeel Mouse!

CyberTouch Glove

by Virtual Technologies Inc. provides vibrotactile feedback to the user six tactile actuators (one on the back of each finger and one in the

palm) are used to provide impulses and vibrations the vibration frequency can range from 0 to 125 Hz. Each actuator

applies a small force of 1.2 N

The Displaced Temperature Sensing System

Provides temperature feedback for virtual environment simulations Uses thermoelectric heat pumps that consist of solid-state N and P-

doped semiconductors sandwiched between ceramic electrical insulators

One of the plates is called the heat source while the other acts as a heat sink

Other Tactile Commercial Interfaces

The TouchMaster tactile interface The Tactool system

Tactile Interfaces Research Prototypes

The HAPTAC tactile interface Tactile interface prototype

The temperature display The Linear Grasper and the Planar Grasper

Tactile Interfaces Research Prototypes

The tactile interface at Karlsruhe

Programmable Tactile Array (TiNi Alloy)

The Teletact interface, University of Salford

Kinesthetic Interfaces Are devices capable of feeling and manipulating objects The majority of these devices can be classified as

exoskeleton devices, tool-based devices, thimble-based devices, or robotic graphics systems

have three main functions: measuring the movements and forces exerted by a part of the

human body, i.e., hand or fingers; calculating the effects of these forces on objects in the virtual

environment and the force response that must act on the user; and

applying the appropriate forces to the user

Commercial and Development Kinesthetic-haptic Interfaces

Product Feature Sensation VendorC

omm

erci

al In

terf

aces

Force Feedback Master Desktop Hand via joystick EXOS,Inc.( Microsoft)Force ExoskeletonArmMaster

Exoskeleton Shoulder andElbow

EXOS,Inc.( Microsoft)

CyberGrasp Force-reflecting exoskeleton: five actuators, one for each finger

Resistive force feedback Immersion Corporation

Impulse Engine 3000 Desktop Hand via joystick Immersion CorporationLaparoscopicImpulse Engine

Desktop Hand via toolhandle

Immersion Corporation

Interactor Vest Torso via vest Aura Systems, Inc.Interactor Cushion Cushion Back via cushion Aura Systems, Inc.HapticMaster Desktop Hand via knob Nissho Electronics

CorporationHand ExoskeletonHaptic Display

Exoskeleton Thumb & indexfinger joints, palm

EXOS,Inc.( Microsoft)

PER-Force 3DOF Desktop Hand via joystick Cybernet SystemsCorporation

PER-ForceHandcontroller

Desktop Hand via joystick Cybernet SystemsCorporation

PHANToM Desktop Fingertip viaThimble

SensAble Devices,Inc.

SAFiRE Exoskeleton Wrist, thumb &index finger

EXOS, Inc.( Microsoft)

Res

earc

h &

Dev

elop

men

t

Robotic Graphics Proof-of-Concept System Robotic graphics Hand via tracker Boeing Computer Services

Force and Tactile Feedback System (FTFS) Robotic graphics Throttle and joystick) Computer Graphics Systems Development Corporation

Elbow Force Feedback Display Exoskeleton Elbow joint Hokkaido

MSR-1 Mechanical Master/Slave Tool-based Active limbs MIT

7 DOF Stylus Tool-based Hand via toolhandle

McGill

Force FeedbackManipulator

Desktop Hand via joystick Northwestern University

Second Generation Master Thimble-based Three fingertips and thumb Rutgers

SPICE Robotic graphics Hand via toolhandle

Suzuki Motor Corporation

SPIDAR Thimble-based Thumb and index finger Tokyo Institute of Technology

Molecular Docking Virtual Interface Exoskeleton Shoulder andElbow

University of

Pen-Based Force Display Tool-based Fingertips or pointed object University of

The Rotary Module Can only simulate particular tasks in a one dimensional

axis (for example, opening a door with a knob that is constrained to rotate around a single axis

An example of such devices is the family of rotary modules developed by Immersion Technologies (such as the PR-3000 device

The Pantograph A 2-DOF force feedback device The Pantograph can reconstruct interactions in real-time,

creating mechanical objects with stiffness or any other physical attributes

Force Feedback Joysticks Are characterized by a small number of degrees of

freedom and produce moderate forces with high mechanical bandwidth

An example is the Wingman Force 3D joystick, developed by Logitech Co.

Wingman Force 3D joystick

The PHANTOM Family

The PHANToM used for 3D “sculpting”(courtesy of SensAble Technology

Co.)

PHANToM Omni

The force feedback system of the Phantom device

The HapticMASTER Device3 DOF cylindrical robotMax force output 250 NStiffness 50 N/mmUses force-in, position-out arrangement

The Immersion Haptic Workstation

The CyberGrasp force feedback glove

The Immersion Haptic Workstation

The CyberGrasp force feedback glove

16 N/finger (continuous?); Weight 539 grams;remote electrical actuators in a control box.

The Immersion Haptic Workstation

CyberForce interface

Quanser Haptic displays

The Haptic Wand System

• The device allows for three translation and two rotation (roll and pitch) degrees of freedom.

• It has a peak exertable force of 9N and a peak exertable torque of 810 N.mm.

• The force feedback workspace measures 48x25x45 cm3 and the rotational workspace measures 170º roll and 130º

The Novint Falcon 3-DOF device designed

originally for the gaming industry with three arms extending out of the device, with one motor connected to each arm

workspace is 12x12x12 cm rate of 1000 Hz The force torques can reach up

to 5N

Research and Development Efforts in Kinesthetic haptic displays

Examples of tension based force feedback devices developed at the Precision and Intelligence Laboratory at the Tokyo Institute of Technology. (a) A 6 DOF force feedback device with 54x54x54 cm workspace. (b) A 24 DOF device with 100x50x50 cm workspace. (c) A 5 DOF device with a 2x2x2 m workspace.

Research and Development Efforts in Kinesthetic haptic displays

The Rutgers Master II A 7 OF Stylus, at McGill University

Research and Development Efforts in Kinesthetic haptic displays

Exoskeleton for Astronauts at Vanderbilt University

Multi‐Fingred master at the Keio University 

Closing Remarks Many research and industry efforts to develop

haptic interfaces of different sizes, actuation mechanisms, and for varying applications

Medical simulators have been very influential in the development of outstandingly accurate and precise haptic devices

The gaming and entertainment industries have been focused on more robust and mechanically durable devices

متشکرم

谢谢！ありがとう！