Haptics Technologies:Bringing Touch to Multimedia

C1: Haptics: General Principles

Roadmap of the course/book Haptic Applications (Chapter 2) General Principles Human Haptic Perception (chapter 3) Haptic interfaces and hardware (chapter 4) Computer Haptics (chapter 5) Multimedia Haptics (chapter 6) Summary, conclusions, and future directions (chapter 7)

Introduction Vision

Via human eyes and most information rich media Humans perceive between 0.3 and 0.7 microns wavelength Temporal and spatial limitations define visual displays

Audition Via human ears and very effective Humans perceive 16 Hz to 20000Hz (1000Hz and 4000Hz) Auditory interfaces are very mature (2D and 3D sound effects)

Touch Distributed all over the human body, up to 1 KHz sensitivity Tactile and kinesthetic Bi-directional flow of information and energy

Haptics explorationHaptics refers to the science of manual sensing (exploration for information extraction) and manipulation (for modifying the environment) through touch

Force representation in a virtual world

1. The user manipulates the generic probe of the haptic device,

2. the device tip position is read and conveyed to the computer.

3. the computer checks for collisions between the simulated stylus and the virtual objects populating the virtual environment.

4. If a collision has occurred, reaction forces/torques will be computed and applied.

5. In the case that no collision is detected, no forces will be computed/applied

Concepts and Terminology -1- Haptic: the science of applying tactile, kinesthetic, or both

sensations to human-computer interactions. It refers to the ability of sensing and/or manipulating objects in a natural or synthetic environment using a haptic interface.

Cutaneous: Relating to or involving the skin. It includes sensations of pressure, temperature, and pain.

Tactile: Pertaining to the cutaneous sense, but more specifically the sensation of pressure rather than temperature or pain.

Kinesthetic: relating to the feeling of motion. It is related to sensations originating in muscles, tendons and joints.

Force Feedback: Relating to the mechanical production of information that can be sensed by the human kinesthetic system.

Concepts and Terminology -2- Haptics or haptic technology: an emerging interdisciplinary field that

deals with the understanding of human touch sensory (Human Haptics), motor characteristics (Machine Haptics) and with the development of computer-controlled systems (Computer Haptics)

Haptic communication: the means by which humans and machines communicate via touch

Haptic device: is a manipulator with sensors, actuators, or both. The most popular are tactile-based, pen-based, and 3 degrees of freedom (DoF) force-feedback devices; more degree of freedom devices exist.

Haptic interface: consists of a haptic device and software-based computer control mechanisms. It enables human-machine communication through the sense of touch.

Haptic perception: the process of perceiving the characteristics of objects through touch

Concepts and Terminology -3- Haptic rendering: the process of calculating the interaction force. Haptic

rendering is therefore a system that consists of three parts, a collision detection algorithm, a collision response algorithm, and a control algorithm.

Sensors and Actuators: A sensor is responsible for sensing the haptic information exerted by the user on a certain object and sending these force readings to the haptic rendering module. The actuator will read the haptic data sent by the haptic rendering module and transform this information into a form perceivable by human beings.

Tele-haptics: the science of transmitting haptic sensations from a remote explored object/environment, using a network such as the Internet, to a human operator.

Tele-presence: the situation of sensing sufficient information about the remote task environment and communicating this to the human operator in a sufficiently natural way.

Concepts and Terminology -4- Virtual Reality (VR): the computer simulation of a real or virtual world

where users can interact with it in real time and change its state to increase realism.

Virtual environment (VE): is an immersive virtual reality that is simulated by a computer and primarily involves audio-visual experiences. A virtual environment is mainly concerned with defining interactive and virtual image displays.

Collaborative Virtual Environments (CVE): is one of the most challenging fields in VR where the simulation is distributed among geographically dispersed computers.

Simulation Engine: is responsible for computing the virtual environment behavior over time.

Collaborative Haptic Audio Visual Environment (C-HAVE): C-HAVE allows multiple users, each with his/her own haptic interface, to collaboratively and/or remotely manipulate shared objects in a virtual or real environment.

Path 1 - Human Haptics The study of human sensing and manipulation

through tactile and kinesthetic sensations

Includes understanding The human sensory system The human perception and cognition system The human motor system

Path 2 - Machine Haptics Involves designing, constructing, and developing

mechanical devices that replace or augment human touch

Measure the poses (positions and/or orientations) and/or contact forces of any part of the human body, and display the computed reaction touch

Divided into two categories: force feedback devices and tactile devices. Force feedback devices display force and/or torque and enable

users to feel resistive force, friction, roughness, etc. Tactile devices present vibration, temperature, pressure, etc. on

the human skin

Path 3 - Computer Haptics Design and development of algorithms and software that computes

interaction forces and simulates physical properties of touched objects

Haptic rendering:

Haptic rendering refers to the set of algorithms and techniques that are used to compute and generate forces and torques in response to interaction between the haptic interface avatar inside the virtual environment and the virtual objects populating the environment

Two types of algorithms: collision detection force computation algorithms

The Roundabout - Multimedia Haptics

Considers Haptics as a new media channel in a complete multimedia system

Defined as: the acquisition of spatial, temporal, and physical knowledge of

the environment through the human touch sensory system and the integration/coordination of this knowledge with other sensory displays (such as audio, video, and text) in a multimedia system

Comprises Haptic media creation, storage, compression, dissemination,

representation, communication (transporation) and security (biometrics)

Haptic-Audio-Visual Multimedia System

Case Study: HugMe interpersonal Communication System

1. The child is wearing a haptic suit (haptic jacket) that is capable of simulating nurturing physical stimuli.

2. The parent uses a haptic device to communicate his/her feelings to his child.

3. A 2.5 dimensional (2.5D) camera is used to capture the image and depth information of the child and send it to the parent

Case Study: HugMe interpersonal Communication System

Human in the Middle!Your child is crying

You can soothe him hugging and tickling

What can we do for this poor child???

When you are away,You cannot reach your hands to hug and tickle

Haptic Teleconferencing -HugMe

With a haptic deviceyou can touch him

With a haptic jackethe can feel touch

How it works?Capture 3D(2.5D) video

Using a depth camera Depth image Color image

Reconstructed 3D scene

How it works? Demo Video

Haptic jacket

Controller with Bluetooth and 9V battery

Fiducial marker for tracking

Pager motor Inner fabric to protect the skin

Further Reading about Haptics

In recent years, there has been extensive research literature on all aspects of haptic systems. Some journals, such as IEEE Transactions on Haptics, IEEE Transactions on Instrumentation and Measurement, ACM Transactions on Graphics, ACM Transactions on Applied Perception, ACM Transactions on Multimedia Computing, Communications and Applications, MIT Press Presence, Springer Multimedia Tools and Applications, Springer Multimedia Systems Journal, and the Electronic Journal of Haptics Research “Haptics-e” frequently publish haptics-based research results. Many other journals, such as Journal of Robotics and Mechatronics, have special issues on this subject.

متشکرم

谢谢！ありがとう！