Multimedia and computertechnologies have becomea part of our everyday lives

in many ways. Before buying newfurniture, we can view it on a com-puter screen from multiple perspec-tives in a virtual living room. Alsoimpressive are the virtual recon-structions of famous historical build-ings, in which computers and multi-media give the user the illusion ofwalking through historical roomsthat no longer exist in reality. Multi-media computer games enjoy greatpopularity – and not only amongyoung people – such as auto racing“à la Michael Schumacher”. How-ever, with the constant improve-ment of these multimedia systems,they often begin to show their defi-6

Getting a Grip onthe Computer WorldIn computer games or other multimedia applications,the user receives only visual and audio sensations.New touch displays help to transmit tactile sensations

german research 1/2004

is achieved with a two-sensor mag-netic gradiometer. To produce mapsof the magnetic-field distribution,the patient is placed in various posi-tions under the sensor and a shorttime sequence of the magnetocar-diogram is recorded. From this dataseries a map of the field distributioncan be generated for any point intime during the heartbeat.

In many cases the magnetocardio-gram supplies information equiva-lent to that of an ECG, but in recentyears clinical research has identifieda variety of diseases in which theMCG is clearly superior to the ECG.This is particularly evident in casesin which the ECG looks normal, de-spite the presence of a cardiac prob-lem, but where the MCG exhibits ir-regularities. These include, for ex-ample, Wolff-Parkinson-White syn-drome, as well as the hours immedi-ately after a myocardial infarction.Furthermore, the occurrence of cir-cular currents in the heart, whosepresence is suspected in the case ofcertain severe cardiac arrhythmias,for physical reasons does not con-tribute to an ECG but does to theMCG. A magnetic map also provedmore suited for locating arrhythmo-genic centres in the heart musclethan an electrical map, which is im-portant prior to surgical intervention.

So far, the advantages of magne-tocardiography have only beenbrought to bear in a few select high-tech medical centres because thetechnical, logistical, and financialburden of installing and operating aSQUID-based system is too heavyfor an ordinary hospital or cardio-logical practice. With the new tech-nology, this expense is reduced tosuch an extent that magnetocardio-graphy will also be affordable andpractical for medical practices andremote hospitals. Since magneto-cardiography is essentially contact-free, public screening for heart con-ditions will be possible once the sys-tem has become market-ready. Thepotential advantages to generalhealthcare are clearly apparent.

PD Dr. Robert WynandsDipl.-Phys. Georg BisonProf. Dr. Antoine WeisUniversity of Fribourg/Switzerland

Engineering Sciences

ciencies in terms of delivering highlevels of realism. With its monitorand speakers, a computer canpresently stimulate only two of thefive human senses. Other sensa-tions important for “grasping” theworld, conveying a sense of motion,force or touch, currently cannot beemulated satisfactorily or at all. Fre-quently, however, it is exactly thesehaptic (relating to the sense oftouch) perceptions that turn out tobe key for a comprehensive, realisticimmersion into virtual, computer-generated worlds and environ-ments.

Customers would certainly ap-preciate not only being able to seetheir future furniture on the screenbut also to feel its surface texture or

possible to perform more realistic ac-tions in virtual environments or evento perform teleactions, for examplevia the Internet, across distances.

What does a computer require inorder to give the user of a multi-modal system haptic sensations inaddition to visual and audio stimuli?First it must comprise display de-vices that are capable of adequatelystimulating the haptic sensory chan-nels. Tactile sensations on the fin-gertips can be conveyed, for exam-ple, by miniature vibration elementsor using pincushion-like displayscomposed of numerous individualmoving rods.

Robot-like mechanical struc-tures can be used to elicit sen-sations of force on the fingers,

hand and elbow. In addition, specialsoftware must be prepared so thatthe computer can generate the spe-cific haptic stimuli on the displaysand coordinate them with one an-other. Despite certain similaritieswith high-quality graphics softwarein the way it can generate three-di-mensional visual sensory impres-sions by computer, correspondinghaptic computer programs stillplace high scientific demands onthe respective modelling, program-ming and computer technology. Inthe end, the human organ, such as a

finger or a hand, interacting withthe virtual environment, must beblended in the computer image as atype of phantom, often called anavatar. This ensures that all visual,audio and haptic information trans-mitted by the computer and per-ceived by the user conveys a harmo-nious overall experience – which isthe crucial factor for realism.

A few examples of the applica-tions developed and executed at theMunich Collaborative ResearchCentre show that the “tactile simu-lation” achievable through multi-modality allows a considerablydeeper and more comprehensivegrasp of computer-generated worldsand objects:

Shortly before the introduction ofthe euro, it was possible not only tovisualise the as-yet-physically-un-available one-euro coin, but also toallow one to feel it with the finger-tips using a haptic coin model andtouch displays. Similar techniques 7

The technology makes it possible to see a phantom hand on the screeninserting a virtual radio into adashboard while controlling themovements and feeling the objectwith one’s own hand.

german research 1/2004

test the quality of the upholstery.Visits to a virtually reconstructedhistorical building could be madesignificantly more realistic if visitorscould run their fingers along the ar-chitectural details and actually haveto exert the force necessary to openor close a door. And of course thesame applies for computer games.Computer game manufacturershave recognised the importance ofhaptic sensations, which is why theysell joysticks and steering wheelsgiving the player authentic tactilesensations in addition to the familiaraudio and visual sensory input.

These examples indicate a recog-nisable trend in the human-comput-er interface – from today’s multime-dia systems to so-called “multi-modal” systems, in which the userreceives multisensory stimuli andcan act and respond by variousmeans. This vision is also at the cen-tre of the research activities of theCollaborative Research Centre“High-Fidelity Telepresence andTeleaction”, located in the Municharea. Telepresence refers to a feel-ing of being physically present withall relevant human senses in a com-puter-generated, virtual world, or ina real environment that is not direct-ly accessible to the person. The in-clusion of the haptic sensory modali-ty in telepresence systems makes it

can also be beneficial when used inmedical training. For example, it ispossible to practice palpating theabdominal wall to locate a beatingartery or to detect a tumour.

Another application is associatedwith “virtual prototyping”, that isproduct develop-ment on the com-puter in the auto-motive or otherindustries, suchthat users cannotonly see and heara car radio in avirtual instru-ment panel, butcan also touch it.By means of a complex hand/armdisplay, they can feel the weight ofthe radio when lifting it, experiencethe smoothness or roughness and

the temperature of the housing andfinally even feel the force requiredto insert it in the instrument panel. Arecently developed mobile hapticdisplay lets users take a walkthrough an extended virtual artgallery, while actually moving

through a con-fined area of alaboratory or, infuture, even intheir own homes.Users can notonly view worksof art such aspaintings orsculptures fromall angles, but

can even touch them and hold them,which as we all know is rarely possi-ble in a real gallery.

At present, however, the inten-sive international research effortshave not yet progressed far enoughthat affordable, mass-market hapticdisplay products will be on the hori-zon yet in the near future. The ap-proaches to multimodality de-scribed here have, however, al-ready proven themselves time and

8

A long-term researchgoal is to enable amore comprehensive andintensive perceptionof virtual environments

again in specific applications, thusdemonstrating a broader applica-tion potential. They will increase inimportance in the near future, forexample in intensifying medicaltraining by use of multimodal organor patient simulators, as well as for“rapid prototyping” in digital prod-uct development. Another area thatwill benefit from multimodality isteleshopping, in which customerswill one day also receive haptic sen-sations of the products whilst brows-ing through a virtual catalogue. Fur-thermore, there are also possibilitiesfor the haptic exploration and ma-nipulation of nanotechnological ormolecular biological structures.

In the longer term, it is altogetherconceivable that computer-generat-ed multimodality will one day alsoextend to other sensory perceptions,such as taste or smell. This wouldmake it possible to experience virtu-al or real remote environments evenmore comprehensively and inten-sively.

Prof. Dr.-Ing. Günther SchmidtTechnische Universität München

A mobile haptic display lets users take a walkthrough an extended virtual art gallery,

while actually moving through a confinedarea. Users can not only view works of art

such as paintings or sculptures from allangles but can even touch and hold them.