A Conceptual Design for Non-verbal Message Transmission Ryoichi Komiya

8/14/2019 A Conceptual Design for Non-verbal Message Transmission Ryoichi Komiya

1/6

A Conceptual Design for Non-verbal Message TransmissionRyoichi Komiya

Faculty of Engineering and ScienceUniversiti Tunku Abdul Rahman, Jalan Genting Kelang Setapak 5330 KL, Malaysia

[email protected]

This article discusses the limitations of the todays telecommunication systems in terms of non-verbal messagetransmission. When it comes to transmitting direct personal messages of an emotional nature such as trust, enthusiasmor sincerity via current communication systems, it is presently virtually impossible. Human beings normally gauge anotherpersons overall personality, character, disposition, education level or other such traits when they physically meet.People sense and understand these personal attributes very quickly and emotive decisions that are subsequently made,such as; this person seems trustworthy and sincere so perhaps it would be well for me to enter into a prospective businesstransaction with him or her. However, if there is a telecommunication system between the two individuals, then whathappens to these intuitive judgments? This non-face-to-face communication limitation is the motivating force for writingthis paper.

Keywords: non-verbal messages, limitations of current telecommunication systems, sensors and actuators fornon verbal messages

1. Introduction

Multimedia has without doubt enormously contributed to enhance business efficiency. We have been able to send a widevariety of media over existing networks such as voice, text, image and video via todays personal computers. Personalcomputers nowadays are one of our most important multimedia telecommunication tools. In fact many of todays cellularphones can be referred to as mobile multimedia terminals because they function as telephones, but with e-mail, stillpicture or video transmission/reception and the Internet access. By exchanging information via multimedia channels it isnow possible to proceed business transactions far more quickly, efficiently and accurately than ever before.However, present multimedia communications content has not been able to contribute significantly to theperson-to-person communications interface in terms of transmitting trust and other such emotions [1].This paper firstly discusses the importance of non-verbal messages in our daily lives and then describes the possibility ofimplementing sensors and actuators to pick up, transmit and regenerate non-verbal messages between the calling andcalled party sites. At the end of the discussion, a conceptual design for a non-verbal telecommunication terminal isproposed.

2. Human to human understanding components in face to face conversation

In daily life humans exchange large amounts of non-verbal information during face-to-face contact. However, when weexchange ideas and opinions by making use of telecommunication systems or the Internet, it is oftentimes very difficult tocommunicate precisely or thoroughly. Why does this happen? According to the data, we get many different types ofinformation during face-to-face conversations, typically; 7% of the information is imparted by direct speech, 38% fromspeech tone and the remaining 55% comes from non-verbal messages such as gestures, posture, facial expression,accessories and clothing and other such personality indicators [2].These emotive messages are referred to as non-verbal messages and are the principal communication mechanisms weall use to understand, evaluate, judge and ultimately get to know a persons unique and individual underlying personalityand character.Humans understand each other most precisely through these non-verbal messages. We use them to understand eachother and even after just a few short word exchanges during a first meeting we use them to decide, for instance, whetherthere is the possibility or not or working well with someone in the business arena. In relationships between males andfemales, the same can be applied. Videophones or multimedia terminals such as personal computers were developed

with the expectation of better sending and receiving these non-verbal messages, but they have not been successful.Of course for the direct exchange of information the telephone and Internet have contributed greatly to the benefit ofbusiness globally and it could be said that for data, graphic and pictorial transmission, provide almost perfecttelecommunication tools.However, current multimedia terminals such as network connected personal computer have not been able to contribute tonon-verbal message exchange at all.The importance of non-verbal message exchange not only makes it an essential part of personal discussions, but also incommercial activities. Because, as in all other spheres of human activity, in business we need to establish trust as well asother emotional (controlled) transmissions, for example; talks between a medical practitioner and a patient, conversationsbetween a banker and his client or price negotiations between a trader and a consumer.

3. Non verbal messages and their significances

Non-verbal behavior has been well studied and findings published [3]. In this section the high level presentation ofnon-verbal messages are summarised in Table l encompassing; physical appearance, clothing, gesture/posture, facial

expressions to vocal tone.From Table 1 we can see that human beings assimilate a wide variety of human character traits and attributes throughthese non-verbal messages when they meet. It is clear that even following the invention and widespread usage of the

Proceeding of the 3rd International Conference on Informatics and Technology, 2009

Informatics '09, UM 2009 RDT3 - 63


2/6

telephone, facsimile and Internet, the need for and number of face-to-face meetings has not been significantly reduced inthe areas such as business, science and technology or international diplomatic negotiations. Due to the increasing needfor these face-to-face meetings and negotiations, business trips have tremendously increased nationally as well asinternationally [4]. This increase in travel of course works diametrically against initiatives such as efforts to reduce globalwarming, to reduce in energy consumption and other such similar environmental issues facing the world.If we accept that non-verbal transmission in telecommunication systems would contribute to reducing the necessity of

travel for face-to-face meetings, then an individuals work mode and lifestyle would change significantly making theprospect of a telecommunications system capable of exchanging both verbal and non-verbal messages a very desirablefuture tool for mankind.Based on Table I, further discussion will be conducted in the following sections.

Table l Non-verbal messages and their significancesNon-verbalmessages

Non-verbal messagecomponents

Significances

Physicalappearance

Body type

Height

Skin colorHair Color

Hair length

Overweight: Slow, sociable, emotional, forgiving, relaxedMedium sized: Confident, energetic, dominant,enterprising, hot-temperedSlim: Tense, awkward, meticulous, tactful, detachedMajority of executives in Fortune 500 companies were morethan 180 cmNo dataGold: funBrown: wholesomeRed: tempestuousBlack: sultryLong: less serious (sexy, rebellious)Short: serious

Clothing When we know someoneWhen we do not know the realperson

His/her clothing has little influence on our perceptionsThe clothes of strangers are a rich source of information aboutthem when no other source exists.

Gesture andmovement

Various gestures and movement Together with facial expressions, a persons posture, gait, limbmovement and other behavior provides information about hisor her emotional state or mood.

Facial behavior 1.Eye brows and forehead2.Eyes and eyelids

3.Lower face (cheeks, nose andmouth)

Sadness and fear: 1Anger: 1&3

Disgust often blends with surprise: 1,2,&3Fear: 2Happiness: 2&3Surprise: 1,2 & 3

Eye behavior SalienceStimulate arousalInvolvement

Eliciting the attention of othersLike/dislikeEye contact

Vocal behavior Vocal qualities

Vocalization-vocal characterizer-vocal qualifierVocal segregate

Tempo, resonance, rhythm control, articulation controlpitch control, glottis control, vocal lip control, pitch range-non-language sound: laughing, crying, whimpering, giggling,snickering, sobbing-intensity, pitch height, extentuh-huh, er, ah, and-ah, you know

3.1 Sensors required to pick up non-verbal messages

We understand that almost all, except vocal behavior, of the non-verbal messages emitted by humans as in the Table 1can be picked up by visual sensors. The most prospective sensor would be the video camera. However, it needs to beascertained which standard video cameras offer the optimum ability to pick up visual non-verbal messages accurately andnaturally.For vocal non-verbal messages it is comparatively easy to collect and transmit by microphone(s).Therefore, essentially all the necessary sensors to pick up non-verbal messages may not be different from theconventional videophone. However, videophones have not to date been generally well accepted by the public and have ahistory of successive commercial catastrophes all over the world.Let us then review the history of the videophone a little in the section below.

4 History of videophone R&D- Its successive failures-

4.1 AT&T [5]

1927: A one-way system was demonstrated in a conversation between Secretary of commerce Herbert Hoover, andsenior officials of AT&T.

1930: A two-way video link was set up between the headquarters in New York of Bell laboratories and AT&T using the




3/6

then latest TV equipment within a closed circuit.1956: Bell laboratories develop a videophone system which could work over the standard telephone network.1971: Bell laboratories completed the R&D of the Picturephone.

4.2 The R&D outcomes of the Picturephone

The Picturephone was a dream of Bell laboratories R&D group because at that time (1971) it was able to handle videoimage as well as voice message exchanges leading them to believe that the Picturephone would become the ultimatesolution to human to human network based contact, the ultimate telecommunication tool.However, because of the restrictions of the existing network infrastructure; such as subscriber loops, transmission andswitching systems, the picture size could only be very small (less than 13 cm by 13 cm) and only in monochrome, and theoverall quality was such that it was never really accepted by the general public at that time.Since that time relevant technologies have improved, especially in terms of video signal compression.However, basic features of the videophone in general have not changed from those of Picturephone, even though thefunction has been commonly implemented in the modern laptop computers. The usual setup is for a video camera to belocated above computer video screen, but it only caters for the persons head and shoulders image and if a calling partymoves out of the shooting range of the video camera, the called party can not see the calling party any more. This hasbeen a specifically disadvantageous aspect of the videophone because the verbal telephone conversation can bemaintained so long as both parties hold the handsets, effectively combining the incompatible elements of one stationaryand one mobile sensor. Hence it should be stressed that one of the most important advantages of the videophone shouldbe the ability to maintain eye contact between the talking parties. The limitation arises because of the physical alignment

between the camera and the screen. There are, however, on the market nowadays videophone systems essentiallysolving the eye contact issue [6].Today, videophone capability in cellular phones has contributed significantly to on site video news gathering by thegeneral public [7].Regardless of these developments, current videophone features do not offer enough flexibility to adequately exchangehuman non-verbal messages.

4.3 Technology assessment of the Picturephone from the view point of non-verbal message exchange

Now, let us think about why the present Picturephone design philosophy is unable to match the features required toadequately communicate non-verbal messages. In relation to the technology assessment of the Picturephone, which wasconducted by Cornell University some 35 years ago [8], it was their view from their analysis results that limitations intechnology were why the Picturephone had not been well-accepted by consumers. However, their assessment methodcan be adapted and applied to the modern technology case.Table ll indicates the results of the assessment.Based on the assessment below we can clearly understand why the lack of adequate functions of the Picturephonerendered it unsuitable from the perspective of non-verbal message transmission.

5.0 The intended terminal configuration of non-verbal message transmission

5.1Use cases

The use cases of non-verbal message transmission are as follows

5.1.1Business scenario

(1)Where trust transmission is required: ie- conversations between a MD and a patient, conversations between lawyersand clients, conversation between bank officers and customers, recruitment interviews, etc.(2) Where enthusiasm transmission is necessary: ie- presentation of a sales talk/pitch (real world/cyber world) on specific

items, recruitment interviews, etc.(3) Where entire human character transmission is necessary: ie- any negotiations; including diplomatic commercial, orsimilar issues, recruitment interviews, etc.(4) Where confidence, precision and accuracy transmission is necessary: ie- teaching activities, political campaigns,technical briefings, recruitment interviews, etc.

5.1.2 Personal/private scenario

(1) Talks between family members(2) Chats between boyfriend and girlfriend(3) Guidance from boss to subordinate (Small Office Home Office application)

5.2 Functional design

5.2.1 Necessary functions for non-verbal message transmission

(1) Video Sensing/Actuating: to shoot/regenerate visual non-verbal messages(2) Audio Sensing/actuating: to pick up/regenerate audible non-verbal messages




4/6

(3) Video/Audio sensing control: to control video/Audio sensing to and from both calling and called parties to maximizesensitivity of shoot/pick up visual/audio non-verbal messages.

(4)Human position detection: to detect the human position in a room to operate Video/Audio control accurately(5) CODEC with bandwidth reduction: to digitize sensed video/audio signals together with appropriate bandwidth

reduction algorithm(6) MUX/DEMUX: to multiplex/de-multiplex digital data stream

(7) TRANSMIT/RECEIVE: to transmit/receive to/from line interface

Table II Technology assessmentNon-verbalmessage

Non-verbal message components Basic features of Picturephone

Physicalappearance

Body type (overweight, medium,slim) including height

Skin color and hair color/length

Body type and height can not be captured by thePicturephone because its video camera has beendesigned to shoot only a persons head and shouldersimage.Depends on the performance and lighting conditions of thevideo camera

Clothing Clothing including accessories The video camera of Picturephone has no capabilities tocapture. Only limited image can be shot.

Gesture andmovement

Head, torso movementHands and arm movementLegs position on chairGaits

Can be captured with limitation (head and shoulders)Only the limited portion can be shotImpossibleImpossible

Facial behavior 1.Eye brows and forehead2.Eyes and eyelids3.Lower face (cheeks, nose and mouth)

Distinctive facial behavior can be captured

Eye behavior Eye contact Impossible because of the video camera and screenalignment, One manufacturer currently offers a videophonewhich can maintain eye contact.We still do not know the actual resolution required foraccurately capturing eye behavior.

Vocal behavior Vocal qualities

Vocalization-vocal characterizer-vocal qualifierVocal segregate

This can be captured by microphone with the bandwidth of 0.3 Hz-3.4 kHz (telephone audiocharacteristics)

We have to know the frequency range of the vocalization

As above

5.2.2 A functional configuration of non-verbal message transmission

The functional configuration of the non-verbal message transmission is illustrated in Fig. 1.This figure only shows the calling partys side configuration. The called partys side functional configuration is identical tothis figure. Functional interface reference points are defined by arrows between each functional block.Detailed functional blocks are presented in Table III and functional interface characteristics are indicated in Table IV.

Fig. 1 Functional configuration of non-verbal message transmission system (calling partys side)

VideoSensing/Actuating

AudioSensing/Actuating

Video/audio sensing control Human position detection

Codec with bandwidth reduction

MUX/DEMUX

Transmit/Receive

A

B

C

D

E

F

G




5/6

Table III Description of functional blocks

Functional block Description Candidate hardwareimplications

Video Sensing/Actuating Video sensing: to shoot visual non-verbal messagesVideo actuating: to regenerate visual non-verbal

messages

Video camera/Display(scanning line resolution to

be studied)Audio Sensing/Actuating Audio sensing: to pick up audible non-verbal

messagesAudio actuating: to regenerate audiblenon-verbal messages

Microphone/Receiver(audio bandwidth to bestudied)

Video/Audio sensing control Video sensing control: to control video sensing to/from both called/calling parties to maximizesensitivity to shoot visual non-verbal messagesAudio sensing control: to control audio sensingfrom both calling/called parties to maximizesensitivity to pick up audio non-verbal messages

Video camera control(zoom in/off, pan, tilt)

Microphone sensitivitycontrol

Human position detection To detect the human position in a room to operateVideo/Audio control accurately

Floor pressure sensorVideo image sensor

CODEC with bandwidth reduction

MUX/DEMUXTRANSMIT/RECEIVE

To digitize sensed video/audio signals together

with appropriate bandwidth reduction algorithmTo multiplex/de-multiplex digital data streamTo transmit/receive to/from line interface

MPEG 4 CODEC

Packetizer/De-packetizerLAN interface card

Table IV Interface characteristics of the Functional Interface Reference Point (FIRP)

FIRP Direction of theinformation flow at

FIRP

Information to be exchanged on the FIRP

A bidirectional Downstream direction: video signal from the called partyUpstream direction: video signal form the calling party

B bidirectional Downstream direction: audio signal from the called partyUpstream direction: audio signal form the calling party

C unidirectional Camera control signal from the called partyD unidirectional Microphone control signal from the called party

E unidirectional Human position indicating signal to control video/audio sensing controlF bidirectional Audio/video sensing control from both calling and called partyG bidirectional Downstream/Upstream multiplexed digital data

5.3 Basic design philosophy of an experimental non-verbal message transmission terminal

Based on the functional configuration of the non-verbal message transmission system, we are now trying to design anexperimental non-verbal message transmission terminal.Basic design philosophy of the terminal is as below;(1) The terminal should be considered as the next generation POTS (Plain Ordinary Telephone Service)(2) The terminal should firstly start as a fixed line services (office room, living room applications)(3) The terminal is to be designed to meet universal requirements for various business and personal communications(4) The terminal should be implemented by combining, where possible, existing consumer products(5) The video image and audio quality should be adequate to transmit non-verbal messages accurately and naturally.(6) The terminal should contribute to overall telecommunication services innovation by making use of NGN [9].(7) The terminal should contribute to a decrease in business trips reducing carbon dioxide emissions.

The actual implementation will be preceded by testing and evaluating the quality of non-verbal message transmission.

6. Conclusions and future works

Non-verbal messages such as physical appearance, body movement, clothing (including accessories), facialexpressions are all very important to fully ascertain a persons emotional state and even his/her background. In the caseof face-to-face conversations, we collect these non-verbal messages intentionally or non-intentionally, but based on thesemessages, we understand or determine almost instantaneously whether we are going to get on well with anotherindividual or not. Because human beings can understand much of another persons background by directly conversingwith them they instinctively receive and accept non-verbal messages of trust, enthusiasm, sincerity and other emotions inthe people they are confronting. However, where there is a telecommunication system (telephone, Internet or even videotelephony) of current technological capability in between the two persons it is difficult if not impossible to receive thesenon-verbal signals. This paper proposes the possibility of accurate non-verbal transmission over the existing network bycombining currently available technology and products.




6/6

REFERENCES

[1] Ryoichi Komiya, How multimedia/multimodal information Can Contribute to the Change of Telecommunication?,In Proceedings of Eighth IEEE International Symposium on Multimedia 2006, San Diego, pp.774-776.

[2] Albert Merabian,http://en.wikipedia.org/wiki/Albert_Mehrabian.

[3] Virginia P. Richmond, James C. McCROSKEY, Nonverbal Behavior in International Relations, Fourth Edition, Allyn andBacon, 2000.

[4] Research: U.S. Business Trips To Fall Through 2012 While Spending Riseshttp://www.management.travel/news.php?cid=Global-Insight-TIA-forecast.Dec-08.04.

[5] Videophonehttp://en.wikipedia.org/wiki/Videophone.

[6] THE EYE CATCHERhttp://greeniii.com/productContent.php

[7] Internet plays key role in Uighur unresthttp://www.taipeitimes.com/News/world/archives/2009/07/09/2003448178

[8] Edward M. Dickson, Potential Impacts on the Video Telephone, IEEE TRANSACTIONS ON COMUNICATIONS,VOL.COM-23, NO. 10, OCTOBER 1975, pp. 1172-1176

[9] Next Generation Networks Global Standards Initiativehttp://www.itu.int/ITU-T/ngn/index.html

BIOGRAPHY

Ryoichi KOMIYA was born in Tokyo, Japan on March 16, 1945. He received his B.E. and Ph.D. degrees from WasedaUniversity, Tokyo, Japan, in 1967 and 1986 respectively. Joining the Electrical Communication Labs of NTT in 1967, hewas engaged in the development of the PCM repeatered line, digital data terminal equipment, video coder/decoders, stuffmultiplexers, ISDN subscriber loop transmission systems and fiber optic remote multiplexer systems.After leaving NTT in 1992, he worker with Siemens, Nippon Telecommunication Consulting Co. Ltd., NTT AdvancedTechnology and the Distribution and Economics University of Japan.He lectured at the Faculty of Engineering and Faculty of Information Technology at Multimedia University, Malaysia from1998 to 2009, where he was responsible for research and development of next generation telecommunication systems,services, terminals, IP networks, virtual education environment, e-commerce terminal, Intelligent TransportationSystVirem, Fiber to The Home and Smart Home programmes.He also worked at the National Institute of Communications Technology, Japan from 2006 to 2008, where he studiedConnected Health Systems using Body Area Network (BAN) systems.From April, 2008 he worked with the Regional Disaster Information System Labs at Iwate Prefectural Univesity in Japan.In June 2009, he joined University Tunku Abdul Raman, Malaysia, where he is conducting Research & Development onICT applications to transport system, medicine and non-verbal communication systems.Professor Komiya is a member of IEEE, ACM and IEICE.



Documents

A Conceptual Design for Non-verbal Message Transmission Ryoichi Komiya