i Access Ws8 Research on Communication

8/13/2019 i Access Ws8 Research on Communication

1/27

Research on(tele)communication tools for

people with disabilities -selected topics

Jane 2011, Antoni Grzanka, Poland


2/27

General introduction to human voice production: source of energy air flo!


3/27

lu

ngs

vocal cords

vocal tract- tongue

- jaw- nasal cavities lips

General introduction to human voice production: key elements


4/27

intensity

vibrations,intonation

- information coding

- voiceless part lips

General introduction to human voice production: functions


5/27

Speech: information layers:

physical: acoustic !ave intensity time dependant fre"uency dependant #voice spectrum$ space dependant

phonetic phonems % spoken &letters', spoken 'orthography(

phonological spoken &grammar', semantics, perception

General introduction to human voice production: information layers


6/27

fre"uency

time

inte

nsity

General introduction to human voice production: main dimensions on spectrogram


7/27

time

fre

"uency

General introduction to human voice production: formants of phonemes )e) )y)


8/27

Auditory path

*imensions of voice in human perception: ho! does the auditory system !ork+
http://c/Users/Antek/Desktop/Praca/Programy%20badawcze/iAccess/Warsztaty/opr046UA.wmvhttp://c/Users/Antek/Desktop/Praca/Programy%20badawcze/iAccess/Warsztaty/opr046UA.wmv


9/27

*imensions of voice in human perception: inaural hearing


10/27

audile fre"uency range 1-.20 000 /z

the range of the largest ear sensitivity 1000.000 /z

fre"uency range of human conversation 200.000 /zhearing threshold 0 d /

threshold of pain 110.130 d /

hearing damage 140 d /

range of perceived acoustic pressures 0,00002.-0 Pa

numer of distinct pure.tones 000

fre"uency resolution 1 /z at 1000 /z

angular resolution 1.35

temporal resolution 0604 sec

hearing loss vs age #17.40 years$ 064 d)year

hearing loss vs age #over 40 years$ 1 d)year

average hearing loss at 80 years 8 d

9rail eye.to.mouth !hen reading aloud 064.2 sec

ource: 9alice iologiczne6 praca zioro!a pod redakc;< =6 >izierskiego6 =arsza!a: Adamantan, 20036 ?@ 7.840.04.-6

*imensions of voice in human perception: acoustic channel


11/27

*imensions of voice in human perception: speech perception !ithout noise


12/27

*imensions of voice in human perception: speech perception !ith noise


13/27

>icrophone acoustic transducer to electric voltage omnidirectional microphones directional microphones arrays of microphones

Amplifier, miBers, filters conditioning of the voltageAnalog to digitalconversion on a sound card #special integrated circuit$

sampling #fre"uency C 2Dhighest fre"uency of the signal$

"uantization #7 its, 1- its, 23 its, E6$ Formatting, compression and filing to the se"uence of samples

9echnologies of speech ac"uisition y computers: from virations to digits . hard!are


14/27

Finite state language model

9ransition

pro,a,ilities

possible pronunciations data

encoding of one phone

>ohri, >6, Pereira, F6, Hiley, >6 #2002$6 =eighted finite.state transducers in speech recognition6 Iomputer peech and anguage, 1-, -776

9echnologies of speech ac"uisition y computers: speech production y computers


15/27

ource: 6 9okuda,/6 Ken, A6=6 lack: An />>.ased speech synthesis system applied to Lnglish, Proc6 2002 =orkshop on peech ynthesis, anta >onica MA

9echnologies of speech ac"uisition y computers: speech production y computers


16/27

Iomination of acoustical and video information in machine speech perception: laryngeoctomees issue


17/27

ource: H6 Pietruch, A6 Grzanka: Oo!el Hecognition of Patients after 9otal aryngectomyusing >el Fre"uency Iepstral Ioefficients and >outh Iontour #!: Journal of AutomaticIontrol$6 20106 z6 1, 206 pp6 .76

ource: H6 Pietruch6 Ph6*6 9hesis, =arsa! 200

Iomination of acoustical and video information in machine speech perception: laiograms N acoustics


18/27

determi e the positio of the eyes

Face detection

eye trackingand link detection

determine the position of the eyes

9he application developed as open source #students circle$ and promoted y largetelecommunication company in Poland

9o control the computer !ithout hands link ro!ser


19/27

1st Level 2nd Level 3rd Level448 neonatal dpt. 57 audiological dpt. 14 audiological dpt.

>odern methods of intervention in hearing prolem: screening #200$


20/27

Lmployed persons !ho,for various reasons, haveimpaired hearing, andhearing aid often onlygives them a chance to

continue !orking lder people !ithphysiological hearing lossfor !hom a hearing aidmakes it possile to

communicate !ith theenvironment

>odern methods of intervention in hearing prolem: amplification chances


21/27

?mage courtesy of =olfgang GstQttner, Mniversity of Frankfurt6

1 microphone

2 signal processor,3 transmitter, receiverI cochlear implantIo cochleaA@ auditory nerveA? auditory rainstem implantI@ cochlear nucleus

at http:))!!!6americanscientist6org)issues)num2)the.design.and.function.of .cochlear.implants)1

ource: *avis @, Happaport J>, >ac*ougall JI #Fall)=inter 18$6RIochlear and Auditory rainstem ?mplants in the >anagement of Acoustic@euroma and ilateral Acoustic @eurofiromatosisR6 >cGill Journal of>edicine #$: 1141206

>odern methods of intervention in hearing prolem: hearing implants
http://www.americanscientist.org/issues/num2/the-design-and-function-ofhttp://www.americanscientist.org/issues/num2/the-design-and-function-of


22/27

Partial deafness treatment - Polish achievement

Gradually increasing clinical material consisting of children !ith preserved residual hearing haveeen consistently presented on the international forum y the team of the ?nstitute6 !ing to thecontinuous development of our program of surgical approach to the cochlea through its round

!indo! !e have een ale, as the first team in the !orld, to single out a completely ne! group ofpatients !ith a hearing impairment, !ho have eBtant large population of ganglions in the apeB ofthe cochlea, representing normal tonotopy, thus enaling them to hear lo! fre"uency sounds,!hile eing unale to hear high sounds . eBplains Prof6 karSyTski6 uch condition of hearinghad een named the partial deafness and it is treated using cochlear implants #Partial *eafnessIochlear ?mplantation . P*I?$6 Prof6 karSyTski, as the first surgeon in the !orld, performed acochlear implantation in an adult patient !ith partial deafness in 20026 Fully satisfactory resultshad een presented in autumn that same year at the /earing Preservation =orkshop in?ndianapolis6 n follo!ing conferences of this series he presented very good results ofimplantation in further groups of patients6 atisfactory results of hearing preservation in overU of adult patients motivated the possiility of applying the same treatment to children6 First inPoland and in the !orld child !ith a partial deafness had een operated y Prof6 karzyTski inthe ?nternational Ienter of /earing and peech in a;etany in 20036 Mntil present day thehomogenous group of patients, children !ith almost 100U of hearing preservation , is constantlygro!ing6 ?t is the only such group presented in the literature in international congresses69heoretical rudiments of complementing lost aility of hearing the high fre"uency sounds !ithcochlear implants had therefore een confirmed in praBis, proving the possiility of synergyet!een residual hearing, preserved in different degrees and possily supplemented y theacoustic stimulation, and electrical hearing6

?nstitute of Physiology and Pathology of /earing: http:))pdt6ifps6org6pl)indeBVen6php

>odern methods of intervention in hearing prolem: partial deafness
http://pdt.ifps.org.pl/index_en.phphttp://pdt.ifps.org.pl/index_en.php


23/27

rain Iomputer ?nterface is an innovative iomedical technology that use rain activityto control computer e"uipment6 Iomputer can e used to control other devices and systems6

Ian use ?nvasive and @on.?nvasive methods to ac"uire signals6 Ian use different methods of rain activity measurement #i6e6 LLG, #f$>H?, 666$6 @o signal input into rain eBcept safe audio.visual feedack stimuliW IompleB analog measurement systems !orking !ith ultra.lo!.voltage iological signals6Advanced digital structure for signal ac"uisition, processing and decision making6 LBternal soft!are modules for universal measurement ) result visualization6 Mse neuro.feedack mechanisms for closed.loop control6 ?nteroperaility !ith eBisting solutions ) open.source soft!are6

=e use self designed LLG e"uipment and firm!are N pen.ource soft!are tools6 =e plan to create ineBpensive and affordale I?)@F e"uipment for home users6

L@HN

A>P?FX

A@AG9

*?G?9A

*?G?9A?G@APHIL

*LI??@N

I@9H

rain computer interface the overvie!


24/27

oft!are: pen ource #I?2000, penOiL, rainay, 66$ N our o!n #Java, Python, MniB$6

rain computer interface: studentsY applications

rain Iomputer ?nterface


25/27


/ard!are: elf designed and developed from scratch6 LBisting for comparison6

Iyernetic Hesearch tudent Group, =arsa! Mniversity of 9echnology, http:))cyer6ise6p!6edu6pl

http://cyber.ise.pw.edu.pl/http://cyber.ise.pw.edu.pl/


26/27


Iommunity: cience Festivals, Ionferences, eminars, Picnics, =orkshops, 666

Iyernetic Hesearch tudent Group, =arsa! Mniversity of 9echnology, http:))cyer6ise6p!6edu6pl
http://cyber.ise.pw.edu.pl/http://cyber.ise.pw.edu.pl/


27/27

R9he person !ho says it cannot e doneshould not interrupt the person doing it6R

? love the Ihinese prover #after o Proctor$,

pecial thanks for help to:professor =iesZa! onopka #[\d], Poland$my students:

9omasz Iedro,>agdalena Iie^licka

Documents

i Access Ws8 Research on Communication