An Attempt to Detect Hearing Disorders by Acoustic Features of the Infant Cry

György Várallyay Jr. Budapest University of Technology and Economics, Dept. of Control Engineering and Information Technology, Laboratory of Medical Informatics,

H-1117, Budapest, Magyar tudósok krt. 2., Hungary, varallyay@bio.iit.bme.hu,

András Illényi Budapest University of Technology and Economics, Dept. of Telecommunications and Media Informatics,

H-1117, Budapest, Magyar tudósok krt. 2., Hungary, illenyi@tmit.bme.hu,

Zoltán Benyó Budapest University of Technology and Economics, Dept. of Control Engineering and Information Technology, Laboratory of Medical Informatics,

H-1117, Budapest, Magyar tudósok krt. 2., Hungary, benyo@iit.bme.hu,

Zsolt Farkas, Gábor Katona Heim Pál Hospital for Sick Children, Laboratory of Medical Informatics,

H-1089, Budapest, Üllői út 86, Hungary, drfarkaszsolt@yahoo.com, katonag@heimpalkorhaz.hu

Most of the hearing disorders might be treatable in the early age of life. There are variant methods to test the hearing abilities of an infant; still none of them cover the whole auditory system by itself. Thus a comprehensive audiometry with high hit probability is required. Crying is the first tool of communication for an infant. Emotions, state of health or several organ functions can be obtained from it. Normally, an auditory feedback works between the produced sound and the brain. When this feedback has problems, crying may differ from the original one. Authors are to find these differences by acoustic analysis. This paper deals with acoustic features of the infant cry in order to detect hearing impairments.

1 Introduction

1.1 The infant cry

Crying is an important information source about the infant. It is a doubtless sign of life at birth. Investigations dealing with the infant cry can be classified into three main categories: investigations of the reason of crying, investigations of the development of crying and investigations of the connection between diseases and crying. Infant cry is a mean of communication. The sound of crying is in the most sensible range of the human auditory sensation area, in order to let the babies express their needs. Crying is different if the baby is hungry, sleepy, bored or need to be changed. Several research teams had been investigated the reason of crying, they tested mostly acoustical and psychological respects [1-4]. The second main category is the investigation of the development of crying. The sound of crying has a development in the same way as the infant itself [5-7]. The cry of preterm infants is acoustically different from the normal term babies in the first three months of life [8]. According to Zeskind et al. the sound of crying is characteristic to the birth process [9]. If there are complications at birth the sound of crying is much sharper and shorter; the time of the pause between crying blocks is longer than they are at normal birth [10]. Crying can also refer to emergency. E.g. normal term infants kept in incubator observably decrease the

volume of their crying, while normally it should be increased [11]. Last but not least, the infant cry can be an important aid in medical diagnostics. Experienced otorhinolaryngologist paediatricians use their impressions from the movements and crying of the infant in the diagnostics. E.g. infants suffers from Mongolism have a very weak sound of crying [12, 13].

1.2 Hardness of hearing

Approximately 0.2-0.3% of the infants is born with at least a medium grade permanent hearing loss (the hearing threshold is above 40 dB) [14]. A baby learns to speak by hearing its environment. It is essential for the correct development of speech to have good hearing in the critical period. This critical period (i.e. in the case of the development of the organ of speech and organ of hearing) is between the first 6 and 24 months of life. There are further repercussions of deafness in the physical, psychic or intellectual development. The hearing system is a complex system, which has numerous elements cooperating together. The main elements are: the pinna, the ear canal, the ear drum, the auditory ossicles (malleus, incus and stapes), the oval and round windows, the Eustachian tube, the cochlea, the nerve fibers and the central nervous system. If any of these elements has functional disorders, hardness of hearing will occur.

227

Forum Acusticum 2005 Budapest Várallyay, Illényi, Benyó, Farkas, Katona

Hearing disorders can be cured in many cases, if they are detected in the first few months of life.

1.3 Basic hypothesis

There is an auditory feedback between the organ of speech and the brain, which instructs the organ of speech to create sound [15]. If this feedback is missing, the created sound may differ from the normal sound. Our basic hypothesis is the following: infants who have hearing disorders or hearing loss cry unlike healthy infants. Our primary goal is to detect deafness by analyzing the sound of crying. By means of the above mentioned auditory feedback the whole hearing system could be tested on this wise. There are several parameters which can be obtained from crying, as length of the crying blocks, as fundamental frequency, as spectral components, as melody contour of crying, as rising time, etc [16]. Authors have already tested several parameters of crying, in this study the rising time of crying will be investigated between healthy and deaf (partly or totally) infants. The melody contour (which correlates with the rising time) is an important parameter to determine the mood of the infant [17, 18]. Authors want to examine, if there is any connection between deafness and the rising time (which is part of the melody). The definition of the rising time will be given later.

2 Data collection

Recordings were mainly made in the Heim Pál Hospital for Sick Children in a quiet room. Authors used a digital camera (SONY DCR-TRV25) in order to recognize the infant and the circumstances of crying. The sampling frequency for audio recording was 48 kHz; there were 16 bits assigned to each sample. No image processing was used. During the recording the infants were sitting on their mothers’ lip, the distance between the microphone and the mouths of the infants was 1-2 meters. For this study, cries from 70 infants were recorded; the length of the cry signals is 27.49 s on the average. The hearing function of all the infants was assessed by means of subjective and objective audiometry. 35 of the babies had hearing loss or deafness; the other 35 had normal hearing. The mean age was 11 months, with a standard deviation of 8 months. There were boys (40) as well as girls (30) within the group. The cries were collected during manual audiometry; the doctor looked into the ears of the infant. The procedure was painful and the baby started to cry. The cries selected for analysis were, whenever possible,

chosen from the start of the cry sequences. The cries for each baby were, however, usually very similar both auditively and on the spectrograms.

3 Method

Authors used their own developed Matlab software to evaluate preprocessing, feature extraction and fundamental frequency detection, and to plot the melody contour and calculate the rising time.

3.1 Preprocessing

Before the acoustic analysis starts, preprocessing is necessary to eliminate the technical defects of the recording and to find the important parts of the whole crying signal. Two main tasks are defined, they are filtering and feature extraction. The recorded sounds may contain unwanted effects, as background noises, echo, etc. Some of these effects can disadvantageously affect the results of the analysis. In the frequency domain the lowest component of the infant cry is not less than 250-300 Hz, thus using a high-pass filter, with a frequency cut-off at 250 Hz is a good solution to reduce most of the background noise [19]. The feature extraction is also called as Automatic Infant Cry Recognition [20]. The basis of this procedure is to find the useful crying signals among pauses and disturbances in the whole signal. Crying signals (see in Figure 1) can be identified by their frequency content or energy [21]. In a typical crying process crying signal is the first signal after respiration.

Figure 1: a 2.4 s long crying signal.

The automatic identification has found 556 crying signals from the 70 infants.

3.2 Fundamental frequency detection

The fundamental frequency (F0) is the smallest, useful frequency component of the spectrum. In the case of crying the range of the fundamental frequency is between approximately 300 and 750 Hz [22-24].

228

Forum Acusticum 2005 Budapest Várallyay, Illényi, Benyó, Farkas, Katona

Each crying signals are divided into successive windows, which are same in size. F0 is calculated to each window. Authors chose such a small window size to be able to ignore the changing of the frequency components within a window. Because crying is a harmonic signal (except crying with disturbances, as hoarseness), the fundamental frequency can be determined by the regular structure of the spectrum. Authors developed an algorithm (Smoothed Spectrum Method) to detect F0 in each window. It is based on the harmonic structure of the spectrum and statistical algorithms. This method works with 97.99 % reliability [25].

3.3 Plotting the melody contour

The melody contour can be obtained by imaging the fundamental frequency as a function of time. There are typical types of the melody contour, as rising, rising-falling, falling, and flat, etc [26]. Crying melodies can be categorized by duration as well. In a previous study it was proved that the duration doesn’t differ between the group of infants of normal hearing and hard of hearing [27]. Because the range of the fundamental frequency comes near the frequency range of the music paper, authors represented the melody contour on a music paper as it was used in the eighties [28]. But the original music paper, which contains five lines (namely E4, G4, H5, D5, F5), has an unequal scaling, so the original music paper cannot be used as a correct frequency scaling. Authors developed the Five Line Method (FLM) to visualize the melody contour, which i.a. applies logarithmic scaling to obtain equal ratio between the neighboring lines [29]. The difference of the five lines between the music paper and the Five Line Method is shown in the following Matlab code.

>>lin_old=[329.63,392.0,493.88,587.33,698.46]; >>lin_old(2:5)./lin_old(1:4) ans = [1.1892, 1.2599, 1.1892, 1.1892] >>lin_new=[329.63,397.7,479.82,578.91,698.46]; >>lin_new(2:5)./lin_new(1:4) ans = [1.2065, 1.2065, 1.2065, 1.2065]

Figure 2 shows an example to the FLM. There is the time in seconds on the X-axis. The Y-axis is a frequency axis, containing the (round) values of the new five lines.

3.4 Calculating the rising time

The rising time (trising) can be only defined if the melody of the crying signal starts with rising; it is the duration of the first rising period.

Figure 2: an example to the Five Line Method.

E.g. on the previous figure (Figure 2) the melody starts with a rising period, the duration of that period (i.e. the rising time) is approximately 0.3 s. To obtain the rising time the following steps are to evaluated. ( ) ( ){ }tyFtM 0= (1)

Where M(t) is the melody contour of the y(t) crying signal. M(t) is a function of time. F0{} means the detection of the fundamental frequency. To find the rising period, the authors check the derived function of the melody contour.

( ) 00

><< risingttdt

tdM (2)

If the derived function is positive at starting, it means the melody starts with a rising period. In this case the rising time is interpretable, and the value of trising can be obtained by finding the first zero value at the derived function of the melody contour. The next figure (Figure 3) shows a melody contour and its derived function. It can be clearly seen that the rising time is at the zero point of the derived function (in this case it is 0.27 s).

Figure 3: The melody contour and its derived function. The rising time is shown with the vertical dashed line.

229

Forum Acusticum 2005 Budapest Várallyay, Illényi, Benyó, Farkas, Katona

The following Matlab code shows how to obtain the rising time from a crying signal in practice. This code contains the before mentioned methods as preproc, ssm and flm functions.

% input is the filename and the window size % output will be the auto_detected rising time %-------------------------------------------- % importing the wave file and the sampling % frequency into variables [y,fs] = wavread(filename); % preprocessing (filtering, etc.) y = preproc(y); % number of windows win_number = floor(length(y)/win_size); %-------------------------------------------- for i = 1:win_number actual_window = y((i-1)*win:number+1:... i*win_number); % calculating the fundamental % frequency to the actual window by % the Smoothed Spectrum Method F0 = ssm(actual_window,fs); melody(i) = F0; end % imaging the melody with the Five Line Method flm(melody,fs,win_size) %-------------------------------------------- % auto-calculating the rising_time start_point = 0; finish_point = ... (min(find(diff(melody)<0))-1)*win_size/fs; rising_time = finish_point - start_point;

The graphical user interface of the rising time detection software is shown below (Figure 4).

Figure 4: The graphical user interface of the rising time detection software

The auto-calculated rising time can be manually corrected if the (experienced) user find a different start time or finish time. The program shows the determined melody contour and calculates the rising time. The start and the finish points are highlighted from the points of the melody. If the user finds both the start time and the finish time true, by clicking the OK button, the calculated rising time will be saved and the next crying signal will appear.

In questionable situations the user can play the crying signal, and listen to the melody to testify the results. If there is no rising part in the beginning of the crying signal or it is not obvious according to the plotted melody contour, the user can choose not to save rising time for the current sample and to get the next crying signal.

4 Results

From the 556 crying signals a total of 265 rising periods (i.e. 265 crying signals started with rising period) were detected. There were 128 crying signals from infants with hearing disorders, and 137 signals from healthy babies. The average of the rising times was 0.3282±0.1487 s, and the median was 0.2987 s. The distribution of the rising times is shown in Figure 5.

Figure 5: The distribution of the rising time.

4.1 Comparison

Comparing the two groups (i.e. the group of infants with hearing disorders and the group of healthy babies) no significant differences were found by the rising time (from the well-known ANOVA method: d.f. = 273, F = 1.31, p = 0.253). At the first group the mean of the rising time was 0.3178±0.1206 s, and the median was 0.2887 s. At the group of healthy infants the mean value was 0.3380±0.1701 s, and the median was 0.2987 s. The results are presented on the following table (Table 1).

230

Forum Acusticum 2005 Budapest Várallyay, Illényi, Benyó, Farkas, Katona

Table 1: The mean, the deviation and the median values of the rising time for both groups.

Rising time (s) Hearing Gender No. of

signals mean SD median

Disorders boys 69 0.2951 0.1056 0.2560 girls 59 0.3442 0.1321 0.3413 total 128 0.3178 0.1206 0.2887 Healthy boys 42 0.2853 0.1303 0.2560 girls 95 0.3612 0.1807 0.2987 total 137 0.3380 0.1701 0.2987 Both boys 111 0.2914 0.1151 0.2560 girls 154 0.3547 0.1636 0.3251 total 265 0.3282 0.1484 0.2987

Authors found differences between the rising time of boys and girls (d.f. = 307, F = 17.02, p < 0.01). At boys the mean of the rising time was 0.2914±0.1151 s, and the median was 0.2560 s. Girls had a 0.3547±0.1636 s mean value and a 0.3251 s median value. The difference is shown on Figure 6.

Figure 6: The distribution of the rising time between boys and girls.

5 Discussion

As experienced otorhinolaryngologists could distinguish the crying of infants with hearing disorders from the crying of healthy infants a few decades before, these differences may still exist. By using a suitable parameter obtained from the crying signal, this separation could be successful. This method would cover the whole auditory system, contrary to the existing objective audiometry methods, which test just a part of the whole system. There are

further advantages of the infant cry based method, as it is a non-invasive method and sound analysis could be evaluated by low cost instruments.

6 Conclusions

The melody contour was determined to each crying signal and the rising time was calculated from the melody. Authors had found 265 melodies which started with rising period. No significant differences were found between the rising time of infants with hearing disorders and healthy babies. Differences were found, although, between boys and girls. By using the one-way analysis of variance a very small (< 0.01) p-value was determined between the rising time of boys and girls.

Acknowledgements

The research has been supported by Hungarian National Scientific Research Foundation (OTKA T0422990).

References

[1] R. G. Barr, B. Hopkins, J. A. Green, ‘Crying as a sign, a symptom and a signal’, Clinics in Developmental Medicine No. 152. London, MacKeith Press (2000)

[2] A. Fort, A. Ismaelli, C. Manfredi, P. Bruscaglioni, ‘Parametric and non-parametric estimation of speech formants: application to infant cry’, Med Eng Phys, Vol. 18. No. 6. pp. 677-691. (1996)

[3] A. Fort, C. Manfredi, ‘Acoustic analysis of newborn infant cry signals’, Med Eng Phys, Vol. 20. No. 6. pp. 432-442. (1998)

[4] F. B. Furlow, ‘Human Neonatal Cry Quality as an Honest Signal of Fitness’, Evolution and Human Behavior, Vol. 18. No. 3. pp. 175-193. (1997)

[5] H. R. Gilbert, M. P. Robb, ‘Vocal fundamental frequency characteristics of infant hunger cries: birth to 12 months’, Int J Pediatr Otorhinolaryngol, Vol. 34. pp. 237-243. (1996)

[6] Z. Benyó, Z. Farkas, A. Illényi, G. Katona, G. Várallyay Jr., ‘Csecsemők hangelemzése különös tekintettel a hallásképességükre’. In: Beszédkutatás 2002. Edited: Gósy Mária. MTA Nyelvtudományi Intézet, Budapest, pp. 100-117. (2002)

[7] H. Rothganger, ‘Analysis of the sounds of the child in the first year of age and a comparison to

231

Forum Acusticum 2005 Budapest Várallyay, Illényi, Benyó, Farkas, Katona

the language’, Early Human Dev, Vol. 75. pp. 55-69. (2003)

[8] A. T. Cacace, M. P. Robb, J. H. Saxman, H. Risemberg, P. Koltai, ‘Acoustic features of normal-hearing pre-term infant cry’, Int J Pediatr Otorhinolaryngol, Vol. 33. pp. 213-224. (1995)

[9] P. S. Zeskind, B. M. Lester, ‘Acoustic features and auditory perceptions of the cries of newborns with prenatal and perinatal complications’, Child Dev, Vol. 49. No. 3. pp. 580-589. (1978)

[10] G. E. Gustafson, J. A. Green, J. V. Cleland, ‘Robustness of individual identity in the cries of human infants’, Dev. Pschobiol, Vol. 27. No. 1. pp. 1-9. (1994)

[11] D. W. Leger, R. A. Thompson, J. A. Merritt, J. J. Benz, ‘Adult perception and emotion intensity in human infant crys: effects of infant age and cry acoustics’, Child Dev, Vol. 67. No. 6. pp. 3238-3249. (1996)

[12] J. Hirchberg, T. Szende, ‘Pathological cry, stridor and cough in infants’, Akadémiai Kiadó, Budapest (1982)

[13] J. Hirschberg, P. H. Dejonckere, M. Hirano, K. Mori, H-J. Schultz-Coulon, K. Vrticka, ‘Voice disorders in children’, Int J Pediatr Otorhinolaryngol, Vol. 32. pp. S109-S125. (1995)

[14] Z. Farkas, O. Ribari, ‘Some Data on the Audiological Situation and Ear Care in Hungary and in some Central and Eastern European Countries’, Scand. Audiol, Vol. 26. pp. 45-55. (1997)

[15] Z. Benyó, Z. Farkas, A. Illényi, G. Katona, G. Várallyay Jr., ‘Information transfer of sound signals. A case study: the infant cry. Is it a noise or an information?’. Proc. 33rd International Congress and Exposition on Noise Control Engineering, Prague, No. 528. pp. 1-8. (2004)

[16] G. Várallyay Jr., Z. Benyó, A. Illényi, Z. Farkas, ‘Methods for the Analysis of Acoustic Biomedical Signals’. Proc. IASTED International Conference on Biomedical Engineering (BioMED 2005), Innsbruck, pp. 434-438. (2005)

[17] K. Michelsson, K. Eklunkd, P. Leppanen, H. Lyytinen, ‘Cry characteristics of 172 healthy 1- to 7-day-old infants’, Folia Phoniatr Logop, Vol. 54. No. 4. pp. 190-200 (2002)

[18] P. Runefors, E. Arnbjornsson, G. Elander, K. Michelsson, ‘Newborn infants' cry after heel-prick: analysis with sound spectrogram’, Acta Paediatr, Vol. 89. No. 1. pp. 68-72. (2000)

[19] G. Várallyay Jr., Z. Benyó, A. Illényi, Z. Farkas, G. Katona, Z. Csákányi, T. Majoros, P. Benedek, T. Patkó, ‘Csecsemősírás számítógépes elemzése’. Proc. BUDAMED ’02, Budapest, pp. 60-61. (2002)

[20] O. F. Reyes-Galaviz, E. A. Tirado, C. A. Reyes-Garcia, ‘Classification of Infant Crying to Identify Pathologies in Recently Born Babies with ANFIS’, LNCS, Vol. 3118. pp. 408-415. (2004)

[21] T. Zhang, C. C. J. Kuo, ‘Audio content analysis for online audiovisual data segmentation and classification’, IEEE Trans. on Speech and Audio Processing, Vol. 9. No. 4. pp. 441-457. (2001)

[22] K. Michelsson, K. Christensson, H. Rothganger, J. Winberg, ‘Crying in separated and non-separated newborns: sound spectrographic analysis’, Acta Paediatr, Vol. 85. No. 4. pp. 471-475. (1996)

[23] K. Michelsson, O. Michelsson, ‘Phonation in the newborn, infant cry’, Int J Pediatr Otorhinolaryngol, Vol. 49. No. 1. pp. S297-S301. (1999)

[24] K. Wermke, W. Mende, C. Manfredi, P. Bruscaglioni, ‘Developmental aspects of infant’s cry melody and formants’, Med Eng Phys, Vol. 24. No. 7-8. pp. 501-514. (2002)

[25] G. Várallyay Jr., ‘Infant cry analyzer system for hearing disorder detection’, Periodica Politechnica, TU Timişoara, Transactions on Automatic Control and Computer Science, Vol. 49/63. pp. 57-60. (2004)

[26] R. Schönweiler, S. Kaese, S. Möller, A. Rinscheid, M. Ptok, ‘Neuronal networks and self-organizing maps: new computer techniques in the acoustic evaluation of the infant cry’, Int J Pediatr Otorhinolaryngol, Vol. 38. pp. 1-11. (1996)

[27] G. Várallyay Jr., ‘Future prospects of the application of the infant cry in the medicine’, Periodica Polytechnica Ser. El. Eng. in press

[28] Z. Makói, Z. Szőke, L. Sasvári, P. Gegesi Kiss, P. Popper, ‘1st cry of newborn after vaginal and cesarean delivery’, Acta Paediatr Hung, Vol. 16. No. 2. pp. 155-161. (1975)

[29] G. Várallyay Jr., Z. Benyó, A. Illényi, Z. Farkas, L. Kovács, ‘Acoustic analysis of the infant cry: classical and new methods’, Proc. 26th Conf. IEEE Engineering in Medicine and Biology, San Francisco, CA, pp. 313-316. (2004)

232