Lipreading in hearing impaired

Lip ReadingZainab Hamadi

Na ta l i e Nader

Hawraa Al -Romani

Lebanese Unive rs i ty-Speech and Language Therapy

Lip-reading = Speech reading

The act or process of determining the intended meaning of a speaker by utilizing all visual clues accompanying speech attempts, as lip movements, facial expressions, and bodily gestures, used especially by people with impaired hearing.

What is Lip-reading?

Lip reading is said to be 80% guesswork

Only 1/3 of speech is visible on the lips

An “expert” lip reader is guessing at 66% of what is said!

When considering lip-reading, be aware that there is a high incidence of visual impairment within the hearing impaired population.. Regular eye checks are essential.

More about lip-reading:

More about lip-reading:• Amplification: Also has a significant role - a little sound in addition to lip-reading, makes a big

difference. Appropriate, functioning hearing aids, worn consistently, do make lip-reading much easier.

• McGurk Effect: Even in the normally hearing population, lip movements influence what listeners

hear. An example of this is known as the 'McGurk Effect' - an illusion caused when what you hear someone say doesn't match the lip configuration that you see. This is why watching dubbed movies is a strange experience.

Lip-reading in hearing:

Speechreading is also used by people who have typical hearing, especially in situations that are noisy or the speech is unclear.

Woodhouse (2009) published five pieces of evidence that shows:

“speech-reading is an integral part of speech processing. Hearing people's perception of speech is influenced by speech-read cues, and those speech-read cues cannot be ignored.”

What is Visible?

Anatomically Physiologically Phonetically Phonemically

What is visible?

Partially Visible

Tongue tip

Cheek musclesTeeth

VisibleLips

Jaw

InvisibleTongue bodyVelumVocal foldsBreathing mechanism

Anatomically - Motorically

Anatomical visibility is described at the level of Positions and Movements

Lips open/closed

Lips spread/rounded

Jaw raised/lowered

Tongue tip raised/lowered

Tongue tip forward/back

Anatomically -MotoricallyLips protruded/retracted

What is visible?

vocal fold vibration, nasal resonance, and airflow patterns are not visible

What is visible?

• Place of articulation of stressed vowels, and of consonants, is mostly visible.

• Duration and rhythm are partially visible.

• Place of articulation of unstressed vowels, voicing and manner of articulation of consonants, pitch and intonation, and loudness are all invisible

What is visible?

• Stressed vowels are mostly identifiable

• Unstressed vowels are not identifiable.

• Consonants articulated at or near the back of the mouth are invisible (e.g., k, g, h).

• Other consonants are only identifiable within groups (visemes) that have different places of articulation.

What is visible?

WHY IS SPEECHREADING DIFFICULT? Problems relate to four main areas. These are:

1. The nature of speech

2. The speaker

3. The environment

4. The speechreader

1-The Nature of Speech

Frontal sounds (e.g. ‘p’) are the easiest to speechread. In contrast, it is very difficult to speechread middle sounds (e.g. ‘t’) or back

sounds (e.g. k, g, h - Kate, Gate and Hate) .

Many words also look alike. Alexander Graham Bell coined the term ‘homophene’ to describe words which look alike but do not sound alike.

e.g. Try saying the following three words without voice to friends. Can they tell the difference? pat, mat, bat

Because so many speech shapes and words look alike, misunderstandings can occur.

Why is speechreading difficult?

• A moustache or beard

• Poor lip movement (either minimal or exaggerated) ‘dead-pan’ faces.

• Hands covering mouths.

• Talking too quickly.

Why is speechreading difficult? 2-The Speaker:Some people’s speaking habits or features make life very difficult for speech reader:

Why is speechreading difficult? 3-The Environment:

Lighting Distance DistractionsPoor lighting that is shining into the speechreader’s eyes rather than on the speaker’s face is a problem. A speechreader should try to be positioned with his/her back to the light source.

A speechreader should be a comfortable distance from the speaker so that hearing and vision are maximised. (not too close or too far)

Distractions in the environment can make concentration difficult for the speechreader. In group situations, a speechreader must try to be in touch with topic changes and who is speaking, otherwise he/she is ‘left out’ very quickly.

o The innate ability to speechread varies from person to person. This can only be improved to a certain extent by training.

o To help speechreading, any visual defects should be identified and corrected if possible.

o Speechreaders should also be prepared to admit when they have not understood, rather than passively pretend they have.

Why is speechreading difficult? 4-The SpeechReader:

Tips for Speechreading

Make sure you can see the speaker’s face clearly.

Hold the conversation in a quiet environment, with good lighting, and not a lot of visual distractions.

Make sure that light is behind you, not the person you are trying to speechread.

Gently remind people that you need to see their face when they forget and look down or away from you.

Tips for Speechreading

Ask for the topic of the conversation, if you are not sure.

If the speaker over–exaggerates, or talks too loudly, gently request that they speak normally.

Remind speakers to move their hands or other objects away from their face.

If you still don’t understand after a repeat, ask the speaker to rephrase.

There are Four general approaches used to impart speechreading skills (Tye-Murray, 1998).

Analytic Synthetic Pragmatic Holistic

Each approach had different models associated with them

1 Analytic

• The Analytic approach was considered a traditional method of instruction for teaching speechreading skills,

• and is generally defined as an approach that focused on the smallest units of speech to understand spoken communication.

• Hallmarks of this approach are rapid syllable drills emphasizing the practice of building smaller verbal units (the syllable or viseme) into words and then sentences (Hanavan, 2013).

definition

1- Analytic Approach:

1- Analytic Approach:

Theory They assume that speech perception is, a bottom-up process Where that 1st task is to analyze the incoming visual signal into its basic components before assembling those components into words and sentences (Kaplan, et al. 1985).

1- Analytic Approach:

Instruction Instruction: concentrates on the details of the sounds - learning to recognize how they look on the lips and practicing their recognition in isolation and in single words

Exercises:

• Some of the practices include syllable discrimination exercises,

• With practice and improved proficiency, similar exercises can be done using words selected based on the syllables intended for practice.

1- Analytic Approach:

where the speech-reader would be presented with two syllables:

(e.g., “GA-BA” or “BA-BA”)

two syllables were the same or different

limitations• While studies have reported from modest to significant improvements in speech

recognition performance, there are limitations to this approach.

1- Analytic Approach:

English-speaking person will creates about 13/15

specific speech movements per second in a comfortably paced

conversation

But even the best trained professional

speechreaders can only register about 8/9

movements per sec at most.

Synthetic2

definition

Top-down approach where the perception of the whole, or the overall meaning of the message is emphasized more so than concentrating on smaller parts.

Exercises for this method include speechreading sentence practice in a known context to aid in determination of overall meaning.

(The sentences used to practice this approach should be “definite, natural, interesting, pleasing, rhythmical and dignified”)

2- Synthetic Approach:

2- Synthetic Approach:

goal The goal of the synthetic approach is for speechreaders to use context and any known information to allow for educated guessing, when some sounds or parts of the sentence is missing from auditory perception.

In other words, the meaning of a given sentence, if the context is known but some of the words are ambiguous, can be inferred.

2- Synthetic Approach:

limitations Include situations when:o Speechreader is unable to determine the context of the message, or is o Unable to determine the overall meaning of a messageo Unknowingly misinterprets the information,

leading to misunderstanding. While repair strategies can help in such situations, it is largely the responsibility of the speechreader to recognize when this is happening and to take the needed steps to mitigate any confusion.

2- Synthetic Approach:

exercises There are many exercises that can be used in this approach, most focusing on giving the speechreader some clues such as key words or a topic that will be discussed, presenting the message and having the speechreader respond in a variety of ways.

For example, the speechreader could:o reply to a specific question,o repeat a passage, or o incorporate details from the presented message into subsequent discussion

2- Synthetic Approach:

Pragmatic3

definition The Pragmatic Approach focuses primarily on creating the ideal environment and context for the person with hearing loss in dialogue.

They employ communication strategies to ensure that residual hearing is maximized to increase comprehension of the speaker.

3- Pragmatic Approach:

(Lind, 2009).

Theory The aim of this approach is for the individual to:

• concentrate on interaction more than the reception of specific cues or sounds,

• to ask for modification of the speakers message, the context in which dialogue takes place.

• develop specific strategies that they can employ in particular situations that are found to be most difficult to communicate.

3- Pragmatic Approach:

The focus of this approach is based on how language is used in social situations and how the use of language can affect those communicating; as a result, most of the exercises are framed from the perspective of interaction.

3- Pragmatic Approach:

exercises Practices and exercises used in the Pragmatic Approach tend to incorporate dialogue between pairs or groups of speechreaders, or the speechreader and the instructor/clinician.

With practice, the speechreader will employ communication strategies to ensure they are fully engaged and correctly understanding the conversation.

3- Pragmatic Approach:

During dialogue btw speechreader & instructor/clinician, the speechreader will have to reply appropriately

The instructor provides prompts & cues to help the person get back on track

if they do not

exercisesOther exercises can include: narratives, discussion, and role-playing.

Cues such as word order and sequencing, tense and word endings, matching language to the situation and context, among others are important considerations when using this approach

3- Pragmatic Approach:

limitations

• The most notable is that it is the speechreader’s existing skills that are maximized, with little emphasis on developing new skills.

• There is no effort made to incorporate a bimodal approach (two sensory inputs including both auditory and visual cues) to overcoming conversation difficulties.

• Another serious limitation, is that there is currently no evidenced-based research to be found on the efficacy of this approach.

3- Pragmatic Approach:

Holistic4

Definition (Eclectic approach)

The Holistic Approach incorporated aspects from the Analytic, Synthetic and Pragmatic approaches.

The main objectives of a Holistic Approach included: “the evaluation and management of overall communication skills, the psychosocial aspects of hearing loss, the education of significant others (e.g. parents, siblings and close friends), hearing aid orientation, an emphasis on improving conversational and interactive skills, and the use of assistive listening devices”

4- Holistic Approach:

goals Yoshinaga-Itano (1988) also described a holistic approach to instructing speechreading techniques to children.

The training goals included:

4- Holistic Approach:

increasing the child’s knowledge of the speechreading process,

their ability to generalize strategies to facilitate more successful communication,

their tolerance for difficult/frustrating communicative situations.

their role in generating personal goals.

their motivation to improve their speechreading abilities

The rationale behind a Holistic Approach is two fold;

Firstly Analytic, Synthetic and Pragmatic approaches each train specific aspects of speech perception and communication strategies that are useful, and the starting point in determining which strategies could be applied is the individual living with hearing loss.

Secondly, while there are both benefits and detriments to the Analytic, Synthetic and Pragmatic approaches, an approach that incorporates aspects of each is flexible and can be adapted to specific situations when called for.

4- Holistic Approach:

benifit The benefit to using a Holistic paradigm is the client-oriented approach, which is much more empowering to the person seeking help in coping with their hearing loss, and ensures a program that is tailored to the individual or groups’ unique needs.

4- Holistic Approach:

limitations Unfortunately, there is a lack of evidence-based research to provide any validity to this particular method, and an obvious need for this remains.

It could be argued however, that the research that upholds the value of the Analytic, Synthetic and Pragmatic Approaches could be transferred to the Holistic Approach as it incorporates aspects of all three of the preceding methods, and the skills imparted are not new inventions without prior precedent.

4- Holistic Approach:

Study: Does lipreading improve? The study addressed the research questions: Does lipreading improve between the ages of 7 and 14 years? Does hearing loss affect the development of lipreading?

children with normal hearing (NH) and with hearing loss (HL) were tested using 4 lipreading instruments.

Results: For both groups, lipreading performance improved with age on all 4 measures of lipreading, with the HL group performing better than the NH group.

Conclusions: • Results showed that children's lipreading ability improves. • Experience plays an important role in the development of this ability. (HL children

lipread better than those with NH)

In Hearing

In Deaf

References:• The Impact of Speechreading Programs on Adults with Hearing Loss, Literature Review

Alena Wickware, Project Manager, National Speechreading Program, The Canadian Hard of Hearing Association .

• http://www.ssc.education.ed.ac.uk/courses/deaf/ddec05f.html

• SLHS 511: Lipreading. Slides and study notes

• http://www.vicdeaf.com.au/files/editor_upload/File/Information%20Sheets/Speechreading%20_Visual%20Cues_.pdf

• http://jslhr.pubs.asha.org/article.aspx?articleid=1802630