CONTROL AND COMMUNICATION FOR PHYSICALLY DISABLED PEOPLE, BASED ON VESTIGIAL SIGNALS FROM THE BODYBY YVONNE MAY NOLAN B.E.

A thesis presented to The National Univerisity of Ireland in fullment of the requirements for the degree of

PHILOSOPHIAE DOCTORin the DEPARTMENT OF ELECTRONIC AND ELECTRICAL ENGINEERING FACULTY OF ENGINEERING AND ARCHITECTURE NATIONAL UNIVERSITY OF IRELAND, DUBLIN

SEPTEMBER 2005

Supervisor of Research: An tOllamh A.M. de Paor

Head of Department: Professor T. Brazil

Abstract

When people become disabled as a result of a road trac accident, stroke or another condition, they may often lose their ability to control their environment and communicate with others by conventional means. This thesis investigates methods of harnessing vestigial body signals as channels of control and communication for people with very severe disabilities, using advanced signal acquisition and processing techniques. Bioelectrical, acoustic and movement signals are among the signals investigated. Some applications are presented that have been developed to assist environmental control and communication. These applications rely on a variety of control signals for operation. Some applications may be controlled by a simple binary switching action whereas others require user selection from a wider range of possible options. A mechanical switch or adjustable knob may be used to interact with these applications but this may not be an option for people who are very severely disabled. The remainder of the thesis focuses on alternative methods of enabling user interaction with these and other applications. If a person who is physically disabled is able to modify some body signal in such a way that two states can be distinguished reliably and repeatedly, then this can be used to actuate a switching action. Reliable detection of more than two states is necessary for multiple-level switching control. As users abilities, requirements and personal preferences vary greatly, a wide range of body signals have been explored. Bio-signals investigated include the electrooculogram (EOG), the electromyogram (EMG), the mechanomyogram (MMG) and the conductance of the skin. The EOG is the electrical signal measurable around the eyes and can be used to detect eye movements with careful signal processing. The EMG and the

MMG are the electrical and mechanical signals observable as a result of muscle contraction. The conductance of the skin varies as a person relaxes or tenses and with practice it can be consciously controlled. These signals were all explored as methods of communication and control. Also, investigation of the underlying physical processes that generate these signals led to the development of a number of mathematical models. These models are also presented here. Small movements may be harnessed using computer vision techniques. This has the advantage of being non-contact. Often people who have become disabled will still be capable of making ickers of movement e.g. with a nger or a toe. While these movements may be too weak to operate a mechanical switch, if they are repeatable they may be used to provide a switching action in software through detection with a video camera. Phoneme recognition is explored as an alternative to speech recognition. Physically disabled persons who have lost the ability to produce speech may still be capable of making simple sounds such as single-phoneme utterances. If these sounds are consistently repeatable then they may be used as the basis of a communication or control device. Phoneme recognition oers another advantage over speech recognition in that it may provide a method of controlling a continuously varying parameter through varying the length of the phoneme or the pitch of a vowel sound. Temporal and spectral features that characterise dierent phonemes are explored to enable phoneme distinction. Phoneme recognition devices developed in both hardware and software are described.

ACKNOWLEGDEMENTS

I would rstly like to thank Harry, my supervisor, for all his support, encouragement and advice and for sacricing his August bank holiday Monday to help me get this thesis in on time!

Thanks also to all the postgrads who have been in the lab in the NRH with me over the past three years - Deirdre, Claire, Catherine, Kieran, Ciaran and Jane. Special thanks to Ted for all his assistance, support and friendship. Thanks also to Emer for generating some of the graphs for this thesis.

Thanks to my parents for their patience and nancial help and to my sisters Tamara and Jill for keeping the house (relatively) quiet to enable me to get some work done.

Thanks to all my friends for understanding my disappearance over the past few months and giving me space to get this thesis nished.

Finally, a big thanks to Conor for being so supportive and patient with me over the past few months, for giving me a quiet place to work and for helping me with the pictures for this thesis!

i

LIST OF PUBLICATIONS ARISING FROM THIS THESISAn Investigation into Non-Verbal Sound-Based Modes of Human-to-Computer Communication with Rehabilitation Applications, Edward Burke, Yvonne Nolan & Annraoi de Paor, Adjunct Proceedings of 10th International Conference on Human-Computer Interaction, Crete, June 22-27 2003, pp. 241-2.

The Mechanomyogram as a Tool of Communication and Control for the Disabled, Yvonne Nolan & Annraoi de Paor, 26th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, San Francisco, CA, September 1-5 2004, pp. 4928-2931.

An Electrooculogram Based System for Communication and Control Using Target Position Variation, Edward Burke, Yvonne Nolan & Annraoi de Paor, IEEE EMBSS UKRI Postgraduate Conference on Biomedical Engineering and Medical Physics, Reading, UK, July 18-20 2005, pp. 25-6.

The human eye position control system in a rehabilitation setting, Yvonne Nolan, Edward Burke, Claire Boylan & Annraoi de Paor, International Conference on Trends in Biomedical Engineering, University of Zilina, Slovakia, September 7-9 2005.

Accepted Paper: Phoneme Recognition Based Software System for Computer Interaction by Disabled People, Yvonne Nolan & Annraoi de Paor, IEEE EUROCON 2005 - International Conference on Computers as a Tool, University of Belgrade, Serbia and Montenegro, November 21-24 2005.

ii

Contents

1 Introduction 1.1 Assistive Technologies . . . . . . . . . . . . . . . . . . . . . . . 1.2 Thesis Layout . . . . . . . . . . . . . . . . . . . . . . . . . . . .

1 2 3

2 Assistive Technology 2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2 Causes of Paralysis . . . . . . . . . . . . . . . . . . . . . . . . . 2.2.1 2.2.2 2.2.3 Neurological Damage . . . . . . . . . . . . . . . . . . . . Spinal Cord Injuries . . . . . . . . . . . . . . . . . . . .

6 6 7 7 9

Diseases of the Nervous System . . . . . . . . . . . . . . 17

2.3 Assistive Technology . . . . . . . . . . . . . . . . . . . . . . . . 18 2.3.1 2.3.2 2.3.3 Importance of a Switching Action . . . . . . . . . . . . . 19 Switch Based Systems . . . . . . . . . . . . . . . . . . . 20 Brain Computer Interfaces . . . . . . . . . . . . . . . . 23

2.4 Communication Device . . . . . . . . . . . . . . . . . . . . . . . 24 2.4.1 Technical Details . . . . . . . . . . . . . . . . . . . . . . 25

iii

2.4.2 2.4.3 2.4.4 2.4.5

The Natterbox Graphical User Inteface . . . . . . . . . . 25 Switch Interface Box . . . . . . . . . . . . . . . . . . . . 26 Other Features . . . . . . . . . . . . . . . . . . . . . . . 26 Possible Future Developments of Natterbox . . . . . . . 31

2.5 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

3 Muscle Signals

33

3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 3.2 The Nervous System . . . . . . . . . . . . . . . . . . . . . . . . 34 3.2.1 3.2.2 Nerves and the Nervous System . . . . . . . . . . . . . . 34 Resting and Action Potentials . . . . . . . . . . . . . . . 38

3.3 Muscles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 3.3.1 3.3.2 3.3.3 Muscle Physiology . . . . . . . . . . . . . . . . . . . . . 41 Muscle Contraction . . . . . . . . . . . . . . . . . . . . . 44 Muscle Action in People with Physical Disabilities . . . . 47

3.4 Electromyogram . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 3.4.1 3.4.2 EMG Measurement . . . . . . . . . . . . . . . . . . . . . 49 EMG as a Control Signal . . . . . . . . . . . . . . . . . . 52

3.5 Mechanomyogram . . . . . . . . . . . . . . . . . . . . . . . . . . 55 3.5.1 3.5.2 MMG as a Control Signal . . . . . . . . . . . . . . . . . 56 MMG Application for Communication and Control . . . 58

3.6 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64

iv

4 Other Biosignals - Eye Movements and Skin Conductance

65

4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65 4.2 The Electrooculogram . . . . . . . . . . . . . . . . . . . . . . . 66 4.2.1 4.2.2 4.2.3 4.2.4 4.2.5 4.2.6 4.2.7 4.2.8 4.2.9 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 66 Anatomy of the Eye . . . . . . . . . . . . . . . . . . . . 67 Eye Tracking Methodologies . . . . . . . . . . . . . . . . 69 The EOG as a Control Signal . . . . . . . . . . . . . . . 76 Target Position Variation . . . . . . . . . . . . . . . . . 84

Experimental Work . . . . . . . . . . . . . . . . . . . . . 86 TPV Based Menu Selection . . . . . . . . . . . . . . . . 94 Limitations of Eyetracking for Cursor Control . . . . . . 99 A Model of the Eye . . . . . . . . . . . . . . . . . . . . . 100

4.3 Electrodermal Activity as a Control Signal . . . . . . . . . . . . 119 4.3.1 4.3.2 4.3.3 4.3.4 4.3.5 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 119 Anatomy and Physiology of the Skin . . . . . . . . . . . 120 Electrodermal Activity . . . . . . . . . . . . . . . . . . . 121 Skin Conductance as a Control Signal . . . . . . . . . . . 123 Non-invasive Measurement of the Sympathetic System Firing Rate . . . . . . . . . . . . . . . . . . . . . . . . . 125 4.4 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130

5 Visual Techniques

132

v

5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132 5.2 Visual Based Communication and Control Systems . . . . . . . 133 5.2.1 5.2.2 The Camera Mouse . . . . . . . . . . . . . . . . . . . . . 133 Reected Laser Speckle Pattern . . . . . . . . . . . . . . 135

5.3 Visual Technique for Switching Action . . . . . . . . . . . . . . 136 5.3.1 5.3.2 5.3.3 5.3.4 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 136 Technical Details . . . . . . . . . . . . . . . . . . . . . . 138 Frame Comparison Method . . . . . . . . . . . . . . . . 139 Path Description Method . . . . . . . . . . . . . . . . . 150

5.4 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157

6 Acoustic Body Signals

159

6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159 6.2 Speech Recognition . . . . . . . . . . . . . . . . . . . . . . . . . 160 6.2.1 6.2.2 Speech Recognition: Techniques . . . . . . . . . . . . . . 160 Speech Recognition: Limitations . . . . . . . . . . . . . . 163

6.3 Anatomy, Physiology and Physics of Speech Production . . . . . 164 6.3.1 6.3.2 6.3.3 6.3.4 Respiration . . . . . . . . . . . . . . . . . . . . . . . . . 165 Phonation . . . . . . . . . . . . . . . . . . . . . . . . . . 165 Resonance . . . . . . . . . . . . . . . . . . . . . . . . . . 169 Articulation . . . . . . . . . . . . . . . . . . . . . . . . . 171

6.4 Types of Speech Sounds . . . . . . . . . . . . . . . . . . . . . . 173

vi

6.4.1 6.4.2 6.4.3 6.4.4

The Phoneme . . . . . . . . . . . . . . . . . . . . . . . . 174 Types of Excitation . . . . . . . . . . . . . . . . . . . . . 177 Characteristics of Speech Sounds . . . . . . . . . . . . . 180 Proposal of a Phoneme Recognition Based System for Communication and Control . . . . . . . . . . . . . . . . 183

6.5 Hardware Application . . . . . . . . . . . . . . . . . . . . . . . 186 6.5.1 6.5.2 Analogue Circuit . . . . . . . . . . . . . . . . . . . . . . 188 Microcontroller Circuit . . . . . . . . . . . . . . . . . . . 192

6.6 Software Application . . . . . . . . . . . . . . . . . . . . . . . . 194 6.6.1 6.6.2 Application for Linux . . . . . . . . . . . . . . . . . . . . 195 Application for Windows . . . . . . . . . . . . . . . . . . 199

6.7 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 210

7 Conclusions

211

7.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 211 7.2 Resolution of the Aims of this Thesis . . . . . . . . . . . . . . . 212 7.2.1 7.2.2 7.2.3 7.2.4 Overview of Current Communication and Control Methods213 Identication of Signals . . . . . . . . . . . . . . . . . . . 213 Measurement Techniques . . . . . . . . . . . . . . . . . . 214 Signal Processing Techniques and Working Systems Developed . . . . . . . . . . . . . . . . . . . . . . . . . . . 217 7.2.5 7.2.6 Patient Testing . . . . . . . . . . . . . . . . . . . . . . . 218 Biological Studies . . . . . . . . . . . . . . . . . . . . . . 220 vii

7.3 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221 7.3.1 7.3.2 7.3.3 7.3.4 7.3.5 7.3.6 The Mechanomyogram . . . . . . . . . . . . . . . . . . . 221 Target Position Variation . . . . . . . . . . . . . . . . . . 222 Visual Methods for Mouse Cursor Control . . . . . . . . 222 Communication System Speed . . . . . . . . . . . . . . . 223 Multi-Modal Control Signals . . . . . . . . . . . . . . . . 223 Other Vestigial Signals . . . . . . . . . . . . . . . . . . . 223

A MMG Circuit

235

B Simulink Models

237

C MATLAB Code for TPV Fit Function

242

D Optimum Stability

244

E Circuit Diagram for Measuring Skin Conductance

249

F

Phoneme Detection Circuit Diagrams and Circuit Analysis 251 F.1 Analogue Circuit . . . . . . . . . . . . . . . . . . . . . . . . . . 251 F.1.1 Pre-Amplier . . . . . . . . . . . . . . . . . . . . . . . . 251 F.1.2 Filtering . . . . . . . . . . . . . . . . . . . . . . . . . . . 252 F.1.3 Amplier . . . . . . . . . . . . . . . . . . . . . . . . . . 254 F.1.4 Rectier . . . . . . . . . . . . . . . . . . . . . . . . . . . 254 F.1.5 Threshold . . . . . . . . . . . . . . . . . . . . . . . . . . 255

viii

F.1.6 Delay and Comparator . . . . . . . . . . . . . . . . . . . 256 F.1.7 Relays . . . . . . . . . . . . . . . . . . . . . . . . . . . . 256 F.2 Microcontroller Circuit . . . . . . . . . . . . . . . . . . . . . . . 259 F.2.1 Microphone . . . . . . . . . . . . . . . . . . . . . . . . . 259 F.2.2 Amplier . . . . . . . . . . . . . . . . . . . . . . . . . . 259 F.2.3 Innite Clipper . . . . . . . . . . . . . . . . . . . . . . . 262 F.2.4 Microcontroller . . . . . . . . . . . . . . . . . . . . . . . 262 F.2.5 Debouncing Circuit . . . . . . . . . . . . . . . . . . . . . 262 F.2.6 Current Amplier and Relay Coils . . . . . . . . . . . . . 263

G PIC 16F84 External Components and Pinout

264

H Phoneme Recognition Microcontroller Code and Flowchart 266

I

Code for Programs I.1 I.2 I.3 I.4 I.5 I.6

273

Natterbox . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 273 USB Switch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 273 MMG Detection Program . . . . . . . . . . . . . . . . . . . . . 274 Path Description Program . . . . . . . . . . . . . . . . . . . . . 274 Graphical Menu . . . . . . . . . . . . . . . . . . . . . . . . . . . 275 Spelling Bee . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 275

ix

List of Figures2.1 The Vertebral Column . . . . . . . . . . . . . . . . . . . . . . . 11 2.2 The Spinal Nerves . . . . . . . . . . . . . . . . . . . . . . . . . 13

2.3 Dasher program . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 2.4 Natterbox GUI . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 2.5 Natterbox Phrases Menu . . . . . . . . . . . . . . . . . . . . . . 30 3.1 The Nerve Cell . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 3.2 Classication of Nerve Fibre Types . . . . . . . . . . . . . . . . 36 3.3 Nerve Fibres . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37 3.4 An Action Potential . . . . . . . . . . . . . . . . . . . . . . . . 39

3.5 Muscle Anatomy . . . . . . . . . . . . . . . . . . . . . . . . . . 42 3.6 The Muscle Fibre . . . . . . . . . . . . . . . . . . . . . . . . . . 43 3.7 Sarcomere . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44 3.8 The Neck Muscles . . . . . . . . . . . . . . . . . . . . . . . . . . 48 3.9 EMG and frequency spectrum . . . . . . . . . . . . . . . . . . . 50 3.10 EMG Dierential Amplier . . . . . . . . . . . . . . . . . . . . 51

x

3.11 Electrode Position

. . . . . . . . . . . . . . . . . . . . . . . . . 51

3.12 MMG showing Muscle Contraction . . . . . . . . . . . . . . . . 57 3.13 MMG Prosthesis Socket . . . . . . . . . . . . . . . . . . . . . . 58 3.14 Accelerometer . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60 3.15 MMG Processing . . . . . . . . . . . . . . . . . . . . . . . . . . 62 4.1 The Outer Eye . . . . . . . . . . . . . . . . . . . . . . . . . . . 67 4.2 Cross section of the eye . . . . . . . . . . . . . . . . . . . . . . . 68 4.3 Pupil and Corneal Reections . . . . . . . . . . . . . . . . . . . 72 4.4 50Hz Video Eyetracker . . . . . . . . . . . . . . . . . . . . . . . 73 4.5 Scleral Search Coil . . . . . . . . . . . . . . . . . . . . . . . . . 74 4.6 EOG Electrode Positions . . . . . . . . . . . . . . . . . . . . . . 75 4.7 EOG recordings . . . . . . . . . . . . . . . . . . . . . . . . . . . 76 4.8 EOG controlled alphabet board . . . . . . . . . . . . . . . . . . 77 4.9 TPV Based Menu Selection Application . . . . . . . . . . . . . 85

4.10 TPV Candidate Target Shapes . . . . . . . . . . . . . . . . . . . 87 4.11 Results of TPV: Experiment 1 . . . . . . . . . . . . . . . . . . . 90 4.12 TPV Experiment 2 Screenshot . . . . . . . . . . . . . . . . . . . 94 4.13 TPV Experiment 2 . . . . . . . . . . . . . . . . . . . . . . . . . 95 4.14 Fit Values . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98 4.15 Eye feedback control loop . . . . . . . . . . . . . . . . . . . . . 102 4.16 Step Response of Eye with Muscle Spindle Inuence . . . . . . . 106

xi

4.17 Nuclear Bag Model . . . . . . . . . . . . . . . . . . . . . . . . . 106 4.18 Unit step response and Bode magnitude diagrams of the muscle spindle controllers . . . . . . . . . . . . . . . . . . . . . . . . . . 109 4.19 Actual EOG and Simulated Saccadic Responses . . . . . . . . . 111 4.20 Feedback Control Loop for Smooth Pursuit . . . . . . . . . . . . 113 4.21 Modied loop for Smooth Pursuit . . . . . . . . . . . . . . . . . 115 4.22 Bode Plot for Gi (s) . . . . . . . . . . . . . . . . . . . . . . . . . 116 4.23 Smooth Pursuit Model Graphs . . . . . . . . . . . . . . . . . . . 117 4.24 Sweat Gland . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122 4.25 Electrodermal response . . . . . . . . . . . . . . . . . . . . . . . 124 4.26 Skin Conductance Model . . . . . . . . . . . . . . . . . . . . . . 126 4.27 Proposed Loop For Firing Rate Output . . . . . . . . . . . . . . 127 4.28 Measured and Modelled Skin Conductance . . . . . . . . . . . . 128 4.29 Measured Skin Conductance and Estimated Firing Rate . . . . . 129 5.1 Camera Mouse Search Window . . . . . . . . . . . . . . . . . . 134 5.2 Speckle Pattern . . . . . . . . . . . . . . . . . . . . . . . . . . . 136 5.3 Webcam . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138 5.4 Filter Graph used for Video Data in application. . . . . . . . . . 139 5.5 Filtered Video Frames . . . . . . . . . . . . . . . . . . . . . . . 143 5.6 Various Thresholding Methods . . . . . . . . . . . . . . . . . . . 145 5.7 Video Frame Histogram . . . . . . . . . . . . . . . . . . . . . . 146

xii

5.8 Path Description . . . . . . . . . . . . . . . . . . . . . . . . . . 152 5.9 Region Finding . . . . . . . . . . . . . . . . . . . . . . . . . . . 155 5.10 Overlapping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156 6.1 The Vocal Organs . . . . . . . . . . . . . . . . . . . . . . . . . . 166 6.2 Waveform of Vowel Sounds . . . . . . . . . . . . . . . . . . . . . 179 6.3 Spectrum of Vowel Sounds . . . . . . . . . . . . . . . . . . . . . 182 6.4 Phoneme Waveforms and Spectra . . . . . . . . . . . . . . . . . 189 6.5 Analogue Circuit Block Diagram . . . . . . . . . . . . . . . . . 190 6.6 Audio signal pre-processing . . . . . . . . . . . . . . . . . . . . 193 6.7 AudioWidget GUI . . . . . . . . . . . . . . . . . . . . . . . . . 200 6.8 Graphical Menu . . . . . . . . . . . . . . . . . . . . . . . . . . . 201 6.9 The X10 Module . . . . . . . . . . . . . . . . . . . . . . . . . . 201 6.10 Phoneme Detection Program Signal and Spectrum . . . . . . . . 206 6.11 The Spelling Bee GUI . . . . . . . . . . . . . . . . . . . . . . . 208 A.1 MMG Circuit . . . . . . . . . . . . . . . . . . . . . . . . . . . . 236 B.1 Simulink MMG Muscle Contraction Detection . . . . . . . . . . 238 B.2 Simulink Model for Eye System . . . . . . . . . . . . . . . . . . 239 B.3 Simulink Model for Smooth Pursuit . . . . . . . . . . . . . . . . 240 B.4 Simulink Model for Firing Rate . . . . . . . . . . . . . . . . . . 241 D.1 Root Locus Varying f0 . . . . . . . . . . . . . . . . . . . . . . . 245

xiii

D.2 Root Locus Varying f1 . . . . . . . . . . . . . . . . . . . . . . . 246 D.3 Root Locus Varying h0 . . . . . . . . . . . . . . . . . . . . . . . 247 D.4 Root Locus Varying h1 . . . . . . . . . . . . . . . . . . . . . . . 248 E.1 Skin Conductance Circuit Diagram . . . . . . . . . . . . . . . . 250 F.1 Circuit Diagram for Phoneme Detection . . . . . . . . . . . . . 257 F.2 Electret Microphone Circuit . . . . . . . . . . . . . . . . . . . . 260 F.3 Circuit Diagram for PIC-Based Phoneme Detection . . . . . . . 261 G.1 Pin-out Diagram for PIC . . . . . . . . . . . . . . . . . . . . . . 265 H.1 Microcontroller Flowchart . . . . . . . . . . . . . . . . . . . . . 272

xiv

List of Tables2.1 Cranial Nerve Damage . . . . . . . . . . . . . . . . . . . . . . . 9

2.2 Incomplete Spinal Cord Injury Patterns . . . . . . . . . . . . . . 14 2.3 Spinal Cord Injuries Motor Classications . . . . . . . . . . . . 15 2.4 Spinal Cord Injury Functional Abilities . . . . . . . . . . . . . . 16 3.1 MMG Experimental Results . . . . . . . . . . . . . . . . . . . . 63 4.1 Icon Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . 92 4.2 TPV Experiment 2 Sequence . . . . . . . . . . . . . . . . . . . . 93 5.1 Program Steps . . . . . . . . . . . . . . . . . . . . . . . . . . . 140

5.2 Video Capture Parameters . . . . . . . . . . . . . . . . . . . . . 141 5.3 RGB24 format . . . . . . . . . . . . . . . . . . . . . . . . . . . 141

6.1 The Phonemes of Hiberno-English . . . . . . . . . . . . . . . . . 176 6.2 Classication of English Consonants . . . . . . . . . . . . . . . . 178 6.3 Spectral Peaks . . . . . . . . . . . . . . . . . . . . . . . . . . . . 203 6.4 Example Relative Harmonic Amplitudes . . . . . . . . . . . . . 205

xv

F.1 Component Values for Phoneme Detection Circuit . . . . . . . . 258 F.2 Component Values for PIC-Based Circuit . . . . . . . . . . . . . 260

xvi

Chapter 1 IntroductionThis thesis arises from work in the Engineering Research Laboratory in the National Rehabilitation Hospital (NRH)1 . Typically, the patients in this hospital are people who have become disabled as a result of a stroke, disease or accident. Advances in medical research are ensuring that more and more people survive from these disabling conditions. It is important that research follows that not only keeps these people alive, but also enables a fullling and worthwhile quality of life. Loss of speech production abilities can be one of the most devastating elements of severe physical disability. Without the means to communicate by conventional methods, people may nd themselves shut o from the outside world. Communication with other people is one of the most important actions that we as humans perform. It is important to be able to converse with loved ones, and to have a means for expressing our emotions, needs and desires. Communication with others allows us to build relationships, make requests, reach our intellectual potential and lead a stimulating and participative life. The independence of people with severe physical disabilities is also an important consideration. Results from the 2002 census from the Central Statistics1

Rochestown Ave., Dun Laoghaire, Co. Dublin, Ireland

1

Oce [1] indicate that there are 159,000 people in this country who provide regular unpaid help for a friend or family member with a long-term illness, health problem or disability. Frequent reliance on family and friends can be frustrating for the disabled person, both for practical reasons and because it can compromise a persons feelings of dignity. As technology advances, it is important to ensure that systems are developed which can provide disabled people with the ability to control their living environment, without needing assistance from others.

1.1

Assistive Technologies

For people who are unable to control their environment and communicate with others by conventional means, there are various systems available which provide alternative methods of performing these tasks. The term augmentative and alternative communication is often used to describe a range of alternative communication techniques, from the use of gestures, sign language and facial expressions to the use of alphabet or picture symbol boards [2]. In order to be able to make use of these systems it is necessary to be able to interact with the system in some way. Perkins and Stenning [3] state that the main objective for people who are unable to use a keyboard is to be able to identify a function or movement over which they have some control and utilise that. This could be from movement of the head, eyes, chin, arms, hands or feet, for example. These movements can be converted into such electrical signals as on or o switches, or, in the case of those with a little more control, variable voltages. People with very severe physical disabilities may only be capable of making very small movements to indicate intent that may be dicult to harness. The focus of this thesis is on investigating advanced methods of signal acquisition and signal processing to enable these signals to be captured and used to control communication and control devices. The principal aims of this thesis may be outlined as follows. 2

Overview of current methods of providing communication and control for disabled people. Identication of alternative signals from the body which may be harnessed from the body for communication and control purposes for people with very severe disabilities. Study of measurement techniques that may be used to acquire these vestigial signals. Investigation of signal processing methods to enable these signals to be correctly interpreted. Development of working systems that demonstrate the capabilities of these techniques. Testing of these techniques and systems with people with severe disabilities. Development of some mathematical models that evolved as a result of studying these body signals.

1.2

Thesis Layout

Some of the causes of paralysis and severe disability are outlined in Chapter 2. An overview of assistive technology applications that may be of relevance to people with very severe disabilities is given and the importance of identifying a switching action is emphasised. An alphabet board based communication tool was developed as part of this work called the Natterbox. This is also described in Chapter 2. The nervous system and the structure of muscle are given in Chapter 3, and the mechanism of muscle contraction is described. Often people who are disabled will retain some ability to contract certain muscles, but not to a sucient 3

extent to enable a mechanical switch to be used. However, the muscle contraction may still be harnessed for communication and control purposes through other means. The electromyogram is the electrical signal observable from the surface of the skin due to action potentials which occur on contraction. The electromyogram as a control signal for prosthetics and for communication and control systems is described. An alternative method of measuring muscle contraction for communication and control purposes is proposed. This method uses the mechanomyogram, which is the mechanical signal observable on the skin surface due to muscle contraction. A mechanomyogram based system for communication and control was developed and this is presented here. Some experiments were also performed with this system to assess its ecacy in controlling an alphabet board. The results of these experiments are reported. Two more biosignals are investigated in Chapter 4, the electrooculogram and the electrical conductance of the skin. The electrooculogram is the electrical signal observable around the eyes which can be used to measure eye movement. An overview of dierent eye movement measurement techniques is given and the electrooculogram is described in more detail. Some limitations of the electrooculogram signal as a communication and control signal are identied and a novel technique is presented that seeks to overcome these limitations to allow the electrooculogram to be used as a control signal. Study of movement of the eyes led to development of a mathematical model of the eye, which is also presented in Chapter 4. This model incorporates the eect of the muscle spindle on the eyes torque and predicts saccadic and smooth pursuit eye movements. The electrical conductance of the skin is also briey explored as a control signal. Electrical skin conductance is related to sweat gland activity on the surface of the skin and may be modulated by tensing or relaxing, as will be discussed. Resulting from this study, a technique for measuring the ring rate of the sympathetic nervous system was developed which uses measurement of the skin conductance as its input. Visual techniques are discussed in Chapter 5, which use a computer camera 4

or another light sensitive device to measure movement. Often people who have become disabled will retain the ability to make ickers of movement of a certain body part, for example a nger or a thumb. If these movements are repeatable then they may be used to indicate intent. A novel algorithm for describing specic paths of motion is presented. This algorithm is incorporated into a software program, which detects specic movements and uses them to generate a switching action. This switching action can then be used to control any communication and control application operable by one switch. Acoustic methods of harnessing signals from the body are explored in Chapter 6. For people who have speech production abilities, there is a wide range of speech recognition technologies available that allow environmental control using the voice. For those who are unable to speak, there may still be ways of harnessing acoustic signals from the body. Often people who have lost the ability to produce speech will still remain capable of producing non-verbal utterances. If these utterances are repeatable then they may be used as the basis of a communication and control system. A number of acoustic based systems were developed as part of the work described here and these are presented in this chapter. A system for controlling a reading machine, an environmental controller and an alphabet board based communication device are given. The conclusions drawn from the research presented here are given in Chapter 7. Suggestions are made for future work in the area of communication and control for disabled people.

5

Chapter 2 Assistive Technology

2.1

Introduction

Assistive technology is dened by Lazzaro [4] as any device that enables persons with disabilities to work, study, live, or play independently. Cook and Hussey [5] describe it as any device or technology that increases or improves the functional capabilities of individuals with disabilities. Assistive technology may oer assistance to people with a wide range of disabilities including vision, hearing, motor, speech and learning impairments. Screen magniers and braille are assistive technologies for blind or partially blind persons. Hearing aids and subtitled lms may be classed as assistive technologies for the deaf. This thesis focuses on assistive technologies for people who, for one reason or another, require assistance to communicate with others and to control their environment. A principal aim of this thesis is to explore ways in which signals from the body may be harnessed so that people with extremely severe physical disabilities can interact with control and communication devices. In this chapter, some of the possible causes of paralysis are rst described in Section 2.2. Section 2.3 reviews some of the available assistive technology devices that may be of benet to such people. An application called the

6

Natterbox is presented in Section 2.4. This communication application was developed as part of this work to act as a testing board for switching action methods described in later chapters.

2.2

Causes of Paralysis

There are many dierent circumstances that will lead to a person requiring the use of an assistive device to communicate with others or to control their environment. Paralysis can result from spinal injury following a road trac accident or other trauma. It can be caused by damage to the brain due to a brain haemorrhage or a tumour. Motor neurone diseases, which cause wasting of the muscle tissue, may eventually lead to paralysis, and necessitate use of a communication and control device. Some of the reasons that may lead to a person becoming severely physically disabled are discussed in this section although this review is by no means extensive. A major focus of this thesis is on exploring a range of available options, so that a suitable assistive technology system may be identied for each individual user, based on their capabilities and requirements, rather than oering one single solution that will allow all severely disabled people to use a control and communication device. Similarly, it is impossible to state here the exact group of people who might benet from the methods described in this thesis. Some of the more common causes of paralysis will now be discussed.

2.2.1

Neurological Damage

Neurological damage, or damage to the brain, can occur due to a number of dierent circumstances. One of the most commonly occurring reasons is due to a stroke. The Irish Health Website [6] estimates that 8500 people in this country suer from a stroke annually.

7

Stroke is not a disease in itself, but a syndrome of neurological damage caused by cerebrovascular disease [7]. Although paralysis is the most commonly associated aspect of a stroke, the stroke syndrome consists of a number of dierent aspects which also include spasticity, contractures, sensory disturbances, psychological impairments, emotional and personality changes and apraxia (the loss of ability to carry out familiar purposeful movements in the absence of paralysis [8]). A stroke occurs when normal blood circulation in the brain is interrupted, either due to occlusion caused by a blood clot (an ischemic stroke) or through sudden bursting of blood vessels (a haemorrhagic stroke). Strokes due to blood clots may be divided into two categories. Cerebral thrombosis occurs due a clot that develops in situ and cerebral embolism is caused by a clot that forms elsewhere in the body and travels up to the brain [7]. Paralysis can result from damage to the frontal lobe and/or damage to the internal capsule bres. The frontal lobe of the brain contains the motor area, which connects to the motor cranial nerve nuclei and the anterior horn cells. The internal capsule of the brain is the narrow pathway for all motor and sensory bres ascending from lower levels to the cortex. Damage to one side of the motor bres or the frontal lobe leads to loss of power in the muscles on the side of the body opposite the lesion [9], a paralysis known as hemiplegia [8]. While paralysis is the main symptom of a stroke relevant here, some other symptoms caused by damage to the cranial nerves are summarised in Table 2.1. The cranial nerves exist in pairs and damage to one of the nerves may result in the symptoms listed at the side of the lesion. Note that damage to the tenth nerve is one of the causes of total or partial loss of speech production abilities. Speech impairments will be discussed in more detail in Chapter 6. Following a stroke, some voluntary movement may return within a few weeks of the incident. This is usually due to a number of causes. Following cerebral infarction and particularly in the case of a cerebral haemorrhage, abnormally large amounts of uid in the surrounding tissue can temporarily 8

Table 2.1: Signs and symptoms of cranial damage, adapted from [10], pg. 100 Nerve V Nametrigeminal

Signs and Symptoms of DamagePain and burning on outer and inner aspect of cheek Loss of sensation over face and cheek

VI VII VIII

abducens facial auditory

Diplopia, external rectus weakness, squint Weakness of face Vertigo, vomiting, nystagmus Deafness and tinnitus

IX X

glossopharyngeal vagus

Loss of taste Dysphagia Paralysis of vocal cord and palate

disrupt neurological function. As the pressure subsides, the neurons in this area may regain function. Motor function may also be restored due to central nervous system reorganisation where other areas of the brain take on the role of voluntary motor control [7]. This partial return of voluntary movement following a stroke may be of enormous benet when considering methods for enabling stroke victims to interact with control and communication systems.

2.2.2

Spinal Cord Injuries

Spinal cord injuries usually occur as the result of a trauma, which is often caused by a road trac accident or a domestic, sporting or work-related injury. The basic anatomical features of the spine and the innervation of the spinal cord will rst be discussed and the classications of spinal cord injury will then be described.

Structure of the Vertebral Column and the Spine The spinal cord is protected by the vertebral column, a line of bony vertebrae that runs down the middle of the back. The structure of the vertebral column 9

is shown in Figure 2.1. When viewed from the side, the vertebral column displays ve curves - an upper and lower cervical curve, and one each thoracic, lumbar and sacral [11]. The sacral curve is not shown in Figure 2.1 but it is located at the very bottom of the vertebral column, from the lumbarsacral junction to the coccyx. The coccyx is better known as the tailbone, which is made up of several fused vertebrae at the base of the spine [12]. The spinal cord terminates before the end of the vertebral column, around the top of the lumbar vertebrae in adults [13]. The lower tip of the spinal cord is called the conus medullaris [8]. The area from the conus medullaris to the coccyx is known as the cauda equina [13].

The Cervical Spine The purpose of the cervical spine is mobility. The two curves in the cervical spine can be divided into upper and lower segments at the second cervical vertebra. The rst cervical vertebra (C1) is called the atlas and the second cervical vertebra (C2) is called the axis. The upper cervical muscles move the head and neck and are principally concerned with positioning of the eyes and the line of vision, hence these muscles are highly innervated to enable these movements to be made with a ne degree of precision [11]. The axis provides a pivot about which the atlas and head rotate. The lower cervical spine (C2-C7) also contribute to movement of the head and neck. The Thoracic Spine and Ribs An important function of the thoracic spine and rib cage is to protect the heart, lungs and major vessels from compression. Due to this, the thoracic area is the least mobile region of the spine. The thoracic vertebrae are numbered T1-T12 and the ribs are numbered R1-12 on each side. The diaphragm muscle bres are attached to ribs R7-R12.

10

Figure 2.1: The Vertebral Column, from pg. 2 in [11]

11

The Lumbar Spine The lumbar spine is made up of ve vertebrae numbered L1-L5. The fth lumbar vertebra (L5) is the largest and its ligaments assist in stabilising the lumbar spine to the pelvis. There are 31 pairs of spinal nerves attached to the spinal column. Each pair is named according to the vertebra to which they are related. The spinal nerves are shown in Figure 2.2.

Classication of Injury Injury of the spinal cord may produce damage that results in complete or incomplete impairment of function. A complete lesion is one where motor and sensory function are absent below the level of injury. A complete lesion may be caused by a complete severance of the spinal cord, by nerve bre breakage due to stretching of the cord or due to a restriction of blood ow (ischaemia) to the cord. An incomplete lesion will enable certain degrees of motor and/or sensory function below the injury [14]. There are recognised patterns of incomplete spinal cord injuries,which are summarised in Table 2.2. A spinal cord injury may produce damage to upper motor neurons, lower motor neurons or both. Upper motor neurons originate in the brain and are located within the spinal cord. An upper motor neuron injury will be located at or above T12. Upper motor neuron injury produces spasticity of limbs below the level of the lesion and spasticity of bowel and bladder functioning. Lower motor neurons originate within the spinal cord where they receive nerve impulses from the upper motor neurons. These neurons transmit motor impulses to specic muscle groups and receive sensory information which is transmitted back to the upper motor neurons. Lower motor neuron injuries may occur at the level of the upper neuron but more commonly are identied when occurring at or below T12. Lower motor neuron injuries produce accidity of the legs, decreased muscle tone, loss of reexes and atonicity of bladder and bowel [14]. 12

Figure 2.2: The Spinal Nerves, pg. 208 in [11]

13

Table 2.2: Patterns of incomplete spinal cord injuries, from text in [14] SyndromeCentral Cord

Damaged Area Common CauseCervical Region Hyperextension injury

CharacteristicsFlaccid arm weakness Good leg function Injured Side Loss of Motor Function Uninjured Side Loss of temperature & pain sensation

Brown-Squard

Hemisection of Spinal Cord

Stab Wound

Anterior Cord

Corticospinal & spinothalamic tracts

Ischaemia & direct trauma

Variable loss of motor function Reduced sensitivity to pain and temperature

Conus medullaris/ cauda equina

Sacral cord or the cauda equina nerves

Flaccid bladder and bowel Loss of leg motor function

Spinal cord injuries due to complete lesions are usually classied according to the level of injury to the spine. Table 2.3 summarises the motor classication of spinal cord injury. The word paraplegia describes lower lesion spinal cord injuries resulting in partial or total loss of the use of the legs. The words tetraplegia and quadriplegia both describe high level spinal cord injuries, usually occurring due to injury of the cervical spine. Both terms mean paralysis of four limbs and the injury causes the victim to lose total or partial use of their arms and legs [15]. The main causes of spinal cord injury may be gauged from gures from the Duke of Cornwall Spinal Treatment Centre, which are given in [16]. For the new patient admissions with spinal injuries for the period 1993-1995, 36% are due to road trac accidents, 6.5% are due to self harm and criminal assault, 37% are due to domestic and industrial accidents and 20.5% are due to injuries at sport. Until recently spinal cord injury was recognised as a fatal condition.

14

Table 2.3: Motor classication of spinal cord injury, adapted from pg. 63 in [14] Level C4 C5 C6 C7 C8 T1 Muscles Deltoids Elbow Flexors Wrist Extensors Elbow Extensors Finger Flexors Finger Abductors Level L2 L3 L4 L5 S1 S4-S5 Muscles Hip Flexors Knee Extensors Ankle dorsiexors Long toe extensors Ankle Plantar Flexors Anal contraction

In the First World War, 90% of patients who suered a spinal cord injury died within one year of wounding and only about 1% survived more than 20 years [16]. The chances of survival from a spinal cord injury began to increase in the 1940s with the introduction of sulfanilamides and antibiotics [14]. Nowadays, due to better understanding and management of spinal cord injury, the outlook has greatly improved for people with spinal cord injuries. There has been a gradual change in the pattern of survival from low-lesion paraplegia in the 1950s, high-lesion paraplegia in the 1960s and low-lesion quadriplegia in the 1970s. Finally, in the 1980s, people with spinal cord injuries at or above C4, resulting in high-lesion quadriplegia, have been surviving in signicant numbers. It is estimated that each year in the USA, 166 sustain injury at C1-C3 and 540 people at C4 [14]. As medicine advances, such individuals will survive in increasing numbers and thus it is important to identify methods for interaction with communication and control systems for this group of severely disabled individuals. The functional ability of tetraplegic patients based on the level of injury are summarised in Table 2.4. In general, movements of the limbs suer more severely than those of the head, neck and truck. Movements of the lower face also tend to be more severely impaired than those of the upper face [10].

15

Table 2.4: Expected functional ability based on level of injury, constructed usinginformation from [16].

Level of Injury Complete lesion below C3

Functional Ability Dependent on others for all care Chin and head movement Can use breath controlled devices

Complete lesion below C4

Dependent on others for all care Chin and head movement Shoulder shrugging possible Can type/use computer using a mouth stick

Complete lesion below C5

Shoulder movement Elbow exion

Complete lesion below C6 Complete lesion below C7

Wrist Extension Full wrist movement Some hand function

Complete lesion below C8 Complete lesion below T1

All hand muscles except intrinsics preserved Complete innervation of arms

16

2.2.3

Diseases of the Nervous System

The words motor neurone disease (MND) and amyotrophic lateral sclerosis (ALS) are often used interchangeably. However, amyotrophic lateral sclerosis may be described more accurately as a type of motor neurone disease, and probably the most well known. Motor neurone diseases aect the motor nerves in the brain and the spinal cord [17] and the term motor neurone disease may be used to describe all the diseases of the anterior horn cells and motor system, including ALS [18]. Motor neurone diseases may be divided into two categories - idiopathic motor neurone diseases and toxin-related motor neurone diseases. An idiopathic disease is one of spontaneous origin [8]. The idiopathic motor neurone diseases include both the familial and juvenile forms of amyotrophic lateral sclerosis. Also included under this category are progressive bulbar palsy (PBP), progressive muscular atrophy (PMA), primary lateral sclerosis (PLS), Madras motor neurone disease and monomelic motor neurone disease [18]. The toxinrelated motor neurone diseases are suspected to be linked to environmental factors [18]. These include Guamanian ALS (due to a high incidence of ALS in Guam), lathyrism and Konzo. The exact gure for the number of people diagnosed with ALS varies, but it is thought to aect between 1-3 in every 100,000 of the population each year [17, 18]. There are an estimated 300 people living with amyotrophic lateral sclerosis at any one time in Ireland [17]. ALS is a progressive fatal disease of the nervous system and the rate of progression depends on the individual [18]. The muscles rst aected by motor neurone diseases tend to be those in the hands, feet or mouth and throat. As ALS progresses, the ability to walk, use the upper limbs and feed orally are progressively reduced. In the terminal stage of the disease, none of these functions can be independently performed and respiratory functions become compromised [18]. At this stage of the disease, it is as important as ever to give the person the best quality of life possible and

17

assistive technologies must be considered that can harness the vestigial signals left to these people. Usually, the motor function of the eye muscles is spared due to the calcium binding proteins in these nerve cells [18] and this feature could be used to provide a method of control and communication, as will be discussed in Chapter 4. Brain computer interface (BCI) technologies are also often considered at the very latest stages of the disease, these will be briey described in Section 2.3.3. Paralysis can also occur due to demyelinating diseases such as multiple sclerosis. A demyelinating disease causes impairment of conduction of signals in nerves as it damages the myelin sheath of neurons. More about the structure of nerves will be described in Chapter 3. Neurological damage resulting in paralysis may also occur due to viral infections such as poliomyelitis or polio [10] or due to bacterial infections such as bacterial meningitis, which aects the uid in the spinal cord and the uid surrounding the brain [19].

2.3

Assistive Technology

Assistive technologies can be of immense benet to people with severe physical disabilities such as those described above. As mentioned already, this thesis focuses mainly on facilitating interaction with two type of assistive technology applications - control and communication. Communication applications are usually described in assistive technology terms as augmentative and alternative communication (AAC) systems [2]. Augmentative and alternative communication systems refer to assistive technology systems designed for people who have limited or no speech production abilities. Alternative communication systems usually consist of some sort of alphabet board or symbolic board [4]. Some alternative communication systems display text to a computer screen, others output the text to a printer and some work in conjunction with speech synthesis systems to speak out the in-

18

tended message. Some are computer operated and some are handheld, such as the handheld LightWriter1 , a dual display keyboard based communication aid. Some, such as Voicemate2 , allow the user to record phrases for digitised playback [4]. Control applications refer to any system that can be operated automatically using a control signal. For example, a control signal could be used to handle an environmental control system to operate appliances in the users environment, such as lights, fans or the television. The reading machine described in Chapter 6 is another example of a system that may be operated using a control signal. Control signals can also be used to operate wheelchairs or electrically powered prosthetics. The electromyogram muscle signal is often harnessed to replace muscle function to control prosthetics for amputees, as described in Chapter 3.

2.3.1

Importance of a Switching Action

The simplest control signal is probably the switching action, which is any action that allows the user to alternate between two possible states, on or o. There are numerous systems in use today that may be operated by pressing a single switch or multiple switches. Such systems are often called switch-activated input systems [2]. A standard computer keyboard may be described as a switch based system for interfacing with a computer. The keyboard usually has around 100 keys or switches and each key press sends a control signal to the processor which is recognised as a dierent letter or symbol by the computer. The combination of two or more key presses may also be used to increase the number of possible control signals [5]. There are many types of commercially available switches and a comprehensive guide to switches is given in [20]. The standard type of switch is the paddle1 2

Lightwriter, Zygo Industries, Inc., P.O. Box 1008, Portland, OR 97202 USA Tash Inc., Unit 1, 91 Station Street, Ajax, Ont. L1S 3H2, Canada.

19

type switch. These mechanical switches have movement in one direction and can be activated by the user by pressing on the switch with any part of the body. For persons who do not have sucient strength or ability to operate these switches there are a number of other types of switches available. These switches include suck-pu switches, wobble switches, leaf switches and lever switches [5, 21]. The switch chosen for a particular individual will depend on the capabilities of the user. For people who are very severely physically disabled, performing a switching action using any of these physical switches may not be an option. In these cases, other methods of harnessing signals from the body to provide a switching signal must be explored. One of the main objectives in developing alternative systems for communication and control is to be able to correctly identify two or more distinct states that a user can voluntarily elicit. If these states can be reliably distinguished, then transition from one state to another can be harnessed as a means of eecting a switching action.

2.3.2

Switch Based Systems

Switches are generally used in one of two ways - in a scanning system or in a coding system. In a coding system, the user taps out a message using some scheme such as the famous Morse code, using the switch. The Morse code software functions like a translator, converting Morse code to text in real time [4]. The coding can either be done using one switch with long switch presses for the dash and short switch presses for the dots, or using two separate switches to represent dots and dashes [2]. Morse code based systems have the disadvantage that the code must rst be learnt by the user. A more popular type of switch-activated input system uses scanning based selection. These systems are usually based on some variation of the rowscanning method described by Simpson and Koester [22]. The user is presented with a screen of options, arranged in rows and columns. The program scans 20

through the rows and the user can select a particular row by pressing a switch. The program then scans through each item on the selected row and the user can select the desired item by pressing a switch again. Row scanning is often used in software alphabet boards and can be used to spell out messages [2]. The idea of switch based menu selection has been around for years. The personal computer became popular in the early 1980s and software based assistive technology systems soon followed. An independent living system known as the ADAPTER program was developed around 20 years ago by a team in Lousiana Tech University in the USA [23]. This program uses the row-scanning method to allow the user to select one of several tasks from a menu. The ve options given are letters, words, codes, phone and environment. The program is designed to be operated with a mechanical switch and the two examples mentioned are a push-button switch and a bulb-pressure switch. If the user selects the letter option on the main menu then they will be presented with a second sub-menu with rows of letters and numbers which allows messages to be spelled out. The word option provides quick access to a list of important words e.g. light, water, bath etc. Selection of the code option allows communication through Morse code by pressing the switch for long or short periods which is then converted to text. The phone option displays a pre-programmed list of names and phone numbers which may be dialled through the computer and the environment option allows control of appliances in the users surroundings. Another scanning based alphabet board system developed around this time is described in [21], in which the scanning device is a hardware logic-based module that uses LEDs to highlight each character. This device can be connected to the computer as a substitute for a manually operated keyboard. The system uses two switches to scan through the characters and enter the required character into the computer. Damper [24] estimates that a communication rate of 6-8 words per minute is typically achieved using an alphabet board based communication system. There have been a number of dierent methods suggested for increasing the 21

rate at which the user can select the letters. Perkins and Stenning [3] experimented with the idea of using two or ve switches to operate an alphabet board and also tested the communication rate with dierent menu layouts. The two layouts tested had 57 characters - one had letters and each number once and the second had additional characters related to frequency of use (e.g. the letter E appears on the board ve times) but no numbers. Simpson and Koester [22] have proposed a method of increasing text entry rate using an adaptive row-column scanning algorithm which increases or decreases the scan delays according to user performance. Although it is not yet implemented as a switch based text entry system, the Dasher program by Ward [25] will briey be described. Rates of 39 words per minute have been claimed for it when operated using a mouse and 25 words per minute when operated using eye tracking. It is a software based program which enables a person to spell out words by steering through a continuously expanding two-dimensional scene containing alphabetical listings of the letters [26]. A screenshot from this program is shown in Figure 2.3. The line in the centre of the screen is the cursor. The user is initially presented with an alphabetical list of letters and the user selects a letter by moving the cursor inside the area of the letter. As the user approaches a letter the letter grows in size. Once the letter is selected the user will again nd themselves presented with another list of letters but the relative sizes of all the letters on the new list is based on the probability of this letter being the desired letter based on the previous letter. Dasher uses a language model to predict this, and the model is trainable on example documents in almost any language [26]. In the example shown in Figure 2.3, the user is spelling out the word demonstration and has already selected demonstrat. As the user moves the cursor closer towards the letter i, the letter grows in size until the user is inside the box. The screenshot also illustrates alternative words that could instead have been selected such as demolished, demonstrated that, demoralise and demonstrative. A number of dierent methods for interfacing with the

22

Figure 2.3: Dasher program - spelling out the word demonstration. Dasher program are suggested on the Dasher website [27], including a mouse, a joystick, eye-tracking and head-tracking. Future possible developments of Dasher are described in [26], and include a suggestion for a modied method for operation using a single switch. This will allow the user to operate Dasher using a switch that changes the direction of cursor movement on activation.

2.3.3

Brain Computer Interfaces

Brain computer interfaces (BCI) may oer another method of providing switching actions in cases of very severe disability. Brain computer interfaces are usually used in situations of very severe disability where there is no other method of communication and control possible. These methods allow the user 23

to interact with the computer using some measurement of brain activity, such as function magnetic resonance imaging (fMRI) or the electroencephalogram, the electrical signal measurable from the surface of the scalp. Correct interpretation of these signals can be used to convey user intention and thus actuate a switching action. The area of brain computer interfaces for the disabled is a huge research area and the interested reader is referred to the IEEE review of the rst international BCI technology meeting [28] as a starting point for more information.

2.4

Communication Device

A software communication device called Natterbox was developed as part of this study, based on an alphabet board. The code for this program is included in Appendix I. Although there are many similar communication programs available commercially, this program was developed for two reasons. Firstly, it was in response to a request made by one of the occupational therapists in the hospital, who had been using a previous version of the same program, which had been developed earlier in our laboratory in the NRH. She was attempting to use the system with a male patient who had suered from a brainstem stroke. The patient had poor visual ability and was also very photosensitive. This rendered him unable to see the letters of the alphabet board on screen. She suggested making each of the rows of the alphabet board a dierent colour, in accordance with the layout of physical alphabet boards used by occupational therapists. An auditory facility was then added which speaks out the colours on each of the dierent rows as they are highlighted. The patient was able to learn which letters corresponded to which coloured row and hence could perform a switching action when the program called out the name of the row that was desired. The program then calls out each letter in that row in turn, and the user can again select the desired letter when it is reached, thus enabling the user to spell out messages.

24

The second benet gained from development of the Natterbox program is that it served as a useful testing board for dierent switching mechansims developed in the work presented here. Since the Natterbox allows the user to spell out words and sentences simply by performing a single switching action, it was an invaluable tool in demonstrating translation of dierent body signals into communication. The Natterbox program as described here was used by a number of dierent patients in the hospital. For each of these patients, a reliable method of interfacing with the program had to be identied and some of the techniques used are discussed in this thesis. As the program developed, various features were added in response to therapist and patient requests. Some of these will now be briey outlined.

2.4.1

Technical Details

The Natterbox program was developed with C++ using the Fast Light Tool Kit3 (FLTK) to develop the graphical user interface. The sound feature was added using tools from the Simple Directmedia Layer4 (SDL), which is a C++ multimedia library designed to provide access to audio devices. The primary advantage of using FLTK and SDL is that they are both cross-platform, making the Natterbox program portable across dierent operating systems.

2.4.2

The Natterbox Graphical User Inteface

The graphical user interface (GUI) of the Natterbox main menu is shown in Figure 2.4, demonstrating a message being spelled out. In Figure 2.4(a), the yellow row is highlighted. The user activates a switch to select this row and the program begins scanning the letters on that row. In Figure 2.4(b), the symbol . is highlighted. The user again activates a switch to select this symbol. Figure 2.4(c) shows that the symbol has appeared on the message banner and3 4

FLTK Website: http://fltk.org SDL Website: http://www.libsdl.org

25

also on the history panel along the right-hand side of the screen.

2.4.3

Switch Interface Box

The switch input required by Natterbox was chosen to be an F2 keypress. Thus Natterbox can be used in one of three ways. Firstly it is operable by simply pressing the physical key on the keyboard. Obviously this is not a very useful interaction method for people with very severe disabilities. Secondly, it may be used in conjunction with another program that is monitoring some signal from the body and will simulate an F2 keypress when it recognises intention. Possible methods for harnessing body signals for these purposes forms much of the remainder of the this thesis. Thirdly, it may be used with a switch interface box. Any arbitrary two way switch, such as those mentioned in Section 2.3.1, can be connected to this box. The switch interface box is connected into the USB port of the computer and a supplementary software application simulates an F2 key press on detection of a switching action. The supplementary program was called USB Switch and the code is given in Appendix I.

2.4.4

Other Features

Phrases Menu Due to requests from the occupational therapists in the hospital, the option of a sub-menu was added to the Natterbox program. This sub-menu provides quick access to a list of commonly used phrases. This menu may be opened by selecting the last row in the main menu. The sub-menu screen is shown in Figure 2.5(a). When the user selects the phrase Turn on or o fan it appears in the message banner back in the main screen. This phrase could be used by the user to request that the fan is turned o if it is already on, or turned o if it is on. 26

(a)

(b)

27

(c)

Figure 2.4: The Natterbox program (a) The program is highlighting the second(yellow) row. (b) When the user selects the second row the user begins scanning the letters on this row. The . button is currently highlighted. (c) The user selects this symbol and it appears above on the banner.

28

Printing Feature An option to print the message to paper was added in response to a request from a patient who wanted a facility for writing letters to her children. This request was fullled by placing an option Print at the bottom of the phrases menu. Selection of this option sends all the text in the history box to an attached printer. This option could be of immense benet to users since it allows the user to prepare lengthy messages in advance.

Cancel Feature A cancel option was added for people who are capable of actuating a second switching action. The second switch input cancels the eect of the last input. Thus if the user has accidently selected a letter they may delete this letter from the message bar by activating the second switch. If the user has accidently selected the wrong row and the program is scanning through each of the items on that row, the user may use the second switch to change back to row scanning.

Three-Switch Mouse A three-switch mouse was developed for one of the patients who was in the hospital who was particularly successful with the Natterbox program. The patient used a push-button switch placed between his thumb and hand to operate the program. He also had head movement on both sides so was able to operate two head switches. The Natterbox program was modied to include a mouse cursor control system using these three switching actions. The patient could exit the alphabet board program by selecting an Exit option at the end of the phrases menu. This switches the program into mouse cursor control mode. The mouse cursor is controlled by the USB Switch program. The head switches may be used to move the mouse cursor either up and 29

(a)

(b)

Figure 2.5: The Natterbox Phrases Menu (a) The program is highlighting the secondphrase Turn on or o the fan. (b) When the user selects this phrase it appears on the banner back in the main menu.

30

down, or left and right. Switching between these two directions is performed using the hand switch. Pressing the hand switch twice in succession actuates a mouse click.

2.4.5

Possible Future Developments of Natterbox

The addition of a submenu to Natterbox containing numbers and punctuation marks could be of great benet. In addition to adding to user dignity by making the messages look more presentable, they could also enable emoticons to be used to add more meaning to messages. Emoticons are being more and more popular nowadays due to emailing, instant messaging and text messaging. Emoticons (emotion icons) are a method of adding symbols to the end of messages to represent dierent facial expressions. These can be used to communicate more eectively what is meant by the message. For instance, the simple term Its ok could be interpreted in a number of dierent ways. It can be intended straightforwardly and this can be emphasised by placing a smiley face symbol at the end of the message i.e. Its ok :-). Conversely, if the person wishes to impart some sort of satirical tone to the message, they may express this by adding the sad smiley Its ok :-( or the angry smiley symbol Its ok :-@, depending on intent. These emoticon symbols are becoming more and more integrated into casual everyday written communications and could oer an immense benet to people who are severely disabled and wish to more eectively convey their emotions when writing messages. The addition of a speech synthesiser to the complete program to allow the messages to be spoken out loud is also being considered.

2.5

Conclusions

This chapter has outlined some of the diseases, conditions and circumstances that may render a person severely physically disabled. A review of assistive 31

technology applications has been given and the importance of generating a switching action has been emphasised. Now that these areas of been discussed, the aims of this thesis may be more accurately dened. This thesis aims to investigate alternative methods of harnessing vestigial signals from people who have been severely paralysed and have very little motor function, such as those with high-level lesions above C4. These people may be unable to operate a mechanical switch and thus require a more complex technique to be identied that will allow a switching action to be actuated. A large part of the remainder of this thesis focuses on methods of harnessing these vestigial signals to provide switching actions and other control signals.

32

Chapter 3 Muscle Signals

3.1

Introduction

This chapter and Chapter 4 investigate methods of harnessing bio-signals from the body for control and communication purposes. The exact criteria required to enable a particular body signal to be described as a bio-signal are not always well dened. In the broadest sense of the word, a bio-signal may refer to any signal from the body related to biological function. Under this denition, all of the signals presented in this thesis would fall under the category of bio-signals, including the signals obtained through video capture techniques, described in Chapter 5, and speech signals obtained through audio signal processing techniques, described in Chapter 6. A more narrow denition of the term biosignals is meant here. A bio-signal as discussed in this thesis refers to any signal that is measurable directly from the surface of the skin. This includes signals such as biopotentials, which are measured voltages from certain sites on the body, but also other electrical signals, such as the electrical skin conductance, and mechanical signals, such as the mechanomyogram. This chapter discusses two bio-signals which may be used to detect muscle contraction. These are the electrical signal, the electromyogram (EMG),

33

and the mechanical signal, the mechanomyogram (MMG). Muscle signal based switching systems may be an option for people who retain some ability to contract certain muscles but may not be able to operate a mechanical switch. This may be because the particular muscle that can be contracted is not suitable for operating a switch or because the muscle contraction is not strong enough to operate the switch. This chapter investigates how deliberate muscle contraction can be used to eect a switching action to operate control and communication systems. The anatomy and physiology of the nerves and the nervous system are rst described in Section 3.2.1. Action potentials and the method of information transfer in the body are described in Section 3.2.2. The anatomy of muscle and the process of muscle contraction are discussed in Section 3.3. Some dierent muscles that may be suitable for use for an EMG-based or MMG-based system are identied in Section 3.3.3. The electromyogram as a control signal is discussed in Section 3.4. Finally the possibility of using the mechanomyogram as a control signal is explored in Section 3.5.

3.23.2.1

The Nervous SystemNerves and the Nervous System

The Nerve Cell The basic building block of the human bodys nervous system is the nerve cell, or neuron. The neurons in the body are interconnected to form a network which is responsible for transmitting information around the body. The spinal cord, the brain and the sensory organs (such as the eyes and ears) all consist largely of neurons. The structure of a neuron is shown in Figure 3.1. The central part of

34

Figure 3.1: The Nerve Cell, from pg. 2 in [29] the neuron is the cell body, or soma, which contains the nucleus. The cell body has a number of branches leading from its centre, which can either be dendrites or axons. The dendrites receive information and the axons transmit information, both in the form of impulses, which will be described in more detail later. There is generally only one axon per cell. The axon links the nerve cell with other cells, which can be nerve cells, muscle cells or glandular cells. In a peripheral nerve, the axon and its supporting tissue make up the nerve bre. A bundle of nerve bres is known as a nerve.

Classication of Nerve Fibres The peripheral nervous system refers to the neurons that reside outside the central nervous system (CNS) and consists of the somatic nervous system and the autonomic nervous system [30]. A nerve bre may be classied as either an aerent nerve bre or an eerent nerve bre. An aerent nerve bre transmits information to the neurons of the CNS and the eerent nerve bre transmits 35

information from the CNS. Aerent nerve bres may further be divided into somatic nerve bres and visceral nerve bres. Visceral aerents are nerve bres from the viscera, which are the major internal organs of the body. All other aerent nerve bres in the body are called somatic aerents. These come from the skeletal muscle, the joints and the sensory organs such as the eyes and ears, and bring information to the CNS. Eerent nerve bres can be categorised as either motor nerve bres or autonomic nerve bres. Motor eerents control skeletal muscle and autonomic eerents control the glands, smooth muscle and cardiac muscle. See Figure 3.2 for a summary of nerve bre classications. The visceral aerent nerve bres and the autonomic eerent nerve bres both belong to the autonomic nervous system. The autonomic nervous system is responsible for controlling such functions as digestion, respiration, perspiration and metabolism which are not normally under voluntary control. The function of perspiration, controlled by the autonomic nervous system will be described in more detail in Chapter 4.

Sensory Organs Skeletal Muscle Joints

Somatic

Visceral

Central Nervous System

Motor

Skeletal Muscle

Autonomic

Viscera

Afferents

Efferents

Cardiac Muscle Smooth Muscle Glands

Figure 3.2: Classication of Nerve Fibre Types

Supporting Tissue Neurons are supported by a special type of tissue constructed of glial cells. These cells perform a similar role to connective tissue in other organs of the body. In a peripheral nerve, every axon lies within a sheath of cells known as 36

Figure 3.3: (A) Myelinated Nerve Fibre (B) Unmyelinated Nerve Fibres, from pg.8 in [29].

Schwann cells, which are a type of glial cell. The Schwann cell and the axon together make up the nerve bre. A nerve bre may be either a myelinated nerve bre or an unmyelinated nerve bre depending on how the Schwann cells are positioned around the axon. Myelinated nerve bres have a higher conduction velocity than unmyelinated nerve bres. About two-thirds of the nerve bres in the body are unmyelinated bres, including most of the bres in the autonomic nervous system, since these processes generally do not require a fast reaction time. In myelinated nerve bres, the Schwann cell winds around the axon several times as shown in Figure 3.3. A lipid-protein mixture known as myelin is laid down in layers between the Schwann cell body, forming a myelin sheath. This sheath insulates the nerve membrane from the conductive body uids surrounding the exterior of the nerve bre. The myelin sheath is discontinous along the length of the axon. At regular intervals there are unmyelinated sections which are called the Nodes of Ranvier. These nodes are essential in enabling fast conduction in myelinated bres [29]. As mentioned in Chapter 2, diseases such as multiple sclerosis damage the myelin sheath of neurons, or dymyelinate the bres along the cerebrospinal axis [10]. Paralysis occurs due to impairment of the conduction of signals in demyelinated nerves. 37

3.2.2

Resting and Action Potentials

The Membrane Potential A potential dierence usually exists between the inside and outside of any cell membrane, including the neuron. The membrane potential of a cell usually refers to the potential of the inside of the cell relative to the outside of the cell i.e. the extracellular uid surrounding the cell is taken to be at zero potential. When no external triggers are acting on a cell, the cell is described as being in its resting state. A human nerve or skeletal muscle cell has a resting potential of between -55mV and -100mV [29]. This potential dierence arises from a dierence in concentration of the ions K+ and Na+ inside and outside the cell. The selectively permeable cell membrane allows K+ ions to pass through but blocks Na+ ions. A mechanism known as the ATPase pump pumps only two K+ ions into the cell for every three Na+ cells pumped out of the cell resulting in the outside of the cell being more positive than the inside. The origin of the resting potential is explained in further detail in [29].

The Action Potential As mentioned already, the function of the nerve cell is to transmit information throughout the body. A neuron is an excitable cell which may be activated by a stimulus. The neurons dendrites are its stimulus receptors. If the stimulus is sucient to cause the cell membrane to be depolarised beyond the gate threshold potential, then an electrical discharge of the cell will be triggered. This produces an electrical pulse called the action potential or nerve impulse. The action potential is a sequence of depolarisation and repolarisation of the cell membrane generated by a Na+ current into the cell followed by a K+ current out of the cell. The stages of an action potential are shown in Figure 3.4.

38

mV

30

3

20

455 70

Threshold

6Resting Potential

1 5

Figure 3.4: An Action Potential. This graph shows the change in membrane potential as a function of time when an action potential is elicited by a stimulus. The time duration varies between bre types.

Stage 1 - Activation When the dendrites receive an activation stimulus the Na+ channels begin to open and the Na+ concentration inside the cell increases, making the inside of the cell more positive. Once the membrane potential is raised past a threshold (typically around -50mV), an action potential occurs. Stage 2 - Depolarisation As more Na+ channels open, more Na+ ions enter the cell and the inside of the cell membrane rapidly loses its negative charge. This stage is also known as the rising phase of the action potential. It typically lasts 0.2 0.5ms. Stage 3 - Overshoot The inside of the cell eventually becomes positve relative to the outside of the cell. The positive portion of the action potential is known as the overshoot.

39

Stage 4 - Repolarisation The Na+ channels close and the K+ channels open. The cell membrane begins to repolarise towards the resting potential. Stage 5 - Hyperpolarisation The membrane potential may temporarily become even more negative than the resting potential. This is to prevent the neuron from responding to another stimulus during this time, or at least to raise the threshold for any new stimulus. Stage 6 The membrane returns to its resting potential.

Propagation of the Action Potential An action potential in a cell membrane is triggered by an initial stimulus to the neuron. That action potential provides the stimulus for a neighbouring segment of cell membrane and so on until the neurons axon is reached. The action potential then propagates down the axon, or nerve bre, by successive stimulation of sections of the axon membrane. Because an action potential is an all-or-nothing reaction, once the gate threshold is reached, the amplitude of the action potential will be constant along the path of propagation. The speed, or conduction velocity, at which the action potential travels down the nerve bre depends on a number of factors, including the initial resting potential of the cell, the nerve bre diameter and also whether or not the nerve bre is myelinated. Myelinated nerve bres have a faster conduction velocity as the action potential jumps between the nodes of Ranvier. This method of conduction is known as saltatory conduction and is described in more detail in [29].

40

Synaptic Transmission The action potential propagates along the axon until it reaches the axonal ending. From there, the action potential is transmitted to another cell, which may be another nerve cell, a glandular cell or a muscle cell. The junction of the axonal ending with another cell is called a synapse. The action potential is usually transmitted to the next cell through a chemical process at the synapse. If the axon ends on a skeletal muscle cell then this is a specialised kind of synapse known as a neuromuscular end plate. In this case, the action potential will trigger the muscle to contract. The physical processes that must occur to enable muscle contraction will be examined in more detail later, but rst the structure of the muscle is described.

3.33.3.1

MusclesMuscle Physiology

There are three types of muscle present in the human body - smooth, skeletal and cardiac. Smooth muscle is the muscle found in all hollow organs of the body except the heart, and is generally not under voluntary control. Cardiac muscle, the only type of muscle which does not experience fatigue, is the muscle found in the walls of the heart which continuously pumps blood through the heart. Skeletal muscle is the muscle attached to the skeleton which is the type of muscle that will be described here. The main function of skeletal muscle is to generate forces which move the skeletal bones in the body. The basic structure of a skeletal muscle is shown in Figure 3.5. Muscle is a long bundle of esh which is attached to the bones at both ends by tendons. The muscle is protected by an outer layer of tough tissue called the epimysium. Inside the epimysium are fasicles or bundles of muscle bre cells. The fasicles are surrounded by another layer of connective tissue called 41

Epimysiumouter layer of the muscle Tendon111111 000000 each muscle bundle 111111 000000 111111 000000 111111 000000 11111 00000 111 000 111111 000000 11111 00000 111 000 111111 000000 11111 00000 111 000 111111 000000 11111 00000 111 000 111111 000000 11111 00000 111 000 11 00 11111 00000 11111 00000 111 000 11 00 11111 00000 11111 00000 111 000 11 00 11111 00000 11111 00000 111 000 11 00 11111 00000 11111 00000 111 000 11 00 11111 00000 11111 00000 111 000 11 00 11111 00000 11111 00000 111 000 11 00 11111 00000 11111 00000 111 000 11 00 11111 00000 11111 00000 111 000 11 00 111111 000000 11111 00000 11111 00000 111111 000000 11111 00000 11111 00000 1111 11 0000 00 111111 000000 11111 00000 11111 00000 1111 0000 111111 000000 11111 00000 1111 0000 111111 000000 1111 0000 111111 000000

Perimysium surrounds

Bone

Muscle fibre (cell)

Fasicle bundle of muscle cells

Endomysium surrounds each cell

Figure 3.5: Muscle Anatomy the perimysium. The individual muscle bre is surrounded by a layer of tissue called the endomysium. The structure of the individual muscle bre will now be described now in more detail.

The Muscle Fibre Each individual muscle bre is a cell which may be as long as the entire muscle and 10 to 100m in diameter. The nuclei are positioned around the edge of the bre. The inside of the muscle bres consists of closely packed protein structures called myobrils which are the seat of muscle contraction. The myobrils run along the length of the muscle bre. These myobrils exhibit a cross striation pattern which is shown in Figure 3.6. The myobrils may be seen in detail using a technique known as polarised light microscopy. Under a microscope, the myobrils exhibit a repeating

pattern of dark and light bands. The dark bands are termed A-bands or anisotropic bands and the light bands are termed I-bands or isotropic bands. Anisotropic and isotropic refer to how the bands transmit the polarized light which is shone on them as part of the microscopy process. The isotropic bands transmit incident polarised light at the same velocity regardless of the direction and so appear light coloured, while the anisotropic bands transmit the light at dierent velocities depending on the direction of the incident light and

42

Muscle Fibre Myofibril

111111 000000 111111 000000 111111 000000 111111 000000 111111 000000 111111 000000 1 0 111111 1 0ABand IBand 000000 111111 000000 1 0 111111 000000 1111111111 0000000000 1 0 111111 000000 1 1 0 0 111111 000000 1111 0000 1 0 1 1 0 0 1111111 1 111 0000000 0 000 1111 0000 1 111 0 000 1 0 1111111 1 111 0000000 0 000 1 111 0 000 1111 0000 1 0 1111 0000 1 0 1111 0000 1 0 1111111 1 111 0000000 0 000 1 111 1 111 0 000 0 000 1111 0000 1 0 1111 0000 1 111 1 111 0 000 0 000 1111 0000 1 0 1111111 1 111 0000000 0 000 1 0 1111 11 0000 00 1 0 1111 0000 1 0 1111 11 0000 00 1 0 1 0 111111111 000000000 1 0 11 00 111111111 000000000 1111111111 0000000000 1 0 11 111111111 000000000 1 0 Sarcomere 00 11 00 111111111 000000000 1 0 11 00 111111111 000000000 1 0 11 00 Z disc 1 0 11 00

11 00 11 00 11 00 11 00 11 00 11 00 11 00 11 00

Figure 3.6: The muscle bre and the myobril cross striation pattern. therefore appear dark coloured. In the middle of the I-band there is a thin dark strip known as the Z-disc. The basic contractile element of muscle is known as the sarcomere and is the region between two Z-discs. The sarcomere is about 2m in length. The myobril is made up of a repeating chain of sarcomeres. A sarcomere consists of one A-band and one I-band. The structure of the sarcomere is shown in Figure 3.7(a). The Z-discs link adjacent thin myolaments, the Ibands, which are about 5nm in diameter. These bands primarily consist of actin, but also contain tropomyosin and toponin [31]. The A-band in the centre of the sarcomere contains thicker myolaments made of myosin which interlink the thin myolaments [29]. These myosin laments are about 11nm in diameter [30]. When the muscle contracts the thin laments are pulled between the thick laments. The position of the actin and myosin laments are shown before contraction in Figure 3.7(a) and during contraction in Figure 3.7(b). The importance of these bands and their role in muscle contraction will be described in the next section.

43

(a)

IBand

ABand

Actin

ZDisc

Myosin

(b)

Figure 3.7: (a) The sarcomere before contraction occurs. The A-band, containingthick myosin laments, and the I-band, containing the Z-disc and the thin actin laments are shown. (b) On contraction of the muscle, the thin actin laments slide between the myosin laments.

3.3.2

Muscle Contraction

The Motor Unit Each eerent motor nerve bre, or motor neuron as they are also known, stimulates a number of muscle bres. The nerve bre, and the muscle bres it innervates, make up the smallest functional unit of muscle contraction known as the motor unit. Each individual muscle bre in a motor unit will be stimulated simultaneously by the nerve bre, so they will each always contract and relax in synchronisation. The force produced by a muscle can be increased by increasing either of two parameters:(i) The number of active motor units. The motor units are roughly arranged in parallel along the length of the muscle so by activating more motor units, more muscle force can be produced. The force