Serious games for upper limb rehabilitation following stroke

Serious games for Serious games for upper limb upper limb

rehabilitation following rehabilitation following strokestroke

J. W. Burke, M. D. J. McNeill, D. K. Charles, P. J. Morrow

School of Computing and Information Engineering

J. H. Crosbie, S. M. McDonoughSchool of Health Sciences

University of UlsterUniversity of Ulster

ContentsContents1. Overview2. Background on Stroke3. Technology for Stroke Rehabilitation4. Previous Studies5. Game Design & Rehabilitation6. University of Ulster Work

Virtual Reality Games Webcam Games Nintendo Wiimote

7. Design Principles in Our Games8. Usability Study9. Final Remarks

OverviewOverviewTHE PROBLEM AREA:Stroke is a leading cause of severe physical

disability.Effective rehabilitation must be early,

intensive and repetitive.Difficult to maintain patient motivation.

THE PROPOSED SOLUTION:Games may be effective in optimising

engagement.We have developed a serious game system for

upper limb post-stroke rehabilitation.

StrokeStroke Impairments: attention and concentration

deficiencies, balance loss, pain, weakness and paralysis.

Can cause depression and fatigue.

Early and intensive practice of functional tasks in an enriched environment show more positive outcomes for upper limb rehabilitation.

Tasks often mundane and boring.Rehabilitation can require patients to travel to

specialised units.High healthcare costs.

Burkey

It has been shown that early and intensive practice of active functional tasks in an enriched environment show more positive outcomes for upper limb rehabilitation - it has been suggested that this may stimulate neural reorganisation of the cerebral cortex leading to recovery, or partial recovery of the upper limb.

Burkey

Whilst these programmes attempt to stimulate the person with a variety of rehabilitation exercises, people with stroke commonly report that traditional rehabilitation tasks can be mundane and boring due to their repetitive nature.

James Burke

Patients may be required to travel far to clinics. Also, the requirement for one-to-one treatment increases cost.

Traditional Therapy VideoTraditional Therapy Video

Technology for Stroke Technology for Stroke RehabilitationRehabilitationBENEFITS:Rehabilitation systems which are novel,

interesting, safe and customisable to a particular user’s abilities and interests.

Motivate, involve and immerse the user.Potential for home use.

◦ Performance data recorded at home, uploaded to remote site.

LIMITATIONS:Clinically valid? [Burdea, 2002; Crosbie, 2008; Rizzo, 2005]

Cost and size of equipment?Expertise required?

Previous StudiesPrevious StudiesBESPOKE: Sensor-based systems [Crosbie, 2004;

Yeh, 2005] Xbox console with P5 glove [Morrow,

2006] Haptics & Robotics [Boian, 2002; Guo,

2007; Podobnik, 2008]

ADAPTED: Gesture Xtreme / IREX [Rand, 2004]

Sony EyeToy [Rand, 2004; Yavuzer, 2008]

Open source games [Herbelin, 2008]

Nintendo Wii [ongoing studies]

REVIEWS:[Burdea, 2003; Crosbie, 2004; Weiss, 2004; Rizzo, 2005; Rizzo, 2008; Flores, 2008]

Designers of rehabilitation tasks can benefit from examining the formulas that commercial game developers use… [Rizzo & Kim, 2005]

MEANINGFUL PLAY:

Conservative response to failure – reward all engagement.

Goals spanning multiple sessions to reward atypical player behaviour.

◦ E.g. Rewards for time spent playing game.

”“Game Design & Game Design & RehabilitationRehabilitation

Inputs choices

Discernable and integrated outcome

Player GameMeaningful playMeaningful play

Feedback

James Burke

Feedback enables a player to measure their progress in achieving their goals, or progression in their skills over time. Feedback is how the game responds to the changes or choices made by the player and is central to creating and maintaining meaningful play. It can be aural, visual and haptic and can be used to signify correct or incorrect actions or responses. Numerical scores, progress bars, character dialogue, controller vibration/force and sound are all obvious forms of feedback in a game. Alternatively, feedback can occur through the player acquiring new skills or game elements – improving player health or weapon armoury, for example, or through the acquisition of more intelligence or special powers such as X-ray vision or invisibility. Such gratifying incentives can lead to increased motivation and enjoyment, creating a greater desire to complete particular tasks and reach certain goals. Player attributes (health, special powers etc) can also be decreased or removed on poor gameplay, actions which encourage players to learn from their mistakes. Without quantifiable advantages for completing tasks successfully and conversely exposing a recognisable disadvantage for poor gameplay, the player is less likely to engage effectively with the game

James Burke

In games designed for rehabilitation, we believe that failure should be handled conservatively. The goal of a game designed for rehabilitation should initially be to encourage engagement and subsequently to reward all engagement with success. This is not to say that the player character should never die, rather that the game should initially be paced so that the player with stroke is encouraged to play without catastrophic results. Stroke survivors may not have been players of video games prior to their trauma. They may be unfamiliar with the equipment, are going to have limited mobility in their affected limbs and consequently the risk of users not engaging with the game is high. By handling failure in a positive way rehabilitation players are more likely to remain engaged and not feel that failure in the game stems from their poor physical abilities as a result of stroke.

Game Design & Game Design & RehabilitationRehabilitationCHALLENGE:Challenge needs to match

patient’s abilities.Maintains engagement.How to maintain challenge?

◦ Manual difficulty adjustment?◦ Adaptivity?◦ Levels of increasing difficulty?

Speed, position and size of game elements can set level of challenge.

In-game calibration can determine suitable level of challenge for player’s abilities.

[Rabin, 2005]

James Burke

At the start of a new game, a player generally desires a low level of challenge to meet their correspondingly low level of ability/familiarity with the game. The more the player plays the game, the more his or her skills and familiarity increase and so the player requires a higher level of challenge if he/she is to continue enjoying playing. If the game is too difficult for the player, either through their lack of ability with the game interface/controller or poor feedback they can become frustrated and quit. Similarly, if the game is not interesting or challenging enough, the player can become bored. Clearly both of these scenarios are unwelcome in any context, including rehabilitation.

James Burke

Once the user becomes more familiar with the system, games which are adaptive may offer the best solution to maintaining an appropriate level of challenge. If players are scoring highly and successfully completing tasks within the available time it is likely that the level of challenge is too easy – the player’s cognitive and motor skills surpass the challenge offered by the game. Conversely, if players are not scoring well and regularly failing to complete tasks then the game is clearly too difficult. In these cases, game adaptivity, which allows dynamic adjustment of game elements to either make the game easier or more difficult by changing the game elements as described above (altering the timing, positioning and size of game elements) may be beneficial.

James Burke

Many games use levels to structure difficulty. Completing a level is possible only when the player understands at least part of the game mechanics and has acquired the necessary knowledge and skills (including controller skills) to progress through the level to the end. The next level then builds on these skills, requiring either acquisition of new skills or honing of existing skills as the difficulty of the game increases and in this way the challenge presented to the player changes in a way that ideally keeps the player interested and feeds his or her enjoyment. Other games may not have recognisable levels as such, but the challenge might increase as particular points in the game are reached, again, indicating that an appropriate level of understanding and acquisition of skills has been achieved.

James Burke

Typically games are harder when the pace of the game is faster (the player has less time to react) and conversely easier when the pace is slower. The speed at which game elements (e.g., characters) move in the game can affect the pace of the game. Games designed for upper limb motor rehabilitation will, clearly, require the movement of various limbs (hands, arms) to point to, reach for, touch, grasp and move game elements. The position of these game elements can be altered to make them easier or more difficult to reach in order to suit the capabilities of the player. Targets can be moved closer to the median line of the player or conversely farther away, making them reachable only by greater extension of shoulder and elbow joints. Game elements can also be made larger (easier to touch) or smaller (harder to touch, requiring more accurate movement).

James Burke

The data acquired from the initial user assessment tests can help to ensure that the initial game challenge presented to the user is neither too trivial nor impossibly difficult.

Uni. of Ulster WorkUni. of Ulster WorkVIRTUAL REALITY GAMES:Bilateral catch task:

◦ Uses an electromagnetic sensorattached to a real physical basket.

◦ User moves basket with both handsto catch falling oranges.

◦ Various adjustable parameters.Adaptive ‘Whack-a-Mouse’ game:

◦ Sensor attached to player’s hand.◦ Encourages arm movement and

visual discrimination.◦ Initial configuration based on

assessment prior to play.Further information - [Ma, 2007]

Slide 10 of

Uni. of Ulster WorkUni. of Ulster WorkWEBCAM GAMES:Simple 2D games which use a webcam image

as input.Game library where all games share patient

profile.◦ Profile stores reaching ability, play-time, scores,

long-term progress, regions of difficulty and skill level.

Played with coloured gloves or any single coloured object.

No attaching of wires required, untethered movement.

Games can be played standing up or seated.Potential for home rehabilitation.No expertise required.

Uni. of Ulster WorkUni. of Ulster WorkSingle Arm Exercise:“Rabbit Chase”

◦ Player must catch a rabbit as it peers out of holes on the screen.

“Bubble Trouble”◦ Player must burst bubbles as

they float around the screen.

Bimanual Arm Exercise:“Arrow Attack”

◦ Player must touch corresponding arrows with both hands simultaneously as they enter boxes. 12

Uni. of Ulster WorkUni. of Ulster WorkInitial reach calibration:Player must roll each ball as

far as they possibly can.Allows system to determine

range of movement and position game elements accordingly.

Camera calibration tool:Allows refining of colour

detection settings.Hope to improve

automation of colourdetection in future.

Webcam Games VideoWebcam Games Video

Uni. of Ulster WorkUni. of Ulster WorkNINTENDO WIIMOTE VIBRAPHONE: Instrument simulator using multiple Nintendo

Wii remotes.Works with PC, interfacing through a

bluetooth dongle.User can create a tune, or play back a tune

in a timedgame mode.

Potential for wrist and armrehabilitation.

Design Principles in Our Design Principles in Our GamesGamesAuditory feedback: Sound

indicators (hit, miss, play note).Visual feedback: Hit or miss

messages, target changing colour, particle effects, score indicator, time indicator, progress graph.

Level of challenge can be manually adjusted.

Optional adaptivity to adjust speed and position of game elements according to player’s successive hits or misses.

James Burke

Speed of game elements determines the difficulty of the game. As well as this, in the bubble game as the difficulty increases adaptively, the bubbles start appearing further from the player's central position. Future versions of the other games may also reposition game elements to become further from the player in order to increase difficulty.

Usability StudyUsability StudyUsability study of 10 able bodied users playing the

two webcam games and vibraphone application.Majority of participants enjoyed all games.Players felt games were enjoyable, replayable

and easy to play due to intuitive controls.Apativity approved by those who noticed it

(>80%), indicating the game would be less enjoyable without it.◦ One user expressed that the adaptivity was too

aggressive.> 80% agreed that the feedback was effective.

◦ Some players reported it was difficult to see their score during gameplay.

With the Vibraphone game, some users found the controls to be too sensitive.

Final RemarksFinal RemarksFURTHER WORK: Single patient case studies for playability

evaluation are underway with people with stroke – initial feedback is very positive.

Larger study planned to assess impact of games on motor function.

How to provide a consistent level of challenge for a patient during rehabilitation?

CONCLUSION: Games have potential to engage and motivate

during rehabilitation. No need for computer or video game experience. Low-cost off-the-shelf hardware may offer an

opportunity for home rehabilitation in addition to traditional therapy.

Questions?Questions?This work was supported by the

Department for Employment and Learning (DEL), Northern Ireland.

E-mail: [email protected]: http://php.infc.ulst.ac.uk/vr-therapy

ReferencesReferences BOIAN, R., SHARMA, A., HAN, C., MERIANS, A., BURDEA, G., ADAMOVICH, S., RECCE, M., TREMAINE, M. and POIZNER,

H., 2002. Virtual Reality-Based Post-Stroke Hand Rehabilitation, 2002 Proceedings of Medicine Meets Virtual Reality Conference, 2002, IOS Press pp. 64-70.

BURDEA, G., 2002. Key Note Address: Virtual Rehabilitation - Benefits and Challenges, 1st International Workshop on Virtual Reality Rehabilitation (Mental Health, Neurological, Physical, Vocational), VRMHR’2002, November 7-8 2002, pp. 1-11.

BURDEA, G., 2003. Virtual Rehabilitation - Benefits and Challenges. International Medical Informatics Association Yearbook of Medical Informatics, , pp. 170-176.

CROSBIE, J.H., LENNON, S., MCGOLDRICK, M.C., MCNEILL, M.D.J., BURKE, J.W. and MCDONOUGH, S.M., 2008. Virtual reality in the rehabilitation of the upper limb after hemiplegic stroke: a randomised pilot study, Proc. 7th ICDVRAT with ArtAbilitation, 2008, pp. 229-235.

CROSBIE, J.H., MCDONOUGH, S.M., LENNON, S., POKLUDA, L. and MCNEILL, M.D.J., 2004. Virtual Reality in the Rehabilitation of the Upper Limb After Stroke: The User’s Perspective, 5th International Conference for Disability, Virtual Reality and Associated Technology, 2004, ICDVRAT / University of Reading pp. 215-224.

FLORES, E., TOBON, G., CAVALLARO, E., CAVALLARO, F.I., PERRY, J.C. and KELLER, T., 2008. Improving Patient Motivation in Game Development for Motor Deficit Rehabilitation, Advances in Computer Entertainment Technology, 2008, pp. 381-384.

GUO, S., SONG, G. and SONG, Z., 2007. Development of a Self-assisted Rehabilitation System for Upper Limbs Based on Virtual Reality, International Conference on Mechatronics and Automation 2007. 2007, IEEE pp. 1452-1457.

HERBELIN, B., CIGER, J. and BROOKS, A.L., 2008. Customization of gaming technology and prototyping of rehabilitation applications, Proc. 7th ICDVRAT with ArtAbilitation, 2008, pp. 211-218.

MA, M., MCNEILL, M., CHARLES, D., MCDONOUGH, S., CROSBIE, J., OLIVER, L. and MCGOLDRICK, C., 2007. Adaptive Virtual Reality Games for Rehabilitation of Motor Disorders. Lecture Notes in Computer Science. Springer Berlin / Heidelberg, pp. 681-690.

MORROW, K., DOCAN, C., BURDEA, G. and MERIANS, A., 2006. Low-Cost Virtual Rehabilitation of the Hand for Patients Post-Stroke, 2006 International Workshop on Virtual Rehabilitation, 2006, IEEE pp. 6-10.

PODOBNIK, J., MUNIH, M. and CINKELJ, J., 2008. HARMiS: hand and arm rehabilitation system, Proc. 7th ICDVRAT with ArtAbilitation, 2008, pp. 237-244.

RAND, D., KIZONY, R. and WEISS, P.L., 2004. Virtual reality rehabilitation for all: Vivid GX versus Sony PlayStation II EyeToy, Proceedings of the 5th International Conference on Disability, Virtual Reality & Associated Technology, 2004, ICDVRAT pp. 87-94.

RIZZO, A.A., 2008. Virtual reality in psychology and rehabilitation: the last ten years and the next! Proc. 7th ICDVRAT with ArtAbilitation, 2008, pp. 3-6.

RIZZO, A. and KIM, G.J., 2005. A SWOT Analysis of the Field of Virtual Reality Rehabilitation and Therapy. Presence: Teleoperators and Virtual Environments, 14(2), pp. 119-146.

WEISS, P.L., RAND, D., KATZ, N. and KIZONY, R., 2004. Video Capture Virtual Reality as a Flexible and Effective Rehabilitation Tool. Journal of NeuroEngineering and Rehabilitation, 1(12).

YAVUZER, G., SENEL, A., ATAY, M.B. and STAM, H.J., 2008. Playstation eyetoy games improve upper extremity-related motor functioning in subacute stroke: a randomized controlled clinical trial. European Journal of Physical and Rehabilitation Medicine, 44.

YEH, S., RIZZO, A., ZHU, W., STEWART, J., MCLAUGHLIN, M., COHEN, I., JUNG, Y. and PENG, W., 2005. An Integrated System: Virtual Reality, Haptics and Modern Sensing Technique (VHS), Proceedings of the ACM symposium on Virtual reality software and technology, 2005, ACM pp59-62.