Getting a Grip on Assistive Technology: Cognitive Testing Disability Questions

  • Published on

  • View

  • Download

Embed Size (px)


Getting a Grip on Assistive Technology: Cognitive Testing Disability Questions. Barbara F. Wilson, Barbara Altman, and Karen Whitaker National Center for Health Statistics Vicki A. Freedman , Jennifer C. Cornman, Lisa Landsberg Polisher Research Institute - PowerPoint PPT Presentation


<ul><li><p>Getting a Grip on Assistive Technology: Cognitive Testing Disability QuestionsBarbara F. Wilson, Barbara Altman, and Karen WhitakerNational Center for Health Statistics Vicki A. Freedman, Jennifer C. Cornman, Lisa Landsberg Polisher Research InstituteEmily M. Agree Johns Hopkins Bloomberg School of Public Health</p></li><li><p>Project Goals To develop a set of instruments for national surveys on health and aging to collect information on assistive technologies (AT) and the environments in which they are used.</p></li><li><p>Methodology of the Cognitive Testing Recruited 28 participants with assorted conditions (ALS, Blindness, Diabetes)Participants used a great variety of assistive technology items (e.g. cane, grab bars, hearing aid, walker, wheelchair, etc.)</p></li><li><p>MethodologyCognitive interviews were conducted over the telephone from one room to another in the NCHS Questionnaire Design Research Laboratory (QDRL).Interviews were videotaped with permission.Interviewing techniques used were think aloud with probes and debriefing.</p></li><li><p>MethodologyThree rounds of testing.Revised instrument after each round.</p></li><li><p>Selected Findings: </p><p>Language and comprehensionResponse option scalesReference periods</p></li><li><p>Findings about comprehension</p><p>Some words were misunderstood, either because they could not be heard over the telephone or because they were unfamiliar (curb cut, corridor, health, pill reminder, stall shower, vision)</p></li><li><p>Findings about interpretationPeople couldnt decide what constituted convenient public transportation if they couldnt walk.or adequate lighting if they couldnt see well.Couldnt decide whether How often do you walk around your neighborhood? was intended to find out if participant socialized or exercised.One man who uses a wheelchair said, I roll around my neighborhood. </p></li><li><p>Findings about response optionsMany participants did not understand that they were being offered five possible response options and should choose one.</p></li><li><p>Transcript of one questionYou said there is a computer in your home. And how hard or easy is it for you to do the followingis it Very hard, Hard, Not hard or easy, Easy, or Very easy to use the keyboard? Shrugs shoulderIts OK.And so you say that its Not hard or easy?Its Not hard.Is it Easy?Yes.Is it Very easy?Its Easy. Thats what it is. Easy.</p></li><li><p>Findings about response optionsMany participants did not understand that Not hard or easy was a single option, the third of five. To solve this, the response options could be read aloud with numbers preceding.This was a problem over the telephone. In another mode (self administered by computer or paper and pencil), this may not have been a problem.</p></li><li><p>Response optionsParticipants did not naturally use the requested metric.</p></li><li><p>Transcript of second questionAnd how hard or easy is it to see the screen?I see it very well.Would you say then its Easy to see the screen?Yes.Is it very easy?Very easy.</p></li><li><p>Response optionsQuestionnaire designers intended that stem of question with five response options be carried forward to four follow-up questions. However, questions 2, 3, and 4 abbreviated the scale to just hard or easy . Interviewer had to repeat options to arrive at a valid answer.</p></li><li><p>Transcript of third questionAnd how hard or easy is it to control the Pointer?Easy.Very easy?Very easy.</p></li><li><p>Transcript of fourth questionAnd how hard or easy is it to follow the instructions?You mean Email?Yes.I learned it, and its no problem now.So its Very Easy?Yes.</p></li><li><p>Problem with bipolar scalesOne woman said that a numbered scale would be easier for her than a 5-point bipolar verbal scale from Very hard to Very easy. In asking for a numbered scale from one to five she is asking for a unipolar one-directional scale. </p></li><li><p>Time framesCognitive testing showed that it was important for questions to include a reference period (In the past 30 days) for use of assistive technology items.Without the time frame, people offered long lists of equipment that was no longer used, confounding subsequent questions.</p></li><li><p>Frequency of AT useIn the first round, frequency of use was asked generically for all AT items, beds, canes, hearing aids. The original question was: How often do you currently use it All the time, some of the time, or never? </p></li><li><p>Frequency of AT use, contd.</p><p>When asked about a cane, this question got responses such as 90 percent of the time, except when I sleep, eat or read the paper or most of the time. Good answers, but not in the requested metric, which was All the time, some of the time, or never.</p></li><li><p>Frequency of AT use, contd.When asked about a hospital bed, How often do you currently use itAll the time, some of the time, never?Response: Am I in bed 24 hours a day?</p></li><li><p>Frequency of AT use, contd.Cognitive testing showed that, in order to make sense, the frequency of using an AT item (e.g. hospital bed) had to be linked to a specific activity (e.g. sleeping). </p></li><li><p>Questions to develop Measures of EffectivenessEffectiveness Concepts:Participation SafetyControlIndependencePain, fatigue, extra time needed to do things</p></li><li><p>Original Effectiveness QuestionsResponse options offered Balanced, forced choice, 5-point bipolar semantic differential scale:Please tell me whether you strongly disagree, disagree, neither agree nor disagree, agree, or strongly agree with each of the following statements.</p></li><li><p>Example: One of 15 Effectiveness questionsBecause I use these items, I have less pain than I used to. Do you Strongly disagree, Disagree, Neither disagree nor agree, Agree, or Strongly agree ?</p></li><li><p>ConclusionsResearchers and QDRL staff jointly reviewed tapes from first two rounds.The project demonstrated that telephone surveys of elderly persons present special problems.The overly long survey was shortened by dropping unworkable questions.Synonyms were found for misunderstood terms.</p></li><li><p>Conclusions, contd.Purpose of confusing questions was clarified.Bipolar response option scales were replaced with unipolar scales that included a Does not apply option.Assistive technology questions were limited to 30-day reference period.Third round went smoothly.Revised instrument will be pilot tested.</p></li><li><p>Balanced, forced choice, 5-point bipolar scalesMeaning of the midpoint was unclear. The midpoint might mean that they neither disagreed nor agreed, that they both disagreed and agreed, or that sometimes the statement was true and sometimes false. Moreover, for some items Neither agree nor disagree just did not fit.</p></li><li><p>ExampleI have less pain than I used to.One participant said she never had pain. If she agreed that she had less, then it would seem as if she previously had pain. If she disagreed that she had less pain, it would sound as if the Assistive Tech did not help. </p></li><li><p>Problems with forced choice,Disagree/Agree, response scaleThere was a hidden assumption that there had been pain. She wanted the option to say Does not apply.</p></li><li><p>Revised question (Unipolar, 3 point, nonforced choice)Because you use these items, how much less painful is it for you to do your daily activities.. No less, a little less, or a lot less? Or does it not apply?</p></li><li><p>ConclusionResearchers and QDRL staff reviewed tapes from first two rounds.Researchers revised, deleted, or fine-tuned the questions.Third round went more smoothly.Revised instrument will be pilot tested.</p></li></ul>


View more >