J O U R N A L O F G E R I A T R I C O N C O L O G Y 2 ( 2 0 1 1 ) 2 5 3 – 2 5 8

Ava i l ab l e on l i ne a t www.sc ienced i r ec t . com

OnQol: Electronic device to capture QoL data in oncology:Difference between patients 65 years or older and patientsyounger than 65 years of age

Alexandra Oliveiraa,⁎, Pedro L. Ferreiraa, b, Bárbara Antunesa, c, Francisco L. Pimentela, d, e

aCentre for Health Studies and Research of the University of Coimbra, (CEISUC), PortugalbFaculty of Economics of the University of Coimbra, PortugalcKing's College London, UKdHealth Sciences Department, University of Aveiro, PortugaleLenitudes, Santa Maria da Feira, Portugal

A R T I C L E I N F O

⁎ Corresponding author at: Rua Bernardo CostE-mail address: alexandra.t.oliveira@gma

1879-4068/$ – see front matter © 2011 Elsevidoi:10.1016/j.jgo.2011.08.001

A B S T R A C T

Article history:Received 28 March 2011Received in revised form 19 July 2011Accepted 9 August 2011Available online 9 September 2011

Objective: To assess the feasibility of electronic data capture for older patients, as well as tocompare preferences between answering a paper version versus an electronic version inpatients with cancer who are older and younger than 65 years of age.Materials and methods: The computer and the paper versions of the QLQ-C30 questionnairewere administered to a sample of 200 cancer patients.Results: 38.7% of older patients preferred the electronic version, 8.1% preferred the paperquestionnaire and 53.2% had no preference between these two methods. No statisticallysignificant differences between the answers on the paper and computer versions werefound. The best correlation in responses was seen in the “diarrhea” and “physicalfunctioning” subscales. Patients preferred to use the computer version in comparison tothe paper version.Conclusions: The computer version of the QLQ-C30 demonstrated similar results to the paperversion and proved to be preferred by the cancer patients included in the study. Thismethodis useful for monitoring patients' QoL in daily oncology practice and in real time, being apotential useful method to support clinical decision making. It is feasible and useful to usethese tools with an older population of patients.

© 2011 Elsevier Ltd. All rights reserved.

Keywords:QoLOncologyData captureQLQ-C30Older patientsGeriatrics

1. Introduction

For cancer patients, survival and well-being are two of themost important outcomes regarding their treatment. This isthe main cause for the increasing importance given toevaluation of Quality of Life (QoL) in oncology. Severalinstruments have been developed for that purpose, such asthe QLQ-C30 of the European Organization for Research and

a, no. 5 1° Dto, 2775-809 Cil.com (A. Oliveira).

er Ltd. All rights reserved

Treatment of Cancer (EORTC).1 However, the role that thesemeasurement instruments play in daily clinical practiceremains vague and many of them are only used for academicresearch (e.g., MSc or PhD theses) or in clinical trials.2,3 Few areconsidered in everyday clinical decision making.4 This prob-ably occurs because the method and time needed for datacollection is not compatible with getting the results in realtime in clinical daily practice. Answers written in paper

arcavelos, Portugal. Tel.: +351 967393279; fax: +351 239 790 514.

.

254 J O U R N A L O F G E R I A T R I C O N C O L O G Y 2 ( 2 0 1 1 ) 2 5 3 – 2 5 8

questionnaires usually need to be manually keyed in acomputer database. When oncologists plan a treatment or atreatment change based on a clinical decision makingprocedure, it is necessary to take into account not only thetraditional clinical parameters of functioning and symptoms,but also patients' QoL evaluations. However, the time avail-able for an appointment is usually very limited, not allowingfor the identification of important problems and for a morethorough discussion. Therefore, a standard measure designedto evaluate patients' QoL could help overcome these obstaclesand potentially give better or more accurate care to thepatient,5 by taking into account all dimensions of that care.

There are several potential benefits to using QoL question-naires. First, is that it allows the patient to better describehis/her complaints related to the disease and/or treatment.Using questionnaires, clinicians have access to the functionaland psycho-social aspects of their patients.6 Secondly, theintroduction of QoL evaluation in oncology departmentsreinforces to the patient that his/her QoL is being taken intoconsideration by the medical team, and that it is appropriateto include it as a part of the communication between them.Finally, the QoL report produced by a computerized systemwith minimal effort from the clinical team can guide them byproviding a concise and valid evaluation of QoL,7 allowing foreach item to have special relevance that may be used torecognize possible “problems” which can be displayed in thereport to the medical team.

This study is comprised of Portuguese patients aged 65 orolder. This particular population has a set of special charac-teristics, namely a low educational level, rare or no previouscontact with computers and questionnaires, a low visualacuity and a need for wearing glasses, which could lead tomore time to complete questionnaires.8 Despite the concernregarding older patients, in terms of resisting new technolo-gies, some studies have suggested however that electronicdata capture are usually preferred by this population, incomparison to the traditional paper method.9,10

2. Objective

The purpose of this study was to assess the feasibility ofelectronic data capture in older patients and to comparepreferences between answering paper versus electronic ques-tionnaires. We also compared the preferences of cancerpatients aged 65 years or older with those provided by patientsbelow that age.

3. Materials and Methods

3.1. Population

A convenience sample of 200 cancer patients was collected. Allwere receiving or had received oncological treatment or hadattended follow-up consultations in the Medical OncologyDepartmentof theSantaMariadaFeira's SãoSebastiãoHospital,inPortugal. Thisdepartment ismainly for outpatients and forallcancer diagnosis in all phases of the illness. Patients wererecruited after their registration for the appointment but before

seeing the doctor, so that the completion of the questionnairebefore themedical appointment was possible. Inclusion criteriaincluded the need to read,write and understand the Portugueselanguageandhaveadiagnosis of cancer. Patientswere excludedif they had neurologic or cognitive impairment which wouldaffect their ability to complete the questionnaires (in paper andcomputer). This information was taken from the clinical fileand, in case of doubt, an oncologist was asked for his/heropinion on the patient's cognitive status. For this analysis,patients were divided into two groups: group I with patients65 years or older (n=62 individuals) and group II with patientsyounger than 65 years (n=138 individuals).

3.2. Instrument

The QoL instrument used for this study was the Portugueseversion 3.0 of the EORTC's QLQ-C30 form. This measure has 30items that reflect the multidimensionality of the quality-of-life construct organized into 15 multi-item and single-itemsubscales: five functional (physical, performance, cognitive,emotional and social), three for general symptoms (fatigue,pain and nausea/vomiting), one for capturing the self percep-tion of health status and QoL as well as six items assessingdyspnea, appetite loss, insomnia, constipation, diarrhea andfinancial difficulties caused by the disease. All questionsutilized the previous week as the time reference.11

This questionnaire is frequently used in oncology, iscurrently well accepted by patients, and is valid, reliable andsensitive to change both among patients and within the sameindividual. It is also a good tool to establish comparisons inclinical research studies among different cultures, since it isadapted in different languages, including Portuguese.1,12

Several studies which assess an individual's QOL have utilizedthis questionnaire.4,7,13–17

3.3. Proceedings

Together with a computer scientist team, we designed theelectronic version of the QLQ-C30, and included it in aspecially developed software for QoL data collection calledOnQol®.18,19 This system is based on a server-client architec-ture and has special requirements related to performance, dueto the psychological effect that the time of response mighthave in the disposition and availability of the patient. All thatis needed is an internet wireless connection. For this research,the data was collected by two laptop computers with touchscreen (pen-touch) and wireless technology. Each item of thequestionnaire appears on its own along with the four possibleanswers. Each patient is only supposed to touch, with thecomputers' pen, on the response that he/she feels is the onethat best describes the current situation. The same instruc-tions and layout of the traditional paper questionnaire wereused in the electronic version aswell as the same format of thepossible answers. The navigation throughout the electronicquestionnaire is possible, so that the patient may go to anyitem if s/he wants to answer or to change a previous response.In the paper version the original format of the QLQ-C30 waskept.

Both researchers that conducted the data collection hadprior training in the use of the OnQol software.20 These

255J O U R N A L O F G E R I A T R I C O N C O L O G Y 2 ( 2 0 1 1 ) 2 5 3 – 2 5 8

researchers approached the patient explaining the objective ofthe study, what they were expected to do and that they werefree to decide whether to participate or not. If they accepted toparticipate then a more thorough explanation and theinformed consent were given.

Data referring to the patient's medical history weregathered from the medical file of each participant. Theinstructions given to participants were similar for paper andelectronic completion. Patients were asked to complete thequestionnaire by choosing the answer that best describedtheir current situation. For the electronic version, there werefour training questions, none of them from the questionnaire,to make sure that respondents understood how the pen-touchtechnology worked. When the questionnaire was finished, areport was produced using the Rasch model through theOnQoL software.18 This report gives a graphic sequencedescribing the scores of measures previously taken, and alsoinformation about each item and the weight in themeasure. Ifan item does not fit the model, this information appears in“red” in the report and the caregiver is alerted to a potentialproblem. The report will be available to the doctor before theconsultation, in order to be considered and evaluated.

3.4. Statistical Proceedings

We used the Statistical Package for Social Sciences (SPSS, version16.0) for data analysis. To compare the answers given by thetwo age groups, in the electronic version, correlation co-efficients were also determined.

To test the feasibility of both versions of the QLQ-C30,paper and electronic, internal consistency was evaluated by

Table 1 – Social-demographic characterization of oncological pa

Variables Values

Age MeanStandard deviation

Gender FemaleMale

Marital status SingleMarried/living togetherWidowDivorced

Educational level Only reads and writes4 years6 years9 years12 yearsCollege degree

Diagnosis BreastColorectalProstateGastricLungLymphomaOvaryKidneyPancreasOthers

computing the Cronbach's α. This indicator is one of the mostused to verify the internal consistency of a group of variables(items).21 It varies between 0 and 1. Values above 0.7 wereconsidered as good internal consistency indicators.22 Thereference value for this study was 0.7.

4. Results

Patients in group I were mainly male (71.0%), with age rangedbetween 65 and 96 years old (mean=71.8, SD=5.9), most ofthem were married or living together and with four years ofeducation (79.0%). Patients in group II were mainly female(71.7%), age ranged between 25 and 64 years old (mean=49.7,SD=10.1), most of them were married or living together andalso with four years of education (49.3%). Data for maritalstatus and educational level are presented in Table 1.

There were no missing data in the total sample. Threepatients were excluded from the total sample because theyhad Alzheimer's disease and four other patients wereexcluded because they were not able to read, understandand respond to the questionnaire.

Regarding each subscale, the correlations between paperand electronic versions, for both groups and for the totalsample were good or very good and were highly significant.The highest correlation for patients 65 years or older occurredon the subscales “financial difficulties” (0.92) and “global scaleof QoL” (0.91). In group II the highest correlation was forsubscales “diarrhea” (0.94) and “dyspnea” (0.95) (Table 2).

In Table 3 we present the internal consistency analysis forthe electronic version of the QLQ-C30 subscales in both

tients for both groups.

Group I (n=62)Patients≥65 years old

Group II (n=138)Patients<65 years

n % n %

71.8 49.75.9 10.1

18 29.0 99 71.744 71.0 39 28.31 1.6 14 10.1

51 82.3 108 78.39 14.5 10 7.21 1.6 6 4.31 1.6 0 0.0

49 79.0 68 49.33 4.8 25 18.14 6.5 16 11.62 3.2 11 8.03 4.8 18 13.17 11.3 48 34.8

19 30.6 24 17.415 24.2 5 3.62 3.2 13 9.47 11.3 9 6.55 8.1 5 3.61 1.6 10 7.21 1.6 5 3.61 1.6 4 2.94 6.5 15 10.9

Table 2 – Correlation between paper–computer.

Dimension Subscales Total sample(n=200)

≥65 years(n=62)

<65 years(n=138)

Functioning scales Physical 0.90 ⁎ 0.86 ⁎ 0.92 ⁎

Performance 0.76 ⁎ 0.80 ⁎ 0.74 ⁎

Emotional 0.89 ⁎ 0.85 ⁎ 0.91 ⁎

Cognitive 0.76 ⁎ 0.88 ⁎ 0.73 ⁎

Social 0.79 ⁎ 0.74 ⁎ 0.80 ⁎

Global QoL 0.88 ⁎ 0.91 ⁎ 0.87 ⁎

Symptoms Fatigue 0.83 ⁎ 0.74 ⁎ 0.87 ⁎

Nausea/vomiting 0.89 ⁎ 0.71 ⁎ 0.93 ⁎

Pain 0.88 ⁎ 0.87 ⁎ 0.89 ⁎

Dyspnea 0.91 ⁎ 0.83 ⁎ 0.95 ⁎

Insomnia 0.89 ⁎ 0.84 ⁎ 0.91 ⁎

Loss of appetite 0.87 ⁎ 0.76 ⁎ 0.92 ⁎

Constipation 0.89 ⁎ 0.89 ⁎ 0.90 ⁎

Diarrhea 0.92 ⁎ 0.86 ⁎ 0.94 ⁎

Financial impact 0.88 ⁎ 0.92 ⁎ 0.87 ⁎

⁎ p<0.001.

256 J O U R N A L O F G E R I A T R I C O N C O L O G Y 2 ( 2 0 1 1 ) 2 5 3 – 2 5 8

groups. Symptoms measured by only one item are notpresented since it makes no sense to determine Cronbach's α.

By observing Table 3 it is possible to verify that, for thesubscales of the electronic QLQ-C30 version, in group I, theCronbach's α varied between 0.31 for “nausea/vomiting” sub-scale and 0.92 for subscale “pain”. For patients younger than65 years old, the same coefficient varied between 0.66 for“nausea/vomiting” subscale and 0.96 for “Global QoL” subscale.

Regarding patients' preference between both paper andelectronic versions, when they were asked which of thequestionnaire versions they felt more comfortable with, 8.1%ofolder patients preferred thepaper versionand38.7%preferredthe computer version. 2.9% of younger patients preferred thepaper version and 44.9%, the electronic version (Table 4).

When asked if, in the future, how theywould like to answerabout their QoL, most patients of both groups preferred thecomputer version (Table 5).

5. Discussion

In this study, the majority of the older patients preferred theelectronic method for quality of life data collection. One of themost commonly expressed reasons for preferring the electronicdata collectionwas that the questions aremuch easier to see on

Table 3 – Internal coherence (α Cronbach).

Dimension Indicators

Functioning scales PhysicalPerformanceEmotionalCognitiveSocial

Global QoLSymptoms Fatigue

Nausea/vomitingPain

the computer screen than on paper, even for those who neededglasses, making it more user friendly for this population.

Other authors have obtained similar results in the oncologysetting5,14 as well as in other areas of medicine. Sixty-fourpercent of rheumatoid arthritis patients preferred answeringquestions about their QoL in touch screen technology, and 33%had no preference between either method.23 Newell et al.24

concluded that the touch screen technology was acceptable byolder patients, and that 89% wanted to continue answeringquestions about their QoL in an electronic way. For this group,73% were more than 50 years old and 59% had never used acomputer before. The electronic versionwas highly acceptableand, in many cases, preferred to traditional paper method,considering age and previous experience with computers. Inanother study, performed in a rheumatology clinic with anelectronic version of the SF-36, there was also high correlationbetween paper and computer versions.25 In addition, dataobtained by computer version provide more information thanthe medical notes on symptoms and functional aspectsrecorded in the clinical files.5 In other areas such as diabetes,psychiatry and gastrointestinal disease, interactive softwareare now well accepted by patients and allow valid datacollection.26,27 Finally, 82% of patients preferred the computerversion and 89% said that theywould be comfortable using thesoftware, in the future, with no help.28

(α) Electronic version≥65 years

(α) Electronic version<65 years

0.81 0.760.89 0.830.80 0.870.69 0.670.33 0.780.69 0.960.88 0.890.31 0.660.92 0.82

Table 4 – Patients' preferences: With which of thequestionnaire versions were you more comfortable?

Electronic versionn (%)

Paper versionn (%)

No preferencen (%)

≥65 years 24 (38.7) 5 (8.1) 33 (53.2)<65 years 62 (44.9) 4 (2.9) 72 (52.2)

Table 5 – Patients' preferences: In the future how wouldyou like to answer questions about your QoL?

Electronic versionn (%)

Paper versionn (%)

No preferencen (%)

≥65 years 37 (59.7) 6 (9.7) 19 (30.6)<65 years 79 (57.2) 5 (3.6) 54 (39.1)

257J O U R N A L O F G E R I A T R I C O N C O L O G Y 2 ( 2 0 1 1 ) 2 5 3 – 2 5 8

In this study, patients in both groups felt more comfortableanswering questions about their QoL through the electronicquestionnaire or had no preference and, in the future mostpatients of both groups stated they would prefer to answer anelectronic questionnaire. The internal consistency of the QLQ-C30 was generally good for both groups of patients, except forthe social functioning scale in group I. Thismaybe related to thefact that this group is sociallymore fragile than the other due totheir age. QoL measurement should be part of the monitoringand systematic evaluation of the treatment given to cancerpatients. ElectronicQoLdatacapturepresentsa clear advantage,since it is easy to use, fast and valid. Furthermore, using theelectronic questionnaires, some of the problems associated tothe database entry process may be solved.29 This process isimmediate, as well as its analysis, potentially increasing thecorresponding clinical utility, by allowing immediate access toresults in a user-friendly layout.23 Also, it is possible to createsoftware that provides an alert for missing responses, so thatthepatienthasanopportunity toanswerany forgotten item,butalso the choice of not answering it.

On theother hand, costs are apotential obstacle in electronicQoL evaluation. It is necessary to buy a specific software andhardware to collect, process and to analyze data in realtime.14,30–32 However, this cost is not as high as the one neededto have an expert to carry out all these procedures every day.Thedevelopment of electronic systems forQoLmeasurement inoncology is, therefore, a promising and feasible method inclinical daily practice5 to systematically evaluate the functionaland emotional needs of their patients. This technology helpsmeet the needs of the traditional biomedical health model andof the increasing request for Patient Reported Outcome in-dicators in the clinical daily practice.34,35 In fact, electronic datacapture allows not only the access to medical history, tosymptoms and to some anatomic observations,5 but also tovalues that reflect the way patients evaluate their global QoL.

6. Conclusion

Despite the existing concerns for evaluating QoL usingelectronic tools, for an older population of cancer patients,we observed that it is possible to use these tools with botholder and younger patients. This technology is a feasible and a

reliable way of data collection for QoL, saving time inadministering the QoL instruments and in the data analysis.The increasing use of self-reported health status in clinicaldaily practice and in research, as well as monitoring changesin health throughout time, suggest the need to collect datamore frequently. Having these results in real time is nowpossible.

In spite of previous concerns on how older people withcancer could view the electronic data capture, our work andprevious work by Newell et al.24 revealed that touch screentechnology was acceptable to patients. The large majority ofolder patients, even those without computer literacy, werewilling to complete the electronic questionnaire. Patients,health care providers, researchers and policy decision makerswill be best served and best informed whenever the relation-ship between QoL and related health variables is known andrecognized. That information can be used not only in researchbut also in clinical daily practice, as well as in policy decisionmaking.33 Thus, measuring QoL, by touch screen technologyin real time, is feasible and practical and could change theculture of data collection in oncology practice.

Author's Contributions

Concept anddesign: AlexandraOliveira and Francisco Pimentel.Data collection: Bárbara Antunes and Alexandra Oliveira.Analysis and interpretation of data: Pedro Lopes Ferreira

and Alexandra Oliveira.Manuscript writing and approval: Alexandra Oliveira,

Francisco Pimentel, Bárbara Antunes, and Pedro Lopes Ferreira.

Disclosures

There were no actual or potential conflicts of interest for anyauthor of the paper, including any financial, personal or otherrelationships with other people or organizations within3 years of beginning the submitted work that could inappro-priately influence, or be perceived to influence our work.

Acknowledgments

Alexandra Oliveira is beneficiary of a fellowship (SFRH/BD/37397/2007) of the Foundation for Science and Technology,Portugal.

R E F E R E N C E S

1. Aaronson NK, Ahmedzai S, Bergman B, Bullinger M, Cull A,Duez NJ, et al. The European Organization for Research andTreatment of Cancer QLQ-C30: a quality of life instrument foruse in international clinical trials in oncology. J Natl Cancer Inst1993;85:365–376.

2. Coates A, Porzsolt F, Osoba D. Quality of life in oncologypractice: prognostic value of EORTC QLQ-C30 scores inpatients with advanced malignancy. Eur J Cancer 1997;33:1025–1030.

258 J O U R N A L O F G E R I A T R I C O N C O L O G Y 2 ( 2 0 1 1 ) 2 5 3 – 2 5 8

3. Dancey J, Zee B, Osoba D, Whitehead M, Lu F, Kaizer L, et al.Quality of life scores: an independent prognostic variable in ageneral population of cancer patients receivingchemotherapy. Qual Life Res 1997;6:151–158.

4. Velikova G, Stark D, Selby P. Quality of life instruments inoncology. Eur J Cancer 1999;35(11):1571–1580.

5. Velikova G, Wright EP, Smith AB. Automated collection ofquality-of-life data: a comparison of paper and computertouch-screen questionnaires. J Clin Oncol 1999;17(3):998–1007.

6. Velikova G, Wright P, Smith Stark D, Perren T, Brown J, PeterSelby. Self-report quality of life of individual cancer patients:concordance of results with disease course and medicalrecords. J Clin Oncol 2001;19(7):2064–2073.

7. Taenzer PA, Speca M, Atkinson MJ, Bultz BD, Page S, HarasymP, et al. Computerized quality-of-life screening in an oncologyclinic. Cancer Pract 1997;5:168–175.

8. Crawley JA, Kleinman L, Dominitz J. User preferences forcomputer administration of quality of life instruments. DrugInf J 2000;34:137–144 [ck.].

9. Nunnally, Bernstein, 1994 cit in Sneeuw Kcaa, Neil K,Sprangers MAG, Detmar SB, Wever LDV, Schornagel JH.Comparison of patient and proxy EORTC QLQ-C30 ratings inassessing the Quality of Life of cancer patients. J Clin Epidemiol1998;51(7):617–631.

10. Drummond HE, Ghosh S, Fergunson A, Brackenridge D,Tiplady B. Electronic quality of life questionnaires: acomparison of pen-based electronic questionnaires withconventional paper in a gastrointestinal study. Qual Life Res1995;4:21–26.

11. Fayers PM, Aaronson NK, Bjordal K, Groenvold M, Curran D,Bottomley A. The EORTC QLQ-C30 Scoring Manual. 3rd ed.Bruxelas: European Organization for Research and Treatmentof Cancer; 2001.

12. Ferreira PL. The Portuguese version of the EORTC QLQ-C30.Proceedings of the 10th International Meeting of EuropeanSociety of Gynaecological Oncology. Bologna, Itália: MonduzziEditore; 1997. p. 527–532.

13. Rogausch A, Sigle J, Seibert A, Thuring S, Kochen MM, HimmelW. Feasibility and acceptance of electronic quality of lifeassessment in general practice: an implementation study.Health Qual Life Outcomes 2009;7:51.

14. Buxton J, White M, Osoba D. Patients' experiences using acomputerized program with a touch-sensitive video monitorfor the assessment of health-related quality of life.Qual Life Res1998;7(6):513–519.

15. Velikova G, Keding A, Harley C, Cocks K, Booth L, Smith AB,et al. Patients report improvements in continuity of care whenquality of life assessments are used routinely in oncologypractice: secondary outcomes of a randomised controlled trial.Eur J Cancer 2010;46:2381–2388.

16. Velikova G, Booth AB, Smith P, Lynch P, Brown J, Selby P.Measuring Quality of Life in routine oncology practiceimproves communication and patient well-being: arandomized controlled trial. J Clin Oncol 2004;22(4).

17. Snyder C, Blackford A, Aaronson NK, Detmar S, Carducci M,Brundage M, et al. Can patient-reported outcome measuresidentify cancer patients' most bothersome issues? J Clin Oncol2011;29.

18. Oliveira A, Antunes B, Pimentel FL, Ferreira PL. QoLinformation system of the oncological patient: comparison of

the traditional paper data collection versus electronic version:using a report to the doctor. Support Care Cancer June 2009;17(7):304.

19. Oliveira A, Ferreira PL, Antunes B, Pimentel FL. OnQol:information system to capture quality of life data fromoncology patients. Book of Abstract. International Society forQuality of Life Research; 2009. p. A52–A53.

20. Oliveira A, Ferreira PL, Antunes B. Pimentel FL Quality of life inoncology, electronic device to collect data. Acta Med Port2010;23:1017–1024.

21. Pestana G. Análise de dados para ciências sociais — acomplementaridade do SPSS. 2nd ed. rev. Lisboa: EdiçõesSílabo; 2000.

22. Yarnold PR, Stewart MJ, Stille FC, Martin GJ. Assessingfunctional status of elderly adults via microcomputer. PerceptMot Skills 1996;82:689–690.

23. Greenwood M, Hakim A, Carson E, Doyle D. Touch-screencomputer systems in the rheumatology clinic offer a reliableand user-friendly means of collecting quality-of-life andoutcome data from patients with rheumatoid arthritis.Rheumatology 2006;45:66–71.

24. Newell S, Girgis A, Sanson-Fisher R, Stewart J. Are touchscreencomputer surveys acceptable to medical oncology patients?J Psychosoc Oncol 1997;15:37–46.

25. Wilson AS, Kitas GD, Carruthers DM, Reay C, Skan J, Harris S,et al. Computerized information-gathering in specialistrheumatology clinics: an initial evaluation of an electronicversion of the Short Form 36. Rheumatology 2002;41:268–273.

26. Lewis G, Sharp D, Bartholomew J, Pelosi AJ. Computerizedassessment of common mental disorders in primary care:effect on clinical outcome. Fam Pract 1996;13:120–126.

27. Pouwer F, Snoek FJ, van der Ploeg HM, Heine RJ, Brand NA. Acomparison of the standard and the computerized versions ofthe Well-Being Questionnaire (WBQ) and the DiabetesTreatment Satisfaction Questionnaire (DTSQ). Qual Life Res1998;7:33–38.

28. Bliven B, Kaufman S, Spertus J. Electronic collection ofhealth-related quality of life data: validity, time benefits, andpatient preference. Qual Life Res 2004;10(1):15–21.

29. Cella DF. Methods and problems in measuring quality of life.Support Care Cancer 1995;3:11–22.

30. Bezjak A, Skeel R, Depetrillo AD, Comis R, Taylor RKM.Oncologists' use of quality of life information: results of asurvey of Eastern Cooperative Oncology Group physicians.Qual Life Res 2001;10(1):1–13.

31. Carlson LE, Speca M, Hagen N, Taenzer P. Computerizedquality-of-life screening in a cancer pain clinic. J Pal Care2001;17(1):46–52.

32. Velikova G, Brown JM, Smith AB, Selby PJ. Computer-basedquality of life questionnaires may contribute to doctor–patientinteractions in oncology. Br J Cancer 2002;86(1):51–59.

33. Greenhalgh J. The applications of PROs in clinical practice:what are they, do they work, and why? Qual Life Res 2009;18:115–123.

34. Lohr KN, Zebrack BJ. Using patient-reported outcomes inclinical practice: challenges and opportunities. Qual Life Res2009;18:99–107.

35. Davis, Cella, 1997 cit. in Baum A, Anderson B. PsychosocialInterventions for Cancer. Washington: AmericanPsychological Association; 2001.