Embed Size (px)
Citation preview
Scientific Research Project
K.F. Driest, BSc. February 2015
Measuring quality of teleconsultation services, the patients’ perspective
The development of a concept questionnaire according to the first phase of the Consumer Quality Index
Measuring quality of teleconsultation services, the
patients’ perspective
The development of a concept questionnaire according to the first phase of the
Consumer Quality Index
Trainee Keiko F. Driest Student number: 6174280 E-mail: [email protected]
Mentor MSc, L. Thijssing Academic Medical Center – University of Amsterdam Department of Medical Informatics Meibergdreef 9 1105 AZ Amsterdam, the Netherlands
KSYOS TeleMedisch Centrum Professor J.H. Bavincklaan 2 - 4 1183 AT Amstelveen, the Netherlands Email: [email protected]
Tutor Prof. Dr. M.W.M. Jaspers Academic Medical Center – University of Amsterdam Department of Medical Informatics Meibergdreef 9 1105 AZ Amsterdam, the Netherlands Email: [email protected]
Location Scientific Research Project Academic Medical Center – University of Amsterdam Department of Medical Informatics Meibergdreef 9 1105 AZ Amsterdam KSYOS TeleMedisch Centrum Professor. J.H. Bavincklaan 2 - 4 1183 AT Amstelveen
Practice Teaching Period December 2013 – February 2015
1
Preface
This thesis is the final assignment of the Master Medical Informatics of the University of Amsterdam.
It describes my scientific research project (SRP). I performed my SRP at the department of Medical
Informatics of the Academic Medical Center (AMC) in Amsterdam and KSYOS TeleMedical Center in
Amstelveen. The aim of this thesis is to develop a standardized concept questionnaire according to
the first phase of the Consumer Quality Index to assess patients’ experiences with teleconsultation
services. This thesis reports on the first step of the development and has as result a concept
questionnaire.
I express my deep gratitude to my mentors Leonie Thijssing, Linda Dusseljee-Peute (the first period)
and my supervisor Monique Jaspers for their guidance, feedback, support and supervision
throughout this research project.
Furthermore I would like to thank Job van der Heijde and Leonard Witkamp for the opportunity to
perform my SRP at KSYOS TeleMedical Center. I have learned a lot in the past year.
Without the patients, the GPs, the dermatologist, Job van der Heijde and Arda van Breda (as
teleconsultation providers) for the focus group and workgroup, there would not be a final concept
questionnaire available. Therefore I thank them for their time and feedback.
I wish to express my gratitude to my family, boyfriend and friends for supporting me and making it
possible to finalize my study. Special thanks to all the people that reviewed my work during the
different stages of progress.
Finally, I thank my fellow students in the room J1b-213.1 at the Medical Informatics department for
the support and the great time. Konec Hry!
Keiko Driest
Amsterdam, February 2015
2
3
Table of contents
Samenvatting ................................................................................................................... 5
Abstract ............................................................................................................................ 7
1 General introduction .................................................................................................. 9
Bibliography ................................................................................................................... 11
2 Literature review: Quality aspects of teleconsultation from the patients’ perspective 12
2.1 Introduction ................................................................................................................ 12
2.2 Method ....................................................................................................................... 13
2.2.1 Search strategy ................................................................................................................................ 13
2.2.2 Study selection ................................................................................................................................ 13
2.3 Results ........................................................................................................................ 15
2.3.1 Quality aspects ................................................................................................................................ 17
2.4 Discussion ................................................................................................................... 19
2.5 Conclusion ................................................................................................................... 21
Bibliography ................................................................................................................... 22
3 Development of the CQ-index for Teleconsultation services ...................................... 24
3.1 Introduction ................................................................................................................ 24
3.2 Background CQ-index .................................................................................................. 25
3.2.1 Development phases CQ/index ....................................................................................................... 25
3.3 Methods...................................................................................................................... 26
3.3.1 Development of the concept CQ-index of Teleconsultation Services ............................................. 26
3.3.2 Study participants of the focus groups ........................................................................................... 26
3.3.3 Focus groups ................................................................................................................................... 27
3.3.4 Mapping the quality aspects of the literature review with the quality aspects of the focus groups
28
3.3.5 Validation Workgroup ..................................................................................................................... 28
3.4 Results ........................................................................................................................ 28
3.4.1 Focus groups ................................................................................................................................... 28
3.4.2 Quality aspects ................................................................................................................................ 28
3.4.3 Clustering ........................................................................................................................................ 29
3.4.4 Importance rating ........................................................................................................................... 29
3.4.5 Mapping .......................................................................................................................................... 35
3.4.6 Validation workgroup ...................................................................................................................... 35
3.4.7 Final Questionnaire ......................................................................................................................... 36
3.5 Discussion ................................................................................................................... 37
3.6 Conclusion ................................................................................................................... 40
Bibliography ................................................................................................................... 41
List of abbreviations ....................................................................................................... 43
4
Appendix A: Approval of the Medical Ethical Commission ................................................ 44
Appendix B: Invitation letter ........................................................................................... 46
Appendix C: Confirmation letter ...................................................................................... 49
Appendix D: Background characteristics .......................................................................... 52
Appendix E: From quality aspect to question ................................................................... 53
Appendix F: Origin of questions and the changed made by the Workgroup ...................... 60
Appendix G: Concept Questionnaire ................................................................................ 64
5
Samenvatting Doel: De vraag van de gezondheidszorg in Nederland groeit. In de afgelopen tien jaar groeide de
uitgaven in de gezondheidszorg met 4,4% per jaar. De verwachting is dat deze groei zich zal
voortzetten. Telemedicine fungeert als een potentieel middel om de kosten van de gezondheidszorg
te verlagen, de toegang tot diensten te verbeteren, de juiste expertise overal beschikbaar te maken,
in een vroeg stadium te zorgen voor de juiste behandeling en het verbeteren van de kwaliteit van de
zorg. De zorg die wordt verstrekt via teleconsultatie diensten heeft bewezen zo efficiënt en effectief
te zijn als fysieke consult met een hoge diagnostische nauwkeurigheid. Echter, slaagt het beschikbare
onderzoek niet in om een bevredigende uitleg van de onderliggende redenen voor de tevredenheid
van de patiënt (of ontevredenheid) met Telemedicine te bieden. Bovendien laat bestaande literatuur
beperkingen zien van patiënt tevredenheid vragenlijsten met betrekking tot Telemedicine diensten
en ook beperkingen in het ontwikkelingsproces van deze vragenlijsten. Specifieker, bij deze studies
ontbreken vaak informatie over de betrouwbaarheid, factoranalyse en de geldigheid van de
vragenlijsten gebruikt om de tevredenheid van de patiënt met Telemedicine diensten te beoordelen.
Bij KSYOS TeleMedisch Centrum is er behoefte om ervaringen van patiënten te meten met de
Teleconsultatie diensten die zij leveren. Er is daarom behoefte naar een gestandaardiseerde en
gevalideerde vragenlijst. De antwoorden op dergelijke vragenlijst kan worden gebruikt om de
kwaliteit van zorg geleverd door teleconsultatie diensten te verbeteren. Het KSYOS Teleconsultatie
Systeem (KSYOS TeleMedisch Centrum) biedt huisartsen de mogelijkheid om te communiceren met
andere zorgverleners (specialisten). Deze mededeling is gebaseerd op de ‘store and forward’
modaliteit. In deze scriptie willen we een gestandaardiseerde concept vragenlijst ontwikkelen.
Methoden: De concept vragenlijst is ontwikkeld op basis van de Consumer Quality Index (CQ-index).
De CQ-index is een gestandaardiseerde methode voor het ontwikkelen van vragenlijsten en het
meten van kwaliteit van zorg vanuit het perspectief van de patiënt. De CQ-index bestaat uit drie
fases. De eerste fase bestaat uit het maken van een concept vragenlijst. De volgende stappen werden
in de eerste fase uitgevoerd: (i) een systematische literatuuronderzoek dat gericht is op het
identificeren van kwaliteitsaspecten met betrekking tot teleconsultatie diensten die patiënten
ervaren als relevant of belangrijk, (ii) twee focusgroepen met patiënten gericht op het identificeren
van kwaliteitsaspecten met betrekking tot de levering van store-and-forward Teleconsultatie
diensten, (iii) het in kaart brengen van de resultaten van de systematische literatuurstudie en de
resultaten van de focusgroepen om een unieke lijst te creëren van de kwaliteitsaspecten, (iv) het
ontwikkelen van vragen aan de hand van de kwaliteitsaspecten en (v) het pre-valideren van het
concept vragenlijst met een werkgroep bestaande uit stakeholders van Teleconsultatie diensten
(patiënten, huisartsen, specialisten, leveraars van de teleconsultatie diensten). De eerste fase van de
CQ-index is uitgevoerd in deze studie.
Resultaten: Uit de literatuur studie zijn er 22 kwaliteitsaspecten gevonden. Uit de focus groepen zijn
er ook 22 kwaliteitsaspecten gevonden. Vijf van de kwaliteitsaspecten uit de focus groepen kwamen
overeen met kwaliteitsaspecten van de literatuur studie. Twee kwaliteitsaspecten uit de literatuur
studie zijn niet gebruikt in het concept vragenlijst. 19 verplichte vragen, geleverd door de CQ-index
zijn toegevoegd aan de concept vragenlijst. De ontwikkelde concept vragenlijst bestaat in totaal uit
56 vragen, gegroepeerd in de thema’s die geleverd zijn door de CQ-index. De concept vragenlijst
bestaat uit: (i) twee introductie vragen (ii) drie vragen betreffende het thema toegang tot zorg, (iii)
vijf vragen betreffende het thema communicatie en informatie, (iv) vijf vragen betreffende het
6
thema bejegening, (v) acht vragen betreffende het thema patiënt regierol, (vi) drie vragen
betreffende het thema competentie, (vii) zes vragen betreffende het thema organisatie van de zorg,
(viii) drie vragen betreffende het thema continuïteit van de zorg, (ix) twee vragen betreffende het
thema effectieve en veilige zorg, (x) zes vragen betreffende het thema kosten en vergoedingen, (xi)
twee vragen betreffende algemeen oordeel van teleconsultatie diensten en (xii) elf vragen
betreffende de demografie van de patiënten.
Discussie: De validiteit en de generaliseerbaarheid van de opgenomen artikelen in de systematische
literatuurstudie zijn laag, omdat de artikelen een lage steekproefomvang hadden, vaak het aantal
respons niet vermeld was en vaak geen achtergrond informatie over hun deelnemers beschikbaar
stelden. Tijdens de literatuur studie zijn er ook geen studies geincludeerd met het type store-and-
forward Teleconsultatie diensten. Kwaliteitsaspecten van Teleconsultatie diensten die patiënten
belangrijk achten kunnen hierdoor worden gemist. In de focusgroepen is er een potentieel
selectiebias opgetreden. Patiënten in de regio van het AMC werden benaderd om vrijwillig deel te
nemen. Één van de focusgroepen voldeed niet aan het vereiste aantal deelnemers. Geen van de
deelnemers in de focus groepen hebben de Teleconsultatie dienst telepulmonology ervaren.
Conclusie: Het belangrijkste voordeel van de ontwikkelde concept vragenlijst om kwaliteit te meten
van de geleverde zorg van teleconsultatie is dat deze is ontwikkeld is vanuit het perspectief van de
patiënt. Een ander voordeel is dat vanwege het brede toepassingsgebied van de systematische
literatuurstudie en focusgroepen met patiënten kan deze toegepast worden op verschillende vormen
van teleconsultatie diensten. De concept vragenlijst kan na validatie (tweede en derde fase van de
CQ-index) worden gebruikt om vanuit het perspectief van de patiënt de kwaliteit van de geleverde
zorg te beoordelen.
Toekomstig onderzoek: Toekomstig onderzoek is gericht op het testen en ontwikkelen van de
validatie en psychometrische eigenschappen van de concept vragenlijst. De tweede (constructie) fase
en de derde (psychometrische) fase van de ontwikkeling van de CQ-index moet worden uitgevoerd
om de ontwikkeling van de CQ-index van Teleconsultatie dienst vanuit de patiënt af te ronden.
Sleutelwoorden: Patiënten perspectief, CQ-index, Teleconsultatie diensten, Kwaliteit van zorg,
Kwaliteit aspecten
7
Abstract Objective: The demand of healthcare delivery in the Netherlands is growing. In the past ten years
the healthcare expenses grew with 4.4% per year. The expectations are that this growth will
continue. Telemedicine acts as a potential source to reduce health care expense, improve access to
services, make the right expertise available anywhere, early provide the appropriate treatment and
improve quality of care. The healthcare provided through teleconsultation services has proven to be
as efficient and effective as face-to-face consultation with high diagnostics accuracy. However, the
available research fails to provide satisfactory explanations of the underlying reasons for patient
satisfaction (or dissatisfaction) with Telemedicine. Furthermore, the literature shows limitations of
patient satisfaction questionnaires concerning Telemedicine services and in the development process
of those questionnaires. More specific, these studies lack details on reliability, factor analysis and
validity of the questionnaires used to assess patient satisfaction with Telemedicine services. There is
therefore a need of a standardized and validated questionnaire with quality aspects of Telemedicine
services that patients perceive as important can be measured. The responses on such a
questionnaire can presumably be used to enhance the quality of care delivered by Telemedicine. At
KSYOS TeleMedical Center there is a need to measure patients’ experiences with the
Teleconsultation services they provide. The KSYOS Teleconsultation System provides General
Practitioners (GPs) the ability to communicate with other healthcare providers (specialists). This
communication is based on store-and-forward modality. In this thesis we aim to develop a
standardized concept questionnaire.
Methods: The concept questionnaire was developed according to the Consumer Quality Index (CQ-
index) methodology. The CQ-index is a standardized method for developing surveys and measuring
healthcare quality from the patient’s perspective. The CQ-index consists of three phases. The first
phase is used to develop the concept questionnaire. The following steps were performed: (i) a
systematic literature review that aimed to identify quality aspects concerning the delivery of
Teleconsultation services that patients perceived as relevant or important, (ii) two focus groups with
patients aimed to identify quality aspects concerning the delivery of store-and-forward
Teleconsultation services, (iii) the mapping of the results from the systematic literature review and
the two focus groups to create a unique list of quality aspects, (iv) the development of questions
from the quality aspects and (v) the pre-validation the concept questionnaire with a workgroup
consisting of stakeholders from the Teleconsultation services (patients, GPs, specialist,
teleconsultation provider). The first phase is conducted in this study and reported in this thesis.
Results: From the systematic literature review 22 quality aspects were revealed. From the focus
groups also 22 quality aspects were revealed. Five of the quality aspects of the focus groups matched
with the quality aspects of the literature study. Two quality aspects of the literature review were not
included into the concept questionnaire. 19 required questions provided by the CQ-index were
included. The final concept questionnaire consisted of 56 questions, clustered according to the
themes of the CQ-index. The concept questionnaire consist of (i) two introduction questions (ii) three
questions concerned the theme access to care, (iii) five questions concerned the theme
communication and information, (iv) five questions concerned the theme interpersonal conduct, (v)
eight questions concerned the theme patient management role, (vi) three questions concerned the
theme competence, (vii) six questions concerned the theme organization of care, (viii) three
questions concerned the theme continuity of care, (ix) two questions concerned the theme effective
8
and safe care, (x) six questions concerned the theme costs and compensation, (xi) two questions
concerned patients general judgment of teleconsultation and (xii) eleven questions concerned the
demographics of the patients.
Discussion: The validity and the generalizability of the included articles in the systematic literature
review are low. The included articles had low sample sizes, often no response rate was reported and
often no descriptive information about their patient sample was reported. We also did not encounter
studies on store-and-forward Teleconsultation services. Quality aspects of Teleconsultation services
that patients deem important can be missed. In the focus groups there is a potential selection bias
for the participants of the focus groups. Patients in the region of the AMC were contacted to
participate voluntarily. One focus group did not meet the requirement of six participants. None of
the participants in the focus groups experienced the Teleconsultation services telepulmonology and
is therefore not represented.
Conclusion: The main advantage of this concept questionnaire to measure quality aspects of
teleconsultation services is that it is developed from the patients’ perspective. Another advantage is
that after validation (second and third phase of the CQ-index) the questionnaire can be used to
assess patients perspectives on quality of care delivered through any kind of teleconsultation service,
because of the broad scope of the systematic literature study and the focus groups conducted with
patients familiar with different kind of teleconsultation services.
Future research: Future research is focussed on the validation and psychometric properties of the
concept questionnaire. The second (constructive) phase and third (psychometric) phase of the
development of the CQ-index needs to be performed to complete the development of the CQ-index
of teleconsultation services from the patients’ perspective.
Keywords: Patients’ perspective, CQ-index, Teleconsultation services, Quality of care, Quality
aspects
9
1 General introduction The demand of healthcare delivery in the Netherlands is growing. In the past ten years the
healthcare expenses grew with 4.4% per year. The expectations are that this growth will continue.
The reasons for the rising costs are ageing of the population, higher welfare, changing society and
new medical technology and treatment options. The number of older people is increasing and the
elderly live longer, but often with chronic diseases putting a burden on the healthcare system. Higher
welfare results in changes in the society, resulting in higher expectations on the healthcare delivery
with less discomfort and a need of patients to be well informed about the risks or treatment of their
diseases. New treatment techniques, drugs and devices contribute to a better healthcare. They are
often cheaper than the older alternatives. The number of treatments increase with more patients
being helped and cured (1).
Taking advantage of telecommunication and information technologies, telemedicine acts as a
potential source to reduce health care expense, improve access to services, make the right expertise
available anywhere, early provide the appropriate treatment and improve quality of care (2). Recent
works in communication technologies have inspired the development of telemedicine to a large
extent.
One example of Telemedicine, Teleconsultation has been used more often as a new means of health
care delivery in the past few years (3,4). The healthcare provided through Teleconsultation services
has proven to be as efficient, and effective as face-to-face consultation with high diagnostics
accuracy. Teleconsultation services in the field of dermatology (teledermatology) prevented 78% of
unnecessary physical referrals, yielding an 18% reduction in conventional costs (5). In the field of
pulmonology (telepulmonology), Teleconsultation services prevented 68% of unnecessary physical
referrals (6). In the field of dermatology (teledermatoscopy), diagnostic accuracy was determined
between teledermatoscopy and face-to-face services. The agreement between the specialist (that
answered the teledermatoscopy) and face-to-face was 66% (7).
Brejaart (2008) shows that perceived usefulness, perceived ease of use and compatibility (degree to
which an innovation like teleconsultation is consisted with existing values and needs of stakeholders)
are factors that significantly influence the acceptance of teledermatology among physicians (8).
Telemedicine literature concerning publications about patient satisfaction with Telemedicine shows a
generally positive trend. As a consequence there is a tendency to presume that further research into
patients’ experiences with Telemedicine is no longer needed. However, the available research fails
both to provide satisfactory explanations of the underlying reasons for patient satisfaction or
dissatisfaction with Telemedicine and to explore factors that contribute or impede patients’
satisfaction with of telemedicine services in depth (9). Patients may for example have concerns about
the technology, their privacy, the impersonal character of the consult, the accuracy of the
Teleconsultation services as opposed to face-to face consultations, communication patterns between
the involved healthcare providers, and whether they can trust the information provided (10).
Therefore, there is a need of a standardized and validated questionnaire with quality aspects of
Teleconsultation services that patients perceive as important can be measured. The responses on
such a questionnaire can presumably be used to enhance the quality of care delivered by
teleconsultation, give choice information to healthcare consumers, advocacy information for patients
and patient organizations to inform their members about the quality of care of telemedicine services,
10
healthcare purchasing information for healthcare insurers, monitoring and policy information for the
government, supervisory information for the healthcare inspectorate and quality information for
institutions and professionals in healthcare with respect Telemedicine services (11).
In this thesis we aim to develop a standardized concept questionnaire. In the Netherlands, the
Consumer Quality Index (CQ-index), a standardized method for developing surveys and measuring
healthcare quality from the patient’s perspective, was introduced in 2006 in order to promote
patient-centred care. In order to obtain reliable and valid questionnaires, the development process is
structured by guidelines. The content validity of questionnaires is ascertained during a qualitative
phase which includes a literature search, patient focus groups and interviews with stakeholders.
After this phase a pilot study on the CQ-index questionnaire should be performed to determine
internal consistency. The CQ-index is characterized by combining patients’ experiences with the
relative importance of each experience item resulting in a list of priorities for improvement of quality
of care. The questionnaire for measuring quality aspects of Telemedicine services from the patients’
perspective will be developed according to the CQ-index guidelines.
Chapter 2 describes the literature review to gain insight in quality aspects perceived as relevant by
patients. The following research question is answered in chapter 2, the literature study:
Which quality aspects concerning the delivery of Teleconsultation services are reported in
scientific literature that patients’ perceive as relevant and important?
Chapter 3 describes (i) the elicitation of patients’ experiences and perceptions of quality aspects for
Teleconsultation during focus groups, (ii) the creation of a complete list of quality aspects of
Teleconsultation services using the mapping method to combine the quality aspects found in the
literature review and focus groups, (iii) the development of questions for the concept questionnaire
derived from the quality aspects of the complete list, (iv) the pre-validation of the concept
questionnaire by stakeholders/workgroup (consisting of patients, GPs and a specialist), (v) the
development of the final concept questionnaire. The following research questions are answered in
chapter 3, describing the method of the development of the standardized concept questionnaire:
What are the positive and negative experiences of patients who experienced
Teleconsultation services?
Which quality aspects of Teleconsultation services, based on these patients’ experiences,
should be incorporated into the validated and standardized questionnaire to assess patient
experiences regarding Teleconsultation services?
Are the questions from the concept questionnaire perceived relevant and clear by the
stakeholders/workgroup (patients, GPs, specialist and Teleconsultation providers) and does
the workgroup find the concept questionnaire complete or is it missing quality aspects?
11
Bibliography
1. Ministerie van Volksgezondheid,, Welzijn en Sport. De zorg: hoeveel extra is het ons waard? 2012.
2. Hjelm NM. Benefits and drawbacks of telemedicine. J Telemed Telecare. 2005;11(2):60–70.
3. Deshpande, MD MBA A, Khoja, MD, PhD S, McKibbon, BSc MLS PhD A, Jadad, MD DPhil FRCP(C)
AR. Real-Time (Synchronous) Telehealth in Primary Care: Systematic Review of Systematic Reviews.
2008 Jan [cited 2015 Jan 13]; Available from: http://www.ncbi.nlm.nih.gov/books/NBK168923/
4. Hersh, MD WR, Hickam, MD, MPH DH, Severance, MPH SM, Dana, MLS TL, Kragers, AMLS, MA KP,
Helfand, MD, MS, EPC Director M. Telemedicine for the Medicare Population: Update [Internet].
2006 [cited 2014 Jul 15]. Available from:
http://archive.ahrq.gov/downloads/pub/evidence/pdf/telemedup/telemedup.pdf
5. Van der Heijden JP, de Keizer NF, Bos JD, Spuls PI, Witkamp L. Teledermatology applied following
patient selection by general practitioners in daily practice improves efficiency and quality of care at
lower cost. Br J Dermatol. 2011 Nov;165(5):1058–65.
6. Thijssing L, van der Heijden JP, Chavannes NH, Melissant CF, Jaspers MWM, Witkamp L.
Telepulmonology: Effect on quality and efficiency of care. Respir Med. 2014 Feb;108(2):314–8.
7. Van der Heijden JP, Thijssing L, Witkamp L, Spuls PI, de Keizer NF. Accuracy and reliability of
teledermatoscopy with images taken by general practitioners during everyday practice. J Telemed
Telecare. 2013 Sep;19(6):320–5.
8. Brejaart J. The adoption of teledermatology; Factors affecting the adoption by physicians. 2008.
9. Mair F, Whitten P. Systematic review of studies of patient satisfaction with telemedicine. Bmj.
2000;320(7248):1517–20.
10. Zilliacus EM, Meiser B, Lobb EA, Kirk J, Warwick L, Tucker K. Women’s Experience of Telehealth
Cancer Genetic Counseling. J Genet Couns. 2010 Apr 22;19(5):463–72.
11. Koopman L, Sixma H, Hendriks M, Boer, de D, Delnoij D. Handboek CQI Ontwikkeling. 2011.
12
2 Literature review: Quality aspects of teleconsultation from the
patients’ perspective
2.1 Introduction In recent years, patient satisfaction gained increasing attention in assessing the quality of healthcare
delivery (1). Patient satisfaction refers to a balanced evaluation of patients’ experiences of the health
care they received, in relation to their personal preferences or expectations (2), regarding multiple
indicators concerning the structure, process, and outcome of care (3). Structure of care concerns all
factors that affect the context in which care is delivered (e.g., number of doctors per patient,
presence of equipment), whereas process of care indicators refer to all the activities performed to
deliver health care (e.g., average patient waiting times, percentage of patients compliant with
treatment). Outcome of care indicators measure the impact of health care on patients’ wellbeing
(e.g., health status, quality of life) (4). The extend to which patients are satisfied with certain health
care delivery modes can influence patients’ willingness to play an active role in their own care (4,5),
can increase their desire or willingness to continue using certain health care services (5–7) and can
lead to higher treatment compliance (5,7). Furthermore a higher patient satisfaction with quality of
care has a positive influence on patients’ physical and mental well-being (7).
A rather new mode of healthcare delivery is Telemedicine. Teleconsultation services is a specific type
of Telemedicine and can be defined as the consultation of a healthcare professional by another
healthcare professional using information technology to bridge the spatial distance between the
participants (8). The information that is communicated between these professionals can either be
synchronously transmitted (i.e. interaction is performed real-time such as video conferencing) or
asynchronously (i.e. pre-recorded or ‘store-and-forward’) (9). Teleconsultation helps eliminate
distance barriers, can improve access to medical services that would often not be consistently
available and decrease waiting time for appointments (10). Patient satisfaction questionnaires are
often used to assess healthcare quality and enhance quality of care (2). However, available research
fails to provide explanations and reasons for patient satisfaction or dissatisfaction with
Teleconsultation services (11). Therefore there is no insight in whether the quality of the healthcare
service is high or whether the quality of the healthcare service should be enhanced.
In order to improve patient satisfaction with this new mode of health care delivery, it is essential to
gain insight in patients’ experiences and expectations regarding the structure, process, and outcome
of health care services delivered through Teleconsultation services. Besides showing what quality
aspects of health care services provided through Teleconsultation services patients perceive as
important, insight in those aspects may guide health care professionals and policy makers in their
focus on improving aspects of the care they provide based on patients’ quality perceptions (2). This is
specifically relevant for the indicators concerning the structure and process of Teleconsultation
services, as these tend to be more actionable than health care outcomes, and because outcomes of
care can be seen as the product of structures and processes (4).
Although there are already questionnaires available that aim to measure patient satisfaction with
general healthcare services and Telemedicine service, Crow et al. (2002) show limitations in
satisfaction studies in general healthcare services (12). Williams et al. (2001) and Mair et al. (2000)
show limitations in satisfaction studies in Telemedicine services applying these questionnaires
(10,11). The studies included in these literature reviews applied questionnaires that were brief,
13
quantitative, lacked validation, lacked standardization and did not report on non-responses. Further,
these studies did not provide sufficient details to determine the type of questions asked to patients
(11,13). Additionally, these studies did not report whether patients valued the questions posed to
assess their satisfaction with the delivery of health care services as important or relevant.
Therefore, there is a need of a standardized and validated questionnaire with which quality aspects
of Teleconsultation services that patients perceive as important can be measured. Before we can
evaluate what specific questions concerning quality aspects of Teleconsultation services should be
incorporated into this standardized and validated questionnaire, insight is needed into which quality
aspects of Teleconsultation services patients find important and relevant. In this chapter, we aim to
answer the following research question:
Which quality aspects concerning the delivery of Teleconsultation services are reported in
scientific literature that patients’ perceive as relevant and important?
To answer this research question we conducted a systematic literature review.
2.2 Method
2.2.1 Search strategy
PubMed and PsycINFO were searched for relevant studies. The search terms we used were related to
“Telemedicine”, “Patient experience”, “Patient satisfaction” and “Quality”. The literature searches
were restricted to English language articles. Figure 1 shows the keywords used and Figure 2 shows
the search strategy used to identify relevant articles.
Searches were constructed using three groups. Group A consisted of terms related to telemedicine,
group B consisted of terms related to patients’ perspective and group C consisted of terms related to
quality of healthcare. Searches in PubMed were constructed using all groups. Searches in PsycINFO
were constructed using group A and from B only the keywords patient satisfaction and patient
experience. The keyterms within a group were searched with “ OR” and between the groups “AND”
was used. A librarian was consulted to help with the search terms and strategy.
2.2.2 Study selection
Two researchers (KD, LDP) independently reviewed and assessed titles and abstracts of the resulting
papers against predefined inclusion criteria. Articles that were included focused on exploring
patients’ satisfaction or patients’ experiences with Teleconsultation services, specifically those who
focused on quality of care delivered through Teleconsultation services. Papers that only reported an
overall satisfaction rate of patients with teleconsultation services were excluded, since these articles
would not provide information on the specific quality aspects of telemedicine/teleconsultation
services that patients perceive as important and relevant.
Inter-reliability of the two reviewers was determined by calculating the Cohen’s kappa. The kappa
coefficient is a chance-corrected measurement of the agreement between reviews. A Cohen’s kappa
of zero means that the agreement between ratings was based entirely on chance and a Cohen’s
kappa of one means that there is a perfect agreement between the reviewers. Kappa values can be
interpreted in terms k <0 is poor, 0 to 0,20 is slight, 0,21 to 0,4 is fair, 0,41 to 0,60 is moderate, 0,61
to 0,8 is substantial, and above is 0,81 almost perfect. (14). For the calculation of the Cohen’s kappa
SPSSv20 was used.
14
After a first exclusion of studies based on the title and abstract, a full-text analysis was performed on
the remaining articles. An article was included by the two reviewers (KD, LT) if: (i) the focus of the
study was on assessing the Teleconsultation service on quality aspects that patients deem important;
(ii) the study provided insights in patients’ perspectives on quality aspects of teleconsultation
services.
The included articles were screened on reports on quality aspects perceived as important and
relevant by the patients. Data on quality aspects was derived from these articles, if the quality aspect
referred to one of the indicators concerning the structure and process of healthcare delivery. Data on
quality aspects was not derived from these articles if: (i) predefined, standardized questionnaires
which had been constructed with no direct input from the patients were used in the study (e.g. with
no possibility for the patients to comment if they find the question important or relevant), and (ii)
the study concerned comparison of face-to-face (traditional) healthcare delivery with
Teleconsultation services, as these studies gave no insight into quality aspects of Teleconsultation
services itself that patients rate as important. The two reviewers (KD, LT) extracted all relevant
quality aspects from the selected articles. Any disagreements were mediated by a third person and
were resolved by consensus.
Quality aspects derived from the selected articles were clustered according to themes of the
Consumer Quality Index (CQ-index) methodology. The CQ-Index is a standardized methodology for
measuring, analysing and reporting customer experiences in healthcare. The themes were: (i) access
1. Telemedicine
2. Telehealthcare
3. Telehealth
4. Telediagnostic
5. Remote
consultation
6. Teleconsultation
7. Teledermatology
8. Telecardiology
9. Telepulmonology
10. Teleophthalmology
11. teleradiology
12. Telenursing
13. Telepharmacy
14. Telerehabilitation
15. Telepsychiatry
16. Teleneurology
17. Patient acceptance
of healthcare
18. Consumer
satisfaction
19. Patient perspective
20. Patient satisfaction
21. Patient experience
22. Quality assurance,
health care
23. Delivery of health
care
24. Quality of
healthcare
Figure 1 Groups of keywords and MeSH terms used in the search strategy. MeSH Terms are in bold. Tiab (title or abstract) are in italic
A: Teleconsultation
services
B: Patients’ perspective
C: Quality of Healthcare
15
to care, (ii) communication and information, (iii) interpersonal conduct, (iv) patient management
role, (v) competence, (vi) organization of care, (vii) continuity of care, (viii) effective and safe care
and (ix) costs and compensation (15).
The following information was extracted from the articles: author, year of publication, study location,
type of telemedicine system, medical field (speciality) in which the Teleconsultation service is used,
methodology used and sample size.
2.3 Results The initial search identified 1474 articles. First 88 articles were removed, because they concerned
duplicates. From the remaining 1474 articles, 1349 were excluded based on the title and abstract
screening. Reasons for exclusion of these articles were: (i) the abstract was not available (n=126), (ii)
article was not written in English (n=48), (iii) patient satisfaction nor patient experiences with
Teleconsultation services were reported (n=610), (iv) no quality assessment or quality aspects
concerning teleconsultation services were reported (n=427), (v) the subject of the articles was not
Teleconsultation services (n=95) and (vi) if a patients’ satisfaction rate was reported, but no
specifications on which quality aspects of Teleconsultation services patients were satisfied on (n=42).
The Cohen’s kappa of the first study selection based on title and abstract was moderate (Kappa=
0,508).
For the remaining set of 37 articles, the reasons for exclusion of articles were: (i) the quality aspects
concerning Teleconsultation services were not reported by the patients themselves (n=25), (ii) the
focus was on tele-monitoring instead of Teleconsultation services (n=1) and (iii) the non-availability
of the full text article (n=4). Ultimately seven articles were included in the review. The Cohen’s kappa
of the inclusion of the full-text analysis of these seven articles was substantial (Kappa = 0,637). The
search flow is shown in figure 2. The included articles were read to extract quality aspects. The
inclusion of each quality aspect was based on consensus between the two reviewers.
The studies were performed in four different countries: two in the USA (16,17), three in Australia
(18–20), one in Colombia (21) and one in Canada (22). The stakeholders using the Teleconsultation
services of these studies were patients and clinical pharmacists (16), patients and nurses (18,22),
patients and specialists (dermatologists or surgeons) (21), patients and physicians (17), patients
(together with the genetic counsellor) and the specialist genetic clinicians (19). The last study
concerned a ‘’willingness to use’’ study of Teleconsultation services by audiology patients and their
healthcare professional (20). The main characteristics of the articles are summarized in table 1.
Four of the seven studies performed face-to-face interviews using a semi structured questionnaire
(16,18,19,22), one study performed the patient interviews over the telephone (21), one study sent
the questionnaires on paper to patients (17) and one article did not describe how the survey among
patients was performed (20).
16
Figure 2 Search Flow
Excluded based on:
Abstract not available (n=126)
language not English (n=48)
No teleconsultation services (n=95)
No patient satisfaction/patient experience (n=610)
No quality assessment (n=427)
No quality aspects (n=42)
Not original article (n=1)
Excluded based on:
Not from a patients’
perspective (n=25)
Telemonitoring (n=1)
Full articles not available
(n=4)
Abstract/title screening for
relevance (n=1386)
Included articles (n=7)
Initial search
strategy in:
PubMed (n=1351)
I
Removal of duplicates (n=88)
Initial search
strategy in:
PsycINFO (n=123)
I
Full articles retrieved(n=37)
Total articles
(n =1474)
17
2.3.1 Quality aspects
A review of the seven articles gave insight in quality aspects of Teleconsultation services that were
reported by the patients themselves as being important and relevant. Table 2-7 summarizes the
quality aspects derived from these articles clustered in to the themes of the CQ-index. There are
themes were no quality aspects were clustered. These themes were: (i) Interpersonal conduct, (ii)
continuity of care and (iii) effective and safe care.
Table 1 Descriptive table of included articles
Author Year Study location
Telemedicine System
Telemedicine specialty
Methodology used
Sample size
Saberi et al (16) 2013 USA Real-time video teleconsultation
HIV Semi structured interviews
14
Wade et al (18) 2012 Australia Real-time video teleconsultation
Tuberculosis Semi structured interviews
11
Lopez et al (21) 2011 Colombia Real-time video teleconsultation
Dermatology and surgery
Telephone questionnaires
121
Zilliacus et al (19)
2010 Australia Real-time video teleconsultation
Cancer Genetic Counselling
Semi structured interviews
12
Sevean et al (22)
2008 Canada Real-time video teleconsultation
General healthcare
Semi structured interviews
10
Dixon et al (17) 2008 USA Real-time video teleconsultation
General healthcare
Questionnaires on paper
30
Eikelboom et al (20)
2005 Australia Teleconsultation Ear and hearing Questionnaires 116
Table 2 Access to care
Quality aspects Description Teleconsultation provided convenient appointment times bf
Patients may feel that because of video teleconsultation services, they could arrange an appointment at a time that was more convenient to them (17,18).
Patients do not need to travel to see a healthcare provider
eg
Because of the use of teleconsultation services, patients could appreciate that they may not need to travel to see the specialist physically (20,22).
Teleconsultation provides a reduction of travel time
adf
Patients may feel that because of the teleconsultation service the travel time to the healthcare provider could be reduced (16,17,19).
a= Saberi et al(16), b= Wade et al(18), c= Lopez et al(21), d= Zilliacus et al(19), e=Sevean et al(22), f= Dixon et al(17)and
g=Eikelboom et al(20)
Table 3 Communication and information
Quality aspects Description The healthcare provider was informative
d
Patients may feel that the healthcare provider can be informative about the patient’s condition. The healthcare provider could be yet more informative rather than giving emotional support to the patient (19).
All question asked by the patients were addressed properly
d
Patients may feel that all the questions they could ask can be addressed properly by their healthcare provider with the use of teleconsultation services (19).
Explanation given by the healthcare provider was clear
d
Patients may feel that the explanation they could receive about their condition from the healthcare provider through the teleconsultation service can be clear and understandable (19).
a= Saberi et al(16), b= Wade et al(18), c= Lopez et al(21), d= Zilliacus et al(19), e=Sevean et al(22), f= Dixon et al(17)and
g=Eikelboom et al(20)
18
Table 4 Patient management role
Quality aspects Description The healthcare provider provided sufficient privacy
abe
Patients may feel that the healthcare provider provided more privacy through teleconsultation in comparison with face-to-face visits, because for example there was no chance of encountering acquaintances (16,18,22).
Patients are less reliant on others for transport to the healthcare provider
g
Patients may rely on others to transport them to see the healthcare provider. By making use of teleconsultation services the patients may feel they need to rely less on transport of others (20).
Patients felt more at ease discussing issues
a
Patients may feel more at ease about discussing issues which they might never bring up in the office, because they are not talking in-person with someone (16).
Teleconsultation diminished the level of trust in the information provided
d
Patients may feel that the nature of teleconsultation services can diminished the level of trust they have in the information provided by the healthcare provider (clinician) (19).
Teleconsultation was not a disruption of patients daily life e
Patients may feel that teleconsultation services will not be an intrusion into their daily lives. The patients could start the video teleconsultation whenever they want and when the teleconsultation is finished they could carry on with their previous activities(22).
a= Saberi et al(16), b= Wade et al(18), c= Lopez et al(21), d= Zilliacus et al(19), e=Sevean et al(22), f= Dixon et al(17)and
g=Eikelboom et al(20)
Table 5 Competence
Quality aspects Description Healthcare providers or patients experienced technical difficulties during teleconsultation
acd
Patients could experience technical difficulties during the teleconsultation services. This could for example be difficulties with the camera, video or microphone (16,19,21).
Patients felt like physical exam was not possible through teleconsultation
f
Patients may feel that a video exam cannot be thorough enough, compared to an examination performed physical (17).
a= Saberi et al(16), b= Wade et al(18), c= Lopez et al(21), d= Zilliacus et al(19), e=Sevean et al(22), f= Dixon et al(17)and
g=Eikelboom et al(20)
Table 6 Organization of care
Quality aspects Description There is a smaller queue at the GP’s office than to go to the specialist physically
g
Patients could experience and appreciate that the time to wait for an appointment to see a specialist can be reduced (20).
Teleconsultation saves time for the patients
e
Teleconsultation services could save time for the patients, because the patients do not need to travel to see the specialist physically (22).
Patients do not have to wait in clinical waiting rooms
a
By using teleconsultation services the patients’ appreciate that they may not have to wait in the clinic waiting room (16).
There is less time devoted to the appointment of the consult g
Patients may feel that time devoted to the appointment could be reduced (20).
Convenient for minor problems abf
Teleconsultation services can be viewed as convenient by the participant for handling minor medical problems. Patients can find teleconsultation a good screening method for the healthcare providers that might alleviate their workload and that gives patients direct contact with the healthcare provider for minor, but significant medical problems (17).
a= Saberi et al(16), b= Wade et al(18), c= Lopez et al(21), d= Zilliacus et al(19), e=Sevean et al(22), f= Dixon et al(17)and
g=Eikelboom et al(20)
19
Table 7 Costs and compensations
Quality aspects Description Patients experienced cost saving through teleconsultation adef
Patients could experience cost savings, because of the use of teleconsultation services (16,17,19,22).
Patients have less travel expenses
e
Patients appreciated that they had less travel expenses to contact the healthcare provider (22).
Patients have less accommodation costs
de
Patients often come from long distances to see their healthcare provider. Therefore some patients arranged accommodation close to the hospital. Patients could experience and appreciate a reduction of accommodation costs (22).
Patients loses less time off work hours
eg
Patients could experience and appreciate that they could be less time off of work, because of the teleconsultation services. Lost wages and time were not of a great concern to them (20,22). Patients who did not live nearby the hospital appreciated that long driving to or arranging accommodation nearby the health care center was no longer needed.
a= Saberi et al(16), b= Wade et al(18), c= Lopez et al(21), d= Zilliacus et al(19), e=Sevean et al(22), f= Dixon et al(17)and
g=Eikelboom et al(20)
2.4 Discussion The overall aim of this research project is to develop a concept questionnaire for Teleconsultation
services from the patients’ perspective that is standardized and validated. With a systematic review
of the literature, we aimed to identify quality aspects concerning the delivery of Teleconsultation
services that patients perceived as relevant or important in scientific literature. The literature review
revealed 22 quality aspects which were clustered into nine themes of the CQ-index. The theme
access to care concerns the accessibility and availability of the health care institution or healthcare
provider. The theme communication and information concerns the degree of transparency of
information by the healthcare institution or healthcare provider and how it is communicated. The
theme interpersonal conduct concerns the aspects that have to do with the way the customer is
treated by the institution or healthcare provider. The theme patient management role concerns the
management role of the patients, whether the patients are able to make choices about the direction
and progress of their healthcare problems. The theme competence concerns how the professional
skills of the healthcare provider are experienced. The theme organization of care concerns how well
the care process as well as the physical environment is supportive of the need for care of the
patients. The theme continuity of care concerns how the cooperation between institutions and
healthcare providers is arranged. The theme effective and safe care concerns whether the care or
treatment contributed to improving the health or quality of life. The theme cost and compensation
concerns whether the patients understand what costs are involved in care and treatment.
Three quality aspects concerned access to care. Three quality aspects concerned the communication
and information. Five quality aspects concerned patients’ management role. Two quality aspects
concerned competence. Five quality aspects concerned organization of care. Finally, four quality
aspects concerned cost and compensation. Our systematic literature study missed quality aspects in
the themes interpersonal conduct, continuity of care and effective and safe care.
Yip et al. 2003 developed a questionnaire for assessing patient satisfaction with Telemedicine
videoconference services. This questionnaire was developed by a literature review and by
consultation of an expert panel. The final questionnaire consisted of fourteen questions (23).
According to the nine themes of the CQ-index, one question concerned access to care, two questions
20
concerned communication and information, two questions concerned interpersonal conduct, one
question concerned the patients management role, two questions concerned competence and four
questions concerned organization of care. Finally, two questions concerned the general judgment of
patients with videoconference services. One theme that our systematic literature study did cover
from the CQ-index and was not covered by the questionnaire from Yip et al., was effective and safe
care. Yip et al. covered the theme interpersonal conduct whereas our systematic literature study did
not cover this theme. Both Yip et al. and our systematic literature review failed to provide quality
aspects with the themes continuity of care and effective and safe care.
Yip et al reported on the validity and consistency of the questionnaire, but not on the origin of each
question. Yip et al. did not report on how they conducted the literature review. Therefore it is
difficult to explain the differences in quality aspects we could derive from the literature and those
quality aspects related to the questions that Yip et al included in their questionnaire. Moreover, their
expert panel consisted of doctors, nurses and experts in telemedicine that ensured the relevance,
clarity and coverage of the questions. The patients as one of the stakeholder group were not
involved in the development of this questionnaire.
Our literature review provides insight into which quality aspects concerning the delivery of
Teleconsultation services are reported relevant and important from the patients’ perspective. It
therefore gives insight in which quality aspects should be covered by a validated and standardized
questionnaire aiming at Teleconsultation services from the patients’ perspective. Our literature
review provides 22 quality aspects of Teleconsultation services from the patients perspective
whereas Yip et al. only used fourteen quality aspects that was developed by an literature review that
is not reported in detail and by an expert panel where other stakeholders than the patients were
consulted.
However, this literature has its own limitations. Four of the seven studies included in the literature
review had a sample size smaller than twenty patients. These patients may not be representative for
the overall population and as a consequence, quality aspects of Teleconsultation services that
patients deem important can be missed in these studies. Further, two of the seven articles did not
provide descriptive information about their patient sample included in their study (17,22). Patients
with different backgrounds can have other needs and preferences concerning Teleconsultation
services. For example patients that are younger tend to be more comfortable with telemedicine
technology (10). Moreover, five of the seven articles did not mention the response rate to the
questionnaires sent (16–18,20,22). Whitten et al. shows that results of patient satisfaction
questionnaires may be biased due to self-selection. Patients who are less satisfied with the
healthcare services are more likely to drop out of the study. Satisfaction studies are limited to those
patients who accept the healthcare service and are willing to fill in a satisfaction questionnaire (10).
Due to these limitations the generalizability and validity these studies are limited.
Another limitation of the literature review is that we did not encounter studies on store-and-forward
Teleconsultation services. Teleconsultation services provide different kinds of modes of interaction.
This can be synchronously transmitted Teleconsultation systems (e.g. videoconferencing) or
asynchronously services (e.g. store-and-forward). Studies on store-and-forward Teleconsultation
services are limited. Also Teleconsultation services can concern different stakeholders, like
healthcare providers and their patients, or healthcare providers and clinical specialists
21
communicating about a patient health status. Six of the seven studies concerned healthcare
providers interacting with their patients through Teleconsultation (16–18,20–22), the remaining
study concerned clinical specialists communicating with their patients (together with their
counsellor) through Teleconsultation services (19). Studies on types of Teleconsultation services
concerning communication of healthcare providers with clinical specialists are limited.
Due to these limitations, the generalizability and validity of the results of these studies on
Teleconsultation services included in this systematic literature review is low. As a consequence, we
may not have gained full insight in the complete set of quality aspects of Teleconsultation services.
Hence, more patient satisfaction studies on store-and-forward interaction modes of Teleconsultation
services concerning the interaction of healthcare providers and their patients should be performed.
A strength of our literature review is that the Cohen’s kappa’s concerning the agreement of
reviewers on inclusion of articles based on their title and abstract was moderate and the kappa
concerning the inclusion of articles based on full text review was substantial.
Recommendations for future research would be to complete the quality aspects with the themes of
the CQ-index where we did not find quality aspects: (i) interpersonal conduct, (ii) continuity of care
and (iii) effective and safe care. Also we recommend to development a standardized and validated
questionnaire for measuring the quality aspects of Teleconsultation services from the patient’s
perspective. The results of this literature review – providing insight in the quality aspects of
Teleconsultation services that patients perceive as important - can be used as a stepping stone for
the establishment of such a validated and standardized questionnaire.
2.5 Conclusion This systematic literature review revealed 22 quality aspects concerning the delivery of
Teleconsultation services that patients’ perceive as relevant and important. The quality aspects were
clustered in nine themes of the CQ-index. Three quality aspects concerned access to care. Three
quality aspects concerned the communication and information. Five quality aspects concerned
patients’ management role. Two quality aspects concerned competence. Five quality aspects
concerned organization of care. Finally, four quality aspects concerned cost and compensation.
However, the included articles have a small sample size, misses descriptive tables of the patients,
misses response rates, the focus is on video conferencing Teleconsultation services, and the focus is
on one type of stakeholders (healthcare providers communicating with patients through
Teleconsultation services). Due to these limitations, the generalizability and validity of the results of
the studies on Teleconsultation services included in this literature review is low. This may have led to
an uncompleted set of quality aspects of all the types of Teleconsultation services. More studies are
therefore needed with store-and-forward Teleconsultation services and Teleconsultation services
between the healthcare provider and specialist to gain fully insight in all the quality aspects of
different types of Teleconsultation services from the patients’ perspective.
22
Bibliography
1. Bleich S, Özaltin E, Murray C. How does satisfaction with the health-care system relate to patient
experience? Bull World Health Organ. 2009 Apr 1;87(4):271–8.
2. Sixma HJ, Kerssens JJ, Campen C van, Peters L. Quality of care from the patients’ perspective: from
theoretical concept to a new measuring instrument. Health Expect. 1998;1(2):82–95.
3. Lebow JL. Research assessing consumer satisfaction with mental health treatment: a review of
findings. Eval Program Plann. 1983;6(3):211–36.
4. Donabedian A. The quality of care. How can it be assessed? JAMA J Am Med Assoc. 1988 Sep
23;260(12):1743–8.
5. Asadi-Lari M, Tamburini M, Gray D. Patients’ needs, satisfaction, and health related quality of life:
towards a comprehensive model. Health Qual Life Outcomes. 2004 Jun 29;2:32.
6. Marquis MS, Davies AR, Ware JE. Patient satisfaction and change in medical care provider: a
longitudinal study. Med Care. 1983 Aug;21(8):821–9.
7. Guldvog B. Can patient satisfaction improve health among patients with angina pectoris? Int J Qual
Health Care J Int Soc Qual Health Care ISQua. 1999 Jun;11(3):233–40.
8. Asbach P, Nerlich M. A telemedicine guideline for the practice of teleconsultation. Stud Health
Technol Inform. 2003;97:1–14.
9. Craig J, Patterson V. Introduction to the practice of telemedicine. J Telemed Telecare.
2005;11(1):3–9.
10. Whitten P, Love B, others. Patient and provider satisfaction with the use of telemedicine:
overview and rationale for cautious enthusiasm. J Postgrad Med. 2005;51(4):294.
11. Mair F, Whitten P. Systematic review of studies of patient satisfaction with telemedicine. Bmj.
2000;320(7248):1517–20.
12. Crow R, Gage H, Hampson S, Hart J, Kimber A, Storey L, et al. The measurement of satisfaction
with healthcare: implications for practice from a systematic review of the literature [Internet]. Core
Research; 2002 [cited 2014 Jul 15]. Available from:
http://www.journalslibrary.nihr.ac.uk/__data/assets/pdf_file/0009/59895/ExecutiveSummary-
hta6320.pdf
13. Williams TL, May CR, Esmail A. Limitations of patient satisfaction studies in telehealthcare: a
systematic review of the literature. Telemed J E Health. 2001;7(4):293–316.
14. De Haan RJ. Cohen’s kappa [Internet]. [cited 2014 Sep 23]. Available from: The Cohen’s Kappa
determines the chance measure of agreement between reviewers. A Kappa of zero means that the
agreement between ratings was based entirely on chance and kappa one is a full agreement.
http://os1.amc.nl/wikistatistiek/index.php?title=Cohen%27s_kappa
23
15. Centrum Klantervaring Zorg. Ontwikkeltraject CQ-index: Informatie voor
patiënt/cliëntvertegenwoordigers in de begeleidende werkgroep. 2010.
16. Saberi P, Yuan P, John M, Sheon N, Johnson MO. A Pilot Study to Engage and Counsel HIV-Positive
African American Youth Via Telehealth Technology. AIDS Patient Care STDs. 2013 Sep;27(9):529–32.
17. Dixon RF, Stahl JE. Virtual visits in a general medicine practice: A pilot study. Telemed E-Health.
2008;14(6):525–30.
18. Wade VA, Karnon J, Eliott JA, Hiller JE. Home Videophones Improve Direct Observation in
Tuberculosis Treatment: A Mixed Methods Evaluation. Neyrolles O, editor. PLoS ONE. 2012 Nov
30;7(11):e50155.
19. Zilliacus EM, Meiser B, Lobb EA, Kirk J, Warwick L, Tucker K. Women’s Experience of Telehealth
Cancer Genetic Counseling. J Genet Couns. 2010 Apr 22;19(5):463–72.
20. Eikelboom RH, Atlas MD. Attitude to telemedicine, and willingness to use it, in audiology patients.
J Telemed Telecare. 2005 Dec 1;11(8):22–5.
21. Lopez C, Valenzuela JI, Calderon JE, Velasco AF, Fajardo R. A telephone survey of patient
satisfaction with realtime telemedicine in a rural community in Colombia. J Telemed Telecare. 2011
Mar 1;17(2):83–7.
22. Sevean P, Dampier S, Spadoni M, Strickland S, Pilatzke S. Patients and families experiences with
video telehealth in rural/remote communities in Northern Canada. J Clin Nurs. 2008
Sep;18(18):2573–9.
23. Yip MP, Chang AM, Chan J, MacKenzie AE. Development of the Telemedicine Satisfaction
Questionnaire to evaluate patient satisfaction with telemedicine: a preliminary study. J Telemed
Telecare. 2003 Feb 1;9(1):46–50.
24
3 Development of the CQ-index for Teleconsultation services
3.1 Introduction The use of Teleconsultation services for healthcare delivery has increased in the past few years (1,2).
To assess healthcare quality and enhance the quality of care, patients’ satisfaction questionnaires are
often used (3). Validated and standardized patient satisfaction questionnaires on traditional
healthcare services are not yet applicable to Teleconsultation services, since the regular workflows of
the general practitioner in comparison to the workflows of the general practitioner during a
teleconsultation with a patient or another healthcare provider differs. It may therefore be relevant to
measure which aspects concerning the quality of healthcare delivered through teleconsultation
patients deem relevant as opposed to aspects of healthcare quality delivered through traditional
healthcare.
Due to the broad definition of telemedicine, patient satisfaction questionnaires of telemedicine
services cannot be applied directly to Teleconsultation services. For example the “Telemedicine
satisfaction and usefulness questionnaire” is a questionnaire that is used to assess patient
satisfaction and usefulness of telemedicine services. The questionnaire consists of various items that
are not applicable to teleconsultation e.g. “The telemedicine equipment is easy to use” and “The
telemedicine system helps me to better manage my health and medical needs” (4). These two
questions refer to patients who use a (monitoring) device at home and are not aimed to assess
quality aspects of Teleconsultation services at e.g. the GP’s office.
There are questionnaires that have been developed and used specifically to assess patient
satisfaction with Teleconsultation services. However Blozik et al. showed limitations of these
questionnaires and in the development process of the questionnaires. More specific, the studies in
this systematic review lack details on reliability, factor analysis (the identification on the relevant
underlying construct) and validity of the questionnaires used to assess patient satisfaction with
Teleconsultation services. Because of these limitations no comparisons between different
Teleconsultation services, quality of care and patient satisfaction can be made adequately over time
(5).
Also, the measurements of patients’ experiences concerning the quality of care they received tend to
generate more objective information than assessments of patients satisfaction about the quality of
care. Patient satisfaction refers to a balance between patients’ experiences and preferences or
expectations (6). Williams et al. showed that patients’ satisfaction findings should be interpreted
cautiously, because it is difficult to extract data of patients’ reflections on the evaluation of the
healthcare quality only, which can be influenced by the study settings. For example, patients may
feel treated special because they receive both telemedicine as face-to-face care (7). In a study on
patients’ perceptions of a Teleconsultation services clinic, 50% of the patients approved to
participate in the study if they saw the specialist from time to time and the patients perceived the
telemedicine clinic as providing mainly a 'monitoring' function (8). Therefore questionnaires
focussing on patient experiences instead of patients’ satisfaction tend to be more adequate to assess
quality of care (6).
A validated and standardized questionnaire focussing on patients’ experiences with Teleconsultation
services is still lacking. In this chapter we describe to the development a first concept questionnaire
25
to assess quality aspects regarding Teleconsultation services from the perspective of the patient. The
first version of the questionnaire was developed using the Consumer Quality Index (CQ-index)
guidelines. The CQ-Index is a standardized methodology for measuring, analysing and reporting
customer experiences in healthcare. This chapter describes the first phase of the development
process towards a validated and standardized questionnaire. To this end we defined the following
research questions:
1. What are the positive and negative experiences of patients who experienced
Teleconsultation services?
2. Which quality aspects of Teleconsultation services, based on these patients’ experiences,
should be incorporated into the validated and standardized questionnaire to assess patient
experiences regarding Teleconsultation services?
3. Are the questions from the concept questionnaire perceived relevant and clear by the
stakeholders/workgroup (patients, GPs, specialist and Teleconsultation providers) and does
the workgroup find the concept questionnaire complete or is it missing quality aspects?
3.2 Background CQ-index The CQ-index methodology was developed by the Dutch Center for Consumer Experience in Health
Care (Centrum Klantervaring Zorg, CKZ). The CQ-index combines the inventory of patients’
experiences of a specific healthcare dlivery with an assessment of their priority. The primary aim of
the CQ-index is to promote patient-centered care. The CQ-index is based on CAHPS1 (Consumer
Assessment of Healthcare Providers and Systems) and QUOTE (QUality Of care Through the patient’s
Eyes) developed by NIVEL (Nederlands instituut voor onderzoek van de gezondheidszorg)2. The CQ-
index has already been developed for several conditions (e.g. chronic dialysis questionnaire(9) and
heart failure (10)), specific patient groups (e.g. diabetes(11)), community series and care settings
(e.g. emergency department questionnaire(12) and hospital care(13)).
The CQ-index offers a framework to develop valid and reliable questionnaires to measure patients’
experiences. These questionnaires can be applied to gain insight into the patients’ perspective on the
evaluated healthcare services.
3.2.1 Development phases CQ/index
The development of a CQ-index consists of three phases. The first phase is the
preparation/qualitative phase. The aim of this phase is to elicit the relevant quality aspects of a
certain healthcare service by conducting a systematic literature review and conducting patient focus
groups. The systematic literature study and the results from the focus groups are used as input to the
discussion with the stakeholders. In this study, the stakeholders concerned patients who experienced
the teleconsult, general practitioners who perform teleconsultations, specialists who receive and
answer the teleconsultations and the provider of the Teleconsultation services. In this chapter we
describe the first phase of the CQ-index.
1 https://www.cahps.ahrq.gov/ 2 Dutch Institution for Research of Healthcare
26
The second phase is the constructive phase. The aim of this phase is to assure that the questionnaire
is relevant, complete, unambiguous and understandable. The deliverable of this phase is a
questionanire that is usable for a pilot study.
The third phase is the psychometric phase. The aim of this phase is to validate the questionnaire by
sending it to a group of 600-1200 patients. The results of the pilot are statistically tested for
reliability of the questionnaire. The deliverable of this phase is the final questionnaire of the CQ-
index.
3.3 Methods
3.3.1 Development of the concept CQ-index of Teleconsultation Services
In this study the first phase of the development of the CQ-index to asses patient experiences with the
quality of healthcare delivered through teleconsultation was conducted. In this phase, we performed
the following steps (i) focus groups with patients who experienced teleconsultation to gain insight in
quality aspects of store-and-forward Teleconsultation services (ii) mapping the results of the focus
groups with the results of the systematic literature study (described in chapter 2) to create an
complete and unique set of quality aspects of Teleconsultation services deemed important by
patients, (iii) translate the quality aspects to questions and (iv) discuss the concept questionnaire for
prevalidation with the work group (patients, GPs, specialist, teleconsultation provider).
3.3.2 Study participants of the focus groups
This study was approved by the Medical Ethical Commission of the Academic Medical Center (AMC)
in Amsterdam (see Appendix A). A list of patients was exported from the KSYOS system inclduing
patients who had experienced a form of teleconsultation in the previous nine months. The
Teleconsultation services that these patients experienced were teledermatology, telepulmonology or
telecardiology. These patients had a complaint about their heart, lungs or skin. The General
Practitioners (or their assistants) took up to four pictures of the skin, made a spirometry or an
electrocardiogram (ECG) and sent it via the KSYOS system to the corresponding medical specialist in
the hospital for evaluation. The patient list was filtered for patients who were at least 18 years old
and lived in: Amsterdam, Abcoude, Amstelhoek, Amstelveen, Bloemendaal, Diemen, Duivendrecht,
Badhoevendorp, Haarlem, Heemstede, Hoofddorp, Mijndrecht, Oudekerk aan de Amstel, Uithoorn,
Wilnis, Zaandam or Zwanenberg. The focus group sessions were held at the AMC.
The general practitioners of the patients who were included in the final list were informed about the
study by telephone and e-mail. A short description of the study was given to the general
practitioners. If the general practitioners had no objection to approach their patients, an invitation
for the focus groups was sent with an application form on which the patients could indicate their
availability and contact information.
The participants/patients who received an invitation letter had two weeks to respond. See Appendix
B for the invitation letter and application form. Participation was on voluntary basis. The patients
who agreed, received one week before the focus group a confirmation letter with the location, date
and directions to the location. See Appendix C for the confirmation letter.
27
All participants received a €20 incentive gift card for their participation in the focus groups and their
travel expenses were returned. Compensation was later increased to €30 gift cards in an effort to
increase recruitment rates.
3.3.3 Focus groups
The aim of the focus groups was to gain insight in quality aspects of store-and-forward
Teleconsultation services and therefore the (negative and positive) experiences of patients who
experienced the store-and-forward Teleconsultation services. The results of the focus groups
together with the results of the systematic literature review were used as input to the development
of the questionnaire for assessing quality of Teleconsultation services from the patient perspective.
The focus group sessions were divided into three sections, the introduction, the focus group itself
and the wrap up. The introduction was given by the chairman, who introduced the research team,
explained the purpose of the study, explained how the data would be processed (anonymously),
distributed the informed consent form and explained the form, provided an overview of the program
of the focus group, gave a definition of Teleconsultation services and let the participants introduce
themselves.
We used concept mapping to organize the results from the focusgroup. This method is very
attractive, because it gives consensus about complex questions (3). It is primarily a group process and
so it is especially well-suited for situations where teams or groups of stakeholders have to work
together. Second, it uses a very structured facilitated approach. In concept mapping, specific steps
are followed by a trained facilitator in helping a group to articulate its ideas and understand them
more clearly (14). We used concept mapping to support the stakeholders in our focus groups to
generate opinions on quality aspects of Teleconsultation services that they deemed important and to
define relationships between those quality aspects based on the CQ index (3).
First the focus group started with the individual brainstorm asking the participants to think about
their positive and negative experiences of teleconsultation. Participants were asked to think about
their teleconsult and reflect on the process and write each experience on a different piece of paper.
After ten minutes the chairman explained the following rules for the discussion: there is no
discussion on the relevance or accuracy of the experience, there is no wrong answer, it is possible to
inspire each other, however no consensus is needed, the experience should be clear and
unambiguous. The chairman wrote the experiences down on the screen in statements and asked if
the statement reflected a patient’s experience. A list was created of all the statements mentioned in
the focus group. If the statement was ambiguous the chairman tried to make it clear and
unambiguous or split the statement up in two (or more) individual statements. Hereafter, a short
coffee/tea break was held. During the break the chairman and assistant wrote the statements on
large post-its. The session begun by asking the participants to look one more time at the created list
and identify missing statements. The chairman then picked the statements on the post-its one-by-
one and asked the participants to which other statement on a post-it it related to. If all the
statements were clustered, the chairman asked the participants to name the clusters with a common
theme. The clustering was performed with the participant group and was based on concensus. Then
all the statements were given a number and a form was distributed to the participants with all the
numbers of the statements and participants were asked if they could give a rating on how important
28
each statement was for them on a Liktert scale from one to five. The importance of each statement
was rated by the participants individually.
The wrap up of the focus group existed of the distribution of the background form and a refund of
travel expenses form. All the participants were thanked for coming and the gift cards were
distributed when participants left the room.
3.3.4 Mapping the quality aspects of the literature review with the quality aspects of the
focus groups
The aim of mapping the quality aspects of the literature review with the quality aspects of the focus
groups was to have a complete overview of the quality aspects without duplicates. This mapping was
performed by matching quality aspects from the focus groups with quality aspects from the
systematic literature review and by removing overlapping aspects. The result of this step was a
unique list of quality aspects. The quality aspects of this unique list were then clustered according to
themes defined by the CQ-index (15). The themes are (i) access to care, (ii) communication and
information, (iii) interpersonal conduct, (iv) patient management role, (v) competence, (vi)
organization of care, (vii) continuity of care, (viii) effective and safe care, (ix) costs and compensation.
The clustered, unique list with quality aspects was used as input for creation of a concept
questionnaire.
3.3.5 Validation Workgroup
A workgroup was established that consisted of different stakeholders, e.g. patients, GPs, specialists
and teleconsultation providers of KSYOS TeleMedical Center. The workgroup received the concept
questionnaire by email and provided feedback to the concept questionnaire. The stakeholders were
asked if they found the questions in the concept questionnaire relevant and clear and if they missed
any questions. The outcome of this pre-validation was a first concept the questionnaire. The concept
questionnaire will be used as input for phase two (constructive) and phase 3 (the psychometric
properties) of the development of the CQ-index for Teleconsultation services.
3.4 Results
3.4.1 Focus groups
We conducted two focus groups with patients in one in May and one in July 2014. For the first focus
group 216 invitations were sent, nine patients responded to the application form, six were able to
meet on the same day and time (one patient cancelled last minute). For the second focus group 186
invitations were sent, fourteen patients responded to the application form, eleven were able to meet
on the same day a time (five cancelled last minute). Figure 3-8 shows the participants’ characteristics
of the first and second focus group in the categories perceived health status, perceived technology
skills, type of teleconsultation experiences, if the participants were referred physically after the
teleconsult. Table 8 shows the basic background characteristics of the participants of the first and
second focus groups. For more background characteristics see appendix D.
3.4.2 Quality aspects
The focus groups also revealed 22 quality aspects. There were eight negative experiences and
fourteen positive experiences of patients with teleconsultation expressed during the focus groups.
The quality aspects and their description from both of the focus groups are summarized in table 9.
29
Fourteen quality aspects were mentioned in the first focus group. In the second focus group eleven
quality aspects were mentioned.
3.4.3 Clustering
The themes chosen by the patients that covered the patient experiences as mentioned in the focus
group were: risks, costs, quality of care, time, convenience, confidence in the GP and efficiency.
Figure 9 shows the clusters of quality aspects that were constructed during the focus group. Three
quality aspects were mentioned in both focus groups, however the patients clustered the quality
aspects in different themes. “The assessment of the teleconsultation was fast” and “Teleconsultation
saved time for patients” overlaps with the themes “Time” from the first focus group and “Efficiency”
from the second focus group. “Patients do not have to travel to see the specialist physically” overlaps
with the theme “Convenience” from the first focus group and “Efficiency” from the second focus
group.
The quality aspects that were clustered in the theme “time” of the focus groups were clustered in
the CQ-index theme “organization of care”. The quality aspects of the themes “costs” and
“convenience” of the focus groups were a combination of the theme “cost and compensation” with
“communication and information” or “access to care”. Two themes of the focus groups matched
partially on the theme of the CQ-index. These were “risk” of the focus group with “effective and safe
care” of the CQ-index and “convenience” of the focus groups with “access to care” of the CQ-index.
The rest of the themes of the focus group did not match directly or partly on a theme of the CQ-
index. This may indicate that the theme made by the participants in the focus groups tends to be
more logical/comprehensible to the participants than the themes the CQ-index provides.
3.4.4 Importance rating
Participants were asked to rate the quality aspects on a Likertscale from one (not important) to five
(very important). Table 10 shows whether the quality aspect was rated as a negative or positive
experience of the participants, the description of the quality aspect itself, the mean, the standard
deviation of the mean on the Likert scale, the number of participants that rated the importance of
the quality aspect and in which focus group the quality aspect was mentioned. The mean of the
importance rating of each quality aspect of the focus groups was high to vary from 3,4 up to 5,0. Four
quality aspects showed a high standard deviation of the mean. One participant scored the relevance
of these items very low. These quality aspects were: “the healthcare provider provided information
about the deductible costs (eigen risico)”, “there is a risk that patients would still need to be seen
physically by the specialist”, “patient was not satisfied with outcome of the teleconsult” and
“patients have clear insight in consequential costs of teleconsultation”. However, the other four
participants scored these quality aspects high: these quality aspects therefore were not excluded for
the further development of the concept questionnaire.
30
Figure 3 Perceived health status Figure 4 Perceived technical skills
Figure 5 Type of teleconsultation experienced by Figure 6 Type of teleconsultation experienced by the participants of focus group 1 the participants of focus group 2
Figure 7 The percentage of physical referrals of the Figure 8 The percentage of physical referrals of the participants after teleconsultation in focus group 1 participants after teleconsultation in focus group 2
0
2
4
6
8
1 2 3 4 5
# p
arti
cip
ants
1= excellent, 5=poor
Perceived health status
focus group 1 focus group 2
0
2
4
6
1 2 3 4 5
# P
arti
cip
ants
1=excellent, 5 = poor
Perceived technology skills
focus group 1 focus group 2
60%
0%
40%
0%
Type of teleconsultation Focus group 1
Telecardiology
Telepulmonology
Teledermatology
Don’t know
83%
0% 17%
0%
Type of teleconsultation focus group 2
Telecardiology Telepulmonology Teledermatology Don’t know
80%
20%
Physically referred Focus group 1
yes
no 50% 50%
Physically referred Focus group 2
yes
no
31
Table 8 Background characteristics of the participants
1st focus group N = 5
2nd focus group N = 6
Age 63 [54-75]* 73 [63-90]* Gender Female 80% 67% Male 20% 33% Country of birth Netherlands 80% 83% Surinam 20% 0% Ghana 0% 17% * mean [min. value - max. value]
Table 9 Results of the quality aspects derived from the focus groups
Quality aspects Description
Safety increase through additional check from specialist
The patients could feel more secure since the specialist performs an additional check on the diagnosis or treatment plan of the GP.
Healthcare provider provided information about the deductible costs (eigen risico)
In the Netherlands patients have to pay a deductible. The care that is given by a GP is covered by the insurance, but in secondary and tertiary care patients have to pay a deductible before the insurance company will pay. The patients could feel missing informed by the GP about the costs of the teleconsult, because the teleconsult has to be payed throughout the deductible.
Patients experience cost saving through use of teleconsultation
Visiting the hospital in the Netherlands is very expensive. The first table of expenses (the deductible) has to be payed by the patient themselves. The medicall bill for performing the teleconsultation is lower, than a physical consult by a specialist in the hospital. Therefore the patients could feel that the teleconsultation system saves costs.
There is a chance that patients would still need to be seen physically by the specialist
By performing a teleconsult, there is a chance that the patient should still be seen physically by a specialist.
Patient missed the face- to- face contact with healthcare provider
Patients could miss the face-to-face contact with the specialist.
Patient was not satisfied with the outcome of the teleconsult
Patients could not be satisfied with the outcome of the teleconsult.
Patient avoids waiting in clinical waiting rooms
Patients could feel it was convenient that they did not have to wait in the waiting room of the hospital to see the specialist.
Patients do not have to see the healthcare provider physically
Patients could feel it was convenient not to see the specialist physically.
Patients have clear insight in consequential costs of teleconsultation
Patients could feel that clear insight in consequential costs of the teleconsultation is needed.
The assessment of the teleconsultation was fast
Patients could feel that the teleconsultation was quickly assessed by the specialist.
32
There is a short waiting time for the (1
st) appointment
When a patient is referred to the hospital to see a specialist there is a waitinglist to make the first appointment. The waitinglist can be weeks till months. Patients could feel that when a teleconsult is send to the specialist, the waiting time to see the specialist is shorter.
Patients do not have to travel to see the specialist physically
Patients could feel that it was more convenient to not have to travel to see the specialist.
Patients missed being direct involved in the teleconsult between the GP and the specialist.
Teleconsultation is based on communication between a GP and a specialist. The GP writes information about the patient to the specialist. The patient is not direct involved in this process. Patients could feel that there is a chance that not all the information of their problem is provided to the specialist or if the specialist needs additional information to give proper diagnosis or treatmentplan the GP doesnot have the right information, because the patient is not involved in this process.
Teleconsultation saved time for patients
Patients could feel that teleconsultation saved them time. The patients did not have to travel to see the specialist in the hospital (saves travel time) and did not have to wait for the consult.
The communication of results to the patient depends on the availability of the GP
Patients could feel that the results of the teleconsultation were late, because the communication of the results depends on whether the GP has time.
Patients have doubts about if the pictures were of sufficient quality.
Patients could have some doubts, if the pictures, ECG’s, spirometries are of sufficient quality to be assess by the specialist remotely.
Teleconsultation result from the specialist took too long
Patients could feel that the time that it took to get the results can be very long. If there was something wrong with them and the results take too long, it could escalate.
Teleconsultation provides short lines of communication between healthcare providers
Patients could feel that the teleconsultation provides short lines of communication between the GP and specialist.
Teleconsultation provides additional support to the GP’s
Patients could feel that teleconsultation gives the GP additional support. It is a backup when the GP lacks knowledge of the problem or treatment.
The referral to the specialist after the use of teleconsultation was rapid
Patients could feel that if there was something wrong with them and the advice of the specialist through teleconsultation was a physical referral, the referral to the specialist would be rapid.
Patients were helped quickly by the specialist through the use of teleconsultation
Patients could feel like they were helped quickly by the specialist through teleconsultation and got the results within a few days.
The communication between healthcare providers was fast
Patients could feel that the communication between the GP and specialist was fast and could therefore be lifesaving.
33
Figure 9 Quality aspects of both focus groups clustered by the participants
First focus group
Second focus group
Chapter 3
34
Table 10 The quality aspect mentioned in the focus groups and if they were a negative or a positive experience, the mean, the standard deviation of the mean (SD), how many participants rated the importance of the quality aspect and in which focus group the quality aspect was mentioned.
Positive/ Negative
Quality Aspects Mean SD # Focus group(s)
+ Safety increase through additional check from specialist 4,2 1,10 5 1 - Healthcare provider provided information about the
deductible costs (eigen risico) 4,2 1,79 5 1
+ Patients experience cost saving through use of teleconsultation
4,4 0,55 5 1
- There is a chance that patients would still need to be seen physically by the specialist
3,4 1,52 5 1
- Patient missed the face-to-face contact with healthcare provider
4 0,71 5 1
- Patient was not satisfied with the outcome of the teleconsult
3,8 1,64 5 1
+ Patient avoids waiting in clinical waiting rooms
4,2 0,84 5 1
+ Patients do not have to see the healthcare provider physically
3,8 0,84 5 1
- Patients have clear insight in consequential costs of teleconsultation
3,8 1,64 5 1
+ The assessment of the teleconsultation was fast
4,9 0,30 11 1,2
+ There is a short waiting time for the (1st) appointment 4,6 0,55 5 1 + Patients do not have to travel to see the specialist
physically 4,0 0,71 9 1,2
- Patients missed being direct involved in the teleconsult between the GP and the specialist.
5 0,00 4 1
+ Teleconsultation saved time for patients
4,6 0,70 10 1,2
- The communication of results to the patient depends on the availability of the GP
4,5 0,5 6 2
+ Patients have doubts about if the pictures were of sufficient quality.
3,8 1,0 6 2
- Teleconsultation result from the specialist took too long 4,3 0,8 6 2 + Teleconsultation provides short lines of communication
between healthcare providers 4,0 1,1 6 2
+ Teleconsultation provides additional support to the GP’s
4,7 0,8 6 2
+ The referral to the specialist after the use of teleconsultation was rapid
4,7 0,5 6 2
+ Patients were helped quickly by the specialist through the use of teleconsultation
4,8 0,4 6 2
+ The communication between healthcare providers was fast
5,0 0,0 6 2
35
3.4.5 Mapping
The systematic literature review (chapter 2) revealed 22 quality aspects that mainly focused on
videoconference Teleconsultation services. These were clustered according to the CQ-index themes.
Three quality aspects concerned the theme access to care. Three quality aspects concerned the
theme the communication and information. Five quality aspects concerned the theme patient’s
management role. Two quality aspects concerned the theme competence. Five quality aspects
concerned the theme organization of care. Finally, four quality aspects concerned the theme cost
and compensation.
The quality aspects derived from the systematic literature review (chapter 2) were mapped on the
quality aspects of the focus groups. Five of the quality aspects of the focus groups matched with the
quality aspects of the literature study. These were: (i) patients avoid waiting in clinical waiting rooms,
(ii) patients experience cost savings through use of teleconsultation, (iii) patients do not have to
travel to see the specialist physically, (iv) teleconsultation saved time for patients and (v) there is a
short waiting time for the (1st) appointment.
Two quality aspects of the literature review were not included into the concept questionnaire. These
were: “teleconsultation was not a disruption of patients’ daily life” and “patients felt like physical
exam was not possible through teleconsultation”. These quality aspects refer specifically to
teleconsultation that is focused on videoconferencing and not for teleconsultation in general.
The mapping of the quality aspects and removal of duplicates resulted in a unique list of 37 quality
aspects. Hereafter the unique list of quality aspects were clustered according to sub domains of the
following nine themes of the CQ-index: (i) access to care, (ii) communication and information, (iii)
interpersonal conduct, (iv) patient management role, (v) competence, (vi) organization of care, (vii)
continuity of care, (viii) effective and safe care and (ix) costs and compensation.
For each quality aspect, a question for assessing its importance from the patients’ perspective was
developed. The questions developed assess each quality aspect. 19 Required questions provided by
the CQ-index methodology were included. There were two questions concerned communication and
information, and three questions concerned interpersonal conduct. There were also questions in
new themes: two question in the introduction, two questions about the healthcare delivery in
general and eleven questions about the background information of the patient.
This concept questionnaire went through an iteration round with the research team to ensure the
questions were clear and unambiguous. This was performed over mail and by a final meeting.
Appendix E provides an overview of the questions developed, the relation of each question to quality
aspects and the cluster to which each quality aspect belongs, and the origin of the quality aspect (the
literature review or focus groups). The CQ-index guideline provides optional and required questions
for each theme of the CQ-index. These questions were all included in the concept questionnaire.
3.4.6 Validation workgroup
The concept questionnaire was sent to a work group for pre-validation. The workgroup consisted of
two patients, two GPs, one specialist (dermatologist) and two teleconsultation providers. The
workgroup received the concept questionnaire and provided feedback on missing quality aspects or
quality aspects that were not clear. The workgroup indicated no missing quality aspects. In response
to the feedback of the workgroup eight questions were modified and thirteen questions were
36
removed. Main reasons for removal were: (i) they were similar to other questions, (ii) the questions
would be hard to judge by patients and (iii) the questions were not relevant to assess
Teleconsultation services. The concept questionnaire was received positively by the workgroup. See
Table 11 which questions were removed and the reasons. In Appendix F the origin of each quality
aspect is reported (focus groups, literature study, required questions of the CQ-index or optional
questions of the CQ-index), modifications to questions or answers and removed questions. The final
concept questionnaire is available in Appendix G.
3.4.7 Final Questionnaire
The final concept questionnaire consisted of (i) two questions for the introduction (ii) three questions
concerned access to care, (iii) five questions concerne
d communication and information, (iv) five questions concerned interpersonal conduct, (v) eight
questions concerned patient management role, (vi) three questions concerned competence, (vii) six
questions concerned organization of care, (viii) three questions concerned continuity of care, (ix) two
questions concerned effective and safe care, (x) six questions concerned costs and compensation, (xi)
two questions concerned their general judgment and (xii) eleven questions concerned the
background of the patients.
Table 11 Questions that were removed and the reasons
# Questions Reason 3 Hielp uw huisarts en/of assistente u
binnen een kwartier na de afgesproken tijd?
This question was an optional question from the CQ-index. The workgroup stated that this question is not related to the Teleconsultation services.
4 Hielp uw huisarts en/of assistente u binnen een kwartier?
This question was an optional question from the CQ-index. The workgroup stated that this question is not related to the Teleconsultation services.
5 Was het een probleem (geweest) om uw huisarts overdag tussen 9.00 en 17.00 uur telefonisch te bereiken?
This question was an optional question from the CQ-index. The workgroup stated that this question is not related to the Teleconsultation services.
6 Kon u een afspraak inplannen op een tijdstip dat voor u goed uitkwam, doordat er gebruik werd gemaakt van teleconsultatie?
This question was a quality aspect from the literature review. This question was removed because the workgroup did not find this question relevant for Teleconsultation services.
7 Hoefde u niet meer naar de specialist af te reizen doordat er gebruik werd gemaakt van teleconsultatie?
This question was a quality aspect from the focus groups and the literature review. This question was removed because of the similarities with question eight and nine.
12 Werd u voldoende geïnformeerd of uw foto's, spirogrammen (longfunctieonderzoek) of ecg’s (hartfilmpje) van voldoende kwaliteit waren om door de specialist op afstand beoordeeld te kunnen worden?
This question was a quality aspect from the focus groups. This question was removed because the stakeholders did not think it was relevant for Teleconsultation.
16 Was de uitleg die uw huisarts en of assistente over uw aandoening gaf voor u te begrijpen?
This question was a quality aspect from the literature review. This question was removed because it was similar to question ten and this is a required question from the CQ-index guidelines.
21 Was uw huisarts beleefd tegen u? This question was a quality aspect of the optional questions from the CQ-index guidelines. This question was removed, because the rest of the quality aspects are from the patients perspective focused on teleconsultation. No information is given on the background of this question in the CQ-index guidelines.
37
23 Kon u meebeslissen over uw behandelplan?
This question was a quality aspect of the optional questions from the CQ-index guidelines. This question was removed, because the rest of the quality aspects are from the patients perspective focused on teleconsultation. No information is given on the background of this question in the CQ-index guidelines.
37 Was het consult/de afspraak met uw huisarts korter, doordat er gebruik werd gemaakt van teleconsultatie?
This question was a quality aspect from the literature review. This question was removed because this is hard to judge for a patient.
39 Was uw uitslag sneller bekend doordat uw huisarts (en specialist) gebruik maakten van teleconsultatie?
This question was a quality aspect from the focus groups. This question was removed because of the similarities with question 40.
41 Heeft u binnen 2 werkdagen de uitslag van het teleconsult ontvangen?
This question was a quality aspect from the focus groups. This question was removed because of the specialist has 2 workdays to respond, but it does not mean that the GP discusses within 2 work days the results with the patient.
55 Zou u teleconsultatie bij uw vrienden en familie aanbevelen?
There was a choice given of the required questions of the CQ-index. There was a choice between this question (55) and question 56. Question 56 was chosen to use because it related to other patients.
3.5 Discussion This study reports on the development of a standardized and pre-validated concept questionnaire to
assess quality aspects of Teleconsultation services from the patients’ perspective. The first phase of
the development of the CQ-index of teleconsultation was accomplished. The first phase concerned:
(i) a systematic literature review that aimed to identify quality aspects concerning the delivery of
Teleconsultation services that patients perceived as relevant or important, (ii) two focus groups with
patients aimed to identify quality aspects concerning the delivery of store-and-forward
Teleconsultation services, (iii) the mapping of the results from the systematic literature review and
the two focus groups to create a unique list of quality aspects, (iv) the development of questions
from the quality aspects and (v) the pre-validation the concept questionnaire with a workgroup
consisting of stakeholders (patients, GPs, specialist, teleconsultation provider).
Yip et al. 2003 likewise developed a questionnaire: for assessing patient satisfaction with
Telemedicine videoconference services by a literature review and by consultation of an expert panel.
The final questionnaire consisted of fourteen questions (16). According to the nine themes of the CQ-
index, one question of Yip’s questionnaire concerned the theme access to care, two questions
concerned the theme communication and information, two questions concerned the theme
interpersonal conduct, one question concerned the theme the patients management role, two
questions concerned the theme competence, four questions concerned the theme organization of
care and two questions concerned the theme the general judgment of patients with videoconference
services. The main differences between our study and the study from Yip et al. are that our study
revealed more questions (quality aspects) and these questions were distributed over all of the nine
themes of the CQ-index. This higher number of questions were revealed in the focus groups with
patients as important stakeholder of the healthcare process of Teleconsultation services. Our study
focused on Teleconsultation services in general, in the systematic literature review quality aspects of
the type video conferencing (real-time) were revealed and in the focus groups quality aspects of the
type store-and-forward Teleconsultation services were revealed. Therefore our study may have
produced a more complete and more valid questionnaire for assessing Teleconsultation services on
its quality from the patient perspective than the questionnaire of Yip et al.
38
Bakken et al. 2006 developed the Telemedicine Satisfaction and Usefulness Questionnaire. Their aim
was to report on Telemedicine satisfaction and usefulness ratings of urban and rural diabetes
patients (4). Shore and Manson 2004 implemented quality assurance to telepsychiatric care of
American Indian veterans (17). Both studies used different statements to assess patients’ satisfaction
and quality of care and concerned videoconference Teleconsultation services. Our study provides a
concept questionnaire that after validation (second and third phase of the CQ-index) can be used to
assess patients perspectives on quality of care delivered through any kind of Teleconsultation
service, because of the broad scope of the systematic literature study and the focus groups
conducted with patients familiar with different kind of Teleconsultation services.
Several studies showed limitations of patient satisfaction questionnaires and in their development
process (5,7,18). Blozik et al. 2014 for example showed lack in reporting relevant details (e.g. details
on reliability, factor analysis, validity and origin of questions) of the development process of patient
satisfaction questionnaire.
Bakken et al. 2006 developed their Telemedicine Satisfaction and Usefulness Questionnaire by use of
four other questionnaires, input of two telemedicine experts and input of the research team. In their
overview on the origin of each question of the questionnaire there are two questions were N/A (not
available) is provided. No indication is given on where these two questions are based on (4). The
methodology that we used in our study which is based on CQ-index provides a structured
development process of questionnaires for assessing healthcare deliveries from the patients’
perspective. The methodology of the CQ-index warrants the development of a standardized,
validated and reliable questionnaire.
However, there are a number of limitations in this study. The first one is potential selection bias of
patients for the focus groups. Patients who received an invitation letter for the focus group were
patients from GPs which we contacted for approval to contact their patients. These were GPs who
worked in the region of the location (AMC) where the focus groups were held. The effect is that
fewer invitations were sent, so fewer patients could indicate that they were interested and willing to
participate in the focus groups. However, we expected to have an even lower response, when we
would have invited participants who had to travel more than 40 minutes to participate in one of the
focus groups. Further, certain patients who had indicated to participate in the focus group did not
show up. At the first focus group five patients showed up instead of six patients and at the second
focus group six patients instead of eleven patients. There were a few last minute cancellations and
therefore we could not anticipate on the minimum amount of patients (shift it maybe to another
day). Focus groups requite the participation of at least six and preferably twelve patients to generate
valid results (3). The lower number of participants in the focus groups may have resulted in less
interaction in the group and as a consequence certain quality aspects concerning Teleconsultation
services may not have been revealed during the sessions. Finally eight patients had experienced
telecardiology, three patients had experienced teledermatology participated in the focus groups and
no patients participated who had experienced telepulmonology. This may also have biased the
results of the focus groups towards telecardiology.
The second limitation is that the chairman in the second group (KD) had no experience in acting as
moderator at a focus group meeting. However, the chairman of the first focus group was present to
help and engage if they were any problems.
39
The strength of this study is that quality aspects from the patients’ perspective of both synchronous
(found in the literature review) and asynchronous (found in the focus groups) teleconsultation were
included in the development of the concept questionnaire.
Another strength of this study is that the concept questionnaire was developed from the patients’
perspective. Further in the pre-validation phase of the questionnaire, other stakeholders (GP’s,
specialist and Teleconsultation service providers) of the process of Teleconsultation services had the
opportunity to provide missing quality aspects or give feedback on unclear questions. Therefore, not
only the patients as important stakeholders were involved in development process of the
questionnaire, but also the other stakeholders involved in the provision of healthcare through the
Teleconsultation services.
Future research focussed on validation of the concept questionnaire is recommended. The second
(constructive) phase and third (psychometric) phase of the development of the CQ-index needs to be
performed to complete the development of the CQ-index of Teleconsultation services from the
patients’ perspective.
The second phase is the cognitive test that is conducted with ten to fifteen patients. The aim of this
phase is to assure that the questionnaire is relevant, complete, unambiguous and understandable.
This can be conducted either by paper (or mail), face-to-face, or a combination of the two.
The third phase is the psychometric where the psychometric properties are determined. This is
conducted by sending the concept of the second phase to 600-1200 patients. Analyses have to be
performed in order to validate and standardize the questionnaire. The analyses consist of: response
analysis, item analysis (non-response, sufficient response and sufficient variation), factor analysis
(correlation between factors), internal consistency reliability (Cronbach’s α) and power analysis. The
final questionnaire can be used to assess quality of Teleconsultation services.
40
3.6 Conclusion This study reports on the first phase of the development of the CQ index of Teleconsultation services.
The aim was to develop a standardized concept questionnaire to assess quality factors of
Teleconsultation serivices from the patients’ perspective. The concept questionnaire consists of 56
questions. The questions were clustered according to the CQ-index methodology into the following
subjects: (i) two questions introduction questions (ii) three questions concerned the theme access to
care, (iii) five questions concerned the theme communication and information, (iv) five questions
concerned the theme interpersonal conduct, (v) eight questions concerned the theme patient
management role, (vi) three questions concerned the theme competence, (vii) six questions
concerned the theme organization of care, (viii) three questions concerned the theme continuity of
care, (ix) two questions concerned the theme effective and safe care, (x) six questions concerned the
theme costs and compensation, (xi) two questions concerned patients general judgment of
teleconsultation and (xii) eleven questions concerned the demographics of the patients. The
advantage of this concept questionnaire is that it is developed from the patients’ perspective. The
responses on such a questionnaire can be used to enhance the quality of care delivered by
teleconsultation, give choice information to healthcare consumers, advocacy information for patients
and patient organizations to inform their members about the quality of care of telemedicine services,
healthcare purchasing information for healthcare insurers, monitoring and policy information for the
government, supervisory information for the healthcare inspectorate and quality information for
institutions and professionals in healthcare with respect Teleconsultation services (3).
However, the concept questionnaire has still to be validated in the (second) constructive phase and
(third) psychometric phase of the development of the CQ-index. The second phase is conducted by
performing a cognitive evaluation with patients to assure that the questions are clear, relevant,
unambiguous and to identify missing quality aspects. The third phase is conducted by sending the
questionnaire to patients and to determine the psychometric properties to validate the
questionnaire.
41
Bibliography
1. Deshpande, MD MBA A, Khoja, MD, PhD S, McKibbon, BSc MLS PhD A, Jadad, MD DPhil FRCP(C)
AR. Real-Time (Synchronous) telehealth in Primary Care: Systematic Review of Systematic Reviews
[Internet]. 2008 [cited 2014 Jul 15]. Available from: http://www.cadth.ca/media/pdf/427A_Real-
Time-Synchronous-Telehealth-Primary-Care_tr_e.pdf
2. Hersh, MD WR, Hickam, MD, MPH DH, Severance, MPH SM, Dana, MLS TL, Kragers, AMLS, MA KP,
Helfand, MD, MS, EPC Director M. Telemedicine for the Medicare Population: Update [Internet].
2006 [cited 2014 Jul 15]. Available from:
http://archive.ahrq.gov/downloads/pub/evidence/pdf/telemedup/telemedup.pdf
3. Koopman L, Sixma H, Hendriks M, Boer, de D, Delnoij D. Handboek CQI Ontwikkeling. 2011.
4. Bakken S, Grullon-Figueroa L, Izquierdo R, Lee N-J, Morin P, Palmas W, et al. Development,
Validation, and Use of English and Spanish Versions of the Telemedicine Satisfaction and Usefulness
Questionnaire. J Am Med Inform Assoc. 2006 Nov 1;13(6):660–7.
5. Blozik E, Allemann Iseli M, Kunz R. Instruments to assess patient satisfaction after teleconsultation
and triage: a systematic review. Patient Prefer Adherence. 2014 Jun;893.
6. Sixma HJ, Kerssens JJ, Campen C van, Peters L. Quality of care from the patients’ perspective: from
theoretical concept to a new measuring instrument. Health Expect. 1998;1(2):82–95.
7. Williams TL, May CR, Esmail A. Limitations of patient satisfaction studies in telehealthcare: a
systematic review of the literature. Telemed J E Health. 2001;7(4):293–316.
8. Mair F, Whitten P, May C, Doolittle GC. Patients’ perceptions of a telemedicine specialty clinic. J
Telemed Telecare. 2000;6(1):36–40.
9. Veer SN van der, Jager KJ, Visserman E, Beekman RJ, Boeschoten EW, Keizer NF de, et al.
Development and validation of the Consumer Quality index instrument to measure the experience
and priority of chronic dialysis patients. Nephrol Dial Transplant. 2012 Aug 1;27(8):3284–91.
10. Oosterhuis T, Triemstra M, Rademakers J. CQ-index Hartfalen: meetinstrumentontwikkeling. 2010
[cited 2014 Jul 16]; Available from:
http://www.centrumklantervaringzorg.nl/fileadmin/site/ckz/documenten/Vragenlijsten/rapport_CQI
_Hartfalen_-_def_versie.pdf
11. Stubbe J. CQ-index diabetes: schaalconstructie, betrouwbaarheid en discriminerend vermogen
van de ervaringenvragenlijst. Utrecht: NIVEL; 2007.
12. Bos N, Sturms LM, Schrijvers AJ, Stel HF van. The consumer quality index (CQ-index) in an
accident and emergency department: development and first evaluation. BMC Health Serv Res. 2012
Aug 28;12(1):284.
13. Stubbe JH, Brouwer W, Delnoij DMJ. Patients’ experiences with quality of hospital care: the
Consumer Quality Index Cataract Questionnaire. BMC Ophthalmol. 2007 Sep 19;7(1):14.
42
14. Trochim WMK. An Introduction to Concept Mapping for Planning and Evaluation. Cornell Univ.
1989(12):1–16.
15. Centrum Klantervaring Zorg. Ontwikkeltraject CQ-index: Informatie voor
patiënt/cliëntvertegenwoordigers in de begeleidende werkgroep. 2010.
16. Yip MP, Chang AM, Chan J, MacKenzie AE. Development of the Telemedicine Satisfaction
Questionnaire to evaluate patient satisfaction with telemedicine: a preliminary study. J Telemed
Telecare. 2003 Feb 1;9(1):46–50.
17. Shore JH, Manson SM. Telepsychiatric care of American Indian veterans with post-traumatic
stress disorder: bridging gaps in geography, organizations, and culture. Telemed J E Health.
2004;10(Supplement 2):S – 64.
18. Whitten P, Love B, others. Patient and provider satisfaction with the use of telemedicine:
overview and rationale for cautious enthusiasm. J Postgrad Med. 2005;51(4):294.
43
List of abbreviations
AMC Academic Medical Center
CQ-index Consumer Quality Index
GPs General Practitioners
KD Keiko Driest
LDP Linda Dusseljee- Peute
LT Leonie Thijssing
SRP Scientific Research Project
Tiab Title + abstract (in search query)
44
Appendix A: Approval of the Medical Ethical Commission
45
46
Appendix B: Invitation letter
Keiko Driest KSYOS TeleMedisch Centrum Amsterdamseweg 206 1182 HL Amstelveen
<Naam>
<Straat>
<Postcode> <Plaats>
Datum: 02-06-2014
Betreft: Uitnodiging groepsdiscussies over de patiëntervaringen van TeleConsultatie
Geachte mevrouw <Naam>,
Onlangs heeft u bij de huisarts zorg ontvangen via TeleConsultatie: de huisarts (of assistente) heeft
een foto van uw huid, een hartfilmpje of een spirogram gemaakt en via het internet naar de medisch
specialist in het ziekenhuis gestuurd ter beoordeling.
Omdat deze vorm van zorg relatief nieuw is, doen wij onderzoek naar de ervaringen van patiënten.
Met toestemming van uw huisarts willen wij u bij deze graag uitnodigen voor het deelnemen aan een
groepsdiscussie.
De groepsdiscussie is van groot belang bij het creëren van inzicht in de kwaliteitsaspecten van
TeleConsultatie vanuit het perspectief van de patiënt. In de groepsdiscussie wordt u gevraagd naar
uw ervaring van de kwaliteit van de geleverde zorg.
Wat levert het u op?
Uw ervaringen worden gebruikt om de zorg en dienstverlening van TeleConsultatie te verbeteren.
Door deel te nemen aan het onderzoek draagt u daaraan bij. Zonder u is het niet mogelijk om inzicht
te krijgen in welke kwaliteitsaspecten voor u als patiënt van belang zijn. Wij vergoeden ook de
eventueel gemaakte reiskosten en aan het einde van het onderzoek ontvangt u van ons als dank voor
deelname een cadeaubon t.w.v. 30 euro.
Wie doen dit onderzoek?
De afdeling Klinische Informatiekunde van het AMC is bezig met het opstellen van een
gestandaardiseerde vragenlijst: Consumer Quality Index (CQ-index) in samenwerking met KSYOS
TeleMedisch Centrum. De CQ-index is binnen de gezondheidszorg de gouden standaard om te meten
wat patiënten belangrijk vinden en wat hun ervaringen zijn met de zorg.
47
Waar en wanneer?
Bij iedere groepsdiscussie zullen 8-12 deelnemers aanwezig zijn. De discussies zullen ongeveer 2 uur
duren. De groepsdiscussies worden gehouden in het AMC te Amsterdam. U kunt op het bijgevoegde
aanmeldformulier aangeven op welke van de 5 data u aanwezig kunt zijn. U ontvangt hierna een
bevestiging van uw deelname en het programma.
Wat gebeurt er met de gegevens?
De resultaten worden anoniem verwerkt. Het verslag van de bijeenkomst is alleen toegankelijk voor
de onderzoekers. In het uiteindelijke rapport en de CQ-index vragenlijst worden geen
persoonsgegevens van u opgenomen. Uw behandelaar wordt niet op de hoogte gebracht van uw
deelname of mening.
Deelname
Graag horen wij binnen twee weken of u bereid bent om mee te doen aan het onderzoek. Dit kan
door middel van het retourneren van het aanmeldformulier, door een e-mail te sturen naar
[email protected] of telefonisch op 020-6000060 (bereikbaar op dinsdag, woensdag en
donderdag van 8.30 – 17:30).
Wij zullen vervolgens contact met u opnemen om de aanmelding te bevestigen.
Indien u nog vragen of opmerkingen heeft met betrekking tot dit onderzoek kunt u contact opnemen
met Keiko Driest ([email protected] of via 020-6000060).
Alvast hartelijk dank voor uw tijd!
Met vriendelijke groet,
<Handtekening> <Handtekening>
Keiko Driest, BSc
Medische Informatiekunde
Klinische Informatiekunde AMC
Dr. J.P. van der Heijden
Manager Research & Development
KSYOS Telemedisch Centrum
48
AANMELDFORMULIER GROEPSDISCUSSIE TELECONSULTATIE
Naam: ____________________________________________________
Geslacht: M / V (omcirkelen wat van toepassing is)
Geboortedatum: ___________________________________________
Adres:_____________________________________________________
Plaats: ____________________________________________________
Telefoonnummer:___________________________________________
Email adres:________________________________________________
Ervaren TeleConsultatie:
TeleDermatologie TelePulmonologie TeleCardiologie Ik weet het niet
Ja, ik ben beschikbaar op:
Maandag 30 juni
10:00 – 12:00 15:00 – 17:00
Dinsdag 1 juli
10:00 – 12:00 15:00 – 17:00
Woensdag 2 juli
10:00 – 12:00 15:00 – 17:00
Donderdag 3 juli
10:00 – 12:00 15:00 – 17:00
Vrijdag 4 juli
10:00 – 12:00 15:00 – 17:00
Nee, ik heb geen interesse om deel te nemen aan het onderzoek en wil in de toekomst voor dit onderzoek
hier niet meer voor benaderd worden.
49
Appendix C: Confirmation letter
Keiko Driest KSYOS TeleMedisch Centrum Amsterdamseweg 206 1182 HL Amstelveen
<Naam>
<Straat>
<Postcode> <Plaats>
Datum: 7 mei 2014
Betreft: Programma groepsdiscussies over de patiëntervaringen van TeleConsultatie
Geachte mevrouw <naam>
Bedankt voor u aanmelding voor het deelnemen aan een groepsdiscussie. Dit is een bevestiging
van u deelname aan de groepsdiscussies. Hierbij versturen we het programma, de datum en tijd
dat is vastgesteld. De routebeschrijving vindt u in de bijlage.
Datum: woensdag 21 mei
Tijd: 15:00 – 17:00
Plaats: J1B-121, AMC, Meibergdreef 9, 1105 AZ Amsterdam
Voorbereiding op de groepsdiscussie
U hoeft zich niet speciaal voor te bereiden op de discussie. Eventueel kunt u vooraf alvast
terugdenken aan de diagnose en behandeling die u kreeg/krijgt voor de TeleConsult:
Welke onderzoeken heeft u gehad? Wat was voor u aangenaam en wat viel u tegen?
Programma
14:45 – 15:00 uur: Ontvangst
Vanaf 15.00: De groepsdiscussie:
Kennismaking
Uitleg over onderzoek
Discussie
Samenvatting
17:00 uur: Einde en eventueel napraten.
Reiskosten
U krijgt uw reiskosten volledig vergoed. U dient hiervoor wel de bonnetjes van openbaar vervoer te
bewaren of het aantal kilometers met eigen auto te registreren.
Geluidsopname
De gehele discussie wordt op geluidsband opgenomen. Misschien is dat voor u even wennen, maar
na enkele minuten denkt u er waarschijnlijk niet meer aan.
50
Privacy
De geluidsbanden, eventuele notities en het verslag van de bijeenkomst zijn alleen toegankelijk
voor de onderzoekers. In het uiteindelijke rapport en in de vragenlijst worden geen
persoonsgegevens van u opgenomen. Uw behandelaar wordt niet op de hoogte gebracht van uw
mening.
Contact
Mocht u verhinderd zijn op genoemde datum en tijdstip of vragen hebben, wilt u dan zo vriendelijk
zijn contact op te nemen. Dit kan door een email te sturen naar: [email protected] of
telefonisch op 020-6000060 (bereikbaar op dinsdag, woensdag en donderdag van 8.30 – 17:30).
Alvast hartelijk dank en tot woensdag 21 mei om 15:00.
Met vriendelijke groet,
<Handtekening> <Handtekening>
Keiko Driest, BSc
Medische Informatiekunde
Klinische Informatiekunde AMC
Dr. J.P. van der Heijden
Manager Research & Development
KSYOS TeleMedisch Centrum
51
Bijlage: Routebeschrijving
Parkeerterreinen
De parkeerterreinen rond het AMC zijn genummerd. Wij verzoeken u te parkeren in de garage P1 of P2. Zie blauwe cirkels in het onderste plaatje.
Parkeerkaartje en betaling
Voor het parkeren betaalt u het eerste drie uur het basistarief van €3, daarna €0,50 per half uur. Denk er vooral aan uw parkeerkaartje uit de auto mee te nemen. De betaalautomaten staan bij de
verschillende uitgangen.
Openbaar vervoer
Metro: Uitstappen metrostation Holendrecht (richting Gein)
54 vanaf Amsterdam CS (via stations Amstel en Bijlmer-Arena)
50 vanaf Isolatorweg (via stations Sloterdijk, RAI en Bijlmer-Arena)
Trein: uitstappen NS-station Holendrecht
Bussen:
45 vanaf station Bijlmer-Arena
47 vanaf station Bijlmer-Arena
Locatie
De groepsdiscussie zal plaatsvinden op de locatie: J1B-121. In de bovenstaande plaatje is het
aangegeven met een groene cirkel. Volg de borden naar afdeling J. Hierna kunt u de lift of de trap
nemen naar de verdieping 1B. Mocht u het niet vinden, neem gerust contact op. Dit kan via het
volgende telefoonnummer: <Telefoonnummer>
P1
P2
52
Appendix D: Background characteristics
Table 12 Background characteristics
Focus group 1 2
Age 63 [54-75]* 73 [63-90]*
Gender
Female 80% 67%
Male 20% 33%
Country
Netherlands 80% 83%
Surinam 20% 0%
Ghana 0% 17%
Marital status
unmarried 40% 17%
married 20% 50%
Living together 20% 0%
Divorced 20% 33%
Highest education
0 None
1 Primary education 0% 0%
2 Lower secondary education 0% 0%
3 Upper secondary education 60% 17%
4 Post-secondary non-tertiary education 0% 17%
5 Short cycle tertiary education 0% 17%
6 Bachelor 0% 50%
7 Master/Doctoral 40% 0%
Employed
Yes 40% 17%
No 60% 83%
If no, reason for unemployment
Retired 67% 100%
Taking care of sick husband 33%
* mean [min. value - max. value]
53
Appendix E: From quality aspect to question
Table 13 From quality aspect to question cluster: access to care
Toegankelijke zorg
Bereikbaarheid van zorg (telefonisch, maken van afspraken, fysiek, website, folder).
L Teleconsultation provides convenient appointment times
Kon u een afspraak inplannen op een tijdstip dat voor u goed uitkwam doordat er gebruik werd gemaakt van teleconsultatie?
nee, helemaal niet een beetje grotendeels ja, helemaal
Beschikbaarheid ( aanwezigheid zorgaanbod, keuzemogelijkheid zorg, nabijheid zorgvoorzieningen).
F-L Patients do not have to travel to see the specialist physically
Hoefde u niet meer naar de specialist af te reizen doordat er gebruik werd gemaakt van teleconsultatie?
nee, helemaal niet een beetje grotendeels ja, helemaal
F Patients do not have to see the healthcare provider physically
Hoefde u de specialist niet meer persoonlijk te zien doordat er gebruik werd gemaakt van teleconsultatie?
nee, helemaal niet een beetje grotendeels ja, helemaal
L Teleconsultation provides a reduction of travel time
Was reizen overbodig, doordat er gebruik werd gemaakt van teleconsultatie?
nee, helemaal niet een beetje grotendeels ja, helemaal
Table 14 From quality aspect to question cluster: communication and information
Communicatie en informatie
Informatie ( schriftelijk, loket, instructie, zorg/leefplan (EPD/ ECD), voorlichting, procedures kwaliteitsbeleid)
F Patients have doubts about if the pictures were of sufficient quality
Werd u voldoende geïnformeerd of uw foto's, spirogrammen (longfunctieonderzoek) of ecg’s (hartfilmpje) van voldoende kwaliteit waren om door de specialist op afstand beoordeeld te kunnen worden?
nee, helemaal niet een beetje grotendeels ja, helemaal
54
Communicatie (mogelijkheid contact te leggen, begrijpelijkheid, uitleg, op maat gesneden).
F Healthcare provider provided information about the deductible costs (eigen risico)
Werd u door uw huisarts en/of assistente ingelicht over de kosten (eigen risico belast) van het teleconsult?
nee, helemaal niet een beetje grotendeels ja, helemaal
L The healthcare provider was informative
Werd u door uw huisarts en/of assistente voldoende geïnformeerd over uw aandoening (wat uw symptomen veroorzaakte)?
nee, helemaal niet een beetje grotendeels ja, helemaal
L All questions asked by the patients were addressed properly
Werden alle vragen die u aan de huisarts en/of assistente stelde naar wens beantwoord?
nee, helemaal niet een beetje grotendeels ja, helemaal
L Explanations given by the healthcare provider were clear
Was de uitleg die uw huisarts en of assistente over uw aandoening gaf voor u te begrijpen?
nee, helemaal niet een beetje grotendeels ja, helemaal
F The communication between healthcare providers was fast
Kreeg u snel bericht over de uitkomst van het overleg tussen uw huisarts en de betrokken specialist?
nee, helemaal niet een beetje grotendeels ja, helemaal
Table 15 From quality aspect to question cluster: interpersonal conduct
Bejegening
Aandacht F Patient missed the face- to- face contact with healthcare provider
Miste u het persoonlijke contact (face-to-face) met de specialist?
nee, helemaal niet een beetje grotendeels ja, helemaal
Tijd
Belangstelling
Serieus nemen.
55
Table 16 From quality aspect to question cluster: patient management role
Regierol klant
Autonomie (vragen stellen, meebeslissen, privacy, participatiemogelijkheden).
L The healthcare provider provided sufficient privacy
Werd uw privacy voldoende gewaarborgd door uw huisarts en/of assistente?
nee, helemaal niet een beetje grotendeels ja, helemaal
L Patients are less reliant on others for transport to the healthcare provider
Zou u afhankelijk zijn geweest van anderen voor vervoer naar het ziekenhuis wanneer u de specialist persoonlijk had moeten bezoeken?
ja nee
Zo ja: Vond u, dat u nu minder afhankelijk was, omdat een persoonlijk bezoek aan het ziekenhuis achterwege kon blijven?
nee, helemaal niet een beetje grotendeels ja, helemaal
F The communication of results to the patient depends on the availability of the GP
Was de huisarts bereikbaar om de uitslag van het overleg met de specialist te bespreken(diagnose/behandelplan)?
nee, helemaal niet een beetje grotendeels ja, helemaal
F Patients missed being direct involved in the teleconsult between the GP and the specialist.
Was u bezorgd dat er eventueel miscommunicatie kon ontstaan tussen de huisarts en specialist, omdat u verder niet meer betrokken werd in het overleg?
nee, helemaal niet een beetje grotendeels ja, helemaal
Versterking ( welbevinden, veiligheid, vertrouwen, geestelijk welzijn).
L Teleconsultation diminished the level of trust in the information provided
Had u vertrouwen in de informatie die uw huisarts en/of assistente aan u verstrekte ?
nee, helemaal niet een beetje grotendeels ja, helemaal
L Patients felt more at ease discussing issues
Voelde u zich op uw gemak om alle zaken te bespreken tijdens het teleconsult?
nee, helemaal niet een beetje grotendeels ja, helemaal
56
Table 17 From quality aspect to question cluster: competence
Deskundigheid
Professionaliteit
Bijscholing
Veiligheid F Safety increase through additional check from specialist
Voelde u zich veiliger omdat de specialist met de huisarts meekeek bij het stellen van uw diagnose of het bepalen van uw behandeling?
nee, helemaal niet een beetje grotendeels ja, helemaal
F Teleconsultation provides additional support to the GP’s
Kreeg u het gevoel dat teleconsultatie de assistente/huisarts extra ondersteuning bood bij het diagnosticeren van uw klacht en/of behandeling?
nee, helemaal niet een beetje grotendeels ja, helemaal
organisatie van de zorg. L Healthcare providers or patients experienced technical difficulties during teleconsultation
Traden er tijdens het teleconsult technische problemen op?
nee, helemaal niet een beetje grotendeels ja, helemaal
Table 18 From quality aspect to question cluster: organization of care
Organisatie van de zorg
Wachttijden F-L There is a smaller queue at the GP’s office than to go to the specialist physically
Heeft u wachttijd voor een (eerste) afspraak bij de specialist kunnen vermijden doordat er gebruik werd gemaakt van teleconsultatie bij de huisarts?
nee, helemaal niet een beetje grotendeels ja, helemaal
F-L Teleconsultation saves time for the patients
Heeft het teleconsult u tijd bespaard?
nee, helemaal niet een beetje grotendeels ja, helemaal
F-L Patient avoids waiting in clinical waiting rooms
Heeft u wachttijden in de wachtkamer van het ziekenhuis kunnen vermijden doordat er gebruik werd gemaakt van teleconsultatie?
nee, helemaal niet een beetje grotendeels ja, helemaal
57
snelheid zorgproces L There is less time devoted to the appointment of the consult
Was het consult/de afspraak met uw huisarts korter, doordat er gebruik werd gemaakt van teleconsultatie?
nee, helemaal niet een beetje grotendeels ja, helemaal
F Patients were helped quickly by the specialist through the use of teleconsultation
Werd u sneller geholpen door de specialist, doordat er gebruik werd gemaakt van teleconsultatie?
nee, helemaal niet een beetje grotendeels ja, helemaal
F The assessment of the teleconsultation was fast
Was uw uitslag sneller bekend doordat uw huisarts (en specialist) gebruik maakten van teleconsultatie?
nee, helemaal niet een beetje grotendeels ja, helemaal
F Teleconsultation result from the specialist took too long
Duurde het lang voordat u de resultaten van het teleconsult kreeg?
nee, helemaal niet een beetje grotendeels ja, helemaal
Heeft u binnen 2 werkdagen de uitslag van het teleconsult ontvangen?
L Convenient for minor problems Vindt u dat teleconsultatie gemakkelijker was voor het diagnosticeren van uw
problemen?
nee, helemaal niet een beetje grotendeels ja, helemaal
Gebruiksvriendelijke omgeving
Hygiëne.
Table 19 From quality aspect to question cluster: continuity of care
Continuïteit van zorg
Ketenzorg
Afstemming
Doorverwijzen F The referral to the specialist after the use of teleconsultation was rapid
Werd u na het teleconsult met spoed doorverwezen naar de specialist?
Zo ja, kon u na de doorverwijzing sneller bij de specialist terecht?
nee, helemaal niet een beetje grotendeels ja, helemaal
ja nee
ja nee
58
samenwerking tussen zorgverleners
F Teleconsultation provides short lines of communication between healthcare providers
Vond u dat teleconsultatie de communicatie tussen de specialist en de huisarts vergemakkelijkte?
nee, helemaal niet een beetje grotendeels ja, helemaal
Table 20 From quality aspect to question cluster: effective and safe care
Effectieve en veilige zorg
Resultaat van zorgproces (behandeling)
F Patient was not satisfied with the outcome of the teleconsult
Was u tevreden over de uitkomst (diagnose/behandeling) van het teleconsult?
nee, helemaal niet een beetje grotendeels ja, helemaal
F There is a chance that patients would still need to be seen physically by the specialist
Werd er door de huisarts en/of assistente aan u uitgelegd dat u mogelijk alsnog doorverwezen kon worden naar de specialist?
nee, helemaal niet een beetje grotendeels ja, helemaal
nazorg die bijdraagt aan een betere gezondheid en/of kwaliteit van leven.
Table 21 From quality aspect to question cluster: costs and compensation.
Kosten/ vergoedingen
Inzicht in kosten F Patients have clear insight in consequential costs of teleconsultation
Was u achteraf bezorgd over de kosten van het teleconsult die u zelf moest voldoen?
nee, helemaal niet een beetje grotendeels ja, helemaal
Vergelijkingsmogelijkheid
Mogelijkheid te kunnen kiezen.
Kosten besparing F-L Patients experience cost saving through use of teleconsultation
Heeft u het idee dat u kosten bespaard heeft doordat een onnodig bezoek aan de specialist is voorkomen door het teleconsult?
nee, helemaal niet een beetje grotendeels ja, helemaal
59
L Patients have less travel expenses Heeft u minder reiskosten gemaakt, doordat er gebruik werd gemaakt van teleconsultatie?
nee, helemaal niet een beetje grotendeels ja, helemaal
L Patients have less accommodation costs
Als u het ziekenhuis persoonlijk had moeten bezoeken, had u dan tijdelijke accommodatie in de buurt moeten regelen?
Zo ja, Kon u deze accommodatie kosten besparen, doordat er gebruik werd gemaakt van teleconsultatie?
nee, helemaal niet een beetje grotendeels ja, helemaal
L Patients loses less time off work hours
Was u minder werktijd kwijt, doordat er gebruik werd gemaakt van teleconsultatie?
nee, helemaal niet een beetje grotendeels ja, helemaal
Abbreviations
F = Focus groups
L= Literature study
F-L = Focus group and Literature study
ja nee
60
Appendix F: Origin of questions and the changed made by the
Workgroup Table 22 Questions of the final concept questionnaire and the origin of the questions
# Question Origin Workgroup Introductie
1 Bent u in de afgelopen 12 maanden in aanraking geweest met teleconsultatie bij de huisarts?
CQ-index Required
2 Met welk soort teleconsultatie bent u in aanraking geweest?
Answers were adapted
Bereikbaarheid/ toegankelijkheid
3 Hielp uw huisarts en/of assistente u binnen een kwartier na de afgesproken tijd?
CQ-index Optional Removed
4 Hielp uw huisarts en/of assistente u binnen een kwartier?
CQ-index Optional Removed
5 Was het een probleem (geweest) om uw huisarts overdag tussen 9.00 en 17.00 uur telefonisch te bereiken?
CQ-index Optional Removed
6 Kon u een afspraak inplannen op een tijdstip dat voor u goed uitkwam, doordat er gebruik werd gemaakt van teleconsultatie?
Literature Removed
7 Hoefde u niet meer naar de specialist af te reizen doordat er gebruik werd gemaakt van teleconsultatie?
Focus group + Literature
Removed
8 Hoefde u de specialist niet meer persoonlijk te zien doordat er gebruik werd gemaakt van teleconsultatie?
Focus group
9 Was reizen overbodig, doordat er gebruik werd gemaakt van teleconsultatie?
Literature
Communicatie en informatie
10 Legde uw huisarts u dingen op een begrijpelijke manier uit?
CQ-index Required
11 Gaf uw huisarts u tegenstrijdige informatie? CQ-index Required
12 Werd u voldoende geïnformeerd of uw foto's, spirogrammen (longfunctieonderzoek) of ecg’s (hartfilmpje) van voldoende kwaliteit waren om door de specialist op afstand beoordeeld te kunnen worden?
Focus group Removed
13 Werd u door uw huisarts en/of assistente ingelicht over de kosten (eigen risico belast) van het teleconsult?
Focus group
61
14 Werd u door uw huisarts en/of assistente voldoende geïnformeerd over uw aandoening (wat uw symptomen veroorzaakte)?
Literature
15 Werden alle vragen die u aan de huisarts en/of assistente stelde naar wens beantwoord?
Literature Question was adapted
16 Was de uitleg die uw huisarts en of assistente over uw aandoening gaf voor u te begrijpen?
Literature Removed
17 Kreeg u snel bericht over de uitkomst van het overleg tussen uw huisarts en de betrokken specialist?
Focus group Replaced question 40
Bejegening
18 Luisterde uw huisarts aandachtig naar u? CQ-index Required
19 Had uw huisarts genoeg tijd voor u? CQ-index Required
20 Nam uw huisarts u serieus? CQ-index Required
21 Was uw huisarts beleefd tegen u? CQ-index Optional Removed
22 Miste u het persoonlijke contact (face-to-face) met de specialist?
Focus group
Zelfbeschikking/Autonomie
23 Kon u meebeslissen over uw behandelplan? CQ-index Optional Removed
24 Werd uw privacy voldoende gewaarborgd door uw huisarts en/of assistente?
Literature Question was adapted
25 Zou u afhankelijk zijn geweest van anderen voor vervoer naar het ziekenhuis wanneer u de specialist persoonlijk had moeten bezoeken?
Literature
26 Vond u, dat u nu minder afhankelijk was, omdat een persoonlijk bezoek aan het ziekenhuis achterwege kon blijven?
Literature Question was adapted
27 Was de huisarts bereikbaar om de uitslag van het overleg met de specialist te bespreken (diagnose/behandelplan)?
Focus group
28 Was u bezorgd dat er eventueel miscommunicatie kon ontstaan tussen de huisarts en specialist, omdat u verder niet meer betrokken werd in het overleg?
Focus group
29 Had u vertrouwen in de informatie die uw huisarts en/of assistente aan u verstrekte?
Literature
30 Voelde u zich op uw gemak om alle zaken te bespreken tijdens het teleconsult?
Literature
Deskundigheid
31 Voelde u zich veiliger omdat de specialist met de huisarts meekeek bij het stellen van uw diagnose of het bepalen van uw behandeling?
Focus group Question was adapted
32 Kreeg u het gevoel dat teleconsultatie de assistente/huisarts extra ondersteuning bood bij het diagnosticeren van uw klacht en/of behandeling?
Focus group
62
33 Traden er tijdens het teleconsult technische problemen op?
Literature
Organisatie van de zorg
34 Heeft u wachttijd voor een (eerste) afspraak bij de specialist kunnen vermijden doordat er gebruik werd gemaakt van teleconsultatie bij de huisarts?
Focus group + Literature
35 Heeft het teleconsult u tijd bespaard? Focus group + Literature
36 Heeft u wachttijden in de wachtkamer van het ziekenhuis kunnen vermijden doordat er gebruik werd gemaakt van teleconsultatie?
Focus group + Literature
37 Was het consult/de afspraak met uw huisarts korter, doordat er gebruik werd gemaakt van teleconsultatie?
Literature Removed
38 Werd u sneller geholpen door de specialist, doordat er gebruik werd gemaakt van teleconsultatie?
Focus group Placed beneath question 34 + Question was adapted
39 Was uw uitslag sneller bekend doordat uw huisarts (en specialist) gebruik maakten van teleconsultatie?
Focus group Removed
40 Duurde het lang voordat u de resultaten van het teleconsult kreeg?
Focus group Replaced by question 17
41 Heeft u binnen 2 werkdagen de uitslag van het teleconsult ontvangen?
Focus group Removed
42 Vindt u dat teleconsultatie gemakkelijker was voor het diagnosticeren van uw problemen?
Literature
Continuïteit van de zorg
43 Werd u na het teleconsult met spoed doorverwezen naar de specialist?
Focus group
44 Kon u na de doorverwijzing sneller bij de specialist terecht?
Focus group
45 Vond u dat teleconsultatie de communicatie tussen de specialist en de huisarts vergemakkelijkte?
Focus group
Effectiviteit en veilige zorg
46 Was u tevreden over de uitkomst (diagnose/behandeling) van het teleconsult?
Focus group
47 Werd er door de huisarts en/of assistente aan u uitgelegd dat u mogelijk alsnog doorverwezen kon worden naar de specialist?
Focus group
Kosten en vergoedingen
48 Was u achteraf bezorgd over de kosten van het teleconsult die u zelf moest voldoen?
Focus group Question was adapted
63
49 Heeft u het idee dat u kosten bespaard heeft doordat een onnodig bezoek aan de specialist is voorkomen door het teleconsult?
Focus group + Literature
50 Heeft u minder reiskosten gemaakt, doordat er gebruik werd gemaakt van teleconsultatie?
Literature
51 Als u het ziekenhuis persoonlijk had moeten bezoeken, had u dan tijdelijke accommodatie in de buurt moeten regelen?
Literature
52 Kon u deze accommodatiekosten besparen, doordat er gebruik werd gemaakt van teleconsultatie?
Literature
53 Was u minder werktijd kwijt, doordat er gebruik werd gemaakt van teleconsultatie?
Literature
Algemeen oordeel
54 Welk cijfer geeft u de teleconsultatie? Een 0 betekent: heel erg slecht. Een 10 betekent: uitstekend.
CQ-index Required
55 Zou u teleconsultatie bij uw vrienden en familie aanbevelen?
CQ-index Required Question was removed
OF (gebruik 55 of 56) OR
56 Zou u teleconsultatie bij andere patiënten aanbevelen?
CQ-index Required
Over uzelf CQ index
57 Hoe zou u over het algemeen uw gezondheid noemen?
CQ index
58 Wat is uw leeftijd? CQ index
59 Bent u een man of een vrouw? CQ index
60 Wat is uw hoogst voltooide opleiding? (een opleiding afgerond met diploma of voldoende getuigschrift)
CQ index
61 Wat is het geboorteland van uzelf? CQ index
62 Wat is het geboorteland van uw vader? CQ index
63 Wat is het geboorteland van uw moeder? CQ index
64 In welke taal praat u thuis het meeste? CQ index
65 Heeft iemand u geholpen om deze vragenlijst in te vullen?
CQ index
66 Hoe heeft die persoon u geholpen? U mag meer dan één vakje aankruisen.
CQ index
67 We willen de vragenlijst blijven verbeteren. We horen dan ook graag wat u van de vragenlijst vindt. Mist u iets in deze vragenlijst? Of heeft u nog opmerkingen of tips? Dan kunt u dat hieronder opschrijven.
CQ index
64
Appendix G: Concept Questionnaire
INTRODUCTIE Deze vragenlijst gaat over uw ervaringen met de zorg. Wij stellen het zeer op prijs als u deze
vragenlijst wilt invullen. Het invullen van deze vragenlijst duurt ongeveer … minuten.
De vragenlijst wordt anoniem en vertrouwelijk gebruikt. Dit betekent dat niemand weet welke
antwoorden u heeft gegeven. Ook worden uw gegevens niet met anderen gedeeld. Verder ziet u
een nummer op de voorkant van deze vragenlijst staan. Dit nummer wordt ALLEEN gebruikt om
te kijken of de vragenlijst al is teruggestuurd. U krijgt dan géén herinnering meer thuis gestuurd.
Deelname aan dit onderzoek is geheel vrijwillig. Het wel of niet meedoen aan dit onderzoek
heeft géén gevolgen voor de zorg die u krijgt.
Als u niet wilt meedoen aan dit onderzoek, zet dan een kruisje in dit vakje
Wilt u daarna deze bladzijde terugsturen in de antwoordenvelop?
Heeft u vragen, dan kunt u bellen met XXXX, telefoonnummer XXXX.
Of u kunt een email sturen naar: XXXX
INVULINSTRUCTIE
Wanneer in deze vragenlijst wordt gesproken over zorgaanbieder, wordt hieronder ook
huisartsenpraktijk, ziekenhuis, zorgorganisatie, zorgcentrum, zelfstandig werkende verzorgende
e.d. verstaan.
Het is belangrijk dat de vragen alleen worden ingevuld door de persoon die in de
begeleidende brief wordt genoemd. Het is niet de bedoeling dat u de vragenlijst aan
iemand anders doorgeeft.
De meeste vragen kunt u beantwoorden door een kruisje te zetten in het vakje van uw
keuze. Graag het kruisje altijd binnen het vakje zetten, anders wordt uw antwoord gemist
bij de automatische verwerking.
Er zijn ook enkele open vragen. Wilt u deze met blokletters beantwoorden in het
aangewezen vakje?
Bij sommige vragen zijn meerdere antwoorden mogelijk. Bij deze vragen staat
aangegeven dat u meerdere vakjes kunt aankruisen.
Soms wordt u gevraagd om enkele vragen in deze vragenlijst over te slaan. U ziet dan
een pijltje met een opmerking. Deze opmerking geeft aan welke vraag u daarna kunt
beantwoorden. Dit ziet er als volgt uit:
Sla de volgende vraag over
Ik wil niet meedoen
met dit onderzoek
Nee
65
Introductie
1. Bent u in de afgelopen 12 maanden in
aanraking geweest met teleconsultatie
bij de huisarts?
Deze vragenlijst is niet op
u van toepassing. Wilt u zo vriendelijk zijn
hem terug te sturen in de bijgevoegde
antwoordenvelop?
2. Met welk soort teleconsultatie bent u in
aanraking geweest?
Telecardiologie bv ECG
Telepulmonologie bv
longfunctieonderzoek
Teledermatologie foto's van
huidafwijking
ik weet niet welke
Bereikbaarheid/ toegankelijkheid
De vragen 3 t/m 5 in deze vragenlijst gaan
specifiek over uw ervaringen met de
bereikbaarheid of toegankelijkheid van de zorg
in de afgelopen maanden.
3. Hoefde u de specialist niet meer
persoonlijk te zien doordat er gebruik
werd gemaakt van teleconsultatie?
4. Was reizen overbodig, doordat er
gebruik werd gemaakt van
teleconsultatie?
Communicatie en informatie De vragen 6 t/m 10 gaan over uw ervaringen met
de communicatie met de zorgverlener en de
informatie die u van de zorgverlener heeft
ontvangen in de afgelopen maanden.
5. Legde uw huisarts u dingen op een
begrijpelijke manier uit?
6. Gaf uw huisarts u tegenstrijdige
informatie?
7. Werd u door uw huisarts en/of
assistente ingelicht over de kosten (eigen
risico belast) van het teleconsult?
8. Werd u door uw huisarts en/of
assistente voldoende geïnformeerd over
uw aandoening (wat uw symptomen
veroorzaakte)?
9. Werden alle vragen die u aan de
huisarts en/of assistente stelde naar
tevredenheid beantwoord?
Nee
Ja
nee, helemaal niet
een beetje
grotendeels
ja, helemaal
nee, helemaal niet
een beetje
grotendeels
ja, helemaal
Nooit
Soms
Meestal
Altijd
Nooit
Soms
Meestal
Altijd
nee, helemaal niet
een beetje
grotendeels
ja, helemaal
nee, helemaal niet
een beetje
grotendeels
ja, helemaal
66
Bejegening De vragen 11 t/m 15 gaan over de bejegening door
de zorgverleners in de afgelopen maanden.
10. Luisterde uw huisarts aandachtig naar
u?
11. Had uw huisarts genoeg tijd voor u?
12. Nam uw huisarts u serieus?
13. Miste u het persoonlijke contact (face-
to-face) met de specialist?
Zelfbeschikking/Autonomie
De vragen 16 t/m 23 gaan over de
mogelijkheid mee te beslissen over uw
behandeling/verzorging in de afgelopen
maanden.
14. Vond u dat uw privacy voldoende
gewaarborgd door uw huisarts en/of
assistente?
15. Zou u afhankelijk zijn geweest van
anderen voor vervoer naar het
ziekenhuis wanneer u de specialist
persoonlijk had moeten bezoeken?
Sla de volgende vraag over
16. Voelde u zich minder afhankelijk,
omdat een persoonlijk bezoek aan het
ziekenhuis achterwege kon blijven?
17. Was de huisarts bereikbaar om de
uitslag van het overleg met de specialist
te bespreken (diagnose/behandelplan)?
18. Was u bezorgd dat er eventueel
miscommunicatie kon ontstaan tussen
de huisarts en specialist, omdat u verder
niet meer betrokken werd in het
overleg?
nee, helemaal niet
een beetje
grotendeels
ja, helemaal
Nooit
Soms
Meestal
Altijd
Nooit
Soms
Meestal
Altijd
Nooit
Soms
Meestal
Altijd
nee, helemaal niet
een beetje
grotendeels
ja, helemaal
nee, helemaal niet
een beetje
grotendeels
ja, helemaal
ja
nee
nee, helemaal niet
een beetje
grotendeels
ja, helemaal
nee, helemaal niet
een beetje
grotendeels
ja, helemaal
niet van toepassing
67
19. Had u vertrouwen in de informatie die
uw huisarts en/of assistente aan u
verstrekte ?
20. Voelde u zich op uw gemak om alle
zaken te bespreken tijdens het
teleconsult?
Deskundigheid
De vragen 24 t/m 26 gaan over de
deskundigheid van uw zorgverleners
21. Gaf het u een gevoel van extra
zekerheid, omdat de specialist met de
huisarts meekeek bij het stellen van uw
diagnose of het bepalen van uw
behandeling?
22. Kreeg u het gevoel dat teleconsultatie de
assistente/huisarts extra ondersteuning
bood bij het diagnosticeren van uw
klacht en/of behandeling?
23. Traden er tijdens het teleconsult
technische problemen op?
Organisatie van de zorg
De vragen 27 t/m 32 gaan over de organisatie
van de zorg.
24. Heeft u wachttijd voor een (eerste)
afspraak bij de specialist kunnen
vermijden doordat er gebruik werd
gemaakt van teleconsultatie bij de
huisarts?
25. Ontving u sneller advies van de
specialist, doordat er gebruik werd
gemaakt van teleconsultatie?
26. Heeft het teleconsult u tijd bespaard?
nee, helemaal niet
een beetje
grotendeels
ja, helemaal
nee, helemaal niet
een beetje
grotendeels
ja, helemaal
nee, helemaal niet
een beetje
grotendeels
ja, helemaal
nee, helemaal niet
een beetje
grotendeels
ja, helemaal
nee, helemaal niet
een beetje
grotendeels
ja, helemaal
nee, helemaal niet
een beetje
grotendeels
ja, helemaal
nee, helemaal niet
een beetje
grotendeels
ja, helemaal
nee, helemaal niet
een beetje
grotendeels
ja, helemaal
nee, helemaal niet
een beetje
grotendeels
68
27. Heeft u wachttijden in de wachtkamer
van het ziekenhuis kunnen vermijden
doordat er gebruik werd gemaakt van
teleconsultatie?
28. Kreeg u snel bericht over de uitkomst
van het overleg tussen uw huisarts en de
betrokken specialist?
29. Vindt u dat teleconsultatie
gemakkelijker was voor het
diagnosticeren van uw problemen?
Continuïteit van de zorg
De vragen 33 t/m 35 gaan over de continuïteit
van de zorg.
30. Werd u na het teleconsult met spoed
doorverwezen naar de specialist?
Sla de volgende vraag over
31. Kon u na de doorverwijzing sneller bij
de specialist terecht?
32. Vond u dat teleconsultatie de
communicatie tussen de specialist en de
huisarts vergemakkelijkte?
Effectiviteit en veilige zorg
De vragen 36 t/m 37 gaan over de effectiviteit
en veiligheid van de zorg.
33. Was u tevreden over de uitkomst
(diagnose/behandeling) van het
teleconsult?
34. Werd er door de huisarts en/of
assistente aan u uitgelegd dat u mogelijk
alsnog doorverwezen kon worden naar
de specialist?
Kosten en vergoedingen
De vragen 38 t/m 43 gaan over de kosten en
vergoedingen van de zorg.
35. Waren de kosten achteraf wat u ervan
verwacht had?
ja, helemaal
nee, helemaal niet
een beetje
grotendeels
ja, helemaal
nee, helemaal niet
een beetje
grotendeels
ja, helemaal
nee, helemaal niet
een beetje
grotendeels
ja, helemaal
ja
nee
nee, helemaal niet
een beetje
grotendeels
ja, helemaal
nee, helemaal niet
een beetje
grotendeels
ja, helemaal
nee, helemaal niet
een beetje
grotendeels
ja, helemaal
nee, helemaal niet
een beetje
grotendeels
ja, helemaal
nee, helemaal niet
een beetje
69
36. Heeft u het idee dat u kosten bespaard
heeft doordat een onnodig bezoek aan
de specialist is voorkomen door het
teleconsult?
37. Heeft u minder reiskosten gemaakt,
doordat er gebruik werd gemaakt van
teleconsultatie?
38. Als u het ziekenhuis persoonlijk had
moeten bezoeken, had u dan tijdelijke
accommodatie in de buurt moeten
regelen?
Sla de volgende vraag over
39. Kon u deze accommodatiekosten
besparen, doordat er gebruik werd
gemaakt van teleconsultatie?
40. Was u minder werktijd kwijt, doordat
er gebruik werd gemaakt van
teleconsultatie?
Algemeen oordeel
De vragen 44 t/m 56 gaan over uw
totaaloordeel van de teleconsultatie.
41. Welk cijfer geeft u de teleconsultatie?
Een 0 betekent: heel erg slecht. Een 10
betekent: uitstekend.
42. Zou u teleconsultatie bij andere
patiënten aanbevelen?
Over uzelf
De volgende vragen gaan over uzelf (degene
die de vragenlijst invult). Deze
achtergrondkenmerken zijn belangrijk om
meer inzicht te krijgen in de kwaliteit van de
grotendeels
ja, helemaal
nee, helemaal niet
een beetje
grotendeels
ja, helemaal
nee, helemaal niet
een beetje
grotendeels
ja, helemaal
ja
nee
nee, helemaal niet
een beetje
grotendeels
ja, helemaal
nee, helemaal niet
een beetje
grotendeels
ja, helemaal
0 heel erg slecht
1
2
3
4
5
6
7
8
9
10 uitstekend
Nooit
Soms
Meestal
Altijd
70
zorg rondom zorg voor verschillende groepen
mensen.
43. Hoe zou u over het algemeen uw
gezondheid noemen?
44. Wat is uw leeftijd?
45. Bent u een man of een vrouw?
46. Wat is uw hoogst voltooide opleiding?
(een opleiding afgerond met diploma of
voldoende getuigschrift)
(a.u.b. in blokletters)
47. Wat is het geboorteland van uzelf?
(a.u.b. in blokletters)
48. Wat is het geboorteland van uw vader?
Uitstekend
Zeer goed
Goed
Matig
Slecht
18 t/m 24 jaar
25 t/m 34 jaar
35 t/m 44 jaar
45 t/m 54 jaar
55 t/m 64 jaar
65 t/m 74 jaar
75 jaar of ouder
Man
Vrouw
Geen opleiding (lager onderwijs: niet
afgemaakt)
Lager onderwijs (basisschool, speciaal
basisonderwijs)
Lager of voorbereidend beroepsonderwijs
(zoals LTS, LEAO, LHNO, VMBO)
Middelbaar algemeen voortgezet onderwijs
(zoals MAVO, (M)ULO, MBO-kort,
VMBO-t)
Middelbaar beroepsonderwijs en
beroepsbegeleidend onderwijs (zoals
MBO-lang, MTS, MEAO, BOL, BBL,
INAS)
Hoger algemeen en voorbereidend
wetenschappelijk onderwijs (zoals
HAVO, VWO, Atheneum, Gymnasium,
HBS, MMS)
Hoger beroepsonderwijs (zoals HBO,
HTS, HEAO, HBO-V, kandidaats
wetenschappelijk onderwijs)
Wetenschappelijk onderwijs (universiteit)
Anders, namelijk:
Nederland
Indonesië/voormalig Nederlands-Indië
Suriname
Marokko
Turkije
Duitsland
(voormalig) Nederlandse Antillen
Aruba
Anders, namelijk:
Nederland
Indonesië/voormalig Nederlands-Indië
Suriname
Marokko
Turkije
Duitsland
(voormalig) Nederlandse Antillen
Aruba
71
(a.u.b. in blokletters)
49. Wat is het geboorteland van uw
moeder?
(a.u.b. in blokletters)
50. In welke taal praat u thuis het meeste?
(a.u.b. in blokletters)
51. Heeft iemand u geholpen om deze
vragenlijst in te vullen?
Sla de volgende vraag over
52. Hoe heeft die persoon u geholpen? U
mag meer dan één vakje aankruisen.
(a.u.b. in blokletters)
53. We willen de vragenlijst blijven
verbeteren. We horen dan ook graag
wat u van de vragenlijst vindt. Mist u
iets in deze vragenlijst? Of heeft u nog
opmerkingen of tips? Dan kunt u dat
hieronder opschrijven.
HARTELIJK BEDANKT VOOR
HET INVULLEN VAN DE
VRAGENLIJST
Anders, namelijk:
Nederland
Indonesië/voormalig Nederlands-Indië
Suriname
Marokko
Turks
Duitsland
(voormalig) Nederlandse Antillen
Aruba
Anders, namelijk:
Nederlands
Fries
Nederlands dialect
Indonesisch
Sranan (Surinaams)
Marokkaans-Arabisch
Turkije
Duits
Papiaments (Nederlandse Antillen)
Anders, namelijk:
Nee
-> Ja
Heeft de vragen voorgelezen
Heeft de vragen in mijn plaats beantwoord
Heeft mijn antwoorden opgeschreven
Heeft de vragen in mijn taal vertaald
Heeft op een andere manier geholpen,
namelijk:
