Eliciting preferences of the community for out of hours care provided by general practitioners: a stated preference discrete choice experiment

Social Science & Medicine 56 (2003) 803–814

Eliciting preferences of the community for out of hours careprovided by general practitioners: a stated preference discrete

choice experiment

Anthony Scotta,*, M. Stuart Watsonb, Sue Rossc

aHealth Economics Research Unit, University of Aberdeen, Foresterhill, Aberdeen AB25 9ZD, UKbDepartment of Public Health, University of Aberdeen, UKcHealth Services Research Unit, University of Aberdeen, UK

Abstract

Access to primary care services is a major issue as new models of delivering primary care continue develop in many

countries. Major changes to out of hours care provided by general practitioners (GPs) were made in the UK in 1995.

These were designed in response to low morale and job dissatisfaction of GPs, rather than in response to patients’

preferences. The aim of this study is to elicit the preferences of patients and the community for different models of GP

out of hours care. A questionnaire was sent to parents of children in Aberdeen and Glasgow in Scotland who had

received a home visit or attended a primary care emergency centre, or were registered with a GP. The questionnaire used

a discrete choice experiment that asked parents to imagine their child had respiratory symptoms. Parents were then

asked to choose between a series of pairs of scenarios, with each scenario describing a different model of out of hours

care. Each model varied by waiting time, who was seen, location, and whether the doctor listened. The response rate

was 68% (3893/5718). The most important attribute was whether the doctor seemed to listen, suggesting that policies

aimed at improving doctor–patient communication will lead to the largest improvements in utility. The most preferred

location of care was a hospital accident and emergency department. This suggests that new models of primary care

emergency centres may not reduce the demand for accident and emergency visits from this group of patients in urban

areas. Preferences also differed across sub-groups of patients. Those who had never used out of hours care before had

stronger preferences for waiting time and the doctor listening, suggesting higher expectations of non-users. Further

research is required into the demand for out of hours care as new models of care become established. r 2002 Elsevier

Science Ltd. All rights reserved.

Keywords: Out of hours; General practice; Discrete choice experiments; Patients’ preferences; Scotland; UK

Introduction

There have been many changes to the delivery of

primary care in many countries, as health care reforms

introduce and experiment with new models of care. The

provision of out of hours (or after hours) care by

primary care physicians is a crucial element of primary

health care in many countries. Concerns about access to

primary care, increasing public expectations, and the

effects on stress and family life for primary care

physicians, have led to reform of out of hours primary

care in several countries. A common theme in these

reforms has been a degree of centralisation and

integration of services through system-wide telephone

triage (Vedsted & Olsen, 1999; Hansen & Munck, 1998)

and growth in co-operative models of out of hours care,

including the establishment of primary care out of hours

emergency centres (Department of Health, 2000; Scot-

tish Out of Hours Study Group, 2001). These changes

have reduced the on-call commitment of primary care

*Corresponding author. Tel.: +44-1224-553866; fax: +44-

1224-662994.

E-mail address: [email protected] (A. Scott).

0277-9536/03/$ - see front matter r 2002 Elsevier Science Ltd. All rights reserved.

PII: S 0 2 7 7 - 9 5 3 6 ( 0 2 ) 0 0 0 7 9 - 5

physicians, increased the proportion of telephone con-

sultations, and changed the location and provider of

health care. The effects of these major changes in service

provision have therefore been supply-led, rather than

demand-led. There is little evidence of the effects on

patients.

In the UK, the provision of out of hours care by

general practitioners (GPs) has changed radically since

1995. As a result of deepening concern felt by GPs about

increasing workload out of hours and the ability to

maintain their 24 hour contractual commitment, the

arrangements for providing care out of hours were

altered in 1995. Funding was made available to set up

GP co-operatives, where many practices formally

collaborate to provide out of hours care. This funding

paved the way for the set up of new primary care

emergency centres open during evenings, nights and

weekends, and staffed by GPs from member practices.

In many areas, this has led to reductions in the numbers

of home visits as more patients are asked to travel to the

emergency centre and more advice is given over the

telephone. The structure and organisation of co-

operatives vary, with some providing nurse triage,

patient and doctor transport, and being supported by

a formal management board and administrative struc-

ture. Others have a less formal structure, and are more

similar to rotas. These changes have influenced mainly

urban areas of the UK, with care in rural areas provided

along more traditional lines with GPs providing their

own cover for their own patients (Hallam and

Henthorne, 1999; Department of Health, 1998).

These arrangements were devised and implemented to

reduce stress and improve the morale of GPs, with little

recourse to the preferences of patients and the commu-

nity. The changes have had important effects on the

process of care, including location of care, waiting times,

and who is seen as well as potential effects on health

status.

Several patient satisfaction studies have been con-

ducted which have used a variety of instruments, and

asked about a variety of different models of care

(McKinley et al., 1997a, b; Salisbury, 1997; Bain,

Gerrard, Russell, Locke, & Baird, 1997). However,

satisfaction studies suffer from several known short-

comings. Although they ask about patients’ experience

of the care they have had, which is relevant, they do not

directly ask about preferences for alternative models of

care. It is difficult to determine the relative importance

of attributes from satisfaction studies, since dissatisfac-

tion (or satisfaction) with an attribute does not

necessarily indicate that it is the most important to

patients (Scott & Smith, 1994; Carr-Hill, 1992).

Satisfaction studies also ignore the notions of sacrifice

and opportunity cost: resources for out of hours care are

finite and choices need to be made about where they are

likely to have the best effect. To have more of one

characteristic means less of another. It is impossible for

patients to receive a home visit within 5min of their call,

from their own GP. It is therefore important to find out,

from the patients’ perspective, which attribute they

would most like to be improved, given that they cannot

have the best level of every attribute. In short, priorities

need to be set.

The aim of this study was to elicit the preferences of

users and non-users (i.e. the community) for different

models of out of hours care, and to examine the relative

importance of attributes of out of hours care. The study

uses a discrete choice experiment which has its origins in

mathematical psychology, market research, and eco-

nomics and has been developed as a method of

examining preferences for attributes or characteristics

of goods and services (Brunel University, 1993). Its

application in health care is growing (Ryan, 1999).

Method

A postal questionnaire was developed which pre-

sented respondents with a number of pairs of scenarios,

where each scenario described a particular model of out

of hours care. For each pair, respondents were asked to

choose which scenario they preferred (Fig. 1). The

attributes for the scenarios were chosen from the

existing literature and from face-to-face and postal

piloting of the questionnaire (Table 1). Several studies

have suggested that who the patient consults is

important, as is the time between initial contact and

actual consultation with a doctor (McKinley et al.,

1997a; Sawyer & Arber, 1982; Prudhoe, 1984; Bollam,

McCarthy, & Modell, 1988; Dixon & Williams, 1988;

Cragg, Campbell, & Roland, 1994).

Furthermore, the current policy context suggests that

the location of the out of hours consultation is

important. The doctor–patient relationship is also a

relevant factor and has been shown by many previous

patient satisfaction studies to be more important than

other aspects of care in general practice (Smith &

Armstrong, 1989; Calnan et al., 1994). The attribute

representing the quality of the doctor–patient relation-

ship was ‘whether the doctor seems to listen to what you

have to say’. This was chosen on the basis of the results

of a previous choice experiment examining the impor-

tance of different attributes of the GP consultation (Vick

& Scott, 1998). ‘Whether the doctor listens’ was found to

be the most important to patients. The scenarios

therefore described out of hours care in terms of the

location of the consultation, time between initial contact

and consultation, who was seen, and whether the doctor

seemed to listen (Table 1).

A vignette was used to place the hypothetical

scenarios in a recognisable and realistic setting. The

scenarios were therefore presented in the context of

A. Scott et al. / Social Science & Medicine 56 (2003) 803–814804

children with respiratory illness (Fig. 1). Children

represent a large proportion of consultations out of

hours, with respiratory disease contributing significantly

to these consultations (Cragg et al., 1994; Morrison,

Gilmour, & Sullivan, 1991). Parents of young children

have also been shown to be the most dissatisfied users of

the service in one of our study sites (Glasgow) (Wilson

et al., 2001).

The attributes and levels (Table 1) were organised into

scenarios using a factorial experimental design. From a

possible 48 scenarios, 16 were generated using a

fractional factorial design. This provides the minimum

number needed to maintain an orthogonal design. One

scenario was chosen to be constant across all pairs

(Scenario A), and the others compared with it (Fig. 1).

The constant scenario was chosen to represent the new

model of an emergency centre. The remaining 15

scenarios were then divided randomly across two

questionnaires, one with eight pairs and one with seven.

Each type of questionnaire was allocated randomly to

respondents.

Once scenarios are paired (whether using a constant

scenario or a random pairing), the orthogonality of the

original main effects design is not guaranteed. Ortho-

gonality was therefore checked by examining the

associations between attribute differences (i.e. the

difference between the levels of each pairwise choice)

in the 15 scenarios. Appendix A shows that associations

between attribute differences were low and not statisti-

cally significant.

• Imagine that during the night, your child is short of breath, wheezing and coughingand that you decide to call a doctor. You have several options about the care you receive. These differ according to who your child sees, where they are seen, the time it takes between making the telephone call and receiving treatment, and whether the doctor seems to listen to what you have to say.

• For each question below, you are asked to choose which type of consultation you would prefer for your child during the night (Consultation A or Consultation B).

1. Which consultation would you prefer? (please tick box below)

Consultation A Consultation B

Where your child is seen: Emergency centre run by GPs

Your home

Who your child sees: A GP who doesn’t work at your practice/health

centre

A GP who doesn’t work at your

practice/health centre

Time taken between thetelephone call and treatment being received:

60 minutes 20 minutes

Whether the doctor seems to listen to what you have to say:

The doctor seems tolisten

The doctor seems to listen

Prefer consultation A Prefer consultation B

(please tick one box)

Fig. 1. Example of a choice presented to patients.

Table 1

Attributes and levels used in the questionnaire

Attribute Levels of each attribute

Where your child is seen Emergency centre run by

GPs

Your home

A hospital accident and

emergency department

Who your child sees A GP from your

practice/health centre

A GP who doesn’t work

at your practice/health

centre

Time taken between the

telephone call and

treatment being received

20min

40min

60min

80min

Whether the doctor

seems to listen to what

you have to say

The doctor seems to

listen

The doctor does not

seem to listen

A. Scott et al. / Social Science & Medicine 56 (2003) 803–814 805

Face validity was assessed during piloting in a

respiratory clinic at Aberdeen Children’s Hospital, and

at the Glasgow and Aberdeen primary care emergency

centres. Internal consistency was examined by analysing

those pairs of scenarios where one scenario was

obviously dominant. The experimental design produced

three scenarios that were dominant, assuming a priori

that individuals prefer shorter waiting times, prefer a

home visit to a GP emergency centre visit, and prefer to

see a GP they know. One questionnaire had two

dominant scenarios and the other version had one. If a

respondent chose inconsistently (i.e. choose the non-

dominant scenario), then all of the respondent’s choices

were removed from the data set. Theoretical validity

was assessed using the signs of the coefficients to

test prior hypotheses about the nature of preferences

(e.g. shorter waiting time is preferred to longer waiting

time).

To examine the effect of respondent characteristics

and past experiences on the relative importance of

attributes, additional questions were asked for demo-

graphic details, past experiences of out of hours care and

wheezing illness, and about a range of access factors

such as the use of a car at night, and availability of

someone else to look after children. For example, it

might be expected that parents with younger or fewer

children are more likely to prefer shorter waiting times

or the doctor listening. Parents’ own characteristics may

also influence their preferences, such as gender and level

of education. It may also be expected that past

experiences of out of hours care might influence

preferences, indirectly testing hypotheses related to

stability of preferences and path-dependence.

The questionnaire was sent to parents of children in

Aberdeen and Glasgow, allowing comparison of two

communities with contrasting social structures (affluent

and one of the most deprived in the UK, respectively).

As a guide to relative deprivation using the Carstairs

deprivation index, around half of Greater Glasgow

Health Board’s population were classified in categories

six and seven (where seven is the highest category of

deprivation). This is compared with none of the

population in Grampian (including Aberdeen) being

classified in these categories (McLaren & Bain, 2000).

Each area had a new co-operative, Grampian doctors on

call service (GDOCS) based in Aberdeen, and Glasgow

emergency medical service (GEMS) in Glasgow.

From each area (and with ethical approval), three

random samples of children aged under 13 years old

were identified: those who had visited a primary care

emergency centre; those who had received a home visit;

and those from the lists of GPs (i.e. who may or may not

have used out of hours care before). Duplicates from the

samples of users and temporary residents were excluded

from this latter sample. Information about those who

had a centre visit or home visit was gathered from the

records of the emergency centres. Records of GDOCS

users were available for a 12-month period and those for

GEMS were available for a 6-month period. On the

basis of previous experience of using choice experiments

in the community, expected response rates and expected

sub-group analysis, we estimated sample sizes of 1800

non-users and 715 in each user group from each area

were required to obtain sufficient responses for analysis

(Pearmain, Swanson, Kroes, & Bradley, 1991). Remin-

ders were sent at 3 and 6 weeks if necessary.

Econometric model

Each model of out of hours care is represented by an

indirect utility function. With two models of out of

hours care (i and j), y�n is a latent variable representing

the difference in utility between them, with n individuals

making a choice. Since it is the choice that is observed

rather than the difference in utility, y�n is binary.

Therefore:

yn ¼ 1 if y�n > 0 and 0 else

and

y�n ¼ ðaþ bxi þ qsn þ einÞ � ðaþ bxj þ qsn þ ejnÞ; ð1Þ

where a; b and @ are coefficients, x are the attributes of

each model of care, s are socio-economic characteristics

reflecting influences on tastes and e is the random

component of utility accounting for the analyst’s

inability to accurately observe individual’s behaviour

(Manski, 1977). Further, assume that there are

taste variations such that the marginal utility of x

depends on s:

b ¼ pþ lsn: ð2Þ

This gives

y�n ¼ ðaþ pxi þ lsnxi þ qsn þ einÞ

� ðaþ pxj þ lsnxj þ qsn þ ejnÞ: ð3Þ

Because the discrete choice experiment presents each

respondent with several pairs of scenarios, multiple

observations from each respondent mean that errors are

not independent and so an error term mn capturing

random variation across respondents is included:

y�n ¼ ðaþ pxi þ lsnxi þ qsn þ ein þ mnÞ

� ðaþ pxj þ lsnxj þ qsn þ ejn þ mnÞ: ð4Þ

Taking differences for each pairwise choice (k), the

equation to be estimated becomes

y�kn ¼ pxk þ lsnxk þ ekn: ð5Þ

Terms common to both indirect utility functions drop

out of the model (i.e. a; qsn; and mn).

However, the inclusion of a constant term (a) anderror term across respondents (mn) can be used to test for

mis-specification due to unobservable attributes and


unobservable interaction terms between GPs’ socio-

economic characteristics and attributes. The constant

term can be interpreted as the difference in the average

utility of scenario i and j; caused by the use of a constantscenario, left/right bias or an omitted dummy variable

that is a function of other included attributes (Scott,

2000). The model to be estimated then becomes

y�kn ¼ aþ pxk þ lsnxk þ ekn þ mn: ð6Þ

This model was estimated using random effects probit

regression. A model with main effects was estimated

first, and used to calculate marginal rates of substitution

between each attribute and the time attribute. A full

model was estimated including main effects and inter-

action terms for which hypotheses existed. This was then

reduced to a more parsimonious model by excluding

variables, one at a time, with p-values >0.10.

Results

Excluding those questionnaires that were not deliv-

ered (586), the final response rate was 3893/5718

(68.1%). Two questionnaires were completed by chil-

dren and were excluded from analysis. Time to complete

the questionnaire ranged from 2 to 60minutes (mean

12minutes), and 1760/3757 (46.8%) of respondents

considered that the questionnaire was easy to answer.

Characteristics of the respondents and their last out of

hours consultation are presented in Tables 2 and 3.

For the regression analysis, the final sample size was

3326 individuals (24,789 observations), after the deletion

of observations where data on the dependent and

independent variables were missing (2391; 8%), and

inconsistent responses (2023; 7.5%). The main effects

model is shown in Table 4. r; measuring the correlationbetween observations of the same respondent, was

statistically significant, suggesting that a random effects

specification was appropriate and that there may be

unobservable interaction terms between attributes and

characteristics of respondents.

The constant term is statistically significant and

negative, suggesting a general preference for scenario

A. Scenario A represented the new co-operative model

with a primary care emergency centre, and Scenario B

represented other models of care. Although this suggests

there was a general preference for new models of care,

other explanations for the significant constant term

should not be ruled out (i.e. ‘right’ bias and omitted

attributes).

Face and content validity were established through

piloting, there was a low level of inconsistent responses

(8%), and the results were consistent with prior

hypotheses (e.g. shorter waiting times are preferred to

longer), confirming the techniques’ theoretical validity.

Using data from this study that has been reported

elsewhere, the instrument used was also found to be

reliable (San Miguel et al., forthcoming).

For the main attributes, the signs of coefficients

suggest that respondents preferred to see a GP from

their own practice or health centre, preferred shorter to

longer waiting times, and preferred the doctor to listen.

These are as expected and confirm the theoretical

validity of the technique. In terms of the location of

Table 2

Descriptive characteristics of respondents

Characteristic Response

From Aberdeen 44% (1701/3890)

Age of parent/guardian (mean years, range) 34 (16–75)

Gender of parent/guardian (% female) 87.4% (3394/3883)

Age of children (mean years, range) 6.9 (0–25)

Number of children (mean, range) 2.2 (0–12)

Highest level of education of parent/guardian University 17% (659/3858)

College 32% (1215/3858)

Secondary 50% (1937/3858)

None 1.1% (43/3898)

Health status of child Excellent 47% (1812/3878)

Good 41% (1586/3878)

Fair 10% (374/3878)

Poor 2% (106/3878)

Child has ever suffered from asthma or other ‘wheezy’ illness Yes 38% (1463/3881)

Child has other medical problems for which they see a doctor regularly Yes 16% (630/3869)

Access to a car during the night Yes 73% (2805/3830)

Availability of others to help look after children during the night Yes 74% (2756/3715)

Used out of hours care before Yes 42% (1634/3890)


care, respondents preferred a home visit compared to a

primary care emergency centre run by GPs, and a

hospital A&E department to a primary care emergency

centre.

The relative importance of each attribute can be

examined by using the main regression coefficients,

which are measured on the same scale, as well as

marginal rates of substitution. The most important

attribute was ‘whether the doctor seemed to listen’,

followed by being seen at an A&E department compared

to a primary care emergency centre, seeing a GP from

your own practice, and being seen at a home visit

compared to a primary care emergency centre. Waiting

time was also a significant predictor, with the impor-

tance of waiting time compared to other attributes

depending on how long they had to wait. The coefficient

gives the relative importance for a difference of 1minute

in waiting time between each model in the scenarios.

It is possible to examine the approximate ‘thresholds’

at which waiting time becomes more important than the

other attributes, by examining the implied trade-offs or

marginal rates of substitution (Table 4). Waiting time

becomes more important than ‘whether the doctor

listens’ if the difference in waiting time was longer than

66minutes. This indicates that respondents would be

willing to wait up to 66minutes to see a doctor who

seems to listen. Waiting time became more important

than ‘who was seen’, if the difference in waiting time was

longer than 14minutes, suggesting that respondents

would be willing to wait 14minutes to see a doctor from

their own practice. Similarly, waiting time was more

important than being seen in an A&E department if the

difference waiting time was longer than 33minutes. This

also represents the extra time respondents are willing to

wait to be seen in an A&E department rather than a

primary care emergency centre. Waiting time became

more important than being seen at home rather than in a

primary care emergency centre, if the difference in

waiting time was longer than 11minutes.

The reduced model is shown in Table 5. The main

effects are of a similar sign, except for the constant term

and whether there is a home visit or a primary care

emergency centre visit. The constant term is now

positive, suggesting a general preference for Scenario

B. This will be a function of the addition of interaction

terms that involve dummy variables, and show that once

respondents’ characteristics are included in the model,

the general preference is for more traditional models of

care, rather than the new co-operative model with a

primary care emergency centre. However, other expla-

nations for the significant constant term should not be

ruled out (i.e. ‘right’ bias and omitted attributes). The

sign of the variable examining preferences for either a

home visits or a visit to a primary care emergency centre

has also changed, suggesting that once respondents’

characteristics are included, a primary care emergency

centre visit is preferred to a home visit.

The coefficients of the interaction terms show how

preferences differ depending on respondents’ character-

istics and past experiences. For location of care, home

visits and A&E visits were more likely to be preferred to

primary care emergency centre visits for those who had a

college and university education, compared to those who

had a secondary school education. Those with children

in fair/poor health compared to excellent health also

preferred to receive a home visit.

There was also evidence that the location of their last

out of hours visit influenced preferences. A home visit

was preferred to a primary care emergency centre visit

by those who had previously had an A&E visit and a

home visit compared to those who never visited before.

Emergency centre visits were more likely to be preferred

to home visits for those who previously had an

emergency centre visit, and those who received a visit

more than a year ago. An A&E department was more

likely to be preferred to a primary care emergency centre

visit for those who previously received a home visit.

Table 3

Characteristics of last out of hours visit for those who had used

an out of hours service

Characteristics

of last out of

hours visit:

Response

When did your

child last see a

doctor during

the night?

Less than 1

month ago

3% (106/3879)

1–6 months ago 15% (591/3879)

6–12 months ago 17% (660/3879)

More than 1

year ago

24% (938/3879)

Never 33% (1266/3879)

Can’t remember 8% (318/3879)

Doctor seemed

to listen

84% (2197/2623)

Doctor did not

seem to listen

10% (258/2623)


Place of visit A&E 13% (353/2631)

Emergency

centre

33% (865/2631)

Home visit 48% (1263/2631)


Time between

phoning and

seeing a doctor

Around 20min 29% (742/2592)

Around 40min 33% (845/2592)

Around 60min 15% (394/2592)

Around 80min 10% (256/2592)

>80min 0.03% (14/2592)


Who was seen GP from

practice

25% (641/2616)

GP not from

practice

68% (1781/2616)



Preferences for seeing a GP from their own practice

(or not) also differed across sub-groups. Those sampled

from Glasgow were more likely to prefer to see a GP

from their own practice, compared to those from

Aberdeen, as were those sampled from the general

population compared to those who were sampled from

home visits. Seeing a GP from their own practice had a

higher marginal utility for respondents with older

children, with a secondary rather than university

education, and for those who saw a GP from their

own practice at their last visit.

A shorter waiting time was preferred by respondents

whose child had not previously suffered from asthma, by

those with fewer and younger children, and by those

with children in excellent compared to good, fair or poor

health. Preferences for waiting time were also influenced

by parent’s characteristics, independent of their chil-

dren’s characteristics. A shorter waiting time was

preferred by those sampled from Glasgow compared

to Aberdeen, by those with a college and university

education compared to secondary education. Past

experiences also influenced the marginal utility of

waiting times, as shorter waiting times were preferred

by those who had never had an out of hours visit

compared to those who visited up to 1 year ago, and by

those who waited 20minutes at their previous visit.

Those who waited more than 1 hour for their last visit

were more likely to prefer longer waiting times,

compared to those who had never had a visit.

Preferences for the doctor–patient relationship were

also influenced by respondents’ characteristics. Those

who were more likely to prefer the doctor who seem to

listen included respondents who were younger, who were

sampled from the general population compared to those

sampled from centre visits, and who had a college or

university education compared to those who had a

secondary education. Respondents’ childrens’ character-

istics also had an effect, with the doctor listening being

preferred by those whose children had no other medical

problems, and who were in excellent rather than poor or

fair health. Past experiences also had an impact. The

doctor listening was preferred by those who had never

received an out of hours visit compared to those who

visited up to 1 year ago, and by respondents whose

doctor had seemed to listen at their last visit.

The version of the questionnaire respondents were

sent also influenced preferences. Although the scenarios

were allocated to each version of the questionnaire

randomly, this does not guarantee that there will be no

systematic differences in preferences. The inclusion of

these variables in the model controls for these biases.

Discussion

The most important attribute was whether the doctor

seemed to listen. This was independent of whether the

patient knew the doctor, perhaps reflecting the real

reason why other studies have reported low satisfaction

with deputising services. This finding is consistent with

other studies examining patient satisfaction with general

practice and the doctor–patient relationship (Williams &

Calnan, 1991; Savage & Armstrong, 1990; Wissow,

Roter, & Wilson, 1994), with out of hours services

(Scottish Out of Hours Study Group, 2001), as well as

previous discrete choice experiments examining the

doctor–patient relationship (Vick & Scott, 1998) and

out of hours care (Morgan, Shackley, Pickin, & Brazier,

Table 4

Regression results (main effects only)

Variable b (SE) MRSa (SE) 95% CI

Constant �0.2015 (0.0399)* — —

Who is seen 0.4686 (0.0254)* �14 (0.83) �15.73 to �12.48Waiting time �0.0332 (0.0006)* — —

Whether the doctor listens 2.1925 (0.0275)* �66 (1.11) �68.17 to �63.82Home visitb 0.3638 (0.0422)* �11 (1.34) �13.58 to �8.32Hospital A&E department visitb 1.0892 (0.0363)* �33 (1.25) �35.23 to �30.35r 0.3227 (0.0121)* — —

�2 log likelihood �9832 % 0’s predicted correctly 96%

Model chi-squared (d.f.) 6959 (5)* % 1’s predicted correctly 66%

Psuedo R2 29% Number of observations 24789

Number of individuals 3326

Notes: *=po0:0001:aMRS ¼ bx=bwaiting time: Standard errors calculated from a Taylor series approximation to the variance of a function of random

variables, where varðMRSÞ ¼ 1=b2waiting time½varðbxÞF2MRS covðbx;bwaiting timeÞ þMRS2 varðbwaiting timeÞ� (Propper, 1988).bRelative to a primary care emergency centre.


Table 5

Regression results (model with interaction terms)

Variable B SE

Constant 0.3586 0.0593***

Who is seen 0.3975 0.0624***

Waiting time �0.0618 0.0024***

Whether the doctor listens 3.3710 0.1189***

Home visita �0.7080 0.1011***

Hospital A&E department visita 1.7586 0.0809***

Centre visit sample*home visitc �0.2574 0.0531***

Home visit sample*home visitc �0.2061 0.0606**

College education*home visitd 0.1926 0.0676**

University education*home visitd 0.1566 0.0786**

Fair/poor health*home visite 0.2012 0.0729**

Last visit more than 1 year ago*home visitf �0.1538 0.0627**

Last visit at A&E*home visitf 0.2747 0.0897**

Last visit at home*home visitf 0.1555 0.0609**

College education*A&E visitd 0.1238 0.0643*

Last visit at home*A&E visitf 0.1732 0.0476**

Grampian Health Board*who is seenb �0.0987 0.0411**

Age of children*who is seen 0.0139 0.0054**

Centre visit sample*who is seenc �0.0988 0.0594*

University education*who is seend �0.1640 0.0710**

Saw a GP from own practice at last visit*who is seenf 0.1377 0.0534**

Grampian Health Board*waiting timeb �0.0027 0.0008**

Whether child has asthma*waiting time 0.0028 0.0009**

Number of children*waiting time 0.0013 0.0004**

Age of children*waiting time 0.0003 0.0001**

College education*waiting timed �0.0029 0.0013**

University education*waiting timed �0.0049 0.0016**

Good health*waiting timee 0.0024 0.0008**

Fair/poor health*waiting timee 0.0056 0.0017**

Last visit up to 6 months ago*waiting timef 0.0029 0.0013**

Last visit between 6 months and 1 year*waiting timef 0.0048 0.0012**

Waited 20min at last visit *waiting timef �0.0023 0.0011**

Waited 60min at last visit *waiting timef 0.0036 0.0013**

Waited over 80min at last visit*waiting timef 0.0052 0.0016**

Other health problems of child*doctor listens �0.1170 0.0374**

Centre visit sample*doctor listensc �0.0861 0.0414**

Age*whether the doctor listens �0.0167 0.0024***

College education*doctor listensd 0.3137 0.0497***

University education*doctor listensd 0.1551 0.0539**

Fair/poor health*doctor listense �0.1727 0.0462**

Last visit up to 6 months ago*doctor listensf �0.2258 0.0461***

Last visit between 6 months and 1 year ago*doctor listensf �0.1429 0.0433**

Doctor listened at last visit *doctor listens 0.1058 0.0331**

Questionnaire*home visit 0.7898 0.1245***

Questionnaire*A&E visit �1.4554 0.0647***

Questionnaire*waiting time 0.0128 0.0014***

r 0.3097 0.0126***

�2 log likelihood �9199 % 0’s predicted correctly 90%

Model chi-squared (d.f) 5916 (45)*** % 1’s predicted correctly 72%

Psuedo-R2 39% Number of observations 24789

Number of individuals 3326

Notes: ***=po0:0001; **=0.0001Xpo0:05; *=0.10Xpp0.05.aRelative to a primary care emergency centre.bRelative to Greater Glasgow Health Board.cRelative to general population sample.dRelative to secondary education.eRelative to excellent health.


2000). The finding suggests that if GP out of hours

organisations want to increase patients’ utility the most,

then they should focus on improving doctors’ commu-

nication skills.

Preferences differed by respondents’ characteristics.

Socio-economic differences are reflected by respondents’

education. Highly educated respondents had a stronger

preference for A&E department visits to primary care

emergency centre visits and home visits to primary care

emergency centre visits. They were also less concerned

about seeing a GP from their own practice, more

concerned about a lower waiting time, and had a

stronger preference for the doctor listening. This may

have implications for the provision of services and the

pattern of demand in affluent compared to deprived

areas. A previous satisfaction study of out of services in

Glasgow showed that those from affluent areas were

more likely to be dissatisfied (Wilson et al., 2001).

The characteristics of parents’ children also influenced

preferences. Respondents with children in fair or poor

health were more likely to prefer a home visit than a

primary care emergency centre visit. Those with children

in excellent health, and who had not previously had

asthma had a stronger preference for a shorter waiting

time. This may be related to limited experience of health

care services or illness where expectations with respect to

waiting times may be high. Parents of children with

other health problems were less likely to prefer the

doctor to listen. Parents with younger and fewer

children were more likely to prefer a shorter waiting

time, whilst those with older children had a stronger

preference to see a GP from their own practice.

Previous utilisation of GP out of hours services

influenced preferences. Those who had never visited an

out of hours service before were more likely to prefer a

visit at a primary care emergency centre compared to a

home visit, and more likely to prefer a shorter waiting

time and for the doctor to listen. Those sampled from

the general population had a stronger preference for the

doctor listening and seeing a GP from their own

practice. This has implications for the education of

potential users about appropriate use of out of hours

services, and also shows that the potential users may

have higher expectations of out of hours care compared

to those who have used the service before.

What happened at the respondent’s last out of hours

visit also influenced preferences, such that respondent’s

preferred what they had previously experienced. This

confirms previous studies, and has implications for when

preferences should be elicited during a health care

episode (Salkeld et al., 2000). Along with the influence of

past utilisation, it suggests that to a certain extent,

preferences are endogenous. It is not necessarily the case

that individuals hold a set of stable and complete

preferences as implied by standard neo-classical micro-

economic theory. Once a dynamic element is introduced,

preferences may be modified by past experiences and

may also exhibit a degree of path-dependence. This

highlights the importance of gathering information

about these past experiences in preference elicitation

studies.

The most preferred model of GP out of hours care for

parents of children who have respiratory symptoms in

urban areas, is a hospital accident and emergency

department where the doctor seems to listen, where

waiting time between the initial phone call and treatment

being received is low, and where they see a doctor from

their own practice. This describes a model of care that

does not exist in most places of the UK, although there

are examples of GPs working in A&E departments, and

of primary care emergency centres located adjacent to

A&E departments. For example, GEMS in Glasgow has

six primary care emergency centres, several of which are

located alongside A&E departments.

These preferences were elicited without any informa-

tion on the costs of providing this model of care and so

it may not be possible, for example, for patients to see a

GP from their own practice in an A&E department. The

results of this study would need to be combined with

costs of different combinations of attributes to establish

the most cost-effective model of care.

Nevertheless, the results have important implications

for the demand for new models of GP out of hours care

which are based in primary care emergency centres and

where the patient may not see a GP from their own

practice. For parents of children with respiratory

symptoms, these new models may not be the most

preferred. The consequence of this may be that new

models of GP out of hours care may not reduce the

demand from parents of young children with respiratory

problems, who may continue to use A&E services. New

primary care emergency centres may not therefore

reduce the demand for A&E services. It also suggests

that primary care emergency centres that are located

within or adjacent to A&E departments may be more

effective at reducing demand at A&E departments, than

centres located away from acute hospitals.

These results should, however, be interpreted with

several issues in mind. First, the results presented here

apply to the parents of children under 13, when their

child has respiratory symptoms. Although this is a

relatively common condition seen by GPs out of hours,

preferences may differ for other groups of individuals,

such as the elderly, and for other types of symptoms of

differing severities. In addition, the study applied to

urban areas only and extrapolation to rural areas would

be difficult.

Second, some individuals responding may not be

using compensatory decision rules. This can be proble-

matic when a lexicographic ordering exists since utility

functions cannot be defined (Scott, 1998). These

individuals have very strong preferences for a given


attribute since they are not prepared to trade off with

other attributes. Generally, individuals may have used a

variety of non-compensatory decision making rules

and heuristics that lie between the extremes of fully

compensatory decision making and lexicographic

orderings (Payne, Bettman, & Johnson, 1992). The

implications of non-compensatory decision making

are that policy recommendations based on the marginal

rate of substitution should be treated with caution.

Our results found that individuals were prepared to wait

an additional 66minutes to see a doctor who seems

to listen. This implies that a policy that reduces waiting

times by 66minutes would generate the same improve-

ment in utility as a policy that improved doctors’

communication skills. However, individuals with a

lexicogrpahic ordering would not be willing to

wait at all. In this case, any policies to reduce

waiting times would not influence their utility, or

demand. For those sub-groups of the sample with a

lexicographic ordering, the implication is that reducing

waiting times would be of no value to them. A further

implication is that inclusion of those with lexicographic

orderings will overestimate the size of regression

coefficients for those attributes where individuals have

dominant preferences.

It is important to find ways of identifying these sub-

groups of individuals, although this may only be

possible through face-to-face interviews that explore

individuals’ preferences in more depth. However, there

may be different reasons for the existence of these

heuristics, that include the complexity of the question-

naire (bounded rationality) or because they genuinely

have a dominant preference. Alternative ways of

examining preferences should also be explored, such as

those reviewed by Ryan et al. (2001).

There are also other issues when interpreting the

attributes. The study has not investigated why respon-

dents preferred the A&E department. This may because

of a higher perceived ‘quality’ of service and faster

access to other hospital facilities. This implies other

attributes of potential importance that require further

research. Furthermore, different wording of the attri-

butes may have influenced responses, especially for the

doctor listening. Further research should be conducted

on framing effects in choice experiments.

A further issue is the treatment of inconsistent

responses. Eight percent of responses were inconsistent,

suggesting those individuals did not understand the task

being set. This compares favourably with other methods

of eliciting preferences in health care (with levels of

inconsistency of between 7% and 77%) (Vick & Scott,

1998). Although the inclusion of dominant scenarios to

check for inconsistency may reduce the number of

choices that involve trading-off attributes, this is not a

major concern and has not manifested itself in a poor

quality econometric model.

Finally, there are several issues about the experi-

mental design. When placing scenarios into pairs, the

property of orthogonality that was in the original linear

design is no longer guaranteed. What is required is a

design that minimises colinearity between attribute

differences. However, it is not necessarily the case that

correlations have to equal zero as there are other aspects

to the design that need to be considered, such as balance

(that the levels of attributes appear with equal

frequency) and minimal overlap of attribute levels. It

has been recognised that, ‘‘y for most combinations of

attributes, levels, alternatives and parameter vectors, it is

impossible to create a design that satisfies (all of) these

principles’’ (Zwerina, Huber, & Kuhfeld, 1996).

A common assumption in choice experiments is that

interactions between the main effects are negligible.

Because there were no specific hypotheses that the main

effects in this study were interdependent, we used a

fractional factorial design and so assumed interaction

effects were negligible. If there are interaction effects,

then reliable estimation of these is only possible if they

are built into the experimental design (or if a full factorial

design is used). However, this does not rule out the

possibility that interaction effects exist. This may reduce

the explanatory power of the model and could lead to

incorrect estimates of regression coefficients for the main

effects, and must be borne in mind when interpreting

results. Although there have been few studies in health

that have examined interactions, previous empirical

studies in transport suggest that interactions between

main effects are negligible (Louviere, 1988).

Further research is required into the demand for out

of hours services, especially as alternative models begin

to emerge. It is not sufficient to reconfigure services to

suit the preferences of providers, as this takes no

account of patterns of demand. Preferences of both

users and non-users need to be accounted for.

Acknowledgements

Thanks go to Nicola Torrance for collecting data, and

to the staff of G-DOCs and GEMS. Thanks also go to

anonymous referees and Cristina Ubach for helpful

comments. This project was funded by the NHS R&D

Primary–Secondary Care Interface Programme. The

Health Economics and Health Services Research Units

are funded by the Chief Scientist Office of the Scottish

Executive Health Department (SEHD). The views in this

paper are those of the authors and not SEHD.

Appendix A. Testing whether the design is orthogonal

The differences between the levels of the paired

attributes were tested for orthogonality. The sample


size for these tests was 15 (the number of pairwise

choices in the questionnaires), and so the power to

detect statistically significant correlations was low, and

so none of the values in the table below are statistically

significant. However, all are close to zero suggesting that

the attribute differences exhibit low associations. Since

there are a mixture of nominal and interval data, the

measures of association used varied and so are not

directly comparable with each other (see Table 6).

References

Bain, J., Gerrard, L., Russell, A., Locke, R., & Baird, V. (1997).

The Dundee out of hours co-operative: Preliminary out-

comes for the first year of operation. British Journal of

General Practice, 47, 573–574.

Bollam, M. J., McCarthy, M., & Modell, M. (1988). Patients’

assessment of out of hours care in general practice. British

Medical Journal, 296, 829–832.

Brunel University. (1993). The valuation of changes in quality in

the Public Services. Report prepared for HM Treasury,

HMSO, London.

Calnan, M., Katsoyiannopolous, V., Ouchavev, V. K., Pro-

khorskas, R., Ramic, H., & Williams, S. (1994a). Major

determinants of consumer satisfaction with primary care in

different countries. Family Practice, 11, 468–478.

Carr-Hill, R. (1992). The measurement of patient satisfaction.

Journal of Public Health Medicine, 14, 236.

Cragg, D., Campbell, S. M., & Roland, M. O. (1994). Out of

hours primary care centres: Characteristics of those attend-

ing and declining to attend. British Medical Journal, 309,

1627–1629.

Department of Health. (1998). GP out of hours services.

Working Group Report, Department of Health, Scottish

Office, Edinburgh.

Department of Health. (2000). Raising standards for patients.

New partnerships in out of hours care. An independent

review of GO out of hours services in England. Department

of Health, London.

Dixon, R. A., & Williams, B. T. (1988). Patient satisfaction with

general practitioner deputising services. British Medical

Journal, 297, 1519–1522.

Hansen, B. L., & Munck, A. (1998). Out of hours services in

Denmark. The effect of a structural change. British Journal

of General Practice, 48, 1497–1499.

Hallam, L., & Henthorne, K. (1999). Cooperatives and their

primary care emergency centres: organisation and impact.

Combined report on seven case studies. Health Technology

Assessment 3.

Louviere, J. (1988). Analysing decision making: Metric conjoint

analysis. California: Sage Publications Inc.

Manski, C. F. (1977). The structure of random utility models.

Theory and Decision, 8, 229–254.

McKinley, R. K., Cragg, D. K., Hastings, A. M., French, D. P.,

Mankau-Scott, T. K., Campbell, S. M., Van, F., Roland,

M. O., & Roberts, C. (1997a). Comparison of out of hours

care provided by patients’ own general practitioners and

commercial deputising services: A randomised controlled

trial. II: The outcome of care. British medical Journal, 314,

190–193.

McKinley, R. K., Mankau-Scott, T. K., Cragg, D. K.,

Hastings, A. M., French, D. P., & Baker, R. (1997b).

Reliability and validity of a new measure of patient

satisfaction with out of hours primary medical care in the

United Kingdom: Development of a patient questionnaire.

British Medical Journal, 314, 193–198.

McLaren, G., & Bain, M. (2000). Deprivation and health in

Scotland. Information and Statistics Division, Common

Services Agency, Edinburgh.

Morgan, A., Shackley, P., Pickin, M., & Brazier, J. (2000).

Quantifying patient preferences for out of hours primary

care. Journal of Health Services Research and Policy, 5,

214–218.

Morrison, J. M., Gilmour, H., & Sullivan, F. (1991). Children

seen frequently out of hours in one general practice. British


Table 6

Measures of association between attribute differences

D WHERE D WHO D LISTEN D TIME

D WHERE — 0.07a/0.38b 0.006c/0.094b 0.22d

D WHO 0.07a/0.38b — 0a/0.07b 0.03d

D LISTEN 0.006c/0.094b 0a/0.07b — 0.03d

D TIME 0.22d 0.03d 0.03d —

al: A measure of association which reflects the proportional reduction in error when values of the independent variable are used to

predict values of the dependent variable. A value of 1 means that the independent variable perfectly predicts the dependent variable. A

value of 0 means that the independent variable is no help in predicting the dependent variable.bMeasures of association based on chi-square including: contingency coefficient (measure of association based on chi-square—the

value ranges between zero and 1, with zero indicating no association between the row and column variables and values close to 1

indicating a high degree of association between the variables. The maximum value possible depends on the number of rows and

columns in a table.); f (a chi-square based measure of association that involves dividing the chi-square statistic by the sample size and

taking the square root of the result); cramer’s V (a measure of association based on chi-square).cGoodman and Kruskal t: Similar to l; except the marginal proportions are used for predictions, rather than modal category.dZ is a measure of association that ranges from 0 to 1, with 0 indicating no association between the row and column variables and

values close to 1 indicating a high degree of association.


Payne, J. W., Bettman, J. R., & Johnson, E. J. (1992).

Behavioural decision research: A constructive processing

perspective. Annual Review of Psychology, 43, 87–131.

Pearmain, D., Swanson, J., Kroes, E., & Bradley, M. (1991).

Stated preference techniques: A guide to practice. Steer

Davies Gleave, Hague Consulting Group.

Propper, C. (1988). Estimation of the value of time spent on NHS

waiting lists using stated preference methodology. Discussion

Paper 49, Centre for Health Economics, University of York.

Prudhoe, R. H. (1984). Deputising services. British Medical

Journal, 288, 718.

Ryan, M. (1999). Using conjoint analysis in health economics.

Paper presented to the International Health Economics

Association, Rotterdam.

Ryan, M., Scott, D. A., Bate, A., van Teijlingen, E., Russell, E.

M., Napper, M., Reeves, C., & Robb, C. (2001). Eliciting

public preferences for health care: a systematic review and

evaluation of methods. Health Technology Assessment, 5(5).

Salisbury, C. (1997). Postal survey of patients’ satisfaction with

a general practice out of hours co-operative. British Medical

Journal, 314, 1594–1598.

Salkeld, G., Ryan, M., & Short, L. (2000). The veil of

experience: do consumers prefer what they know best?

Health Economics, 9(3), 267–270.

San Miguel, F., Ryan, M., & Scott, A. (forthcoming). Testing

the assumptions of completeness and stability of preferences

using discrete choice experiments. Journal of Economic

Behaviour and Organisation, forthcoming.

Savage, R., & Armstrong, D. (1990). Effect of a general

practitioner’s consulting style on patient’s satisfaction: A

controlled study. British Medical Journal, 301, 968–970.

Sawyer, L., & Arber, S. (1982). Changes in home visiting and

night and weekend cover: The patient’s view. British


Scott, A. (1998). Giving things up to have more of others. The

implications of limited substitutability in eliciting prefer-

ences for health and health care. Discussion paper 01/98,

Health Economics Research Unit, University of Aberdeen,

1998.

Scott, A. (2000). Agency and incentives in general practice. Ph.D.

Thesis, University of Aberdeen.

Scott, A., & Smith, R. (1994). Keeping the customer satisfied:

Issues in the interpretation and use of patient satisfaction

surveys. International Journal for Quality in Health Care, 6,

353–359.

Scottish Out of Hours Study Group. (2001). A comparison of

models of delivery of out of hours general medical services in

Scotland. Final Report to Scottish Executive, Department

of General Practice, University of Edinburgh.

Smith, C. H., & Armstrong, D. (1989). Comparison of criteria

derived by government and patients for evaluating

general practitioner services. British Medical Journal, 299,

494–496.

Vedsted, P., & Olesen, F. (1999). Effect of re-organised after-

hours family practice service on frequent attenders. Family

Medicine, 31, 270–275.

Vick, S., & Scott, A. (1998). Agency in health care. Examining

patients’ preferences for attributes of the doctor–patient

relationship. Journal of Health Economics, 17, 511–644.

Williams, S. J., & Calnan, M. (1991). Key determinants of

consumer satisfaction with general practice. Family Prac-

tice, 8, 237–242.

Wissow, L. S., Roter, D. L., & Wilson, M. E. H. (1994).

Pediatrician interview style and mothers’ disclosure of

psychosocial issues. Pediatrics, 93, 289–295.

Wilson, P., McConnacchie, A., O’Donnell, C. A., Ross, S.,

Moffat, K. J., & Drummond, N. (2001). Assessing

dissatisfaction with an out of hours service: Reasons and

remedies. Health Bulletin, 59, 37–44.

Zwerina, K., Huber, J., & Kuhfeld, W. (1996). A general method

for constructing efficient choice designs. Working Paper,

Fuqua School of Business, Duke University.


Documents

Eliciting preferences of the community for out of hours care provided by general practitioners: a stated preference discrete choice experiment