HSRU is funded by the Chief Scientist Office of the Scottish Executive Health Department. The author accepts full responsibility for this talk. Health

HSRU is funded by the Chief Scientist Office of the Scottish Executive Health Department. The author accepts full

responsibility for this talk.

Health Services Research Unit University of Aberdeen

Health Services Research Unit University of Aberdeen

Data collection (1) Data collection (1) Quantitative:Quantitative:

Structure, Process Structure, Process and Outcomeand OutcomeClare Robertson

HSRU is funded by the Chief Scientist Office of the Scottish Government Health Directorates. The author accepts full responsibility for this talk.

Health Services Research Unit

OutlineOutline

1. What to measure – structure, process, outcome

2. Studies of structure, process and outcome

3. Outcome measurements

4. Developing a new scale


Purpose of HSRPurpose of HSR

• To provide information regarding health care delivery

‘Health services research must provide the evidence by which health services activities are judged’ Crombie IK (1996)


Research processResearch process1. Identify the research question

eg Is keyhole surgery better than open surgery for hernia repair?

2. Decide upon research designeg randomised controlled trial

3. Choose what to measureeg recurrence of hernia

4. Decide how to measure eg physical examination at 1 year


Health care systemHealth care system

O u tco m e (o u tp u t)

P ro cess (th ru p u t)

S tru ctu re (in p u t)


Deciding what to measure Deciding what to measure II

• Structurethe physical features of health care

eg premises, number of staff, range and type of equipment, medical records

• Processinteraction between health professionaland patient

eg examinations, investigations, referrals, interactions


Deciding what to measure Deciding what to measure IIII

• Intermediate (surrogate) outcomedescribes measures which may be either process or short-term outcomes which are closely related to the outcome of interest and highly predictive of iteg measuring blood pressure when evaluating treatments for stroke


Deciding what to measure Deciding what to measure IIIIII

• Outcome changes in a patient’s current and future health status that can be attributed to antecedent

health care (the resulting changes in the health

of that patient)

eg quality of life measures, survival, cure rate


EXAMPLE- MRSAEXAMPLE- MRSAMRSA (sometimes referred to as the superbug) methicillin-

resistant Staphylococcus aureus.MRSA infections are more difficult to treat due to the

antibiotic-resistance of the bacteria. • septicaemia (blood poisoning), • septic shock (widespread infection of the blood that

leads to a fall in blood pressure and organ failure), • severe joint problems (septic arthritis), • Bone marrow infection (osteomyelitis), • internal abscesses anywhere within the body, • inflammation of the tissues that surround the brain and

spinal cord (meningitis), • lung infection (pneumonia), and • infection of the heart lining (endocarditis).


Example - MRSAExample - MRSA• Those in hospital are more likely to develop

MRSA infections because they often have an entry point for the bacteria to get into their body, such as a surgical wound, a catheter, or an intravenous tube.

• MRSA infections are diagnosed by testing blood, urine or a sample of tissue from the infected area for the presence of MRSA bacteria.

• MRSA is usually passed on by human contact, often from the skin of the hands.


MRSA- preventionMRSA- prevention

• Organisation– Regular cleaning of hospitals

• Health care professionals– Handwashing -fast-acting, special

antiseptic solutions (alcohol rubs or gels) before examination of patients

– disposable gloves when they have physical contact with open wounds


MRSA- preventionMRSA- prevention

• Patients– Keep hands and body clean – wash your hands after using the toilet

(hand wipe)– wash your hands or clean them with a

hand-wipe before and after eating• Visitors

handwashing when enter patient areas


Structural studies of care I studies of care IWhen appropriate

– exploratory studies in new areas– ensuring minimum standard of care– structure is expected to be strongly

related to outcome

Methods– observation survey – questionnaire survey


Structural studies of care IIStructural studies of care II

Advantages – easy to conduct– valuable when strong link between

structure and outcome

Disadvantage– link between structure and outcome is

tenuous


Process studies of care IProcess studies of care I

When appropriate– process expected to be strongly related

to outcome (intermediate outcome)– quality of care– mechanisms- (behavioural)Methods– observation of process– self report instruments– case note review– routine data sources


Process studies of care IIProcess studies of care II

Advantages– easy to conduct (?behavioural change) – stronger links between process and

outcome

Disadvantages– no guarantee that changes in process

will improve outcome


Outcome studies of care IOutcome studies of care IWhen appropriate

– any evaluation

Methods– self administered outcome measure – interview administered outcome measure – case note review – routine data sources– trial report forms = web based(clinical

outcomes)


Outcome studies of care IIOutcome studies of care IIAdvantages

– ‘gold standard’ for evaluation

Disadvantages– difficult to measure:

validityreliabilitytiming of evaluationaccessibility feasibility

– problems of attribution– valuation of costs


Difficulties associated with Difficulties associated with outcome measurement outcome measurement

• Consider – How specific an outcome should be measured? – Whose values should be used?– When should assessment take place?– How will it be assessed?

• Subjective experience of health varies across individuals and time

What would you measure to assess the effects of the intervention on the ‘health’ of patients?


What type of outcome?What type of outcome?

• critical events • condition/disease specific

– clinical• biomedical/biological markers• symptom

– functional– educational– economic

How general?


What type of outcome?What type of outcome?How general? (continued)

• client group specific• general

– health profiles– health indices using indirect valuation– health indices using direct valuation

• patient satisfaction, health related knowledge and behaviour


Examples: Condition specific Examples: Condition specific outcome measurement Ioutcome measurement I

• clinical – biomedical/biological markers

eg peak flow rate, blood pressure

– symptomseg wheeze,


Examples: Condition specific Examples: Condition specific outcome measurement IIoutcome measurement II

• functionaleg disturbed nights in previous month

• educationaleg knowledge about asthma drugs

• economiceg time and travel to clinic


Examples: Client group Examples: Client group specific IIspecific II

Hospital Anxiety and Depression Scale (HADS)

• anxiety– 7 items (4 point scale)

• depression

– 7 items (4 point scale)


ExamplesExamples: GeneralGeneral outcome measurement I I

Health profile: eg SF-36 (36 items in 8 dimensions)

– General health perceptions (5 items)– Physical function (10 items)– Role limitation physical (4 items)– Role limitation emotional (3 items) – Mental health (5 items)– Pain (2 items)– Social functioning (2 items)– Energy (4 items)


Examples: General outcome Examples: General outcome measurement IImeasurement II

Health indices – single value for the health status of

each patient surveyed– either indirect valuation or direct

valuation


Examples: Health indices IExamples: Health indices I

Indirect valuation: • EUROQOL-5D (EQ-5D)

– 5 dimensions represented by one value• mobility• self care• usual activities• pain• anxiety

• Visual analogue scale (VAS)


Examples: Health indices IIExamples: Health indices II

Direct valuation: enables patients to value their health status directly without reference to components of that health status

eg time trade offstandard gamblemagnitude estimate


ExampleExample

• Compare two different “types” of knee replacement surgery in patients

• What outcomes might you be interested in?


Developing a new scaleDeveloping a new scale

• Identify any suitable published scales• Devise new items • Select items• Avoid bias• Combine items to form a scale• Assess reliability• Assess validity


Devising new itemsDevising new items

• Sources– literature review

– clinical observation

– expert opinion


Devising new itemsDevising new items

• Defining responses– binary– categorical– ordinal– interval– continuous


Selecting itemsSelecting items• Interpretability

– reading levelFlesch score

– jargonhow long have you had hypertension?

– value laden wordsdo you often go to your doctor with trivial problems?

– positive and negative wordingnot, rarely, never I rarely feel well agree/disagree


Assessing validity I Assessing validity I • Face validity

appear to be assessing desired qualities

• Content validitysampling all relevant and important areas

• Criterion validitycorrelated with gold standard

• concurrently - new simple scale and SF36• predictive - prescribing behaviour scale

and actual prescribing


Assessing validity IIAssessing validity II

• Construct validityscale correlated with other measures of underlying construct or measures independent of construct

time trade off and SF36time trade off and sleep patterns


Assessing reliabilityAssessing reliability

• Influence of observers– Intra observer – Inter observer

• Test-retest reliability• Internal consistency


Avoiding bias IAvoiding bias I• Social desirability

– unwilling to report things they see as socially unacceptable

• Acquiescent– agree with statements regardless of

what the content is– disguise intent of scale – use subtle items– random response technique


ExampleExampleImagine you want to evaluate the implementation of a shared care scheme with guidelines for those newly diagnosed with asthma (cough and wheeze)

Usual care = see general practitioner only as per usual care

Shared care = see general practitioner and go to asthma clinic (3 monthly in 1st year/ 6 monthly yr 2 and 3) to see consultant – re use of asthma drugs, asthma controlGP given guidelines re referral and treatment


QuestionsQuestions

What would you measure (& how & when)? Structure, process and outcome?

Consider: advantages/disadvantages


Further readingFurther reading

• Bowling A. Research methods in health: Investigating health and health services, 2nd edition. Open University Press, 2002.

• McDowell and Newell (1995). Measuring health. A guide to rating scales and questionnaires. Oxford University Press.

• Wilkin, Hallam and Doggett (1992). Measures of need and outcome for primary health care. Oxford University Press.


Next time on HSRNext time on HSR

Everything you ever wanted to know about questionnaires (and possible a whole lot more)!

Documents

HSRU is funded by the Chief Scientist Office of the Scottish Executive Health Department. The author accepts full responsibility for this talk. Health