A Systematic Review of A&F interventions: the good, the bad, and the uglyHeather Colquhoun, CIHR and KT Canada Postdoctoral Fellow, OHRI

Research Team

Jamie C Brehaut, Ottawa Hospital Research Institute (OHRI)

Kevin Eva, University of British Columbia (Vancouver)

Jeremy Grimshaw, OHRI

Anne Sales, University of Michigan (Ann Arbor)

Noah Ivers, Women’s College Research Institute (Toronto)

Susan Michie, University College London (UK)

Heather Colquhoun, OHRI

Kelly Carroll, MSc, OHRI

Mathieu Chalifoux, OHRI

Funding: CIHR KT Priority Announcement

Systematic Review of studies of A&F

• Not focused on effectiveness, but rather the details of the interventions• Extent, type and purpose of theory use• Consistency with theoretical constructs believed to

be important for A&F• Intervention description

Methods

• Systematic Review• 140 RCT’s of A&F from the Cochrane Review• Data extraction form, pilot X 2• Extracted by two reviewers, one consistent• Separate extraction with consensus for disagreements

A&F Interventions

• To whom?• What exactly? • Where and how delivered?• How much?• Why?

• Give results, challenges, examples of variations

To whom?

• Was the FB given to the person in whom the behaviour change was desired?

Who was FB given to? Count (%)

Individual 72 (51%)

Group 25 (18%)

Individual and Group 23 (16%)

Unclear 20 (14%)

Yes No Unclear

130 (92%) 6 (5%) 4 (3%)

What exactly?FB about Yes No Unclear

Behaviour 111 (79%) 9 (6%) 20 (15%)

Outcomes 20 (14%) 102 (73%) 18 (13%)

Other 45 (32%) 95 (68%) n/a

Individuals Performance

81 (58%) 50 (36%) 9 (6%)

Groups Performance

90 (64%) 38 (27%) 12 (9%)

Aggregate data 114 (81%) 15 (11%) 11 (8%)

Individual data 35 (25%) 95 (68%) 10 (7%)

What exactly?

• Feedback about ‘Other’ 45 (32%)• Accuracy of their diagnosis• Cost• Risk data for patients (stroke, infection)• Education as FB (FB on what type of antibiotic they

should have prescribed) • Survey on barriers to change

What exactly?

• Was the FB graphical?

• Did the FB address the behaviour to be changed?

• Was there a clear comparison in the FB?

Yes No Unclear

124 (89%) 9 (6%) 7 (5%)

Yes No Unclear

47 (36%) 73 (52%) 20 (14%)

Yes No

104 (74%) 36 (26%)

Comparison for A&F

Other: internal standard, “plan wide scores” for comparison, systematic review, algorithm,

others previous performance + thresholds set by expert panel, others previous

performance + own previous performance + drug formulations that accounted for >60%

of charges + the total charges

How delivered?

• Was the FB given face to face?

Yes No Unclear

62 (44%) 68 (49%) 10 (7%)

How much – Total feedback

• Unclear: 33 (24%)• Clear: 107 (76%)

• Once: 33• Twice: 21• Three times: 13• Four times: 13• >four times: 27 (range 5-78)

• Frequency• Challenges

When and how much?

• Was the lag time between collection of the FB and the provision of the FB clearly stated?

• If yes, what was it?• Days: 6• Weeks: 23• Months: 56• Years: 1• Mix: 2

Yes No

88 (63%) 52 (37%)

Why A&F?Intervention rationale in A&F trials

Rationale for intervention Count (%)N=140

Empirical (only) 51 (36%)

Intuitive 39 (28%)

No rationale 37 (26%)

Theory 13 (9%)

Reporting

• “The data was presented in an easy to follow format”• “Extensive feedback was provided twice a week”• “Feedback was always discussed in detail”• “The exact nature of the A&F was decided within each

audit group and supported by the researcher with help and advice from the tutor”

Hospital Report Card: Given to individuals not given to target, about behaviour, about the group, aggregate, comparison: multiple, given 1X, 12 indicators

Given to individuals (mailed), individual and group performance, aggregate data, multiple comparison, target yes, behaviour yes, key message, every 2 months for 6 months (3)

Given to individuals, individual

performance?, ind data,

target yes, behaviour no, monthly for 9

months

Self-monitoring, face to face,

given to individual, individual

performance, weekly for 4

months?

“To be displayed in a conspicuous

location in their office”

Summary

• Some consistency but also wide variation• Reporting is poor in many areas• Rationale for the intervention is lacking

Thank you

Given to individuals by mail, individual and group performance, multiple comparisons, quarterly for 18 months (6), target yes, behaviour yes