[Workshop] Implementation of screening (Oct10)

Alex Mitchell www.psycho-oncology.info/workshop

Department of Cancer & Molecular Medicine, Leicester Royal Infirmary

Department of Liaison Psychiatry, Leicester General Hospital

IPOS 2010IPOS 2010

WORKSHOP Day 2

Implementation of Screening:Screening studies, Short methods, HADS and longer methods, implementation, future of screening

WORKSHOP Day 2

Implementation of Screening:Screening studies, Short methods, HADS and longer methods, implementation, future of screening

Schedule Day 2Schedule Day 2

930-10.00 – Introduction to research task 1. design 2. evaluation

10.00-11.00 – T3 Screening in Cancer: Instruments & Validity

11.30 – 12.30 – Group work #2

1.30-2.30 – T4 Screening in Cancer: Implementation and future

3.00 – 4.00 – Presentation of Research task

Group Work #2Group Work #2

930-10.00 – Introduction, groups and issues

10.00-11.00 – T1 Basic science of screening

11.30 – 12.30 – Group task #1

1.30-2.30 – T2 Symptoms, Burden, Help, Needs in Cancer

3.00 – 4.00 – Evaluation of a screening paper

Group Work #2Group Work #2

Read paper in your group……..

1.What is being tested?

2.What is the comparison?

3.Is the tool effective?

4.Is the tool acceptable?

5.Did the tool make a difference?

T1. Are We Looking for Distress?T1. Are We Looking for Distress?

How Often

What method?

n=226Comment: Frequency of cancer specialists enquiry about depression/distress from Mitchell et al (2008)

1,2 or 3 Simple QQ15%

Clinical Skills Alone73%

ICD10/DSMIV0%

Short QQ3%

Other/Uncertain9% Other/Uncertain

Use a QQ15%

ICD10/DSMIV13%

Cancer StaffCurrent Method (n=226)

Psychiatrists

Comment: Current preferred method of eliciting symptoms of distress/depression

ICD10/DSMIV24%

Short QQ24%

Long QQ8%

Algorithm26%

Short QQ23%

ICD10/DSMIV0%

Cancer StaffIdeal Method (n=226)

Psychiatrists

Effective?

Comment: “Ideal” method of eliciting symptoms of distress/depression according to clinician

T2. Are We finding it?T2. Are We finding it?

How successful are we (routinely)?

Comment: Slide illustrates diagnostic accuracy according to score on DT

11.815.4

30.4 28.9

41.9 42.9 40.7

66.771.4

26.124.4

19.4 19.0

22.2 14.3

43.546.7

38.7 38.1

25.921.4

5.911.1

Zero One Two Three Four Five Six Seven Eight Nine Ten

Judgement = Non-distressedJudgement = UnclearJudgement = Distressed

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

Pre-test Probability

CHEMO+

CHEMO-

Baseline Probability

COMMU+

COMMU-

Detection sensitivity = 50.6%Detection specificity = 79.4%Overall accuracy = 65.4%.

Comment: Slide illustrates performance of chemotherapy vs community nurses in oncology T125 – Sat am

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27

GP Accuracy – Detection of Distress by GHQ ScoreGP Accuracy – Detection of Distress by GHQ ScoreMcCall et al (2007) Primary Care Psychiatry - Recognition by Severity

Comment: Slide illustrates raw number of people identified by severity on the GHQ. Although the % detection increases with severity, the absolute number decreased due to falling prevalence

Nine Ten

Eleven

Fiftee

Eighteen

Ninetee

Proportion MissedProportion Recognized

HADS-D

Testing Clinicians: A Meta-AnalysisTesting Clinicians: A Meta-Analysis

Methods (currently unpublished)

12 studies reported in 7 publications. 2 studies examined detection of anxiety, 8 broadly defined depression (includes HADS-T)3 strictly defined depression and 7 broadly defined distress.

9 studies involved medical staff and 2 studies nursing staff.

Gold standard tools including GHQ60, GHQ12 HADS-T, HADS-D, Zung and SCID.

The total sample size was 4786 (median 171).

Testing Clinicians: A Meta-AnalysisTesting Clinicians: A Meta-Analysis

All cancer professionalsSE =39.5% and SP =77.3%.

OncologistsSE =38.1% and SP = 78.6%; a fraction correct of 65.4%.

By comparison nursesSE = 73% and SP = 55.4%; FC = of 60.0%.

When attempting to detect anxiety oncologists managedSE = 35.7%, SP = 89.0%, FC 81.3%.

Presented at IPOS2009

GPs vs Oncologists vs NursesGPs vs Oncologists vs Nurses

Who is better?

Bayesian analysis

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

GP+GP-Baseline ProbabilityNurse+Nurse-Oncologist+Oncologists-

Comment: Doctors appear to be more successful at ruling-in or giving a diagnosis, nurses more successful at ruling out

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

Ave Confidence+

Ave Confidence-

Above Ave Confidence+

Above Ave Confidence-

High Confidence+

High Confidence-

Low confidence = more cautious, fewer false positives, more false negatives

High confidence = less cautious, more false positives, low false negatives

T3. Screening Tools in CancerT3. Screening Tools in Cancer

Clinician Opinion

Patient Opinion

Observation

Interview

Visual

Self-Report

DepressionScreening

VA-SES

HAMD-D17

PhysicalGeneral

Signs ofDS

CDSS#10

MADRAS10

Trained

ConfidentSkilledClinician

SMILEY

Clinicians Methods to Evaluate Depression

Unassisted Clinician Conventional Scales

Ultra-Short (<5) Short (5-10) Long (10+)Untrained Trained

Routine Implementation

Acceptability ?

Accuracy? Accuracy?

vsComment: schematic overview of methods to evaluate depression

example

Clinicians Methods to Evaluate Depression

Conventional Scales

Short (5-10) Long (10+)

HADS-D BDIexample example

Comment: This is a reminder of the structure of the HADS scale, this version adapter for cancer.

HADS – Pros vs ConsHADS – Pros vs ConsADVANTAGES DISADVANTAGES

HADS – Pros vs ConsHADS – Pros vs ConsADVANTAGES

Well knownShort (7 items)Well testedDepression & anxiety coveredSelf-report

DISADVANTAGES

Can be too longValidation stats not goodWhich version?Distress, anger, needs not coveredScoring complexHADS-t not recommendedRoyalty fee

Inadequate Data(n=11)

No data (n= 250)

No reference standard(n= 293)

Accuracy or Validity Analyses(n= 210)

HADS Validity Analyses(n=50)

HADS in CancerInitial Search (n= 768)

ScaleTypes

Sample Size (cases)

HADS-T(n=26)

HADS-D(n=14)

HADS-A(n=10)

Less than 30(n=22)

More than 100(n=8)

30 to 100(n=20)

Review articles (n= 16)

Depression(n=22)

Any Mental Ill Health(n=24)

Anxiety(n=4)

OutcomeMeasure

No interview standard(n=149)

Validity of HADS vs depression (DSMIV)Validity of HADS vs depression (DSMIV)

SE 71.6% (68.3)

SP 82.6% (85.7)

Prev 13%

PPV 38%

NPV 95%

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

HADS7v8+

HADS7v8-

Depression_HADS-d (7v8)

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

HADS7v8+

HADS7v8-

Depression_HADS-d (7v8)

British Journal of Cancer (2007) 96, 868 – 874

DistressThermometer

AnxietyThermometer

DepressionThermometer

AngerThermometer

TenNineEightSevenSixFiveFourThreeTwoOneZero

Comment: Slide illustrates scores on ET tool

ET - Table of Cut-PointsET - Table of Cut-Points

Distress Thermometer

Anxiety thermometer

Depression Thermometer

Anger Thermometer

Help Thermometer Cut-point

Insignificant 39.0 25.6 50.1 55.7 54.3 0,1

Minimal 20.1 22.5 18.3 13.6 15.4 2,3

Mild 16.9 16.5 12.2 10.5 12.2 4,5

Moderate 12.0 14.5 9.8 6.6 6.6 6,7

Severe 11.9 20.8 9.5 13.6 11.2 8,9,10

DepT23%

AngT18%

AnxT47%

Nil41%

Non-Nil59%

AnxT AngT

T4. How Valid Are the ToolsT4. How Valid Are the Tools

Validity of Methods to Evaluate Depression

Unassisted Clinician Conventional Scales

Ultra-Short (<5) Short (5-10) Long (10+)Untrained Trained

DT vs HADS-T Validity (n=660)DT vs HADS-T Validity (n=660)

SE SP AUC CUT

DT – 71.9% 78.4% 0.814 cut point >=4

AnxT – 75.7% 73.4% 0.821 cut point >=5

DepT – 77.6% 82.2% 0.855 cut point >=3

AngT – 77.5% 77.6% 0.823 cut point >=2

HelpT - 69.1% 80.8% 0.809 cut point >=3

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

HADSd+

HADSd-

HADS-T+

HADS-T-

HADS-A+

HASD-A-

Depression_HADS

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

1Q+1Q-Baseline ProbabilityDT+DT-2Q+2Q-HADSd+HADSd-HADS-T+HADS-T-BDI+BDI-EPDS+EPDS-HADS-A+HASD-A-

Depression_all

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

DT+ [N=4]DT+ [N=4]Baseline Probability1Q+ [N=4]1Q- [N=4]2Q+2Q-DT/IT+DT/IT-HADST+ [N=13]HADST+ [N=13]PDI+PDI-

Mitchell AJ. Short Screening Tools for Cancer Related Distress A Review and Diagnostic Validity Meta-analysis JNCI (2010) in press

Distress

Validity of DT vs depression (DSMIV)Validity of DT vs depression (DSMIV)

SE 80%

SP 60%

PPV 32%

NPV 93%

DT vs DSMIV DepressionDT vs DSMIV Depression

SE SP PPV NPV

DTma 80.9% 60.2% 32.8% 92.9%

DTLeicesterBW 82.4% 68.6% 28.0% 98.3%

DTLeicesterBSA 100% 59.6% 26.8% 100%

BSA = British South Asian BW= British White

T5. How to Choose A Cut-OffT5. How to Choose A Cut-Off

Distress ThermometerDistress Thermometer

Distress Thermometer – Pooled TableDistress Thermometer – Pooled Table

ScoreRansom 2006

Tuinman2008

Mitchell 2009

Lord 2010

Hoffman 2004

Gessler2009

Clover 2009

Jacobsen 2005 Sum

Proportion

Zero 68 38 61 123 14 27 65 71 467 18.4%

One 72 31 42 68 5 26 39 46 329 12.9%

Two 77 22 35 44 5 18 30 54 285 11.2%

Three 65 37 42 46 8 23 45 46 312 12.3%

Four 51 29 29 30 8 7 21 31 206 8.1%

Five 41 46 62 40 11 13 41 48 302 11.9%

Six 38 32 23 28 2 16 26 31 196 7.7%

Seven 36 21 23 38 2 15 32 16 183 7.2%

Eight 18 12 18 29 6 9 19 15 126 5.0%

Nine 16 5 8 14 3 3 13 9 71 2.8%

Ten 9 4 7 20 4 0 9 13 66 2.6%

Sum 491 277 350 480 68 157 340 380 2543

Proportion 19.3% 10.9% 13.8% 18.9% 2.7% 6.2% 13.4% 14.9%

Distress Thermometer – Pooled

Proportion

18 .4 %

12 .9 %

11.2 %12 .3 %

11.9 %

2 .8 % 2 .6 %

7.7% 7.2 %

Zero One Two Three Four Five Six Seven Eight Nine Ten

Insignificant SevereModerateMildMinimal

British Journal of Cancer (2007) 96, 868 – 874

SampleSample

We analysed data collected from Leicester Cancer Centre from 2008-2010 involving 531 people approached by a research nurse and two therapeutic radiographers.

We examined distress using the DT and daily function using the question:

“How difficult have these problems made it for you to do your work, take care of things at home, or get along with other people?”

“Not difficult at all =0; Somewhat Difficult =1; Very Difficult =2; and Extremely Difficult =3”

Dysfunction in 531 cancer patientsDysfunction in 531 cancer patients

Unimpaired Mild Moderate Severe

Unimpaired by DT ScoreUnimpaired by DT Score

10.00%

20.00%

30.00%

40.00%

50.00%

60.00%

70.00%

80.00%

90.00%

1 2 3 4 5 6 7 8 9 10 11

DepT23%

Distress69%

Dysfunction76%

26%28% 22%

Of the 293 Non-Nil

DysfunctionDistress

Mean DT Scores?Mean DT Scores?

Unimpaired Mild Moderate Severe

Mean DT Score 2.1 4.1 5.9 6.5

Std Deviation 2.54 3.0 2.56 3.59

Sample Size 296 182 39 14

Simplified DT Range* 0-3 4-5 6-7 8-10

DT distribution by ImpairmentDT distribution by Impairment

0 1 2 3 4 5 6 7 8 9 10

Typically severely impared

Typically mod impared

Typically mildly impared

Typically unimpared

None at all

Extreme and incapacitating

Very Severe and very disabling

Moderately Severe and disabling

Moderate and quite disabling

Moderate and somewhat disabling

Mild-Moderate and slight disabling

Mild but not particularly disabling

Very mild and not disabling

Minimal but bearable

Minimal and not problematic

None at all

Dt vs DysfunctionDt vs Dysfunction

ROC plot from Book 1

0.00 0.25 0.50 0.75 1.000.00

1.00Sensitivity

1-Specificity

Distress Thermometer(+ve), M(-ve)

Optimal Cut to Define Distress on DTOptimal Cut to Define Distress on DT

At a cut-off of 2v3 (>=3)Sensitivity =67.8%; PPV =60.3%; UI+ = 0.409Specificity = 68.9%; NPV = 70.3%; UI- = 0.484

At a cut-off of 3v4 (>=4)Sensitivity =58.9%; PPV =65.6%; UI+ = 0.386Specificity = 75.9%; NPV = 70.3%; UI- = 0.534

At a cut-off of 4v5 (>=5)Sensitivity =50.9%; PPV =67.85; UI+ = 0.345Specificity = 81.1%; NPV = 67.9%; UI- = 0.55

T6. Screening in Cancer: ImplementationT6. Screening in Cancer: Implementation

Clinician Opinion

Patient Opinion

Comment: Slide illustrates actual gain in meta-analysis of screening implementation in primary care

ScreenRoutine vs At-Risk vs Identified

Low High

Follow-up Care

?? Desire for Help

Meetable Unmet Needs

800 Patients Approached

100 Not Willing (13%) 700 Patients Willing (87%)

500 Staff Willing (71%)TAU

402 Data Collected (80%)Screen Data

Leicester: DT/ET ImplementationLeicester: DT/ET Implementation T177 t680

Pre-Post Screen - DistressPre-Post Screen - Distress

Before After

Sensitivity of 49.7% 55.8% =>+5%

Specificity of 79.3% 79.8% =>+1%

PPV was 67.3% 70.9% =>+4%

NPV was 64.1% 67.2% =>+3%

There was a non-significant trend for improve detection sensitivity (Chi² = 1.12 P = 0.29).

Qualitative Aspects: CommunicationQualitative Aspects: Communication

DISTRESS

43% of CNS reported the tool helped them talk with the patient about psychosocial issues esp in those with distress

28% said it helped inform their clinical judgement

DEPRESSION

38% of occasions reported useful in improving communication.

28.6% useful for informing clinical judgement

2x2 Clinician Help Table : ACTUAL HELP2x2 Clinician Help Table : ACTUAL HELPClinician thinks:Unmet Needs

Clinician thinks no Unmet Needs

Patient Says:Help Wanted

=> Intervention => Low grade

Patient Distressed => Intervention =>??

Patient Not distressed orHelp Not Wanted

=> Monitor? => discharge?

2x2 Clinician Help Table : ACTUAL HELP2x2 Clinician Help Table : ACTUAL HELP

Clinician thinks:Unmet Needs

Clinician thinks no Unmet Needs

Patient Says:Help Wanted (60)

Helped 21/35 (60%)

Helped 11/23(48%)

Patient Distressed

Helped 65/102(63%)

Helped 31/62(50%)

Patient Not distressed orHelp Not Wanted

Helped 8/35(23%)

Helped 20/117(17%)

b. Intervention and helpb. Intervention and helpPREDICTORS

1. patient desire for help

2. number of unmet needs

3. clinicians confidence

4. patient reported anger

RCT using DT Carlson et al 2010RCT using DT Carlson et al 2010

Screening for Distress in lung and breast cancer outpatients: A randomized controlled trial Linda Carlson Tom Baker Cancer Centre, University of Calgary

1) Minimal Screening: the Distress Thermometer (DT) [n=365]

2) Full Screening: DT, Problem Checklist, Psychological Screen for Cancer (PSSCAN) [n=391] a personalized report

3) Triage: Full screening plus optional personalized phone triage [378]

Advanced AspectsAdvanced Aspects

Algorithms

Structured interviews

Computerized testing

Item-banking

Screening in subgroups

T7. ExtrasT7. Extras

Unfiled

Cancer Population

CNS Assessment

Possible case

Depression

Screen #1+ve

n = 200 No Depression

Sp 55%

Se 70%

n = 800

N = 1000

TP = 140

FP = 360Probable Non-Case TN =440

FN = 60

PPV 28% NPV 88%

Screen #1-ve

YieldTP = 140

TN = 440

FN = 60

FP = 360

NPV 88%

PPV 28%

Sp 55%

Se 70%

Cancer Population

CNS Assessment

Possible case

Depression

Screen #1+ve

n = 200 No Depression

Sp 55%

Se 70%

n = 800

N = 1000

TP = 140

FN = 60

PPV 28%

Oncologist Assessment Sp 80%

Sp 40%

NPV 88%

Probable Depression TP = 56

FN = 84

PPV 44% NPV 77%

Screen #1-ve

Screen #2+ve

Cumulative YieldTP = 56

TN = 728

FN = 144

FP = 72

NPV 83%

PPV 44%

Sp 91%

Se 28%

Credits & Acknowledgments

Elena Baker-Glenn University of NottinghamPaul Symonds Leicester Royal InfirmaryChris Coggan Leicester General HospitalBurt Park University of NottinghamLorraine Granger Leicester Royal InfirmaryMark Zimmerman Brown University, Rhode IslandBrett Thombs McGill University CanadaJames Coyne University of PennsylvaniaNadia Husain University of Leicester

For more information www.psycho-oncology.info

[Workshop] Implementation of screening (Oct10)

Health & Medicine

Corporate Presentation Oct10 Infoedge

Risalah Oct10,

HES UserGuide Oct10[1]

Oct10 Spreads Magazzine

Waters Risk Compliance Oct10 Web

CD23 Oct10 Crosstabs

Lifeline policy summary oct10

GM Customer Specifics Oct10

Yahoo digital citizenship oct10

Portal Issue3 Sept-Oct10[1]

Anabatic brief profile-oct10

Implementation of Universal Screening

Tidings Oct10

bulletin oct10 2021

Poultry Farm Oct10

ISMB2014 slides Oct10

CD25 Oct10 Crosstabs

Blb next oct10

ITP Notes Oct10

Smart grid oct10 sso