What do screeners refer for checking? Allan Wilson Lead Biomedical Scientist in Cellular Pathology Advanced Practitioner in Cervical Cytology BAC 24th

What do screeners refer for checking?

Allan WilsonLead Biomedical Scientist in Cellular

PathologyAdvanced Practitioner in Cervical Cytology

BAC 24th October 2013

Plan for today

• What do screeners refer for checking?

• Monklands audit of checkers• National survey of checker

role• Interactive session


Screeners referral decisions

• What do screeners refer to checkers?

• What influences screeners referral patterns?

• How do we monitor referrals?• Can we influence over-referral

and under-referral?


Why do screeners refer slides for checking?

• Marked inflammation ?dyskaryosis• Bland nuclear enlargement• Atrophy• ?Koilocytes / HPV / dyskeratosis• ?GA / ?Endometrium• ?Small cells• ?degenerate only• Local issues


Some ground rules.....

• Images are representative of slides• All slides changed from BNA to

negative by checkers are double screened

• All from Thinprep slides• All stained using imager stain• Limited biopsy confirmation • Our opinions on BNA Negative

could be wrong......


Why do screeners refer slides for checking?• Marked inflammation







Why do screeners refer slides for checking?• Bland nuclear enlargement

– ?perimenopausal or post menopausal changes.







Why do screeners refer slides for checking?• Atrophy





Why do screeners refer slides for checking?• ?HPV / ?Koilocytes



3a





Why do screeners refer slides for checking?• ?glandular abnormality / ?

endometrium







•Immature squamous metaplasia




Why do screeners refer slides for checking?• ?degenerate only



2aBAC 24th October 2013


• Check small cells ?high grade







• Other oddities........



What influences screeners referral patterns?

• Confidence levels• Personal issues• Fatigue• Previous performance• Ability• Training• Post –invasive audit or review


Over referral v Under referral (false +ve v false –ve!)

• We tend to notice over referral more quickly as it has a “nuisance” value!

• Under referral is potentially more dangerous and more difficult to detect

• Usual balance between sensitivity and specificity


How do we monitor referrals?

• Gut feeling?• Worksheets• Spreadsheets• LIMS• Do we have any idea of what a

“normal” referral rate is?• What should we do if a referral

rate is deemed to be too low?


How can we influence screener referrals?

• Feedback!• More feedback!• Training• Setting targets, self audit• Selected slide reviews• Don’t just live with over or under

referral


Monklands Checker audit


What is a checker?

• Poorly defined• How do we appoint checkers?• How do we train?• How do we assess competence?• Variable role• Often overstretched (HPV roles)


Monklands checker survey

• Four checkers (13-25 years experience)

• Asked to record categories of slides and numbers called negative or referred over an approximate 3 month period

• Reflective of rota system in operation


Summary of results

• 348 slides in audit• 56 referred to medic/AP as BNA+

•36 BNA•16 Low grade dyskaryosis•3 High grade (moderate)•1 severe dyskaryosis ?invasive

• 52 reported as BNA+ by medic/AP• 4 reported as negative


Monklands checker survey (2013)

Checker Slides checked

Signed out Referred to AP/medic

1 174 140 (80%) 34 (20%)

2 82 70 (87%) 12 (13%)

3 60 54 (90%) 6 (10%)

4 32 28 (87%) 4 (13%)

Total 348 292 (84%) 56 (16%)


Monklands checker survey (2006)

Checker Slides checked

Signed out Referred to AP/medic

1 72 49 (68%) 23 (32%)

2 132 59 (44%) 73 (56%)

3 86 60 (69%) 26 (31%)

4 87 67 (77%) 20 (23%)

5 105 89 (84%) 16 (16%)

Total 482 324 (67%) 158 (33%)

Monklands checker survey

• Can we suggest referral or signing out targets based on these figures?

• Are there any outliers?• We simply don’t know if checker 2 is

cautious (or lazy!) or checker 5 is confident (or dangerous!)

• We need to do more work over a longer period and involve more labs


What type of slides prompted most referrals?

Slides checked

Signed out Referred to AP?Medic

Inflammation 160 134 (84%) 26 (16%)Nuc. enlargement 84 63 (74%) 21 (26%)Atrophy 4 4 (100%) 0?HPV 13 9 (70%) 4 (30%)?GA 21 19 (90%) 2 (10%)Check small cells 30 27 (90%) 3 (10%)?degenerate 36 34 (94%) 2 (6%)TOTAL 348 290 (83%) 58 (17%)


Screener analysisScreener referred Reported as

BNA+1 28 8 (29%)2 56 20 (35%)3 24 2 (8%)4 76 12 (16%)5 6 06 82 8 (10%)7 6 4 (66%)8 16 09 28 0

10 26 0

What can we learn from this?

• Use to focus training/updates/review sessions

• Reduce inappropriate referrals and encourage appropriate referrals

• Invasive and HG audit suggests no issue with checkers reporting referred slides

• Possible link between low referral rates and low high grade sensitivity.......


National survey on checker role


Who replied?

• Web based survey on BAC website• Survey Monkey• 98 responses• 76 laboratories• 85% of UK cytology labs


Health warning on analysis

• Duplicate entries from same lab• How do you select the “correct

response”• Multiple responses when only one

choice requested• Accuracy of responses• The “doomed” lab


Cervical Cytology labs in the UK

• England 71• Northern Ireland 4 • Scotland 9• Wales 4

•Total 88


Do you have staff who perform a Checker role?

• Yes: 75 (99%)• No: 1 (1%)


Which statement best describes their role in your department

• They screen all abnormal slides referred by primary screeners

•30 (39%)• They only screen difficult slides, ?

BNA’s, “please check dots” cases•10 (13%)

• Perform both of the above roles•43 (57%)


Do your checkers also primary screen?

•Yes: 69 (91%)•No: 7 (9%)


Do you set a minimum number of slides that they must screen?

•Yes: 38 (50%)•No: 31 (41%)•No response: 7 (9%)


If you do set a minimum figure, what is that figure?

• Wide range of responses from 20 to 3750!

• 34 labs did not respond• Top three responses were:

•3000 15 (20%)•750 8 (11%)•1000 6 (8%)


Do your checkers participate in rapid review/preview?


•


Do you monitor the performance of checkers?



If you do monitor checkersperformance what do you measure?

• Workload•48 (63%)

• Number of referred abnormal slides from screeners that they agree/disagree with

•22 (29%)• Number of referred abnormals from

checkers to consultant that are agreed/disagreed with

•26 (34%)


Does your laboratory require a secondopinion on slides where the checker has changed the primary screener report from abnormal to negative?

•Yes: 59 (74%)•No: 17 (22%)


If you do is it for:

• All such slides•31 (41%)

• For High Grade changes only•26 (34%)

• No response•19 (25%)


IQC of checkers

If you give a checker a ?small cell slide, how would you know if one checker consistently called them negative and one consistently called them severe dyskaryosis?


Audit trails and CPA………

CPA insist that we record who embeds tissue, who cuts sections and what batch number of Pap stains we use but do we record every opinion given on every slide?


Over view of survey (1)

• Most labs use checkers to screen all abnormals and “? BNA’s”

• Most checkers also primary screen and participate in IQC

• Less than half of responding labs monitor the rate of slides changed from BNA to negative


Over view of survey (2)

• A third of labs do not get a second opinion on slides changed from BNA to negative

• 41% of labs set no minimum workload level

• Wide variation among those labs that do set a minimum workload level


Conclusions

• Support and recognition of checker role

• Ensure opinions are recorded• Consider setting targets based on

local practice and monitoring• A minimum check should be a

double screen of all slides reported as BNA and above by screener but changed to negative by checker


Conclusions (2)

• Review monitoring systems for screener referral and checker reporting

• Monitor screener referral rates (especially low referral rates)

• Monitor BNA “turnaround” rates by checkers

• National guidance on referral rates???• We cannot (and never will) agree on BNA!


Interactive session










And finally........



•Thank you for listening

•Any Questions?


Documents

What do screeners refer for checking? Allan Wilson Lead Biomedical Scientist in Cellular Pathology Advanced Practitioner in Cervical Cytology BAC 24th