National Cataract Audit. Data Validation... · Eliminate poor practice Use scale to reduce personal...

National Cataract Audit

National Cataract Audit

Why?

• Quality assure NHS services Show surgeons / centres their results

Learn from the best

Eliminate poor practice

Use scale to reduce personal burden of audit on individuals

Risk adjust to avoid penalising surgeons who take on difficult cases

Cataract Audit participation

Looking ahead – 114 of 125 NHS Centres signed up

4

2018 Report

• Surgeon and Centre results on NOD website

– 1918 Surgeons invited to check their latest results

• 92 Centres being reported

– Majority Medisoft

– 4 OpenEyes

– 7 Other

• 181,965 Operations

6

Is all this really worth the effort?

PCR % RATE 2010-11* 2011-12 2012-13 2013-14 2014-15 2015-16 2016-17 Overall

N = 64,923 N = 82,879 N = 89,602 N = 100,333 N = 107,255 N = 148,073 N = 182,064 N = 775,129

Consultant surgeons

(485,757) 1.55 1.48 1.46 1.33 1.27 1.18 1.10 1.28

Career grade surgeons

(88,272) 1.82 1.64 1.78 1.25 1.69 1.29 1.54 1.55

More experienced

trainee surgeons

(171,458)

2.87 2.65 2.41 2.33 2.21 2.17 1.95 2.29

Less experienced

trainee surgeons

(29,642)

3.66 3.62 3.02 2.35 2.57 2.53 2.33 2.71

Overall 2.00 1.85 1.77 1.56 1.56 1.46 1.38 1.59

Overall 30% reduction in PCR rate from 2010-11 to 2016-17 *Years run from Sept to August

Data Validation and Data Completeness Overview and Data Entry

Professor John Sparrow,

Consultant Ophthalmologist and NOD Audit Clinical Lead

Data Completeness

• Number of cases reported – Recent joiners acknowledged as such in report (date)

– Comparison between submitted data and NHS digital (pro-rata)

• Complications – 100% PCR or some other complication or none

• VA Pre- & Post-op – Variable between centres

• Risk indicators data – PCR

– VA Loss (doubling or worse of visual angle)

– Important for case complexity adjustment

Number of Cases and Estimated % Completeness

Data from 2015-2016 Report https://www.nodaudit.org.uk/

Data Completeness

Number of cases reported – Recent joiners acknowledged as such in report (date)

– Comparison between submitted data and NHS digital (pro-rata)

• Complications – 100% PCR or some other complication or none

• VA Pre- & Post-op – Variable between centres

• Risk indicators data – PCR

– VA Loss (doubling or worse of visual angle)

– Important for case complexity adjustment

Data Completeness

Number of cases reported – Recent joiners acknowledged as such in report (date)

– Comparison between submitted data and NHS digital (pro-rata)

Complications – 100% PCR or some other complication or none

• VA Pre- & Post-op – Variable between centres

• Risk indicators data – PCR

– VA Loss (doubling or worse of visual angle)

– Important for case complexity adjustment

% Cases with Missing Pre-op VA by Centre

% Cases with Missing Post-op VA by Centre

Data Completeness

Number of cases reported – Recent joiners acknowledged as such in report (date)

– Comparison between submitted data and NHS digital (pro-rata)

Complications – 100% PCR or some other complication or none

VA Pre- & Post-op – Variable between centres

• Risk indicators data – PCR

– VA Loss (doubling or worse of visual angle)

– Important for case complexity adjustment

Data Completeness

Number of cases reported – Recent joiners acknowledged as such in report (date)

– Comparison between submitted data and NHS digital (pro-rata)

Complications – 100% PCR or some other complication or none

VA Pre- & Post-op – Variable between centres

• Risk indicators data – PCR

– VA Loss (doubling or worse of visual angle)

– Important for case complexity adjustment

% Cases with Specific Ocular Pathologies

% Cases with Ocular Pathology

Data Completeness

Number of cases reported – Recent joiners acknowledged as such in report (date)

– Comparison between submitted data and NHS digital (pro-rata)

Complications – 100% PCR or some other complication or none

VA Pre- & Post-op – Variable between centres

Risk indicators data – PCR

– VA Loss (doubling or worse of visual angle)

– Important for case complexity adjustment

Data Validation

• Contributors invited to check

– Number of cases reported

– Adverse outcomes

• PCR

• VA Loss (needs both Pre & Post-op VA)

– Risk indicators data

• For case complexity adjustment

Known

Number cases done % PCRs

Unadjusted data 4503 2.07% 93

Benchmark = 1.10%

Adjusted% = 1.25%

Adjusted% = Benchmark * ObservedN / ExpectedN

ExpectedN (from case complexity) is unknown

ExpectedN = Benchmark * ObservedN / Adjusted% PCRs

ExpectedN = 82

ObservedN is higher than expectedN, therefore

adjusted rate of 1.25% is higher than benchmark of 1.10%

Conclusion:

Centre's recorded case complexity is overall less complex

than the average complexity from all centres

Case Complexity Centre Example

Known

Number cases done % PCRs

Unadjusted data 606 1.65% 10

Benchmark = 1.10%

Adjusted% = 0.92%

Adjusted% = Benchmark * ObservedN / ExpectedN

ExpectedN (from case complexity) is unknown

ExpectedN = Benchmark * ObservedN / Adjusted% PCRs

ExpectedN = 12

ObservedN is lower than expectedN, therefore

adjusted rate of 0.92% is lower than benchmark of 1.10%

Conclusion:

Surgeon's recorded case complexity is overall more complex

than the average complexity from all surgeons

Case Complexity Surgeon Example

Data Completeness

Number of cases reported – Recent joiners acknowledged as such in report (date)

– Comparison between submitted data and NHS digital (pro-rata)

Complications – 100% PCR or some other complication or none

VA Pre- & Post-op – Variable between centres

Risk indicators data – PCR

– VA Loss (doubling or worse of visual angle)

– Important for case complexity adjustment