Risk adjustment and other reporting issues

Shalini Santhakumaran

NDAU Statistician

Imperial College London

Why report outcomes ?

Improvements in neonatal care and outcomes

Audit

Comparison

Commissioning Accountability

Transparency

Data Issues

Completeness

Accuracy

Data Issues

Completeness

Accuracy

Agreed case definitions

Data Issues

Completeness

Accuracy

Agreed case definitions Neonatal transfers

Analysis issues

Unadjusted data is useful for some purposes

For others a more detailed analysis is required

Analysis issues

Unadjusted data is useful for some purposes

For others a more detailed analysis is required

3 reasons for variation in outcome:

Analysis issues

Unadjusted data is useful for some purposes

For others a more detailed analysis is required

3 reasons for variation in outcome:

1. Differences in case-mix

2. Random variation

3. Differences in care provided

Analysis issues

1. Differences in case-mix

2. Random variation

3. Differences in care provided

Selection of variables

Use of appropriate statistical models

Cannot completely control for case-mix

Survival Probability Calculator

Survival Probability Calculator

Survival Probability Calculator

Analysis issues

1. Differences in case-mix

2. Random variation

3. Differences in care provided

Occurs by chance even if the underlying mortality rate is the same for all providers

Illustrate significance using funnel plots

Funnel plot for adjusted SMR

95% confidence interval limits = “warning”99.8% confidence interval limits = “alarm”

● = complete networks

○ = incomplete networks

Analysis issues

1. Differences in case-mix

2. Random variation

3. Differences in care provided

Not necessarily due to good/poor performance

‘Constant risk fallacy’ and other local effects

Cannot tell us whether deaths were preventable

Needs to be linked to process measures

The National Neonatal Database

Individual, not aggregated

Population-based

Detailed clinical record

Electronic

Collaborative access to denominator data

The National Neonatal Database

Individual, not aggregated

Population-based

Detailed clinical record

Electronic

Collaborative access to denominator data

…ideal for reporting outcomes

Acknowledgements

All neonatal units contributing to the NDAU

NDAU Team

NDAU Steering BoardJane Abbott (BLISS) Jacquie Kemp

Prof. Peter Brocklehurst Prof. Azeem Majeed

Prof. Kate Costeloe Prof. Neena Modi

Prof. Liz Draper Prof. Andrew Wilkinson

Imperial College London Academic Neonatal Medicine Unit