Audit Made Simple

8/6/2019 Audit Made Simple

1/13

Produced in partnership by Smile-on and Denplan

Clinical Audit Made Simple IIA step-by-step guide through eight audits for dental practices


2/13

3

Introduction 4

Using this package 6

Audit requirements and funding in the

General Dental Services 7

What is audit and why should I do it? 8

Choosing a topic and setting a standard 10

The audits 11

Antimicrobial prescribing Periodontal record keeping

Patient waiting times

Quality of radiographs Crown assessments

Emergency appointments

Record keeping The success of endodontic treatment

Recording, sampling and analysing data 19

Implementing changes and re-auditing 22

Useful contacts and resources 23

Acknowledgements: Diana Scarrott and Mike Mulcahy

2005 Denplan Ltd, Smile-on Ltd

All rights reserved. No part of this publication may be reproduced, stored in a retrieval system or

transmitted in any form or by any means, electronic, mechanical, photocopying, recording or

otherwise without the prior permission of the copyright owner.

This booklet and the accompanying CD-Rom are copyright. The materials are intended for the

use of the original purchaser alone in his/her practice for the purpose of personal or team clinical

audit activity. Electronic materials are copy-protected.

Contents


3/13

54

Dr Jacob Bronowski (famed for his TV series "The Ascent of Man")

once commented that the success of the human race lay in the fact

that humans are "wired to learn". Watch a baby trying to walk, a child

learning to ride a bike, an undergraduate dental student trying to

record the perfect alginate impression - they have a lot in common.

We all learn from our mistakes, from

what goes right and even more

from what goes wrong. It was Sir

Humphry Davy, in the early 19th

Century, who famously noted: I

thank God I was not made a

dextrous manipulator; the most

important of my discoveries have

been suggested to me by my

failures. Today, we would call it

experiential learning.

It is easy to believe that any skill

once learned especially a complex

one can be dismissed as

completed. But as professionals,we have a duty to our patients and

ourselves to constantly improve our

skills. I can ride my bike to the

shops, but if I needed the skills of a

professional stunt rider, I would

have to spend many hours a day

practising.

The outcome of this improvement is

not only highly satisfying for the

professional, but more rewarding for

the client too.

Two recognised ways of improving

professional skills are clinical audit

and peer review. For a definition of

how these two approaches differ

see the table opposite.

Peer Review

A joint approach to

improvement, involving a

number of (usually 4 8)

dentists

Has a Convenor, who organises

the group

May include a review or

discussion of clinical approaches

to a particular problem

A collaborative audit, where the

participating dentists measure

their outcomes separately

Lasts no more than 6 months

(to receive funding)

Has a definable objective

Clinical Audit

Usually involves a single clinician

or a clinical team (practice staff)

Has a set objective, problem or

area to look at

Starts off with setting a standard

what ought to happen

Measures actual practise

Includes researching forreferences or expert views

Analyses actual practice

performance to check for

shortfalls

Lasts no more than 4 months (to

receive funding)

Comes to a conclusion, or

measures again when new

systems have been introduced

Is a continuous spiral of

improvement

"Audit" sounds cold and off-putting, like "tax returns" or "management

accounts", but these activities are absolutely essential to the success ofmodern business. Carl Sewell, highly regarded for his work into customer

care in the United States, regards the need to measure everything as one of

the 5 tenets of retaining satisfied customers for life.

Introduction

Clinical Audit INTRODUCTION


4/13

76

'Clinical Audit Made SimpleII' has been designed to address the

needs of the busy general dental practitioner. This easy-to-use tool

will help you undertake eight audits and facilitate the entire process.

If you are already familiar with audit, use this package to save time

and get more out of your audit activities.

This package consists of this guide

and a CD-Rom or online course that

includes eight 'cook-book' audits.

From the easiest to the most

difficult, the audits provided are:

Antimicrobial prescribing: are you

prescribing consistently?

Radiographs: audit is a regulatory

requirement

Waiting times: how long do your

patients wait?

Periodontal monitoring: how

effective is your periodontal

therapy?

Crown assessments: how do

crowns from different labs

compare?

Endodontics: what causes failure?

Emergency appointments: how

can they be accommodated more

easily?

Record keeping: are your patient

records complete?

With each audit comes a brief for

the audit leader, a brief for the team,

data collection forms and analysis

tools, typically a Microsoft Excel

spreadsheet.

The spreadsheet accompanying the

crown audit is designed to allow

you to audit other topics.

We hope you will enjoy using this

package.

It is an NHS Terms of Service requirement for UK dentists to carry

out 15 hours of peer review or clinical audit activity over a three-year

period. This requirement is intended to improve dental services for

patients by improving the standards of care.

Much of the current audit and peer

review funding system is going to

change. At the time of publication,

you are entitled to payment for your

audit activity. To claim, you will need

to submit forms to your Local

Assessment Panel (LAP). You

should contact your LAP before

starting your audit. We have

provided all the required forms and

contact details for your LAP on the

CD-Rom/web course.

Different forms are used in England,

Wales, Scotland and Northern

Ireland. If you live in Scotland,

please consult your Health Board or

Local Dental Committee for advice

on payment for audit. In Northern

Ireland, contact the Central

Services Agency.

Using this package Audit requirements and funding

in the General Dental Services


5/13

98

Audit has been defined as The systematic, critical analysis of the

quality of dental care, including the procedures and processes used

for diagnosis, intervention and treatment, the use of resources and

the resulting outcome and quality of life as assessed by both

professionals and patients. But it's really just about measuring andimproving our own working practices.

Audit is an NHS Terms of Service

requirement for UK dentists (see

'Audit in the GDS'). This

requirement was introduced in May

2001 to improve dental services for

patients by improving the standards

of care. A few dentists are leading

the way, but a survey of GDPs

published in the British Dental

Journal1 showed that:

6% had a reasonable

knowledge of audit

16% had attended a course on

the subject

63% thought the introduction

of audit was a step in the

right direction

39% thought audit would lead

to an improvement in

standards

43% wanted further training

in the subject

Even if you are not subject to the

NHS requirement to conduct audit

or peer review, improving patient

care is a powerful reason to do it.

Clinical audit is also recognised by

the General Dental Council as

verifiable continuing professional

development. It can, therefore, also

help you to fulfil your CPD

requirements.

The Commission for Healthcare

Improvement (CHI) has stated, "We

can no longer think about

effectiveness of care as an isolatedprofessional matter. There have been

too many instances of highly

publicised failures of healthcare for

that attitude to remain, and clinical

governance - the pursuit of the

organisational approach to quality -

has become a professional

imperative."

This approach looks straightforward, but there are pitfalls along the way.

Using this guide and the provided cook-book audits will help you to get started.

1 Mercer PE Long AF Ralph JP.Audit Activ ity and Uptake Br Dent J 184(3) 158 - 1622Principles for Best Practice in Clinical Audit, National Institute for Clinical Excellence:Radcliffe Medical Press 2002.

CHI points out that audit is at the heart of clinical governance because:

It is a method of reviewing the quality of everyday care to patients

It builds on the existing tradition of healthcare professionals reviewing

notes and cases in an attempt to improve their care

As an approach, it is systematic, clear and reliable

It can confirm high quality, or indicate areas for improvement

Audit is useful because it is easy to apply in principle. It follows a spiral

of activity with six simple steps:

What is audit and why should I do it?

Clinical Audit WHAT IS AUDIT AND WHY SHOULD I DO IT?


6/13

1110

The first stage of audit is to select a topic.When choosing a topic it

is important to select a subject that is quite straightforward and

simple to measure and review.The cook-book audits we've provided

vary in their difficulty. If you're new to audit, start with our audit of

antimicrobial prescribing.

You should avoid, as far as possible,

topics that require a considerable

amount of arbitrary assessment. Its

also important to choose a topic that

will really tell you something useful

about your practice. You don't have

to focus on clinical topics.

Choose a topic that you can actually

improve. It may seem unnecessary

to point out, but auditing something

like the incidence of malocclusions

or geographic tongue is not really

going to lead anywhere. Also try to

choose a topic that can be subject

to a standard. You can find external

standards from fully researched

scientific literature or expert groups

such as the British Dental

Association, or Faculty of General

Dental Practitioners (UK) 3. But it is

also acceptable, and even

encouraged, to set internal practice

standards. If setting your own

standard, it should be reasonable

and should stretch you to encourage

improvement.

Remember that there are

differences between criteria and

standards. A criterion is a condition

that should be satisfied (e.g.

patients should wait no longer than

15 minutes for an appointment).

A standard expresses how often, or

how consistently that criterion is

satisfied (e.g. at least 90% of

patients should be seen within 15

minutes of arriving).

Criteria may be set by you in the

absence of an authoritative external

consensus. Criteria are based on

standards of evidence-based care.

The web is a particularly rich

source for finding criteria and we've

given other sources at the end of

this guide.

Your local audit facilitator 4 may be

able to help if you have problems in

setting a standard or the criterion.

This simple review introduces the concepts of audit and enables

you to assess the prescribing pattern for antimicrobial usage in

your practice.

It helps to indicate the appropriate

or inappropriate usage of

antimicrobials and suggests sources

for obtaining up to date information,

based on the Faculty of General

Dental Practitioners guidance.

By reviewing a selection of patients

for whom you have prescribed in

recent months (using records

selected at random from your files),

this audit gives an overview of your

prescribing patterns and helps you

to identify any patterns that might

conflict with current advice.

Data collection is made easy with

the provided spreadsheets, which

display your results in a format easyto share with the dental team. It

also enables you to compare

prescribing patterns with colleagues

in other practices.

The accompanying instruction

sheets clearly describe how the

whole practice can become

involved in the audit process and

how a supportive environment can

be created for the discussion of

individual prescribing patterns.

3See page 23 for details of likely sources of standards4See page 23 for details of contacting your facilitator

Choosing a topic and setting a standard The audits:

Antimicrobial prescribing


7/13

1312

Periodontal status and any pathology or risk factors should be

recorded for all dentate patients at each examination or at least

annually. This audit, carried out either retrospectively or currently

on patients seen over a period of time, indicates the periodontal

data collected and related information such as age and risk factors.

Breaking down the data into basic

periodontal examination (BPE)

scores by sextant and scoring also

for risk factors, the analysis

performed by the "intelligent

spreadsheet" allows you to identify

a number of profiles for your

patients. Your findings may reveal

previously unknown links and will

certainly provide the basis for a

team discussion about effectively

targeting and providing care for

these patients.

One powerful suggestion that may

come from your audit results is torethink how the practice targets its

oral health promotion messages. Is

smoking or alcohol consumption an

issue amongst your periodontally

involved population? Is there a

strong age or gender bias? Is there

a link to coronary artery disease?

Repeating the audit after a period

will also enable you to track any

changes in BPE that take place as

a result of changes you might

implement.

Keeping patients waiting on a regular or frequent basis is wasteful

of their time and yours! It increases anxiety for many patients and

creates management issues within the dental team.

This audit can involve all members

of the team and is a prospective

exercise that can be carried out

within two to three weeks in most

practices.

By collecting data on patient waiting

times and the time allocated for

each patient within the surgery, the

audit helps to identify any

inaccuracies in allocating time for

clinical procedures, which could be

delaying successive appointments.

The outcomes from the audit help

to clearly identify what remedial

action can be taken and can form

the basis for a "whole team"

discussion on time management

leading to "quick wins" for the

practice in improving efficiency and

customer service.

Repeat audits can be undertaken

periodically to enable the whole

practice to see what benefits can be

obtained and to maintain a "spiral"

of increasing standards.

The audits:

Periodontal record keeping

The audits:

Patient waiting times


8/13

1514

Undertaking a review of radiographic quality is often one of the first

formal audits undertaken in a dental practice. Equally, this audit

can be used as a periodic check on the radiography standards that

are being achieved.

A basic standard is introduced

(based on definitive guidance from

the Faculty of General Dental

Practitioners Selection Criteria in

Dental Radiography). Your standard

can then be incrementally

increased for subsequent audits as

a means of continually improving

the quality of films.

This audit can involve different

members of the dental team and

can be a valuable learning aid for

those involved in the taking and

processing of radiographs. The data

collection spreadsheet helps to

provide both detailed ratings for

different aspects of the process,

such as angulation, exposure and

process quality. It gives an overall

score for each film and the data

display shows which aspects of the

procedure could benefit from

improvement.

The data sheet provides for a short

pilot audit of 20 films and can be

expanded to include up to 250

radiographs for longer audits. It is

worth checking with your Local Audit

Facilitator how many films should be

included in an audit submitted for

funding under the GDS.

This slightly more complex audit is set up for comparison of the

quality of different types of crowns provided by different laboratories.

The quality of the crown is assessed in terms of factors such as

marginal fit, occlusion, patient acceptability and so on.

The variables listed in the different

headings can be over-written in the

data collection and analysis

spreadsheets and in this way, this

audit structure can be used to

review almost any type of treatment

or service provided and to measure

its acceptability, or the extent to

which it meets different criteria.

The data display enables

comparisons to be made between

any of the variables, so for

instance, it could be used for

restorations using different

materials, for surgical procedures

or even for staff appraisal.

As with any of the audits in the

"cook book", multiple downloads of

the data sheets can be made,

allowing for repeat audits or for

varying any of the headings or fields.

Although a number of factors

affecting outcome are included,

which are drawn from scientific

reports in the literature, individual

variables can be substituted to

permit different analyses to be

carried out.

The audits:

Quality of radiographs

The audits:

Crown assessments


9/13

1716

The emergency patient can pose a risk to effective time-

management in even the best-organised practice. Patients who

re-attend with urgent problems may impact on the time allocated

for routine procedures, and the cost in terms of patient acceptance

may also be significant.

This audit helps the practice to

record emergency visits over a

period of time and to explore the

data analysis to uncover any

underlying trends. The numbers of

patients attending on different days

or at different times may allow for

better planning, or alternatively an

analysis of the presenting

conditions may reveal areas for

potential improvement in clinical

treatment, case selection or the

management of chronic conditions.

Again, the analysis headings can be

modified as required to meet the

needs of different practices, and

multiple contemporary audits may

reveal differences in clinical

performance or time allocation for

emergencies between staff.

Good record keeping is at the heart of good quality clinical care. As

well as permitting efficient case management and alerting the

practitioner to inherent risk factors such as medical history, good

records promote good diagnosis based on thorough examination

findings. Not least, excellence in record keeping is established as aprimary factor in the avoidance of medico-legal issues.

Based on ideal criteria set out in

the Faculty of General Dental

Practitioners guide Examination

and Record Keeping, as well as the

advice of defence bodies, this

audit encourages a review of a

selection of patient records and

makes it easy to see where any

deficiencies may lie.

Repeat audits can be carried out at

intervals to enable monitoring of

improvements.

Many of the criteria can be

assessed by other members of the

dental team and from the

randomised selection of records for

review, to the final discussion of

the results, there is much to be

gained for all practice members in

auditing this fundamental aspect of

good dental care.

The audits:

Emergency appointments

The audits:

Record keeping


10/13

1918

Before you embark on data collection, make time to discuss the

audit project with your team. Trying to fit in a few minutes to

discuss the project at the end of a busy surgery session will not

be helpful. Team involvement helps to keep everyone motivated

and to work within agreed time limits. Also, allow adequate time toconduct the audit. Set a time limit for collecting data, analysing it

and making improvements.

Forms are provided to help you

collect data for the audits on this

package. Generally speaking, data

collection is easiest if you prepare

forms for the task, which can then

be carried out by any team member.

Getting the team to help with data

collection makes them feel part of

this important task. There are

instruction sheets for the team with

each audit.

For confidentiality, your data should

be anonymous. To do this, whileallowing a check for accuracy, you

could have a patient-identification

list. One is included on the

antimicrobial prescribing audit. In

this way, you have one form on

which you allocate a number to

each patient. Then, on your data

collection sheet, you only need to

list numbers not patient names.

Once you review the collected data

and believe it to be error free, you

should destroy the patient

identification list. Patient names

should never appear in your results.

For the audits we've provided, you

should collect data in the way

we've described in the audit

briefings. We've designed the audit

collection methods to ensure bias

does not creep into your data.

Never intentionally exclude cases

because you feel they don't fit orthat they will not give you the

desired findings. Honesty during

data collection is essential: you are

only fooling yourself if you tamper

with the data.

Recording, sampling and analysing data

This audit permits an analysis of root canal treatment carried out

on past patients. Endodontic treatment is categorised by tooth

type, pre-treatment condition and radiographic outcome, and this

permits an analysis of the success rate of different types of

treatment under different circumstances over time.

It is suggested that as patients

come in for treatment, those who

have had endodontic treatment in

the past are identified and put aside

for data entry at, say, weekly

intervals. Once a sample of 100 to

150 cases (or however many you

can accumulate in a reasonable time

period) has been identified, the

outcome of the treatment against

time can be assessed using the

spreadsheets.

The audit can be repeated at future

intervals to permit measurement of

improvement in outcome, as well as

to identify any particular factors that

predispose to greater or lesser

success rates. The results will help

to show which factors might be

modified and provide pointers to

changes in technique, case selection

or referral protocols.

The audits:

The success of endodontic treatment


11/13

2120

With your data collection forms in hand, you next need to choose

your sample size. For the audits on this package, we've suggested

sample sizes that are big enough to give meaningful results and

small enough not to need too much work. It can be a difficult concept

to accept, but sample sizes do NOT need to vary dependent onyour list size.

Mathematically it is accepted that certain sample sizes will yield certain levels

of reliability. To be 95% confident with your results, as we suggest you should

be, you need to observe a certain amount of cases. Lets say your sample

includes 50 cases. You could, therefore, say with 95% confidence that 26% to

54% (ie 40% +/-14%) have a particular characteristic. We've provided a table

that will help you determine this for your results.

For 95% confidence in your results:

If submitting your application to a LAP for funding, you will need to indicate

your sample size. This will also have a direct bearing on how many hours

you will spend on the audit (or your team will spend if they are collecting the

data for you). We have given some indications on how long the various

audits may take in the table shown. Please remember that the amount of

time you spend on each audit may vary depending on your practice

circumstances. Discuss this with your team and/or your audit facilitator.

Sample Percentage of cases with a particular characteristic

20% 40% 60% 80%

20 +/-18% +/-22% +/-22% +/-18%

50 +/-11% +/-14% +/-14% +/-11%

100 +/-8% +/-10% +/-10% +/-8%

500 +/-3.5% +/-4.5% +/-4.5% +/-3.5%

1000 +/-2.5% +/-3% +/-3% +/-2.5%

Model audit title Sample size Indicative hourssuggested taken to audit

Waiting times 200 6

Quality of radiographs 10 x 2 4

Periodontal records 200 5

Antimicrobial prescribing 20 3

Crown assessment 200 6

Record keeping 25 4

Emergency appointments 150 5

Endodontic success 100-150 6

Recording, sampling and analysing data

Clinical Audit RECORDING, SAMPLING AND ANALYSING DATA


12/13

2322

There is little point in auditing if you then ignore the results or fail to

apply the information gained. You are not auditing just to look for

things that have gone well. Its equally important to recognise

strengths as well as weaknesses. By doing this yourself, you're

setting an example to the whole team to identify positive as well asthe negative outcomes without the fear of consequence.

In most cases, you will also come

across information you didnt have

before you started the audit. For

instance, when auditing radiographs,

it might be discovered that the

established practice in the darkroom

did not match the advice given with

the chemicals or films used. In many

instances, there will be the discovery

that some information is still missing:

this might need further research or

possibly a more detailed audit of that

part of the process. Perhaps with our

crowns audit you will find that one

type of crown has a more distinctmargin than other types. This might

lead to the discovery that there are

innate limits to the accuracy of

different materials.

There may be a number of ways in

which the practice team might

respond to the outcome of your

audit. If they are disillusioned or

downcast, then it is likely that your

approach has been too blame-

oriented. The ideal outcome is that

the audit will either generate the

desire to improve by re-organising or

re-evaluating the process, or it will

create a thirst for more knowledge.

Either of these is good, both would

be excellent.

Finally, the best way to review

changes in knowledge, attitude or

procedure is to re-audit. This is a

crucial part of the process. To ensure

you remember to re-audit include a

reminder in your practice diary.

Interestingly, a paper published in

the Journal of the Royal Society of

Medicine (June 2001) noted that only

24% of the audits included a re-auditphase as part of the planned audit

process. The authors concluded that

failure to close the loop undermined

its effectiveness.

A provision for re-audit can be built

into your original audit application.

However, your LAP will require a

fresh application as it is often

difficult at the outset of your first

audit to establish exactly what you

will re-audit.

British Dental Association

Information Centre

64 Wimpole Street

London W1G 8YS

Tel: 020 7563 4545Fax: 020 7935 6492

www.bda-dentistry.org.uk

Faculty of General Dental

Practitioners (UK)

The Royal College of Surgeons

of England

35-43 Lincolns Inn Fields

London WC2A 3PN

Tel: 020 7405 3474

www.rcseng.ac.uk/dental/fgdp/

For information on audit in

Wales contact:

Department of Postgraduate

Education in Wales. Dental School,

Heath Park, Cardiff CF14 4XY

Tel: 029 2074 2594Forms are online at:

www.dentpostgradwales.ac.uk

For information on audit in

Scotland contact:

Scottish Council for Postgraduate

Medical and Dental Education

2nd Floor Hanover Buildings

66 Rose Street

Edinburgh EH2 2NN

Tel: 01436 670949

Forms are online at:

www.nes.scot.nhs.uk/dentistry/

General/Audit

For information on audit in Northern

Ireland, contact the Central Services

Agency on 0289 0535649

Resources

The Department of Healths document:

"Modernising NHS Dentistry Clinical

Audit and Peer Review in the General

Dental Services" can be found at:

www.doh.gov.uk/pdfs/moddent.pdf

Medline

www.ncbi.nlm.nih.gov/entrez/query.fcgi

West Midlands Local Assessment Panel

www.dentistry.bham.ac.uk/lap/

Cochrane Oral Health Group

www.cochrane-oral.man.ac.uk

Centre for Evidence Based Dentistry

www.ihs.ox.ac.uk/cebd

Confederation of Dental Employers

www.codeuk.com

Local Assessment Panels are listed

on the BDA website.

Implementing changes and re-auditing Useful contacts and resources


13/13

Denplan Limited, Denplan Court

Victoria Road, Winchester, SO23 7RG, UK

Telephone +44 (0) 1962 828000. Fax +44 (0) 1962 840846.Email [email protected]

Denplan Limited. Registered number 1981238 England.Registered office 107 Cheapside, London EC2V 6DU United Kingdom.

Denplan Limited is an Appointed Representative of AXA PPP healthcare Limited which is authorised and regulated by the Financial Services Authority.The registered office of AXA PPP healthcare is 107 Cheapside, London EC2V 6DU United Kingdom. Registered number is 3148119.

Denplan Limited can only offer the dental insurance of AXA PPP healthcare Limited. Calls may be monitored or recorded for training purposes.TR21 03/05 SKY

Clinical Audit Made Simple II

Smile-on Ltd., Treasure House19-21 Hatton Garden, London EC1N 8LF.Tel: 020 7400 8989

smile-on.comThe future of dentistry is here

IN PARTNERSHIP WITH

The most fruitful lesson is the conquest of ones own error. Whoever

refuses to admit error may be a great scholar, but he is not a great learner.

- Wolfgang Goethe

We are part of a learning as well as a caring profession.

If you experience any problems with the CD-Rom, please call 020 7843 6850

www.denplan.co.uk

www.smile-on.co.uk

SUPPORTED BY

Documents

Audit Made Simple