Sina Keshavaarz M.DPublic Health &Preventive Medicine

Measuring level of performance& sustaining improvement

The audit cycle

Planning data collection

the user group to be included, with any exceptions noted

the healthcare professionals involved in the users’ care

the time period over which the criteria apply

Identifying users

Audit staff must be very careful about the accuracy, timeliness and completeness clinical records. It can help to use certain data collection strategies, including:

multiple sources of information direct observation encounter sheets completed at the time by

the healthcare professional.

Sampling users

How many of the users (study population) do I need to select? How do I choose a representative sample? In audit, it is usual to compare the proportion of users whose

care is in accordance with the criteria before changes in care with the proportion after the changes.

The number needed in the sample is determined by two factors:

the degree of confidence wanted in the findings

resource constraints (time, access to data, costs).

Calculating sample sizes for proportions Interval sampling Two-stage sampling Rapid-cycle sampling

Handling data

Data sources If the required data are not collected

routinely, a specific paper or electronic encounter sheet can be devised for healthcare professionals to record additional information during each consultation

Individual user

Data abstraction tools

Data abstraction tools Data for an audit are generally collected retrospectively Data collection forms must specify precisely the information to

be abstracted from the record, and they should be clear and easy to use.

Retrospective or concurrent data collection? Concurrent data collection gives a team more immediate

feedback on its current performance and can act as a positive reinforcement to improve or maintain practice.

Data analysis

The analysis can range from a simple calculation of percentages, through to relatively sophisticated statistical techniques.

the analysis should be as simple as possible, the findings should be presented simply and clearly.

Quantitative analysis is concerned with numerical data – the more common form of data in audit. Many of the examples of audit included in this book employ quantitative analysis.

Qualitative analysis

A data collection strategy may include descriptive elements, such as additional comments within a questionnaire or transcripts from focus groups or interview

qualitative methods provide a means of assimilating a rich source of information on people’s experiences around a clinical topic

sustaining improvement

Improvements in care implemented as a part of clinical audit must be monitored, evaluated, sustained, and reinforced within a supportive environment.

Structures and systems must be developed to enable organisations to integrate improvements within a planned strategy.

A culture is required that makes the user’s experience the primary motivation for improvements, creates confident staff who do not fear reporting or confronting inadequate performance, and has clear and constant objectives.

Monitoring and evaluating changes

Collecting data for a second time, after changes have been introduced, is central to both assessing and maintaining the improvements made during clinical audit

Using IT A well thought out and integrated IT strategy can help data

collection

Clinical performance indicators

It is important to realize that only the minimum number of essential indicators should be included in monitoring

Whenever possible, authoritative, evidence-based sources of guidance on selecting performance indicators and advice on audit criteria (such as those in the technology appraisals and guidelines produced by NICE) should be used

Other methods of continued monitoring

Errors, adverse incidents, and significant event audit can also be used for continued monitoring. Comments from users may be included as sources of information about performance.

they depend on an environment that fosters the reporting of errors and adverse incidents and they are no substitute for systematic monitoring.

Evaluating audit quality

The quality of clinical audit programmes must be evaluated as part of the wider clinical governance agenda

The checklists can be used by audit leaders and clinicians to evaluate the methods they have used

Maintaining and reinforcing improvement common factors have been identified including:

reinforcing or motivating factors built in by the management to support the continual cycle of quality improvement

integration of audit into the organization's wider quality improvement systems

strong leadership.

Appropriate organisational development

cultural change, ensuring that the shared values and beliefs of the organisation support the ideas of quality improvement

adequate training, so that staff can gather and analyse data accurately

an organisational structure that coordinates and monitors quality improvement work quickly and effectively.

three principal recommendations, which can be described as vision, constancy and management (Garside, 1998).

. The desired end-state or vision should be explicitly articulated, alliances with external organisations should be forged while achieving it, and it should be retained in difficult times of resistance to change.

four interdependent processeshave been found to support the lasting impact of clinical audit:

a strategy that recognizes audit activity, combined with an achievable plan of quality

Improvement a culture that supports the concept of planned audit activity, leading to improvements in quality of which everyone in the organization is aware and supportive

IT processes that can provide accurate information about the organization, allowing sensible decisions to be made about where audit is needed and whether changes have had the desired effects

appropriate structures to support and implement the changes that are suggested

Continue

Leaders must constantly show commitment to the desired direction of change, and that they mean what they say.

The details must be managed carefully, for example, by appointing a person to oversee the task, creating project teams, making time available, and arranging for a source of transitional funding to be available.

Using existing frameworks At the strategic level, programs of audit linked to national

issues The responsibility for providing education, training, and support

for audit teams lies at the organizational level Leadership Changing the organizational culture The learning organization Knowledge management Sustained quality improvement in practice