1

Laboratory Internal Auditor Course

Daren C. ValentineIT Director, A2LAPh: 301 644 3213

Email: dvalentine@a2la.org

#1 Rule of Auditing

If you cannot express the non-conformance in the words of the management system standard, the

appropriate technical specification or test method, or the company’s own policies and procedures,

THEN YOU DO NOT HAVE A NON-CONFORMITY

2

© A2LA 20104

What Is An Audit, Really?

1. Main purpose: to verify compliance

2. Focus: to find evidence of compliance3. The goal should never be to keep digging until nonconformities

are found

4. Auditors should not take pleasure in finding fault!

QMS in Lab Operations

PROCESS CONTROL

Quality Control (QC)

QUALITY MANAGEMENT SYSTEMS (QMS)

Includes all functions with emphasis on those that influence the safety of persons and/or the (quality) validity of technical decisions or results

• All business operations functions

• All management and administrative functions

• All finance functions

• All HR functions

• All purchasing and contract review functions

• All marketing and communications functions

• All maintenance functions

Pre - Preparation Test, Calibration. Data Validation Post

Sales Inspection, Cert. Sales

CONTROL OF DATA

Quality Assurance (QA)

Contact Sampling Reporting Feedback

Why perform Internal Audits?

• Internal audits are the most comprehensive method of determining if the management system is working for us:

• Is it supporting the work of the people in the laboratory?

• Does it help provide a safe environment?

• Does it help us produce only technically valid results?

• Internal audits tell us of our management system is:

• Implemented;

• Effective, and

• Allows for continual improvement.

3

Continual Improvement (17025)

4.7 4.8 4.9 4.10 4.14 5.9

4.9 4.10

4.11 4.12

4.15Clause

GoverningReference

When the deviation Today Tomorrow??occurs

Formal Action

Internal Audit CyclePlanning

Implement the Audit Program

Conduct each Audit

Close out and follow up

• Based on previous results, frequency, scope, resources, etc.• Established by top management and the quality team• Agreed by all participants, auditees, auditors

• Plan for each individual audit occurrence• Determine resources, scope, objectives for each audit• Each individual audit agreed by all participants, auditees, and

auditors.

• Document review, checklists, timings for each audit agreed• Onsite - opening meeting, interviews, findings, report, closing

meeting.

• Auditee action – corrective/preventive action• Formal close out of findings• Follow-up as required, including any further audits

System Readiness for Audit

Documented system

Structured system

Implemented system

4

Documented System Quality policy document

Processes / procedures

SOPs / instructions

Specifications

Standards

Codes of practice

Regulations

Records

Structured System

WHY

WHOWHATWHENWHERE

HOW

RECORDS

Policy

Procedures

WorkInstructions

Proof

Level 1

Level 2

Level 3

Level 4

Implemented System

Do people have access to the policies and procedures that govern their work?

Do people maintain records that demonstrate their use of the policies and procedures that govern their work?

• Records indicate the level of implementation of Documents, Processes and Procedures

• Records provide evidence that a system is being used

5

Internal Audit Program

Approved by top management and contains:• Resource allocations• Scope and objectives of audit activities• Timelines

Audit Area / Period Q1–2007 Q2–2007 Q3–2007 Q4-2007

Admin Sally George

Engines Karen

Polymers Bruce

Sample Reception / dispatch Jill

Developing the Audit Plan

See the example on page 14 of Chapter 2.

Note the list of requirements on page 15 of Chapter 2 when formally presenting the plan during an opening meeting.

© A2LA 201015

Audit Approaches

Periodic Department/Section Audits

Audit of the whole management system on a department by department schedule

Pros

More manageable Cons

Can miss interfaces between departments

6

Audit Approaches

Horizontal Audits Based upon the periodic audit of a particular system

procedure in the management system (e.g. records; calibration; training; test methods)

ProsDiscover differences in application throughout organization

ConsDoesn’t address all of the combined inputs which produce the

final product or test data.

Audit Approaches

Test Process/Vertical Audits Selection of a particular test or calibration method, and

auditing all of the inputs, operations and activities required to produce the data output

ProsExamines the interfaces between all inputs and

procedures ConsMisses inconsistent application of procedures between

different activities

Best Audit ApproachTest Report

Review of New Work

Sample Receipt

Test Methods

Equipment Calibration

Data Acquisition

Sample Disposition

7

© A2LA 201019

Depth of Audits

Full audit of a procedure

Partial audit Phased audits

Non-conforming work evaluations Root cause investigations

Corrective action monitoring Follow-up audits

Opening Meeting

1. Introductions and thanks

2. Audit scope and objectives

3. Audit plan

4. Audit methods and procedures

5. Confidentiality

6. Respond to questions

7. Depart for the tour of the facility

Audit Schedule / Plan

1. Agreement on the planned approach

2. Respond to necessary adjustments to the plan

3. Confirm arrangements for logistics and resources

4. Confirm arrangements and timings for subsequent meetings

5. Confirm arrangements for the end of the audit

8

Audit Methods

1. Clearly explain investigation activities

2. Be open about the process and emphasise its transparency

Confidentiality

1. Confirm audit team’s responsibilities

2. Confirm auditing authority’s responsibilities

Respond to Questions

Be prepared to handle questions from the auditees – focus on the benefits of the approach and the transparency of the activities.

9

© A2LA 201025

Four Human Relation Aspects

Attitudes towards auditing

Communication skills Listening skills

Reducing tension

26

Audit AttitudesAudits…

are fact-finding missions, not fault-finding safaris focus on verifying compliance are not meant as fodder for auditors should not give the auditor “joy” when writing

deficiencies are meant to improve the quality of the work in

your laboratory are just another tool to improve the laboratory’s

management system

© A2LA 201027

Audit Perspectives

We have talked about the #1 Rule of Auditing, BUT…

What do you want to see? What do you think you see?

Or what’s actually there?

10

Communication Skills Keep it simple: clear, brief, direct, focused Three ways of receiving information:

Seeing, hearing, experiencing Spend most of your time listening

Stay in the adult mode Equal, not parent/child or teacher/student

Adults: thinking, assessing, proactive, logical, mature Parents/teachers: bossy, authoritarian, instructing, nurturing Children/students: emotional, reactive, impulsive, rebellious

Express comments and questions in a positive way It’s not just what we say, but how we say it And please use the person’s name!

Listening Skills Why don’t we listen?

Get bored and we’re ready to say something ourselves We’re thinking about our reply while they’re talking We’re jumping to conclusions or making assumptions Slow down! We think 5 times faster than we talk

If we don’t listen, we will miss: Important statements The significance of what was said The interactions between the people present during the audit

And we will convey complete disinterest to the person

Listening Skills A good listener

Keeps the focus on the speaker Is willing to see things from another perspective Refrains from finishing someone else’s sentence Remains calm if attacked, rather than becoming defensive Is accepting of the person, if not their actions Uses “active listening” skills – the concept of being a “mirror”

Attending skills Shows we are prepared to listen Is being sensitive to the speaker’s feelings Is asking relevant questions and using minimal encouragers Is mirroring the mode and pace of the other person’s speech

11

Listening Skills

Body Language Relaxed, alert, smiling Lean forward slightly Face squarely and at eye level Open posture Remember the “message”

Eye Contact Expresses interest Establishes trust Focus softly and shift gaze

Reducing the Tension Effects of tension

Defensiveness, reluctance, aggression Poor performance and mistakes by interviewee

Reduce by: Being human Putting people at ease Projecting the right image – quiet confidence Challenging the issue, not the person Avoiding absolutes and attacking statements

Asking Key Questions

What is happening?Why is it happening?Where is it happening? Why is it happening

there?When was it done? Why was it done then?Who did it? Why was it done by that person?How was it done? Why was it done that way?

12

Direct Questions

Require definite answers Establishes something factualUseful for clarifying detailBrings discussion back on trackOften results in a “yes” or “no”, like:

“Do you calibrate these instruments daily?”Not effective in having the auditee explain

Hypothetical Questions

Poses the unusual “What do you do when you find that the

instrument is out of calibration?”What does the response indicate?Has a contingency been planned for?Encourages the auditee to explain further

Clarifying Questions

Prevents misunderstanding “Do I understand that you will perform daily

instrument calibrations only when the daily intermediate checks are unacceptable?”

Enables the person to explain even furtherRepetitive question to further clarifyAVOID leading questions

13

Other Questions Answer your own questions

Avoids questioning the obvious Reassures the person while you examine more

Get confirmation Same question, different perspective Same question, different person

Even silence can be useful Gives person time to think Places person under pressure to respond But generates tension (so use with care)

Limitations of Questions

Can’t always take answers at face value

Answers may not be the truth May be intentionally misleading

Coached responses But remember:

Auditors are in the evidence business

© A2LA 201039

Information or Evidence?

Information is what you hear!

Evidence is what you see!!

So – we need to go see…

14

Inspect Facility & Equipment

Evaluate design layout of laboratory Proper environmental conditions and monitoring?

Sufficient lighting? Temperature? Effective separation between areas? Appropriate security? Good housekeeping? Suitable equipment, range, precision? Status of equipment, maintenance, calibration?

Examine Documents

Documents available for use? Implemented? Is there proper document control for

Internal documents? External documents?

Document changes properly made?Draft procedures in use?Obsolete documents removed? Verify that testing and calibration follows written

procedures

Examine Records

Quality and technical records in place? Records allow traceability and repetition? Procedures being followed for archiving? Confidentiality maintained? Audit trail? Records include identity of all personnel? Contemporaneous records? Complete? Error corrections properly made?

15

Observe Activities

Witness demonstrations of procedures Confirms what we heard from our questions Confirms that analysts are following procedures Determines status of equipment and setup Enables us to judge technical skills Establishes the level and depth of training Gains information on supervision

Show and Tell

Questions Initiated by the auditor Goes along the spokes

“Show and Tell” Lead by the auditee Goes around the rim

Responsibilities for Conduct

The auditor is solely responsible for the success of the audit. If a team, it is the team leader.

Chair meetings (all of them) Declare findings Sign the report Official link between team and auditees.

16

Sampling

Discuss the process and pick some records along the way. Best when the process is simple and records are few.

Start with random selection of records and discuss the process based on the records. Best when process is complex and records are many.

Facing Conflict

Remain calm - create distance and space Tell someone (lead auditor) or senior

auditee Tell someone else (quality manager) Pause the audit until the situation is

resolved

Recording Nonconformities

1. No Secrets

2. Disclose/Communicate

3. Vet

17

Daily Debreifs

1. Daily to prevent surprises later

2. List of observations - not findings

3. Keep auditee current with progress of audit

When Something Seems Wrong Is it really wrong?Do the analysts know its wrong?What is their explanation? Is it an isolated event, or a symptom of a deeper

problem?Why didn't the management system detect it?What lapse in the management system allowed

this to happen?

Audit Findings

Two required elements of every finding:

• A statement of the objective evidence, and• A reference to the requirement

18

Evidence Supports Findings

Sources of audit evidence:

• Inspecting facilities and equipment• Examining documents• Examining records• Observation of activities

What is an “audit finding”?

Definition:

“result of the evaluation of the collected audit evidence against audit criteria”

Findings can be good or “not so good”

Can indicate conformity, nonconformity or opportunities for improvement

© A2LA 201054

Writing Deficiencies

Clear

Concise Non-confrontational

Non-blaming Non-personal

19

Writing a Deficiency

Structure of a deficiency statement

The Requirement

Description of the deficiency

The objective evidence

© A2LA 201056

Writing Deficiencies

Directly related to the management system standard, laboratory policies and procedures, technical procedures or customer requirements, etc.

And based on a clear statement of objective evidence recorded during the audit

Writing the Report Solely the responsibility of the auditor (or lead auditor).

Only the auditor (or the lead auditor) MUST sign the report. Only the auditor (or the lead auditor) declares the non-

conformances.

Good lead auditors get consensus.

20

Contents of the Report

The report must:

Be a factual description of the audit activities it covers, and

Provide a fair and accurate picture of the quality system audited.

Structure of the Report

The report contains:

Cover page Audit Summary

Activity Summary Findings

Cover PageThe cover page contains:

• Report number• Dates• Scope and objectives• Reference documents• Audited documents• Team names• Auditee names• Signature of lead auditor (and….)• Page numbers…

21

Audit Summary Page

The summary page contains:

• Report number• Auditor overall comments• Auditor conclusions• Auditor recommendations• Date of next planned activity• Page numbers

Activity Summary Page

The activity summary page contains:

Report number List of observations (areas/activities)

List of associated references List of deviations and other finding notation numbers

Auditor activity comments Page numbers

Findings Page(s)

Each finding page contains:

Finding (NCR) log number Indication of type of finding

Identification of process owner, date and reference Description of the finding

Space for root cause analysis, corrective action, and follow up

22

Team Debriefing

After the conclusion of the audit activity, gather the team together to discuss how it went and look for areas to improve.

Three Questions:

1. Does this condition (or might it) present unacceptable risk of recurrence to the organization (or its people or visitors)?

2. Has this condition prevented or might it prevent the organization from producing technically valid results or making technically valid decisions?

3. Does the condition represent a true departure from the Quality System?

Determining the need for Root Cause Analysis

Answers:

Yes/No?

Yes/No?

Yes/No?

Results of the Three Questions:

1. ANY “yes” answers means a full root cause analysis is REQUIRED2. If ALL answers are “no,” then root cause is not required and only

remediation (correction or prevention) is needed.

Determining the need for Root Cause Analysis

23

Root Cause Process

1.Direct Cause. Ask the first “Why” and identify the Direct Cause.

2.Root Cause. Ask three or four more “Why’s” back along the chain of events and get to a condition or circumstance whose resolution that will permanently resolve the problem (and is still within your scope of responsibility). Find the hole in the System and fill it. Use the indicators from the ISO/IEC 17025 Principle of “Capacity” to help.3.Document the results of the analysis as part of the overall Corrective or Preventive Action.

Example 1Why was I late for work this morning?

PROBLEM: Why didn't I get to work on time? DIRECT CAUSE

CONTRIBUTING CAUSE: Car wouldn't start Why? CONTRIBUTING CAUSE: Battery was dead. Why? CONTRIBUTING CAUSE: Dome light stayed on all night. Why?

ROOT CAUSE: Kids played in car, left door ajar.

Example 2Received ticket for safety violation.

Car exhaust too loud.Muffler knocked loose from tail pipe.

Daughter hit pothole.Potholes in road.

Winters damaged roads.Govt won't approve extramoney for better roads.

Govt doesn't have extra money.Govt spent money on other legal issues.

Too many lawyers in politics. Solution? Drive car in Sweden where there are fewer lawyers.

24

The Chain of Causes1. Direct Cause: The cause that directly resulted in an event. (The first

cause in the chain.)• This is the answer to your first question (your problem statement).

2. Contributing Cause: A cause that contributed to an event but, by itself, might not have caused the event (the causes before the direct cause) or are back in the chain of events between the Direct Cause and the Root Cause.• For a very simple problem there may not be any contributing causes.

3. Root Cause: The fundamental reason for an event, which if corrected, would prevent recurrence or first-time occurrence. (The first cause in the chain).• Root Causes that are outside the ability of an organization to address it are not

useful for anything. Stick to the ones we can actually fix.

Root Cause Process

1. Determine the Direct Cause.2. Determine the Root Cause. 3. Document the results

Identify the Direct Cause

1. Direct Cause. Ask the first “Why” and identify the Direct Cause.

OR

2. Visualise it: Take a mental picture of the actual occurrence (incident, deviation, etc.). You are looking at the Direct Cause in that picture.

25

Root Cause Process

1. Determine the Direct Cause. 2. Determine the Root Cause.3. Document the results

Identify the missing pieces• Knowledge/skill of the people involved• Environment surrounding the work:

• Physical• Psychological• Equipment

• Quality control / assurance of the work• Suitability of the procedures involved

Root Cause ListingPossible root causes include the following categories.

• Personnel Factors• Environmental Factors• Quality Factors • Procedural Factors• Organisational Factors

26

Personnel Factors• Physical capacity• Intellectual capacity• Physical or physiological stress• Emotional or psychological stress• Individual skill• Individual knowledge• Care and attention.

Environmental Factors• Physical plant and facilities• Environmental conditions• Tools and equipment• Materials and supplies• Maintenance• Wear and tear

Quality Factors• Quality control• Quality assurance• Quality system

27

Procedural Factors• Standard procedures• Specifications• Implementation• Selection of support

Organisational Factors• Leadership• Communications• Motivation

Root Cause Process

1. Determine the Direct Cause. 2. Determine the Root Cause. 3. Document the results

28

Record Steps Taken• Steps taken to eliminate root cause• Follow up on implementation• Verification audit• Subsequent actions

What if Problem Recurs?• Correct implementation?• Original root cause determination incorrect?• Back to the drawing board• New root cause• New corrective action

What if Problem Recurs?• Correct implementation?• Original root cause determination incorrect?• Back to the drawing board• New root cause• New corrective action

29

Summary Internal Audits are focused on determining compliance A good auditor collects objective evidence and evaluates that evidence

against stated requirements A well written deficiency contains two elements:

A statement of the objective evidence A relationship to the requirements

Audit findings are followed by: Remedial action Root cause analysis

Corrective action If corrective action fails to prevent recurrence, then its back to the

drawing board!