Upload
others
View
1
Download
0
Embed Size (px)
Citation preview
1
Laboratory Internal Auditor Course
Daren C. ValentineIT Director, A2LAPh: 301 644 3213
Email: [email protected]
#1 Rule of Auditing
If you cannot express the non-conformance in the words of the management system standard, the
appropriate technical specification or test method, or the company’s own policies and procedures,
THEN YOU DO NOT HAVE A NON-CONFORMITY
2
© A2LA 20104
What Is An Audit, Really?
1. Main purpose: to verify compliance
2. Focus: to find evidence of compliance3. The goal should never be to keep digging until nonconformities
are found
4. Auditors should not take pleasure in finding fault!
QMS in Lab Operations
PROCESS CONTROL
Quality Control (QC)
QUALITY MANAGEMENT SYSTEMS (QMS)
Includes all functions with emphasis on those that influence the safety of persons and/or the (quality) validity of technical decisions or results
• All business operations functions
• All management and administrative functions
• All finance functions
• All HR functions
• All purchasing and contract review functions
• All marketing and communications functions
• All maintenance functions
Pre - Preparation Test, Calibration. Data Validation Post
Sales Inspection, Cert. Sales
CONTROL OF DATA
Quality Assurance (QA)
Contact Sampling Reporting Feedback
Why perform Internal Audits?
• Internal audits are the most comprehensive method of determining if the management system is working for us:
• Is it supporting the work of the people in the laboratory?
• Does it help provide a safe environment?
• Does it help us produce only technically valid results?
• Internal audits tell us of our management system is:
• Implemented;
• Effective, and
• Allows for continual improvement.
3
Continual Improvement (17025)
4.7 4.8 4.9 4.10 4.14 5.9
4.9 4.10
4.11 4.12
4.15Clause
GoverningReference
When the deviation Today Tomorrow??occurs
Formal Action
Internal Audit CyclePlanning
Implement the Audit Program
Conduct each Audit
Close out and follow up
• Based on previous results, frequency, scope, resources, etc.• Established by top management and the quality team• Agreed by all participants, auditees, auditors
• Plan for each individual audit occurrence• Determine resources, scope, objectives for each audit• Each individual audit agreed by all participants, auditees, and
auditors.
• Document review, checklists, timings for each audit agreed• Onsite - opening meeting, interviews, findings, report, closing
meeting.
• Auditee action – corrective/preventive action• Formal close out of findings• Follow-up as required, including any further audits
System Readiness for Audit
Documented system
Structured system
Implemented system
4
Documented System Quality policy document
Processes / procedures
SOPs / instructions
Specifications
Standards
Codes of practice
Regulations
Records
Structured System
WHY
WHOWHATWHENWHERE
HOW
RECORDS
Policy
Procedures
WorkInstructions
Proof
Level 1
Level 2
Level 3
Level 4
Implemented System
Do people have access to the policies and procedures that govern their work?
Do people maintain records that demonstrate their use of the policies and procedures that govern their work?
• Records indicate the level of implementation of Documents, Processes and Procedures
• Records provide evidence that a system is being used
5
Internal Audit Program
Approved by top management and contains:• Resource allocations• Scope and objectives of audit activities• Timelines
Audit Area / Period Q1–2007 Q2–2007 Q3–2007 Q4-2007
Admin Sally George
Engines Karen
Polymers Bruce
Sample Reception / dispatch Jill
Developing the Audit Plan
See the example on page 14 of Chapter 2.
Note the list of requirements on page 15 of Chapter 2 when formally presenting the plan during an opening meeting.
© A2LA 201015
Audit Approaches
Periodic Department/Section Audits
Audit of the whole management system on a department by department schedule
Pros
More manageable Cons
Can miss interfaces between departments
6
Audit Approaches
Horizontal Audits Based upon the periodic audit of a particular system
procedure in the management system (e.g. records; calibration; training; test methods)
ProsDiscover differences in application throughout organization
ConsDoesn’t address all of the combined inputs which produce the
final product or test data.
Audit Approaches
Test Process/Vertical Audits Selection of a particular test or calibration method, and
auditing all of the inputs, operations and activities required to produce the data output
ProsExamines the interfaces between all inputs and
procedures ConsMisses inconsistent application of procedures between
different activities
Best Audit ApproachTest Report
Review of New Work
Sample Receipt
Test Methods
Equipment Calibration
Data Acquisition
Sample Disposition
7
© A2LA 201019
Depth of Audits
Full audit of a procedure
Partial audit Phased audits
Non-conforming work evaluations Root cause investigations
Corrective action monitoring Follow-up audits
Opening Meeting
1. Introductions and thanks
2. Audit scope and objectives
3. Audit plan
4. Audit methods and procedures
5. Confidentiality
6. Respond to questions
7. Depart for the tour of the facility
Audit Schedule / Plan
1. Agreement on the planned approach
2. Respond to necessary adjustments to the plan
3. Confirm arrangements for logistics and resources
4. Confirm arrangements and timings for subsequent meetings
5. Confirm arrangements for the end of the audit
8
Audit Methods
1. Clearly explain investigation activities
2. Be open about the process and emphasise its transparency
Confidentiality
1. Confirm audit team’s responsibilities
2. Confirm auditing authority’s responsibilities
Respond to Questions
Be prepared to handle questions from the auditees – focus on the benefits of the approach and the transparency of the activities.
9
© A2LA 201025
Four Human Relation Aspects
Attitudes towards auditing
Communication skills Listening skills
Reducing tension
26
Audit AttitudesAudits…
are fact-finding missions, not fault-finding safaris focus on verifying compliance are not meant as fodder for auditors should not give the auditor “joy” when writing
deficiencies are meant to improve the quality of the work in
your laboratory are just another tool to improve the laboratory’s
management system
© A2LA 201027
Audit Perspectives
We have talked about the #1 Rule of Auditing, BUT…
What do you want to see? What do you think you see?
Or what’s actually there?
10
Communication Skills Keep it simple: clear, brief, direct, focused Three ways of receiving information:
Seeing, hearing, experiencing Spend most of your time listening
Stay in the adult mode Equal, not parent/child or teacher/student
Adults: thinking, assessing, proactive, logical, mature Parents/teachers: bossy, authoritarian, instructing, nurturing Children/students: emotional, reactive, impulsive, rebellious
Express comments and questions in a positive way It’s not just what we say, but how we say it And please use the person’s name!
Listening Skills Why don’t we listen?
Get bored and we’re ready to say something ourselves We’re thinking about our reply while they’re talking We’re jumping to conclusions or making assumptions Slow down! We think 5 times faster than we talk
If we don’t listen, we will miss: Important statements The significance of what was said The interactions between the people present during the audit
And we will convey complete disinterest to the person
Listening Skills A good listener
Keeps the focus on the speaker Is willing to see things from another perspective Refrains from finishing someone else’s sentence Remains calm if attacked, rather than becoming defensive Is accepting of the person, if not their actions Uses “active listening” skills – the concept of being a “mirror”
Attending skills Shows we are prepared to listen Is being sensitive to the speaker’s feelings Is asking relevant questions and using minimal encouragers Is mirroring the mode and pace of the other person’s speech
11
Listening Skills
Body Language Relaxed, alert, smiling Lean forward slightly Face squarely and at eye level Open posture Remember the “message”
Eye Contact Expresses interest Establishes trust Focus softly and shift gaze
Reducing the Tension Effects of tension
Defensiveness, reluctance, aggression Poor performance and mistakes by interviewee
Reduce by: Being human Putting people at ease Projecting the right image – quiet confidence Challenging the issue, not the person Avoiding absolutes and attacking statements
Asking Key Questions
What is happening?Why is it happening?Where is it happening? Why is it happening
there?When was it done? Why was it done then?Who did it? Why was it done by that person?How was it done? Why was it done that way?
12
Direct Questions
Require definite answers Establishes something factualUseful for clarifying detailBrings discussion back on trackOften results in a “yes” or “no”, like:
“Do you calibrate these instruments daily?”Not effective in having the auditee explain
Hypothetical Questions
Poses the unusual “What do you do when you find that the
instrument is out of calibration?”What does the response indicate?Has a contingency been planned for?Encourages the auditee to explain further
Clarifying Questions
Prevents misunderstanding “Do I understand that you will perform daily
instrument calibrations only when the daily intermediate checks are unacceptable?”
Enables the person to explain even furtherRepetitive question to further clarifyAVOID leading questions
13
Other Questions Answer your own questions
Avoids questioning the obvious Reassures the person while you examine more
Get confirmation Same question, different perspective Same question, different person
Even silence can be useful Gives person time to think Places person under pressure to respond But generates tension (so use with care)
Limitations of Questions
Can’t always take answers at face value
Answers may not be the truth May be intentionally misleading
Coached responses But remember:
Auditors are in the evidence business
© A2LA 201039
Information or Evidence?
Information is what you hear!
Evidence is what you see!!
So – we need to go see…
14
Inspect Facility & Equipment
Evaluate design layout of laboratory Proper environmental conditions and monitoring?
Sufficient lighting? Temperature? Effective separation between areas? Appropriate security? Good housekeeping? Suitable equipment, range, precision? Status of equipment, maintenance, calibration?
Examine Documents
Documents available for use? Implemented? Is there proper document control for
Internal documents? External documents?
Document changes properly made?Draft procedures in use?Obsolete documents removed? Verify that testing and calibration follows written
procedures
Examine Records
Quality and technical records in place? Records allow traceability and repetition? Procedures being followed for archiving? Confidentiality maintained? Audit trail? Records include identity of all personnel? Contemporaneous records? Complete? Error corrections properly made?
15
Observe Activities
Witness demonstrations of procedures Confirms what we heard from our questions Confirms that analysts are following procedures Determines status of equipment and setup Enables us to judge technical skills Establishes the level and depth of training Gains information on supervision
Show and Tell
Questions Initiated by the auditor Goes along the spokes
“Show and Tell” Lead by the auditee Goes around the rim
Responsibilities for Conduct
The auditor is solely responsible for the success of the audit. If a team, it is the team leader.
Chair meetings (all of them) Declare findings Sign the report Official link between team and auditees.
16
Sampling
Discuss the process and pick some records along the way. Best when the process is simple and records are few.
Start with random selection of records and discuss the process based on the records. Best when process is complex and records are many.
Facing Conflict
Remain calm - create distance and space Tell someone (lead auditor) or senior
auditee Tell someone else (quality manager) Pause the audit until the situation is
resolved
Recording Nonconformities
1. No Secrets
2. Disclose/Communicate
3. Vet
17
Daily Debreifs
1. Daily to prevent surprises later
2. List of observations - not findings
3. Keep auditee current with progress of audit
When Something Seems Wrong Is it really wrong?Do the analysts know its wrong?What is their explanation? Is it an isolated event, or a symptom of a deeper
problem?Why didn't the management system detect it?What lapse in the management system allowed
this to happen?
Audit Findings
Two required elements of every finding:
• A statement of the objective evidence, and• A reference to the requirement
18
Evidence Supports Findings
Sources of audit evidence:
• Inspecting facilities and equipment• Examining documents• Examining records• Observation of activities
What is an “audit finding”?
Definition:
“result of the evaluation of the collected audit evidence against audit criteria”
Findings can be good or “not so good”
Can indicate conformity, nonconformity or opportunities for improvement
© A2LA 201054
Writing Deficiencies
Clear
Concise Non-confrontational
Non-blaming Non-personal
19
Writing a Deficiency
Structure of a deficiency statement
The Requirement
Description of the deficiency
The objective evidence
© A2LA 201056
Writing Deficiencies
Directly related to the management system standard, laboratory policies and procedures, technical procedures or customer requirements, etc.
And based on a clear statement of objective evidence recorded during the audit
Writing the Report Solely the responsibility of the auditor (or lead auditor).
Only the auditor (or the lead auditor) MUST sign the report. Only the auditor (or the lead auditor) declares the non-
conformances.
Good lead auditors get consensus.
20
Contents of the Report
The report must:
Be a factual description of the audit activities it covers, and
Provide a fair and accurate picture of the quality system audited.
Structure of the Report
The report contains:
Cover page Audit Summary
Activity Summary Findings
Cover PageThe cover page contains:
• Report number• Dates• Scope and objectives• Reference documents• Audited documents• Team names• Auditee names• Signature of lead auditor (and….)• Page numbers…
21
Audit Summary Page
The summary page contains:
• Report number• Auditor overall comments• Auditor conclusions• Auditor recommendations• Date of next planned activity• Page numbers
Activity Summary Page
The activity summary page contains:
Report number List of observations (areas/activities)
List of associated references List of deviations and other finding notation numbers
Auditor activity comments Page numbers
Findings Page(s)
Each finding page contains:
Finding (NCR) log number Indication of type of finding
Identification of process owner, date and reference Description of the finding
Space for root cause analysis, corrective action, and follow up
22
Team Debriefing
After the conclusion of the audit activity, gather the team together to discuss how it went and look for areas to improve.
Three Questions:
1. Does this condition (or might it) present unacceptable risk of recurrence to the organization (or its people or visitors)?
2. Has this condition prevented or might it prevent the organization from producing technically valid results or making technically valid decisions?
3. Does the condition represent a true departure from the Quality System?
Determining the need for Root Cause Analysis
Answers:
Yes/No?
Yes/No?
Yes/No?
Results of the Three Questions:
1. ANY “yes” answers means a full root cause analysis is REQUIRED2. If ALL answers are “no,” then root cause is not required and only
remediation (correction or prevention) is needed.
Determining the need for Root Cause Analysis
23
Root Cause Process
1.Direct Cause. Ask the first “Why” and identify the Direct Cause.
2.Root Cause. Ask three or four more “Why’s” back along the chain of events and get to a condition or circumstance whose resolution that will permanently resolve the problem (and is still within your scope of responsibility). Find the hole in the System and fill it. Use the indicators from the ISO/IEC 17025 Principle of “Capacity” to help.3.Document the results of the analysis as part of the overall Corrective or Preventive Action.
Example 1Why was I late for work this morning?
PROBLEM: Why didn't I get to work on time? DIRECT CAUSE
CONTRIBUTING CAUSE: Car wouldn't start Why? CONTRIBUTING CAUSE: Battery was dead. Why? CONTRIBUTING CAUSE: Dome light stayed on all night. Why?
ROOT CAUSE: Kids played in car, left door ajar.
Example 2Received ticket for safety violation.
Car exhaust too loud.Muffler knocked loose from tail pipe.
Daughter hit pothole.Potholes in road.
Winters damaged roads.Govt won't approve extramoney for better roads.
Govt doesn't have extra money.Govt spent money on other legal issues.
Too many lawyers in politics. Solution? Drive car in Sweden where there are fewer lawyers.
24
The Chain of Causes1. Direct Cause: The cause that directly resulted in an event. (The first
cause in the chain.)• This is the answer to your first question (your problem statement).
2. Contributing Cause: A cause that contributed to an event but, by itself, might not have caused the event (the causes before the direct cause) or are back in the chain of events between the Direct Cause and the Root Cause.• For a very simple problem there may not be any contributing causes.
3. Root Cause: The fundamental reason for an event, which if corrected, would prevent recurrence or first-time occurrence. (The first cause in the chain).• Root Causes that are outside the ability of an organization to address it are not
useful for anything. Stick to the ones we can actually fix.
Root Cause Process
1. Determine the Direct Cause.2. Determine the Root Cause. 3. Document the results
Identify the Direct Cause
1. Direct Cause. Ask the first “Why” and identify the Direct Cause.
OR
2. Visualise it: Take a mental picture of the actual occurrence (incident, deviation, etc.). You are looking at the Direct Cause in that picture.
25
Root Cause Process
1. Determine the Direct Cause. 2. Determine the Root Cause.3. Document the results
Identify the missing pieces• Knowledge/skill of the people involved• Environment surrounding the work:
• Physical• Psychological• Equipment
• Quality control / assurance of the work• Suitability of the procedures involved
Root Cause ListingPossible root causes include the following categories.
• Personnel Factors• Environmental Factors• Quality Factors • Procedural Factors• Organisational Factors
26
Personnel Factors• Physical capacity• Intellectual capacity• Physical or physiological stress• Emotional or psychological stress• Individual skill• Individual knowledge• Care and attention.
Environmental Factors• Physical plant and facilities• Environmental conditions• Tools and equipment• Materials and supplies• Maintenance• Wear and tear
Quality Factors• Quality control• Quality assurance• Quality system
27
Procedural Factors• Standard procedures• Specifications• Implementation• Selection of support
Organisational Factors• Leadership• Communications• Motivation
Root Cause Process
1. Determine the Direct Cause. 2. Determine the Root Cause. 3. Document the results
28
Record Steps Taken• Steps taken to eliminate root cause• Follow up on implementation• Verification audit• Subsequent actions
What if Problem Recurs?• Correct implementation?• Original root cause determination incorrect?• Back to the drawing board• New root cause• New corrective action
What if Problem Recurs?• Correct implementation?• Original root cause determination incorrect?• Back to the drawing board• New root cause• New corrective action
29
Summary Internal Audits are focused on determining compliance A good auditor collects objective evidence and evaluates that evidence
against stated requirements A well written deficiency contains two elements:
A statement of the objective evidence A relationship to the requirements
Audit findings are followed by: Remedial action Root cause analysis
Corrective action If corrective action fails to prevent recurrence, then its back to the
drawing board!