Upload
carroll-sams
View
150
Download
1
Tags:
Embed Size (px)
Citation preview
FUNDAMENTALS OF AUDITING
Carroll Sams
Reference:
FUNDAMENTALS of
QUALITY AUDITING
By B. Scott Parsowith
ContentsChapter 1 Overview of Quality Auditing 1
Chapter 2 Audit Definitions and Applications 9
Chapter 3 Phase 1--Audit Initiation and
Preparation 26
Chapter 4 Phase 2--Performance of an Audit:
On-Site Evaluation 40
Chapter 5 Phases 3 & 4--Reporting and Closure
Phases 55
Chapter 6 Establishing the Framework for an
Audit Program 64
Chapter 7 The First Seven Elements of a Quality
System 69
Chapter 8 The Second Seven Elements of a
Quality System 78
Chapter 9 The Final Seven Elements of a
Quality System 87
Chapter 10 Statistical Process Control 96
Chapter 11 Statistical Sampling for Auditing 102
Chapter 1Overview of Quality Auditing
1
Audit Overview
• Auditing is an art and a science.
• Audits are snapshots in time.
• Audits should not be confrontations.
• Auditors investigate, verify, and confirm.
• The audit should be used as a tool for continual improvement of the quality management system.
2
Auditor Knowledge
• Quality management system requirements
• Product specifications
• Process procedures
• Investigation techniques
• Sampling techniques
• Communication techniques
3
Auditor Traits
• Observant
• Honest
• Investigative
• Questioning
• Through
• Communicative
• Adaptable
• Cooperative
• Polite
• Professional
• Persistent 4
Two Types of Quality Standards
• System standards– Examples
• ANSI/ISO/ASQ Q9000 standards
• QS-9000 automotive standards• ISO 14000 environmental standards
• Product standards (specifications)– Examples
• Chemical and physical property requirements
• Dimensional tolerances
5
Review and Verify Documentation, Implementation, and Effectiveness of
• Policy manuals
• Procedure manuals
• Work instructions, drawings, etc.
• General quality management system requirements
• Individual product specifications
6
Audits Can
• Enhance internal quality management systems
• Be part of continual quality improvement efforts
• Help establish trusting relationships internally and externally
• Find problems before someone else does!
7
Audits Are Not
• Acceptance of product
• Punch lists
• Public hangings
• Fault Finding
• Ill-prepared
• Antagonistic
• Biased
• Rambling
8
Chapter 2Audit Definitions and Applications
9
What Is an Audit?• ANSI/ISO/ASQC Q10011-1-
1994: “A systematic and independent examination to determine whether quality activities and related results comply with planned arrangements and whether these arrangements are implemented effectively and are suitable to achieve objectives.”
• Quality Applications:– System audit– Process audit– Product audit 10
III
III
Institute of Internal Auditors
• Definition from IIA– “Internal auditing is an
independent, objective assurance and consulting activity designed to add value and improve an organization’s operations. It helps an organization accomplish its objectives by bringing a systematic, disciplined approach to evaluate and improve the effectiveness of risk management, control and governance processes.”
11
Additional Audit Terms• Compliance audit
– Compliance to standards– Review of processes– Review of data
• Operational audit– Time studies– Work studies
• Readiness review by internal management– ANSI/ISO/ASQ Q9000 system
• Management review– Processes, procedures, & controls
• Survey– Comprehensive evaluation 12
System Audit Definition
“ A documented activity performed to verify, by examination and evaluation of objective evidence, that applicable elements of the quality management system are appropriate and have been developed, documented, and effectively implemented in accordance and in conjunction with specified requirements.”
13
Types of Audits
• Internal audits– First-party audits
• Management system audits• Self-assessments
• External audits– Second-party audits
• Sister company audits• Supplier audits
– Third-party [extrinsic] audits• ABS Registrar audits
• OSHA Safety audits• KPMG Accounting audits
14
Types of Audits(continued)
• Survey– Comprehensive evaluation of
facilities, resources, economic stability, personnel, capability
• System audits– Verification that a quality
management system exists and is being followed
• Process audits– Verifies procedures exist, correct, &
are being followed under standard, rushed/adverse conditions
• Product audits– Verifies inspector capabilities and
correct use of specifications 15
Hierarchy of Audits
Survey
System audit
Process audit
Product audit
16
Characteristics of Audits
• Are independent of doer– Performed by a person not
responsible for production or services in area being audited
• Have measurement criteria– Specifications (quality
management system and product requirements)
– Attribute data (good/bad)
– Variables data (actual measurements)
17
Characteristics of Audits(continued)
• Are performed by competent personnel– Trained audit team members
• Are cost effective– Prevention of problems
• Are samples of the system– Isolate one small moment in time.– Represent the total population
• Must be sure the sampling procedure is correct.– Avoid making incorrect
assumptions 18
Two Types of Sample Process Errors
• Type I error (alpha error)– We assume something is
unacceptable when it is actually acceptable
• Producer’s Risk
• Type II error (beta error)– We assume something is
acceptable when it is actually unacceptable
• Consumer’s Risk
• Sample consideration– Size, amount, location, & errors
19
Use of Audits
• Continual improvement efforts– Internal and external
• Benchmarking
• Value-added suppliers
• Certification– Internal (quality management
system)
– External (suppliers)
Note: Do not use audits alone to determine acceptance of product.
20
Why Audit?
• Competition– External--We want low-cost, high-
value suppliers– Internal--We want to verify our
systems are adequate/effective
• Regulation– Imposed by customers
– Self-imposed– General quality standards or
internal quality standards
• Self-preservation to audit ourselves before someone else does– OSHA 21
Audits Improve Performance if They
• Meet management needs– Provide feedback on areas
needing improvement
• Provide a forward look in time– Can predict where process
controls or product measurements are needed
• Measure effectiveness and compliance to internal and external policy and contract requirements
22
Who Is the Customer?
• There are three customers (clients) we must satisfy on each audit.– Auditee
• Person being audited (internal or external)
– Boss or manager• Give authority to perform internal
audit
– Buyer of the service (client)• Buyer of service on external audit
23
What Managers Do• Responsible for entire system
and its various processes
• Continual process improvement– Plan– Do– Check (Study)
• audits address the “check” function of the cycle by checking and monitoring a system
– ActPlan
DoCheck
Act
24
Audit Administration
• Formal audit procedures– Initiation and preparation 25%
• Areas to be audited
• Team composition and training• Notification of auditee• Checklist development• Transportation
– On-site audit performance 50%
– Reporting and closure 25%• Audit report• Audit records• Tracking and closure
25
Chapter 3Phase 1 -- Audit Initiation and Preparation
26
Audit Initiation Phase
• Audit initiated by client– Develop audit plan
• Lead auditor (audit manager) and client (approves)
• Time allocated for audit
– Client• Determine need for audit• The purpose of audit
• The type of audit
• The objectives of audit
– Lead auditor• Scope of the audit
27
The Preparation Phase
• Items to be covered in the audit plan, which must be defined:– Objective [purpose, manuals,
procedures, work instructions]
– Scope [facility, product, process, system elements, departments]
– Resources [trained auditors, technical expertise, team size]
• Length of audit =
< 500 employees: 1-3 days
500-1000 employees: 4-5 days
– Initial contacts
– Checklists– Past history 28
Lead Auditor Responsibilities
• Prepare the audit plan– 50% of work
• Select the audit team– Based on plan
• Brief the audit team– Safety issues
– Brief auditee
• Submit a report– 2 weeks maximum
29
Desirable Auditor Characteristics
• Knowledge– Quality, production, engineering
& procurement– requirements– techniques
• Open-mindedness
• Patience
• Interest
• Tenacity
• Professional attitude/integrity
• Good listening skills30
Desirable Auditor Characteristics [continued]• Inquisitiveness
• Good verbal/written skills
• Analytical skills
• Honesty
• Ethics
• Diplomacy
• Good planning skills
• Experience
• Objectivity
• Empathy31
Auditor Musts
• Knowledge of:– Verification techniques
• Show me…
• Objective evidence
– Standards– Sampling techniques– Human nature
• Four faces of Frank
– Time management• You are in control
32
Undesirable Auditor Characteristics
• Argumentative
• Opinionated
• Lazy
• Easily influenced
• Inflexible
• Impulsive--jump to conclusions
• Gullible
• Uncommunicative
• Devious
• Poor at planning
• Unprofessional 33
Contact the Auditee• Informal contact
– Date
– Purpose– Scope
• Formal contact by letter/memo– Time and date of audit– Purpose– Scope– Activities to be audited– Interfacing organizations– Applicable quality requirements– Identification of team members– Preliminary schedule 34
Perform Desk Audit
• Review quality system documentation of auditee– Policy Manual/Procedures
• Determine what control systems are in place– What facts are available?
• What is the product?– Become familiar with product
• Review prior audits
• Investigate exemptions or waivers
• Review performance history35
Develop Checklists that Provide:
• Structure– Define before audit/use team
• Required coverage– Not designed to stifle creativity.
• A communication document– A prompt to your next question.
• A place to record data– Objective evidence is required for
anything less than satisfactory.
• Help in time management– Pace of the audit
36
Auditors--Points to Consider
• Auditors can use:– Canned checklists– Yes/no checklists [attributes]
– Rating checklists [variables]
• Things to consider when developing checklists:– What is process during normal
operations?• Rushed conditions?• When there is a problem?
37
Summary of the Preparation Phase
• Define the audit objective
• Establish the audit scope
• Allocate resources
• Contact the auditee
• Develop checklists
• Review history
• Understand the process and control systems
38
Results of the Preparation Phase
• Audit plan– auditee, purpose, scope, activities
to be audited, team, standards
• Checklist questions
• Items and reference location of each requirement
• Initial evaluation– Based on documentation received
• Desk audit• Past audit results
• Plan for action– Areas needing attention/audited 39
Chapter 4Phase 2--Performance of an Audit: On-Site Evaluation
40
The Performance Phase
• Hold opening meeting
• Conduct on-site audit
• Review controls of quality management system
• Verify that the system is working– Is it documented, implemented,
and effective?
• Share information
41
The Opening Meeting
• Must have:– Entire audit team present– Auditee’s plant manager/staff
• Requires two-way communication– Trade introductions– Restate objectives– Form initial impressions– State any areas of concern
– Settle logistics; space, lunch, etc.
• Should take no more than 30 minutes 42
The Opening Meeting [continued]
• If needed:– Allow 15 minute maximum for
auditee to present quality management system enhancements
• Distribute the checklist(s)
• Define the audit schedule
• Be on time
43
On-Site Evaluation
• Auditors can be compared to florists:– Collects evidence [flowers]
– Puts it on worksheets [baskets]– Arranges the data [flower
arrangement]– Delivers the report [delivers
flowers in baskets]
44
Causes of Quality Issues
• Lack of top management support
• Lack of organization
• Lack of training
• Lack of discipline
• Lack of resources
• Lack of time
• Lack of teamwork
• Lack of knowledge
• Lack of consistency– Look for symptoms of these
issues 45
Evaluating Information• Formal [documented] control
systems vs. Informal [understood/verbal] control systems– Does top management believe in
quality and are these beliefs communicated throughout the organization?
– Is the control system used by management?
– Is the control system adequate?• Are key variables identified?
– Is the control system working?• How is it monitored? 46
Verification Methods
• Three primary methods used:– Tracing
– Corroboration
– Sampling [discussed in Chapter 11]
A Contract Random
HorizontalVertical
47
Tracing Techniques
• Techniques of tracing– Contract -- Series of audits
performed on a contract as the contract progresses [2nd party]
– Random -- Review what is taking place at time of audit [3rd party]
– Horizontal• Backward -- From the end of
process [1st, 2nd, 3rd party]
• Forward -- From the beginning of process [1st, 2nd, 3rd party]
• Middle -- Backward or forward from some critical point in process [1st, 2nd, 3rd party]
48
Tracing Techniques [continued]
• Vertical -- Reviews management layers for elements such as quality goals and quality communications
• Departmental -- Reviews numerous quality elements within a department
• Element -- Reviews a quality element in various departments
49
Corroboration
• A statement is not a fact until it is corroborated by someone else or is verified by a document.– The facts must agree:
• Two different auditors• Two different records• Two different interviews
– Combination of the above
• Tie conclusions to what is tangible– Qualitative [some documents reviewed
did not have…]
– Quantitative [out of 50 documents reviewed, 5 were found to have…]
• Forms, records, procedures50
Sources of Data Collection• Physical evidence
– Reinspect, retest, recheck
– Documentation and records
• Sensory observations– Rusty parts
– Nonconforming parts stored with good parts
– Observation of tasks being performed
• Comparisons and relationships– Patterns and trends
• Interviews and questions– Valuable source of information
– No place for sexism or biases 51
Effective Interviews
1.Put person at ease.– Your presence is threatening.
2. Explain your purpose.– Demonstrate competence.
3. Use proper questioning techniques.– Ask open-ended questions.– Use “pregnant pause”.– Ask why five times/five w’s.
4. Analyze and verify what is said.– Verify a claim.– Write notes out loud/no secrets.
5. Explain your next step.– Conclude cordially. 52
Brief the Auditee During the Audit
• Short and to the point– Discuss verifications and
concerns– About 5-10 minutes– In each area as you progress
through the facility
53
Auditor Team Meetings
• Informal– End of each day with audit team
• Auditor sharing– Facts, conclusions, problems
• Replanning– Changes to the audit schedule
• Reporting– Preliminary audit results
• Status of the audit or areas covered– Areas completed, areas to be
examined, concerns & issues 54
Chapter 5Phases 3 and 4--Reporting and Closure Phases
55
Reporting Phase
• Informal report– Exit meeting--60 minutes
• Formal report– Written & distributed within two
weeks after the audit
“Clear and precise reports”
• Statements of noncompliance contain– Requirement– Location
– Evidence– Finding(s) 56
The Exit Meeting
• Same people as opening meeting
• Lead Auditor presents summary
• Disclaimer about sampling
• Lead auditor reads findings
• Explain follow-up and corrective action response process.
57
Findings• Observation
– Write what you were doing.• “While auditing Purchasing I found
drawings for new suppliers were not being marked on form…”
• Attribution– Restate system’s deficiencies
relating to the standard or written documentation of company.
• “This is a finding against Procedure P01-01 which requires all new drawings sent to suppliers…”
• Explanation– Explain why this is a finding
• “The drawing must be recorded to ensure supplier has latest version.”
58
Degree of Severity of Findings
• The report should order findings by their importance.– Major finding--conflict with
contractual or procedural requirements. [6 maximum]
– Minor finding--Isolated observations in conflict with contractual or procedural requirements. [6 maximum]
– Concerns--Observations that may not be part of the required quality management system, but are worthy of further consideration.
59
Content of the Formal Audit Report
• Formal audit report from audit team to client should include:– Purpose and scope of audit
– Participant’s names– Background information– Summary of the results
– Identified weaknesses in order of importance
– Date and signature of lead auditor
60
The Closure Phase
• Auditee has 30-45 days to respond with plan of action.
• Auditee’s corrective action evaluated.
• Formally close corrected items.
• Items requiring on-site verification are reaudited.
• Letter of closure issued.
• All documentation filed.
61
Retention of Audit Documents
• “Working papers” should be maintained for at least one audit cycle for follow-up audits.
• Follow the required Quality Record Procedure requirements.
62
Pitfalls to Avoid
• Inadequate planning/preparation
• Inadequate communication prior to audit
• Lack of clearly defined scope
• Inadequate knowledge of standards
• Improperly trained auditors
• Failure to reevaluate implemented corrective actions
63
Chapter 6Establishing the Framework for an Audit Program
64
Establishing the Framework for an Audit Program• Top management gives authority
to establish audit function– Policy Q17-01
– Procedure P17-01
• Audit administrator is assigned– Management Representative
• Purpose and scope established– Minimum compliance purposes– Continuous quality purpose
65
General Principles to Administering an Audit
• Objectivity– Independence– Attack problem, not people
– Avoid conflict– Verify data
• Confidentiality– Information should not be shared
by auditor from company to company
66
Audit Staff
• Auditors are selected on the basis of:– Experience
– Knowledge– Communication skills– Personal traits
– Judgement
67
Audit Records Retention
• Permanent records– Notification letter– Audit plan
– Pre-assessment questionnaire– On-site audit/evaluation– Working papers– Audit report letter– Closure letter
– Auditor training/qualification
• Nonpermanent records– Notes, supporting documents
• Audit records kept 3 years 68
Chapter 7The First Seven Elements of a Quality Management System
69
The First Seven Elements of a Quality Management System
1. Management responsibility
2. Quality system
3. Contract review
4. Design control
5. Document control
6. Purchasing
7. Customer-supplied product
70
1--Management Responsibility [checklist]
• Management Responsibility
• Quality policy
• Planning
• Responsibility, authority & communication
• Management review
• Resource management
• Infrastructure
• Work environment
• Customer satisfaction
• Improvement 71
2--Quality System [checklist]
• General requirements
• Documentation requirements
• Quality Manual
• Quality management system planning
I quality manual
II procedures
III work instructions
IV Forms, tags, labels72
3--Contract Review [checklist]
• Customer focus– Interested parties
• Customer-related processes– Determine mutually acceptable
processes
• Review of requirements related to the product
• Customer communication– Product information– Amendments– Customer complaints/feedback
73
4--Design Control [checklist]
• Design and development– Planning– Inputs
– Outputs– Review– Verification– Validation
• Control of design and development changes
74
5--Document & Data Control [checklist]
• Control of documents– Approval– Review and update
– Current revision status– Latest versions at points of use– Documents remain legible and
retrievable – External documents are controlled– Obsolete documents are identified
75
6--Purchasing [checklist]
• Purchasing– Purchasing process– Purchasing information
– Verification of purchased product
76
7--Control of Customer Supplied Product [checklist]• Customer property
– Ingredients or components– Packaging materials
– Customer material in storage– Transport to a third party– Customer intellectual property
77
Chapter 8The Second Seven Elements of a Quality Management System
78
The Second Seven Elements of a Quality Management System
8. Product identification and
traceability
9. Process control
10. Inspection and testing
11. Inspection, measuring, and
test equipment
12. Inspection and test status
13. Control of nonconforming
product
14. Corrective & preventive action79
8--Product Identification and Traceability [checklist]• Identification and traceability
– Drawings– Documents
– Bar codes– Computer– Tagging– Labeling– Handling/storage
• Segregation/quality status
• Quantities
• Records of traceability
• Shipment records 80
9--Process Control [checklist]
• Product realization– Planning of product realization
• Production and service provision– Control of production and service
provision– Does not include servicing
– Validation of special processes
• Monitoring and measurement of processes– Reaction time– Yield 81
10--Inspection and Testing [checklist]• Monitoring and measurement of
product [from receiving inspection to product delivery]– Receiving inspection and testing
• Lot sampling plans• Skip lot
• Certifications
– In-process inspection and testing• First piece setup• Process audits
– Final inspection and test• 100% inspection• Lot sampling• Product audits
82
11--Inspection, Measuring, and Test Equipment [checklist]
• Control of measuring and monitoring devices– Calibration procedures
– Calibrated at defined intervals– Identified and safeguarded– Protected from damage– Personnel trained– Maintain records– Control environment– Confirm computer software
83
12--Inspection and Test Status [checklist]
• Identification and traceability– Status of material maintained
throughout process• Paper records
• Computer records• Product tags• Bar codes• Traceability of product with releases
– Identification of responsible personnel for conformance decisions
• Records of product release
84
13--Control of Nonconforming Product [checklist]
• Identification [NCMR]– Marking, tagging, computer
• Segregation– Physical, records, computer
• Disposition– Evaluation by authorized personnel
• Accept with repair by concession• Accept without repair by concession• Rework to specifications• Return to supplier [vendor]• Regraded for alternate application
• Scrap85
14--Corrective and Preventive action [checklist]• Identification of problem
– Causes• Common or special
• Repeat or isolated• Root cause determination• Evaluation of impact
– Costs
– Performance
– Safety
– Customer satisfaction
• Implementation of corrective actions [prevent recurrence]
• Implementation of preventive actions [prevent occurrence] 86
Chapter 9The Final Seven Elements of a Quality Management System
87
The Final Seven Elements of a Quality Management System
15. Handling, storage, packaging, preservation, and delivery
16. Quality records
17. Internal quality audits
18. Training of personnel
19. Servicing
20. Statistical techniques
21. Quality cost analysis
88
15--Handling, Storage, Packaging, Preservation, and Delivery
• Handling– Prevent damage/deterioration
– Maintain identification
• Storage– Maintain identification/traceability
• Packaging– Prevent damage/maintain identification
• Preservation– Protect from deterioration
• Delivery– Protection of quality after inspection
• Feedback system 89
16--Control of Quality Records• Examples of quality records:
– Management review– Employee education, skills,
training, and experience– Production criteria– Design records– Supplier evaluations and actions– Special process validations– Customer property damage
– Calibration and verification– Internal audits– Authorizing release of product
– Nonconforming product– Corrective/preventive action 90
17--Internal Quality Audits• Scheduling
– Frequency, dates, time involved
• Objective– Type of audits
• Scope– Location, element, audit team
• Procedures– Training program, checklists
• Implementation– On-site evaluation
• Reporting– Corrective actions
• Follow-up 91
18--Training of Personnel• Training needs planned/identified
– Certifications for specific employees
• Training is provided– Competency based on qualifications
• Employee training is verified– Qualification/certification on the
basis of;• Education, training, experience, and
evaluation of skills verified
• Human factors are considered– Inspire continuous improvement
– Be knowledgeable about quality– Provide a good work environment
92
19--Servicing
• Field servicing of products or services to meet requirements– Performed to written procedures
• [P19-01]
• Liability– Warranty
• Claims/traceability
– Ability to recall• Records, designs, specifications,
process controls, testing results
• Safety– Risks
• [MSDS] 93
20--Statistical and Analytical Techniques
• Process control techniques– Flowcharts– Control charts
– Trend charts– Histograms– Pareto charts– Cause/effect diagram– Inspection plans
• Assessment tools for applications– Benchmarking– FMEA – Reliability predictions 94
21--Quality Cost Analysis• Standard cost system
– Standard cost vs actual cost
• Cost of nonconformance– Value lost to company
• Quality cost system– Prevention--Blue $
• QIP process
– Appraisal--Blue $• Inspection, calibration, testing
– Internal failure--Red $• Scrap, rework
– External failure--Red $• Returns, warranty claims
95
Chapter 10Statistical Process Control
96
UCL
LCL
X
Key Process and Product Variables
• Process map [flowchart]– Identification of:
• Quality control points
• Location of activities• Who is responsible for activities
– Comparison to external and internal customer requirements [quality plan]
– Process and inspection data feedback to control system
Start or end
Go/no go
Task
97
Variability of Processes and Products
• Causes of variability in processes– Man, machine, material, methods,
mother nature
• Evaluation of the causes of variability– Special causes– Common causes
Process
MethodMaterial
MachineMan
Mother Nature
98
Statistical Control charts and Process Stability
• Statistical limits [based on past data]– Minimum 20 samples
• Statistical control– Common cause variation
• Data points outside limits– Out of control
• Data points exhibiting patterns– Runs, trends, 7 consecutive
• Determination of special causes or process changes– Notes 99
Types of Data Collection
• Variables Control Charts– Measurement
• X-bar, R control chart– Samples are averaged, range plotted
• Individual, moving range [Xmr]– One unit measured, range plotted
• Attributes Control Charts– Count
• Nonconforming products– P charts
– NP charts
• Nonconformities per unit– C charts
– U charts 100
Signals vs Noise
• Out of control signals or patterns– Data points beyond the control
limits
– Data points in a run [7 or more points above or below central line]
– Data points in a trend [7 or more points going up or down and crossing the central line]
– Data points in a periodic configuration
– Data points stratified 101
Chapter 11Statistical Sampling for Auditing
102
Aspects of Sampling
• Sampling obtains information from a part of the total population or field
• Sampling is used in:– Accounting– Business– Industry– Quality auditing
102
Methods Used Must Suit Needs of the Audit
• Types of approaches– Estimation sampling
• Provides the auditor with an estimate of the frequency of occurrences of an event in the field and considers sampling error
• Desired sample precision is based on risk and consequences of making an error
– Inventory accuracy vs airplane engine defects
– Discovery sampling• It is used to find one nonconformance
within a given field size
– Acceptance sampling• Is based on a continuous process and is
not suggested for use in quality audits103
Sample Selection
• Establish what is to be sampled depending on the scope of the audit– Critical work instructions
– Highly stressed, high-volume, high precision areas
– Contributory factors that make the process successful
– Time allocated
104
How to Select an Unbiased Sample
• Equal chance of selection
• After process is running consistently
• Selection of predetermined number of random samples– Use of random number tables– Use of calculator
– Use of a book
105
Deck of cards to show biased sample