Embed Size (px)
Citation preview
An Evaluation of the EPA Quality Assurance Annual Report and Work An Evaluation of the EPA Quality Assurance Annual Report and Work Plan from the Perspective of a Traditional Cost of Quality ModelPlan from the Perspective of a Traditional Cost of Quality Model
Lora S. Johnson, Director of Quality AssuranceGary J. Foley, Laboratory Director
USEPA, National Exposure Research Laboratory
18th Annual National Conference onManaging Quality Systems for Environmental Programs
Cincinnati, OhioApril 16, 1999
NERLNERL
OVERVIEWOVERVIEW
REPORTING ISSUES
COST OF QUALITY CATEGORIES
ELEMENTS OF THE QUALITY ASSURANCE ANNUAL REPORT AND WORKPLAN (QAARWP)
QAARWP VS. COST OF QUALITY CATEGORIES
ORD APPROACH FOR FY98/99
A POTENTIAL SOURCE OF COST DATA
IDEAS FOR REFINING THE QAARWP
NERL
REPORTING ISSUESREPORTING ISSUES
Managers need information so that appropriate resources can be allocated to quality assurance activities.
What information is needed?
The Agency uses the Quality Assurance Annual Report and Work Plan (QAARWP) to provide this information.
How does the EPA approach compare to a traditional private sector assessment of the “cost of quality?”
How should it be measured?
How should it be collected?
How should it be represented?
NERL
COST OF QUALITY CATEGORIESCOST OF QUALITY CATEGORIES
The costs of all activities specifically designed to prevent poor quality in products and services.
Quality assurance project plansTraining in qualityQA review
PREVENTION
The costs associated with measuring, evaluating, or auditing products or services to assure conformance to quality standards and performance requirements.
CalibrationPerformance evaluation samplesTechnical system audits
APPRAISAL
NERL
COST OF QUALITY CATEGORIESCOST OF QUALITY CATEGORIES
The costs resulting from products or services not conforming to requirements or customer/ user needs.
FAILURE
Costs that occur prior to delivery
Reworking publicationsToo many signaturesFailed Experiments
INTERNALFAILURE
Costs that occur after delivery
???
EXTERNALFAILURE
NERL
ELEMENTS OF THE QAARWPELEMENTS OF THE QAARWP
QA Annual Report
Total FTE’sFederal FTE’s - QA ManagementFederal FTE’s - QA Technical SupportContract FTE’s - QA Direct SupportContract FTE’s - QA Nontechnical SupportContract $$ for Travel-Oversight Activities$ for Travel/Training about QAResources adequate?
Quality Management Resources
FROM THE EPA QUALITY MANUAL, 7/98FROM THE EPA QUALITY MANUAL, 7/98
Method used to assess training needsCourse names and suppliersAttendeesCourse goals met?
Training
NERL
ELEMENTS OF THE QAARWPELEMENTS OF THE QAARWP
QA Annual Report (continued)
Innovative PracticesTechnical AssessmentsTechnical AssistanceNew/revised QA GuidancePublicationsAwards
Management Accomplishments
FROM THE EPA QUALITY MANUAL, 7/98FROM THE EPA QUALITY MANUAL, 7/98
MSR’s - InternalMSR’s - External
Management Assessments
Work Plan
(same elements as above)Quality Management Resources
ActivitiesNERL
QAARWP VS. COST OF QUALITYQAARWP VS. COST OF QUALITY
A, PQuality Management Resources
PTraining
A, PManagement Accomplishments
AManagement Assessments
NERL
CATEGORIESCATEGORIES
A = Appraisal Cost P = Prevention Cost
ORD APPROACH FOR FY98/99ORD APPROACH FOR FY98/99
ACCOUNT FOR RESOURCES IN A SIMILAR WAY TO PROMOTE COMPARABILITY IN THE ORD RESOURCE SUMMARY TABLE
GOAL:
DIRECTORS OF QA FOR NERL, NHEERL, NRMRL MET WITH QAD REPRESENTATIVE VIA TELECONFERENCE
Quality Management Resources = Sum of the Division QA Managers and the Director’s of QA i.e. staff listed on EPA’s QA Community list, adjusted for percentage of time spent on the QA function
METHOD:
EXAMPLE:
GROUP AGREED TO ACCOUNTING RULES FOR ELEMENTS IN QUALITY MANAGEMENT RESOURCES CATEGORY
NERL
A POTENTIAL SOURCE OF COST DATAA POTENTIAL SOURCE OF COST DATA
Vice-president’s “National Performance Review” seeks to “build a government that works better and costs less.” Customer service standards are developed across the Federal government.
FEDERAL GOVERNMENT CUSTOMER SERVICE INITIATIVESFEDERAL GOVERNMENT CUSTOMER SERVICE INITIATIVES
Congress passes the Government Performance and Results Act of 1993 that seeks to improve Federal program effectiveness and public accountability by promoting a new focus on results, service quality, and customer satisfaction.
ORD’s products are knowledge based therefore “failures” are hard to measure. Customer surveys are a potential tool for measuring both success and failure.
Including customer survey data in the QAARWP would close the loop on cost of quality items.
NERL
IDEAS FOR REFINING THE QAARWPIDEAS FOR REFINING THE QAARWP
Need to compare apples to applesBetter Definition of Terms:
Need some measure of “cost of failure”
Include customer survey data:
NERL
Is information helpful for making management decisions?
Examine use of the QAARWP:
Present data graphically
QA FTEs/FTEsQA FTEs/$QA FTEs/ # APMs
Index QA resources to some measure of productivity:
HYPOTHETICAL SCENARIOSHYPOTHETICAL SCENARIOS
NERL
QA EffectPositive
Why are we bothering?
QA Hurts
QA IndexCustomer Satisfaction
Time
Customer Satisfaction
Time
QA Index
Customer Satisfaction
Time
QA Index
COMMENTSCOMMENTS
CONCLUSIONCONCLUSION
The QAARWP is a significant element of the EPA quality system. To make it a more useful management tool, some estimate of “failure” needs to be included.
NERL
Recent efforts in the area of customer satisfaction may provide a surrogate measure of external failure at least to the EPA R&D laboratory community.
