Welcome, Panel of Examiner and Process Development

Members!

Washington State Quality Award

PEPD #1 Training

2008

WSQA Return Examiner 2008 2

First things first!

• THANK YOU!• WSQA’s mission: Improving the way we live, learn,

and work in Washington by helping organizations improve through the use of the Baldrige Criteria for Performance Excellence

WSQA Return Examiner 2008 3

Objectives For This Class• WSQA Overview

• Ethics Review

• PEPD Time commitment

• PEPD Roles and Responsibilities

• Lite and Full Examination Process Review

WSQA Return Examiner 2008 4

Introductions

• Name, organization, what you do• Questions and expectations?• What do you like to do when

you’re not working?• PEPD Members

1. WSQA organization overview

WSQA Return Examiner 2008 6

What is WSQA?

• Created by State law to:• Promote excellence• Recognize achievement• Educate and train

• Patterned after the Baldrige National Quality Award• Awards presented annually by the Governor • Awarded to private, public, and not-for-profit

organizations in manufacturing, service, education, and healthcare

WSQA Return Examiner 2008 7

What is WSQA?

WSQA is a 501(c)(3) nonprofit organization supported by • Corporate donations• Individual memberships• Workshop, collaborative and conference fees• Many (many, many!) volunteer hours

WSQA Return Examiner 2008 8

Who is WSQA?

• Examiners• Lite Assessment• Full Assessment• Certification follows assessment

• Senior Examiners• Level 3 Team Leads• Scorebook Editors

• Panel of Examiner & Process Development (PEPD) Mentors• Panel of Judges• Board of Directors• Advisory Panel• Executive Director and support team

WSQA Return Examiner 2008

2. WSQA Ethics Review

• Ethics Statement• Conflict of Interest form and statement

9

WSQA Return Examiner 2008 10

Conflict of interest issues

• WSQA seeks to avoid conflict of interest to protect process integrity

• Conflicts are frequent!• First step in assignment process: determine

any conflicts

WSQA Return Examiner 2008

Conflict of Interest Exercise

• Pick any organization (not yours)

• Complete the Conflict of Interest form for that organization

11

WSQA Return Examiner 2008

3. PEPD Time Commitments• Monthly meeting 1 ½ hour• Lite team mentoring Quarterly

• E-mails ½ hour• Review examiner inputs stage 1- 1.5 hour• Follow up with team members- .5-1hour• Review consensus comments- 2 hours• Consensus meeting 4-6 hoursTOTAL 10 hours for lite process

• Team involvement 1-2 hours /month (optional)

TOTAL TIME COMMITMENT: 4- 6 hours/month

12

WSQA Return Examiner 2008

4. PEPD Roles & Responsibilities

• Process Ethics • Process Integrity- Team follows process• Process Completion• Examiner growth- Mentoring• Process Improvement

• Monthly meetings• Examiner training input• Process improvement input

13

WSQA Return Examiner 2008

PEPD TEAMS

• Examiner Training• Process Review• Governance• Repeat Applicant Process

14

WSQA Return Examiner 2008

5. WSQA Full and Lite Process

Comparison Table- website

http://www.wsqa.net/criteria.php

Time Table – website

15

Stages of the Assessment Process

No

Receive applications

Stage 1:Independent Review

Individual Scorebooks

Stage 2:Consensus Review

Consensus Scorebook

Lite or Fullapplication?

Selected forsite visit?

Full

Judges recommend award level to WSQA Board

Lite

Stage 3:Site Visit

Yes

Feedback report

Executive Briefing

WSQA Symposium & Awards Ceremony

Final product

Editor updates

scorebook

Lead & Mentor meet with Judges Panel

WSQA updates

scorebook

Lead & Mentor meet with

Judges Panel

Consensus Scorebook

Site visit scorebook

WSQA Return Examiner 2008 17

Preparing for independent review

• Read the criteria• Read the application• Familiarize yourself with the online tool

• Site: http://www.shawresources.com/• Login information in email from WSQA• Training at https://cc.readytalk.com/play?id=o6d1b4j9

(part 1) and https://cc.readytalk.com/play?id=gc87umbs (part 2)

• Check in with Team Lead on Key Factors and after entering category 1 and associated results

WSQA Return Examiner 2008 18

Independent review

1. Select the most relevant Key Factors (from list)2. Read the criteria for the category3. Read (and annotate) the relevant section of the

application4. Identify and record the applicant’s processes5. Record observations. Note missing responses and

significant observations6. Recommend a scoring range for the category

WSQA Return Examiner 2008 19

Independent review steps - process

1. Review related criteria2. Read the category under review, take notes3. Select relevant key factors from list4. Identify & record processes5. Record observations on ADLI:

• Approach• Deployment• Learning• Integration

6. Identify any significant or missing responses

WSQA Return Examiner 2008 20

Record observations on “ADLI”

“Learning” refers to • Refining the approach through cycles of

evaluation and improvement• Encouraging breakthrough change to your

approach (innovation)• Sharing refinements and innovations with other

relevant work units and processes

WSQA Return Examiner 2008 21

Record observations on “ADLI”“Integration” refers to the harmonization of

• Plans• Processes• Information• Resource decisions• Actions• Results• Analyses

to support key, organization-wide goals

WSQA Return Examiner 2008 22

Record observations on “ADLI”

Questions to ask in analyzing “Integration” • Do the individual components of a performance

management system operate as a fully interconnected unit?

• Is the approach aligned with the organizational needs identified in the Organizational Profile?

WSQA Return Examiner 2008 23

Process diagnosis: key conceptsIntegration examples:

• Alignment of objectives and action plans with strategic challenges AND mission, vision, values

• Alignment of product/service delivery methods with KEY customer requirements

• Alignment of key process measures with KEY customer requirements

WSQA Return Examiner 2008 24

Results evaluation: key concepts

Letci

WSQA Return Examiner 2008

Application

Criteria

Consensus planning call

Independent review

scorebooks

Editor or Lead

1. Consolidate scorebooks

Single consensus scorebook

Category Lead

2. Review and group team’s observations

Grouped observations

Category Lead

3. Select relevant Key Factors

4-6 (3-5) Relevant Key

Factors

Observations (from Ind. Review)

Key Factors

For your category/categories only

1

Numbers in parentheses apply for Lite assessment only

WSQA Consensus: Step-by-step

WSQA Return Examiner 2008

Category Lead

4. Select and develop comments

About 6 comments

Grouped observations

1

Below-the-line observations

Category Lead

5. Write rationales for below-the-line observations

Rationales

Below-the-line observations

Category Lead

6. Exchange feedback with Back-up, update

comments

Updated comments

Back-up’s comments &

feedback

2

Lite: 4-6

Scoring recommendation

WSQA Consensus: Step-by-step

WSQA Return Examiner 2008

2

Team

7. Review team comments, prepare

notes, enter key theme ideas

Notes

Consensus-ready

comments

Team

8. Hold consensus call(s) or meetings

Updated consensus scorebook

Draft consensus scorebook

Draft Key Theme topics

Editor

9. Prepare consensus scorebook

Updated Comments,

score

Key themes, site visit

recommendation

FULL only

End

Consensus scorebook

FULL only

WSQA Consensus: Step-by-step

WSQA Return Examiner 2008 28

Consensus preparation and call(s)

• Each Examiner responds with agreement, suggestions for changes, or disagreement, stating rationales

• Category Lead facilitates consensus• Team Lead and/or PEPD mentor leads final

scoring discussion• Can take anywhere from 4-8 hours

(depending on what?)

WSQA Return Examiner 2008 29

Comment, scoring considerations

• “Benefit of the doubt”• Give credit for what’s in the application

• Don’t penalize for incidental exceptions• Not every process must show complete “DLI”• Not every results example must include “TCI”

WSQA Return Examiner 2008 30

Comment, scoring considerations

• For LITE, only one score for entire results Category 7• Propose a score for results related to your

categories• Be prepared to reach team consensus for all of

Category 7• Using scoring language can help the

applicant understand comments and score

WSQA Return Examiner 2008

6. Examiner Software

Down load manual and webinar from our website:

http://www.wsqa.net/Extimetable.php

Log on to Examiner software at:https://www.shawresources.com/pea2007

31

WSQA Return Examiner 2008 32

THANK YOU!!

Your support and participation as Examiners helps us all by helping WSQA

fulfill its mission!