Transcript Welcome, Panel of Examiner and Process Development Members!

Welcome, Panel of Examiner
and Process Development
Members!
Washington State Quality Award
PEPD #1 Training
2008
First things first!
• THANK YOU!
• WSQA’s mission: Improving the way we live, learn,
and work in Washington by helping organizations
improve through the use of the Baldrige Criteria
for Performance Excellence
WSQA Return Examiner 2008
2
Objectives For This Class
• WSQA Overview
• Ethics Review
• PEPD Time commitment
• PEPD Roles and Responsibilities
• Lite and Full Examination Process Review
WSQA Return Examiner 2008
3
Introductions
• Name, organization, what you do
• Questions and expectations?
• What do you like to do when
you’re not working?
• PEPD Members
WSQA Return Examiner 2008
4
1. WSQA organization overview
What is WSQA?
• Created by State law to:
• Promote excellence
• Recognize achievement
• Educate and train
• Patterned after the Baldrige National Quality Award
• Awards presented annually by the Governor
• Awarded to private, public, and not-for-profit
organizations in manufacturing, service, education,
and healthcare
WSQA Return Examiner 2008
6
What is WSQA?
WSQA is a 501(c)(3) nonprofit organization
supported by
•
•
•
•
Corporate donations
Individual memberships
Workshop, collaborative and conference fees
Many (many, many!) volunteer hours
WSQA Return Examiner 2008
7
Who is WSQA?
• Examiners
• Lite Assessment
• Full Assessment
• Certification follows assessment
• Senior Examiners
• Level 3 Team Leads
• Scorebook Editors
•
•
•
•
•
Panel of Examiner & Process Development (PEPD) Mentors
Panel of Judges
Board of Directors
Advisory Panel
Executive Director and support team
WSQA Return Examiner 2008
8
2. WSQA Ethics Review
• Ethics Statement
• Conflict of Interest form and statement
WSQA Return Examiner 2008
9
Conflict of interest issues
• WSQA seeks to avoid conflict of interest to
protect process integrity
• Conflicts are frequent!
• First step in assignment process: determine
any conflicts
WSQA Return Examiner 2008
10
Conflict of Interest Exercise
• Pick any organization (not yours)
• Complete the Conflict of Interest form for
that organization
WSQA Return Examiner 2008
11
3. PEPD Time Commitments
• Monthly meeting 1 ½ hour
• Lite team mentoring Quarterly
• E-mails ½ hour
• Review examiner inputs stage 1- 1.5 hour
• Follow up with team members- .5-1hour
• Review consensus comments- 2 hours
• Consensus meeting 4-6 hours
TOTAL 10 hours for lite process
• Team involvement 1-2 hours /month (optional)
TOTAL TIME COMMITMENT: 4- 6 hours/month
WSQA Return Examiner 2008
12
4. PEPD Roles & Responsibilities
•
•
•
•
•
Process Ethics
Process Integrity- Team follows process
Process Completion
Examiner growth- Mentoring
Process Improvement
• Monthly meetings
• Examiner training input
• Process improvement input
WSQA Return Examiner 2008
13
PEPD TEAMS
•
•
•
•
Examiner Training
Process Review
Governance
Repeat Applicant Process
WSQA Return Examiner 2008
14
5. WSQA Full and Lite Process
Comparison Table- website
http://www.wsqa.net/criteria.php
Time Table – website
WSQA Return Examiner 2008
15
Receive applications
Stages of the Assessment Process
Individual
Scorebooks
Stage 1:
Independent Review
Final product
Consensus
Scorebook
Stage 2:
Consensus Review
Lite or Full
application?
Lite
Editor
updates
scorebook
Consensus
Scorebook
Site visit
scorebook
Lead & Mentor
meet with
Judges Panel
WSQA
updates
scorebook
Feedback
report
Executive
Briefing
Full
Lead & Mentor meet
with Judges Panel
Selected for
site visit?
No
Yes
Stage 3:
Site Visit
Judges recommend
award level to
WSQA Board
WSQA Symposium
& Awards
Ceremony
Preparing for independent review
•
•
•
Read the criteria
Read the application
Familiarize yourself with the online tool
•
•
•
•
Site: http://www.shawresources.com/
Login information in email from WSQA
Training at https://cc.readytalk.com/play?id=o6d1b4j9
(part 1) and https://cc.readytalk.com/play?id=gc87umbs
(part 2)
Check in with Team Lead on Key Factors and after
entering category 1 and associated results
WSQA Return Examiner 2008
17
Independent review
1.
2.
3.
4.
5.
6.
Select the most relevant Key Factors (from list)
Read the criteria for the category
Read (and annotate) the relevant section of the
application
Identify and record the applicant’s processes
Record observations. Note missing responses and
significant observations
Recommend a scoring range for the category
WSQA Return Examiner 2008
18
Independent review steps - process
1.
2.
3.
4.
5.
Review related criteria
Read the category under review, take notes
Select relevant key factors from list
Identify & record processes
Record observations on ADLI:
•
•
•
•
Approach
Deployment
Learning
Integration
6. Identify any significant or missing responses
WSQA Return Examiner 2008
19
Record observations on “ADLI”
“Learning” refers to
• Refining the approach through cycles of
evaluation and improvement
• Encouraging breakthrough change to your
approach (innovation)
• Sharing refinements and innovations with other
relevant work units and processes
WSQA Return Examiner 2008
20
Record observations on “ADLI”
“Integration” refers to the harmonization of
•
•
•
•
•
•
•
Plans
Processes
Information
Resource decisions
Actions
Results
Analyses
to support key, organization-wide goals
WSQA Return Examiner 2008
21
Record observations on “ADLI”
Questions to ask in analyzing “Integration”
• Do the individual components of a performance
management system operate as a fully
interconnected unit?
• Is the approach aligned with the organizational
needs identified in the Organizational Profile?
WSQA Return Examiner 2008
22
Process diagnosis: key concepts
Integration examples:
• Alignment of objectives and action plans with
strategic challenges AND mission, vision, values
• Alignment of product/service delivery methods
with KEY customer requirements
• Alignment of key process measures with KEY
customer requirements
WSQA Return Examiner 2008
23
Results evaluation: key concepts
Letci
WSQA Return Examiner 2008
24
WSQA Consensus: Step-by-step
Application
Criteria
Consensus
planning call
Independent
review
scorebooks
Observations
(from Ind.
Review)
Key Factors
Editor or Lead
Category Lead
Category Lead
1. Consolidate
scorebooks
2. Review and group
team’s observations
3. Select relevant Key
Factors
Single
consensus
scorebook
Grouped
observations
4-6 (3-5)
Relevant Key
Factors
For your category/
categories only
Numbers in
parentheses apply for
Lite assessment only
WSQA Return Examiner 2008
1
WSQA Consensus: Step-by-step
1
Grouped
observations
Below-the-line
observations
Back-up’s
comments &
feedback
Category Lead
Category Lead
Category Lead
4. Select and develop
comments
5. Write rationales for
below-the-line
observations
6. Exchange feedback
with Back-up, update
comments
About 6
comments
Rationales
Updated
comments
Below-the-line
observations
Scoring
recommendation
Lite: 4-6
WSQA Return Examiner 2008
2
WSQA Consensus: Step-by-step
2
Consensusready
comments
Draft
consensus
scorebook
Updated
Comments,
score
Team
Team
Editor
7. Review team
comments, prepare
notes, enter key
theme ideas
8. Hold consensus
call(s) or meetings
9. Prepare consensus
scorebook
Updated
consensus
scorebook
Key themes,
site visit
recommendation
Notes
Draft Key
Theme topics
FULL
only
FULL
only
WSQA Return Examiner 2008
Consensus
scorebook
End
Consensus preparation and call(s)
• Each Examiner responds with agreement,
suggestions for changes, or disagreement,
stating rationales
• Category Lead facilitates consensus
• Team Lead and/or PEPD mentor leads final
scoring discussion
• Can take anywhere from 4-8 hours
(depending on what?)
WSQA Return Examiner 2008
28
Comment, scoring considerations
• “Benefit of the doubt”
• Give credit for what’s in the application
• Don’t penalize for incidental exceptions
• Not every process must show complete “DLI”
• Not every results example must include “TCI”
WSQA Return Examiner 2008
29
Comment, scoring considerations
• For LITE, only one score for entire results
Category 7
• Propose a score for results related to your
categories
• Be prepared to reach team consensus for all of
Category 7
• Using scoring language can help the
applicant understand comments and score
WSQA Return Examiner 2008
30
6. Examiner Software
Down load manual and webinar from our
website:
http://www.wsqa.net/Extimetable.php
Log on to Examiner software at:
https://www.shawresources.com/pea2007
WSQA Return Examiner 2008
31
THANK YOU!!
Your support and participation as
Examiners helps us all by helping WSQA
fulfill its mission!
WSQA Return Examiner 2008
32