Transcript Document

2008 ONE Award
Examiner Training
May 2008
ONE Award
Today’s session is sponsored by:
ONE Award
Presenting Sponsors
ONE Award
Gold Sponsors
ONE Award
The ONE Award is aimed at recognizing
best practices in non-profit organizations.
We seek to identify, promote and reward
our region’s
Organizations of Noteworthy
Excellence
ONE Award
Agenda
•
•
•
•
•
•
•
•
Overview & Process
Roles, Responsibilities & Expectations
Criteria, 2008 Criteria Changes
Reviewing and Analyzing the Application
Independent Review
Consensus Process
Site Visit Process Overview
Questions & Comments
ONE Award
Timetable
•
•
•
•
•
•
•
Applications Due – May 23 at 5:00 PM
Applications Distributed - May 28
Consensus Review Due – June 23
Site Visit Selection – July 8
Site Visit Training – July 11 or 14
Site Visits Conducted - July 21- August 1
Site Visit Reports Due - August 15
ONE Award
Timetable (cont.)
•
•
•
•
Finalists & Winners Determined – late August
Finalists Announced – early September 7
Award Presentation Luncheon – Oct. 22
Feedback Reports Sent - October 31
ONE Award
Key Roles & Responsibilities
•
•
•
•
Examiners
Selectors
Award Team
Sponsors
ONE Award
Examiner Standards
• Confidentiality
• Conflict of Interest
• Code of Ethical Conduct
• Signed Statement Required
ONE Award
Understanding the Criteria
•
•
•
•
•
•
Organizational Profile, 4 Categories
“What” (Approach) questions
“How” (Deployment) questions
Objective
Non-Prescriptive
Weighted Emphasis on Results
ONE Award
Understanding the Criteria
• Organizational Profile
• Sets the parameters for examiner review
1. Key Characteristics
– Purpose, Vision, Mission, Values
– Programs, Services, Customers, Workforce Profile
– Regulations, Governance, Certification
ONE Award
Understanding the Criteria
2. Competitive Environment
– Numbers and types, competitive success factors
Strategic Challenges and Advantages
– Mission related, Operations, HR
ONE Award
Understanding the Criteria
Category 1 - People (150 points)
Leadership, Learning & Motivation
Category 2 - Principles (200 points)
Customer Focus, Governance & Social
Responsibility
Category 3 - Process (200 points)
Strategic & Operational Planning, Knowledge
Management, Process Management
ONE Award
Understanding the Criteria
Performance Category
1. People Results (100 points)
Employee/Volunteer Satisfaction, Learning & Development
2. Principles Results (250 points)
Product/Service Performance, Financial & Market, Governance,
Social Responsibility
3. Process Results (100 points)
Operational Performance, Accomplishment of Strategy &
Action Plans
ONE Award
Reviewing Smaller Organizations
• Size does not affect criteria applicability
• Size does provide context (calibrate your
expectations)
• Systems likely to be less formal
• Don’t have the people or resources to dedicate
to all areas in same way as others
• Good governance still an imperative
• Ethical behavior is a significant requirement
ONE Award
2008 Criteria Changes
• Italicized in criteria
• Organizational Profile
– Key Performance Indicators
– Workforce Profile – key segments and their
requirements/expectations
– Strategic Challenges/Advantages
– Core Competencies
ONE Award
2008 Criteria Changes
• Category 1 - People
– 1.1 c – Senior leaders communication
– 1.1 e – Organizational improvement, focus
on mission and goals, organizational
learning
– 1.3 c – Evaluating training needs
ONE Award
2008 Criteria Changes
• Category 2 - Principles
– 2.1 a – Use of voice of the customer to understand
customers’ changing needs
– 2.1 c – Reviewing listening and learning methods to
meet changing needs
– 2.2 b – Ensuring that customer access mechanisms
meet customers’ needs
– 2.4 b – Evaluation and improvement of governance
system – staff and Board, individually and collectively
ONE Award
2008 Criteria Changes
• Category 3 - Process
– 3.1 d – Strategic objectives – address strategic
advantages and challenges
– 3.2 b – Tracking progress on action plans
– 3.3 a – Determining resources needed to
complete action plans
– 3.3 c – Emergency/disaster preparation
– 3.4 d – Driving process improvement, engaging
workforce members in the process
ONE Award
2008 Criteria Changes
• Category 3 - Process (cont.)
– 3.6 a – Determining core competencies,
relationship to creating value for customers
– 3.6 b,c – Old 3.7 (2007) combined into 3.6 Key
work processes for creating value, contributing
to success and sustainability, supporting value
creation
– 3.6 d – Process performance measurement
ONE Award
2008 Criteria Changes
• Category 4 – Performance
– 4.7 a – Process performance – key work
processes for creating customer value and
support processes
ONE Award
Independent Review
ONE Award
Key Factors Worksheet
•
•
•
•
•
Provides the context for individual review
What do they do?
Who are their customers?
Who is their workforce?
What are the key factors (internally and
externally) for the organization?
• Key performance indicators
• Core competencies, challenges, advantages
ONE Award
Key Factors Worksheet - Sample
• Key Characteristics
– Provides mental health counseling to youth/adults in 9 local
clinics in 3 county area
– Key customers include youth under 17 years old and their
parents/guardians – requirements include transportation, quick
access to service, confidentiality, collaborative approach with
parents
– Key performance indicators – lives successfully impacted,
customer satisfaction, retention through course of treatment,
funder loyalty
– 9 member Board, rotating 3 year term, 10 employees, 15 regular
clerical volunteers, 300 event volunteers
– Key funders, regulators include…
ONE Award
Key Factors Worksheet - Sample
• Competitive Environment
– Competitive success depends upon easy access to services
and ability to operate efficiently
– Competes with private practitioners as well as 3 other local
agencies who offer similar services
– Competes for funding with numerous small and large local
and regional agencies
– Competes with more lucrative private practices for
qualified, passionate staff
– Competes with community service agencies for volunteers
ONE Award
Key Factors Worksheet - Sample
• Organizational and Strategic Challenges,
Advantages, Core Competencies
– Core competencies – individualized counseling approaches,
customer service
– Changing requirements from funders place greater
demands on employees
– Highly competitive market for qualified counselors
– IT systems make information access difficult
– Potential county budget cuts threaten survival
ONE Award
Reviewing the Application
•
•
•
•
•
•
•
•
Dual customer - applicant and selectors
Bring your knowledge
Not an audit
No advice-giving permitted
Review the Criteria
Review the Glossary of Key Terms
Read the Application completely
Complete the Key Factors Worksheet
ONE Award
Reviewing the Application
• Approach – Deployment – Results
• Approach – methods used to address
criteria requirements
• Deployment – the extent to which the
approach is implemented
• Results – outcomes achieved related to
criteria requirements
ONE Award
Reviewing the Application
• Tips for Effective Application Review
–
–
–
–
–
–
Keep the applicant’s Key Factors in mind
Remember the size of the applicant
Don’t assume anything, but give the…
Benefit of the doubt
The applicant wants developmental feedback
Don’t block a winner too soon
ONE Award
Preparing Item Worksheets
Item 1.1 - How do senior leaders lead?
Key Process
Setting Vision and
Values
Encouraging Frank
Two-Way
Communication
Senior Leaders –
Communicating
Key Decisions
Promoting an
Environment of
Legal and Ethical
Behavior
Creating a
Sustainable
Organization
Evidence of systematic approach
Evidence of deployment
Evidence of systematic process
improvement
ONE Award
Preparing Item Worksheets
Item 1.1 - How do senior leaders lead?
Key Process
Evidence of systematic approach
Evidence of deployment
Evidence of systematic process
improvement
Setting Vision and
Values
+ Reviewed and revised by Board
- No indication of timing for review
+ Posted throughout office
+ Shared at staff meetings
- Not clear how Vision and Values shared
with volunteers
- No evidence of process review or
revision/improvement
Encouraging Frank
Two-Way
Communication
+ Forums held with employees and
volunteers
+ Informal walk-around process, open
door policy
+ Forums held four times per year for
employees
+ Annual volunteer forum
- Not clear when walk-arounds are
conducted or by whom
+ Feedback collected for improvement at
forums
- Not clear how feedback used
-No evidence of overall review of
communications methods
Senior Leaders –
Communicating
Key Decisions
+ Call ”all organization” meetings as
required
- Not clear how volunteers are included
+ Feedback on communication
effectiveness in workforce satisfaction
survey
Promoting an
Environment of
Legal and Ethical
Behavior
+ Values include commitment to ethics
- Ethical standards not indicated
-No process to share expectations or
reinforce
- No indication of sharing expectations to
new volunteers
- No evidence of review of approach to
promoting desired environment
Creating a
Sustainable
Organization
Sustainability factors are reviewed yearly
with Board
+ Leaders attend futurists conference
+ Leaders meet with employees 2 times
each year to discuss key issues
- No evidence of process to review and
improve approach to identify and
communicate sustainability factors
ONE Award
Scoring Item Worksheets
Item 1.1 - Scoring (place an “X” beneath the most appropriate scoring band for both Approach and Deployment)
0-5%
Approach
Not addressed
10-25%
No systematic
approach,
information is
anecdotal
30-45%
50-65%
70-85%
90-100%
A systematic
approach is
evident that
responds to
most of the
basic Category
requirements
A systematic
approach is
evident that
responds to all
requirements
of the Category
A mature,
effective,
systematic
approach that
responds to all
of the
requirements
of the Category
A mature ,
effective,
systematic
approach is
evident that is
of national role
model quality
The approach
is fully
deployed with
no gaps
The approach
is fully
deployed and
has no gaps or
weaknesses in
any areas
X
Deployment
No deployment
is evident
Early stages of
deployment
The approach
is deployed in
most areas
X
The approach
is fully
deployed with
very few gaps
ONE Award
Scoring – Categories 1-3
Category 1 - Scoring (place an “X” beneath the most appropriate scoring band and choose a percentage
score within the band that ends with either 0 or 5)
Approach/
Deployment
0-5%
10-25%
30-45%
50-65%
70-85%
90-100%
Not addressed,
no deployment is
evident
No systematic
approach,
information is
anecdotal. Early
stages of
deployment
A systematic
approach is
evident that
responds to
most of the basic
Category
requirements.
The approach is
deployed in
most areas
A systematic
approach is
evident that
responds to all
requirements of
the Category.
The approach is
fully deployed
with very few
gaps
A mature,
effective,
systematic
approach that
responds to all
of the
requirements of
the Category.
The approach is
fully deployed
with no gaps
A mature ,
effective,
systematic
approach is
evident that is of
national role
model quality.
The approach is
fully deployed
and has no gaps
or weaknesses
in any areas
X
Score for Category 1 - 45% x 150 = 68 points
ONE Award
Preparing Results Worksheets
Item 4.1 – What are your human resource results?
Key
Measurement
Areas
Key Results
Provided/Not
Provided
Performance
Levels
Performance
Trends
Comparative
Performance
Indicators of Employee
and Volunteers WellBeing, Satisfaction and
Dissatisfaction
+ Employee sat.
+ Employee
engagement index
- Results not provided
for volunteer
satisfaction
+/- Employee sat. at 4.5
out of 5
+ 80% of employees
engaged in mission –
1st year measured
-Sat. level of 4.5 is
constant for past 4 years
- Employee sat. based
on survey response rate
of 15%
+ Ranks in upper 25% of
similar organizations for
employee satisfaction
- Response rate for similar
organizations is 65%
Indicators of
Employee/Volunteer
Learning and
Development
+ Training hours
+ Conferences
attended
+ Certifications
achieved
+ 800 hours in 2006
+ 6 people attended 8
conferences
+ 2 employees attained
certifications
-Training hours per
employee/key
volunteers declined
each of past 3 years
- Lowest level in 5 years
- No comparative data
provided for any measure
ONE Award
Scoring Results Worksheets
0-5%
Performance
Levels
No performance
results or poor
performance
10-25%
30-45%
A few results are
reported with some
good performance
in a few results
Good performance
levels are reported in
many results areas
50-65%
Good performance
levels are reported
for most results
areas
70-85%
90-100%
Good to excellent
performance results
are reported in most
results areas
Excellent, role model
performance results are
reported in all results
areas
X
Performance
Trends
No trend data are
provided or
trends are
adverse
Little trend data
are reported and
some trends are
adverse
Early stages of
developing trends
are evident
No adverse trends
exist, no poor
performance levels
exist for key
performance areas
Most strong
performance and
improvement in
performance levels
are sustained over
time
Excellent improvement
trends and consistently
excellent performance
are reported in most
results areas
Early stages of
obtaining
comparative
information are
evident
Some trends and/or
current performance
levels show good to
very good relative
performance
Many to most
reported trends and
current performance
levels show areas of
leadership and very
good relative
performance
Evidence of benchmark
leadership is
demonstrated in most
areas
50-65%
70-85%
90-100%
X
Performance
Comparisons
Comparative
information is not
provided
Little comparative
information is
provided
X
Overall Score
Item 4.1
0-5%
10-25%
30-45%
ONE Award
Scoring – Category 4
Scoring Band for People Results – Item 4.1
Overall
Score**
0-5%
10-25%
30-45%
50-65%
70-85%
90-100%
X
**(place an “X” beneath the most appropriate scoring band and choose a percentage score within the band
that ends with either 0 or 5)
People Results - Score for Item 4.1 - 50% x 250 = 125 points
ONE Award
Consensus Process
ONE Award
Consensus Process
• Work in teams
• Review worksheets
• Develop feedback ready comments based on
worksheets
• Score Categories 1-3 and Items 4.1 through 4.7
• One scorebook – team score
• Submit final feedback report – comments and scores
by June 23
ONE Award
Developing Comments
•
•
•
•
•
•
Strengths
Opportunities for Improvement (OFIs)
Criteria-based
Key factor related
Complete thoughts, complete sentences
Actionable, but not prescriptive
ONE Award
Writing Comments
•
•
•
•
1-3 sentences per comment
The applicant is known as “the applicant”
State in a factual manner
Make sure comments don’t contradict other
area comments
• Don’t comment on how the application is
presented or written
• Useful to applicant and selectors
ONE Award
Writing Comments
• Comment Format
–
–
–
–
Strengths first, followed by OFIs
Refer to item area – 1.1, 2.4. 3.2, 4.3, etc.
Don’t use “would,” “should” or “could”
Professional, polite, positive tone
ONE Award
Developing Comments
• Based on independent review worksheets
• Categories 1-3 – most important 4-6 comments per
Category
• Strengths and Opportunities for Improvement
• Possible Comment Areas
–
–
–
–
–
Approach or deployment strength
Potential role model practice (very important, extremely rare!)
Gap in approach and/or deployment relative to criteria requirements
No response to key factor in the Org. Profile
Example (storytelling) rather than system
ONE Award
Developing Comments
•
•
•
•
•
Based on worksheets – strengths and OFIs
4-6 comments for Item 4.1 (People)
6-8 comments on Items 4.2 to 4.6 (Principles)
4-6 comments for Item 4.7 (Process)
Possible Comment Areas
– Favorable/unfavorable performance level and/or trends
– Favorable/unfavorable performance in areas that are key to
the organization
– Absence of results, comparative data
– Favorable/unfavorable comparative performance
ONE Award
Sample Comments
• Strength Comment
Item 1.1 – The applicant’s senior leaders and Board set
and review organizational vision, mission and values as a
component of the annual Strategic Planning Process.
Many different communications methods, including the
organizational newsletters and monthly staff meetings,
are used on an ongoing basis to make sure that these are
well understood by all employees and volunteers.
ONE Award
Sample Comments
• Strength Comment
Item 2.4 – The applicant ensures fiscal accountability
throughout the organization through a series of periodic
internal audits conducted by key staff as well as an
annual independent financial audit. Audit results are
reviewed within 10 days of receipt basis by senior
leaders, and the Board reviews these results as part of
each quarterly meeting.
ONE Award
Sample Comments
• Strength Comment
Item 3.6 – The applicant reviews its core competencies –
Service Delivery and Service Design – during the Strategic
Planning process and relies on these competencies to
strengthen its 3 key service processes – needs assessment,
collaborative service design and care delivery. These
processes are improved annually utilizing input from
customers, employees, volunteers and other key stakeholders,
as well as data developed from customer feedback
instruments.
ONE Award
Sample Comments
• Strength Comment - Performance
Item 4.1 – The results of the applicant’s annual
workforce satisfaction survey indicate that overall
employee satisfaction (those giving a score of “5” on a
scale of 1 to 5) has increased from 41% in 2003 to 68%
in 2007. Satisfaction among volunteers, who play a key
role in the applicant’s service delivery process, has risen
from 75% to 82% (scores of “5”) in that same time
frame.
ONE Award
Sample Comments
• Strength Comment - Performance
Item 4.4 – The applicant has reduced dependency on
government funding through increased donations and
other forms of private support as a source of revenue,
addressing a key strategic challenge for the applicant.
Private support has risen from 15% of its annual revenue
in 2003 to almost 40% in 2007, while growing overall
revenue by 30% in that same time period.
ONE Award
Sample Comments
• Strength Comment - Performance
Item 4.7 – The applicant has reduced the time it takes to
complete the monthly closing of its books from 19 days
in 2003 to 9 days in 2007, enabling senior leaders access
to key decision data on a more timely basis.
ONE Award
Sample Comments
• OFI Comment
Item 1.3 – It is not clear what system or approach, if any,
the applicant uses to ensure that knowledge that has been
accumulated by long term employees and volunteers is
transferred to other employees during their tenure or prior
to the potential departure of such knowledgeable
workforce members.
ONE Award
Sample Comments
• OFI Comment
Item 2.3 – While the applicant expresses that its
customers are satisfied, and indicates that its has received
numerous letters from satisfied customers, it is not clear
what systematic approach the applicant uses to determine
the satisfaction of its customers with its services provided
on an ongoing basis or how the applicant gathers data on
customer experiences to develop actionable data on
which to base improvement efforts.
ONE Award
Sample Comments
• OFI Comment
Item 3.4 – While the applicant indicates that it conducts
monthly performance reviews, both by senior leaders and
with its Board, it is not clear how or if the results of those
reviews are shared with those individuals responsible for
making decisions in the affected performance areas.
ONE Award
Sample Comments
• OFI Comment
Item 4.1 – While the applicant indicates that its customer
satisfaction results have risen by 300% over the past three
years (2004 through 2006), lacking actual data on the
specific level of performance achieved, it is difficult to
determine whether the applicant is achieving its goal of
95% customer satisfaction.
ONE Award
Sample Comments
• OFI Comment
Item 4.6 – While the applicant indicates that it requires
that each of its workforce members conduct themselves
at the highest levels of ethical behavior, no data are
provided to indicate whether the applicant tracks this key
performance indicator or whether the applicant is
achieving this desired level of performance.
ONE Award
Sample Comments
• OFI Comment
Item 4.7 – The applicant indicates that a key factor in its
ability to compete effectively in its service is
minimization of delivery errors, yet its service delivery
results actually reflect an increase in percentage of errors
each of the past three years (2005 through 2007) from 6%
to 8%. Given such a trend, it is not clear how the
applicant will be able to ensure its sustainability in a
highly competitive market.
ONE Award
Comments Exercise
• Strategic planning is strong
• Applicant states that customers are consistently
satisfied
• The applicant’s service delivery process is
systematic
• Applicant’s key value creation process
successfully create value
ONE Award
Comments Exercise
• Item 3.1 -The applicant needs to talk more about
its human resource plans.
• All items - The application does not flow well
making it difficult to understand what they are
trying to tell me. The use of bullet points rather
than complete sentences would be an
improvement.
ONE Award
Comments Exercise
• Item 4.1 – 4.7 - Applicant indicates results are
strong in most areas
• Item 4.1 – 4.7 - Results are not very good for key
factors
• Item 4.1 – 4.7 - Comparisons are generally
satisfactory
• Item 4.2 -The applicant seems to do pretty well
here
ONE Award
Comments Exercise
• Item 3.1 - The applicant’s strategic planning
process is satisfactory, but if the applicant would
include more time spent on conducting its SWOT
analysis, then the planning process would be a
significant strength.
ONE Award
Comments Exercise
• Item 1.1 - The applicant’s leadership team does an
outstanding job, and has much experience with
meeting organizational challenges. The leadership
team really understands its market and its
customers so well that they are able to anticipate
customer needs and meet them.
ONE Award
Comments Exercise
• Item 3.6 - The applicant’s approach to process
management is clearly best practice, and other
organizations would benefit by adopting a similar
approach.
ONE Award
Comments Exercise
• Item 3.4 - In order to improve its performance
review system, the applicant would benefit from
adopting a weighted balanced scorecard approach.
This would not only make it easier for the
applicant to fully understand organizational
performance but it would also helps auditors and
examiners to better understand their successful
practices.
ONE Award
Comments Exercise
• Item 1.1 - Leaders in role model organizations
must do an excellent job of walking their talk. The
applicant’s leaders seem to do a good job of doing
so.
ONE Award
Comments Exercise
• Item 3.4 - Comparing data and understanding
performance is critical to improving an
organization. If the applicant would designate one
individual to spend full time developing a
database of reliable and useful comparative data,
and build an IT system capable of delivering it to
those who need it, they could be a role model
organization.
ONE Award
Comments Exercise
• Item 3.6 - The applicant works to improve its
processes, but if it adopted a Six Sigma approach,
it would really accelerate its improvement. Such
improvement would really make funders more
willing to support the organization, leading to a
higher level of likelihood that the organization
would be able to successfully sustain itself.
ONE Award
Comments Exercise
• Item 2.4 - The applicant does not tell me enough
about the details of their budgeting process or the
internal controls that they have in their accounting
system for me to say that it is an effective system.
ONE Award
Comments Exercise
• Item 4.1 – 4.7 - Based on the consistent high
levels in many of the applicant’s results, it is clear
the applicant has strong leadership, great customer
relationships, a strong workforce, and must use a
high performing approach to improving its
processes (possibly PDSA or another good
system). This applicant does great things and is a
true role model.
ONE Award
Comments Exercise
• Item 1.2 - If I were in charge of the applicant
organization, I would strengthen the human
resources management system and increase the
investment in training. Otherwise, the applicant
will fall behind other organizations in efforts to
get maximum performance from their employees
and volunteers.
ONE Award
Consensus Process
•
•
•
•
Team Process
One Scorebook
Consensus Comments Only
Process Tips
– Combine and Synergize
– Link the Category and Associated Result Area
– Develop Team Score Based on Team Scorebook
ONE Award
Site Visit Process
•
•
•
•
•
•
•
Review Consensus Scorebook
Identify Key Site Visit Issues
Verify and Clarify on Site
Revise Scorebook based on Site Visit Findings
Revise Score based on Site Visit Findings
Customers – Applicant and Selectors
Training week of July 9