Transcript ZW - Sheffield Hallam University

Total quality management
based on self-assessment
The experience with
Andre Vyt
PARTNERS IN QUALITY
MANAGEMENT
in Flanders
PROZA exists in different versions
•
•
•
•
•
Higher education
Secondary education
Vocational training for handicapped persons
Adult education & training centers
Health Care & Social care centers
The goal of
is
• to diagnose strengths and weaknesses of all
relevant aspects in the organization
• inspired by an all-encompassing and recognized
framework (EFQM Excellence model)
• by the process-owners (lecturers, researchers,
administrators, technicians ...) in quality teams
• together with all parties involved: supervisors,
colleagues, and clients (students, qualified
professionals…)
• to set up well-supported specific actions for
improvement
Inspired by EFQM Excellence
Basic
conditions
Core
processes
Results
The 9 areas of
attention
Policy and
strategy
Resources and
co-operation
Leadership
Core
processes
Human
resources
policy
Personnel
satisfaction
Student
satisfaction
Appreciation by
society
Performance
of the
organization
The content of PROZA
• It contains more than 60 focal points over
the 9 areas of attention
• In a focal point 25 specific items are to be
checked/answered
• The 25 items are grouped into 5 phases,
according to the level of quality
management
Areas and points of attention
Focal points in core processes
•
•
•
•
•
•
•
Admissions coaching
Communication with students
Curriculum design
Design of course units
Implementation of the curriculum
Teaching and tutoring
Student coaching
Focal points in core processes
•
•
•
•
•
•
•
•
Practice-based learning
Internationalisation
Testing and assessment
Graduation and alumni-oriented activities
Postgraduate education
Public service provision
Research
Development and practice of arts
The 5 phases of development
1. Quality is variable
Quality is linked to the individual
2. Initiation of process-based thinking/acting
Starting to work systematically
3. (High level of) Professionalism
Quality is guaranteed
4. Systematic innovation
Continuous improvement
5. Aiming for excellence
Recognition of expertise and high level of quality
The PDCA-cycle of QM
(Deming)
quality
do
plan
check
evaluate - control
act
assurance
improvement
time
The higher the level attained,
the better the
• Formulation and management of processes
Measurements and information gathering
•
Systematisation and reliability
Openness and transparency
•
Attunement and integration
Innovation and improvement
•
Involvement of different persons
Client and external orientation
1: Quality linked to the
individual
•
•
•
•
•
•
Quality is the result of individual effort
Short-term thinking
Problems are resolved when they arise
Limited exchange of information
Lack of follow-up commitments
A minimum of quality is present
2: Initiating process-based
thinking/acting
• Formulation of needs and priorities
• Overall commitments enforceable to a
limited extent
• Increasing attention to information
management
• Real causes of most problems are known
• Performance is evaluated regularly
• Improvement is clearly noticeable
3: (High level of) Professionalism
•
•
•
•
•
Planned approach
Availability of reliable data
Binding by written commitments
Continuity of service is guaranteed
Level of quality achieved is supported
by assessment data
• Information flow is effective
4: Systematic innovation
• Systematic analysis and assessment of
operations
• Use of quality indicators as a policy
instrument
• Structural problem detection & handling
• Innovative thinking
• Emergence of synergy effects
5: (Aiming for) excellence
• Frames of reference, indicators and
working methods internationally
recognized as best practice
• Broad base of support for moving
beyond current levels of quality
• All quality indicators are moving in a
positive direction
• Large number of personnel are
internationally recognized experts
E.g., the focal point “course
units” has items such as:
1. Do lecturers know the function of their course
unit in the curriculum?
2. Do colleagues work together in a team to design
course components?
3. Is the design of nearly all course units geared to
the frame of reference of the department?
4. Are goals of improvement defined on a yearly
basis for all course units?
5. Is the design of specialized course units
entrusted to experts?
Testing and assessment
1. Do all students receive a copy of the exam
regulations in time?
2. Is there an attempt to safeguard objective
assessment by involving more assessors?
3. Are assessment methods for each course
component clearly geared to the objectives?
4. Are assessment methods systematically adjusted
following evaluation?
5. Is the quality of the assessment methods
regularly tested by external experts?
Student admission coaching
Answering the questions
• Answering categories: yes / no / not
applicable / impossible to answer
• Lack of measurement data provides
relevant information on the need to
professionalize the management of
information
• Concrete questions provide immediate
goals of improvement
The method used in PROZA
• The checklists are filled out individually by
the team members
• They gather to agree in consensus on the
scores of the items and deduce the level of
quality management
• but also to identify immediately specific goals
of improvement
• that are selected according priority and
written down in a self-evaluation report
Score (in consensus)
• Actual level: phase within which 3 items
are positive
• Potential or growth level: phase within
which 1 item is positive
• … each time on condition that in
previous phases at least 3 items are
positive
Chart of scores within an area
Chart of scores over the 9 areas
The PROZA
self-assessment process
• Preparation
• Diagnosis
• Improvement
Preparation
• Policy decision to use PROZA and formation
of a steering group
• Getting people on board by engagements
and good communication
• Initial screening with quickscan to
determine which areas (and foci) to address
• Formation of self-assessment groups
• Agreement on working method and prior
clarification of terminology and gradation
words
Screening with the quickscan
• 200 statements that probe the overall
level of quality (20 per area, 40 for core
processes)
• To be answered with
– Adequate (A)
– Must be improved / not adequate (NA)
– Important (I)
– Not so important (NSI)
• Preferably by the steering group or key
persons
The Quickscan
The Quickscan
Carrying out the diagnosis
• Relevant information on the foci can be
made/kept available
• Gaining a view on hot issues by an inventory of
individual scores before the meeting
• Value of individual comments on the questions
• Short-circuiting endless discussions:
– Right/wrong is less essential than identifying
improvement
– Eventual agreement on skipping items on which only
one member answered negatively
From diagnosis to
improvement
• Diagnosis and proposals for improvement
are formulated on a report form
• Assignment of a priority value
• Definitive selection of ultimate priorities by
steering group or management
• For each selected goal, an action plan
must be drawn up on a predefined form
by an improvement team
Work sheets in PROZA
Action plan
•
•
•
•
•
•
•
Concrete goal
Responsible person
Actions that have to be taken
Persons involved
Time plan
Expected results / deliveries
Budget
Client-orientation of PROZA
• Evaluation criteria and items were tested on
content and applicability in different HE
institutions.
• Assessment questions have grown out of
practical experience and are based on a
consensus between TQ managers
• The checklists are accompanied by a users
guide and a list of concepts
• The checklists are delivered electronically, so
that institutes can adjust words to match the
structure of the organisation
Some pro’s of PROZA
• Exhaustive and flexible: separate areas
and focal points can be analysed for
different units in the organisation
• It is easy to deduce questionnaires
• More than 1000 items result in very
specific goals of quality improvement
• Based on individual analysis and on
consultation with others in teams
Links between self-evaluations
and external quality assurance
Self-evaluation
Themes for audits
Quality management policy
Human resources
Means and finances
Learning goals
Programme content and organization
Outside school practice training
International policy and activities
Training methods and tools
Assessment and examination
Counseling and guidance
Study load and effectiveness
Research, arts and societal services
Graduates
Implementation requirements
• Improvement as the final goal: diagnosis
and scoring is only the starting point.
• Explicit policy and engagement on all levels
of the management.
• Agreement on aspects of confidentiality and
openness of reports
• Explicit rules about implementation of
selected goals, follow-up and feedback
Avoiding pitfalls
• If not properly introduced and coached, it may be
perceived and labeled as a mass of paperwork
which demands more energy than that it should
help to improve quality.
• If not complemented by a leading and steering
management, self-evaluation may bog down in
the diagnostic process.
• If the instrument is put forward as to be used
primarily on teaching processes , it may result in
adverse effects of “shooting the piano-player”
instead of improving the piano.
Guaranteeing the achievement
of goals
• Diagnoses by self-evaluation teams can exist
alongside the management instead of in the heart
of the management, leading to quality
management on paper instead of effective
improvement in reality
• Avoid misuse by management to “solve”
management problems, or to evaluate employees.
• Diagnoses must not only result in clear objectives
of improvement but must also be made concrete
in realistic project planning according to SMART
rules.
Criteria for project goals
• Relevant
• Understandable
• Measurable
• Behaviorable
• Attainable
• Specific
• Measurable
• Agreed/acceptable
• Relevant/realistic
• Time-related
For further information…
www.prose.be
www.prose.be
[email protected]