Transcript International Deans’ Course Latin America

DoQup
Training Seminar
Alicante, 24-26 April 2013
Organisation for the Quality Assurance
of Institutions and Study Programmes
Guillermo Bernabeu, University of Alicante
1
The context
Spain: QA for HE
A shared view on QA
The
Context
From Berlin Communique
We adopt the standards and guidelines for quality assurance in
the European Higher Education Area as proposed by ENQA.
We commit ourselves to introducing the proposed model for peer
review of quality assurance agencies on a national basis, while
respecting the commonly accepted guidelines and criteria.
We underline the importance of cooperation between nationally
recognized agencies with a view to enhancing the mutual recognition
of accreditation or quality assurance decisions.
Spain:
QA for HE
THE SPANISH AGENDA
The Spanish Context: Timing
Initial steps in 2003 with a position paper from the Ministry of Education, Science and Sports
(six different ministers ...).
ECTS credits and Diploma Supplement introduced very early (2003).
QA for Spanish HE (or ANECA) defined in 2002.
A modification of the Spanish Universities Law introduced in 2007.
(Too) long discussion about the BMD structure: Catalogue versus Register, 3 or 4 years for
Bachelors, ... .
Different Royal Decrees regulating the BMD structure … were … published in 2005, 2007, 2010
and 2011.
Degree Structure and Study Programmes “kick-off” ended in 2010.
THE SPANISH AGENDA
The Spanish Context: Space
Public universities funding in Spain is regional based while HE
regulation is a competence of the national authorities.
University autonomy is in the Spanish Constitution.
78 universities (50+28), 17 autonomous regions, 1.500.000
students at Spanish Universities, ...
ANECA, 4 QA regional agencies in ENQA&EQAR, 6 QA regional
agencies, … .
Higher Education Spanish System
EVALUATION EX-ANTE: VERIFICATION
FOLLOW-UP PROCEDURE
Public information
Outcomes
indicators
ACCREDITATION (after 4 or 6 years)
Fulfillment
Quality
Assurance
RUCT
SIIU
A
ACREDITA
Accreditation System in Spain
STAGE 1
Ex-ante accreditation
Verificación
To verify that the design of the study
programme includes the criteria established
by the Ministry whose interpretation and
application is made by ANECA.
A public report is delivered by ANECA.
Higher Education Spanish System
Accreditation Process in Spain
1. DESCRIPTION OF THE DEGREE
EX-ANTE
2. JUSTIFICATION
ACCRED.
BM
3. LEARNING OUTCOMES
4. STUDENT ENTRY AND ADMISSION
5. PROGRAMME PLANNING
6. ACADEMIC STAFF
7. RESOURCES AND SERVICES
8. ANTICIPATED OUTCOMES
RD 1393/8 (Modif.
861/2010) sets out how
the report shlod be
requested for the
verification of Official
Degrees
9. INTERNAL QUALITY ASSURANCE SYSTEM
10. IMPLEMENTATION AGENDA
Evaluation Criteria
1. DESCRIPTION OF THE DEGREE
Each degree design must include a description that is adequate and consistent
with the level and/or academic validity so there is no confusion concerning its
content or, where applicable, qualification for professional purposes.
2. JUSTIFICATION
Each degree design must be relevant in terms of the corresponding learning
and/or research experience, be appropriate to the academic field that it refers to
and/or comply with other similar existing studies.
• In the case of the Master ’ s, external equivalents with studies in other
countries may be provided.
• In the case of the Master’s, justification must be provided for the research,
academic or professional orientation of the programme.
• In the case of a Master’s with a professional or research orientation, the
design must refer to the situation of R+D and innovation in the professional
sector.
Evaluation Criteria
3. COMPETENCES
The competences to be acquired by students must be consistent with those
required in order for the degree to be awarded and the qualifications stablished
in the EHEA. Consequently, these competences must be assessable and
consistent with the corresponding level of the Bachelor or Master’s degree.
4. STUDENT ENTRY AND ADMISSION.
The degree programme must have accessible systems that regulate and provide
clear information on the different means of entry, admission and new student
orientation.
• In the case of Master’s degrees, in addition to the envisaged entrance exams,
state the body that is responsible for admissions, together with the applicable
procedures and requirements.
Evaluation Criteria
5. PROGRAMME PLANNING.
The programme of studies must constitute a proposal for study that has been
designed in a coordinated way and takes into consideration the student ’ s
dedication within a given period of time and the competences to be acquired in
the degree proposal.
In the case of a Master’s degree with a professional orientation, the approach
given to adequate professional practice will be of special importance, as well
as collaboration agreements with private enterprise and other institutions for
this practice.
6. ACADEMIC STAFF.
Teaching staff and other support human resources in each degree must be
adequate in order for the general aims and competences envisaged in the
degree design to be achieved.
7. MATERIAL RESOURCES AND SERVICES.
Material resources and services that are necessary for the development of the
envisaged activities must be adequate for the envisaged aims and competences
in the programme of studies to be achieved.
Evaluation Criteria
8. EXPECTED RESULTS
The degree design must include the anticipated outcomes in connection with
the degree's efficiency and the general mechanisms for assessing the learning
outcomes of the students.
Evaluation Criteria
GRADUATION RATE:
the percentage of students who complete the course in the time envisaged in the
programme of study (d) or take one year longer (d+1) in relation to their entrant cohort.
This gives a measure of overall academic performance.
DROP OUT RATE:
the percentage relation between the total number of students in a new entry cohort
who should have obtained the degree the year before who neither registered this year
nor the year before. In the case of one year Master’s programmes, the following
definition shall be applied: percentage relationship between the total number of
students in a new entry cohort that should have obtained their degree the previous
academic year and that did not enrol in either that academic year or the subsequent
one.
EFFICIENCY RATE:
the percentage relation between the total number of credits in the programme of study
and the total number of credits in which the group of graduated students in a particular
academic year had to enrol in throughout their studies.
Evaluation Criteria
9. QUALITY ASSURANCE SYSTEM
The degree design must include an internal quality assurance system that
ensures the control, review and continuous enhancement of the programme.
Specify the body or unit in charge of the quality assurance system for the
programme of study (structure and composition), together with the internal rules of
procedure. Details must be provided in this section on how participation in this body by
teaching staff, students, academic managers, support staff and external stakeholders is
organized.
Establish how the development of the programme of study is to be reviewed (aims,
competences, planning, etc.) through the application of adequate mechanisms and
procedures applied periodically for gathering and analysing information on:
•The quality of the programme and the teaching staff.
•The quality of placement/work experience and mobility programmes.
•Graduate employment and degree satisfaction.
•The satisfaction of the different groups involved (students, academic staff and
administration and services staff, etc.) and the attention paid to suggestions and
complaints.
AUDIT
Evaluation Criteria
QUALITY ASSURANCE SYSTEM
Have adequate and systematic mechanisms and procedures for decision- making that
ensure the enhancement of the programme of study. These should include:
The quality of the programme and the teaching staff.
The quality of placement/work experience and mobility programmes.
These mechanisms and procedures must at least provide for who is in charge, the way
that decisions are made and follow-up of decisions that are made.
Define the criteria that establish the limits whereby the degree is finally suspended. A
procedure needs to be established beforehand that describes the system set up to
define these criteria in the institution/university, together with its review, approval and
periodic updating.
Establish mechanisms to publish the information on the programme of study, its
development and outcomes, and for this to be sent to all those involved and/or
interested (students, teaching staff, support staff, prospective students, external
stakeholders, etc.).
Evaluation Criteria
10. SCHEDULE FOR IMPLEMENTING THE DEGREE
The process of introducing new degree programmes must be planned
in terms of a schedule, and there will need to be a mechanism, where
applicable, to help students from existing courses to adapt to the new
programme of study.
Evaluation Procedures
Evaluation ex-ante (verification)
•Evaluations
Commissions for Bachelor/Master/Doctorate and for
the five tematic areas: Arts and Humanities, Sciences, Health, Social
Sciences and Law and Engineering and Architecture.
•Global Commissions for Bachelor/Master/Doctorate
•Cooperation with regional agencies (those in ENQA)
Evaluation Procedures
Evaluation ex-ante (verification)
•Evaluations
Commissions for Bachelor/Master/Doctorate and for
the five tematic areas: Arts and Humanities, Sciences, Health, Social
Sciences and Law and Engineering and Architecture.
•Global Commissions for Bachelor/Master/Doctorate
•Cooperation with regional agencies (those in ENQA)
Accreditation System in Spain
STAGE 2
Follow-up procedure
Seguimiento
To check, by non-intrusive means, that
the institution is implementing the study
programme according to the approved
design.
Legal Framework
RD 1393/2007 (modif. RD 861/2010), art. 27.2 dice:
"Once initiated the implementation of programmes related to the official
degrees of the University Register, and RUCT degrees, ANECA or
assessment bodies that the autonomous communities determine, the
project monitoring will be carried out by the University Council. "
"To this end, the ANECA and corresponding assessment bodies, in
collaboration with the Ministry of Education and the corresponding
Autonomous Communities jointly develop a protocol that will include
defining minimum common core criteria and indicators for monitoring
procedure curricula. "
25
Legal Framework
 AVAP which implements the university quality assessment in the
Valencian Community, is in charge of performing the functions
described in Article 27.2 of Royal Decree 861/2010 cited above, to
develop a protocol that will include defining a minimum common core
criteria and indicators for the monitoring procedure curriculum.
 Resolution of December 15th 2010, the president of the AVAP, by
adopting the Track Protocol for official degrees in the Valencian
Community.
 Completed by Document Development Monitoring Protocol
26
Sistema de Garantía Interna de
Calidad
A general view
Verification
Curriculum
Development
Tracking
Implantation
AVAP
DEGREES
PROTOCOL
Acreditation
Quality
Assurance
System
continuous improvement
27
- Understand the importance of tracking the official titles of
the University
- Analyze the aspects involved in monitoring
- Understand the importance of having unified all the
information involved in the Monitoring
28
 2008, one center participated in the AUDIT program,
ANECA (Faculty of Philosophy and Letters)
 ANECA Certificate 16/09/2009
 Implanted in the Centre during the year 2009/2010
 2010, participe with the rest of the UA Centres


ANECA Certificates 14/01/2011
Implanted in the Centre during the year 2011/2012
 Prepare reports to be sent to verify all degrees and
masters of the UA
 Including in the corresponding paragraph 9 of the AUDIT
program procedures
29
 AVAP presents its protocol for monitoring the qualifications
(year 2010/11)
 Experimentally participated with two masters of the Faculty of
Philosophy and Letters
 2011/2012 we participated with 5 and 2 Masters Degrees
and the internal track of 2 Masters that participated the
previous year
30
Set the framework for quality of the centers, whose
composition and functions are set out in Chapter 3 of SGIC:
1.Center Management Team
2.Faculty or School Board
3.Quality Commission
4.Quality Coordinator
5.Improvement Groups
6.Qualification Commission (Document support for application
verification degrees (Masters) in UA)
31
AUDIT Review
What is AUDIT?
A support tool for universities main mission is that universities
incorporate continuous improvement strategies, develop
and control their actions, revised and redefined to achieve
some purposes consistent with the requirements of quality
that marks the European Higher Education Area
AIM: to support and enhance development and
implementation of Internal Quality Assurance Systems in
all faculties and universities in Spain.
32
AUDIT
33
AUDIT Review
OBJETIVES:
 Facilitate the development and implementation of
quality systems
 Implement a procedure leading to the recognition of
such systems
 Contribute to the improvement of the quality of
academic programs
 Give a step towards the construction of a European
Quality Space
34
AUDIT Review
tells us what to do
tells us how to do it
Quality
Manual
Procedures
Manual
Other documents
(DOCENTIA, POI, PEUA, etc.)
Records
35
AUDIT Review
CONTENTS OF A QUALITY MANUAL
C1. SGIC
C2. Presentation of the center
C6. Learning orientation
C7. Academic and support staff
C3. Structure of the center for
development SGIC
C4. Quality policy and objectives
C5. Quality assurance of training
programs
C8. Resources and services
C9. Training results
C10. public information
STRATEGIC PROCESSES: PE
PE01. Establishment, review and updating of policy and quality objectives
PE02. Policy faculty and staff of the UA
PE03. Design of the training offer
36
What is AUDIT?
KEY PROCESSES : PC
PC01. Degree training offer
PC03. Own degree training offer
PC05. Review and improvement of qualifications
PC07. Support and guidance of students
PC09. Student mobility
PC11. Occupational guidance
PC13. Public information
PC02. Master degree training offer
PC04. PhD degree training offer
PC06. Defining income students profiles
PC08. Development and evaluation of learning
PC10. External management practices
PC12. Analysis of academic results
SUPPORT PROCESSES : PA
PA01. Control and management of
documentation and records
PA03. Satisfaction of stakeholders
PA02. Degree suspension
PA05. Management of academics and PAS
PA07. Service Management
PA06. Management of material resources
PA08. Admission, registration and records
management
PA04. Treatment of complaints and suggestions
MEASUREMENT PROCESSES : PM
PM01. Review, analysis and continuous improvement of the SGIC
37
AVAP Monitoring
• December 2010, publication of the
resolution establishing what will follow in
the universities of the Valencian
Community
38
AVAP Monitoring
OBJETIVES:
 Ensure effective implementation of the teaching following the
curriculum title.
 Ensure public availability of information to different stakeholders.
 Identify any weaknesses in the development of lessons and analyze
the actions taken for correction.
 Provide recommendations for improvement in the implementation of
the curriculum.
 Demonstrate progress in the development of the title for the Internal
Quality Assurance System (SIGC) as regards both the review of the
implementation of the curriculum and the proposed actions to improve
its design and implementation.
 Identify best practices for dissemination in the university system
39
AVAP Monitoring
STRUCTURE OF PROCEDURE:
1.Universities elaborate an annual report tracking each new title implanted.
2.AVAP constitutes evaluation committees (integrated by renowned academics,
EHEA experts and university students) to analyze the reports and other public
information.
3.AVAP issues individual reports (biennial) to be sent to the universities.
4.Universities, if necessary, claim for interim reports.
5.AVAP issues final reports tracking each title to be sent to the universities,
Ministry and University Council.
6.AVAP will publish a global report of the total qualifications tracking and
universities of the Valencian Community
40
AVAP Monitoring
CRITERIA FOR MONITORING:
1.Information for the society and future students
2.Information on the development and operation of title
3.Upgrades / modifications of the curriculum
4.Recommendations made in the external evaluation reports
5.Assessing the implementation of the Internal Quality Assurance System
(SIGC)
6.Evaluation of indicators
41
AVAP Monitoring
Criterion 1
Criterion 2
F03-PC05
F04-PC05
Criterion 3
Criterion 4
Criterion 5
Criterion 6
Information for the society and
future students Aspects to evaluate
a) Description of the title, general and specific skills e.
b) Training schedule.
c) If present, specify the profession of the degree
(only in the case of a degree linked to a regulated
profession)
d) Regulations for permanency.
e) Rules of recognition and credit transfer.
f) Admission Profile (access and admission
requirements for the degree, special entrance
tests)
g) Pre-registration information (documents to be
submitted, deadlines, etc.)
h) ANECA verification report and AVAP report and
recommendations.
AVAP Monitoring
Criterion 1
Criterion 2
F03-PC05
F04-PC05
Criterion 3
Criterion 4
Criterion 5
Criterion 6
Information on the development and
operation of title Aspects to evaluate
a) Availability and adequacy of the guidelines
b) Scheduling of teaching including the structure of
the curriculum, schedules, classrooms.
c) List of staff teaching subjects / courses indicating
the teaching category.
d) Form in which takes place the end of the old plan
and the implementation of adaptive courses.
e) External information practices and the end of work
studies
f) Student mobility (mobility organization by title,
centers, exchange programs)
g) Commitments to obtain title competence
(qualification procedure in order to assess the
progress and results of student learning).
AVAP Monitoring
Criterion 1
Criterion 2
F03-PC05
F04-PC05
Criterion 3
Criterion 4
Criterion 5
Criterion 6
Upgrades / modifications of the
curriculum
Aspects to evaluate
a) The amendments are based on objective and
previously
analyzed
information
by
the
management bodies of the title.
b) The substantial changes that have occurred have
been reported to the University Council for
assessment.
c) The information that changes as a result of the
amendments to the title has been implemented in
the corresponding website.
AVAP Monitoring
Criterion 1
Criterion 2
F03-PC05
F04-PC05
Criterion 3
Criterion 4
Criterion 5
Criterion 6
Recommendations made in the
external evaluation reports
Aspects to evaluate
a) Actions that have been developed with the
recommendations proposed in the Verification
Report and where issued, if the AVAP.
b) Actions that follow the recommendations of the
monitoring reports of AVAP.
AVAP Monitoring
Criterion 1
Criterion 2
F03-PC05
F04-PC05
Criterion 3
Criterion 4
Criterion 5
Criterion 6
Internal Rating System of Quality
Assurance
Aspects to evaluate
a) SIGC Development, in relation to the qualification
monitored and report submitted to the program
b) Development of the mechanisms used to:
c) improving the quality of teaching and teachers.
d) internships and mobility programs.
e) measuring the labor market and the satisfaction of
graduates.
f) measure the satisfaction of the various
stakeholders involved in the title
g) attention to suggestions and complaints.
h) Compliance with the criteria for termination of title.
i) Improvement actions developed from the analysis
of the results.
j) Making decisions on curriculum development
based on SGIC.
AVAP Monitoring
Criterion 1
Criterion 2
F03-PC05
F04-PC05
Criterion 3
Criterion 4
Criterion 5
Criterion 6
Indicator evaluation
Aspects to evaluate
a)
b)
c)
d)
e)
f)
g)
h)
Rate of Return of Title (CURSA)
Title Dropout rate (CURSA)
Efficiency rate of graduates (CURSA)
Graduation rate title (CURSA)
Enrollment ratio (AVAP)
Rate of supply and demand (AVAP)
PDI rate with a doctoral degree (AVAP)
Full time PDI Rate (AVAP)
AVAP Monitoring
Degree verification
memory
Criterion 1: Information for the
society and future students
Claims /
recommendations
AVAP
Criterion 2: Information on the
development and operation of
title
F03-PC05
F04-PC05
Performance reports
Satisfaction reports
Criterion 3: Upgrades/
modifications of the curriculum
Criterion 4: Recommendations
made in the external evaluation
reports
Criterion 5: Internal Rating
System of Quality Assurance
SGIC
F01-PM01
F02-PM01
F01-PC05
F02-PC05
Service satisfaction
reports
Interest groups
satisfaction reports
Indicators evaluation
Recommendations
Criterion 6:Indicator evaluation
External evaluations
reports
Monitoring
PC05 modification
 Modify the PC05. Monitoring and improving qualifications
OBJETIVE
Establish the process by which the Centers of the UA held the
annual monitoring of official titles, according to memory
and verified with respect to the monitoring requirements for
proper accreditation according to RD 1393/2007, RD
861/2010 and Resolution of 15 December 2010 of the
AVAP, to prepare the annual monitoring report for each of
their titles.
49
Monitoring Workflow
 Set different tasks involved in tracking and qualifications in
the review and improvement of the system of internal
quality assurance
 Qualification Coordinator
 Quality Center Coordinator
 UTC
 Vice President with responsibility for Quality
 Governing Council
 AVAP (established by monitoring protocol)
50
Monitoring. AstUA
 It is a computer application for sustainable monitoring:












Verified memory
AVAP / ANECA arguments or recommendations
Changes to verified memory
AVAP tracking reports
Monitoring reports of the site visit
Performance reports to qualifications
Employability reports
Reports of services satisfaction
Reports of stakeholders satisfaction
Report complaints and suggestions
AVAP indicators
System Monitoring of Title Quality Assurance
 Proceedings of the Quality Assurance Committee of the Centre
51
Monitoring. AstUA
Document
Manager
SGIC
Manual
Indicator
Manager
Expected
Values
procedures
manual
Record
Formats
Other
Historical
series
AVAP
/verifies
Other
Monitoring
Reports
Management
SGIC
Accreditation
Management
Accreditation
Report?
AVAP
Report
Guided
All information
accessible and
controlled
52
Thank you for your attention
53