Transcript xx - Council of Europe Office in Belgrade

Experience of self evaluation as part of a
long-term change process : the UA as a
case study from Portugal
Conference on self-evaluation of HE institutions, Belgrade, Serbia
J.A. Rafael and M. Assunção , 10-11 July 2008
Contents
• About the University of Aveiro
• profile
• organisation
• QA internal implementation
• Setting the scene at European and National level
• Legal changes
• Opportunity to re-think QA in UA
Portuguese Public Universities
The University of Aveiro as a
Regional Network for Education and Training
The U. Aveiro incorporates University Departments and
Polytechnic Schools (since 1997). Four years ago a
programme aimed at post-secondary education was started
and is now run by the Polytechnic Schools.
17 University departments (Aveiro)
35 graduation degrees
Ol. DE AZEMÉIS
AVEIRO
4 Polytechnic Schools (3 cities)
15 graduation degrees
16 CETs (11 cities)
AGUEDA
University Departments
engineering
› Electronics and Telecommunications
› Environment and Planning
› Ceramics and Glass Engineering
› Mechanical Engineering
› Civil Engineering
education
› Sciences of Education
› Didactics and Educational
Technology
health Sciences
science
› Chemistry
› Biology
› Physics
› Mathematics
› Earth Sciences
arts and social sciences
› Languages and Cultures
› Media and Artistic Studies
› Social and Political Sciences
› Economics, Management and
Industrial Engineering
Registered Students
source: GAGI, December 2007
Research Centres
13 Research Units
4 Associated Labs
source: Research Institute, May 2007
Functional Organisation & Management
Rector
Scientific
Council
Research
Institute
Senate
Pedagogic
Council
Undergraduate University
Education Institute
Departments (17)
Postgraduate University
Education Institute
Polytechnic
Education Institute
Polytechnic
School 1
…
Polytechnic
School 4
Study Programme Matrix Organisation
Study Programme 1 Study Programme 2
[Degree Director]
Dept. 1
●
Dept. 2
●
[Degree Director]
●
…
…
Dept. m
Resources
●
●
……
Study Programme n
[Degree Director]
●
Research Units Matrix Organisation
Research Unit1
[Unit Coordinator]
Research Unit 2
[Unit Coordinator]
●
●
●
●
…
…
Dept. m
Resources
Research Unit n
[Unit Coordinator]
●
Dept. 1
Dept. 2
……
●
Promotion of a Quality Culture: the beginning (1)
•
1992, Portuguese Rectors Conference hosts a seminar on Quality
Systems (GB, F, NL) for HE
•
1995 CRE institutional quality audit
•
1997, Vice-Rector for Quality
•
1999 Follow-up of CRE audit
•
Initiatives to create awareness of the quality issue
– Atenção UA!
– Hosting a CRE/IMHE seminar - UA was presented as a case study
– Repensar os Curricula (“Bologna before Bologna”)
Promotion of a Quality Culture: the beginning (2)
•
Decision to develop a single (integrated) information system
•
Creation of specific bodies responsible for quality development
– GAGI (Information Management Office)
– GAQAP (Quality, Evaluation and Procedures Office)
•
Redefinition of the university structure to include institutes for
coordination of function
Internal Processes (1)
• Transversal instruments
– LEGUA (Legislation, Regulation and Procedures
Knowledge Base)
• Quality Protocol
– Single Information System
• Supporting and monitoring all the university activities in an
integrated way
Single Information System
Internal Processes (2)
•
Teaching activities
– Managed through the Education Institutes
• Centrally
• Locally through the figure of the Study Programme Director
– Transparency of the global information system enabling that teaching
and learning activities have their results available to all relevant actors
– Indicators referring access numbers, drop-out rates, time to graduation
and student performance are produced on a semester basis;
employment data and feedback from employers - initially on a five year
cycle, now on an annual basis
– Since 1994, on a five-year cycle basis, the UA study programmes have
been evaluated in accordance with a national framework
Study Programme Evaluation (at national level)
• Promoted by FUP (Portuguese University Foundation) and,
afterwards, CNAVES
• Comprises
– Self-evaluation (SE)
• SE Committee appointed by the Head of the Department
responsible for the Study Programme
• Based on standard evaluation guidelines (include questionnaires to
teachers and students)
– External evaluation
• Evaluation team appointed by FUP, constituted by experts of other
Portuguese universities
• Site visit
– To analyse and discuss the SE report with SE Committee,
Teachers and Students
• Evaluation report with recommendations
– Recommendations of the evaluation report were discussed and totally
or partially implemented
Internal Processes (3)
• Research activities
– Research Institute coordinates the activities of research units and
encourages quality practices through support for high quality research
– At Research Unit/Associated Labs level there exist internal mechanisms
which regulate production of research, promoting quality through
internal funding rules
– Research activities are subject to an external quality processes in the
sense that they are only funded if they meet the requirements of the
funding institution (e.g. FCT, EU, …)
– The Research Units are also subject of an external evaluation
programme by FCT, carried out by international panels, on a three-year
basis
– Academic career itself foments quality research by valuing some types
of publications over others and making promotion more dependent on
research rather than teaching or other activities
Research Evaluation (at national level)
• Promoted by FCT (Science and Technology Foundation)
• Comprises
– Activity report
• Written by the Coordinator of the Research Unit
• Based on standard evaluation form issued by FCT
– External evaluation
• Evaluation team designated by FCT, constituted by
international experts
• Site visit
– To analyse and discuss the Activity Report with the members of
the Research Unit
• Evaluation report and a classification of the Research Unit
(from Poor to Excellent) with direct impact on Unit funding
Internal Processes (4)
• Administrative processes
– Definition and management of these quality processes are the
responsibility of the appropriate Head of Service
– Mechanisms which monitor and regulate administrative
processes are enforced by an internal network of quality-pivots
coordinated centrally (by GAQAP)
– They are disseminated by internet through LEGUA, and/or
incorporated into informatics applications that support actual
activities
– Annually GAQAP gives a general questionnaire to all staff and a
specific one to Heads of Units for identifying staff training needs
– Implementation of a system for evaluation of administrative and
technical staff within the scope of the national SIADAP
(Integrated System for Evaluation in Public Administration)
Setting the scene at European level (1)
•
The initiative to introduce the European Higher Education Area was
launched in 1998, in Paris, and confirmed, one year later, in Bologna
where ministers from 29 countries signed a declaration to the purpose
of creating a borderless EHEA by 2010.
•
The central idea of the Bologna process is to promote student and
staff cross-border mobility through:
– the adoption of a system of comparable two cycle degrees
(Bachelor and Master) and,
– the introduction of a harmonized credit system ECTS (European
Credits Transfer System).
Setting the scene at European level (2)
•
Quality has always played a role in Higher Education but its
importance increased enormously during the last decade.
•
Among the new challenges facing Higher Education Institutions are of
paramount importance the combined requirements of promoting the
Bologna process and participating in the European Higher Education
Area and of creating the European knowledge society.
•
In both cases quality is seen as a key factor to achieve these
objectives.
Setting the scene at European level (3)
•
Why Quality?
– To guarantee the necessary trust between institutions, which is
essential to make sure they will accept students who have acquired
a certain number of credits in another institution, particularly in
another country, the quality of institutions and its audit, as well as
the recognition of degrees, have gained a central role in the whole
process
Setting the scene at European level (4)
•
However
– The emergence of quality as a key factor to the successful
implementation of the EHEA is quite recent.
– In 2003 it became the first policy objective of the Berlin
Communiqué (September 2003). There, it is stated that quality of
higher education is at the heart of the setting up of a European
Higher Education Area and consequently it is essential the
development of quality assurance at institutional, national and
European level
Setting the scene at European level (5)
•
Henceforth the need to develop mutually shared criteria and
methodologies on quality assurance.
•
It is also stressed that the primary responsibility for quality assurance in
higher education lies with each institution itself.
•
It also invited the QA community, together with the Higher Education
Institutions, to develop an agreed set of standards, procedures and
guidelines on quality assurance and to explore the possibility of a peerreview of QA agencies
Setting the scene at European level (6)
•
In 2005 the E4 group (ENQA, EUA, ESIB and EURASHE) proposed
to create an European Register for QA agencies.
•
In April 2007, at the 4th EUA Convention, a recommendation was
issued emphasizing two points:
–
quality should be all embracing and quality assessment must
also recognize diversity and encourage, rather than constrain,
innovation and creativity in teaching and learning.
–
the European Register of QA agencies, expected to be approved
at the 2007 Ministers meeting, has been developed by an
innovative partnership between stakeholders and is seen as an
important block in this area, and an essential component of the
global attractiveness of European Higher Education.
Setting the scene at National level (1)
•
The Portuguese Higher Education System has profoundly changed in the
last thirty five years: the Public University Sector was enlarged and a private
higher education sector emerged and expanded very rapidly. An important
feature is the existence of a binary divide between university-level and
polytechnic-level programmes.
•
The system now comprises:
– University Public Sector
• 14 universities and one non-integrated university institute
• 4 Military University Institutions
– Polytechnic Public Sector
• 15 Institutes
• 21 non-integrated colleges which include nursing, education, military and
police
Setting the scene at National level (2)
•
The private sector is much larger in terms of number of institutions, with
more than 100, but accounting for less than 30% of the students, revealing
the fragmentation of the system.
•
It is also noteworthy that all the public universities are comprehensive in
terms of fields of study and they are, by large, the main actors of research in
Portugal, carried primarily by the academic staff.
Setting the scene at National level (3)
•
In the current context of stabilization in the number of enrolments and
severe shortage of public funds for higher education, the system and the
institutions face considerable constraints.
•
This setting presents great challenges for the regulation and steering of the
system, requiring clear policies on the overall mission and goals to attain,
supported by transparent and stable funding mechanisms and by a regular
and consistent evaluation framework.
•
Failing to do this will not only hinder institutions’ efforts to adopt a strategic
approach, but will also fail to provide the adequate information for students,
parents and other stakeholders, an issue of paramount importance for the
enrolment decision process and for the relationships between institutions
and the society.
Evaluation at National level: the institutions
•
Most public universities have sought international expertise to carry out a
institutional evaluation. This was the case, in particular, of the
Institutional Evaluation Programme run by the European University
Association (EUA), formerly the Council of European Rectors (CRE).
•
During the nineties most public universities, Aveiro, Porto, Minho,
Technical University of Lisbon, University of Lisbon, Universidade Nova
de Lisboa, University of Algarve and Universidade Católica, have invited
EUA to carry out a evaluation of the institution. Some of these institutions
have also done one follow-up.
•
These exercise was self-contained within the institution evaluated, so the
benefits of the exercise were not extended to the different type of
institutions and a systemic perspective could not be obtained.
Evaluation at National level: the system
•
•
•
The Higher Education sector in Portugal is supporting the Glasgow
declaration: Strong Universities for a strong Europe. Europe needs strong
universities as motors in the knowledge triangle of education, research and
innovation.
At national level it is accepted that Portugal needs universities/Higher
Education Institutions that are able to move out of the shadows of nation
state and are able to do both: go for excellence in research as well as in
teaching and provide broad, equitable access to basic higher education in
Europe.
In 2005 the government launched their programme which included as a
priority the integration the Higher Education System in the European Area.
In this context it was emphasised the need to carry out an all embracing
evaluation of the system, according to the best practices in Europe and
elsewhere.
New developments and new legal framework (1)
• Background: Portuguese government promoted discussion on three
levels:
–
–
–
–
HE system level (OECD);
setting-up of a national evaluation system (ENQA);
Institutional level (EUA)
Ministry (of CTES) supports programme for institutional evaluation
New developments and new legal framework (2)
•
•
During 2006 ENQA reviewed the accreditation and quality assurance
practices in the Portuguese higher education against the best European
practices and following the Guidelines for Quality Assurance in the
European Higher Education Area and issued recommendations that are
now being implemented at national level.
These recommendations emphasised the European trend of a distinction
between accreditation at programme level and academic audit and
evaluation at institutional level
New developments and new legal framework (3)
•
The Portuguese Quality Assurance Agency will adopt the following
definition of accreditation:
– The characteristic of accreditation is that the process is based on
established external criteria and that the result of the process is a
decision as to whether these criteria have in fact been met by the
programme under accreditation.
New developments and new legal framework (4)
•
The government decided to encourage ALL the Portuguese Higher
Education Institutions, universities and polytechnic public and private to
apply for an Institutional Evaluation according to the EUA International
Evaluation Programme and set up the following focus for such an
evaluation:
– The main characteristics, positive and negative of the Portuguese
Higher Education institutions, their stage of development in the
European context and their efficiency.
– The level of institutional autonomy and the practices related to
governance and management. The report should make suggestions
for improvement.
New developments and new legal framework (5)
– The institutional capacity for change and the way institutions respond
to the increasing challenges facing higher education.
– The access and the way students are recruited. Special attention
should be paid to LLL and the opening of the institution to the new
publics and professional continuous education.
– The relevance of the diplomas awarded.
– The diversification of funds and the way these are managed.
– Recommendations on how to promote the rationalization of he offer
and diversification of institutions.
New developments and new legal framework (6)
• 2007
– RJIES
• Law that establishes new governance models for HEI.
Institutions should have new statutes until June 2008
– RJAES
• Law that establishes the legal environment for the evaluation
of HE (focus on evaluation of study programmes)
New developments and new legal framework (7)
• The UA decided (2006) to apply for EUA’s institutional evaluation
programme
• The UA is finalising its statutes
– Changes in university governance bodies, both at central and
Department/School levels, may/will lead to a new approach and
organisation for the quality assurance mechanisms
Management
Organic Unit 1
…..
Organic Unit n
Central
Management
Operational
role
Local
Management
Administrative
support
Research
Education
Administrative
support
Research
Education
Re-Structuring the QA Organisation
Key areas of the Action Plan
• From self-evaluation (SWOT) and contribution of external
evaluation team
– Performance indicators
– Bologna Process
– Partnerships
– Funding
– Organisation and governance
Thanks for your Attention