Transcript Dr. Marc Luwel - Research Trends

Performance Based Funding of
Research in OECD Member
States
1
Marc Luwel
Hercules Foundation
CWTS, Leiden University
Washington DC
October 17, 2012
OVERVIEW
• Highlights / Conclusions from the 2010 OECD
Workshop ‘Performance-based Funding for Public
Research in Tertiary Education Institutions’
• Flemish approach to performance based (research)
funding (PB( R)F)
–
–
–
–
Background
Built-up
Impact/outcome
Personal comments
2
TERMINOLOGY
DIFFERENCES BETWEEN EUROPE & US


Fundamental / ‘Blue-sky’ research ↔ Basic
research
Basic research & development ↔ Applied
research
3
PART I
2010 OECD WORKSHOP
‘PERFORMANCE-BASED FUNDING FOR
PUBLIC RESEARCH IN TERTIARY
EDUCATION INSTITUTIONS’
4
HIGHER EDUCATION SYSTEM
TRANSFORMED
• Education / formation : from elitist to mass
education but highest form in education chain
• Knowledge production: fundamental and
applied research
• Outreach activities : Knowledge
dissemination/ valorisation
5
UNIVERSITY FUNDING : DIVERSIFICATION
•
Education
– Higher educational landscape : strongly entrenched in national (regional) culture
–
Europe – Implementation of the Bologna Declaration
– Tuition fee : for-free to full cost
– Public funding of education : inverse relation with tuition fee
– (Continental) Europe : von Humbuldt model – all universities are research
universities & (fiction ?) equal quality among universities
•
Research at universities
– Increased public funding (knowledge critical for competitiveness, societal challenges)
– Revenues from contract research for/in collaboration with firms, public authorities
•
Outreach activities
– ‘Bayh-Dole Act’ type legislation
– Valorisation of IPRs : spin-offs, licencing , …
•
Endowments : Tax code / large cultural differences – Limited in Continental Europe 6
GOVERNMENT’S CHANGING ROLE
• From actor to facilitator and regulator
• Growing pressure on state treasury
– Large societal challenges (aging population,
health, security, resources, …)
– Public infrastructure (mobility, …)
– State of public finances and tax rates – factor in
international competitiveness
7
IMPACT ON HEIS
• Greater autonomy
-
Government stops (micro)managing universities
Government lays down regulatory framework
Introduction of lump sum / block funding
Univ :(some) freedom to set up degree programs
• Accountability to:
– Parliament and Government
– Public at large
8
NEW FORMS OF MANAGEMENT
• Performance based research funding (PBRF)
in HEIs
– PBRF: Concept with many faces
– 14+ OECD-countries use some form of PBRF
– RAE/UK is oldest
9
PBRF : RATIONALE AND ASSUMPTION
• Rationale (implicit / explicit)
– Promote excellence / greater selectivity
– Concentrate / better manage limited resources
• Assumption
– (Research) Performance can be made visible/
measured
– Multidimensional concept
10
PBRF : METHODOLOGY
• Peer view:
–
–
–
–
Generic process of self regulation of the business of science
High credibility
But limitations and biases
“It has been said that democracy is the worst form of government
except all the others that have been tried.”
• Indicators based on ex-post evaluation
• Combination of both
→ Increasingly more sophisticated but no ideal methodology
11
PBRF : 3 TYPES OF INDICATORS
• First order
– Input, processes, results
• Second order
– JIF, CWTS-crown index, (family) H-index (!)
• Third order
– Review panels / ratings (departments)
→ Indicators aggregated at university level to allocate block
funding
12
PBRF : IMPLEMENTATION
• Conception and implementation of PBRF
often in close collaboration with HEIs
• BUT critical comments (from academics)
– Validity of the indicators
– Their use in funding formula
– Intended and unintended consequences
→ anecdotal and limited evidence-based information
(mostly on RAE)
13
PBRF : EFFECTS
• More transparent university funding
• Universities more accountable
• University management response : towards a culture of
excellence but also strategic positioning
• Improvement of information management systems
• A driver for the development of Science of Science and
Innovation Policy
14
PART 2
FLEMISH APPROACH TO PERFORMANCE
BASED (RESEARCH) FUNDING (PB( R)F):
A CASE STUDY
15
BACKGROUND
• Belgium : in a process of devolution of
competences from federal government to
the regions
• 1988 : Flanders became responsible for
(higher) education and nearly all
competences related to Science and
Innovation
16
BACKGROUND -2
• Beginning of the 90’s : Public R&D
expenditures well below EU average
• Since 1995 (except for 2009-2010) :
increase in public R&D funding
– Today : around EU average with 1.85% GDP
– Objective : 2020 – Public sector 2% GDP (and
1% private sector)
17
BACKGROUND – 4
• 5 universities ↔ 6 mio inhabitants
– 2 large universities : KU Leuven (37.000
students and 1.500 faculty) and UGent
– 2 medium sized universities : VUB (10.000
students, 700 faculty) and UAntwerp
– 1 smaller university (only a few degree
programs): UHasselt (3.000 students and 204
faculty)
18
BUILT-UP PBRF – STARTING POINT
Step by step process with Ministers and Universities working closely
together
Starting point : 1990
-
Very limited experience with quantitative science studies in Flanders
-
Universities had some degree of autonomy and lump sum funding (block
funding) consisting of 2 parts
-
a baseline funding (dependent on ‘historical’ data) and
an amount based in weighted number of students
(Correction for inflation)
19
BUILT-UP PBRF – 1990-2000 (1)
• Mainly an increase of the university research funding
– Lump sum: teaching & research
– Impression too many degree programmes /the number of
students → lack of political will to increasing substantially
the lump sum
– Additional funding mechanism for blue sky research :
separate block grant / lump sum for research (‘BOF’) to
allow universities to development their own research
strategy
20
BUILT-UP PBRF - 1990-2000 (2)
• BOF funding formula : the fraction allocated to each university
based on a) block grant, b) number of master students and c)
number of PhD’s-degrees
• Increasing interest in research evaluation
– Having limited in-house knowledge, the universities
commissioned from CWTS a series of bibliometric studies on
research in natural, life and technical sciences (first study: 1991,
UGent) and also in humanities and social sciences (1999, large
scale study)
– The Flemish Government started using bibliometric tools in
evaluation studies (first study commissioned from CWTS: 1994,
ICT in Flanders)
21
BUILT-UP PBRF – 2001 – 2005 (1)
– Fl. Gov. (2001) funded an S&T Indicator Unit to
carry out quantitative studies (prof. K Debackere)
– 2003 revision of the regulation on BOF : New
funding formula using bibliometric data
– Bibliometric data commissioned from S&T
Indicators Unit
22
BUILT-UP PBRF – 2001 – 2005 (2)
• A (input) : each university’s share in total number of Master
degrees, total number of PhD-degrees and in the total lump
sum allocated to all the Flemish universities
• B (output and visibility): each university’s share
– in the total number of SCI publications authored by the
Flemish universities;
– in the total number of citations to these publications
• relative measures to partition yearly a global amount of
money!
• Weight of part B : starting at 10% evolved to 30%
23
BUILT-UP PBF – 2006-2010 (1)
• Fl. Gov. subsidises the Centre for R&D Monitoring
(ECOOM); an interuniversity consortium (successor of
the S&T Indicators Unit) (http://www.ecoom.be/)
• Revision of the BOF funding formula : 2006 (minor) and
2008 (major)
• Additional funding mechanism for applied/ basic research :
separate lump sum / block funding (‘IOF’) to allow universities
to development their own policy for applied research and
innovation
• Revision of the funding formula for lump sum
24
BUILT-UP PBRF – 2006-2010 (2)
• Revision of the BOF funding formula in 2006:
PART A
• Added as a component (human resources
management) : the university’s share in newly
appointed faculty from three groups:
– Underrepresented gender
– Obtained their PhD-degree at another university
– With a substantial period working abroad
25
BUILT-UP PBRF – 2006-2010 (3)

Major revision of the BOF funding formula in 2008 –

PART A : each university’s fraction of the total lump sum
allocated to the Flemish universities
REPLACED BY
each university’s share in the total number of
academic staff (tenured and non-tenured (incl. PhDstudents) at the Flemish universities
26
BUILT-UP PBRF – 2006-2010 (4)

PART B

Publications with weighting factors (incl. JIF) from
SCIE or SSCI with an impact factor
 SCIE or SSCI without an impact factor
 AHCI
 Proceedings STP and SSHP
 From 2011 – publications in VABB-SHW


Citations
27
BUILT-UP PBRF – 2006-2010 (5)

VABB-SHW : the Flemish bibliometric basic database
for the Humanities and the Social Sciences

Along the lines of the Norwegian approach

Responsibility of the Flemish universities / technical
assistance by ECOOM

Standing Committee selects publications (serial literature,
monographs, edited books, …) not covered by WoS

Criteria : starting point is the Governmental regulation but
elaborated by Standing Committee ; weighting factors, …

BUT : Quality review by independent panel; its members not
working in Flanders ! (First time : 2012) ‘ Trust but verify’

http://www.ecoom.be/index.php?id=101&L=0
28
BUILT-UP PBRF – 2006-2010 (6)

IOF : block funding for applied research and valorisation

Funding formula based on a university’s share in:
Number of academic staff (BOF definitions)
- Number of PhD-degrees
- Number of publications and citations (BOF formula)
-
Revenues from contract research
 Revenues from EU Framework Programmes
 Number of patent applications (0.5) at and patents
granted (1.0) by EPO and USTPO
 Number of spin-offs

29
BUILT-UP PBRF – 2006-2010 (7)
 Revision
of the funding formula for the
university’s lump sum / block funding


Component education takes o.a. the number of
Ba & Ma degrees delivered by each
university
Component research based on BOF funding
formula
30
SOME EFFECTS ON PERFORMANCE
Evolution of the number of WoS
publications and PhD degrees
14,000
1,300
13,000
1,200
12,000
1,100
11,000
1,000
10,000
900
9,000
800
8,000
700
7,000
600
6,000
500
5,000
4,000
WoS publ
1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010
5,740 6,474 6,753 6,765 7,201 7,446 8,358 8,478 10,11 9,953 10,81 12,59 13,08 13,05
Doctoraten 580
670
595
669
723
811
862
924 1,007 1,070 1,091 1,204 1,228
400
31
TRENDS
IN USE OF
ENGLISH
AND
WOS COVERAGE.
PERCENTAGES OF ALL SCHOLARLY JOURNAL ARTICLES FROM
FLEMISH (F) AND NORWEGIAN (N) UNIVERSITIES THAT WERE
PUBLISHED IN ENGLISH AND
COVERED BY WEB OF SCIENCE 2005-2009
(‘SIVERTSEN G., OSSENBLOK T. & ENGELS T.C.E. (TO BE PUBLISHED)’.
70%
60%
50%
40%
30%
English - N
20%
English - F
WoS - N
10%
WoS - F
32
0%
2005
2006
2007
2008
2009
COULD BE DONE BETTER




BOF funding formula : by far and large too
complex
Allocation model for BOF and IOF and research
component of university’s lump sum is based on a
university’s share in a sum of money available for
all universities ►not taking into account
absolute but relative variations !
Benchmarking is between Flemish universities,
should be with top universities world wide, but
like with like
Some strategic behaviour ? But the effects of the
funding model still badly documented except for the
macro trends
33
QUESTIONS?
34
Thank YOU !