Transcript Five Myths
Five Myths
About Funding Scientific Research (in Austria) ….
and what Evaluation can do to make them more ‘Evidence Based’.
Five Myths
• „No Money for the Humanities and Social Sciences in Austria!“ • „There is a funding gap between basic research and applied research!“ • „Engeneering is treated unfair!“ • Networks, Fretworks • Impacts – now!
Myth 1:
„No money in the humanities / social sciences“
• Humanites get a raw deal… (Der Standard, 31. Mai 2005) • Social Sciences and Humanities are „starved out“ financially (Austrian Green Party, 13. Mai 2005) • …marginalisation of the Humanities…“.
(M. Nießen, DFG)
Mapping the Social Sciences & Humanities in Austria…
• No Evaluation / No Benchmarking Exercise in the field • Lack of data – Contract research of the ministries?
• No vivid programme scene • … but looking at further empirical evidence…
R&D in the higher education sector, 2002
R&D personnel number of R&D units headcount FTE R&D expenditures in 1000 EUR Natural sciences Technical sciences Medicine (incl. clinics) Agriculture, Forestry & Veterinary Medicine Social Sciences & Humanities >> Social Sciences >> Humanities 197 173 144 44 411 208 203 6.469
3.502
7.284
1.060
6.757
3.775
2.982
4.865,2 2.690,6 6.025,6 847,5 4.993,8 2.718,4 2.275,4 387.193
173.493
333.516
70.089
301.813
165.755
136.058
Source: FTB 2005
Contract Research, 2003
Natural sciences Technical sciences Medicine (incl. clinics) total sum 11.099.561
in % 19,6 7.472.237
13.264.064
13,2 23,5 bm:bwk 8.794.489
1.686.947
12.848.845
bm:vit 617.055
5.214.199
89.969
bm:wa 187.097
132.180
Agriculture, Forestry & Veterinary Medicine Social Sciences & Humanities >> Social Sciences >> Humanities 2.997.521
21.698.122
14.735.356
6.962.766
5,3 12,3 464.558
38,4 18.189.699
26,1 11.226.933
6.962.766
1.129.160
1.129.160
10.433
950.817
950.817
Source: FTB 2005
FWF-project funding Acceptance Rates, 1998-2003
• Highest Acceptance Rate – Natural Sciences and Humanities: 58% • Lowest Acceptance Rate – Agriculture and Social Sciences: 35% • Funding Rates – Quite homogeneous – 70 % Human Medicine – 80% Humanities Source: Streicher 2004
A „Benchmarking Exercise“
• Benchmark Project – Natural Sciences, male co-ordinator – Age 40-50, Size 150 – 250 k€ – Approval Rate: 52,4% Variable Technical Sciences Human Medicine % Difference in approval rate - 8,5 -15,1 Agriculture, Forestry, VetMed Social Sciences -18,1 -19,2 Humanities + 4,5 (A cautionary remark: It would be wrong to interpret the co-efficents causally) Source: FWF Evaluation, Streicher, Schibany 2004
A „Benchmarking Exercise“
• Yes, projects of a different scientific flavour face significantly different chances – Against a benchmark project, Social Sciences are rejected far more frequently – Humanities are (slightly but significantly) more successful Source: Streicher 2004
Take into account…
• Classification • Structural Issues – Age?
– Fragmentation of Research Units?
– Perspectives for younger researchers?
– Researchers = working poor?
• Quality – Kind of Indicators • ……
Heterogeneous average working loads
(in % of total working hours) Natural sciences Technical sciences Medicine (incl. clinics) >> without clinics >> clinics Agriculture, Forestry & Veterinary Medicine Social Sciences & Humanities >> Social Sciences >> Humanities teaching & training 29,5 31,3 16,8 24,7 14,0 25,6 45,0 43,8 46,5 R&D 64,4 61,5 36,7 65,8 26,3 57,0 47,4 48,5 46,1 other tasks 6,1 7,2 46,5 9,5 59,7 17,4 7,6 7,7 7,4 Source: FTB 2005
Conclusions • There is never enough money for doing research – No evidence, that Humanities / Social Sciencies are treated unfair • „Not enough money for the humanities / social Sciencies?“ – This is an urban legend
Challanges for the future:
• Evaluators – Evaluators should be • Sceptical, • suspicious of everybody – Triangulation is necessary!
– Quantitative Methods are valuable sources of information • Stakeholders – Ask the big questions (from time to time), too.
– Give the evaluators the degrees of freedom to answer these questions.
Mythos 2:
Funding Gaps between basic and applied sciences
„Funding Gap“ Basic Sciences Applied Sciences
Dream Nightmare Reality
Risk Aversity & FFF
Source: FFF Evaluation, Arnold 2004 Source: FFF Evaluation, Jörg 2004 Source: FFF Evaluation, Arnold 2004 • Overall [FFF] tends to take too little risk.
• FFF funding practice is risk-averse.
• [The linear model] is a misleading oversimplification that encourages us to make poor policy decisions.
FWF Projects: Commercial Output & Usability
41%: results are relevant for industry 30%: important lab results 20%: working prototypes exist 13% research results are suitable for commercialization straight away Source: FWF Evaluation, Streicher, Schibany 2004
101.51
~ 70 Millionen € CDG K-Ind / Knet Kplus 7.3 (2003) K-Ind ?
18.9
Informationstechnologien: FIT-IT Nanotechnologie: Nano-Initiative Luftfahrt: TAKE OFF Weltraum: ASAP & ARTIST Verkehrstechnologien: ISB & A3 BRIDGE: Translational Research & Brückenschlagprogramm 5.9
10.8
3.5
11.6
8 5.11 (2004 Translational FWF) 127.15
Basic Sciences Applied Sciences
Conclusions
• There is no funding gap (anymore) • „Funding gap: No guiding principle for policymakers (anymore) • Funding Gap: Urban Legend II Source: mid term Evaluation FIT-IT, 2005
Challenges for the Future I
• In a NIS, there is the need for Evaluation of Systems
(from time to time)
• In a NIS, there is the need for Evaluation of Portfolios
(from time to time)
Challenges for the future II
• Room for „curiosity driven Evaluation“ • Methodological Development – Evaluation is no pure science, but – It is no consulting business, too.
– Of cause, Evaluation must have a sound scientific basis • Ensure degrees of freedom – Budget!
– TORs • Fight Evaluation Fatigue – Realistic expectations – sufficent time spans
Next Steps
Paper, part of the conference….
„New Frontiers in Evaluation“ www.fteval.at/conference06
24./ 25. April 2006 Vienna, Austria
Team
Michael Dinges, Joanneum Research Michaela Glanz, WWTF Brigitte Tempelmaier, WWTF