Transcript Science for Sustainable development SSD

20 YEARS OF SCIENTIFIC RESEARCH IN
HEALTH/WORK/ENVIRONMENT
September 6, 2012
Thoughts of a reviewer
Prof Dick Heederik, PhD
IRAS, Utrecht University, The Netherlands
Reviewing scientific proposals
and programs

Involved in BESLPO project evaluation and
the SSD Health program evaluation as
panel member
Program evaluation SSD
Health






Scientific quality
Networking
Internationalization
Policy relevance
Coverage of the program
Characteristics of the program








Project level (network, budget, duration …)
Program level (calls, budget, …)
Follow-up committee
International projects/EU
Clusters
Dissemination
Relevance of the program
Other considerations
Developments in the scientific
community






Scientific production more dominated by
teams, even in field traditionally dominated by
solists
Teams produce more highly cited papers
Development is seen in all areas, over time,
even after removal of self-citations
Networks have become the dominant and most
prominent way to go
Research at disciplinary frontiers and in novel
areas is often inter-disciplinary
Research management becomes interested in
R&D structures
The role of peer review in project
and proposal selection?
http://www.bishop-hill.net/
Criteria voor Quality Assessment
in Peer Review (NIH)





Significance impact (does the
project address an important
problem or critical barrier to
progress in the field)?
Investigators (well suited to the
project)
Innovation (shift current research
or practice paradigms)
Approach appropriate?
Will the scientific environment
contribute to succes?
Scientific Quality: publications,
citations, publication networks …..
From intuitive interpretation to quantitative analysis …
Quality: characteristics of good
research groups

Leaders of high performing research
groups survey:
 High performance research
(publications, citations (normalized for
group size)
 Stronger research commitment
 More effort in group management
 Spent more time on network
management
 All rounders

Verbree et al., Rathenau institute, NL
Quality: different types of
excellent groups




Output types correlate poorly:
publications, citations, productivity,
citations per publication
…, and have different determinants.
So, it also depends to some extent
on what is asked
Verbree et al. 2012 Rathenau
Institute
Peer review program evaluation:
output evaluation parameters
Program performance
~1.8 Meuro/year internal support, ~ 70% external projects
Impact in different sub-fields
SSD



Too early to make a formal
quantitative analysis of impact of the
BELSPO Health program
Does this result in unbiased
impression given the likely additional
funding from other sources?
In essence evaluation of participating
groups
Networking and
internationalization




Strong interdisciplinary collaboration
(PARHEALTH, S2Nano, SHAPES)
Projects did not make use of
additional funding possiblities to
finance international partners
Some groups had strong
international networks but connection
with international research
community could be strengthened
Collaboration with industry limited
(S2Nano)
More formal approaches to analyze networks:
44-cluster co-authorship network of papers at
the 10% highly-cited threshold (Rosas et al.
PLoSone, 2011 )
Collaborative output
Dissemination




Follow-up committee not for all projects
useful, for others effective
Projects which have a stronger basic
research focus could benefit from a
scientific steering committee
More options for dissemination should
be considered (internet databases,
software tools, etc.)
To make scientific results available for
society may require an additional
research cycle
Dissemination




Role in evaluation of future exposure
standards (PARHEALTH)
Results can be used by local
planners (cost benefits of various
modes of transport SHAPES)
Use of developed concepts in testing
guidelines (S2Nano)
Breakthrough technology (ANIMO)
Coverage of the field




20% of diseases associated with environmental factors
.. (Kirsh-Volders et.al. 2012)
Occupational exposures (chemical, biological, physical)
Environmental exposures (outdoor, indoor)
Do we know the priorities in our field (risk, impact, time,
DALY)?
The field environment and health




Small populations at risk, high risks (MICATR)
Large populations at risk, low risks (SHAPES,
PARHEALTH)
New emerging risks (S2Nano)
New approaches/technologies (MIC-ATR,
ANIMO)
Overall appreciation





Small program
Relevant for capacity building in
Belgium
Relevant for public health in relation
to the environment in Belgium
The program delivers value for
money
Effect of most projects is beyond the
project period
Environment and health: funding



Public health
Public funding versus industry funding
Mixed funding (Health Effects Institute)?
Where are we going?