Transcript Universiteit Leiden

Measuring Quality and Impact of the Social
Sciences
Concepts, Opportunities and Drawbacks
Pre-Conference of the 10th International Conference on
Science and Technology Indicators
University of Vienna, September 17, 2008
Anthony F.J. van Raan
Center for Science and Technology Studies (CWTS)
Leiden University
This presentation will highlight recent
CWTS projects:
* Benchmarking & Evaluation
* HEFCE
* Identification of Excellence
From these recent studies we present
empirical results for social science fields
particularly concerning:
*
*
*
*
WoS coverage
Characteristics of WoS publications
Characteristics of n-WoS publications
Bibliometric results and peer judgments
First the basic principles of
bibliometric analysis
Basic Concept: Quality
Scientific performance relates to achieved quality in the
contribution to the increase of our knowledge
(‘scientific progress’)
(1) as perceived by others: peer review
(2) as measured by advanced bibliometric analysis
Basic issues for research assessment, also in
the social sciences:
* Objectivity
* Transparency
* How to handle interdisciplinarity, definition of
fields
* Different ways, prestige and intensity of
publication
* Role of co-authors in publications
* Orientation of research: local vs. global
* Language
* Ageing of research results
* PhD training
* Time dimension of awards
* Socio-economic impact
Citing Publications
networks leading, possibly, to different dynamics, e.g., for the initiation and spread of epidemics.
In the context of network growth, the impossibility of knowing the degrees of all the nodes comprising the network due to the filtering process—
and, hence, the inability to make the optimal, rational, choice—is not altogether unlike the “bounded rationality” concept of Simon [17].
Remarkably, it appears that, for the description of WWW growth, the preferential attachment mechanism, originally proposed by Simon [10],
must be modified along the lines of another concept also introduced by him—bounded rationality [17].
We thank R. Albert, P. Ball, A.-L. Barabási, M. Buchanan, J. Camacho, and R. Guimerà for stimulating discussions and helpful suggestions. We
are especially grateful to R. Kumar for sharing his data. We thank NIH/NCRR (P41 RR13622) and NSF for support.
Cited Publications
[1] S. H. Strogatz, Nature (London) 410, 268 (2001).
[2] R. Albert and A.-L. Barabási, Rev. Mod. Phys. 74, 47 (2002).
[3] S. N. Dorogovtsev and J. F. F. Mendes, Adv. Phys. (to be published).
[4] R. Albert, H. Jeong, and A.-L. Barabási, Nature (London) 401, 130 (1999).
[5] B. A. Huberman and L. A. Adamic, Nature (London) 401, 131 (1999); R. Kumar et al., in Proceedings of the 25th International Conference on
Very Large Databases (Morgan Kaufmann Publishers, San Francisco, 1999), p. 639; A. Broder et al., Comput. Netw. 33, 309 (2000); P. L.
Krapivsky, S. Redner, and F. Leyvraz, Phys. Rev. Lett. 85, 4629 (2000); S. N. Dorogovtsev, J. F. F. Mendes, and A. N. Samukhin, ibid. 85, 4633
(2000); A. Vazquez, Europhys. Lett. 54, 430 (2001).
[6] M. Faloutsos, P. Faloutsos, and C. Faloutsos, Comput. Commun. Rev. 29, 251 (1999); G. Caldarelli, R. Marchetti, and L. Pietronero,
Europhys. Lett. 52, 386 (2000); A. Medina, I. Matta, and J. Byers, Comput. Commun. Rev. 30, 18 (2000); R. Pastor-Satorras, A. Vazquez, and A.
Vespignani, arXiv:cond-mat/0105161; L. A. Adamic et al., Phys. Rev. E 64, 046135 (2001).
[7] F. B. Cohen, A Short Course on Computer Viruses (Wiley, New York, 1994); R. Pastor-Satorras and A. Vespignagni, Phys. Rev. Lett. 86,
3200 (2001); Phys. Rev. E 63, 066117 (2001).
[8] F. Liljeros, C. R. Edling, L. A. Nunes Amaral, H. E. Stanley, and Y. Åberg, Nature (London) 411, 907 (2001).
[9] A.-L. Barabási and R. Albert, Science 286, 509 (1999).
[10] Y. Ijiri and H. A. Simon, Skew Distributions and the Sizes of Business Firms (North-Holland, Amsterdam, 1977).
[11] G. Bianconi and A.-L. Barabasi, Europhys. Lett. 54, 436 (2001).
[12] A. F. J. Van Raan, Scientometrics 47, 347 (2000).
[13] We consider a modification to the network growth rule described earlier in the paper: at each time step t, the new node establishes m new
links, where m is drawn from a power law distribution with exponent gout.
[14] For n(I) = const, one recovers the scale-free model of Ref. [9].
[15] It is known [11] that, for an exponential or fat-tailed distribution of fitness, the structure of the network becomes much more complex; in
particular, the in-degree distribution is no longer a power law. Hence, we do not consider in this manuscript other shapes of the fitness
distribution.
[16] L. A. N. Amaral, A. Scala, M. Barthélémy, and H. E. Stanley, Proc. Natl. Acad. Sci. U.S.A. 97, 11 149 (2000).
[17] H. A. Simon, Models of Bounded Rationality: Empirically Grounded Economic Reason (MIT Press, Cambridge, 1997).
All calculations are corrected for self-citations!
What do citations measure?
- Many studies showed positive correlations
between citations and qualitative judgments
- In principle it is valid to interpret citations in
terms of intellectual influence which is an
important aspect of scientific quality
- Thus, the concepts of citation impact and
scientific quality do not coincide ‘automatically’
Total publ universe
non-WoS publ:
Books
LNCS
ArXiv
Book chapters
Google Scholar
Conf. proc.
Source expansion
Reports
VOLUME 88, Number 13 PHYSICAL REVIEW LETTERS
VOLUME 88, Number 13 PHYSICAL REVIEW LETTERS
Scopus
WoS sub-universe
Compendex
Stefano Mossa,1,2 Marc Barthélémy,3 H. Eugene Stanley,1 and Luís A. Nunes Amaral1
1 Center for Polymer Studies and Department of Physics, Boston University, Boston, Massachusetts 02215
2 Dipartimento
di Fisica,LETTERS
INFM UdR, and INFM Center
for Statistical
VOLUME 88, Number 13 PHYSICAL
REVIEW
1 April
2002 Mechanics and Complexity,
Università di Roma “La Sapienza,” Piazzale Aldo Moro 2, I-00185, Roma, Italy
CEA-Service de Physique de la Matière Condensée, BP 12, 91680 Bruyères-le-Châtel, France
Truncation of Power Law 3Behavior
in “Scale-Free”
Network Models
(Received 18 October 2001; published 14 March 2002)
Stefano Mossa,1,2 Marc Barthélémy,3 H. Eugene Stanley,1 and Luís A. Nunes Amaral1
1 Center for Polymer Studies and Department of Physics, Boston University, Boston, Massachusetts 02215
2 Dipartimento di Fisica, INFM UdR, and INFM Center for Statistical Mechanics and Complexity,
Università di Roma “La Sapienza,” Piazzale Aldo Moro 2, I-00185, Roma, Italy
3 CEA-Service de Physique de la Matière Condensée, BP 12, 91680 Bruyères-le-Châtel, France
(Received 18 October 2001; published 14 March 2002)
We formulate a general model for the growth of scale-free networks under filtering information
conditions—that is, when the nodes can process information about only a subset of the existing nodes in the
network. We find that the distribution of the number of incoming links to a node follows a universal scaling
due to Information Filtering
form, i.e., that it decays as a power law with an exponential truncation controlled not only by the system size
We formulate a general model for the growth of scale-free networks under filtering information
but also by a feature not previously considered, the subset of the network “accessible” to the node. We test our
Stefano Mossa,1,2 Marcconditions—that
Barthélémy,3 H.is,Eugene
Stanley,1
and process
Luís A. Nunes
Amaral1
when the
nodes can
information
about only a subset of the existing nodes in the
model with empirical data for the World Wide Web and find agreement.
1 Center for Polymer Studies andnetwork.
Department
of Physics,
University,
02215to a node follows a universal scaling
We find
that theBoston
distribution
of theBoston,
numberMassachusetts
of incoming links
2 Dipartimento di Fisica, INFM
Center
Statistical
Mechanics
and Complexity,
form,UdR,
i.e., and
that INFM
it decays
as afor
power
law with
an exponential
truncation controlled not only by the system size
DOI: 10.1103/PhysRevLett.88.138701 PACS numbers: 89.20.Hh, 84.35.+i, 89.75.Da, 89.75.Hc
Università di Romabut“LaalsoSapienza,”
Piazzale
Aldo Moroconsidered,
2, I-00185,theRoma,
by a feature
not previously
subsetItaly
of the network “accessible” to the node. We test our
3 CEA-Service de Physiquemodel
de la with
Matière
Condensée,
91680Wide
Bruyères-le-Châtel,
France
empirical
data forBPthe12,World
Web and find agreement.
(Received 18 October 2001; published 14 March 2002)
VOLUME
Number 13 TherePHYSICAL
LETTERS
is a great dealREVIEW
of current interest
in understanding the1 April
structure2002
and growth mechanisms of global networks [1–3], such as the World Wide
DOI: 10.1103/PhysRevLett.88.138701 PACS numbers: 89.20.Hh, 84.35.+i,
89.75.Da,88,89.75.Hc
We formulate a general model for the growth of scale-free networks under filtering information
Web (WWW) [4,5] and the Internet [6]. Network structure is critical in many contexts such as Internet attacks [2], spread of an Email virus [7], or
conditions—that is, when the nodes can process information about only a subset of the existing nodes in the
of human epidemics
[8]. In all theseNetwork
problems, theModels
nodes with the largest number of links play an important role on the dynamics of the
Truncation of Powerdynamics
Law Behavior
in “Scale-Free”
network. We find that the distribution of the number of incoming links to a node follows a universal scaling
system. It is therefore important to know the global structure of the network as well as its precise distribution of the number of links.
due
Information
Filtering
Thereasis a power
great deal
current
interest in understanding
the structure
andbygrowth
mechanisms
as theto
World
Widestudies report
form, i.e., that it decays
law ofwith
an exponential
truncation controlled
not only
the system
size of global networks [1–3], suchRecent
empirical
that both the Internet and the WWW have scale-free properties; that is, the number of incoming links and the
and the Internet
[6].ofNetwork
structure
is criticaltointhe
many
contexts
suchouras Internet attacks [2], spread of annumber
Email virus
[7], or links at a given node have distributions that decay with power law tails [4–6]. It has been proposed [9] that the scale-free
but also by a featureWeb
not (WWW)
previously[4,5]
considered,
the subset
the network
“accessible”
node.
We test
of outgoing
Mossa,1,2
Barthélémy,3
andmay
LuísbeA.explained
Nunes Amaral1
dynamics
all these
problems, the nodes with the largest number of links playStefano
an important
roleMarc
on the
dynamics
theEugeneandStanley,1
model with empirical
data forofthehuman
Worldepidemics
Wide Web[8].andInfind
agreement.
structure
of theofH.
Internet
the WWW
by a mechanism referred to as “preferential attachment” [10] in which new nodes link
1 Center
for Polymer
Department
of Physics,
Boston University,
Boston,
Massachusetts
02215 links to these nodes. Here we focus on the stochastic character of the
system. It is therefore important to know the global structure of the network as well as its precise
distribution
of the Studies
number and
of tolinks.
existing nodes
with a probability
proportional
to the
number of existing
Dipartimento
di Fisica,
INFM
UdR,
for Statistical
and Complexity,
Recent empirical studies PACS
report numbers:
that both 89.20.Hh,
the Internet84.35.+i,
and the89.75.Da,
WWW have
scale-free properties;2 that
is, the number
of incoming
linksandattachment
andINFM
the Center
DOI: 10.1103/PhysRevLett.88.138701
89.75.Hc
preferential
mechanism,
whichMechanics
we understand
in the following way: New nodes want to connect to the existing nodes with the largest
di Roma
“La
Piazzale
Moro 2,degree—because
I-00185, Roma, Italy
number of outgoing links at a given node have distributions that decay with power law tails [4–6]. It hasUniversità
been proposed
[9]number
that Sapienza,”
theofscale-free
links—i.e.,
withAldo
the largest
of the advantages offered by being linked to a well-connected node. For a large network it
3 CEA-Service
Physique
Matière
91680
France nodes, so a new node must make a decision on which node to connect with
structure of the Internet and the WWW may be explained by a mechanism referred to as “preferential
attachment”de[10]
in which
nodesCondensée,
link a new BP
isdenotlanew
plausible
that
node12,will
knowBruyères-le-Châtel,
the degrees of all existing
(Received
18onOctober
2001;
Marchthe2002)
to existing nodes with a probability proportional to the number of existing links to these nodes. Here we focus on the stochastic
the published
basedcharacter
whatofinformation
it has14about
state of the network. The preferential attachment mechanism then comes into play as nodes with a
There is a great deal of current interest
in understanding
themechanism,
structure andwhich
growth
global
networks
the World
Wideto the existing nodes
preferential
attachment
wemechanisms
understand inofthe
following
way:[1–3],
New such
nodesaswant
to connect
the largest
largerwith
degree
are more likely to become known.
We virus
model
growthit of scale-free networks under filtering information
Web (WWW) [4,5] and the Internet [6].
Network
structure with
is critical
in many
contexts such asofInternet
attacks [2],
spread
of an Email
[7], ora generalnode.
number
of links—i.e.,
the largest
degree—because
the advantages
offered
by being
linked
toformulate
a well-connected
For aforlargethenetwork
conditions—that
is, whenonthewhich
nodesnode
can toprocess
about only a subset of the existing nodes in the
dynamics of human epidemics [8]. Inisallnotthese
problems,
largest
number of all
linksexisting
play annodes,
important
role on
themust
dynamics
the
plausible
that athenewnodes
nodewith
willthe
know
the degrees
so a new
node
make of
a decision
connectinformation
with
We find then
that the
distribution
number
system. It is therefore important to know
of the
network
its the
precise
distribution
of the number
ofnetwork.
links.mechanism
basedtheonglobal
what structure
information
it has
aboutasthewell
stateas of
network.
The preferential
attachment
comes
into playofasthenodes
withofa incoming links to a node follows a universal scaling
form, i.e.,
decays as a power law with an exponential truncation controlled not only by the system size
Recent empirical studies report thatlarger
both the
Internet
and likely
the WWW
have known.
scale-free properties; that is, the number of incoming
linksthatandit the
degree
are more
to become
a feature not previously considered, the subset of the network “accessible” to the node. We test our
number of outgoing links at a given node have distributions that decay with power law tails [4–6]. It has been proposed [9]butthatalsothebyscale-free
modelnew
withnodes
empirical
structure of the Internet and the WWW may be explained by a mechanism referred to as “preferential attachment” [10] in which
link data for the World Wide Web and find agreement.
to existing nodes with a probability proportional to the number of existing links to these nodes. Here we focus on the stochastic character of the
PACS numbers: 89.20.Hh, 84.35.+i, 89.75.Da, 89.75.Hc
preferential attachment mechanism, which we understand in the following way: New nodes want to connect to the existing nodesDOI:
with 10.1103/PhysRevLett.88.138701
the largest
number of links—i.e., with the largest degree—because of the advantages offered by being linked to a well-connected node. For a large network it
is not plausible that a new node will know the degrees of all existing nodes, so a new node must make a decision on which node to connect with
based on what information it has about the state of the network. The preferential attachment mechanism then comes into play as nodes with a
There is a great deal of current interest in understanding the structure and growth mechanisms of global networks [1–3], such as the World Wide
larger degree are more likely to become known.
Web (WWW) [4,5] and the Internet [6]. Network structure is critical in many contexts such as Internet attacks [2], spread of an Email virus [7], or
dynamics of human epidemics [8]. In all these problems, the nodes with the largest number of links play an important role on the dynamics of the
system. It is therefore important to know the global structure of the network as well as its precise distribution of the number of links.
Recent empirical studies report that both the Internet and the WWW have scale-free properties; that is, the number of incoming links and the
number of outgoing links at a given node have distributions that decay with power law tails [4–6]. It has been proposed [9] that the scale-free
structure of the Internet and the WWW may be explained by a mechanism referred to as “preferential attachment” [10] in which new nodes link
to existing nodes with a probability proportional to the number of existing links to these nodes. Here we focus on the stochastic character of the
preferential attachment mechanism, which we understand in the following way: New nodes want to connect to the existing nodes with the largest
number of links—i.e., with the largest degree—because of the advantages offered by being linked to a well-connected node. For a large network it
is not plausible that a new node will know the degrees of all existing nodes, so a new node must make a decision on which node to connect with
based on what information it has about the state of the network. The preferential attachment mechanism then comes into play as nodes with a
larger degree are more likely to become known.
*CWTS is in license agreement negotiations
8,000 j; 1,000,000p/y
with Scopus
*CWTS currently compares Scopus- vs. WoS
coverage
*CWTS bibliometric algorithms can be applied
Medline
to
Scopus
data
Refs > nWoS
VOLUME 88, Number 13 PHYSICAL REVIEW LETTERS
1 April 2002
VOLUME 88, Number 13 PHYSICAL
REVIEWofLETTERS
1 April
2002
Truncation
Power Law Behavior
in “Scale-Free”
Network Models
due to Information Filtering
1 April 2002
Truncation of Power Law Behavior in “Scale-Free” Network ModelsVOLUME 88, Number 13 PHYSICAL REVIEW LETTERS
Stefano Mossa,1,2
Marc Barthélémy,3 H. Eugene Stanley,1 and Luís A. Nunes Amaral1
due to Information
Filtering
VOLUME 88, Number 13 PHYSICAL REVIEW LETTERS 1 Center for
1 April
Polymer2002
Studies and Department of Physics, Boston University,
Boston,ofMassachusetts
Truncation
Power Law02215
Behavior in “Scale-Free” Network Models
2 Dipartimento
di Fisica,and
INFM
and INFM
Center for Statistical Mechanics and Complexity,
Stefano Mossa,1,2 Marc Barthélémy,3
H. Eugene Stanley,1
LuísUdR,
A. Nunes
Amaral1
due to Information Filtering
Università
Roma “LaBoston,
Sapienza,”
Piazzale Aldo
Truncation of Power Law1Behavior
in “Scale-Free”
Network
Models
Center for Polymer
Studies and Department
of Physics,
Boston diUniversity,
Massachusetts
02215Moro 2, I-00185, Roma, Italy
3INFM
CEA-Service
de Statistical
Physique deMechanics
la MatièreandCondensée,
BP 12, 91680
Bruyères-le-Châtel,
France
2
Dipartimento
di
Fisica,
INFM
UdR,
and
Center
for
Complexity,
Stefano
Mossa,1,2
Marc
Barthélémy,3
H.
Eugene
Stanley,1
and
Luís
A.
Nunes
Amaral1
due to Information Filtering
(Received 18 October 2001; published 14 March 2002)
Università di Roma “La Sapienza,” Piazzale Aldo Moro 2, I-00185, Roma, Italy
1 Center for Polymer Studies and Department of Physics, Boston University, Boston, Massachusetts 02215
de Physique
de laA.Matière
Condensée,
BP 12, 91680 Bruyères-le-Châtel, France 2 Dipartimento di Fisica, INFM UdR, and INFM Center for Statistical Mechanics and Complexity,
Stefano Mossa,1,2 Marc Barthélémy,33 CEA-Service
H. Eugene Stanley,1
and Luís
Nunes
Amaral1a general
We
formulate
model 2002)
for the growth of scale-free networks
under difiltering
information
(Received Boston,
18 October
2001; published
14 March
Università
Roma “La
Sapienza,” Piazzale Aldo Moro 2, I-00185, Roma, Italy
1 Center for Polymer Studies and Department of Physics, Boston University,
Massachusetts
02215
conditions—that
is, when
the nodes can process information about only
a subset of de
thePhysique
existing de
nodes
in the Condensée, BP 12, 91680 Bruyères-le-Châtel, France
3 CEA-Service
la Matière
2 Dipartimento di Fisica, INFM UdR, and INFM Center for Statistical Mechanics
and
We Complexity,
find that
the distribution
the number
of incoming
links to a node follows(Received
a universal18scaling
We formulate
general
fornetwork.
theRoma,
growth
scale-free
networksof under
filtering
information
October 2001; published 14 March 2002)
Università di Roma “La Sapienza,”
Piazzalea Aldo
Moromodel
2, I-00185,
Italyitofdecays
form,
i.e.,information
that
as aonly
power
law with
an exponential
truncation
is, when
nodes can
process
about
a subset
of the
existing nodes
in the controlled not only by the system size
3 CEA-Service de Physique de conditions—that
la Matière Condensée,
BP the
12, 91680
Bruyères-le-Châtel,
France
by a feature
not previously
the asubset
of thescaling
network
“accessible”
to themodel
node. We
We2001;
find published
that the distribution
ofbutthealso
number
of incoming
links to aconsidered,
node follows
universal
We
formulate
a general
for test
the ourgrowth of scale-free networks under filtering information
(Received network.
18 October
14 March 2002)
empiricaltruncation
data for thecontrolled
World Wide
andthe
findsystem
agreement.
form, i.e., that it decays as a power law model
with anwith
exponential
not Web
only by
size
conditions—that is, when the nodes can process information about only a subset of the existing nodes in the
alsogrowth
by a feature
not previously
considered,
thefiltering
subset ofinformation
the network “accessible” to the node. Wenetwork.
test ourWe find that the distribution of the number of incoming links to a node follows a universal scaling
We formulate a general model forbut the
of scale-free
networks
under
DOI:
10.1103/PhysRevLett.88.138701
PACS numbers: form,
89.20.Hh,
84.35.+i,
89.75.Da,
89.75.Hc
empirical data
the aWorld
Webexisting
and findnodes
agreement.
i.e., that
it decays
as a power
law with an exponential truncation controlled not only by the system size
conditions—that is, when the nodes canmodel
processwithinformation
aboutforonly
subsetWide
of the
in the
but also by a feature not previously considered, the subset of the network “accessible” to the node. We test our
network. We find that the distribution of the number of incoming links to a node follows a universal scaling
10.1103/PhysRevLett.88.138701
89.20.Hh,
form, i.e., that it decays as a power law withDOI:
an exponential
truncation controlled notPACS
onlynumbers:
by the system
size 84.35.+i, 89.75.Da, 89.75.Hc model with empirical data for the World Wide Web and find agreement.
but also by a feature not previously considered, the subset of theThere
network
the node.
We intestunderstanding
our
is a“accessible”
great deal oftocurrent
interest
the structure and growth mechanisms of global networks [1–3], such as the World Wide
DOI: 10.1103/PhysRevLett.88.138701 PACS numbers: 89.20.Hh, 84.35.+i, 89.75.Da, 89.75.Hc
model with empirical data for the World Wide Web and find agreement.
Web (WWW) [4,5] and the Internet [6]. Network structure is critical in many contexts such as Internet attacks [2], spread of an Email virus [7], or
of human
epidemicsand
[8].growth
In all these
problems,
nodes
with the[1–3],
largest
links Wide
play an important role on the dynamics of the
There is a great deal of current interest indynamics
understanding
the structure
mechanisms
of the
global
networks
suchnumber
as theofWorld
DOI: 10.1103/PhysRevLett.88.138701 PACS numbers: 89.20.Hh,
89.75.Da,
89.75.Hc
system.84.35.+i,
Itstructure
is therefore
important
to know
the global
the network
as well
as its
precise
Web (WWW) [4,5] and the Internet [6]. Network
is critical
in many
contexts
such asstructure
Internetofattacks
[2], spread
of an
Email
virusdistribution
[7], or of the number of links.
empiricalthestudies
thatlargest
both the
Internet
and play
theThere
WWW
is, the numbertheof structure
incomingand
linksgrowth
and the
dynamics of human epidemics [8]. In all Recent
these problems,
nodesreport
with the
number
of links
an important
role on
theproperties;
dynamics
ofintheunderstanding
is a have
great scale-free
deal
of current
interestthat
mechanisms of global networks [1–3], such as the World Wide
outgoingoflinks
a givenas node
distributions
that (WWW)
decay
law
tails [4–6].
It has been
proposed
[9] thatin the
scale-free
system. It is therefore important to know number
the globalof structure
the atnetwork
well ashave
its precise
distribution
of thewith
number
links.
Web
[4,5]power
andofthe
Internet
[6]. Network
structure
is critical
many
contexts such as Internet attacks [2], spread of an Email virus [7], or
of theandInternet
and thehave
WWW
may beproperties;
explained that
bydynamics
ais,mechanism
referred
to as “preferential
attachment”
whichwith
newthenodes
linknumber of links play an important role on the dynamics of the
empiricalthestudies
report
bothstructure
the Internet
the WWW
scale-free
the number
of epidemics
incoming
links
of human
[8]. Inand
all the
these
problems,[10]
theinnodes
largest
There is a great deal of current interest inRecent
understanding
structure
andthat
growth
mechanisms
of
global
networks
[1–3],
such
as
the
World
Wide
existing
nodes withthat
a probability
proportional
to the[4–6].
number
of existing
links to[9]
these
Here the
we global
focus on
the stochastic
character
of theas its precise distribution of the number of links.
number ofstructure
outgoingis critical
links atina many
given contexts
nodeto have
decay
power
law tails
Itorhas
been
thatnodes.
thetoscale-free
system.
It is proposed
therefore
important
know
structure
of the network
as well
Web (WWW) [4,5] and the Internet [6]. Network
suchdistributions
asattachment
Internet attacks
[2],with
spread
of an
virus [7],
preferential
mechanism,
which
weEmail
understand
in the
following
way:
nodes
toboth
connect
to the existing
largest properties; that is, the number of incoming links and the
of the Internet
the the
WWW
may
be explained
mechanism
referred
“preferential
attachment”
[10]studies
inNew
which
newwant
nodes
linkthe Internet
Recent
empirical
report
that
and the nodes
WWWwith
havethescale-free
dynamics of human epidemics [8]. In all structure
these problems,
the nodesandwith
largest
number
linksbyplaya with
an important
role ontotheasdynamics
of
the
number
oftooflinks—i.e.,
the largest
thenumber
advantages
offered
by
well-connected
node.that
Fordecay
a largewith
network
nodes with
the number
of existing
linksdegree—because
toofthese
nodes. of
Here
we
focusofonoutgoing
the stochastic
ofa the
linksbeing
atcharacter
alinked
given tonode
have distributions
powerit law tails [4–6]. It has been proposed [9] that the scale-free
system. It is therefore important to know totheexisting
global structure
of atheprobability
network asproportional
wellis asnotitsplausible
precise
distribution
of thewill
number
links.
that
a
new
node
know
the
degrees
of
all
existing
nodes,
so
a
new
node
must
make
a
decision
on
which
node
to
connect
with
mechanism,
we understand inthattheis,following
way:ofNew
nodes links
want and
to connect
to ofthetheexisting
the largest
structure
Internetnodes
and with
the WWW
may be explained by a mechanism referred to as “preferential attachment” [10] in which new nodes link
Recent empirical studies report that bothpreferential
the Internetattachment
and the WWW
have which
scale-free
theitnumber
incoming
the The
based properties;
on what of
information
hasoffered
about the
state oflinked
the network.
attachment
mechanism
into play
as nodeslinks
withto athese nodes. Here we focus on the stochastic character of the
of links—i.e.,that
withdecay
the largest
degree—because
theItadvantages
by[9]being
to atowell-connected
a large network
it thento comes
existingpreferential
nodesnode.
with For
a probability
proportional
the number
of existing
number of outgoing links at a given nodenumber
have distributions
with power
lawdegree
tails [4–6].
has
proposedknown.
that the scale-free
are
more
likelybeen
to become
is notbeplausible
node willreferred
knowlarger
the
of all
existing
nodes,
so ainnew
nodenewmust
makelink
a decisionattachment
on which node
to connect
withwe understand in the following way: New nodes want to connect to the existing nodes with the largest
preferential
mechanism,
which
structure of the Internet and the WWW may
explainedthatbyaanew
mechanism
to asdegrees
“preferential
attachment”
[10]
which
nodes
based ontowhat
information
it has about
statenodes.
of the Here
network.
The preferential
attachment
mechanism
comes into with
play the
as nodes
a
of links—i.e.,
largestwith
degree—because
of the advantages offered by being linked to a well-connected node. For a large network it
to existing nodes with a probability proportional
the number
of existing
links tothethese
we focus
on the stochastic
character
of number
the then
degree are more
to become
known.
is not plausible that a new node will know the degrees of all existing nodes, so a new node must make a decision on which node to connect with
preferential attachment mechanism, whichlarger
we understand
in thelikely
following
way: New
nodes want to connect to the existing nodes with the largest
number of links—i.e., with the largest degree—because of the advantages offered by being linked to a well-connected node. For a large networkbased
it on what information it has about the state of the network. The preferential attachment mechanism then comes into play as nodes with a
larger degree are more likely to become known.
is not plausible that a new node will know the degrees of all existing nodes, so a new node must make a decision on which node to connect with
based on what information it has about the state of the network. The preferential attachment mechanism then comes into play as nodes with a
larger degree are more likely to become known.
Target expansion
1 April 2002
Truncation of Power Law Behavior in “Scale-Free” Network Models
due to Information Filtering
1 April 2002
Truncation of Power Law Behavior in “Scale-Free” Network Models
due to Information Filtering
Network of publications (nodes)
linked by citations (edges)
Lower citation-density
Higher citation-density
e.g., applied research,
social sciences
e.g., basic natural
medical research
CPP
FCSm
JCSm
Expected values for normalization
Absolutely necessary but……are they
appropriate?
CWTS applies two types of field definitions:
Journal
Field = set of journals
‘established fields’
scientific medium-grained structure
+ reference-based re-definition
(expansion) of fields
Main field: Social and Behavioral Sciences
Major field, e.g. Economics & Business
SOCIAL AND BEHAVIORAL SCIENCES
ECONOMICS AND BUSINESS
APPL PREV PSYCHOL
APPL PSYCHOL-INT REV
BEHAV SCI LAW
EDUCATIONAL SCIENCES
BRIT J GUID COUNS
CAREER DEV Q
COUNS MANAGEMENT
PSYCHOL AND PLANNING
CYBERPSYCHOL BEHAV
ERGONOMICS
POLITICAL SCIENCE AND
EUR J PSYCHOL
ASSESS
PUBLIC ADMINISTRATION
EUR J WORK ORGAN PSY
GROUP PSYCHOLOGY
ORGAN MANAGE
HUM FACTORS
HUM PERFORM
HUM RESOUR MANAGE
INT J AVIAT PSYCHOL
INT J OFFENDER
SOCIAL AND THER
BEHAVIORAL
SCIENCES, INTERDISCIPLINARY
INT J SELECT ASSESS
J APPL PSYCHOL
J APPL SPORT PSYCHOL
SOCIOLOGY AND ANTHROPOLOGY
J BEHAV DECIS MAKING
AGRICULTURAL ECONOMICS & POLICY
BUSINESS
BUSINESS, FINANCE
ECONOMICS
INDUSTRIAL RELATIONS & LABOR
EDUCATION & EDUCATIONAL RESEARCH
EDUCATION, SCIENTIFIC DISCIPLINES
EDUCATION, SPECIAL
PSYCHOLOGY, EDUCATIONAL
All publication titles + abstracts (~30,000,000)
journals
havefields
been grammatically parsed to enable
bibliometric analysis by themes/concepts/
instruments and to create word-correlation based
maps of science
AREA STUDIES
MANAGEMENT
PLANNING & DEVELOPMENT
ACTA GENETICAE MEDICAE ET GEMELLOLOGIAE
AMERICAN JOURNAL OF HUMAN GENETICS
AMERICAN JOURNAL OF MEDICAL GENETICS
ANIMAL BLOOD GROUPS AND BIOCHEMICAL GENETICS
ANNALES DE GENETIQUE
ANNALES DE GENETIQUE ET DE SELECTION ANIMALE
ANNALS OF HUMAN GENETICS
ATTI ASSOCIAZIONE GENETICA ITALIANA
BEHAVIOR GENETICS
BIOCHEMICAL GENETICS
CANADIAN JOURNAL OF GENETICS AND CYTOLOGY
CANCER GENETICS AND CYTOGENETICS
CARYOLOGIA
CHROMOSOMA
CLINICAL GENETICS
CURRENT GENETICS
CYTOGENETICS AND CELL GENETICS
CYTOLOGIA
DEVELOPMENTAL GENETICS
ENVIRONMENTAL MUTAGENESIS
EVOLUTION
GENE
GENETICA
GENETICA POLONICA
GENETICAL RESEARCH
GENETICS
GENETIKA
HEREDITAS
HEREDITY
HUMAN BIOLOGY
HUMAN GENETICS
HUMAN HEREDITY
IMMUNOGENETICS
INDIAN JOURNAL OF GENETICS AND PLANT BREEDING
JAPANESE JOURNAL OF GENETICS
JAPANESE JOURNAL OF HUMAN GENETICS
JOURNAL DE GENETIQUE HUMAINE
JOURNAL OF HEREDITY
JOURNAL OF IMMUNOGENETICS
JOURNAL OF MEDICAL GENETICS
JOURNAL OF MENTAL DEFICIENCY RESEARCH
JOURNAL OF MOLECULAR EVOLUTION
MOLECULAR & GENERAL GENETICS
MUTATION RESEARCH
PLASMID
SILVAE GENETICA
THEORETICAL AND APPLIED GENETICS
THEORETICAL POPULATION BIOLOGY
EGYPTIAN JOURNAL OF GENETICS AND CYTOLOGY
REVISTA BRASILEIRA DE GENETICA
ANNUAL REVIEW OF GENETICS
JOURNAL OF CRANIOFACIAL GENETICS AND DEVELOPMENTAL BIOLOGY
JOURNAL OF INHERITED METABOLIC DISEASE
PRENATAL DIAGNOSIS
ADVANCES IN GENETICS INCORPORATING MOLECULAR GENETIC MEDICINE
CHEMICAL MUTAGENS-PRINCIPLES AND METHODS FOR THEIR DETECTION
DNA-A JOURNAL OF MOLECULAR & CELLULAR BIOLOGY
EVOLUTIONARY BIOLOGY
TERATOGENESIS CARCINOGENESIS AND MUTAGENESIS
TSITOLOGIYA I GENETIKA
ADVANCES IN HUMAN GENETICS
PROGRESS IN MEDICAL GENETICS
GENETICS SELECTION EVOLUTION
MOLECULAR BIOLOGY AND EVOLUTION
SOMATIC CELL AND MOLECULAR GENETICS
BIOTECHNOLOGY & GENETIC ENGINEERING REVIEWS
EXPERIMENTAL AND CLINICAL IMMUNOGENETICS
GENE ANALYSIS TECHNIQUES
JOURNAL OF MOLECULAR AND APPLIED GENETICS
JOURNAL OF NEUROGENETICS
TRENDS IN GENETICS
DISEASE MARKERS
ANIMAL GENETICS
GENETIC EPIDEMIOLOGY
JOURNAL OF GENETICS
INTERNATIONAL RELATIONS
POLITICAL SCIENCE
PUBLIC ADMINISTRATION
PSYCHOLOGY,
PSYCHOLOGY,
PSYCHOLOGY,
PSYCHOLOGY,
PSYCHOLOGY,
PSYCHOLOGY,
PSYCHOLOGY,
PSYCHOLOGY,
PSYCHOLOGY,
APPLIED
BIOLOGICAL
CLINICAL
DEVELOPMENTAL
EXPERIMENTAL
MATHEMATICAL
MULTIDISCIPLINARY
PSYCHOANALYSIS
SOCIAL
DEMOGRAPHY
SOCIAL ISSUES
SOCIAL SCIENCES, BIOMEDICAL
SOCIAL SCIENCES, INTERDISCIPLINARY
ANTHROPOLOGY
ETHNIC STUDIES
FAMILY STUDIES
SOCIOLOGY
WOMEN'S STUDIES
cluster
Field = clusters of
concept-related
publications
new, emerging often interdisc. fields
scientific fine-grained structure
Social Sciences
Top-50 EU universities, their top-10% publications in this field
Now specific sub-field CPP/FCSm values can be calculated,
for instance for research on democracy
But, obviously, the finer grained, the more ‘noisy’
Basic Performance Indicators
•P
Ouput: Number of publications in internationally
refereed CI-covered journals
•C
Absolute Impact: Number of (self-ex) citations
to these publications
•H
Hirsch-index
• CPP
Output-normalized Impact: Average number of
cits/pub of the institute
• JCSm Average number of cits/pub of the journal set
used by the institute
• FCSm Average number of cits/pub of all journals of a
specific field in which the institute is active
(FCSm)
• p0
Percentage of not-cited publications
CWTS Key Research Performance Indicators:
• JCSm/FCSm Relative impact of the used journal set
• CPP/JCSm
Internat. journal-normalized impact
• CPP/FCSm
Internat. field & doc-normalized impact
• Pt/Πt
Contribution to the top-5, 10, 20,..%
• P*CPP/FCSm Size & Impact Together: Brute Force
Basic research
High CPP
low FCSm, but
high JCSm
low FCSm
Applied research,
engineering
high FCSm
high FCSm, but
low JCSm
low CPP
Up to factor ~20
Internal WoS-coverage of social
science fields
results from HEFCE and Benchmark projects
Table 3.1:Internal
coverage
percentages ofofthemain
Thomsonfields
Scientific/ISI
Internal
WoS
coverage
of Citation
Indexes
science
80-100%
Biochem & Mol Biol
Biol Sci – Humans
Chemistry
Clinical Medicine
Phys & Astron
Internal Coverage Percentage
60-80%
40-60%
Appl Phys & Chem
Mathematics
Biol Sci – Anim & Plants
Economics
Psychol & Psychiat
Engineering
Geosciences
Soc Sci ~ Medicine
<40%
Other Soc Sci
Humanities & Arts
What is the internal WoS coverage and
how is it calculated?
Example: EUR 2000-2004
major field
CLINICAL MEDICINE
BIOL SCI: HUMANS
BIOL SCI: ANIMALS & PLANTS
MOLECULAR BIOLOGY & BIOCHEM
PHYSICS AND ASTRONOMY
CHEMISTRY
MATHEMATICS
GEOSCIENCES
APPLIED PHYSICS AND CHEMISTRY
ENGINEERING
MULTIDISCIPLINARY
ECONOMICS
PSYCHOL, PSYCHIATRY & BEHAV SC
SOCIAL SC RELATED TO MEDICINE
OTHER SOCIAL SCIENCES
HUMANITIES & ARTS
P 00-04 Avg Nr Refs
5.376
2.597
124
671
5
45
119
33
177
313
61
471
341
573
251
80
31,18
38,18
36,28
42,82
19,40
26,51
27,33
32,42
25,58
25,97
30,33
36,56
36,76
31,70
33,73
42,53
Refs<1980 %Refs<1980 Refs Non-CI
8.138
4.117
333
915
29
74
445
66
375
727
101
1.961
975
956
789
846
5%
4%
7%
3%
30%
6%
14%
6%
8%
9%
5%
11%
8%
5%
9%
25%
15.989
7.484
659
1.559
15
143
1.019
521
864
3.210
150
7.510
2.983
4.836
4.185
1.889
Refs CI %Refs CI
143.516
87.546
3.507
26.255
53
976
1.788
483
3.289
4.193
1.599
7.748
8.578
12.374
3.491
667
90%
92%
84%
94%
78%
87%
64%
48%
79%
57%
91%
51%
74%
72%
45%
26%
TABLE 1:
SHARE OF UNL REFERENCES TO WEB OF SCIENCE PAPERS
Main Field
CLINICAL MEDICINE
BIOL SCI: HUMANS
BIOL SCI: ANIMALS & PLANTS
MOLECULAR BIOLOGY & BIOCHEM
PHYSICS AND ASTRONOMY
CHEMISTRY
MATHEMATICS
GEOSCIENCES
APPLIED PHYSICS AND CHEMISTRY
ENGINEERING
MULTIDISCIPLINARY
ECONOMICS
PSYCHOLOGY, PSYCHIATRY & BEHAV SC
SOCIAL SCIENCES RELATED TO MEDICINE
OTHER SOCIAL SCIENCES
HUMANITIES & ARTS
P
00-06
329
495
318
656
281
796
115
143
723
447
19
124
10
43
60
37
Avg Nr
Refs
33.6
35.2
36.9
39.2
25.0
37.8
18.8
29.9
18.8
19.3
27.2
31.3
28.8
29.8
27.2
57.6
%Refs
<1980
5%
7%
12%
7%
17%
12%
16%
10%
11%
8%
9%
11%
7%
5%
12%
36%
Nr. Refs
>1979
10,461
16,199
10,347
23,950
5,793
26,549
1,824
3,842
12,040
7,951
470
3,468
267
1,215
1,439
1,365
%Refs
CI
87%
88%
77%
90%
82%
86%
57%
61%
76%
45%
85%
55%
45%
74%
47%
17%
Internal WoS coverage (%) of submitted publications per UoA
From: Moed, Visser, Buter, 2008
SS&H
SS&H
SS&H
SS&H
SS&H
SS&H
SS&H
SS&H
SS&H
SS&H
SS&H
SS&H
SS&H
SS&H
SS&H
SS&H
SS&H
Sports-related Subjects
Economics and Econometrics
Accounting and Finance
Geography
Business and Management Studies
Anthropology
Library and Information Management
Built Environment
Archaeology
Linguistics
Art and Design
Social Work
Sociology
Education
Social Policy and Administration
Town and Country Planning
Politics and International Studies
698
1,688
164
2,697
3,205
299
366
560
327
199
194
344
933
1,230
948
520
1,040
65
51
40
39
36
35
35
34
31
30
28
27
24
24
24
23
18
Table 3.3: Estimated CI-coverage (1991-2006), based on the extent to which
references within CI-covered publications are also CI-covered
Internal WoS coverage for all main fields of science
Field
1991
1996
2001
2006
MEDICAL & LIFE SCIENCES
AGRICULTURE AND FOOD SCIENCE
BASIC LIFE SCIENCES
BASIC MEDICAL SCIENCES
BIOLOGICAL SCIENCES
BIOMEDICAL SCIENCES
CLINICAL MEDICINE
HEALTH SCIENCES
66%
87%
76%
72%
86%
82%
50%
66%
89%
75%
74%
87%
82%
47%
73%
93%
80%
80%
90%
85%
57%
75%
93%
84%
82%
90%
85%
62%
NATURAL SCIENCES
ASTRONOMY AND ASTROPHYSICS
CHEMISTRY AND CHEMICAL ENGINEERING
COMPUTER SCIENCES
EARTH SCIENCES AND TECHNOLOGY
ENVIRONMENTAL SCIENCES AND TECHNOLOGY
MATHEMATICS
PHYSICS AND MATERIALS SCIENCE
STATISTICAL SCIENCES
75%
77%
38%
60%
46%
58%
75%
49%
79%
80%
37%
60%
46%
57%
78%
46%
82%
86%
42%
69%
55%
58%
81%
52%
86%
88%
43%
74%
62%
64%
84%
58%
37%
54%
32
54%
26
42%
67%
58%
33%
52%
28
48%
33
37%
62%
53%
34%
52%
29
53%
43
44%
71%
57%
45%
53%
32
59%
40
54%
69%
64%
32%
26%
28%
33%
29%
43%
32%
40%
35
27
35%
27%
23
23%
17
17%
59
59%
33%
33
22%
22
36
31
36%
31%
24
24%
18
18%
59
59%
34%
34
27%
27
35
30
35%
30%
27
27%
20
20%
66
66%
36%
36
29%
29
43
36
43%
36%
36
36%
24
24%
72
72%
40%
40
34%
34
LAW, ARTS AND HUMANITIES
CREATIVE ARTS, CULTURE AND MUSIC
HISTORY, PHILOSOPHY AND RELIGION
LAW AND CRIMINOLOGY
LITERATURE
17%
24%
27%
14%
14%
23%
32%
12%
16%
25%
32%
11%
14%
27%
31%
11%
MULTIDISCIPLINARY JOURNALS
78%
83%
87%
87%
LANGUAGE,
INFORMATION AND COMMUNICATION
ENGINEERING SCIENCES
CIVIL ENGINEERING AND CONSTRUCTION
ELECTRICAL ENGINEERING
AND TELECOMMUNICATION
INFORMATION
AND COMMUNIC
SCIENCES
ENERGY SCIENCE AND TECHNOLOGY
LANGUAGE
AND
LINGUISTICS
GENERAL AND INDUSTRIAL ENGINEERING
INSTRUMENTS AND INSTRUMENTATION
MECHANICAL ENGINEERING AND AEROSPACE
LANGUAGE, INFORMATION AND COMMUNICATION
SOCIAL
AND BEHAVIORAL SCIENCES
INFORMATION AND COMMUNICATION SCIENCES
LANGUAGE AND LINGUISTICS
ECONOMICS AND BUSINESS
SOCIAL AND BEHAVIORAL SCIENCES
EDUCATIONAL
ECONOMICS ANDSCIENCES
BUSINESS
EDUCATIONAL SCIENCES
MANAGEMENT
AND PLANNING
MANAGEMENT AND PLANNING
POLITICAL
SCIENCE AND PUBLIC ADMIN
POLITICAL SCIENCE AND PUBLIC ADMINISTRATION
PSYCHOLOGY
PSYCHOLOGY
SOCIAL AND
AND BEHAVIORAL
SCIENCES, INTERDISCIPLINARY
SOCIAL
BEHAV SCIENCES,
INTERDISC
SOCIOLOGY AND ANTHROPOLOGY
SOCIOLOGY AND ANTHROPOLOGY
1991-2006
purple: non-WoS ref
light blue: CI ref
100%
References non-ISI
References ISI
90%
80%
70%
60%
50%
40%
30%
20%
10%
AGRICULTURE
AND FOOD
SCIENCE
BASIC LIFE
SCIENCES
BASIC MEDICAL
SCIENCES
BIOLOGICAL
SCIENCES
BIOMEDICAL
SCIENCES
CLINICAL
MEDICINE
HEALTH
SCIENCES
2006
2001
1996
1991
2006
2001
1996
1991
2006
2001
1996
1991
2006
2001
1996
1991
2006
2001
1996
1991
2006
2001
1996
1991
2006
2001
1996
1991
0%
1991-2006
purple: non-WoS ref
light blue: CI ref
100%
References non-ISI
References ISI
90%
80%
70%
60%
50%
40%
30%
20%
10%
ECONOMICS AND
BUSINESS
EDUCATIONAL
SCIENCES
MANAGEMENT
AND PLANNING
POLITICAL
SCIENCE AND
PUBLIC
ADMINISTRATION
PSYCHOLOGY
2006
2001
1996
1991
2006
2001
1996
1991
2006
2001
1996
1991
2006
2001
1996
1991
2006
2001
1996
1991
2006
2001
1996
1991
2006
2001
1996
1991
0%
SOCIAL AND
SOCIOLOGY AND
BEHAVIORAL
ANTHROPOLOGY
SCIENCES,
INTERDISCIPLINARY
External WoS-coverage of social
science fields
results from HEFCE and Evaluation projects
TABLE 2: EXTERNAL COVERAGE FOR UPPSALA PAPERS 2002 - 2006
What is the external WoS coverage and
how is it calculated?
Research Unit
Total papers
Total
Journal Journal
(in journals,
paper
Wos papers Papers Coverage procs, books) Coverage
Example: Uppsala
2002-2006
Arts
61
UPPSALA
Centre for Gender Research
Centre for Multiethnic Research
Dep of ALM (Archives, Libraries, Museums)
Social
Sciences
Dep
of Archeology
and Ancient History
Dep
of
Art
History
Dep of Business Studies
Dep of Cultural Anthropology and Ethnology
Dep of Domestic
Dep ofSciences
History
of History of Science
and Ideas
DepDep
of Economic
History
Dep of Literature
Dep of Dep
Economics
of Musicology
Dep
of Philosophy
Dep of Education
The Uppsala Progr for Holocaust and Genocide Studies
Dep of Eurasian Studies
Social Sciences
Dep of Government
Biology
Dep of Bioorganic Chemistry
Dep ofDep
Information
Science
of
Cell and Molecular Biology
Biology
Depof of
Lawand Evolution
Dep
Ecology
of Evolution,
GenomicsResearch
and Systematics
Dep ofDep
Peace
and Conflict
Dep of Physiology and Developmental Biology
DepLinnaeus
of Psychology
The
Centre for Bioinformatics
Dep of Social and Economic Geography
Chemistry
Dep
of
Sociology
Dep of Biochemistry and Organic Chemistry
Dep and
of Materials
Chemistry
Inst for Housing
Urban
Research
Dep of Photo Chemistry and Molecular Science
Dep of Physical and Analytical Chemistry
8.502
11.403
75%
16.436
52%
567
17
37
1
12
3
20
5 380 67
1
21 12
6
94
25 72
9
5
0 77
Wos
papers
3
140
41 7
1
10
6 29
1
4
11%
46%
8%
15%
1.083
7%
8%
70
6%
59
Journal
13%
6%
52
Papers
2%
93
14%
34%
22
25%
1.089
77
43
43
35%
142
34
30%
151
42%
Journal
152
120
0%
Coverage
242
44%
18
67
27%
9
6%
22%
2%
7%
4% 2.476
3% 316
Total
papers
4%
99
(in
6% journals,
4% 133
procs,
books)
1%
131
6%
15% 102
11%
380
1.083
35%
2.476
9
1.265 27 1.383
32
31 39
322 1.265 345
5 419
390
275 12 307
234
260
44 149 45
12
911
951
15
201
216
316 31 328
139
75
91%
82%
48
93%
1.383
164
93%
90%
34
90%
186
98%
23
96%
54
93%
96%
74
98%
97%
6%
36%
1.528
39
65%
366
91%
3%
441
353
35%
320
80%
51
52%
1.072
28%
223
410
42%
69
422
211
83% 180
82%
109
88%1.528
88% 307
78% 156
73%
86% 281
58
85%
90% 170
77% 242
65
372
66
384
94%
88%
Earth Sciences
340
433
79%
711
48%
Educational Sciences
8
60
13%
174
5%
15%
7%
Total
25%
paper
0%
Coverage
31%
6%
4%
15%
15%
28%
83%
2%
8%
53%
21%
9%
13%
Candidates for Spinoza Award 2006, M€ 1.5
Field
From:
Van
Leeuwen
2006
Amerindian Languages
Medical Biochemistry
Computer Science
General Linguistics
Clinical Psychology
Theoretical High Energy Physics
Neuroscience
Entomology
Cell Biophysics/Tumor Immunology
Strategy & International Business
Biology
Zoology
Polymer Technology
Public International Law
Social Psychiatry
Experimental Physics
Molecular Genetics
Soft Condensed Matter
Economics
Bio-Chemisal Analysis Systems
Epidemiology
Psychiatry and Addiction
Child Neurology
Stochastics
Genetic Epidemiology
Environmental Biotechnology
Theoretical Physics
P total
100
130
111
108
175
178
105
134
225
69
180
115
151
108
140
290
52
78
269
102
222
326
121
63
263
200
113
Books/
proceed
31
22
11
3
1
32
31
13
17
6
30
9
5
3
34
11
24
65
10
7
7
4
11
Total
WoS
Ext WoS
Coverage
3
97
37
30
72
131
97
110
193
27
134
91
121
11
136
228
38
66
23
90
150
108
107
44
226
188
90
3%
75%
33%
28%
41%
74%
92%
82%
86%
39%
74%
79%
80%
10%
97%
79%
73%
85%
9%
88%
68%
33%
88%
70%
86%
94%
80%
84% of the total number of publications submitted to the 2001 RAE
from science-related departments were published in WoS-covered
journals.
For Mathematics publications the WoS coverage is only slightly lower
(82%),
It is substantially lower for Social Sciences and Humanities (25%)
From: Moed, Visser, Buter, 2008
WoS coverage of submitted publications per main field and per
subject group
Field/subject group*
Total
Nr.
Papers
%
Papers
in
WoS
Total
Papers
in Jrnls
%
Papers
in Jrnls
% WoS
papers in
set of Jrnl
Papers
Per main field
Science
Mathematics
Soc Sci + Humanities
95,056
6,634
91,324
84.1
81.8
24.9
87,712
6,105
47,972
92.3
92.0
52.5
91.1
88.9
47.4
Per subject group*
Clinical Medicine*
Health Sciences*
Subjects allied to Health*
Biological Sciences*
Physical Sciences
16,541
10,621
10,203
16,694
17,190
96.6
85.0
73.7
93.5
88.0
16,412
10,078
9,408
16,252
16,462
99.2
94.9
92.2
97.4
95.8
97.3
89.6
79.9
96.0
91.9
Engineering & Computer
Science
23,807
70.1
19,100
80.2
87.4
External WoS coverage (%) of submitted publications per UoA
SS&H
Econ and Econometrics
2,879
67.5
2,482
86.2
78.3
SS&H
SS&H
SS&H
SS&H
SS&H
SS&H
SS&H
SS&H
SS&H
SS&H
Geography
Sports-related Subjects
Town & Country Planning
Business & Management St
Library & Inform Manag
Sociology
Anthropology
Politics and Internat St
Social Policy & Administr
Built Environment
4,890
1,301
1,478
9,746
1,259
3,519
1,180
4,382
3,912
2,471
61.6
60.5
38.1
37.9
31.7
29.2
27.5
26.4
25.8
24.5
3,863
1,099
1,083
7,977
743
1,846
523
2,240
1,937
1,532
79.0
84.5
73.3
81.8
59.0
52.5
44.3
51.1
49.5
62.0
78.0
71.6
52.0
46.3
53.7
55.6
62.1
51.7
52.1
39.5
SS&H
SS&H
Social Work
Accounting and Finance
1,642
779
22.9
21.7
887
664
54.0
85.2
42.4
25.5
SS&H
SS&H
American Studies
Education
475
8,662
19.8
16.0
189
4,805
39.8
55.5
49.7
28.8
100%
y = 0.0087x
2
R = 0.9068
80%
Internal WoS Coverage
What is the correlation between internal and
60%
external WoS coverage?
40%
20%
0%
0
10
20
30
40
50
60
External WoS Coverage (%)
70
80
90
100
Characteristics of WoS publications in
social science fields
results from HEFCE and Benchmark & Evaluation projects
PH Y SIC S
time lag & citation window
35000
30000
25000
20000
15000
10000
5000
0
Cits91
Publications from 1991,….1995
Cits92
Cits93
Cits94
Cits95
Cits96
Cits97
Cits98
Cits99
Cits00
Cits01
Cits02
* SO C IO L O G Y
2500
2000
1500
1000
Main differences with the natural and
medical sciences:
*Lower numbers (more than 1 order of
magnitude….)
*Slower rise , broader peak and much slower decay
(less hectics…)
500
0
Cits91
Cits92
Cits93
Cits94
Cits95
Cits96
Cits97
Cits98
Cits99
Cits00
Cits01
Cits02
BIBLIOMETRIC INDICATORS FOR MAJOR FIELDS
University
P
Rank
C
Rank
CPP/
Rank
FCSm
P*CPP/
Rank
FCSm
CLINICAL MEDICINE
ERASMUS UNIV ROTTERDAM
5.376
23
77.806
21
1,56
53
8.413
22
LEIDEN UNIV
RADBOUD UNIV NIJMEGEN
UNIV AMSTERDAM
UNIV GRONINGEN
UNIV MAASTRICHT
UNIV TILBURG
UNIV UTRECHT
VRIJE UNIV AMSTERDAM
3.904
4.205
4.622
3.021
3.120
59
4.757
3.682
57
47
37
96
90
34
63
50.276
46.421
59.862
32.463
39.274
629
55.591
47.934
58
63
38
105
82
47
61
1,36
1,24
1,42
1,23
1,39
1,28
1,32
1,46
97
148
78
156
85
114
74
5.298
5.213
6.573
3.718
4.343
75
6.280
5.360
62
64
40
102
81
44
59
ERASMUS UNIV ROTTERDAM
471
20
1.946
34
1,18
118
558
33
LEIDEN UNIV
RADBOUD UNIV NIJMEGEN
UNIV AMSTERDAM
UNIV GRONINGEN
UNIV MAASTRICHT
UNIV TILBURG
UNIV UTRECHT
VRIJE UNIV AMSTERDAM
37
78
330
237
289
430
104
302
246
187
41
67
49
25
160
45
227
276
1.647
863
1.020
1.684
283
995
190
179
45
93
79
42
177
81
1,52
0,92
1,47
0,95
1,02
1,18
0,80
0,90
74
174
83
165
146
117
210
182
56
72
486
226
296
509
83
271
201
187
48
98
77
43
177
83
ECONOMICS
HIGHLY CITED PAPERS FOR MAJOR FIELDS
University
P01-03
Ptop20% Rank
Ptop10% Rank
Ptop5% Rank
Ptop1% Rank
ERASMUS UNIV ROTTERDAM
2.929
851
23
476
22
254
25
71
21
LEIDEN UNIV
RADBOUD UNIV NIJMEGEN
UNIV AMSTERDAM
UNIV GRONINGEN
UNIV MAASTRICHT
UNIV TILBURG
UNIV UTRECHT
VRIJE UNIV AMSTERDAM
2.152
2.318
2.611
1.640
1.805
29
2.642
2.057
574
598
745
398
487
10
738
600
62
58
35
103
78
37
57
313
309
398
192
267
3
382
323
60
62
36
109
77
42
58
150
164
212
98
134
2
217
167
72
63
39
114
78
38
61
27
24
44
15
29
0
32
32
90
103
48
150
83
72
72
ERASMUS UNIV ROTTERDAM
282
68
34
29
49
9
75
1
83
LEIDEN UNIV
RADBOUD UNIV NIJMEGEN
UNIV AMSTERDAM
UNIV GRONINGEN
UNIV MAASTRICHT
UNIV TILBURG
UNIV UTRECHT
VRIJE UNIV AMSTERDAM
17
54
193
147
166
278
66
186
4
9
54
24
26
68
5
34
211
171
45
97
90
34
202
71
1
4
30
9
17
34
2
15
221
168
46
114
80
41
199
89
1
3
16
6
6
17
0
8
178
133
46
96
96
41
227
83
0
0
1
1
1
1
0
0
133
133
83
83
83
83
133
133
CLINICAL MEDICINE
ECONOMICS
IMPACT COMPARED TO WORLD SUBFIELD AVERAGE
2000 - 2006
ECONOMICS
3,00
DUKE
2,50
NORTHWESTERN
VIRGINIA
2,00
CPP/FCSm
PITTSBURGH
KINGS COLL
LEIDEN
1,50
OXFORD
UvA
LAUSANNE
JOHNS HOPKINS
MCGILL
EUR
UM
MANCHESTER
RUG
RU
TORINO
BASEL
UvT
LEUVEN
IMP COLL
1,00
LSE
CAMBRIDGE
VU
GENT
UTRECHT
BARCELONA
MILANO
GENEVE
0,50
NAPOLI
0,00
0
100
200
300
400
500
TOTAL PUBLICATIONS
Black coloured squares above (below) the horizontal reference line represent groups
for which the impact (CPP) is significantly above (below) world average (FCSm)
600
EUR 200-2006 Benchmark Study
P
%
173,4
86,7
78,7
26,6
2,5
47%
24%
21%
7%
1%
C
CPP/
FCSm
ECONOMICS
ECONOMICS
MANAGEMENT
BUSINESS
BUSINESS, FINAN
AGRIC ECON&POL
507,0
431,2
384,2
80,8
6,5
0,90
1,11
1,41
1,23
0,96
TABLE 6: HIGHLY CITED PAPERS FOR THE UNIVERSITY AND INSTITUTES
Research Unit
P 0203 Ptop 5% E(Ptop 5%) A/E(Ptop 5%)
Biology
Chemistry
Earth Sciences
Engineering
Mathematics and Computer Science
Medicine
Pharmacy
Physics
Social Sciences
438
344
104
186
181
1.432
308
368
151
30
35
9
20
16
95
17
21
10
21,8
17,2
5,3
9,3
8,9
71,8
15,5
18,2
7,5
1,37
2,03
1,69
2,15
1,80
1,32
1,10
1,15
1,33
Uppsala University
3.190
228
159,5
1,43
RESEARCH AND IMPACT PROFILE
COMPARISON CHART 2000 - 2005
LEIDEN UNIV
PSYCHIATRY
PSYCHOLOGY, MULT
* PSYCHOLOGY, SO
PUBL ENV OCC HLT
* PSYCHOLOGY, EX
* PSYCHOLOGY, CL
UNIV ZURICH
1,028331892
1,161701811
0,8652083
0,972944083
0,845662773
0,925778809
0,93637077
0,860002345
0,937883234
0,599794868
1,159763899
0,461296324
* PSYCHOLOGY, DE
0,643706864
* ANTHROPOLOGY
0,766388575
* EDUCATION *ED
1,742351882
# ASIAN STUDIES
1,223234261
* POLITICAL SC
1,813498253
2,181555537
# LANGUAGE&LING
1,710102432
0,361855648
* PSYCHOLOGY, ED
0,714722537
BEHAVIORAL SC
1,058451869
REHABILITATION
0,467699105
2,911953195
# HISTORY
0,654775446
0,941907378
0,674202907
1,144437183
0,317647587
0,873929771
0,127077691
1,251511155
* INFORM SC&LIBR
3,549813304
HEALTH CARE SC&S
1,498792549
* PUBLIC ADMIN
0,89885929
SPORT SCIENCES
1,437596678
* PSYCHOLOGY, AP
1,124854594
0,7734368
* LANGUAGE&LING
0,320689302
0,816961813
# RELIGION
1,576815757
* AREA STUDIES
0,484413159
# PHILOSOPHY
0,835714364
1,046249014
* SOC SCI,INTERD
1,701706725
0,988357391
3,015952381
0,569773338
1,452590319
1,207450867
0,727394586
Characteristics of non-WoS publications in
social science fields
results from HEFCE and Benchmark & Evaluation projects
P
Publication Type
WoS journal article
Authored book
Chapter in book
Conf contribution
n-WoS journ article
840
13
44
29
93
Chemistry
WoS journal article
Authored book
Chapter in book
Conf contribution
n-Wos journ article
1,281
2
7
1
9
Physics
WoS journal article
Authored book
Chapter in book
Conf contribution
n-Wos journ article
1,352
18
12
31
8
Unit of Assessment
Psychology
p0(%)
9
46
70
97
85
6
50
57
100
78
6
44
75
55
50
7155
81
54
1
68
C/P
8.5
6.2
1.2
0.0
0.7
16076
9
19
0
12
12.5
4.5
2.7
0.0
1.3
19947
340
17
120
9
14.8
18.9
1.4
3.9
1.1
C
Top-10% (of impact) of EU publications in
Political Science, Economics, and Psychology
1997-2003, 4-y citation window (to calculate
their impact)
From references all WoS-references removed,
only non-WoS references (with freq > 2) have been
analyzed
Total about 28,000
From: Nederhof, van Leeuwen, van der Wurff 2008
Percentage of references dating prior to 1980
70%
60%
From these:
50%
% <1980
40%
% <1980 articles
% <1980 books
30%
% <1980 handbooks
20%
% <1980 theses
10%
0%
Political
science
Economics
Psychology
Books and journals dominant among post-1980
references to non-WoS items
100%
90%
% Books
80%
% Journals
70%
60%
50%
40%
30%
20%
10%
0%
Political science
Economics
Psychology
Items with small frequencies among post-1980 references to
non-WoS items
(>1979)
1980)
% total items (>
6%
5%
4%
3%
2%
1%
0%
Political science
% Handbook
Economics
% Thesis
% Proc
Psychology
% Reports
% Working P
Top-50
non-WoS
>1980
references
by document
type
Top-ranked
publications
according
to document
type
Books
J.Articles
Chapters
Manuals
Handbook
Edit.vol
Unknown
Political sc
84%
7%
0%
0%
0%
5%
5%
Economics
89%
3%
0%
0%
3%
0%
6%
Psychology
68%
3%
3%
24%
0%
0%
3%
Field
Bibliometric results and peer
judgments
results from HEFCE and Benchmark & Evaluation projects
Qualitative versus quantitative assessment
peer review reputation may have strong influence
includes 'tacit knowledge' (e.g., instrument building)
includes credits: expectations, we believe that…, ahead of time…
takes products other than journals papers into account
fashion and hypes perhaps less influential
bibliometric reputation much less influential
analysis
only 'codified knowledge'
no credits: only past performance, evidence-based
products other than journal papers less important
fashion and hypes perhaps more influential on the short term
TABLE 5: BIBLIOMETRIC STATISTICS FOR DEPARTMENTS AND INSTITUTES 2002 - 2006
CPP/ CPP/ JCSm/ Self
JCSm FCSm FCSm Cit
Department
P
C
C+sc
CPP
Pnc
Biology
1.265
8.174
10.814
6,46
28%
1,01
1,36
1,35
24%
Dep of Bioorganic Chemistry
Dep of Cell and Molecular Biology
Dep of Ecology and Evolution
Dep of Evolution, Genomics and Systematics
Dep of Physiology and Developmental Biology
The Linnaeus Centre for Bioinformatics
32
322
390
275
234
44
82
2.183
2.163
1.741
1.909
271
170
2.868
2.835
2.268
2.565
372
2,56
6,78
5,55
6,33
8,16
6,16
34%
21%
31%
32%
24%
36%
0,36
0,92
1,00
1,05
1,12
1,15
0,55
1,22
1,42
1,36
1,50
1,67
1,54
1,32
1,41
1,29
1,33
1,45
52%
24%
24%
23%
26%
27%
Chemistry
911
4.603
6.311
5,05
35%
1,08
1,35
1,25
27%
201
316
65
372
951
1.549
1.228
1.229
1.815
432
3.058
Dep of Biochemistry and Organic Chemistry
Social Sciences
380
Dep of Materials Chemistry
Dep of Photo Chemistry and Molecular Science
Dep of Physical and Analytical Chemistry
292
2.266
Dep of Business Studies Earth Sciences
21 340 838
44
Dep of Domestic SciencesEngineering
25 622 117
1.847
Dep of Economics
41
75
Mathematics and Computer Science
453
843
Dep of Government
27
54
Centre for Image Analysis
22
59
of Information Technology
589
Dep of Information Dep
Science
31 265 347
Dep of Mathematics
167
195
Dep of Psychology
149
549
Medicine
3.556 24.034
Inst for Housing and Urban Research
31
67
30%
1.9484,73
3,89 4,08
42%
4,49
6,09
35%
31%
522,46 2,10
46%
1562,97 4,68
2.763
49%
88 1,83
1.240
1,86 55%
73 2,00
81
2,68 32%
868 411
2,22 11,19
54%
291
1,17 59%
737 3,68
30.552 6,76 28%
89 2,16
1.316
0,99
1,09 23%
43%
1,26
1,44 1,25
1,30 32%
0,91
1,11
1,09
1,15
1,41
1,53
62%
0,94
24%
1,07 1,35
49%
1,17 1,11
52%
0,84 0,71
1,2235%
1,24
1,13 0,97
42%
1,08 1,22
35%
1,07
1,29
1,34
0,80
0,88
0,99
1,26
0,64
0,95
1,44
0,85
1,01
3,61
0,86
0,96
1,13
0,92
32%
26%
1,23
0,79
33%
0,85
32%
1,13
27%
32%
2,40
33%
0,99
21%
1,16
36%
Dep of Genetics and Pathology
Dep of Medical Biochemistry and Microbiology
Dep of Medical Cell Biology
Dep of Medical Sciences
Dep of Neuroscience
Dep of Oncology, Radiology and Clinical Immunology
Dep of Public Heatlh and Caring Sciences
Dep of Surgical Sciences
Dep of Women?s and Children?s Health
535
509
253
975
490
524
372
443
196
4.579
3.623
1.004
7.376
2.872
3.684
1.568
3.272
579
5.820
4.946
1.467
8.981
3.872
4.771
2.030
3.856
763
8,56
7,12
3,97
7,57
5,86
7,03
4,22
7,39
2,95
24%
23%
34%
28%
27%
26%
35%
29%
38%
1,04
0,88
0,70
1,20
1,03
1,14
1,03
1,24
0,77
1,37
1,14
0,74
1,38
1,03
1,17
0,92
1,52
0,74
1,31
1,30
1,06
1,15
1,00
1,03
0,89
1,22
0,96
21%
27%
32%
18%
26%
23%
23%
15%
24%
Pharmacy
763
3.912
5.379
5,13
29%
1,05
1,11
1,06
27%
1,01 20%
1,52
0,80
1,34
0,78
0,66
1,03
1,26
15%
25%
15%
26%
16%
26%
25%
4,0
2,5
4,0
3,5
4,5
3,5
4,0
Normalised citation impact parameters per subject group and per
rating
Normalised Citation Impact Distribution
RATING
# Depts
MEAN
STD
P25
MEDIAN
P75
Social Sciences and Humanities
1
5
0.45
2
58
1.12
3a
291
0.92
3b
145
0.82
0.29
1.92
1.36
1.23
0.40
0.18
0.17
0.00
0.45
0.64
0.68
0.62
0.65
1.24
1.20
1.10
4
5
5*
1.80
1.30
1.27
0.37
0.53
0.64
0.84
1.03
1.09
1.31
1.59
1.68
366
389
151
1.02
1.26
1.34
Comparison WoS vs. Scopus
results from one of the HEFCE projects:
see tomorrow
Martijn Visser and Henk Moed:
“Comparing Web of Science and Scopus on a paper-bypaper basis”
Conclusion
Advanced bibliometric analysis is a powerful tool to make
research assessment more objective, transparent and
effective, particularly in the natural science and medical
fields, and also in many of the engineering and social
science fields but both internal and external WoS/Scopus
coverage are absolutely necessary parameters to assess
the validity of WoS/Scopus based measurements
(including the non-WoS/Scopus publications…)
As always, never use it as a stand-alone tool.
But also: it is an effective instrument for measuring
interdisciplinarity, knowledge flows and knowledge
diffusion -even for non-WoS/Scopus publications!
Thank you for your attention
more information: www.cwts.leidenuniv.nl
Appendix
According to an influential Swiss scientist:
Bibliometric investigations are clearly not very reliable…. In
particular, the "frequency of citation" does not account for the
quality of the researchers, because
(1) it depends more often on the social recognition of the
researcher than excellence of his/her scientific work;
(2) it favors researchers who work on fashionable topics;
(3) it favors the fields of knowledge which traditionally publish
shorter articles compared to those where publications are longer;
(4) it cannot differentiate between the fashion and the substance
of a paper;
(5) it can favor the authors of "surveys", who are very frequently
cited, compared to the authors of focused research papers;
(6) a position article or even an erroneous article can be criticized
and consequently well cited.
According to an influential Swiss scientist:
How to increase your ‘bibliometric values’













Write your name on papers by your PhD students
Ignore your publisher’s copyright: put your paper online
Work in a popular area so that many others can cite you
Write survey papers, not research papers
Never change your established research area
Avoid innovative and new (but risky) projects
Chose catchy titles for your papers
Emphasize quantity instead of quality
Do not lose valuable time, avoid events like this one
Concentrate on paper production, not good teaching
Heavily cite you own (and your friend’s) papers
Never publish more than a single ‘least publishable unit’
Cannibalize your old papers: refurbish and republish them
Main anecdotal objections against citation
analysis
-
Mendel Syndrome
Wittgenstein Syndrome
Lowry Effect
Einstein effect
Old boys clique
Disgusting anyway
A scientist has index h
if h of his/her N papers have at least h
citations each
and the other (N-h) papers have no more
than h citations each
Hirsch (h-) index AFJ van Raan =
18
Correlation of h-index (h) with number of citations (C)
for all chemistry groups in the Netherlands
100
y = 0.394x0.4543
R2 = 0.8793
h
10
1
1
10
100
C
1000
10000
Correlation of h-index (h) with number of publications (P)
for all chemistry groups in the Netherlands
100
y = 0.7293x0.5186
R2 = 0.4859
h
10
1
1
10
100
P
1000
Correlation of h-index (h) with CPP/FCSm
for all chemistry groups in the Netherlands
100
0.5331
y = 6.9566x
R2 = 0.2161
h
10
1
0.10
1.00
CPP/FCSm
10.00
Large European University
Among top 25 % in
publication output
and citation impact
0
CITATION I PACT RANK PCTL
Top 25%
ENG
ECON
CLM
25
Impact
ranking
(MULTI)
(SOC)
BIOL-HU
APC
MOLB
CHEM
GEO
50
PSY
BIOL-AP
(A&H)
SOC-MED
75
PHYS
Bottom 25%
100
100
Bottom 25%
MATH
75
50
PUBLICATION RANK PTCL
Publ.ranking
25
0
Top 25%
‘Top’ research university
0
CITATION I PACT RANK PCTL
Top 25%
25
University has
a top position
in each discipline
Impact
ranking
50
CLM
SOC-MED
MATH
ENG
BIOL-HU
PSY
ECONBIOL-AP
GEO MOLB (A&H)
(MULTI)
APC PHYS (SOC)
CHEM
75
Bottom 25%
100
100
75
Bottom 25%
50
PUBLICATION RANK PTCL
Publ.ranking
25
0
Top 25%
Citation-counting scheme based on
‘roof-tile’ method:
Citation years
1995
1996
1997
1998
1999
2000
2001
2002
1995
1996 1997 1998 1999 2000 2001 2002
1995
1996 1997 1998
1996 1997 1998
1998
1997
1998
1997 1998
1998
1998
1998
1998
1998
1999
1999
1999 2000
1999
1999
2000
1999 2000
1999
1999
2000
1999
1999 2000
2000
2000
2000
2000
2001
2001
2001
2001
2001
2001
2001
2002
2002
2002
2002
ISI IF
Example: The Lancet
Publs
2000+01
Art
784
Not
144
Rev
29
Subtot 957(a)
Let
Edi
Other
Total
4181
1313
1421
7872
Cits
2002
7134
593
232
7959(b)
4264
905
909
14037 (c)
Citations in 2002__________
Citeable documents in 2000 and 2001
14037 (c)
957 (a)
IF=14.7
CWTS IF
Citations to Art/Not/Rev in 2002
Art/Not/Rev in 2000 and 2001
7959 (b)
957 (a)
IF=8.3
Citations to Art/Let/Not/Rev in 2002
Art/Let/Not/Rev in 2000 and 2001
7959+4264
957+4181
IF=2.4
Manipulability of citation indicators
proposed in this study
To which extent are our citation-based indicators
sensitive to manipulation?
Can one increase actual citation impact by:
(1) Increasing author self citation?
Author self-citations are not included: increasing
author self-citation has no effect
(2) Publishing in high impact journals?
A case study of 2,000 UK senior authors with
>10 p/y revealed that journal impact explains
~20% of the variance in the citation impact
rates.
Journal impact is therefore not a dominant
determinant of actual citation impact at the level
of individual senior authors.
(3) Collaborate more intensively?
Some studies report positive correlation between
number of authors and citation impact, but they
ignore differences in authoring practices among
research fields.
Author self-citations are not included in this
study. It all depends upon who collaborates with
whom. There is also the issue of causality: ‘good’
research may attract high-impact collaborators.
(4) Publishing with US authors
because they overcite their own
papers?
Studies found no conclusive evidence that US
scientists in science fields excessively cite papers
originating from their own country.
(5) Publishing less, only the very best
papers?
One would expect a higher citation impact per
paper. Longer term effects of such a publication
strategy are uncertain. PhD students need papers
in their CV’s. It may become difficult for a group
to attract good PhD students if its policy is to let
them publish only a few papers. Another factor
is that publications also enhance the visibility of a
group’s research activities. If a group starts
publishing substantially less papers, this may
lead to a lower visibility and hence to a lower
citation impact, even per paper.
(6) Making citation arrangements?
A high impact group receives its citations from
dozens of different institutions. The distribution
of citations amongst citing institutions is very
skewed. The contribution of the tail of the
distribution to the citation impact is relatively
large. Making arrangements with a few
institutions will not lead to a substantial increase
in citation impact.
More information: www.cwts.leidenuniv.nl
mapping example:
http://studies.cwts.nl/projects/leiden-benchmark
Application of Thomson-ISI Impact Factors for
research performance evaluation is
irresponsible
*
*
*
*
Much too short ‘Citation window’
No Field-specific Normalization
No distinction between document types
Calculation errors/inconsistencies
nominator/denominator
* Underlying citation distribution is very skew:
IF-value heavily determined by a few very highly
cited papers
* L A N G U A G E & L IN G
600
500
400
300
200
100
0
Cits91
Cits92
Cits93
Cits94
Cits95
Cits96
Cits97
Cits98
Cits99
Cits00
Cits01
Cits02
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
What is quality?
Numbers are order of magnitude lower > examples (e.g., profiles)
H-index example social sciences
National publications (also the case for engineering!)
Coverage
Figures about the role of books vs. journal papers (Uppsala data?)
Language
Citation window
non-WoS analysis, target known
non-WoS analysis, target unknown (CHE-2 results)
Societal relevance of social sciences > try sustainability maps
for social science themes!
Database problems (EC-ASSIST list)
Social science data from benchmark studies
Norwegian Association of Higher Education Institutions
classification of sources
European Reference Index for the humanities journal classifications
ERIH-ESF-HERA
Library Catalog Analysis: number of library copies per book title,
e.g., Worldcat (Linmans); exploratory study of 43 catalogs in
economics (ask HM…)
Use slides of HMs and TNs CHERPA presentations!
bibliometrics is more than an instrument of research performance
analysis, it van also reveal patterns of knowledge development
and diffusion
*
*
*
*
*
*
*
*
Is the lifetime of a book longer than that of a journal article?
Flemish study on social sciences and humanities!
Nature of citations may be different
Hierarchy of books through reputation of publishers?
Results of our HEFCE analysis of the RAE 2001
Figures on p.37-40 in HEFCE Scoping report
Leiden Benchmarking: social sciences and humanities: order of magnitude,
ranking, trend, soc sc & hum profiles, ‘bolletjes’ charts
Field
Political science
Economics
Psychology
N
4742
9062
14132
TABLE 1: PUBLICATION TYPES FOR UPPSALA UNIVERSITY AND INSTITUTES 2002 - 2006
Research Unit
UPPSALA
WoS
Non- WoS
Journal articles
Regular Non
Cdyear
Citeable 2007
Journal articles
Conf proc Bk Ch Books Thesis Rep Pat Oth Coll (ed.) Proc (ed.)
Regular Rev Book
Doct Lic
Rev
8.502
270
4
2.845
56
474
2.773
2.260
381
291 102 826 49 344
15
42
Arts
61
Biology
1.265
Chemistry
911
Earth Sciences
340
Educational Sciences
8
Engineering
622
Languages
23
Mathematics and Computer Sc 453
Medicine
3.556
Pharmacy
763
Physics
1.057
Social Sciences
380
Theology
5
16
38
17
27
12
21
8
88
6
11
33
2
1
2
1
-
492
116
38
92
52
56
383
142
510
52
127
698
169
14
2
2
1
1
6
12
10
5
3
168
5
5
8
1
67
1
35
1
3
61
123
106
88
95
226
57
528
142
521
100
21
472
574
10
416
57
26
52
57
16
285
45
218
36
23
819
230
65
5
1
1
11
4
30
21
22
5
15
165
40
34
28
9
6
13
17
17
7
58
9
8
78
11
3
1
2
9
-
1
13
2
3
21
3
-
2 36 - 34
6 20 4 10
5
3 2
2
4 26 - 17
- 22 - 12
14 12 13 10
- 18 - 52
36 318 32 13
10 9 1 47
3
1 4
6
13 35 7
10 347 - 123
1
4
- 16
Results of the HEFCE study of the bibliometric
characteristics of the 2001 RAE
from: Moed, Visser, and Buter, 2008
Science Mathem Soc+Hum
-----------------+--------+--------+--------+
Authored book
|
690 |
112 | 13657 |
|
0.36 |
0.06 |
7.08 |
|
4.77 |
0.77 | 94.45 |
|
0.73 |
1.69 | 14.95 |
-----------------+--------+--------+--------+
Chapter in book |
1645 |
152 | 21515 |
|
0.85 |
0.08 | 11.15 |
|
7.06 |
0.65 | 92.29 |
|
1.73 |
2.29 | 23.56 |
-----------------+--------+--------+--------+
Conference
|
4186 |
180 |
2785 |
contribution
|
2.17 |
0.09 |
1.44 |
| 58.54 |
2.52 | 38.95 |
|
4.40 |
2.71 |
3.05 |
-----------------+--------+--------+--------+
Edited book
|
218 |
9 |
3489 |
|
0.11 |
0.00 |
1.81 |
|
5.87 |
0.24 | 93.89 |
|
0.23 |
0.14 |
3.82 |
Total
14459
7.49
23312
12.08
7151
3.70
3716
1.93
n = publ type of a main field as % of all publs of all 3 main fields
(e.g., 13,657 / 193,014)
n = publ type of a main field as % of same publ type of all 3 main
fields (e.g., 13,657 / 14,459)
n = publ type of a main field as % of all publs of the main field
(e.g., 13,657 / 91,324)
TREND ANALYSIS FOR ERASMUS UNIVERSITY
CLINICAL MEDICINE
Time Period
1997
1998
1999
2000
2001
-
2000
2001
2002
2003
2004
P
Rank
C
Rank
CPP/FCSm
Rank
P*
CPP/FCSm
Rank
3.808
3.921
4.019
4.170
4.363
27
26
25
26
23
23.686
23.470
24.249
25.704
27.415
20
20
21
22
20
1,71
1,55
1,53
1,52
1,57
38
52
53
59
52
6.501,1
6.095,1
6.139,5
6.344,6
6.853,7
22
21
22
26
23
2002 - 2005
2003 - 2006
3.368
2.341
22
23
Time Period
Ptop 20%
Rank
1.115
1.095
1.090
1.150
22
23
24
22
588
561
584
632
P
Rank
280
302
320
364
389
30
26
22
20
20
1997 1998 1999 2000 -
2000
2001
2002
2003
33.038
31.856
16
15
1,68
1,73
35
36
5.660,4
4.049,6
18
15
Ptop 5%
Rank
Ptop 1%
Rank
23
27
25
22
335
301
318
344
20
26
24
24
79
73
76
92
21
25
25
20
C
Rank
CPP/FCSm
Rank
P*
CPP/FCSm
Rank
363
339
374
536
528
44
50
43
32
33
1,07
0,95
0,94
1,08
0,98
129
161
153
120
146
300,2
288,3
301,6
393,2
382,6
51
52
54
43
49
Ptop 10% Rank
ECONOMICS
Time Period
1997
1998
1999
2000
2001
-
2000
2001
2002
2003
2004
2002 - 2005
2003 - 2006
296
205
18
19
Time Period
Ptop 20%
Rank
62
65
72
90
50
45
41
33
1997 1998 1999 2000 -
2000
2001
2002
2003
627
665
30
30
Ptop 10% Rank
23
25
28
39
68
67
60
46
1,09
1,18
139
119
323,6
241,1
39
37
Ptop 5%
Rank
Ptop 1%
Rank
12
11
12
15
65
75
73
59
3
1
1
2
54
91
91
73
Figure 1
Output and impact compared world field average
1999 - 2006
Erasmus University and 20 benchmark institutions
5.50
5.00
UNIV CHICAGO (USA)
4.50
HARVARD UNIV (USA)
4.00
STANFORD UNIV (USA)
MIT (USA)
UNIV MARYLAND COLLEGE PARK (USA)
CPP/FCSm
3.50
UNIV PENN (USA)
DARTMOUTH COLL (USA)
UNIV N CAROLINA DUKE UNIV (USA)
CHAPEL HILL
(USA)
NORTHWESTERN
UNIV
(USA)
3.00
UNIV OXFORD (GREAT
BRITAIN)
2.50
COPENHAGEN BUSINESS
SCH (DENMARK)
INSEAD (FRANCE)
LONDON BUSINESS SCH
(GREAT BRITAIN)
UNIV AMSTERDAM
STOCKHOLM (NETHERLANDS)
SCH ECON
UNIV TILBURG
(SWEDEN)
UNIV GRONINGEN
(NETHERLANDS)
(NETHERLANDS)
ERASMUS UNIV
ROTTERDAM
(NETHERLANDS)
UNIV MAASTRICHT
(NETHERLANDS)
2.00
1.50
1.00
0.50
0
100
200
300
400
500
600
TOTAL PUBLICATIONS
Black coloured squares above (below) the horizontal reference line represent groups
for which the impact (CPP) is significantly above (below) world average (FCSm)
700
Top-ranked publications according to
country of origin
Country of first author of Top 50 publ > 1980
Field
US
UK
EU continent
Other
Unknown
Political sc
50%
20%
18%
5%
7%
Economics
71%
9%
9%
6%
6%
Psychology
50%
12%
21%
12%
6%
In the set of ‘best’ publications submitted to the 2001 RAE it was
found that journal articles constitute 73% of submitted papers from
all Subject Groups.
For science-related Units of Assessment we find 92%. The profile for
Mathematics is quite similar to that for Science.
In Social Sciences and Humanities books are important publication
sources. The shares of authored books and book chapters are 15 and
24%, respectively.
Most highly cited non-WoS items by document type
Journal
Hand-
Soft-
Proc-
Book
Article
book
Manual
ware
eedings
Thesis
Other
Political sc
54
19
10
3
2
14
4
5
Economics
75
52
42
2
2
4
5
8
Psychology
150
112
30
430
32
11
11
2
The comparison of WoS and Scopus coverage of the 2001
RAE ‘best’ publications shows that Scopus coverage is
especially better in the Subject Groups Subjects allied to
Health (e.g., clinical dentistry, nursing, pharmacy), and to
a lesser extent also in Engineering & Computer Science
and Health Sciences.
In Clinical Medicine, Biological Sciences and Physical
Sciences, however, Scopus coverage is slightly lower than
WoS coverage.
from: Moed and Visser 2008, Appraisal of Citation Data Sources, HEFCE-report
Percentage of WoS papers found in Scopus
Publ Year
WoS
Database
Segment
Nr WoScovered
Journals
Nr WoS
Articles and
Reviews
1996
1996
1996
Science
Soc Sc & Hum
Total
5,320
2,610
7,930
673,271
88,583
761,854
% WoS
Articles and
Reviews
found in
Scopus
93
57
89
2005
2005
Science
Soc Sc & Hum
6,146
2,719
867,748
94,629
97
72
2005
Total
8,865
962,377
95
Distribution of %WoS papers found in Scopus, science fields
100
1996
2005
% WoS Journals
80
60
40
20
0
0
10
20
30
40
50
60
70
% WoS Papers found in Scopus
80
90
100
WoS vs Scopus coverage of 2001 RAE ‘best’ publications
Total
submitt
publ
D
i s
c
i p
l i n
% in
WoS
% in
Scopus
Δ
e
Science
Mathematics
Social Sciences and Humanities
95,056
6,634
91,324
84.1
81.8
24.9
84.4
80.1
25.9
0.3
-1.7
1.0
SSHU
SSHU
SSHU
SSHU
SSHU
SSHU
SSHU
SSHU
SSHU
SSHU
SSHU
SSHU
SSHU
SSHU
Town & Country Planning
Social Work
Accounting and Finance
Built Environment
Social Policy & Administr
Politics and Internat St
Sociology
Business & Managem St
Geography
Education
Econom & Econometrics
Sports-related Subjects
Library & Inform Manag
Anthropology
Total
submitt
publ
1,478
1,642
779
2,471
3,912
4,382
3,519
9,746
4,890
8,662
2,879
1,301
1,259
1,180
% in
WoS
% in
Scopus
38.1
22.9
21.7
24.5
25.8
26.4
29.2
37.9
61.6
16.0
67.5
60.5
31.7
27.5
57.4
36.7
34.9
35.9
34.1
34.6
37.0
45.5
68.6
22.5
72.0
63.4
34.4
28.0
Δ
+19.4
+13.8
+13.2
+11.4
+8.3
+8.1
+7.8
+7.6
+7.0
+6.5
+4.5
+2.9
+2.7
+0.4
INTERNAL COVERAGE OF THE CITATION INDEX BY MAIN FIELD
Main Field
P
00-05
Avg Nr
Refs
Refs
<1980
CLINICAL MEDICINE
BIOL SCI: HUMANS
BIOL SCI: ANIMALS & PLANTS
MOLECULAR BIOLOGY & BIOCHEM
PHYSICS AND ASTRONOMY
CHEMISTRY
MATHEMATICS
GEOSCIENCES
APPLIED PHYSICS AND CHEMISTRY
ENGINEERING
MULTIDISCIPLINARY
ECONOMICS
PSYCHOLOGY, PSYCHIATRY & BEHAV SC
SOCIAL SCIENCES RELATED TO MEDICINE
OTHER SOCIAL SCIENCES
HUMANITIES & ARTS
3,893
2,421
754
1,257
1,492
871
233
134
514
373
126
35
633
292
291
220
33.3
39.0
41.2
40.5
36.7
34.5
21.5
40.4
24.7
21.5
30.5
38.9
40.3
28.9
34.9
38.7
6,950
4,449
5,638
2,930
4,898
3,608
957
578
1,382
686
215
160
2,789
597
1,469
2,477
%Refs
Refs
<1980 Non-CI
5%
5%
18%
6%
9%
12%
19%
11%
11%
9%
6%
12%
11%
7%
14%
29%
Refs CI %Refs CI
11,637 110,945
6,447 83,588
6,611 18,805
3,968 44,001
7,555 42,320
3,717 22,693
1,680
2,375
2,169
2,673
2,081
9,256
3,151
4,185
339
3,291
593
608
7,296 15,406
2,153
5,698
5,649
3,047
5,063
973
91%
93%
74%
92%
85%
86%
59%
55%
82%
57%
91%
51%
68%
73%
35%
16%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90% 100%
AGRICULTURE AND FOOD SCIENCE
BASIC LIFE SCIENCES
BASIC MEDICAL SCIENCES
2006
BIOLOGICAL SCIENCES
BIOMEDICAL SCIENCES
CLINICAL MEDICINE
HEALTH SCIENCES
ASTRONOMY AND ASTROPHYSICS
CHEMISTRY AND CHEMICAL ENGINEERING
COMPUTER SCIENCES
EARTH SCIENCES AND TECHNOLOGY
ENVIRONMENTAL SCIENCES AND TECHNOLOGY
MATHEMATICS
PHYSICS AND MATERIALS SCIENCE
STATISTICAL SCIENCES
CIVIL ENGINEERING AND CONSTRUCTION
ELECTRICAL ENGINEERING AND TELECOMMUNICATION
ENERGY SCIENCE AND TECHNOLOGY
GENERAL AND INDUSTRIAL ENGINEERING
INSTRUMENTS AND INSTRUMENTATION
MECHANICAL ENGINEERING AND AEROSPACE
INFORMATION AND COMMUNICATION SCIENCES
LANGUAGE AND LINGUISTICS
ECONOMICS AND BUSINESS
EDUCATIONAL SCIENCES
MANAGEMENT AND PLANNING
POLITICAL SCIENCE AND PUBLIC ADMINISTRATION
PSYCHOLOGY
SOCIAL AND BEHAVIORAL SCIENCES, INTERDISCIPLINARY
SOCIOLOGY AND ANTHROPOLOGY
CREATIVE ARTS, CULTURE AND MUSIC
HISTORY, PHILOSOPHY AND RELIGION
LAW AND CRIMINOLOGY
LITERATURE
MULTIDISCIPLINARY JOURNALS
purple: non-WoS ref
light blue: CI ref
References ISI
References non-ISI
INFORMATION AND COMMUNICATION SCIENCES
LANGUAGE AND LINGUISTICS
ECONOMICS AND BUSINESS
EDUCATIONAL SCIENCES
MANAGEMENT AND PLANNING
POLITICAL SCIENCE AND PUBLIC ADMINISTRATION
PSYCHOLOGY
SOCIAL AND BEHAVIORAL SCIENCES, INTERDISCIPLINARY
SOCIOLOGY AND ANTHROPOLOGY
CREATIVE ARTS, CULTURE AND MUSIC
HISTORY, PHILOSOPHY AND RELIGION
LAW AND CRIMINOLOGY
LITERATURE
MULTIDISCIPLINARY JOURNALS
2006
purple:
non-WoS ref
References ISI
lightReferences
blue: CI
ref
non-ISI