Transcript Bibliometrics: the black art of citation rankings Angela Carritt Juliet Ralph October 2010 These slides are available on http://www.bodleian.ox.ac.uk/services/training /wiser/presentations.

Bibliometrics: the black
art of citation rankings
Angela Carritt
Juliet Ralph
October 2010
These slides are available on
http://www.bodleian.ox.ac.uk/services/training
/wiser/presentations
Overview of Session
• What are bibliometrics?
• Why bother?
• Problems
• Calculating impact factors using Web of Science
•
•
•
…for an article
…for a researcher
…for a research group / department / institution
• Calculating impact factors using Scopus
• Calculating journal impact factors using the Journal Citation
Reports
What are bibliometrics?
•…the statistical analysis of books, articles, or other
publications. Oxford English Dictionary
•…”ways of measuring patterns of authorship, publication
and the use of literature” HEFCE, Bibliometrics and the
Research Excellence Framework (REF)
• Use of citation analysis to measure the impact of research
Citations to an individual paper
2010
2010
2010
2009
2008
2008
Citations to a researcher
Citations to a research
group/department
Citations to a journal
Continued….
• Wide range of formulae developed which aim to use
bibliometrics to assess research impact accurately
• Some of the things that are sometimes considered:
•
•
•
•
•
•
•
•
Times cited
Number of articles published
Number of self citations
Number of authors (of cited work)
Type of article
Period over which articles were published
Prestige of citing journal
Subject/discipline…
The power behind
Why bother?
• Research Excellence Framework
• Benchmarking of departments and research groups
• Grant applications
• Recruitment of individuals
Lots of problems...
• Self-citations
• Negative citations
• Insignificant citations
• Multiple authors/research groups
• Incomplete citation lists - does not include citations in
books... or other publications not indexed by Web of
Science/Scopus…poor coverage of conferences
• Not comparable across disciplines – may disadvantage
researchers in interdisciplinary fields
• Review articles are more highly cited than original research
• More...
http://www.slideshare.net/guest633b30/bibliometrics-and-scientometrics-1065282
© http://ceochef.files.wordpress.com/2008/07/pinch-of-salt.jpg
Individual articles
• On Web of Science - use Cited Reference Search
• Better than General Search at retrieving Variants (incorrect
citations),
• Example:
• Effectiveness of PowerPoint presentations in lectures.
Author(s): Bartsch RA, Cobern KM
• Source: COMPUTERS & EDUCATION
• Volume: 41 Issue: 1 Pages: 77-86
• Published: AUG 2003.
Cited reference variant
Analyzing a highly cited article
Analyzing the Citing articles
…by Highest citing authors
…by Highest citing journals
Impact factors for a researcher
Citation report for an author
Citation map for an article
Citation map
Other services offering citation searching - SCOPUS
• Sciences and Social Sciences
• Results include journal articles and web pages
• Each reference to a paper shows the number of times an
article has been cited
Citation analysis in Scopus
…for a research group
• Not easy!
• But can search by department or institution
• Web of Science - Search the Address field
• Strict rules regarding syntax
• Search by postcode or name of department/college,
e.g. OX1 2JD
• univ oxford same pathol
• oxford same trinity
• oxford same Ludwig Inst Canc Res
• Scopus
• Affiliation Search
• Much less strict!
• By name not postcode
Searching by Address
Check the abbreviations lists
Trial and error
Scopus Affiliation Search
Scopus Affiliation ID
Impact Factors - Journal Citation Reports (JCR)
• Compare relative importance of journals by measuring their impact
• Uses
•
•
Help you to decide where to publish
Help librarians to make decisions about journal purchase, retention etc
• Abuses
•
Have been used to measure research impact of individual and research
groups
• Based on citation data from Web of Science
• Covers
•
> 5,900 journals in science and technology
•
> 1,700 journals in the social sciences
Immediacy Index
Measures how quickly articles are cited.
Calculated:
no. of citations to articles published this year ÷ no. of
articles published this year.
Impact Factor - Number of times the
“average” article published in the
previous 2 or 5 years was cited this year.
Calculated:
no. of citations to articles published in the
last 2 (or 5) years ÷ no. of articles
published in same period.
Cited Half-Life - How many
years you have to go back to
account for 50% of citations
to the journal. e.g. 50% of
citations were to articles
published in the last 3.5
years. The rest cited earlier
articles.
Detailed view
Detailed view continued
Detailed view continued
Citations TO the journal by year of
cited article (e.g. 333 of this year’s
citations to Biological Review were
to articles published in in 2005 )
Detailed view continued
Citations from Biological
Review (to other journals
and self cites) by year of
cited article E.g. 334
citations from Biological
Reviews journal cited
articles published in 2007
Type of articles included
Eigenfactor Metrics
• Use “Google style”
algorithms
• Take into account prestige of
citing sources
• Attempts to measure how
often the average
researcher would encounter
the journal
• http://wellformed.eigenfactor.org/
Google’s PageRank from
http://en.wikipedia.org/wiki/PageRank
Eigenfactor: Score & Article Influence
• Eigenfactor – increases with the size of the journal
• Article Influence – Takes into account number of articles
published. More comparable to the JCR impact factor
Journal Impact Factors: Problems
Use with caution…Results are skewed by many factors…
• Size
• Frequency / time of publication
• Type of content - review articles are more heavily cited than original
research…
• Journals that are not indexed by WOS are disadvantaged
•
Non English Language journals disadvantaged
•
“Cited” only journals
• Problems when journals change names
• Results are not comparable across discipline (some journals in the
wrong discipline)
• Journal impact factors should NEVER be used to assess impact of
researchers / groups etc
Where will it all end…
• And how much will it all cost…
Watch this space!
Brief bibliography
General
• Broadus, R. N., “Towards a definition of Bibliometrics” Scientometrics,
vol. 12, nos 5-6, (1987) 373-379 @
www.springerlink.com/content/v111750n14086384/fulltext.pdf
• HEFCE papers, reports, papers and pilots on the use of bibliometrics
in the REF @ www.hefce.ac.uk/research/ref/Biblio/
H-Index
• Hirsch, J. E. (15 November 2005). "An index to quantify an individual's
scientific research output". PNAS 102 (46): 16569–16572 @
www.pnas.org/content/102/46/16569.abstract
• Ball, P. “Index aims for fair ranking of scientists”, Nature 2005 Aug 18
436: 900
Brief bibliography
Journal Impact Factors and the JCR
• The Thomson Reuters Impact Factor (originally published in the Current
Contents print editions June 20, 1994) @
http://thomsonreuters.com/products_services/science/free/essays/impact_factor/
• Garfield, E. "The agony and the ecstasy: the history and meaning of the
Journal Impact Factor“ Paper at the International Congress on Peer Review
And Biomedical Publication, Chicago, September 16, 2005 @
http://garfield.library.upenn.edu/papers/jifchicago2005.pdf
Want more…
• Wikipedia entries on the following topics include useful bibliographies: the
h-index, journal impact factors and the Eigenfactor
• A Google Scholar (http://scholar.google.co.uk/) search will return many useful
articles including subject studies on the use of bibliometrics