Transcript WISER: Bibliometrics II The Black Art of Citation Rankings Angela Carritt Juliet Ralph May 2011 These slides are available on http://www.bodleian.ox.ac.uk/services/training /wiser/presentations.

WISER: Bibliometrics II
The Black Art of Citation Rankings
Angela Carritt
Juliet Ralph
May 2011
These slides are available on
http://www.bodleian.ox.ac.uk/services/training
/wiser/presentations
Overview of Session
• What are bibliometrics?
• Why bother?
• Problems
• Bibliometric measures for
•
•
•
•
•
…an article
…a journal
…a researcher
…an institution
…a country
• using Web of Science, Scopus, and other analytical tools
What are bibliometrics?
•…the statistical analysis of books, articles, or other
publications. Oxford English Dictionary
•…”ways of measuring patterns of authorship, publication
and the use of literature” HEFCE, Bibliometrics and the
Research Excellence Framework (REF)
• Use of citation information to measure the impact of
research
Citation analysis
Why bother?... REF?
• Pilot exercise concluded that citation information “is not sufficiently
robust to be used formulaically or as a primary indicator of quality; but
there is considerable scope for it to inform and enhance the process of
expert review” HEFCE www.hefce.ac.uk/research/ref/Biblio/
• Research Excellence Framework 2014
• 36 “Expert panels” will set out criteria for measuring the “quality of
research outputs” – including whether to use bibliometrics or not.
Criteria statements for each panel will be announced in late 2011 –
watch this space!
• More @ http://www.hefce.ac.uk/research/ref/Biblio/
Why bother
• Benchmarking of departments and research groups
• Grant applications
• Recruitment of individuals
• Where to publish
Lots of problems...
• Self-citations
• Negative citations
• Insignificant citations
• Multiple authors/research groups
• Incomplete citation lists - does not include citations in
books... or other publications not indexed by Web of
Science/Scopus…poor coverage of conferences
• Not comparable across disciplines – may disadvantage
researchers in interdisciplinary fields
• Review articles are more highly cited than original research
• More...
http://www.slideshare.net/guest633b30/bibliometrics-and-scientometrics-1065282
Impact of an individual article
• Web of Science counts Times Cited
• Cited Reference Search better than General Search at
retrieving Variants (incorrect citations).
•
No other database does this!
• How many times has this article been cited?
• Effectiveness of PowerPoint presentations in lectures.
Author(s): Bartsch RA, Cobern KM
• Source: COMPUTERS & EDUCATION
• Volume: 41 Issue: 1 Pages: 77-86
• Published: AUG 2003.
Cited reference variant
View the citing articles
Analyze the Citing articles
…see who’s cited it the most
…where it’s been cited
Turn it into a Citation Map
Citation map
Highly cited articles in your field
• ScienceWatch
• http://www.sciencewatch.com/
• Weekly tracking of highly-cited papers and topics
• free Web resource for science metrics and analysis.
• Includes interviews, essays, podcasts and profiles from
scientists, journals, institutions, and nations.
• Uses citation data from Thomson Reuters (owner of ISI
Web of Science).
The h-index: to quantify an
individual’s research output
Aims to measure
productivity and impact.
Your h-index in Web of Science
Do an author search.
Get list of papers by them.
Sort by Times Cited.
Click on Create Citation Report.
This analyses the batch of papers.
And calculates h-index.
h-index=75
75 articles cited
75 times or more
Citation tracking & analysis in SCOPUS
• Scopus covers 18,000 journals & conference proceedings
• Science, Mecicine, Social Sciences & Humanities
• Each record for a paper shows the number of times it has
been cited in Scopus since 1996
• Similar analytical tools to Web of Science
• www.scopus.com
Your h-index in Scopus
Search for author.
Get list of papers.
Tick Select All.
Click on View citation overview.
h-index = 73
based on citations in
Scopus post-1996 to
537 papers
h-index = 63.
Drops if exclude
self-citations
Your h-index and Google Scholar
www.harzing.com
Calculate it with Publish or Perish
Caveat emptor
Bibliometrics for an institution
•
•
•
•
Best tool is on Scopus:
Affiliation Search
Search by institution name
university of oxford retrieves same
results as oxford university
Scopus Affiliation Search
Get an overview of its
publications
Overview of
department,
college,
division
sometimes
possible too.
Recommendation
(1) All authors should
be encouraged to use
the phrase ‘Oxford
University’ or
‘University of Oxford’
in their publication
address, to ensure
that the publication
may be captured by
citation databases.
(2) [may also] cite
either dept or college.
Oxford & Symplectic Elements
For researchers & departments
• A record keeping tool for research outputs.
• Boasts automatic searching of databases such as Web of
Science & Scopus.
• Facility to run reports for a department.
• For more information go to
• www.admin.ox.ac.uk/pras/research/symplectic/
• The solution for your department?
• Contact [email protected]
Bibliometrics for journals
• Bibliometrics can be used to find the most influential journal
in your subject area
• ISI Journal Citation Reports (Web of Science)
• Eigenfactor
• SCImago Journal Ranking (SJR) and SNIP
Bibliometrics & journals: Uses and Abuses
• Uses
• Help you to decide where to publish
• Help librarians to make decisions about journal
purchase etc
• Abuses
• Have been used to measure research impact
of individual and research groups
Journal Citation Reports (JCR)
• Based on citation data from Web of Science
• Covers
• > 5,900 journals in science and technology
• > 1,700 journals in the social sciences
Immediacy Index
Measures how quickly articles are cited.
Calculated:
no. of citations to articles published this year ÷ no. of
articles published this year.
Impact Factor - Number of times the
“average” article published in the previous 2
(or 5) years was cited this year.
Calculated:
no. of citations to articles published in the
last 2 (or 5) years ÷ no. of articles
published in same period.
Cited Half-Life - How many
years you have to go back to
account for 50% of citations
to the journal. e.g. 50% of
citations were to articles
published in the last 3.5
years. The rest cited earlier
articles.
Detailed view
Detailed view continued
Detailed view continued
Citations TO the journal by year of
cited article (e.g. 333 of this year’s
citations to Biological Review were
to articles published in in 2005 )
Detailed view continued
Citations from Biological
Review (to other journals
and self cites) by year of
cited article E.g. 334
citations from Biological
Reviews journal cited
articles published in 2007
Type of articles included
Eigenfactor Metrics
• Take into account prestige
of citing sources
• Use “Google style”
algorithms
• Attempts to measure how
often the average
researcher would encounter
the journal
• http://wellformed.eigenfactor.org/
Google’s PageRank from
http://en.wikipedia.org/wiki/PageRank
Eigenfactor: Score & Article Influence
• Eigenfactor – increases with the size of the journal
• Article Influence – Takes into account number of articles
published. More comparable to the JCR impact factor
Equivalent metrics by Elsevier
Journal
Analyzer
in
Scopus
uses
2 new
metrics
(on
Analytics
tab) :
SJR &
SNIP.
SJR – an alternative impact factor
• SCImago Journal Rank developed by Elsevier in
partnership with Spanish academics.
• Scopus is its data source:
• 50% more journals than ISI Web of Science.
• Weights citations according to the status of the citing
journal and aims to measure journal prestige rather than
popularity.
• More info at http://www.info.sciverse.com/scopus/scopusin-detail/tools/journalanalyzer/
SNIP – also by Elsevier
• Source Normalized Impact per Paper.
• Measuring contextual citation impact of scientific journals
• context =the characteristics of its subject field.
• SNIP aims to account for differences in citation potential
and topicality across research fields.
• ‘Citation potential’ (citation frequency) higher in life
sciences than maths, engineering, social science.
• Higher in basic science than applied or clinical journals.
• Scopus is again the data source.
• http://www.journalmetrics.com/
Bibliometrics by country
Another use of SJR data.
Compare countries.
It’s free!
www.SCImagojr.com
Favourably reviewed in Nature
SJR & SNIP freely available – not
dependent on subscriptions to
Scopus.
Worthy challenger to ISI.
Journal Impact Factors: Problems
Use with caution…Results are skewed by many factors…
• Size
• Frequency / time of publication
• Type of content - review articles are more heavily cited than original
research…
• Journals that are not indexed by WOS / Scopus are disadvantaged
•
Non English Language journals disadvantaged
• Problems when journals change names
• Results are not comparable across discipline (some journals in the
wrong discipline)
• Journal impact factors should NEVER be used to assess impact of
researchers / groups etc
Brief bibliography
General
• Broadus, R. N., “Towards a definition of Bibliometrics” Scientometrics,
vol. 12, nos 5-6, (1987) 373-379 @
www.springerlink.com/content/v111750n14086384/fulltext.pdf
• HEFCE papers, reports, papers and pilots on the use of bibliometrics
in the REF @ www.hefce.ac.uk/research/ref/Biblio/
H-Index
• Hirsch, J. E. (15 November 2005). "An index to quantify an individual's
scientific research output". PNAS 102 (46): 16569–16572 @
www.pnas.org/content/102/46/16569.abstract
• Ball, P. “Index aims for fair ranking of scientists”, Nature 2005 Aug 18
436: 900
Brief bibliography
Journal Impact Factors and the JCR
• The Thomson Reuters Impact Factor (originally published in the Current
Contents print editions June 20, 1994) @
http://thomsonreuters.com/products_services/science/free/essays/impact_factor/
• Garfield, E. "The agony and the ecstasy: the history and meaning of the
Journal Impact Factor“ Paper at the International Congress on Peer Review
And Biomedical Publication, Chicago, September 16, 2005 @
http://garfield.library.upenn.edu/papers/jifchicago2005.pdf
Want more…
• Wikipedia entries on the following topics include useful bibliographies: the
h-index, journal impact factors and the Eigenfactor
• A Google Scholar (http://scholar.google.co.uk/) search will return many useful
articles including subject studies on the use of bibliometrics
Help & info
•See our Bibliometrics
guide at
http://ox.libguides.com/
bibliometrics