Transcript Journals in the arts and humanities: their role and evaluation Professor Geoffrey Crossick Warden Goldsmiths, University of London.

Journals in the arts and
humanities: their role and
evaluation
Professor Geoffrey Crossick
Warden
Goldsmiths, University of London
2
3
Distinctive place of journals in
arts & humanities research
 Diversity of output
monographs, edited collections
edited texts, journal articles
practice outputs
 No clear hierarchy of esteem
amongst as well as within each
challenge for RAE and promotion panels
esp problems for practice outputs
4
Why do we want to know about
journals? Explore the terms
 Use of journals to indicate:
activity and productivity
impact - relevance and use
quality - peer judgment
5
Measurements and proxies
Activity &
Productivity
Quantitative
methods &
data
Impact
Quality
Tangible &
intangible
evidence
& data
Qualitative
methods & data
Increasing use of proxy metrics to infer impact and quality
Especially in context of changes to RAE
6
Why is it so hard to rank journals
or use citation data?
 Rank according to what criteria?
 Impact factors very difficult
 Citation behaviour is very different
not cumulative cf science
old work remains highly cited [See ISI list]
critical discourses as mode of research
citation not clear sign of quality/influence
the culture of the footnote
 Diverse outputs - not just paper-based
 Print output different arts & humanities
 Lower % paper outputs in ISI-Journals
7
Highest citations in ISI
humanities journals 2000
1 Karl Marx
2 Lenin
3 Shakespeare
4 Aristotle
5 The Bible
6 Plato
7 Freud
8 Chomsky
9 Hegel
10 Cicero
8
RAE submissions 1996 & 2001
Humanities, Languages and Arts
 Journal articles?
1996 sciences
engineering
social sciences
33%
90%
57%
42%
2001 - 37%
96%
78%
54%
 Books?
1996 - 51%
sciences
6%
engineering
8%
social sciences 32%
2001 - 52%
3%
5%
28%
 Book chapters
 Other
9
1996 - 4%
1996 - 11%
2001 2001 –
3%
9%
Journal publication & RAE quality
Journal articles as % outputs to RAE2001 by discipline
10
UoA
Law
3b
83
3a
71
4
61
5 5*
53 43
Asian studies
43
56
55
42 29
History
37
36
33
33 34
Art & design
7
7
10
Music
11
15
15
9
23 27
ISI journals & the arts & humanities
 Publication in ISI journals often small % overall
outputs RAE 2001
 Philosophy highest at 52%
 Library & information management 40%
 Most other subjects in 20%-29% range
 English & French just 21%
 Below 20% in Italian, Theology, Art & Design and
Middle Eastern & African Studies
11
Current projects on quality & journals
Issue of evaluating quality very current
So too is assessing standing of journals
But they’re not the same thing at all
And decreasingly so arts & humanities
Reflect on these:
 RAE and metrics
 European Science Foundation’s ERIH
 Humanities Indicators Project





12
RAE metrics
 Metrics-driven RAE
 ‘Neither citations nor RC income’
 AHRC expert group
 outputs but no proxies for their quality
 Post-2008: STEM cf rest of disciplines
 ‘robust indicators’ being sought – for STEM
 primarily HESA and bibliometric data
 arts & humanities 2013+? earlier impact
 consultation non-science 2009-10
 national bibliometrics consortium
13
European Reference Index for
the Humanities

European Science Foundation
“The ERIH lists will help to identify excellence
in Humanities scholarship & should prove
useful for the aggregate bench-marking of
national research systems…in determining
the international standing of the research
activity in a given field in a country.
However, as they stand, the lists are not a
bibliometric tool”
14
Methods & goals of ERIH
 Expert Panels in 15 disciplinary areas
 ESF member lists, iterative consultation
 Categorisation (lists emerging this year)
 A = high-ranking international level
 B = standard international level
 C = important local/regional level
 Resistance to hierarchy: why?
 Say above all to strengthen peer review
 How, if at all, will it be used?
15
Humanities Indicators Project
 American Academy of Arts & Sciences
 many variables on state of humanities
 Publications element:
 Focus is monograph publications data
no quality indicators sought
 No plans to look at journal publishing
 Making humanities count: the importance of
data
16
Arts & humanities journals:
the challenge of bibliometrics
 Is there a challenge?
 Does anyone want to do bibliometrics with arts
& humanities journals?
 Little interest UK or elsewhere
 In many ways for good reasons set out here
 Yet looming is RAE post-2013….
can we build ‘robust measures’ without them?
can it be done with them?
17