Transcript Folie 1
Altmetrics for large, multidisciplinary research groups A case study of the Leibniz Association Alexandra Jobmann (IPN) & Isabella Peters (ZBW) Anita Eppelin (ZB MED), Christian Hoffmann (Universität St. Gallen), Sylvia Künne (IfW), & Gabriele Wollnik-Korn (ZB MED) Bibliometrics impact 3 footprints Seite 2 Altmetrics 2 1 1 impact 1 1 footprints Seite 3 Motivation for study 1. Initiatives that demand for new approaches in research evaluation (e.g., DORA) 2. Leibniz Association’s evaluation guidelines ask for appropriate public outreach and engagement in public discourse how to measure? 3. Research showed significant disciplinary differences: coverage and impact (Haustein & Siebenlist, 2011; Haustein et al., 2013; Holmberg & Thelwall, 2013; Mohammadi & Thelwall, 2013) Seite 4 Research Questions 1. Where and to what extent are the publications of the institutions of the Leibniz Association covered on social media platforms? 2. What impact do publications of the members of the Leibniz Association have on users (i.e., altmetrics)? 3. What tools can be used to assess research impact? What challenges might occur? Seite 5 Methods 1. Webometric Analyst for the collection of missing DOIs via Crossref 2. Checked DOIs and retrieved DOIs 3. ImpactStory for the collection of DOI-based altmetrics data (e.g., Twitter mentions, Mendeley readers) Seite 6 Data • Disciplines of the Leibniz Association 1. humanities and educational research 2. economics, social sciences, spatial research 3. life sciences 4. mathematics, natural sciences, engineering 5. environmental sciences • • • Articles of 12 institutes Correct DOIs Altmetrics 2.834 1.762 (62%) 1.739 (99%) 2-3 institutes of each discipline Articles in conferences/ journals and book chapters Publication years: 2011, 2012 Seite 7 Results 6000 350 5000 300 250 4000 200 3000 150 2000 tweets Mendeley readers • Mendeley attracts readers across disciplines • Enviromental Science reluctantly uses Twitter 100 1000 50 0 0 A: Humanities and B: Economics, Social Educational Research Sciences, Spatial (n=128) Research (n=387) Mendeley readers C: Life Sciences (n=454) D: Mathematics, Natural Sciences, Engineering (n=429) E: Environmental Sciences (n=341) tweets (Altmetrics.com) Seite 8 Results • Social media use is discipline-specific 16 14 12 10 8 6 4 2 0 A: Humanities and Educational Research (n=128) blog B: Economics, Social C: Life Sciences (n=454) Sciences, Spatial Research (n=387) Facebook D: Mathematics, Natural E: Environmental Sciences Sciences, Engineering (n=341) (n=429) Google+ f1000 Seite 9 Results • Where do disciplines find their readers? f1000 Mendeley readers tweets (Altmetrics.com) Google+ Facebook blog 0% 20% 40% 60% 80% 100% A: Humanities and Educational Research (n=128) B: Economics, Social Sciences, Spatial Research (n=387) C: Life Sciences (n=454) D: Mathematics, Natural Sciences, Engineering (n=429) E: Environmental Sciences (n=341) Seite 10 Results • Altmetrics can complement missing data (e.g., life sciences) 50 120 40 100 30 80 60 20 40 10 0 20 articles PubMed Central citations 0 Mendeley readers citations 140 20 18 16 14 12 10 8 6 4 2 0 institute 2 120 100 80 60 40 20 articles PubMed Central citations 0 Mendeley readers Seite 11 readers 160 60 citations 180 readers 70 institute 1 Results • Institutes from the same discipline (e.g., life sciences) find readers on different platforms 9 300 8 250 7 200 5 150 4 3 tweets 6 100 2 50 1 0 0 C1 blog C2 Facebook Google + tweets (Altmetrics.com) Seite 12 Lessons Learned • Chosen tools determine quality of data • Tools and altmetrics providers change settings • Chosen identifiers affect data • PubMedID is more popular than DOI • Missing or erroneous identfiers in social media • Multiple identifiers for one publication Underestimation of real numbers Collection of publication data • Missing DOIs on institutes‘ websites • Double-entry of publication on websites • Carry out data download at the same time Seite 13 Lessons Learned • Aggregated numbers may give wrong picture (e.g., discipline basis) • Sum html views: 2,447 (n=2) - Sum readers: 921 (n=76) 60 1800 1600 50 readers 40 1200 1000 30 800 20 600 400 10 0 Html views 1400 200 articles of institute A1 (humanities and educational research) Mendeley readers 0 PLoS html views Seite 14 Lessons Learned • Mendeley is the platform which covers a substantial amount of papers and shows reasonable user activity • Look for good coverage/ usage ratio • However, some disciplines prefer other platforms • Get to know the community preferences • Respect reader/ community choices • Altmetrics should not substitute, but can complement citation data • Comparability of altmetrics not given – same situation as in traditional citation analysis Seite 15 Thank you! Alexandra Jobmann, IPN [email protected] Prof. Dr. Isabella Peters, ZBW [email protected] Seite 16 References • Haustein, S., & Siebenlist, T. (2011). Applying social bookmarking data to evaluate journal usage. Journal of Informetrics, 5(3), 446–457. • Haustein, S., Peters, I., Bar-Ilan, J., Priem, J., Shema, H., & Terliesner, J. (2013). Coverage and adoption of altmetrics sources in the bibliometric community. In Proceedings of the 14th International Society of Scientometrics and Informetrics Conference, Vienna, Austria, Vol. 1 (pp. 468-483). Retrieved from http://www.issi2013.org/Images/ISSI_Proceedings_Volume_I.pdf • Holmberg, K., & Thelwall, M. (2013). Disciplinary differences in Twitter scholarly communication. In Proceedings of the 14th International Society of Scientometrics and Informetrics Conference, Vienna, Austria, Vol. 1 (pp. 567-582). Retrieved from http://www.issi2013.org/Images/ISSI_Proceedings_Volume_I.pdf • Mohammadi, E. & Thelwall, M. (2013). Assessing the Mendeley readership of social sciences and humanities research. In Proceedings of the 14th International Society of Scientometrics and Informetrics Conference, Vienna, Austria, Vol. 1 (pp. 200-2014). Retrieved from http://www.issi2013.org/Images/ISSI_Proceedings_Volume_I.pdf Seite 17