Transcript “The coolest thing to do with your data will be thought of by someone else” Dave Pattern Library Systems Manager University of Huddersfield [email protected] www.daveyp.com/blog/ Graham Stone Information.

“The coolest thing to do with your data
will be thought of by someone else”
Dave Pattern
Library Systems Manager
University of Huddersfield
[email protected]
www.daveyp.com/blog/
Graham Stone
Information Resources Manager
University of Huddersfield
[email protected]
@Graham_Stone
Using Usage Data…
• …to improve existing services
• …to gain insights into user behaviour
• …to measure the impact of the library
– do the students who use the library the
most get the highest grades?
Defining Using Usage
JISC Activity Data Programme
• User activity data
– record of a user’s actions on a website or software
system or other relevant institutional service
• Attention data
– record of what a user has viewed on a website or
software system or other relevant institutional
service
http://bit.ly/g6S5wH
Defining Usage Data
some library examples...
• Circulation Transactions
– user A borrowed this book on 23/Jan/2011
• E-Resource Usage
– user B logged in and accessed ScienceDirect 126
times during 2009
• Entry Stats
– user C entered the library 12 times in Dec/2009
Usage Data at Huddersfield
5
Keyword Cloud
based on keyword search data
6
Borrowing Suggestions
based on circulation data
7
Borrowing Suggestions
based on user’s recent loans
8
Course Level New Book Lists
analysis of course loans & Dewey
9
Search Suggestions
based on keyword search data
10
Guided Keyword Searches
based on keyword search data
11
Journal Suggestions
based on link resolver logs
12
The Impact of Serendipity
13
Trends in Borrowing
unique titles circ’ing per acad. year
unique titles (bib#s)
80,000
70,000
60,000
50,000
40,000
30,000
20,000
10,000
0
2000 2001 2002 2003 2004 2005 2006 2007 2008 2009
14
Trends in Borrowing
# of items borrowed per acad. year
average number of items borrowed per academic year
25
17.10
20
1
6.5
.3
.9
6
1
2001
2002
17
18.92
7.6
.1
7
1
.1
7
1
2003
2004
2005
1
.7
.4
8
1
2006
2007
18
9.0
.5
9
1
2008
2009
1
15
10
2000
15
Discussing Open Data
JISC TILE Project meeting (Jun 2008)
16
Sharing Data
Why should I release my data?
• “The coolest thing to do with your data
will be thought of by someone else.”
– Rufus Pollack (Open Knowledge Foundation)
http://m.okfn.org/files/talks/xtech_2007/
Sharing Data
Why should I release my data?
• “Absolutely, couldn’t agree more.
The point is not so much whether this
statement might be true or not, so
much as what it does to your thinking
and planning if you decide to take it as
an article of faith.”
– Paul Walk (UKOLN)
http://bit.ly/9ao3c4
Sharing Data
usage data release (Dec 2008)
19
Sharing Data
BA Multimedia Design
20
Non & Low Usage Project
24
Non & Low Usage Project
digging deeper into data since 2005!
25
Non & Low Use Project
digging deeper into data
Measuring Library Impact
is there a link between usage & grade?
27
Non & Low Use Project
Results
• Not a cause and effect
relationship
• Never proven
statistically significant
• Potential for
collaboration on future
projects
http://www.flickr.com/photos/atoach/3344411469/
JISC Activity Data Call
Library Impact Data Project
To prove the hypothesis that…
“There is a statistically significant
correlation across a number of
universities between library activity data
and student attainment”
Project reports
http://library.hud.ac.uk/blogs/projects/lidp/
• Themed posts
–
–
–
–
–
–
The Project Plan
Hypothesis
Users
Benefits
Technical and Standards
Licensing & reuse of software
and data
– Wins and fails (lessons along
the way)
– Final post
Data requirements
• For each student who graduated in a given year, the
following data was required:
– Final grade achieved
– Number of books borrowed
– Number of times e-resources were accessed
– Number of times each student entered the library,
e.g. via a turnstile system that requires identity
card access
– School/Faculty
Legal issues
• Consultation with JISC Legal, University legal officer and data
protection officer
• Ensured that any identifying information is excluded before it
is handled for analysis
• Excluded any small courses to prevent identification of
individuals e.g. where a course has less than 35 students
and/or fewer than 5 of a specific degree level
• Received guidance from the Using OpenURL Activity Data
Data issues
• Anticipated that there may be problems in getting enough
data to make the project viable
– Potential partners were asked to confirm that they could
provide at least 2 of the 3 measures of usage as well as
student grades
– Huddersfield has provided definitions on the data required
and the form the data can be accepted in
• Some partners have already run into some issues with data
collection, but it is felt that there is still enough information to
prove the hypothesis one way or another
Can we prove the hypothesis?
• Not quite!
• Due to the data not
being continuous, a
correlation cannot be
calculated
http://www.flickr.com/photos/26015375@N06/3715306069/
Further statistical tests
• Running a Kruskal-Wallis
test
– to indicate whether there is a
difference between values
e.g. between levels of eresource usage across degree
results
– THEN we analyse the data
visually to check which
variables to compare
What we think we can prove
• That the relationship and
variance means that you
can believe what you see
• And you can believe it
across a range of data, e.g.
subjects
• So library usage does
impact on students
attainment
100
83
80
70
65
60
60
45
41
40
30
27
20
0
1
2:1
MetaLib logins
2:2
books borrowed
3
Remember the disclaimer!
Not a cause and effect relationship
400
Library Impact Data Project
book loans inc. renewals (2009/10)
350
300
279
250
249
242
230
200
236
218
180
170
150
156
151
135
124
126
100
105
100
50
0
First class honours
Upper second class honours Lower second class honours
2007/08
2008/09
2009/10
Third class honours
Pass - degree awarded
without honours
Library Impact Data Project
book loans & Athens (2009/10)
41
Library Impact Data Project
library PC logins & visits (2009/10)
42
Linking back to non/low usage
• Our research shows that for books and e-resource
usage, there appears to be a statistical significance
across all partner libraries
• If we know that there is a link between usage and
attainment
– We can link this back to non/low usage
Measuring Library Impact
2008/9 – library visits
15.5% of students who gained
a 1st never visited the library
34% of students who gained
a 3rd never visited the library
44
Measuring Library Impact
2008/9 – MetaLib usage
70% of those who gained a
3rd logged in to e-resources
20 times or less over 3 years
10.5% of students who gained a
1st logged in more than 180 times
45
Measuring Library Impact
2008/9 – book loans
15% of students who gained
a 1st never borrowed a book
34% of students who gained
a 3rd never borrowed a book
46
Profiling non/low users
• Flesh out themes from the focus groups
– to advise on areas to work on
• Check the amount and type of contact subject teams have
had with the specific courses
– to compare library teaching hours to attainment
• Baseline questionnaire or exercise for new students
– To establish the level of information literacy skills for new
students
• Target our users by concentrating staff resources at the right
point
Next steps for the project
• Pull out any themes from the focus groups
• Release the data on an Open Data Commons Licence
• Release a toolkit to help others benchmark their data
Further work
• Do cuts to the information budget mean that attainment will
fall?
• Can we add more value by better use of resources?
• What about gender and socio-economic background?
• Can we link books borrowed on reading lists to attainment
• What about VLE use?
• Interest from ACRL (USA), University of Wollongong
(Australia), the Netherlands and Denmark
Acknowledgements
• Dave Pattern and Bryony Ramsden
• Phil Adams, Leo Appleton, Iain Baird, Polly Dawes,
Regina Ferguson, Pia Krogh, Marie Letzgus, Dominic
Marsh, Habby Matharoo, Kate Newell, Sarah
Robbins, Paul Stainthorp
Thank you
• http://library.hud.ac.uk/blogs/projects/lidp/
• http://eprints.hud.ac.uk/10949/
This work is licensed under a
Creative Commons Attribution 3.0
Unported License