Transcript Twitter: http://www.ukoln.ac.uk/web-focus/events/workshops/eim-2010-11/ #ukolneim Evidence, Impact, Metrics Institutional Web Services: Evidence for Their Value Welcome Brian Kelly UKOLN University of Bath Bath, UK Acceptable Use Policy Recording/broadcasting of this talk, discussing the content using.

Twitter:
http://www.ukoln.ac.uk/web-focus/events/workshops/eim-2010-11/
#ukolneim
Evidence, Impact, Metrics
Institutional Web Services:
Evidence for Their Value
Welcome
Brian Kelly
UKOLN
University of Bath
Bath, UK
Acceptable Use Policy
Recording/broadcasting of this talk,
discussing the content using Twitter, blogs,
etc. is permitted providing distractions to
others is minimised.
Sharing ideas and discussions during day in
encouraged – please flag confidential items
Blogs:
http://ukwebfocus.wordpress.com/
http://blogs.ukoln.ac.uk/evidence-impact-metrics/
UKOLN is supported by:
This work is licensed under a Attribution-NonCommercialShareAlike 2.0 licence (but note caveat)
Introduction
About Me
Brian Kelly:
• UK Web Focus: a national advisory post
• Long-standing Web evangelist
• Established Institutional Web Management
Workshop (IWMW) in 1997
• Based at UKOLN at the University of Bath
UKOLN:
• A national centre of expertise in digital
information management
• Funded by the JISC
• A JISC Innovation Support Centre
2
Introduction
3
Evidence, Impact, Metrics
Evidence, Impact, Metrics (EIM):
• UKOLN activity funded by the JISC
Aims to:
• Explore ways of gathering evidence which
can demonstrate the impact of services
and devise appropriate metrics to support
such work
By:
• A number of events
• The EIM Blog
• Your input and involvement
Introduction
4
Evidence, Impact, Metrics
Areas of Work
• Institutional Web services (today):
How can we demonstrate the
effectiveness and impact of institutional
Web services? What metrics are relevant?
What concerns may there be?
• The Social Web:
How can we demonstrate the
effectiveness and impact of Social Web
services? What metrics are relevant?
What concerns may there be?
• ….
The Context
•The need to demonstrate value in light of
Government cuts
5
Introduction
6
Workshop Aims
By the end of the workshop we should have:
• Discussed ways of gathering evidence to
demonstrate the value and impact of
institutional Web sites.
• Identified ways that the value and impact can be
measured and articulated to third parties
• Explored ways in which such measures can be
standardised in order to provide level of
consistency, whilst acknowledging institutional
diversity
We should also:
• Be in a position to develop and/or commission
work to gather and interpret evidence
Introduction
Concerns
7
Possible Concerns
Talk of ‘value’, ‘impact’ & ‘metrics’:
• Will lead to league tables (& relegation)
• Will take effort away from service delivery
• Will lead to ‘gaming the system’
Thoughts:
• Are these concerns valid, or are they naive?
• How do we use impact indicators in appropriate
ways e.g.
 Understand usage patterns
 Inform developments
 Identify shortcomings
 Learn from (& help) one’s peers
 Support transparency
Prisoner’s Dilemma
Concerns
We can all benefit by:
• Sharing best practices
• Sharing examples of flawed
approaches
• Agreeing on ways forward
But concerns that:
• We might be seen to be doing
badly
• We want to learnt from others,
but not share our experiences
The aim is to demonstrate the value of institutional Web
services across UK HE. That is a clear unambiguous
message to sell (to e.g. THE)
8
Introduction
Draft Programme
Time
Content
10.00-10.30
Arrivals
10.30-10.45
Introduction to EIM Work [BK]
Subject to change
About You
9
11.00-11.30
Breakout Session 1: Identifying Key Areas
11.30-12.00
Case Study: University of Strathclyde [Colin Hamilton]
12.00-13.00
Breakout Session 2: Producing a Specification
13.00-14.00
Lunch
14.00-14.20
Report Back: Presenting the Specification
14.20-14.50
Institutional Dashboards [RS]
14.50-15.00
Discussion
15.00-15.15
Coffee
15.15-15.45
Action Plans: Personal, Institutional & National
15.45-16.00
Conclusions and What next?
Introduction
E
10
About You
In small groups cover:
• Where you work
• What you do
• Your areas of interest
• Specific issues would you like to see
addressed
• Expertise you can contribute
Introduction
11
Questions
Any questions or comments?
Next
12.00-13.00
E
Producing a Specification
In your groups:
• Agree on clearly-defined area(s)
• Develop the outline of a specification
which developers can use to produce a
system for demonstrating impact & value
You will need to:
• Identify data requirements
• Explain how data is to be processed
• Describe the expected outputs and how
they can be used
• Summarise concerns, difficulties, etc.
12
12.00-13.00
Questions
Any questions or comments?
13
Next
14.00-14.20
P
Presenting Your Specification
Present your outline specification so that
developers or external agencies are able to
understand your requirements and make a
proposal to implement a solution
14
14.00-14.20
Questions
Any questions or comments?
15
Next
15.15-15.45
D
Action Plans
What Next? Building on Today’s Work
• What will you do next?
• What should your institution do next?
• What related work needs to be done at a
regional or national level?
What Next? Additional Areas of Work
• What additional areas could the EIM
activity address?
16
15.15-15.45
Conclusions
What can we conclude?
17