Transcript Twitter: http://www.ukoln.ac.uk/web-focus/events/workshops/eim-2011-07/ #ukolneim Metrics and Social Web Services: Quantitative Evidence for their Use & Impact Surveying Our Landscape From Top To Bottom Brian Kelly UKOLN University of Bath Bath, UK Acceptable.

Twitter:
http://www.ukoln.ac.uk/web-focus/events/workshops/eim-2011-07/
#ukolneim
Metrics and Social Web Services:
Quantitative Evidence for their Use & Impact
Surveying Our Landscape From
Top To Bottom
Brian Kelly
UKOLN
University of Bath
Bath, UK
Acceptable Use Policy
Recording this talk, taking photos,
discussing the talk using Twitter,
blogs, etc. is permitted but please try
to minimise distractions to others.
Blog:
http://ukwebfocus.wordpress.com/
Twitter:
@briankelly
@ukwebfocus
UKOLN is supported by:
This work is licensed under a Attribution-NonCommercialShareAlike 2.0 licence (but note caveat)
Idea from Cameron Neylon
You are free to:
copy, share, adapt or re-mix;
Off the record
questions and
comments
should be
flagged
photograph, film or broadcast;
blog, live-blog or post video of
Evidence &
comments
this presentation provided that:
You attribute the work to its author and respect the rights
and licences associated with its components.
22
Slide Concept by Cameron Neylon, who has waived all copyright and related or neighbouring rights. This slide only CCZero.
Social Media Icons adapted with permission from originals by Christopher Ross. Original images are available under GPL at:
http://www.thisismyurl.com/free-downloads/15-free-speech-bubble-icons-for-popular-websites
Services being considered:
Twitter, Facebook,
YouTube, iTunes,
Technorati, Wikio &
Slideshare
3
Twitter: Personal Trends
I thought I used Twitter
regularly ever since signing
up in March 2007.
Evidence from Tweetstats:
• Significant use began in
Jan 2008 (100+ tweets)
• No use in Mar-Jun 2008
Reality:
• Significant use began in Apr 2008 at MW 2008 conf
• Needed community in order to gain benefits
Implications:
• Our memories may be incorrect
• Data may be wrong (due to Twitter downtime??)
4
Institutional Use of Twitter
Questions:
• What can commercial social media
analytics services tell us about institutional
Use of Twitter?
• What can we learn from the approaches
the services take?
• What are their strengths and
weaknesses?
• Can we / should we develop alternatives?
5
Social Media Analytic Summaries
Social media analytic summaries for Russell
Group Universities
Outliers may be interesting
6
Peerindex Comparisons (1)
Peerindex’s ‘topic
fingerprint’ for
Oxford / Cambridge:
• Similar profiles
• Oxford covers
news, politics &
history
• Cambridge
covers
technologies
7
Peerindex and Klout group comparisons
for Russell Group Unis available
(data used in this talk is available freely for reuse under
CC0 licence)
Peerindex Comparisons (2)
Peerindex’s ‘topic fingerprint’
for:
@psychemedia (blue blob)
& @mweller (grey outline)
Example used of four established
bloggers / Twitterers
8
Do such comparisons
tell us anything useful?
Peerindex For Participants
Peerindex table of
rankings for attendees.
Is this:
• A self-fulfilling echochamber / a clique?
• Easily gamed (e.g.
by signing up to
service)?
Scenario 1: Research funding determined by flawed metrics but Unis still
headhunt researchers with valuable RAE scores
Scenario 2: Social Media metrics flawed but Unis headhunt marketing /
outreach people with valuable scores
Scenario 3: “IMAGINE a world in which we are assigned a number that
indicates how influential we are. This number would help determine whether
9you receive a job …It’s not science fiction. It’s happening to millions”
Slideshare: Personal Use
Steve Wheeler tweet on Aug 2010:
Ironically there were 15 people in my
audience for this Web 3.0 slideshow
but >12,000 people have since
viewed it http://bit.ly/cPfjjP
In Dec 2010 blog post reported:
• ~16K views &1.5K embedded views
Blog post suggested:
• Value of embeddable resources
(Slideshare sets OERs free)
• Ways of comparing value of
experiences of 15 local audience &
22.8K (now) remote viewers
10
Slideshare at Events (1 of 2)
IWMW events used
Slideshare since
2006.
Statistics show:
• Overall nos of
views (<1,000
live audience)
and suggest:
But does decline since 2008 peak indicate: • Potential for
amplification of
• Slideshare is past it (use Slideboom,
parallel sessions
…)?
• Viewing slides is passe
• Potential value
• Benefits of longstanding availability?
of aggregation of
slides
11
Slideshare at Events (2 of 2)
Statistics of most
popular slides
suggest:
• Popularity of
non-HE
speakers’ slides
• Popularity of
slides not in
corporate house
style
12
This has some positive correlation with
experiences of local audience
Could this inform selection of speakers in
future?
Slideshare: Lessons
Lessons:
• Benefits of providing embeddable resources to
enhance access
• Informing policy: encouraging workshop facilitators
to upload their slides
• Challenging orthodoxy: value of house style
• Apparent decline in numbers may reflect longevity
of availability of resources
Implications:
• What can the evidence and reflections tell us
about teaching and learning repositories?
13
Slideshare Premium account provides richer statistics (trend
analysis?) indicates value of business intelligence for $19/month.
Metrics For Personal Blogs
Metadata for use in
comparison with similar blogs
Or blogs.ouseful.info
Top 3% of all blogs
Top 13% of technology blogs
Or top 0.2% of all blogs
Top 1.3% of technology blogs
14
Technorati gives
authority & ranking for
1.2 M blogs
Authority measures
blog's standing &
influence on scale of 01000 (high good).
Ranking given for
Technorati Authority of
all sites (low good)
Rank of 3,045 means
top 0.2% of all blogs.
Technology rank of 376
(of 30,133) means top
1.2%
Metrics For Personal Blogs
Why the drop?
Note usefulness of keywords
15
For blogs with a
“JISC” keyword:
MASHe:
Top 0.8%
Top 2% technology
JISC AM Team:
Top 6%
May be value in:
• Comparisons
(learning)
• Aggregation
(value of HE)
But …
Wikio
MASHe also ranked highly but not listed as
Technology blog (so 4 of top 100 blogs here!)
Wikio also provides metrics
for (registered) blogs
List of top technology blogs
shown
Together with display of
trends for ranking, nos. of
posts, links and backlinks,
OUseful had peak at
#18 in Nov 2008
What happened from
Jun-Sep 2010?
16
Discussion
17
Criticisms: Flawed Approaches
• Blogs have variety of purposes so inappropriate
to reduce to 1 dimension
Response:
• A degree can be seen as I/2.1/2.2 or 3
• Some blogs have similar purpose and can be
compared
Criticisms: Technical
• Results can be volatile & may be skewed by blog
technologies; migration of blogs; …
Response:
• Could improve
• Not significantly different from University rating?
Facebook Trends in UK HE
Development since 2007:
• Blog post in 2007 reported 7 UK Unis on Facebook:
Aston, Cardiff, Kent and the University of Central
Lancashire (UCLan)
• Jun 2008 survey reported on 8 most popular UK HE
pages: OU top with 7,539 fans
• Aug 2010 survey showed
growth over 2 years (OU 380%;
Cardiff 13,000% and others
from 400-700%)
Result of decision to be proactive in Fb marketing? If so,
did it provide an ROI?
18
Facebook
Survey of Russell
Group Unis Fb use:
•
•
•
•
Branded Fb URL: 7
Fb page: 7
Fb group: 1
No easily found Fb
presence: 5
• Neglected or unofficial Fb
presence: 1
• Range of ‘likes’: 2,047 –
137,395
Recent report on ‘ROI of
Fb advertising’ suggested
“1 Facebook fan = 20
additional visits to your
website”
19
How would we know if Cardiff‘s 20,035 Likes have provided an ROI?
YouTube Edu
Survey of YouTube
Edu accounts:
• 1M+ views at
Coventry Uni
• Lack of comments
Example of stats available
20
iTunes Edu
Numeric data not readily
obtained from iTunes Edu
Is this:
• To be welcomed as an
escape from beancounting?
• Of concern since
gathering qualitative
evidence can be costly?
Blog post analysed how services marketed: “lots of content”;
“free content”; “available on iPods”; …
21
Conclusions
To conclude quantitative evidence:
• Can inform policy decisions
• Can influence individual decision-making
The data:
• Needs to be gathered!
• May be flawed and should be questioned
The data analysis:
• May inform (and surprise)
• May be flawed
But:
• Links with value & financial ROI not yet
established
22
Questions
Any questions or comments?
23