Transcript Let’s Chat: Using Data to Champion Virtual Reference Services Let’s Chat: Using Data to Champion Virtual Reference Services 2008 Virtual Poster Session.

Let’s Chat: Using Data to Champion Virtual Reference Services
Let’s Chat: Using Data to Champion
Virtual Reference Services
2008 Virtual Poster Session. American Library Association Annual
Conference 2008. Evaluation of Virtual Reference Services
(MERS).
Presenters:
 Susan Gardner
Assistant Reference Coordinator, University of Southern California
Libraries
[email protected]
Shahla Bahavar
Reference Coordinator, University of Southern California Libraries
[email protected]

Gardner & Bahavar 2008
Page 1 of 27 pages
Let’s Chat: Using Data to Champion Virtual Reference Services
Abstract Part 1
This poster session chronicles the collection and presentation of virtual
reference statistics to persuade administrators and librarians at the University
of Southern California (USC) to support chat reference.
USC is part of the OCLC QuestionPoint academic chat collaborative. However,
some librarians and administrators felt that the service was not used often and
that the quality of service was poor.
Gardner & Bahavar 2008
Page 2 of 27 pages
Let’s Chat: Using Data to Champion Virtual Reference Services
Abstract Part 2
The coordinators put together the following data and presented it to librarians
and administrators:

QuestionPoint chat and email statistics over the past 2 years, including trend
lines

Data on chat patron satisfaction surveys

List of the peak times that USC patrons were using the chat service

Data illustrating that USC patrons asked more questions than the average
institution in the collaborative

Data on chat shifts that needed more librarians
Gardner & Bahavar 2008
Page 3 of 27 pages
Let’s Chat: Using Data to Champion Virtual Reference Services
Abstract Part 3
As a result, the coordinators were able to get the support necessary to

increase USC’s chat hours in the collaborative

gear chat hours more strategically towards USC’s peak user hours

plan for the piloting of a new IM (instant messaging) chat service
Gardner & Bahavar 2008
Page 4 of 27 pages
Let’s Chat: Using Data to Champion Virtual Reference Services
Overview of USC and Chat

USC contributes to the QuestionPoint OCLC Academic Reference Cooperative by answering
questions from both local and non-USC patrons.

In Fall 2007, USC served 7 hours per week in the cooperative, answering questions from both
USC and non-USC patrons:
- Tuesdays 1-5pm
- Thursdays 2-5pm

In return, USC received 24/7 coverage for their patrons during all hours they were not staffing the
cooperative.

Additional coverage to USC-only patrons was given during a pilot project “Subject Chat”
(monitoring only the USC queue) 4 hours per week:
- Mondays, 10-11am
- Tuesdays, 11-noon
- Wednesdays, 10-11am
- Thursdays, 10-11am
Gardner & Bahavar 2008
Page 5 of 27 pages
Let’s Chat: Using Data to Champion Virtual Reference Services
Reference Coordinators Face Chat
Challenges

Some older librarians uncomfortable with chat

Perception by some of the library faculty that chat was
a waste of time and not the most efficient way to reach our patrons
unpopular with our users
3. handled poorly by non-USC librarians (poor quality)
1.
2.

Busy schedules: finding time to staff chat in addition to the time
necessary to staff all twelve of our reference service points

Occasional technical problems
Gardner & Bahavar 2008
Page 6 of 27 pages
Let’s Chat: Using Data to Champion Virtual Reference Services
General Trends in USC Reference:
Statistics, FY 2005-FY 2007
Fiscal Year
Total Transactions
Basic
Advanced
In Person
Phone
Email
In Person
QuestionPoint
Phone
Email
Chat
2005
49024
26670
5432
3288
8453
2230
2514
437
2006
54443
27715
6017
2998
9977
3381
3660
695
2007
51530
23603
5194
2416
8505
3406
5447
2959
Gardner & Bahavar 2008
Page 7 of 27 pages
Ju
l
A -05
ug
S -05
ep
O 05
ct
N 05
o
v
D -05
e
cJa 0 5
n
F -06
eb
M 06
ar
A 06
pr
M -06
ay
Ju 06
n0
Ju 6
l-0
A 6
ug
S -06
ep
O 06
ct
N 06
o
v
D -06
e
cJa 0 6
n
F -07
eb
M 07
ar
A 07
pr
M -07
ay
Ju 07
nJu 0 7
l
A -07
ug
S -07
ep
O 07
ct
-0
7
Let’s Chat: Using Data to Champion Virtual Reference Services
Email Decreasing, Chat Increasing
July 2005-October 2007
350
Email
Chat
300
250
200
150
100
50
0
Gardner & Bahavar 2008
Page 8 of 27 pages
Let’s Chat: Using Data to Champion Virtual Reference Services
Email Trend Line, July 2005-Oct. 2007
180
160
140
120
100
Email
Linear (Email)
80
60
40
20
Gardner & Bahavar 2008
-0
7
S
ep
Ju
l-0
7
07
M
ay
-
7
07
M
ar
-
Ja
n0
ov
-0
6
N
-0
6
S
ep
Ju
l-0
6
06
M
ay
-
06
M
ar
-
6
Ja
n0
ov
-0
5
N
-0
5
S
ep
Ju
l-0
5
0
Page 9 of 27 pages
Ju
A l-0
ug 5
S -0
ep 5
O -0
ct 5
N -05
o
D v-0
e 5
c
Ja -0
n 5
F -0
eb 6
M -0
a 6
A r-0
p 6
M r-0
ay 6
Ju -06
n
Ju -06
A l-0
ug 6
S -0
ep 6
O -0
c 6
N t-0
o 6
D v-0
e 6
c
Ja -06
n
F -0
eb 7
M -0
ar 7
A -07
p
M r-0
ay 7
Ju -07
nJu 0 7
A l-0
ug 7
S -0
ep 7
O -0
ct 7
-0
7
Let’s Chat: Using Data to Champion Virtual Reference Services
Chat Trend Line, July 2005-Oct. 2007
350
300
250
200
Email
Chat
150
Linear (Chat)
100
50
0
Gardner & Bahavar 2008
Page 10 of 27 pages
Let’s Chat: Using Data to Champion Virtual Reference Services
Results: Chat User Satisfaction Surveys,
July 2005-Oct. 2007 (based on 242 responses)
Do you feel the librarian fully understood your
questions?
No
6%
N/A
12%
Yes
82%
Gardner & Bahavar 2008
Page 11 of 27 pages
Let’s Chat: Using Data to Champion Virtual Reference Services
Results: Chat User Satisfaction Surveys,
July 2005-Oct. 2007
Were you satisfied with the answer you received to your
reference question?
No Answer
5%
Not Satisfied
9%
Somewhat
14%
Satisfied
72%
Gardner & Bahavar 2008
Page 12 of 27 pages
Let’s Chat: Using Data to Champion Virtual Reference Services
Results: Chat User Satisfaction Surveys,
July 2005-Oct. 2007
Is this the first time you have used this service?
No Answer
5%
No
22%
Yes
73%
Gardner & Bahavar 2008
Page 13 of 27 pages
Let’s Chat: Using Data to Champion Virtual Reference Services
Results: Chat User Satisfaction Surveys,
July 2005-Oct. 2007
Will you use this service again?
Never N/A
2% 4%
Maybe
17%
Very Likely
77%
Gardner & Bahavar 2008
Page 14 of 27 pages
Let’s Chat: Using Data to Champion Virtual Reference Services
Results: Chat User Satisfaction Surveys,
July 2005-Oct. 2007
The ease of using this online reference is?
N/A
4%
Very Diff.
1%
Difficult
2%
Easy
25%
Very Easy
68%
Gardner & Bahavar 2008
Page 15 of 27 pages
Let’s Chat: Using Data to Champion Virtual Reference Services
Results: Chat User Satisfaction Surveys,
July 2005-Oct. 2007
Comments:

Overall, 78% of the free-form, optional comments were positive
Sample Positive Comments:




“This is the most positive librarian I’ve ever encountered. Thanks
for the great customer service”
“The best service, so stress relieving for students”
“It’s easy and fun to work with”
“Wow, that was very easy. I was surprised that the librarian wasn’t
from USC, but she still seemed knowledgeable and very helpful.
Thank you for such a great service”
Gardner & Bahavar 2008
Page 16 of 27 pages
Let’s Chat: Using Data to Champion Virtual Reference Services
Virtual Reference Data Presented to
Library Faculty and Administration
Meeting #1: Reference Forum attended by 25 library faculty and staff,
and 1 administrator
Discussion topics:





Pilot projects
New reference initiatives
Review policies and procedures (email & chat), especially email
referrals and chat follow up questions (email responses)
Reference statistics: analysis and possible service changes
?? Whatever issues you want to talk about!
Gardner & Bahavar 2008
Page 17 of 27 pages
Let’s Chat: Using Data to Champion Virtual Reference Services
Virtual Reference Data Presented to
Library Faculty and Administration
Meeting #2 Team Leaders & Coordinators (TLC)





Team Leaders for Science and Engineering, Social Sciences, Arts & Humanities, &
Undergraduate
Coordinators of Reference, Instruction, Collection Development, Outreach, Technical
Services, Access Services
Director for Research & Collections
20 minute Reference Presentation covering material similar to Meeting #1
Eleven attendees were at the meeting
Gardner & Bahavar 2008
Page 18 of 27 pages
Let’s Chat: Using Data to Champion Virtual Reference Services
From OCLC QuestionPoint Administrator

During January – October 2007, USC librarians were only picking up 27.68% of the
total number of chat questions that came from USC patrons. This is below average:
the Cooperative average was 60%, and the percentage for the Greater Western
Library Alliance (Iowa State, U Arkansas, U Hawaii, and USC) was 75.51%. USC is
not handling as many questions as are asked by USC patrons

“USC has a popular chat service and receives far more chat requests than other
GWLA libraries. The students are using the service during non-traditional hours (late
at night, early in the morning) as well as during the ‘business’ day”

Suggestions:
-If USC librarians are unable to pick up non-USC patrons because too many other
academic librarians are online, consider changing your shifts (we have many
shifts that are understaffed) so USC can contribute their fair share
-
-
-If possible, staff more hours, especially times when many USC students are using
the service
Gardner & Bahavar 2008
Page 19 of 27 pages
Let’s Chat: Using Data to Champion Virtual Reference Services
Top 5 Chat Shifts

USC students asked the most questions during shifts between 2-8pm on weekdays
(between January-October 2007).
1.
#1: 6-7pm (139 questions)
2.
#2: 7-8pm (135 questions)
3.
#3: 2-3pm (121 questions)
4.
#4: 4-5pm (111 questions)
5.
#5: 5-6pm (110 questions)
Gardner & Bahavar 2008
Page 20 of 27 pages
Let’s Chat: Using Data to Champion Virtual Reference Services
QP Academic Consortium Needs Librarians
Day
1st Timespan Needed
2nd Timespan Needed
Monday
Midnight-7am
5pm-midnight
Tuesday
Midnight-6am
4pm-midnight
Wednesday
Midnight-3am
3pm-midnight
Thursday
Midnight-7am
3pm-midnight
Friday
Midnight-6am
3pm-midnight
Saturday
ALL TIMES
Sunday
ALL TIMES
Gardner & Bahavar 2008
Page 21 of 27 pages
Let’s Chat: Using Data to Champion Virtual Reference Services
Proposal #1: Increase Chat Hours

Proposed additional chat hours on QuestionPoint, spring semester,
through Subject Chat Pilot Project
-Mondays, 4-6pm (Psychology, Sociology, Gerontology)
-Tuesdays, 6-8pm (Anthropology, Communication)
-Wednesdays, 6-8pm (Science & Engineering)

Subject-based chat emerged to simultaneously increase local chat
coverage within the consortium while offering customized chat service for
selected disciplines.
-Would also be covering the local USC chat queue in QuestionPoint, not just
subject-specific questions- for an extra 6 hours of coverage per week.

Proposed Spring “IM Chat Pilot” Project Hours

Librarians cover while they sit at the Reference Desk in the Leavey
Undergraduate Library:
-Monday-Thursdays, 12-5pm (will default to QuestionPoint
cooperative chat when librarians are unavailable)
Gardner & Bahavar 2008
Page 22 of 27 pages
Let’s Chat: Using Data to Champion Virtual Reference Services
Outcome #1: Spring 2008 Chat Coverage
Increased
i.
ii.
iii.
iv.
Library faculty were influenced by our chat statistics.
We had many more volunteers than anticipated to do subject
chat as well as support for the IM pilot project.
We increased monitoring of the USC-only queue in
QuestionPoint by an additional 14 hours by offering subjectbased chat in 13 different disciplines.

These librarians also take any general questions from the
USC queue, thus taking the burden off the cooperative
somewhat.
Number of hours we added are during times that QP needed
additional librarians, and/or during times our students asked the
most questions.
Gardner & Bahavar 2008
Page 23 of 27 pages
Let’s Chat: Using Data to Champion Virtual Reference Services
New Subject Chat Schedule

American Studies & Ethnicity
Mondays
3-5pm

Anthropology
Tuesdays
6-8pm

Business
Wednesdays
2-4pm

Communication
Tuesdays
6-8pm

Gerontology
Mondays
4-6pm

English/Literature
Wednesdays
Fridays
7-8pm
7-8pm

Linguistics
Mondays
10-noon

Philosophy
Mondays
10-noon

Psychology
Mondays
5-6pm

Religion
Mondays
10-noon

Science & Engineering
Mondays
7-9pm
Wednesdays
6-8pm

Sociology
Mondays
4-5pm

U.S. & Canadian History
Mondays
3-5pm
Gardner & Bahavar 2008
Page 24 of 27 pages
Let’s Chat: Using Data to Champion Virtual Reference Services
Outcome #2: IM Pilot Project Launched

Created IM accounts on multiple chat interfaces

Set up chat widgets on various web pages (Leavey Library page, Ask-ALibrarian page, e-resources page)

Monitored with one system, Pidgin, that records the transcripts

Piloted with Leavey librarians and staff who are already serving scheduled
shifts at the Leavey Undergraduate Reference desk

When no IM is available, the link leads patrons to the QuestionPoint
cooperative 24/7 chat service

Expanded service hours from original proposal to include both
staff and librarians, Mon.-Thurs. 10am-5pm and Fri. 10am-2pm.
Gardner & Bahavar 2008
Page 25 of 27 pages
Let’s Chat: Using Data to Champion Virtual Reference Services
Future Goals: Continuing to Enhance Virtual
Reference Services

Continue monitoring virtual reference statistics
 trends
 statistics for IM and subject chat pilot projects
 cooperative statistics
 user satisfaction surveys

Do our own transcript analysis of the content of all IM and cooperative QuestionPoint
chat transactions for quality control

Develop further virtual reference training for library faculty and staff as a result of
analysis
Gardner & Bahavar 2008
Page 26 of 27 pages
Let’s Chat: Using Data to Champion Virtual Reference Services
Contact the Authors for More Information
Presenters:

Susan Gardner
Assistant Reference Coordinator, University of Southern California Libraries
[email protected]
Shahla Bahavar
Reference Coordinator, University of Southern California Libraries
[email protected]

Gardner & Bahavar 2008
Page 27 of 27 pages