Data Collection and Beyond

Assessment at the U.Va. Library

Jim Self University of Virginia Library James Madison University February 10, 2012

Today’s Program

 A short history of stats at U.Va.

 Adding qualitative info to the mix  Focusing on the user experience  Designing for your user  Questions, comments

Why Measure or Count?

 To evaluate  To compare  To improve

When not to collect data

 When someone has already done it  When the data will not be useful  When the results do not justify the costs

Making Data Meaningful

 Summarize  Compare  Analyze  Present

6

“…

but to suppose that the facts, once established in all their fullness, will ‘speak for themselves

’

is an illusion

.”

Carl Becker Annual Address of the President of the American Historical Association, 1931

Data at the U.Va. Library

 Long involvement with ARL statistics  Statistical compilations  Comparisons with peers  Mining existing records  Unit cost calculations  Calculation of turnaround times  Customer surveys  Staff surveys  Performance indicators and targets  Ongoing involvement with VIVA statistics

Management Information Services

 MIS committee formed in 1992  Evolved into a department 1996-2000  Coordinates collection of statistics  Publishes annual statistical report  Conducts surveys  Coordinates assessment  Resource for management and staff

Reasons to Collect Data “

No questions arise more frequently in the mind of the progressive librarian than these:  Is this method the best?

 Is our practice adapted to secure the most effective administration?

 Are we up to the standards set by similar institutions? The success with which we answer them depends much on the success of our administration”

J.T. Gerould, 1906

Message from the 21

st

Century

 Use data to IMPROVE  Services  Collections  Processes  Performance  Etc., etc.

 Don’t use data to preserve the status quo  Karin Wittenborg, ALA Annual, Orlando 2004

U.Va. Library Surveys

    Faculty  1993, 1996, 2000, 2004  Response rates 59% to 70% Students  1994, 1998, 2001, 2005  Separate analysis for grads and undergrads  Response rates 43% to 63% LibQUAL+ 2006  Response rates 14% to 24% Annual Surveys 2008 +    Student samples One third of faculty Response rates 29% to 47%

Sharing the Data

www.lib.virginia.edu/mis

4,0 3,5 3,0 5,0 U.Va. Library Services and Resources: Overall Importance by Group 4,5 Collections Reference/Info Srvs Ugrads Grads Instruction Faculty Library Spaces

14 Overall Satisfaction University of Virginia Library 1993-2010 4,60 4,40 4,20 4,00

4,09 4,01 3,96 Faculty Grads 4,27 4,17 4,15 Undergrads

3,80 1993/94 1996/98 2000/01 2004/05 2008/10

Weekly Visits to a U.Va. Physical Library 1993-2011 90% 70%

84% 79% 69% Undergrads 67% Grads

50%

38% Faculty

30%

21%

10% 93/94 96/98 00/01 04/05 07/09 10/11

U.Va: Monitoring the Online Catalog Satisfaction with VIRGO since 1993 5,00 4,50 4,00 3,50 3,00

Faculty Ugrads Grads

Using Data at U.Va          Additional resources for the science libraries (1994+) Redefinition of collection development (1996) Initiative to improve shelving (1999) Clemons Library open 24 hours (2000) Additional resources for the Fine Arts Library (2000) Support for transition from print to e-journals (2004) New and improved study space (2005-06) Increased appreciation of the role of journals (2007) Re-design of main floor of Clemons Library (2008)   Enhanced usability of discovery tools (2010) Dedicated space for graduate students (2011)

“To assess, in general is to determine the importance, size or value of; to evaluate.”

SPEC Kit 303: Library Assessment, December 2007.

Why Should Libraries Do Assessment?

 To learn about our communities  To respond to the needs of our users  To improve our programs and services  To support the goals of the communities http://www.keepmoneylocal.org/2010/03/buy-local/

An Assessment-Focused Institution:

 Collects, analyzes and uses data for management, program development, and decision-making  Emphasizes ongoing communication with customers, opportunities for collaboration, qualitative measures and circular process of continuous improvement

Common Library Measures

          Revenues and expenditures Community size Staff size and salaries Collections size Circulation and other collections usage Interlibrary borrowing and lending Instruction sessions Service measures (hours, patrons, reference use) Computer use detail Web site usage

What is missing?

The Challenge For Libraries

 Traditional statistics:  emphasize inputs – how big and how many  do not tell our story  do not measure service quality  do not include collaborative partners and services  Need measurements from the perspective of the user  Need the culture and the skills to answer a basic question:

Time For A New Assessment Model

User-Centered Library

   View services & activities through the eyes of users Users determine quality Our services and resources add value to the user

Assess the Value Provided the Community Through

     Online resources and services In-library resources and services In person services outside the library Partnership with others involved in learning Contribution to learning, research and life

Moving forward…

 Continued collection of quantitative data  Expansion of the use of qualitative information  Creation of a user experience team  Integration of web design, development and assessment