Telling the story

Qualitative approaches to measuring the performance of emerging library services Professor Peter Brophy Centre for Research in Library & Information Management (CERLIM) Manchester Metropolitan University United Kingdom 1

Library goodness

• How good is this library?

– Quality – Effectiveness – Efficiency – Economy – Equity • How much good does this library do?

– Impact and outcomes – Value Orr (1973) 2

The future library (2001)….

• The invisible intermediary – “Every reader his/her information object” • The memory institution – “The collected wisdom of the centuries” • The learning centre – “Where people meet and exchange ideas” • The community resource – “The street-corner, drop-in, citizens’ centre” • The traditional (but hybrid) library – “The legacy of the past and the promise of the future” BUT ALL ARE FOCUSED ON THE USER 3

Image Three men in a boat

Libraries: a threat to their users?

4

“Library” users in …

1: The networked landscape – Personal search replaces ‘ask a librarian’ – Global search of the global library • Google, flickr, del.icio.us; PubMed; Google Scholar – If there’s no response in 3 seconds, try elsewhere • •

Then: Resources scarce, attention abundant Now: Attention scarce, resources abundant

[Dempsey] – ‘Phone a friend’ • Social networking services, wikis, …. [SCONUL VAMP] – Where’s the text?

• Discovery to delivery (d2d) essential – Satisficing • •

Then: what is the best source of authoritative information?

Now: which is the most readily accessible source of adequate information?

– Network tools which are used are those embedded in workflow 5

“Library” users in …

2: The virtual learning environment – Self-contained online environments • all you need to study xology in one place 6

You log into WebCT

7

and that’s where you stay

“Library” users in …

• The virtual learning environment – Self-contained • all you need to study xology in one place – Mandated by institutions • If you study here you have to use it – Mandated by teachers • Used for course development • Used for delivery • Used for debate • Used for assessment – Inhospitable to embedding the library – Environment is embedded in learning workflow (‘learnflow’) 9

“Library” users in …

3: Research practice – Big ‘science’ shifting to computational use of huge and dynamic datasets (satellites, particle accelerators, human genome) • Astronomy, physics, medicine, geology • Linguistics, history • Cross-disciplinary – Need is for ‘curation’ throughout the lifecycle – Librarians challenged re. skills/knowledge/organisation – Data is embedded in research practice 10

“Library” users in …

4: Scholarly publication – Pressure for open access through ‘self’ publication from some academics – Libraries prominent in running repositories – Take-up by academic authors patchy – Because it’s not embedded in their workflow?

11

“Library” users in …

5: Personal creativity – Shift from ‘information chain’ to ‘creative circle’ – d2d [discovery to delivery] to c2c – [creation to creation]?

Create Publish Learn Enquire Enjoy View  Discuss Acquire 12

Supporting creativity

Helping people tell their own stories – and publishing them for others to see – and their responses 13

Supporting creativity

Interpret Validate Identify/ Acquire Preserv e Enquire Publish View Create  Describ e Acquire Provide Share Learn Discuss Affirm Enjoy Enable Assess Design Teach Disseminate/ Communicat e 14

What do all these have in common?

• The E word:

Embedding

• ‘Users’ need library services which are embedded in their – Workflows – Learnflows – Leisureflows – Lifeflows 15

Emerging new/revised roles for the librarian

Supporting learning, research and life where they occur – and surfacing that support within the users’ lifeflows 16

What’s distinctive about libraries?

1. They provide access to paid-for content 2.

They have ‘intelligence’ about users Information intelligence User universe User population

User interface

Library as intermediary

Source interface

Information population Information universe User intelligence 17

So, getting close to the users is critical

– Engage with the users’ languages and their processes of learning, research, leisure and … life – Because they are not going to come to you 18

Linguistic codes

• The way a ‘language’ divides up the world – it’s take on ‘how things are’ • e.g. the way scientists speak of ‘matter’ having 16 dimensions • or the fact that Hopi Indians have a conception of time that can’t be explained in English 19

Language games

“Let us imagine a language ...The language is meant to serve for communication between a builder A and an assistant B. A is building with building-stones; there are blocks, pillars, slabs and beams. B has to pass the stones, and that in the order in which A needs them. For this purpose they use a language consisting of the words 'block', 'pillar', 'slab', 'beam'. A calls them out; - B brings the stone which he has learnt to bring at such and-such a call. - Conceive of this as a complete primitive language.” [Wittgenstein:

Philosophical Investigations

] 20

Being part of the dialogue

Learning the language game – Terminology – Concepts – Shared meaning Playing the language game – Credibility – Trust 21

Academic disciplines as social spaces

“An academic discipline ……is not primarily content, in the sense of facts and principles. It is rather primarily a lived and historically changing set of distinctive social practices. It is in these practices that ‘content’ is generated, debated and transformed via certain distinctive ways of thinking, talking, valuing, acting and, often, writing and reading.” Gee, 2003 22

Learning as social practice

“Learning is a social process that occurs through interpersonal interaction within a cooperative context. Individuals, working together, construct shared understandings and knowledge.” Johnson et al.

23

To share in the these social networks, you need …

• A deep understanding of how the discipline or activity is practised • leading to credibility as a contributor to the enterprise • and thus to the standing and peer regard to supply solutions • The following could be well-placed – Subject librarian – Community librarian – Knowledge officer 24

So how do we measure performance?

"Sometimes I lie awake at night, and I ask, 'Where have I gone wrong?' Then a voice says to me, ‘This is going to take more than one night’." Charles Schultz/Peanuts 25

So how do we measure performance?

• How do you measure the performance of any social system?

– Ethnographic approaches • Living in the lifeflow – Moderated by independent, expert assessment • Assisted self-assessment – Painting rich pictures • Use of narrative and story 26

Ethnography

• Culture. The focus of concern is on the shared culture of groups of people, examining and seeking to understand the shared patterns of behaviour, values, norms and standards.

• Holism. The focus is on the whole system being observed, rather than on an isolated part. Meaning and purpose can only be discerned within the context of the whole system.

• In-depth studies. Ethnography is not interested in the data collected by counting or even by questioning, but relies on ‘living in’ the group that is being investigated.

• Chronology. Ethnography is interested in change, in how people and societies alter and reinvent themselves over time. [after Sarantakos, 1998] 27

Methodological requirements

1.

2.

3.

• • Speak the language Investigators judgements are not credible unless they themselves have credibility with the target group Therefore an element of self-evaluation is needed • • Understand the problem space • Investigators must understand the potential of library & information services so as to be able to offer improved solutions to users’ problems • Therefore LIS expertise is also needed Maintain independence Findings are not credible unless an investigator with no vested interest can sign them off Therefore an element of independent evaluation is needed 28

What kinds of methods?

• Observation • Conversation • Self-reflection • Textual analysis

Internal

• Questionnaires • Interviews • Focus groups

External

29

How do we construct meaning when faced with complexity?

• Use of narrative “One reason why we live in a soup of narratives, why narratives permeate our lives and understanding, is that resorting to narratives is the way in which we have learned to cope with our world … Even while scientists and schoolteachers have been telling us to abandon these unscientific approaches, and adopt linear abstract thinking, the human race has used its common sense and stubbornly – to some extent surrepticiously – stuck with narratives as the most usable tool to cope with complexity” Stephen Denning (2001) 30

Unveiling meaning through narrative

You could make an objective list of the actions you performed over the last week, but if it were simply a “factual” account, it would not mean anything. But if you told us what you had done in the last week, not only would your story acquire meaning, but in telling it, both you the narrator and we the listeners would be compelled to reflect on it in order to gain a greater understanding of what had gone on. ……Just as history does not exist in nature, but is created in the telling, so, too autobiography and the medical case history emerge out of transactions which mean they are at the same time both less and more than the ‘facts’ of the case.

Greenhalgh & Hurwitz 31

Unveiling meaning through narrative

“Steadily increasing recognition of the importance of narrative in mainstream management is now inevitable. …… Narrative thinking is contributing to an emerging view of organizations that more accurately reflects not only the traditional structural, process-oriented, control based aspects of an organization but also the living, flowing aspects of organizations – where talking, thinking, dreaming, feeling human beings work and play and talk and laugh and cry with each other, in a way that is organic and self adjusting and naturally innovative.” (Denning in Brown et al.) 32