Transcript Dias nummer 1

Knowledge organization in
classical databases
Keynote speech in Krakow May 19. 2014
Birger Hjørland,
Royal School of Library and Information Science
(RSLIS)
Introduction
• We have knowledge organization (KO) in
physical libraries (for book shelving as well as
for catalogs)
• We have KO in subject bibliographies, which
from the 1960s became online information
systems. I name these the “classical databases”
• We have the Internet with search engines such
as Google.
17-07-2015
Dias 2
Introduction
This presentation focuses on the classical
databases. It does so, because I believe that
these databases are the most important ones
from the perspective of KO as well as library
and information science (LIS) in general.
I do not find that book shelving and library
catalogs have the same importance as finding
tools as have search engines and classical
databases (this function is not important
enough to justify an academic discipline devoted
to KO).
17-07-2015
Dias 3
Introduction
Internet engines are extremely important for
many people and many tasks. They are
important to study – also from the perspective
of LIS/KO.
They do not, however, provide sufficient control
over what is found. Such a control is in my
opinion better established in classical databases.
Also mainstream IR-research (computer
science) does not seem to aim at providing full
control to users.
17-07-2015
Dias 4
Introduction
Therefore, this presentation will make a case for
(1) The important role of human expertise in
searching (expert searchers and “information
literate” users)
(2) The role of library and information science
and KO in the design and use of classical
databases.
(3) The importance of Boolean search models
17-07-2015
Dias 5
Introduction
An underlying issue seems to concern the kind
of retrieval system for which one should aim.
It is suggested that Julian Warner’s (2010)
differentiation between
(a) computer science traditions, aiming at
automatically transforming queries into
(ranked) sets of relevant documents, and
(b) an older library-orientated tradition aiming
at increasing the “selection power” of users
seems to be important.
17-07-2015
Dias 6
Classical databases
By “classical databases” I mean the typical
bibliographical databases found in database
hosts such as Dialog (which unfortunately
terminated November 1, 2013) and ProQuest.
Well known examples are EconLit, MEDLINE,
PsycInfo and Science Citation Index/Web of
Science.
(Voss, 2013, presents different types of
databases, including “record databases”, p. 79)
17-07-2015
Dias 7
Classical databases
Levels of KO in classical databases:
1. The selection of documents to be indexed in
the database.
2. The design of the bibliographic record in fields
and subfields
3. The design of thesauri and other forms of
controlled vocabularies constructed by LISpersonnel
4. The actual indexing of documents in the
database
5. The organization of records in sets by the user
(based on the Boolean retrieval model)
17-07-2015
Dias 8
Classical databases
It is my point that none of these levels should
be understood as neutral technical tasks, but
should be recognized as tasks requiring taking a
stand in relation to different values, paradigms
and conceptions of the field (for example,
medicine).
(1) Although a database such as MEDLINE is
very comprehensive (>20 million records) it is
still a selection based on some notions of
medicine, in particular concerning the coverage
of fringe fields (also its formal limitation to
journal articles has implications).
17-07-2015
Dias 9
Classical databases
(2) The record structure is very complicated in
many databases (allowing the database host
and the user to make important choices of
which elements may be used as subject access
points).
Again, a given structure is not neutral, but
tends to facilitate some queries at the cost of
other queries.
Users in Evidence Based Medicine (EBM), for
example, have expressed the need to be able to
identify systematic reviews in MEDLINE (that
might be done in a separate field, or in an
existing field such as document type).
17-07-2015
Dias 10
Classical databases
(3) Controlled vocabularies cannot once and for
all provide a neutral and objective connection
between concepts. As expressed in this quote:
While unitary documentary languages ensure a
maximum of mutual understanding [. . .], they do so
by legitimizing a particular ideological and
sociopolitical worldview, and by silencing other
meanings, voices, and ways of knowing [. . .].
Unitary documentary languages embody a belief in
the existence of a unified body of knowledge. They
express a belief in the possibility to capture reality
isomorphically in "information," and presuppose a
neutral ground from which to judge the truth-value of
different theories. (Tuominen et al., 2003)
17-07-2015
Dias 11
Classical databases
(4) The actual indexing of documents is not just
a mechanical process of “matching” terms in
documents with descriptors in a thesaurus. It is
an interpretative process in which the relevance
of the document and the meaning of its terms
are translated into systems terminology in a
way that anticipates its fruitfulness.
(5) The organization of records in sets by the
user is again a creative process utilizing all
available possibilities and depending on the
user’s knowledge of the genres and conventions
used in the literature in the domain – as well as
his or her knowledge of the thesaurus etc.
17-07-2015
Dias 12
Classical databases
To the degree that classical databases are based
on less fruitful philosophical perspectives (or
lack of qualifications and subject knowledge),
there should be room for improvement.
Classical databases are, however, challenged by
search engines such as Google as well as by
basic assumptions driving information retrieval
research in computer science.
I argue, however, that these challenges are not
based on a proper scientific or theoretical basis:
The approach used in classical databases is still
superior for serious tasks.
17-07-2015
Dias 13
Challenges from IR
There are different arguments raised against
classical databases, among them are:
(1) classical databases are difficult to use for
non-experts
(2) classical databases are expensive to develop
and maintain
(3) the Boolean search model is weak compared
to other retrieval models
17-07-2015
Dias 14
Challenges from IR
Concerning (1). This is true. This is also the
reason we educate information specialists to
provide expert searching and to teach
“information literacy” (IL).
By the way: IL is in my opinion a field that is
secondary to and depending on deep knowledge
on searching and KO. If you are not expert in
searching and KO you are not able to tell
whether the user’s behavior reflects IL or not!
Concerning (2). Of course classical databases
are expensive. But serious users need to have
control over their search. This should be an
option when scholarliness is taken seriously.
17-07-2015
Dias 15
Challenges from IR
Concerning (3), Boolean search is often
considered a less efficient IR-model. The
criticism seems rather massive. Van Rijsbergen
considers the Boolean retrieval a tool “so ‘blunt’
that it is almost useless” (Rijsbergen, 1991, p.
85).
Larson (2010, p. 2557) lists seven problems
with Boolean searches. Beside being difficult to
use, his main arguments are that it is either too
restrictive or too inclusive. In Hjørland (in
press) I argue that this is false.
17-07-2015
Dias 16
Challenges from IR
Larson argued: ”A search for “A OR B OR C” will
retrieve any record with at least one of the
terms, but no priority is given to those that
include more than one of the terms”.”
My answer is: If a search for “A OR B OR C”
turns out to be too inclusive, the searcher may
just search (A OR B) AND (A OR C) AND (B OR
C). Many additional strategies are available to
increase precision (again dependent on the
content and structure of the database). Larson’s
criticism is therefore wrong.
17-07-2015
Dias 17
Challenges from IR
Boolean retrieval is a kind of exact match
retrieval, which is defined in the following way:
• query specifies precise retrieval criteria
• every document either matches or fails to
match query
• result is a set of documents.
Opposed to best match retrieval:
• query describes good or “best” matching
document
• result is a ranked list of documents
• result may include estimate (from Croft, 1998)
17-07-2015
Dias 18
Challenges from IR
I do not find that the arguments and findings, which
have been raised against the Boolean retrieval model
are convincing.
Also I believe that this issue is of utmost importance
for KO and LIS. The relevance of doing research and
teaching in which human searchers have full control
of what is found and what is not found is the core
issue for our field. I see “exact match retrieval” as a
cornerstone in this endeavor.
17-07-2015
Dias 19
Challenges from IR
Julian Warner (2010) found that the computer
tradition is based on the basic assumption which
may broadly be termed “query transformation”,
and implies that a user’s verbal query –
formulated before the start of the search – is
transformed by an information system into a set
of documents or bibliographical records.
Warner finds that the underlying methodology
tends to reify the concept of relevance and that
the underlying indexing philosophy in the
searched material is neglected and is just seen
as given. Finally he finds that this approach
contains an implicit teleology aiming at the
construction of a perfect system.
17-07-2015
Dias 20
Challenges from IR
Contrary to the dominant paradigm of the
computer orientated tradition, Warner presents
a tradition that is based more in library science
and the practice of indexing. This tradition is far
older, but less influential today. In his opinion
there are especially two valuable elements in
this tradition. The first is an explicit priority of
“selection power” that is the users’ possibilities
to make relevant distinctions during a search.
The second is the recognition of the need for
human labour to create this selection power
17-07-2015
Dias 21
Challenges from IR
I find that the “classical databases” based on
“exact match” principles are the tools that
represent the most advanced form of selection
power ever developed in relation to
bibliographical searches.
But how is this to be evaluated? What are the
empirical findings?
17-07-2015
Dias 22
Evaluating search systems
In medicine there is an approach termed
“evidence based medicine” and a document type
and genre termed “systematic review”.
In order to evaluate if a certain treatment
should be preferred, the “evidence” is
considered. All documents contributing to
deciding the issue should be found and taken
into account.
Literature searches and databases are
taken very seriously. Often teams of highly
trained doctors and information specialists use
many months to search the relevant documents.
No cost is spared in the best reviews.
17-07-2015
Dias 23
Evaluating search systems
By implication, the documents reported in a
high quality systematic review, may be
considered a gold standard which may serve
as key evaluating databases, indexing quality,
search techniques etc.
This evaluation method and its basis in scientific
criteria of what should be found are very
different from the criteria usually applied in
computer science and LIS (which mostly use,
for example, user based criteria).
17-07-2015
Dias 24
Evaluating search systems
A recent study by Gehanno, Rollin & Darmoni
(2013) found that of 738 original studies
included in their gold standard database all were
found in Google Scholar.
“The coverage of GS [Google Scholar] for the
studies included in the systematic reviews is
100%. If the authors of the 29 systematic
reviews had used only GS, no reference would
have been missed. With some improvement in
the research options to increase its precision,
GS could become the leading bibliographic
database in medicine and could be used alone
for systematic reviews.”
17-07-2015
Dias 25
Evaluating search systems
This finding of course challenges my main
thesis. However, it has to be confirmed by other
studies. An most importantly: It still recognizes
that although Google Scholar has a better
coverage compared to MEDLINE, it cannot today
provide the same search opportunities (the
relevant documents are in GS but cannot be
identified in subject searches).
Karimi et al. (2009) found that in the case of
systematic reviews wherein comprehensiveness
takes precedence over efficiency (precision),
Boolean queries offer advantages over rankingbased approaches.
17-07-2015
Dias 26
Evaluating search systems
Karimi et al. (2010, p. 10) also examined the
effectiveness of exploiting MEDLINE metadata in
searching and concluded that “separate indexing
for metadata is not helpful in retrieval for the
systematic review search task. These results are
in line with other studies that show little or no
benefit in using MEDLINE metadata for
biomedical information retrieval”.
However, Karimi et al.’s experiment is difficult
to interpret from the description given, and the
references seem to have been misread. The
empirical evidence in my opinion still supports
the value of MEDLINE’s metadata.
17-07-2015
Dias 27
Conclusion
If we, with Warner (2010), accept that an
important goal in the LIS tradition has been to
improve the searchers’ selection power, then
exact match models should be recognized. A
great range of “recall devices” and “precision
devices” exists. In order to utilize such devices
in optimal ways, the user has to know about the
databases, search facilities, documents, genres,
languages, paradigms, etc. in which he or she is
searching. This should be part of what is often
termed “information literacy”.
17-07-2015
Dias 28
Conclusion
It is this kind of knowledge that is clearly
alternative to the mainstream IR-traditions and
which is also more naturally connected with LIS
and the online-searching community. Given the
Boolean model, the goal for KO may be
understood as improving the bibliographical
records in ways which improve searchers’
selection power.
The case is not just that users are able to
perform more informed searches by applying
exact match techniques. The problem seems
also to be that information scientists and
computer scientists are brought to a state of
confusion by best match systems.
17-07-2015
Dias 29
Conclusion
As Robertson (2000) has argued, “The term
precision device itself was coined in the former
context [set-based retrieval]: whether it is a
valid concept for the latter [ranked-based
retrieval] is not obvious. To put it even more
directly […] Precision devices aren't what they
used to be!”
If alternatives to Boolean searches mean that
we are not able to improve the precision of
searches in either practice or theory (because
we have no clear understanding of the
underlying mechanisms and their mutual
interactions), we are in a deeply problematic
situation.
17-07-2015
Dias 30
Thank you for your attention!
(interested persons can have a copy of the proofs of my
full paper to be published in JASIST
Email: [email protected] )
17-07-2015
Dias 31
References
Croft, B. (1998). Exact-match retrieval. PowerPoint
presentation. (©Howard Turtle and Bruce Croft). Retrieved
from: http://www.cpe.ku.ac.th/~arnon/Mirror/ir-p/ir6/
Gehanno JF, Rollin L, Darmoni S.(2013) Is the coverage of
Google Scholar enough to be used alone for systematic
reviews. BMC Medical Informatics and Decision Making
2013, 13:7 Retrieved January 6, 2014 from
http://www.biomedcentral.com/1472-6947/13/7
Hjørland, B. (2011). Evidence based practice. An analysis
based on the philosophy of science. Journal of the American
Society for Information Science and Technology, 62(7),
1301–1310.
Hjørland, B. (2014). Knowledge organization in classical
databases: The case for Boolean retrieval and human
decision-making during search. Journal of the Association
for Information Science and Technology, in print.
17-07-2015
Dias 32
References
Karimi, S., Pohl, S., Zobel, J., & Scholer, F. (2009). The
challenge of high recall in biomedical systematic search. In
Proceedings of the 3rd InternationalWorkshop on Data and
Text Mining in Bioinformatics, November 2009 (pp. 89–92).
Hong Kong, China: ACM. Retrieved from:
http://www.nicta.com.au/pub?doc=3413
Karimi, S., Pohl, S., Scholer, F., Cavedon, L., & Zobel, J.
(2010). Boolean versus ranked querying for biomedical
systematic reviews. BMC Medical Informatics and Decision
Making, 10, article 58. Retrieved from:
http://www.biomedcentral.com/1472-6947/10/58
17-07-2015
Dias 33
References
Larson, R.R. (2010). Information retrieval systems. In M.J.
Bates & M.N. Maack (Eds.), Encyclopedia of library and
information sciences (3rd ed.) (IV, 2553–2563). New York:
CRC Press.
17-07-2015
Dias 34
References
Rijsbergen, C.J. van (1991). Towards an information logic. In
Proceedings of SIGIR-89, 12th ACM Conference on Research
and Development in Information Retrieval, pages 77–86,
Cambridge, MA. Retrieved January 6, 2014 from
http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.
93.5156&rep=rep1&type=pdf
Robertson, S. (2000). Salton award lecture: On theoretical
arguments in information retrieval. Talk given at SIGIR
2000, Athens, July 2000. ACM SIGIR Forum 34(1), 1–10.
Retrieved 2012-07-18 from:
http://www.sigir.org/forum/S2000/salton%5Flecture.pdf
17-07-2015
Dias 35
References
Tuominen, K., Talja, S., & Savolainen, R. (2003).
Multiperspective digital libraries: the implications of
constructionism for the development of digital libraries.
Journal of the American Society for Information Science and
Technology, 54(6), 561–569.
17-07-2015
Dias 36
References
Voss, Jakob (2013). Describing data patterns: A general
deconstruction of metadata standards. Berlin: Humboldt
University (Dissertation). http://edoc.hu-berlin.de/
Warner, J. (2010). Human information retrieval. Cambridge,
MA: MIT Press.
17-07-2015
Dias 37
END
17-07-2015
Dias 38