Transcript Measuring Information Architecture

Flexible Search and Navigation
using Faceted Metadata
Prof. Marti Hearst
University of California, Berkeley
Search Engines Meeting, April 2002
Research funded by:
NSF CAREER Grant, NSF IIS-9984741
1
The Flamenco Project Team
Ame Elliott
Jennifer English
Marti Hearst
Rashmi Sinha
Kirsten Swearingen
Ping Yee
http://bailando.sims.berkeley.edu/flamenco.html
2
Motivation
• Web search works well now
– Gets people to the appropriate web sites
– Finds starting points
• Web SITE search is NOT ok
– Results still overwhelming
– Not well-integrated with the information
architecture
– People prefer to follow links anyhow
3
Recent Study by Vividence Research
• Spring 2001, 69 web sites
– 70% eCommerce
– 31% Service
– 21% Content
– 2% Community
• The most common problems:
53%
32%
32%
27%
25%
15%
13%
had poorly organized search results
had poor information architecture
had slow performance
had cluttered home pages
had confusing labels
invasive registration
inconsistent navigation
4
Following Hyperlinks
• Works great when it is clear where to
go next
• Frustrating when the desired directions
are undetectable or unavailable
Free Text Search
• Can specify anything
• Can result in a disorganized mess
5
An Analogy
hypertext
text search
Wanted: An All TerTrain Vehicle!
6
Main Idea
• Integrate the search seamlessly into the
information architecture
• Use hierarchical metadata to
– Allow flexible navigation
– Provide query previews
– Organize search results
– Both expand and refine the search
7
The Challenges
• Users don’t like new search interfaces
• How to show lots more information
without overwhelming or confusing?
8
Main Idea
• Use metadata to show where to go next
– More flexible than canned hyperlinks
– Less complex than full search
– Help users see and return to what
happened previously
9
An Important Trend in Information
Architecture Design
• Generating web pages from databases
• Implications:
– Web sites can adapt to user actions
– Web sites can be instrumented
10
A Taxonomy of WebSites
high
Complexity
of Data
low
Web-based
Catalog Sites Information
Systems
WebPresence
Sites
ServiceOriented
Sites
low
high
Complexity of Applications
From: The (Short) Araneus Guide to Website development, by Mecca, et al,
Proceedings of WebDB’99, http://www-rocq.inria.fr/~cluet/WEBDB/procwebdb99.html
11
Faceted Metadata
12
Metadata: data about data
Facets: orthogonal categories
GeoRegion
+ Time/Date
+
Topic
13
Faceted Metadata: Biomedical
MeSH (Medical Subject Headings)
www.nlm.nih.org/mesh
14
Mesh Facets (one level expanded)
15
Questions we are trying to answer
•
•
•
•
How many facets are allowable?
Should facets be mixed and matched?
How much is too much?
Should hierarchies be progressively
revealed, tabbed, some combination?
• How should free-text search be
integrated?
16
How NOT to do it
• Yahoo uses faceted metadata poorly in
both their search results and in their
top-level directory
• They combine region + other
hierarchical facets in awkward ways
17
Yahoo’s use of facets
18
Yahoo’s use of facets
19
Yahoo’s use of facets
20
Yahoo’s use of facets

Where is Berkeley?
 College and University > Colleges and Universities
>United States > U > University of California > Campuses
> Berkeley
U.S.
States > California > Cities >Berkeley > Education >
College and University > Public > UC Berkeley
21
Problem with Metadata Previews as
Currently Used
– Hand edited, predefined
– Not tailored to task as it develops
– Not personalized
– Often not systematically integrated with
search, or within the information
architecture in general
22
Recipe Collection Examples
23
From soar.berkeley.edu (a poor example)
24
25
From www.epicurious.com (a good example)
26
27
28
29
Epicurious Metadata Usage
• Advantages
– Creates combinations of metadata on the fly
– Different metadata choices show the same
information in different ways
– Previews show how many recipes will result
– Easy to back up
– Supports several task types
• “Help me find a summer pasta,'' (ingredient type + event type),
• “How can I use an avocado in a salad?'' (ingredient type + dish
type),
• “How can I bake sea-bass'' (preparation type + ingredient type)
30
Metadata usage in Epicurious
Ingredient
Dish
Cuisine
Prepare
Recipe
31
Metadata usage in Epicurious
Ingredient
Dish
Cuisine
Prepare
Dish
Cuisine
Prepare
Select
I
32
Metadata usage in Epicurious
Ingredient
I
>
Dish
Cuisine
Prepare
Dish
Group by
Cuisine
Prepare
33
Metadata usage in Epicurious
Ingredient
I
>
Dish
Cuisine
Prepare
Dish
Cuisine
Group by
Prepare
34
Metadata usage in Epicurious
Ingredient
I
>
Dish
Cuisine
Prepare
Dish
Cuisine
Prepare
Cuisine
Group by
Prepare
I
Select
35
Epicurious Basic Search
Lacks integration with metadata
38
39
Epicurious: Usability Study
• People liked the browsing-style
metadata-based search and found it
helpful
• People sometimes preferred the
advanced search
– For more constrained tasks
– But zero results are frustrating
• People dissprefer the standard simple
search
40
Missing From Epicurious
• How to scale?
– Hierarchical facets
– Larger collection
• How to integrate search?
• How to allow expansion in addition to
refinement?
41
Metadata Interface for Image Search
42
Current Approaches to Image Search
• Visual Content and Cues, e.g.,
• QBIC (Flickner et al. ‘95)
• Blobworld (Carson et al. ‘99)
• Body Plans (Forsyth & Fleck ‘00)
– Color, texture, shape
– Move through a similarity space
• Keyword based
– Piction (Srihari ’91)
– WebSeek (Smith and Jain ’97)
– Google image search
43
Architects and City Planners
• Common activitie:
– Use images for inspiration
• Browsing during early stages of design
– Collage making, sketching, pinning up on walls
• This is different than illustrating powerpoint
• Maintain sketchbooks & shoeboxes of images
– Young professionals have ~500, older ~5k
• No formal organization scheme
– None of 10 architects interviewed about their
image collections used indexes
• Do not like to use computers to find images
44
The Collection
• ~40,000 images from the UCB
architecture slide library
• The current database and interface is
called SPIRO
• Very rich, faceted, hierarchical metadata
45
Architects’ Image Use
• Common activitie:
– Use images for inspiration
• Browsing during early stages of design
– Collage making, sketching, pinning up on walls
– This is different than illustrating powerpoint
• Maintain sketchbooks & shoeboxes of images
– Young professionals have ~500, older ~5k
• No formal organization scheme
– None of 10 architects interviewed about their image
collections used indexes
• Do not like to use computers to find images
46
Development Timeline
• Needs assessment.
– Interviewed architects and conducted contextual inquiries.
• Lo-fi prototyping.
– Showed paper prototype to 3 professional architects.
• Design / Study Round 1.
– Simple interactive version. Users liked metadata idea.
• Design / Study Round 2:
– Developed 4 different detailed versions; evaluated with 11
architects; results somewhat positive but many problems
identified. Matrix emerged as a good idea.
• Metadata revision.
– Compressed and simplified the metadata hierarchies
• Design / Study Round 3.
– New version based on results of Round 2
– Highly positive user response
47
The Interface
• Nine hierarchical facets
– Matrix
– SingleTree
• Chess metaphor
– Opening
– Middlegame
– Endgame
• Tightly Integrated Search
• Expand as well as Refine
• Intermediate pages for large categories
48
49
50
51
52
53
54
55
56
Usability Study on Round 3
• 19 participants
– Architecture/City Planning background
• Two versions of the interface
– Tree (one hierarchical facet at a time)
– Matrix (multiple hierarchical facets)
• Several tasks
• Subjective responses
– All highly positive
– Very strong desire to use the interface in future
– Will replace the current SPIRO interface
57
Study Tasks
1. High Constraint Search:
Find images with metadata assigned from 3 facets
(e.g., exterior views of temples in Lebanon)
1.1)
1.2)
1.3)
Start by using a Keyword Search
Start by Browsing (clicking a hyperlink)
Start by using method of choice
2. Low Constraint Search:
Find a low-constraint set of images (metadata in
one facet)
3. Specific Image Search:
Given a photograph and no other info, find the
same image in the collection
4. Browse for Images of Interest
58
Interface Evaluation
• Users rated Matrix more highly for:
–
–
–
–
Usefulness for design work
Seeing relationships between images
Flexibility
Power
• On all except “find this image” task, users
also rated the Matrix higher for:
– Feeling “on track” during search
– Feeling confident about having found all relevant
images
59
Overall Preferences:
Matrix vs. Tree
Matrix
Tree
Simple
search
(e.g.
images of
deserts)
Complex
search (e.g.
exteriors of
temples in
Lebanon)
Find
images
like this
one
OVERALL
PREFERENCE
13
14
16
16
5
4
3
3
60
User Comments - Matrix
• “Easier to pursue other queries from each individual
page”
• “Powerful at limiting and expanding result sets. Easy
to shift between searches.”
• “Keep better track of where I am located as well as
possible places to go from there.”
• “Left margin menu made it easy to view other
possible search queries, helped in trouble-shooting
research problems.”
• “Interface was friendlier, easier, more helpful.”
• “I understood the hierarchical relationships better.”
61
User Comments – Tree
• Pro
– “Simple”
– “More typical of other search engines I’d use”
– “Visually simpler and more intuitive…Matrix a bit
overwhelming with choices.”
• Con
– “I found SingleTree difficult to use when I had to refine my
search on a search topic which I was not familiar with. I
found myself guessing.”
– “SingleTree required more thought to use and to find specific
images.”
– “I do not trust my typng and spelling skills. I like having
categories.”
62
Task Completion Times
(Find Image is an artificial task: given a photo and
no other info, find it in the collection.)
63
When Given A Choice …
For each interface, one task allowed the
user to start with either a keyword
search or the hyperlinks.
3 chose to search in both interfaces
11 chose to browse in both interfaces
4 chose to search in Matrix, browse in Tree
1 chose to browse in Matrix, search in Tree
64
Feature Usage (%)
Refining
Use of Features to Refine Search
"More" in disambiguation
Disambiguate keyword search
Search within
Drill by clicking "All N items"
Matrix
Tree
Drill from large category
Drill from image detail
Drill in matrix
Drill above images
0.00%
5.00% 10.00% 15.00% 20.00% 25.00% 30.00%
65
Feature Usage –
Expanding / Starting Over
Use of Features to Expand Search / Start Over
Back
Search all, mid-task
Go back to start midsearch
Matrix
Tree
Expand from image
detail
Expand by clicking X
Expand search using
breadcrumbs
0.00%
5.00%
10.00%
15.00%
20.00%
25.00%
66
Feature Usage (%)
Types of Actions
Action Categories
Start over/backup
Arrange results
Matrix
Tree
Expand search
(increase # of results)
Refine search (reduce
# of results)
0.00%
10.00%
20.00%
30.00%
40.00%
50.00%
60.00%
67
Interface Evaluation
• Users rated Matrix more highly for:
–
–
–
–
Usefulness for design work
Seeing relationships between images
Flexibility
Power
• On all except “find this image” task, users
also rated the Matrix higher for:
– Feeling “on track” during search
– Feeling confident about having found all relevant
images
68
Application to Medline
69
Summary and Conclusions
70
Summary
• Two Usability Studies Completed
– Recipes: 13,000 items
– Architecture Images: 40,000 items
• Conclusions:
– Users like and are successful with the dynamic
faceted hierarchical metadata, especially for
browsing tasks
– Very positive results, in contrast with studies on
earlier iterations
– Note: it seems you have to care about the
contents of the collection to like the interface
71
Summary
• Validating an approach to web site search
– Use hierarchical faceted metadata
dynamically, integrated with search
• Many difficult design decisions
– Iterating and testing was key
• Bits and pieces were there in industry
– The approach is being picked up too
– One is very similar now: endeca.com
72
Summary
• We have addressed several interface problems:
– How to seamlessly integrate metadata previews with
search
• Show search results in metadata context
• “Disambiguate” search terms
– How to show hierarchical metadata from several
facets
• The “matrix” view
• Show one level of depth in the “matrix” view
– How to handle large metadata categories
• Use intermediate pages
– How to support expanding as well as refining
• Still working on it to some extent
73
Advantages of the Approach
• Supports different search types
– Highly constrained known-item searches
– Open-ended, browsing tasks
– Can easily switch from one mode to the
other midstream
– Can both expand and refine
• Allows different people to add content
without breaking things
• Can make use of standard technology
74
Some Unanswered Questions
• How to integrate with relevance
feedback (more like this)?
– Would like to use blobworld-like features
• How to incorporate user preferences
and past behavior?
• How to combine facets to reflect tasks?
75
Thank you!
For more information:
bailando.sims.berkeley.edu/flamenco.html
76