Transcript Characterizing Audience for Informational Website Design: A Case Study Jennifer Turns, Ph.D. Assistant Professor, Technical Communication Faculty Affiliate Program for Educational Transformation through Technology (PETTT) Center for Engineering.

Characterizing Audience for
Informational Website Design:
A Case Study
Jennifer Turns, Ph.D.
Assistant Professor,
Technical Communication
Faculty Affiliate
Program for Educational Transformation through Technology (PETTT)
Center for Engineering Learning and Teaching (CELT)
Acknowledgements: This work has been supported by the Program for Educational Transformation through Technology (PETTT). Many
people have contributed to this work including Scott Macklin, Tracey Wagner, Aaron Louie, Brett Shelton, Kristina Liu, Alice Tanada,
1/30
Jake Burghardt, Julianne Fondiller, Regina Yap, Ralph Warren, and Dr. Frederick Matsen.
Overarching UCD Questions
• How do design teams incorporate information
about users into their design process?
• When characterizing users/audience?
–
–
–
–
Which dimensions are most significant?
Which methods under which circumstances?
How will the insights inform design?
What are realistic expectations?
2/30
Informational Website Design
Some of my projects in this domain…
• Arthritis Source:
Medical information on the web
• Teaching challenges of engineering faculty:
Information for educators
• Also
– Legal information online
– Information about architecture building methods
3/30
Today’s Goal
Use a case study of audience analysis for website
design to reflect on decisions in audience analysis
• Describe Arthritis Source, informational website
• Describe our approach to audience analysis
• Present sample of results from our audience
analysis, and their impacts on the design
• Discuss moving beyond the case study
4/30
Arthritis Source
• Developed in 1995 by
Dr. Frederick Matsen
• Focus on arthritis
• Authorized information
• User-centered information
• Research test bed
5/30
Arthritis Source Content
• Articles as basic metaphor
– Content is organized into articles
– Examples: Rheumatoid Arthritis,
Rotator Cuff Surgery, Pregnancy
• Templates underlie articles:
– The template for each article is the set
of questions answered in each article
– All content is based loosely on five
templates: conditions, surgery,
treatment, living with, medication
– Templates are informed by research
on user questions
• Content is dynamically generated
6/30
Arthritis Source Authoring
• Authors
–
–
–
–
–
Volunteer
Are subject matter experts (e.g., doctors)
Are not trained as technical communicators
Create content for entire template or specific question
Can create content online or in Word
• Authoring is actually distributed
– Team creates templates based on user information
– “Authors” create content
– Administrator edits
7/30
Multiple Access Paths
• Multiple Access Paths
– Navigation comes from
article templates
– Spotlighted content
– Question-based search
8/30
Embedded, Ongoing Evaluation
• Online survey
– “Tell us about yourself”
• Quick polls
– “How useful is this article”
• Online quizzes
– “What do you know…”
• Online research studies
– “Exploring use over time”
9/30
Arthritis Source Community
Community includes
Share responsibility for
• Users
• Technical Communication
Professionals
• Domain Experts (i.e.,
doctors)
• Learning Scientists
• Developers
• Administrator
• Determining scope
• Creating content
• Evaluation quality
10/30
Arthritis Source Timeline
Design
‘00
‘01
‘02
1995
Audience
Analysis
11/30
Audience Analysis
• Goals
– Inform our own design and evaluation
– Contribute to broader discussion
• Decisions
– Dimensions?
• Inform design, Speak to team, Theoretical traditions?
– What methods?
• Breadth/Depth, Acknowledge distributed nature of users
12/30
Multidisciplinary Influences
• Dimensions
–
–
–
–
–
–
Roles
Goals
Knowledge
Circumstances of Use
Culture
Ergonomics
• Theoretical Perspectives
–
–
–
–
–
–
–
–
Technical Communication
Reader Response Theory
Cognitive Science
Constuctivism
Distributed Intelligence
Situated cognition
Socio-Cultural Theory
Human Factors
13/30
Defining Dimensions
• Role – Dominant persona of users (job, affiliation)
• Goals – Reason for the interaction
• Knowledge – The extent and nature of prior relevant knowledge
• Circumstances of Use – Setting, resources, strategy, timing
• Culture – Group level beliefs, language, preferences
• Ergonomics – Relevant perceptual & motor abilities, skills
14/30
Method – Online Survey
• Questions: Adaptive, ~25 questions
• Participants:
– Duration: 9/1/2000 – 7/2/2001 (10 months)
– 472 respondents / 710 starts
• Analyses1 –
–
–
–
–
Descriptive Statistics
Content Analysis
Qualitative Coding
Statistical Analysis
1Acknowledgments:
Tracey Wagner, Kristina Liu, Alice Tanada, Kristen Schuyler
15/30
Method - Phone Interview
About Visit
About Knowledge of Condition
• Could you tell me about your
visit or visits to the Arthritis
Source?
• Could you tell me what you
were trying to do when you
visited the Arthritis Source?
• Did you benefit from your visit
or visits to the Arthritis Source?
• What kind of information do
you think other arthritis patients
should know?
• Could you tell me what you think
arthritis is in general?
• Could you tell me how RA/OA
affects the body?
• Do you know what contributes to
getting RA/OA?
• Do you know how RA/OA is
diagnosed? If no, Do you
remember what your doctor told
you about your diagnosis?
• What is most difficult to
understand about RA/OA?
16/30
Phone Interview
• Participants – 20 users (10 OA, 10 RA)
• Analyses1
– Conceptions/misconceptions
– Overarching Goals
– Specific Information Needs
1Acknowledgments:
Tracey Wagner, Kristina Liu
17/30
Mapping Data to Categories
Data and Sources
Online Survey
Visitor Type
Age
Home Community
Geographical Area
Type of Arthritis
Level of Education
Time since diagnosis
Name of Condition
Why visiting
Came in from
Use of site in past
Sources of information
Phone Interview
Knowledge of Condition
Goals
Specific Information Needs
Role
Goals
XXX
X
Circumstances of
Knowledge
Use
XX
X
X
X
X
X
XX
Culture
X
X
XX
XX
Ergonomics
X
XX
XX
XX
X
XXX
XX
X
X
XX
XX
XX
XXX
XXX
XXX
18/30
Results - Overview
• What we learned…
– 19% international (culture)
– 26% rural (circumstances of use)
– 21% over 60 (ergonomic, through vision implications)
• Highlight specific examples where results had
identifiable impact on design
–
–
–
–
Role
Goals
Knowledge
Circumstances of Use
19/30
Roles
• Users with many roles
Other
20%
– Person with pain
– Person with condition
that they do not
consider arthritis
– Person who is
exploring whether they
have arthritis
Student
1%
Researcher
2%
Medical
Professional
5%
Relation
10%
n=462/472
• “Person with arthritis”
is too simplistic…
Person with
Arthritis
62%
20/30
Role – Design Implications
• Added new types of information
– Differential diagnosis
• Identified writing guidelines…
– Avoid statements that assume reader has the
particular condition… (avoid -- “your
condition” or “you need to…”)
21/30
Knowledge – Misconceptions?
• From interview data, we identified several
possible misconceptions:
–
–
–
–
–
Low bone density is associated with Osteoarthritis
Not drinking enough milk increases the risk of Osteoarthritis
Bone spurs cause arthritic pain
Osteoarthritis causes bone erosion.
OA is caused by the "wear and tear" of the joints due to overuse
and aging.
– There is little you can do
22/30
Knowledge – Design Implications
• Added material to confront misconceptions
• Added “common myths” question to
generic templates
• Developed Osteoarthritis knowledge quiz
– 2513 Respondents for question 1
– 470 Respondents for entire 7-question quiz
23/30
Goals – Social Support?
Description
Percentage
(n=458)
1. No indication for social support
75%
2. Implicit desire for social support
(mostly in the form of direct arthritis questions raised
and requests for other links/sites)
20%
3. Seeking doctor’s (or hospitals) referrals
2%
4. General desire to talk to people
3%
5. Request for live interaction on the site
0%
6. Seeking local, face-to-face support
0%
24/30
Goals – Design Implications
• On providing access to social support
– Reduced priority for this within our system
– Added content to help users find to support
• On learning from a surprise
–
–
–
–
20% of statements were direct questions
Indicates need to help users start with own questions
Question-based search permits this directly
Templates are based on user questions
25/30
Circumstances of Use - Virtual
• Coming from…
Other
18%
– Improve placement of
site on search engines
Bookmark
7%
Website
22%
Referred by
6%
Search Engine
47%
• Implications
n=372/472
– Orient users coming
from search engines accelerated
implementation of the
navigation based on the
article templates.
26/30
Ongoing Audience Research
• Ongoing analysis of user questions (goals)
– From emails
– From question-based search
– Collected earlier…
• Studying use over time through user online
journaling (goals, circumstances of use,
knowledge and learning)
27/30
Where we’ve been
• Starting Point:
When characterizing users/audience…
• Which dimensions are most significant?
• Which methods under which circumstances?
• How will the insights inform design?
• What are realistic expectations?
• Contribution here: Case Study…
28/30
Moving beyond case study
Framing audience analysis decisions
• Effort: Manager perspective
– Resources to design, collect, analyze, interpret, decide
• Insight: Researcher perspective
– Rigorousness, representativeness, triangulation?
• Impact: Designer perspective
– Clarity of links to design, Persuasiveness of the data
29/30
Designing Audience Analysis
• Effort: Manager perspective
– Resources to design, collect, analyze, interpret, decide
• Insight: Researcher perspective
– Rigorousness, representativeness, triangulation?
• Impact: Designer perspective
– Clarity of links to design, Persuasiveness of the data
Observation: Opportunities lie here…
30/30
Web-based Health Information
• Site quality
– Owner credentials, update dates (Hoffman, 2000)
• Quality of information
– Comprehensiveness (e.g., Chen, 2000)
– Accuracy (e.g., Chen, 2000)
– Providing references (e.g., Hellawell, 2000)
• Findability of information
– Time required (e.g., Gotwald, 2000)
– Getting to real questions (e.g., Lechner, 1996)
• Need for evaluation methods
– (e.g., Wu, 2000, Delamsthe, 2000, Charatan, 1999)
31/30
When users are learners?
• Characteristics of learners
–
–
–
–
Growth (Soloway, 1994)
Diversity (Soloway, 1994)
Motivation (Soloway, 1994)
Prior Understandings (NRC, 1999)
• Thinking about implications
–
–
–
–
Users
Learners
What to know and why
Knowing users over time
Variations (how many to know)
Time required to truly “know”
32/30