Transcript From Scatterbrained to Focused: UI Support for Today’s Crazed Info Worker

From Scatterbrained to
Focused: UI Support for
Today’s Crazed Info Worker
Microsoft
Engineering
Excellence
June 6
2006
Visualization and
Interaction for
Business and
Entertainment
Mary Czerwinski, Principal Researcher
Manager, VIBE, Microsoft Research
Overview
• Background Studies
– Diary study
– Large display findings
• Information worker productivity
• Programmer productivity and business
intelligence
• Future directions
Diary Study: Motivation
• Hypothesis: Current software does not support
multitasking well
– How bad/universal is the problem?
• Seek SW design ideas…
– Research shows users developing workaround
strategies
– Interruptions research shows harmful effects of
incoming notifications on current task
– Memory for To Do’s poor, undersupported
– Need to better understand task switching and
multitasking
Method
• 10 multitasking users recruited
• An excel spreadsheet was used as a
diary “template” to be filled out each day
• Diaries emailed back to me each
evening
• Participants instructed to write down
every “task switch”
– how hard to switch, # of docs required, # of
interrupts experienced, task time, anything
forgotten, notes, etc.
Partial diary for MS (6 hours)
Task Frequencies Breakdown
Indicative of
Difficulty Tracking
Tasks
Frequency of Task Type
Task Tracking
13%
Downtime
0%
Email
23%
Telephone Call
8%
Meeting
6%
Personal
5%
Routine Task
27%
“Returned to”
Tasks from this
group
Project
18%
Frequency of Task Shift
Initiators
Frequency of Switch Causes
Telephone Call
14%
App Prompt
Deadline
Appointment 1%
2%
9%
Email
Emergency
3%
1%
New Information
Request
3%
Next Task
19%
Self-Initiated
40%
Other Person
1%
Return to Task
7%
Difficulty Switching by Type
Rated Difficulty Sw itching to Task
Difficulty Switching (1=Low,
2=Med, 3=High)
3
2
Other Tasks
Returned-to Tasks
1
0
Task Type
Task Length by Type
Task Duration by Task Type
160
Average Task Duration (Mins)
140
120
100
Other Tasks
80
Returned-to Tasks
60
40
20
0
Task Type
Document Requirements by Task
Type
Number of Documents by Task Type
3
Average # of Docs
2.5
2
Other Tasks
1.5
Returned-to Tasks
1
0.5
0
Task Type
Interruptions by Task Type
Number of Interruptions by Task Type
2
Average Number of Interruptions
1.8
1.6
1.4
1.2
Other Tasks
1
Returned-to Tasks
0.8
0.6
0.4
0.2
0
Task Type
Discussion of Findings
• During a given week, KWs task shift an awful
lot (avg. 10 task shifts a day)
• Long-term projects are more complex shifts
– Lengthier (11.25% of the week), more documents,
interrupts, “returns”
– Rated significantly harder to return to
• Negative influence of interrupts on multitask
performance and memory well known
• Passage of time also takes its toll
• What designs will help?
General Design Ideas from
Participants
• Smarter, adjustable To Do list tracking & alarming
– In the projects versus just in Calendar
– Consider sticky notes for partial / future tasks
• Auto-categorization of email and files
• Better reminders for things forgotten
– Track events we know about and visualize them, or rely on
user manual tagging
• Better user adaptivity
– e.g., knowing what kinds of paste operations a user
typically performs and automating them
Focus on Returned to Tasks
• Elapsed time spanned hours to days
• Maintaining desktop state isn’t always the
answer
– Often, users said they were waiting on info from
other people or places (web, server)—prospective
reminders needed here
– Info came in via phone, email, web, or personal
contacts (better app integration needed here)
– But reminding about task context and info assembly
/ layout was a key problem identified
About the same time…Large
Display Findings
• Started exploring how user behavior changes as
displays increase in size and resolution
• Found that users were significantly more
productive when performing knowledge work
(multitasking, task switching) with large displays
• Less window management=less cognitive load
• But still needed help with task management
Tools for Task Management
• GroupBar joins related items in
the taskbar, remembers spatial
layouts of tasks (Smith et al.,
2003)
– Desktop “snapshots”
– Can “rehydrate” tasks with the
press of a button
• Scalable Fabric and VibeLog
(AVI 2004)
– Over 5000 downloads of SF
– Logging of task activity
Color Plate 1. Scalable Fabric showing the representation of three
tasks as clusters of windows, and a single window being dragged
from the focus area into the periphery.
New iWorker Productivity Solutions
• Task Tracking
• Event logging: StatusWriter
• Dev team navigation tracking
• FacetMap and FaThumb
• Sensing and adapting
Swish:
Microsoft
Engineering
Excellence
June 6
2006
Visualization and
Interaction for
Business and
Entertainment
Semantic Analysis of Window
Titles and Switching History
Nuria Oliver, Greg Smith, Chintan
Thakkar, Arun Surendran
Automatic Window Clustering
• Goal:
• Assist users in managing their tasks
• Assumption:
• Windows belonging to the same task share some
common features that can be identified from data
• In SWISH we explore:
• Title-based clustering (“Re: Dad, get me Potter-6” &
“Amazon: Harry Potter and the Half Blood Prince” )
• Behavior-based (switching history) clustering
(Looking into the MSDN Library while coding in Visual Studio)
Cluster 1
Harry
Potter
Book
Cluster 2
Top 3
keywords
Donna
Review
Malayeri
Cluster 3
A few exemplary titles
Expedia
Flight
Trip
Applications
• GroupBar
– Automatic or semi-automatic clustering
– Automatic keyword extraction  labels for the groups
• Implicit Query:
– Display relevant information to the current window
• Automatic Window Clean-up
Application
– Users open dozens of Explorer windows, and are too
lazy to kill them
– Collapse unused, unrelated windows to a single
cluster
– Provide an option after a timeout, to kill all together
StatusWriter
• Automatic
status
report
writer
• View time
spent by
app/doc
• View by
day, week,
month,
etc.
Status Writer Continued
• Can also view
by day
• Exports text
info to Excel
for further
analysis
• Future version
to include
– Calendar
– Tagging
– SWISH++
Clipping Lists and
Change Borders
Peripheral Information Display
Microsoft
Engineering
Excellence
June 6
2006
Visualization and
Interaction for
Business and
Entertainment
Tara Matthews, Mary Czerwinski,
George Robertson, and Desney Tan
Why Would Abstraction in
Peripheral Information Help?
• Imagine…
– You are balancing 5 tasks
– You have 18 windows open on your desktop
– You are waiting on the next draft of a paper, code to
be checked in to CVS, and an email
• You want to know
…when should you switch back to a task?
…when you switch tasks, what were you working on?
…when new info arrives, can you safely ignore it?
Study of Proposed Solutions:
Clipping Lists and Change Borders
• Compare interfaces w/ varying types of abstraction
– All interfaces based on Scalable Fabric (SF)
• Abstraction types:
– Change detection
– Semantic content extraction
• 4 interfaces:
SF
Semantic Content
Extraction (Clippings)
SF + Change Detection
Semantic Content
Extraction + Change
Detection
Baseline: Scalable Fabric
• Tasks as piles
• Windows shrunken
SF
Clippings
SF +
Change
Detection
Clippings +
Change
Detection
Change Borders
• Adds red borders around windows changing
content
• Border turns green when change is complete
SF
List
SF +
Change
Detection
List +
Change
Detection
Red Change Border
Green Change Border
Clipping Lists
• Extracts window
content
• Two ways to select
content
– Default: title bar
– User WinCut
– Future: AI
• Goal of selection:
– Help w/ recognition,
resumption timing, and
flow
SF
List
SF +
Change
Detection
List +
Change
Detection
Clipping Lists + Change Borders
• Extracts window content
• Adds green highlight to
task boundary &
windows that have
changed
SF
List
SF +
Change
Detection
List +
Change
Detection
Study Results
• Task flow
• Resumption timing
• Reacquisition
Average Time in Seconds
– Is more effective
than both change
detection and
scaling
– Significantly
benefits:
700
680
660
640
620
600
580
560
540
SF
SF +
Clippings
Change
Clippings
+ Change
Average Time to Resume Quiz
90
Average Time in Seconds
• Semantic content
extraction (Clipping
Lists)
Average Task Times
80
70
60
50
40
30
20
10
0
SF
SF +
Change
Clippings
Clippings
+ Change
Programmer Productivity: Team
Tracks
• We have observed devs struggling with unfamiliar code
– Inefficient navigation to find task-relevant code
– Misleading results of text searches
– Disorientation from too much navigation, too many
open files, interruptions
– [DeLine, Khella, Czerwinski, Robertson SoftVis ’05],
[Ko, Aung, Myers ICSE ’05]
• Team Tracks guides code exploration
– Records the team’s code navigation during
development
– Mines that data to prune the working set and guide
navigation
Evaluating Team Tracks
• Study 1: Does nav frequency indicate importance?
– Setup: Four programming tasks, then ratings
questionnaire and quiz
– Dependent measures: code paths, task completion,
ratings, quiz scores
– Hypothesis: Navigation frequency correlates to
importance rating [reported at SoftVis ’05]
• Study 2: Does Team Tracks improve productivity?
– Use Team Tracks with Group 1’s navigation data
– Same set up and dependent measures
– Hypothesis: Team Tracks improves task completions
and quiz scores
Navigation frequency does correlate
with importance ratings
• Pearson product
moment correlation,
r=0.79, p<0.01
Team Tracks does improve task
completion rates and quiz scores
•Improved task completion rates
–All completed tasks 1 and 2
–Task 3 (localized code): 1 / 7 without, 3 / 9 with
Team Tracks
–Task 4 (dispersed code): 1 / 7 without, 7 / 9 with
Team Tracks
•Group 2 quiz scores significantly higher
2.04, p<.03
–IE 8.0 team deployment ethnography next
–Added annotations and other features
t(16)=-
FaThumb
A Facet-based Interface for Mobile Search
Microsoft
Engineering
Excellence
June 6
2006
Visualization and
Interaction for
Business and
Entertainment
Amy K. Karlson (U of Maryland), George
Robertson, Daniel C. Robbins, Mary Czerwinski,
Greg Smith
VIBE Group
Microsoft Research
FaThumb: Overview + Video
• Keypad is leastcommon-denominator
–
–
–
–
Cell-phone
Remote control
ATM
Number key-pad
• Typing text is hard
• Let users browse
data attributes
taxonomy (facets)
Current Query
Search Terms
Results
Facet
Navigation
Menu
Standard
keypad
FacetMap-Faceted Search of all
your stuff
Sensing
• HealthGear
• Braincomputer
interaction
Future Directions
• Information worker productivity
– Intelligent summaries and visualizations of tasks
• IR & Info Vis
–
–
–
–
FacetMap
FaThumb on SmartPhone
Info vis toolkit prototype in January
Rich desktop search client
• Interaction techniques
– Adaptive UI: study predictability
– Other, step-based UIs
• Sensing
– HealthGear: whole new line of research and networking to be
done
– BCI: actually use it while running real applications
Thank you for your
attention!
Microsoft
Engineering
Excellence
June 6
2006
Visualization and
Interaction for
Business and
Entertainment