Transcript Vis 2003 MyLifeBits Challenge for Visualization Visualization: “Form follows function” Visualization: Science? Engineering? Or Art? Seattle, WA 22 October 2003 Gordon Bell Microsoft Research.

Vis 2003
MyLifeBits Challenge for Visualization
Visualization: “Form follows function”
Visualization: Science? Engineering? Or Art?
Seattle, WA
22 October 2003
Gordon Bell
Microsoft Research
My affair with Visualization
•
•
•
•
•
•
•
•
1958 wrote programs that displayed on the 32x32 scrolling display of Deuce, a machine
that came from NPL that Turing worked on. Interactive Marriage proposal program.
1959 MIT Speech Lab and TX-0 10” 1K x 1K points, basis for DEC display. Displayed
sliders, sonograms, traditional engineering graphs. Made a Land use map of Boston for
Gwen Bell, a student at Harvard School of Design. They later formed the Graphics lab or
such displays.
1960 DEC. Type 30 . Spaceware was built on it, but at one of the joint computer
conferences, Ed Jacks of GM watched the display the whole time and got IBM to build
them displays for car design.
1961 saw the difference of logic diagrams vs. logic equations; i.e. east vs west coast
design
1986-1989 Dana>Ardent>Stardent “the graphics supercomputer”.
1987 Commissioned the NSF Study for visualization.
1987 FCCSET report for the Internet proposal claimed we need the net for visualization.
1990 Keynoted first visualization conference in SF. Met the Fakespace team and
became an investor. They produced the boom for VR, large scale displays, tables, etc.
Outline
•
•
•
•
•
•
Now I find I was doing interactive vis in 1958.
Review of Vis 1990 keynote
Visualization (Google) 7M; 4.4M data
Vis 1990 then and now
Shneiderman, Tufte, & examples
Can you help MyLifeBits?
– Timelines
– Document … knowledge management
1987 FCCSET Report
for NREN
Titan Graphics Supercomputer c1988
25Kpolygons/sec
Keynote from Visualization I.
23 October 1990
Or how I saw it then…
before the web, but after NREN (Internet.)
How I "See" saw Visualization
13 years ago*
The Turing TEST for Visualization: 3D, dynamic computer
graphics and computer imaging are aimed at photo
realism, ...
i.e. the standard of comparison is limited by a view or
photo of the phenomenon itself (reality).
The insight comes from within the user.
Technology helps somewhat in the viewing.
Few of today's vizs provide the insight that compare with
the creative representations that Tufte collected.
Challenge: Make the computer do what the artists did!
Users need Vizicalcs (…otherwise everything is a creation)
Training and integration into the computing mainstream is
still the bottleneck to use
*form mental image of something not visible
Computation is the Driving Force
for Visualization
4. Analysis-by-synthesis in a real time environment
Real, experimental science that simulates and simultaneously
compares observations. Courtesy of Web Services
3. Simulation* is the Third Paradigm of Science & Engineering, after
2. Experimentation (taking and plotting data) and
1. Theory
0. Talk is for philosophers
*The best way to advance a science is by building a computer (i.e.
program) for that phenomenon in an "analysis-by-synthesis"
fashion.
Viz has come about through integrated computation and graphics,
with a push from supercomputing.
The NSF Visualization Report
Domain & Needs:
Steer simulation and computation
W/S driven use of supers
Graphics-oriented programming environments
Higher-order viz of scalar, vectors ... tensor
fields
Dynamic viz of fields and flows
Massive data sets for signals and images
Algorithms and archs for parallel graphics & IP
Framework for international Viz standards
The NSF Visualization Report
(Cited) Benefits
Integrated set of portable tools
Scientific progress and leadership
Scientific productivity
Standardization of research tools and the
cross-discipline flow of knowledge
Safeguard American Industrial
Competitiveness
Making the Advanced Supercomputing
facilities useful
Any Effect of the NSF
Visualization Report?
Raised awareness of the opportunity for Viz by
creating a word and focus
Got funding at various centers (e.g. UC/SD)
Galvanized a collection of people working in
the area
...
Ultimately stimulated this conference* a year or
two before it might have otherwise happened
*Reaction to size and lack of applications focus
at SIGGRAPH
Searching for Vizicalc:
"have data, need a picture"
Users wanted to visualize their data:
how much and how fast can it be displayed?
how long to get the first picture?
Applications done on ad hoc case at a time
basis
New tools such as AVS came about because of
limited resources that had to apply the first
graphics supercomputers.
Progress: 1990 - 2003
• The big bang: Internet and the web
– 1987 FCCSET Report set fiber goal by 2000
• Moore’s Law
–
–
–
–
–
10s of Mips >> 1-4 Gips; Custom rendering…
10s of Mbytes >> 100 Gbytes
25 K polygons/sec >> 25 M polygons/sec
Mpixels = Mpixels; mono; but lower cost
1.5 Mbps > 1-2 Gbps (peak)
• Progress in Vis
– Realistic rendering… see complete virtual scenes/sets
– Expensive, large screens and caves:
• Using “2001, A Space Odyssey” progress in:
– Visualization was much, much greater
– AI, including speech was much, much less
Princeton 6K x 3K+ pixel; 18’x8’; $100K
Vis 2003
Problem and opportunity:
have many nouns…
Need:
create verbs
Taxonomy
• Pseudo phenomena (ICs, molecules, colored
physical parameters e.g. T, p, fields for flow, stress
• Photo realism
– Real world: movies, VR, Augmented Reality,
– People and digital immortality
– Telepresence… ever emerging, potentially important
• Data visualization… Tufte gallery candidates?
– Small no. of dimensions
– Large dimensionality information maps
• Ordinary text and user visualization beyond UI
Shneiderman’s insights
“exploratory data analysis and visualization process is
…characterized by mantra for visual information seeking:
“Overview, zoom and filter, details-on-demand”
“The future … larger, higher resolution screens, that
present perceptually-rich & information-abundant displays.
… the flood of information can be turned into a river of
knowledge...
Human perceptual skills are quite remarkable and largely
underutilized”
Tufte on Graphic Excellence
Tufte
Tufte Napoleon Map Paris-Moscow
Data maps… near and dear
Network flow of trade
Hillis Taxonomy
Axon
Kiviat/radar of car parameters
Evolution of the “Ideal State”
at Each Stage:
Entrepreneurials
Stage IV.
Market Development
Business Plan
Manufacturing
Marketing
Product
Development
Sales
Stage III.
Product Development
Stage II.
Seed
Stage I.
Concept
Technology/
Engineering
CEO
Team
Control
Fundability
Board of Directors
Cash
Many dimensions.. Inverse pattern
recognition
Tufte:
On PowerPoint
I am data
The guinea pig
• Gordon Bell is digitizing his life
• Has scanned/encoded virtually all:
– All computer files
– Personal documents (memos and email,
bills, legal documents, papers written, …)
– Books written (and read when possible)
– Photos
– Posters, paintings, photo of things (artifacts, …medals, plaques)
– CD collection
– Home movies and videos
• Now recording real time: phone, radio, TV (movies), web
pages… started meetings, conversations to come
• Paperless throughout 2002. 12” scanned, 12’ discarded.
• Only 30 GB!!!
The MyLifeBits Challenge
• Files, photos, documents
• Challenge is viz relevant for organizing,
searching, and utilizing your shoebox… data
problem
• Tufte vs. Visioneers:
– Visualization is an art.
– Visualization is science and it has to be automatic!
• Challenge:
– 1. What are the questions viz can help answer?
– 2. What viz can help make a life or after life interesting?
• We have many nouns. You make verbs.
Statistics of use
Visualization Needs & Possibilities
•
•
•
•
•
•
•
•
Time lines
Video cliplets, photos searching
Automatic story creation from content
Related document maps
Concept mapping of entire space
Collections & taxonomies
Synthesis of talking heads “in their words”
Tweening from photos to recreate entire movies
of the experience.
Personal LifeLog Applications
Self
Diary/Journal
Tutor
Mentor
Advisor
Others
Application used by:
Babysitter
Financial
Manager
Medical
Manager
Companion
Caretaker
Parole
Officer
Assistant
for Elderly
Pers Flight
Recorder
Meeting
Prep
Personal
Assistant
Photo
Album
Autobiography
Captain’s
Log
Conservator
Biography
Baby
Book Trustee
Obituary
Executor
Others
Application controlled by:
Personal
Proxy
Self
Radio capture
tool
Telephone
capture tool
PocketPC
transfer tool
PocketRadio
player
TV capture tool
Internet
MyLifeBits store
Radio EPG
tool
TV EPG
download
tool
database
MAPI
interface
Browser tool
files
Legacy
applications
MyLifeBits Shell
Voice
annotation
tool
Text
annotation
tool
Legacy email
client
MyLifeBits organization: time and space
Timeline/
Context
(space)
Personal
(some $s)
GB Co.
(angel, etc.)
Professional
ACM, etc., …
@Microsoft.com,
New co’s.
Archival
(time)
Working
1900 1910 1920 1930 1940 1950 1960 1970 1980 1990 2000 2010
C,L
m
d
CGB...
GB SR
Where
WCa
KvMO
6-year
--GS-HS--MIT DEC---+++++.+++---++++
Education
Work
mB,L
d
B ABos
KF SB
P B
KV-----mit,F cmu
Bell Elec
DECcmuDEC
ComputerMuseum
Books
Computers
Awards..
E,NSF MSFT
M B
BN
SiValley
SBN HiTechVent
4-6 11 VAX E T
Time-machine Computing
Rekimoto
Lifestreams: View by time
David Gelernter et al
Rock & Roll History graph
Timeline view tells a story
MyLifeBit v2 clustered time view
…Information Visualization within a Digital Video Library

Colored notches show
where matches occur
…Information Visualization within a Digital Video Library

Keyword color
matches color in
ranking
Here are the photos
Getting the user to tell a story is the
ultimate in media value




A story is a “layout” in time and space
Most valuable content (by selection, and by being well annotated)
Stories must include links to any media they use (for future navigation/search –
“transclusion”).
Cf: MovieMaker; Creative Memories PhotoAlbums
Dapeng was an
intern at BARC
for the summer
of 2000
We took him to
lunch at our
favorite Dim Sum
place to say
farewell
At table L-R: Dapeng, Gordon, Tom, Jim,
Don, Vicky, Patrick, Jim
Digital Immortality:
CMU Interactive Einstein:
U of MD display of health data
http://www.research.ibm.com/history/gallery.htm
Document Changes History Flow
Tn
Temporal Analytics ThemeRiver

Challenge: identify
patterns, trends, and
relationships over time
in large data
collections

Breakthrough: show
change over time
using smooth, stacked
curves to enhance
understanding;
include context of
external events.
Haystack: Ozone interface
screenshot


Portal
Agents
construct each
window
Information Visualization within a Digital
Video Library

Christel & Martin (CMU),
Information Visualization
within a Digital Video
Library, Intelligent
Information Systems,
June 1998.
Inxight VizServer
http://www.inxight.com/map/
More inxight
Engineered for broad deployment
Now under pilot evaluation and deployment
Windows based: client/server, stand alone, or web based
Business Sensitive
60
Tn
Towards Multiple Simultaneous
Hypothesis Queries
Each Hypothesis on a Ray
 Challenge:

Trade
Protection
Measures
Multiple hypothesis
analysis
Japan
Protection
Measures
Japan
 Breakthrough:

Interactive and visual
method for analysis of
multiple hypothesis
Trade
Protection
 Impact:

First example of
hypothesis and
scenario based
analytics
Measures
Trade
Protection
Business Sensitive
61
Visualizing document relationships
Boyack, Wylie & Davidson
WebScout (MSR Cambridge)
Clustered browsing
of the Fine Arts Museum of San Francisco
http://www.cs.berkeley.edu/~kobus/research/projects/browse.html
The brain
The Axon Idea processor
..\..\Cyberall MyLifeBits\Axon Idea Processor.mht
Mind mapping
Innolab 3D File manager
Webmap
Webmap
http://www.webmap.com/trademapdemo.html
Smartmoney.com
Smartmoney.com Whole market
The end…