Transcript Visualizer 2000: Networked, Collaborative Environment for

VisualizeR 2000: Networked,
Collaborative Environment for
VR-Multimedia Synthesis?
SDSC CHAT
26 January 2000
Margaret Murray
CAIDA/SDSC Technical Manager
[email protected]
SDSC CHAT 26 Jan 00
1
Introduction:
• UCSD’s VisualizeR employs a user interface
that supports the innate human ability to
correlate and compare.
• VisualizeR software architecture is flexible
and extensible.
• Can VisualizeR serve as a springboard for
developing a networked collaborative UI?
SDSC CHAT 26 Jan 00
2
Outline:
• Examine the existing VisualizeR design and
implementation.
• Identify user interface criteria / issues.
• Examine ways in which VisualizeR might
evolve to enable collaborative interaction.
• Discuss alternative designs.
SDSC CHAT 26 Jan 00
3
Anatomic VisualizeR
• VR-Multimedia Synthesis ‘94 - present
• Complex 3D anatomic models
• Extant 2D clinical imagery
• Mentored lessons
• Multiple I/O configurations
SDSC CHAT 26 Jan 00
UCSD SOM
Anatomic VisualizeR is a
project at the Learning
Resources Center,
Helene Hoffman,Ph.D.
P.I.
4
Project goals:
(before there was code…)
• Prototype video demonstration by Helene
Hoffman ‘94 included videoconferencing
• Highly successful!
– Grant expanded to allow system development
– Prototype attracted buyers before anything was
built
SDSC CHAT 26 Jan 00
5
Today: a Complete Implementation
• Adjunct to existing curriculum
– Targeting difficult-to-visualize content
– Successful for many different audiences
– No videoconferencing
• ‘99 video: Anatomic VisualizeR in action
• Runs standalone or with shared content.
• For further information, contact:
– Helene Hoffman, Ph.D. (858)534-4123
– [email protected] http://cybermed.ucsd.edu
SDSC CHAT 26 Jan 00
6
VisualizeR Features
• Type-specific media objects (3D, 2D, video,
text, audio, etc.) are managed as display
blocks movable in 6DOF.
• Blocks respond to events and messages.
– Both core and device-specific user cmd events
– Messages can be tailored to app needs
• 6DOF Interaction tools organized as special
blocks.
SDSC CHAT 26 Jan 00
7
VisualizeR Strengths
• Intuitive, gestural interface commands take
advantage of spatial cognition capabilities.
• Content, logic, and display are well
separated; scenegraph based.
• Flexible device config; easily customized.
• Supports scripted as well as real-time
interactions.
• Enables uniquely insightful visualizations
SDSC CHAT 26 Jan 00
8
Visualizer Weaknesses
• Open Inventor based
• Current SGI platform too expensive for
ubiquitous deployment
• Best available I/O devices inadequate and/or
expensive
• Difficult to scope marketable ‘critical mass’
SDSC CHAT 26 Jan 00
9
Evaluating VisualizeR’s
User Interface
• Usability? (First criteria)
• Performance?
• Platform compatibility / availability?
• Graphics and windowing APIs ?
• I/O protocols?
• Usability? (Last criteria)
SDSC CHAT 26 Jan 00
10
Current VisualizeR Platform
• Modular code base promotes extensions and reuse:
– Rich, compact feature set for block design
– SGI C++ Development Environment 7.2.1
• Required software libraries:
– Open Inventor
– RogueWave
– X/Motif
– STL
• Silicon Graphics hardware
SDSC CHAT 26 Jan 00
11
Current VisualizeR Graphics
• Every block is movable in 6DOF by the user.
– No collision detection
• Special 3D 6DOF interaction tools and Xvr widgets
• Images are texture mapped onto 6DOF rectangles.
• OpenInventor (.iv) polygonal file format.
– Use translators to incorporate other model files
• HW Graphics: SGI 2-processor Octane with MXE
graphics
SDSC CHAT 26 Jan 00
12
Current VisualizeR Protocols
• Built-in event and messaging system (FACET)
• Serial
– devices: trackers, spaceball, pinchgloves,
MicroScribe
• Ethernet
– User may initiate telnet session block
– Shared content files via nfs
– Simple perl script for database content lookup
(returns URL)
SDSC CHAT 26 Jan 00
13
Current VisualizeR Performance
• 100K triangles/sec monoscopic (no latency)
• 50K triangles/sec stereoscopic (no latency)
• Serial devices run at 9600 - 115400 baud
• Multi-threaded app runs better with two
CPUs
SDSC CHAT 26 Jan 00
14
Current VisualizeR I/O Devices
• Input:
– Ascension Flock of Birds motion trackers with
fakespace pinchgloves
– Spaceball with mouse
– Sensable Technologies Phantom haptic device
– Standard mouse and keyboard
• Output:
– High-resolution monoscopic or CrystalEyes
stereoscopic display
– 600x480 head mounted displays (HMD)
SDSC CHAT 26 Jan 00
15
Usability: Glove
Gesture Commands
• Select or pick
• 6DOF Move
• Remove
• Special pointer
• 6DOF Tools
SDSC CHAT 26 Jan 00
16
Usability: Status Indicators
• 6DOF context
– Depth cues
– Shadows
– Grid
– Transparency
– Highlight
• Avatars for hands
• GRACE in corner
SDSC CHAT 26 Jan 00
17
VisualizeR Internals (how things work...)
• How is the application organized?
• How is it extensible?
• How does it support flexible configuration?
• How might VisualizeR evolve to support
collaborative interactions in a networked 3D
graphical environment?
SDSC CHAT 26 Jan 00
18
VisualizeR SW Architecture
• External files
• Modular components
–
–
–
–
–
Managers
VisR
Blocks
Xvr
FACET
• Device drivers
• System and library
calls
SDSC CHAT 26 Jan 00
19
Dynamic Application Config
• External Editors
– No need to reinvent (for models, images, text)
– User settings via Xresources mechanism
• Lesson Scripting
– Messages manage scene content and behaviors
– Lesson Pages may or may not be treated
sequentially
SDSC CHAT 26 Jan 00
20
External Files (Independent of App)
• Settings
• Content
• Lesson
Scripting
• Searchable
database
SDSC CHAT 26 Jan 00
21
Device Drivers
• Selectable at startup
• Loaded only when
needed
• Flexible command
mappings
SDSC CHAT 26 Jan 00
22
Application Organization
• Facet
• Xvr
• Blocks
(resource
or app
specific)
• VisR (gen
purpose
or app
specific)
• Exploits
DSOs
SDSC CHAT 26 Jan 00
23
Ideas for Enhancing VisualizeR
1. Port application as is to Linux (or NT).
2. Convert from OpenInventor to more scalable
Java3D.
3. Single user: Partition code into distributed
display/UI clients that talk to centralized
resource server.
4. Multi-user: Add distributed, collaborative UI.
5. Add dynamic interaction features.
SDSC CHAT 26 Jan 00
24
1: Port VisualizeR to Linux or NT
• Not a research project.
• Mouse and keyboard are poor at 6DOF.
• Linux concerns:
– Support for specialized HW
– Growth path?
• NT concerns:
– Concerns about scheduling and memory management.
– Limits on multi-threading?
– Growth path?
• Easier to move to Linux than NT.
SDSC CHAT 26 Jan 00
25
2. Convert OpenInventor to Java3D
• Not simply a matter of substituting API calls.
• May require conversion of all C++ code to
Java?
• Can a pseudo layer module successfully
interface between application graphics calls
and graphics libs?
SDSC CHAT 26 Jan 00
26
3. Partition App and Distribute Tasks
• Separate display/interaction (user client) from
resource managers (VisR server).
• How many clients can a VisR server manage
in real-time?
– Multiple single-user contexts?
– Multi-cast application?
• Expand existing event/msg paradigm?
SDSC CHAT 26 Jan 00
27
4. Enable Multi-User Collaboration
• Requires efficient partitioning of app transactions.
– User and scene context databases; atomic transactions
• Change script input from file to a network pipe?
• Create a new block for graphical MOO interaction.
– Text based chat board (multi-cast app?)
– Avatars for multiple users in scene; ID groups
– Graphical changes to avatar correlated to emotes
– Tools for single writer, multiple readers available for
collaborative use
SDSC CHAT 26 Jan 00
28
5. Enable Dynamic Interactions
• Efficient collision/correlation detection
– What method works best? (e.g. Ming Lin’s I-COLLIDE API).
– Useful for tools that identify and exploit behaviors of graphical
objects
– Already have a block to block msg passing mechanism
• Dynamic surface deformation
–
To achieve more efficient graphical updates
• Update shared graphic objects (user changes)
SDSC CHAT 26 Jan 00
29
Discussion
• Trying to build a graphical “instrument”
– Achieve real-time interaction response times
– Is scalability an issue depending on #users?
– Keep seat cost low
• Striving for visualization and interaction
ensemble
SDSC CHAT 26 Jan 00
30
In Closing: A Collaborative Analogy
(symphonic score excerpted from Rachmaninoff’s 1st Piano Concerto)
• >32 people
• Time sync
• Individual and
group
expression
• Graph+symbols
• Status+direction
SDSC CHAT 26 Jan 00
31
Acknowledgements
• Software Development.
–
–
–
–
Margaret Murray (CAIDA/SDSC)
Robert Curlee (CAIDA/SDSC)
Mark Danks (Stormfront Studios)
Ram Prayaga (UCSD CogSci)
• 3D models and imagery
– Alicia Fritchle (UCSD SOM/LRC)
– Dr. D Rigamonte (Uniformed Services Health Sciences Univ.)
• Ongoing vision & support
– Helene Hoffman, Ph.D. (UCSD SOM/LRC)
– Dr. Richard Satava (Yale)
SDSC CHAT 26 Jan 00
32