Transcript Interactive Visualization in 4k A Tutorial Jürgen P. Schulze, Ph.D.

Interactive Visualization in 4k
A Tutorial
Jürgen P. Schulze, Ph.D.
California Institute for Telecommunication
and Information Technology (Calit2)
University of California San Diego
PRAGMA 14 – March 10, 2008
Bio
• M.Sc. Computer Science (1999)
• Ph.D. Computer Science (2003)
• Post-doc at Brown University (2003-05)
• Post-doc at UCSD (2005-2006)
• Project Scientist at UCSD (since 2006):
– 1 staff programmer (hiring another)
– 5 students (2 graduate, 3 undergraduate)
UCSD
• Founded 1960
• Number of students: ~26,000
Taiwan
California Institute for Telecommunications
and Information Technology (Calit2)
•
New Laboratory Facilities
–
–
•
Nanotech, Chips, Radio, Photonics, Grid,
Data, Applications
Virtual Reality, Digital Cinema, HDTV,
Synthesis
Over 1000 Researchers in Two Buildings
–
–
Linked via Dedicated Optical Networks
International Conferences and Testbeds
UC San Diego
UC Irvine
Preparing for an World in Which
Distance Has Been Eliminated…
Calit2 Digital Cinema Theater
200 Seats, 8.2 Sound, Sony SRX-R110, SGI
Prism w/21TB, 10GE to Computers/Data
Stereo Wall (C-Wall)
• Dual HD resolution
•
•
•
(1920 x 2048 pixels)
with JVC HD2k
projectors
2 Linux PCs w/Nvidia
Quadro 5600
Passive stereo
Ascension Flock of Birds
tracking system
w/Wanda
Outreach
The StarCAVE
•
•
•
•
•
•
•
•
•
Computers: 15 Dell XPS PCs with Quad
Core Intel CPUs
OS: CentOS Linux
Graphics cards:
2 Nvidia Quadro 5600 per node
Projectors: 30 JVC HD2k (1920x1080
pixels), ~30 megapixelsm per eye
Stereo: passive with circular
polarization filters
15 screens, ~8 x 4 feet each
Optical tracking system by ART
Visualization software:
COVISE, OpenSceneGraph
Programming Language: C++
Calit2’s 225 Megapixel HiPerSpaceTiled Display
Overview
• Interactive graphics
• Graphics frameworks
• COVISE/OpenCover
• DaVinci project
Computer Graphics and Virtual Reality
Computer Graphics
• non-real-time rendering
• focus on image quality
• can be non-interactive,
configuration file driven
• in interactive
applications: progressive
refinement possible
(simple data set during
mouse interaction, full
detail when no
interaction)
Virtual Reality
• real-time rendering (15+
fps)
• focus on frame rate
• interactive, input device
driven
• no progressive
refinement because of
head-tracked user
Low Level Graphics API
• OpenGL is most widely used
– available for Linux and Windows
– fast
– state-machine
– good support for new graphics cards
• DirectX
– requires Windows
– very good support for new features in graphics
cards
Multi-Display Support Mechanisms
• Distributed scene graph:
– e.g., OpenSG (not OpenSceneGraph)
– automatically updates on all cluster nodes upon changes
– support for very large scene graphs
• Event-synchronized:
– e.g., Covise, VR Juggler/Clusterjuggler
– master node sends input events to render nodes
– each render node runs a complete copy of the code
– scene graph size limited to single node
• Pixel map based:
– e.g., SAGE (EVL)
– master node sends pixel images to render nodes
Graphics APIs vs. Software Frameworks
• Graphics APIs:
– provide interface to graphics routines
– e.g., OpenGL, SGI Performer, OpenSceneGraph, OpenSG,
Java3D
• VR Frameworks:
– integrate:




graphics API
VR input device support
flexible display device configurations
audio support
Frameworks: Differences
•
•
•
•
•
•
•
•
•
•
•
•
Size of user base, on-line forums
Free, open source, commercial
Supported platforms
Programming language
Availability and quality of documentation
Input device support
Display device support
GUI API, widget library
Synchronization of cluster nodes
Supported graphics APIs and scene graphs
Data distribution paradigm on clusters
Supported standard visualization algorithms (volume rendering,
particle tracing, etc.)
Software Frameworks: Examples
• COVISE (HLRS)
• SAGE (EVL)
• CGLX (Calit2)
•
•
•
•
•
•
•
•
•
•
•
VTK (Kitware)
WorldToolkit (Sense 8)
CAVELib (VRCO)
Ygdrasil (UIC/EVL)
VR Juggler (Iowa State)
DIVERSE (VA Tech)
Avocado (GMD)
FreeVR (Bill Sherman)
dVS (Division)
Lightning (Fraunhofer IAO)
Amira (TGS/Mercury)
SAGE
HD Video from
BIRN Trailer
Macro View of
Montage Data
Micro View of
Montage Data
Live Streaming
Video of the RTS2000 Microscope
HD Video from the
RTS Microscope
Room
LambdaCam Used to Capture the Tiled Display on a Web Browser
Source: David Lee,
NCMIR, UCSD;
Jason Leigh, EVL, UIC
SAGE Developed Under
Jason Leigh, and Luc Renambot, EVL
4k Projectors at Calit2
• Sony SRXD R110:
– Resolution: 4096x2160 pixels
– Video input: 4 x HD-SDI
– 10,000 Lumens
– Refresh rate: 24 or 30 Hz
– SXRD = Silicon X-tal Reflective Display:
reflective liquid crystal microdisplay
• JVC DLA-SH4K:
– Resolution: 4096x2400 pixels
– 3,500 Lumens
– Video input: 4 x Dual-Link DVI-D
– Refresh rate: 60 Hz
Sony 4k Projector with HD-SDI
• Miranda DVI to HD-SDI converters
• Miranda boxes send incorrect resolution
parameters to DVI port
• Solution:
– connect JVC HD2k projector
– run 'nvidia-settings' tool
– save EDID file
• Configure EDID file in xorg.conf
• Use Twinview on each graphics card
• Use Xinerama between graphics cards
Quadrant Synchronization
• Synchronized buffer swap
• Hardware level: NVidia G-Sync II board
• If Nvidia driver does not work try:
http://www.nvidia.com/object/linux_display_amd64_169.07.htm
l
Nvidia G-Sync II
4k PC at Calit2
• Dell XPS 710
• Intel quad core CPU
• 4 GB RAM
• 500 GB harddisk
• Dual Nvidia Quadro 5600 graphics
• Nvidia G-Sync II board
• CentOS 4.6 (ROCKS compatible)
cards
Operating System
• CentOS 4.6 (ROCKS compatible)
• Basic installation steps:
– Download CentOS ISO image at:
http://isoredirect.centos.org/centos/4/isos/
– Burn DVD and install OS from DVD
– Update OS (with round red icon at top of screen)
– Install latest graphics driver
– use "nvidia-settings" tool to configure display
resolution
– Install additional libraries as needed with rpm or
yum
• Detailed
instructions for IVL are at:
http://ivl.calit2.net/wiki/index.php/Guide_to_install_Linux_on_lab_computer
xorg.conf for Sony 4k projector
Created with 'nvidia-settings' tool
Section "Screen"
Identifier
"Screen0"
Device
"Videocard0"
Monitor
"Monitor0"
DefaultDepth
Option
24
"TwinView" "1"
Option
"metamodes" "DFP-0: 1920x1080_60 +1920+0, DFP-1:
1920x1080_60 +0+0"
Option
"CustomEDID" "DFP-0:/etc/X11/edid_miranda.bin; DFP1:/etc/X11/edid_miranda.bin"
SubSection
Depth
"Display"
24
Modes
"1920X1080" "1600x1200" "1280x1024" "1024x768"
"800x600" "640x480"
EndSubSection
EndSection
COVISE
• Developed
at
University of
Stuttgart
• Support for
desktop and
VR display and
input devices
• Support for
tiled displays
OpenCover
• COVISE's virtual reality renderer
• Can run inside or separate from COVISE
• Written in C++
• Based on OpenSceneGraph
• No license required
• Closed source, but open programming interface
(API)
• Modular application system
• Modules can run concurrently
• Download from HLRS web site
OpenCover Plugins
• C callbacks for initialization,
rendering,
message passing
• C++ module class
• Access to XML configuration file
• OpenSceneGraph based
• Support for straight OpenGL code
• Run-time linking (plugins compile separately)
OpenCover Plugin Structure
• C plugin interface code
• Plugin code is in C++ class
ModelViewer* plugin = NULL;
// global instance of C++ plugin class
int coVRInit(coVRPlugin* m) // initialization; called once at start-up
{
if (plugin==NULL) plugin = new ModelViewer(m);
return 0;
}
void coVRPreFrame() // called once before rendering
{
plugin->preFrame();
}
void coVRDelete(coVRPlugin *)
{
delete plugin;
}
// called when plugin is removed
OpenCover Menu System
• Programming interface similar to Java/AWT
• Widgets: menu, sub-menu, label, action
button, check box, radio button, dial, slider
• Support for custom dialog windows
• All menus live in 3D space
Covise Configuration File
•1
window, 1 pipe, 1 screen
<COVER>
<PipeConfig>
<Pipe display=":0.0" name="0" screen="0" pipe="0" />
</PipeConfig>
<WindowConfig>
<Window width="3840" height="2160" left="0" bottom="0" comment="FRONT" window="0"
pipeIndex="0" name="0" decoration="false" />
</WindowConfig>
<ChannelConfig>
<Channel comment="FRONTLeft" channel="0" left="0" right="3840" bottom="0" top="2400"
stereoMode="LEFT" windowIndex="0" name="0" />
</ChannelConfig>
<ScreenConfig>
<Screen width="2400" comment="FRONTLEFT" h="0.0" originX="0" originY="0" originZ="0"
height="1350" p="0.0" r="0.0" name="0" screen="0" />
</ScreenConfig>
<Stereo enable="off" value="off" />
<StereoMode value="NONE" />
<NumPipes value="1" />
<NumScreens value="1" />
<NumWindows value="1" />
</COVER>
Visualization of multi-spectral images
• Goals:
– Concurrent real-time
–
–
visualization of multiple large,
high-resolution images
Research of user interface: how
best to interact with multispectral images for experts and
general public
Comparison of immersive
environment (3D, large field of
view) with high resolution
displays (4k, tiled display wall)
Walking Into a DaVinci Masterpiece
• Image size:
•
~400 megapixels
(20,000 x 20,000
pixels)
Multiple layers:
visible, IR, UV, Xray
The multi-spectral DaVinci painting on the C-Wall
Visible Light
Infrared Light
Walking into a Leonardo Masterpiece
• Co-registered modality images
• Support for translucent osgTerrain images
• Images can lock to screen
• Zoom by turning pointer
• Pan by moving pointer
• Fade between modalities by user distance
from screen
• Single button interaction: simple user
interface
• Support for multiple image sets
• Co-registered marker
Rendering ultra-high resolution images
• Uses OpenSceneGraph's osgTerrain engine
• Supports RGB images and 2 1/2D terrain
• Clipmapping approach: mipmapped tiles
• Dynamic loading manager loads only visible
tiles
• Constant frame rate
User Interaction
• At Calit2 we use an Ascension Flock of Birds
• 6 degrees of freedom (DOF)
• Only wand is used in auditorium installation
• Distance of wand from screen determines
modality
• Wand allows pan + zoom
• 2D visualization with 6 DOF interaction
Future Work
• Master G-Sync use
• Develop lower cost solution
for user
interaction: Nintendo Wii remote?
• Support >3 modalities
• Two-handed interaction for pan + zoom
• Learn from NCHC: stereo 4k at Calit2!
Demonstration on 4k Projector
More Information
• IVL Wiki:
•
•
http://ivl.calit2.net/wiki/
Homepage:
http://www.calit.net/~jschulze
E-Mail:
[email protected]
Thank you!