Transcript Extensible 3D (X3D) Graphics Requirements for Video on the Web Don Brutzman W3C Workshop, Video on the Web 12 December 2007 Web3D Consortium www.web3D.org Naval Postgraduate.

Extensible 3D (X3D) Graphics
Requirements for Video on the Web
Don Brutzman
W3C Workshop, Video on the Web
12 December 2007
Web3D Consortium www.web3D.org
Naval Postgraduate School
Monterey California USA
Topics
X3D Overview
Functional Requirements
• Video in X3D Scenes
• Converting 3D Models to Video
Recommendations
X3D Overview
www.web3D.org
The Web3D Consortium is a public-private
partnership of industry, agencies, universities
and individuals working on open standards
for real-time 3D communication on the Web.
Web3D develops, implements, evaluates and
writes the Extensible 3D (X3D) Graphics
specification.
Today’s www.web3D.org page
Community rules
Thanks to steady innovation by Web3D
members and users, new features continue to
evolve and grow into X3D capabilities
Working groups identify requirements and write
implementable specifications
Individuals can directly extend specification
capabilities without formal “permission”
ISO
Formal annual review and ratification by the
International Standards Organization (ISO)
have made X3D an approved standard for
real-world use, both on and off the Web
Experts from 12-15 nations review our specs
Immediate adoption by other governing bodies
helps to increase deployment
W3C
Further collaboration by Web3D Consortium
with the World Wide Web Consortium (W3C)
has made X3D a "first-class citizen" on the
Web, providing excellent (and growing)
interoperability with other XML standards
Current joint work includes efforts on Efficient
XML Interchange (EXI) for compression and
processing improvements.
Intellectual Property Rights (IPR)
Web3D and W3C have similar policies
Any known patented technology must be
declared by members prior to consideration
by working groups
Any patented technology contributions must be
licensed on a royalty-free (RF) basis for
inclusion in an openly used Web standard
http://www.web3d.org/membership
Caveat: any legal problem can be solved, but
only in advance
Open Source
Open for any use, without license fees
Free = freedom to innovate
• Not necessarily free cost (more like “free puppy”)
Common shared example implementation(s)
• Not a reference implementation – the specification/standard
hopefully provides that – but required nevertheless
Open source implementations are necessary to break
logjams when company participants can’t resolve
technical issues
Also provides self-sustaining business model for
continued activity, improvement
IPR summary
Royalties incompatible with Web architecture
Open standards + open source are both needed
• Open source by itself is necessary but insufficient
• Proprietary implementations are great too
Standards organizations and IPR agreements
provide a stable playing field for long term
X3D Profiles for Extensibility
Different levels of
content complexity
Browsers can support
increasing levels of
capability
Authors can use the
proper palette for
intended delivery
MovieTexture node is
Immersive Profile
Example syntax
<Shape>
<Box/>
<Appearance>
<MovieTexture/>
<TextureTransform/>
</Appearance>
</Shape>
Family of X3D specifications
Abstract specification describes “how it works”
Equivalent encodings
XML .x3d, ClassicVRML .x3dv, Binary .x3db
Scene Access Interface (SAI)
Consistent programming in EcmaScript, Java
X3D Specifications honeycomb
diagram
The key challenge is scalability
Because the only information systems capable
of scalably growing to match global scope are
the Internet and the World Wide Web, X3D
deliberately follows the architectural
principles of World Wide Web.
Architecture of the World Wide Web, Volume
One http://www.w3.org/TR/webarch
Functional Requirements
Video within X3D Scenes
3D Geometry from Video
Linkability
Linkability
• Video content needs to be referencable by URI
• X3D approach allows ordered list of equivalent
URI addresses for reliable portability, access
“Bookmark” additions are desirable for video,
similar to using Viewpoints as X3D bookmarks
• Further annotation of location or time as part of
URI is under consideration
Planet 9 Raygun demos
raygun.planet9.com
Playability
Internal playability
• Video is often used as a user-facing “billboard”
within the X3D scene
• Can be special display or match a particular view
• Mixed rendering is challenging but feasible
• Performance requirements are demanding
External playability
• Anchor links or scripts may launch external video
displays for multimedia experience
Interoperability
Small number of required formats:
• PNG JPEG GIF images, WAV audio, MPEG-2 video
• Motivated by performance and royalty-free (RF)
• Alternate formats are allowed but might not be
supported, url list can alleviate this problem
• X3D browsers not (yet?) plugging in video codecs,
but plugin interoperability continues to improve
Adoption of imagery, audio and video formats
supported by W3C is shared goal for Web3D
Planet 9 RayGun scene 1
Webcam stream as MovieTexture Billboard
Animation
Need synchronization of video playback with
3D-centric event-based behavior model
• Necessary for consistent playability
• Connect simple X3D animation interpolators,
sequencers
• Unlocks both Ecmascript and Java, either inside
X3D Script node or externally in HTML page
Future work assignment
• investigate possible alignment of X3D event model
with W3C’s Synchronized Multimedia Integration
Language (SMIL)
Metadata overview
X3D included document metadata (similar to
html meta tags) and typed Metadata nodes
• Also Humanoid Animation (H-Anim) joint naming
Some metadata conventions are emerging
• SAVAGE Modeling and Analysis Language (SMAL)
No metadata vocabularies yet formalized by
Web3D
• Won’t reinvent other unencumbered standards
Metadata requirements
Video metadata of interest
• duration, loopability, whether file or streamed
• time, geographic location for precise positioning in
virtual environment
• Camera direction (and perhaps settings)
• Ability to add other metadata extensions
Metadata needs to be inspectable by browser
• Use cases available for video embedded within, or
external to, video content stream
Construction of 3D geometry
Current research demonstrating ability to
automatically generate 3D models from
carefully registered/recorded video streams
• Variety of techniques are being developed
Applications include city modeling, tracking
human pedestrians, classifying produced
objects, querying associated databases, etc.
Planet 9 RayGun scene 2
Numerous efforts
generate matching
3D geometry using
video sources
Security
Scene authors often want to encrypt or digitally
sign 3D content
Content is king, but creation is hard
W3C recommendations for XML Security appear
appropriate: encryption, signature, PKI
Security needs to be compatible with digital
rights management (DRM) scheme
Stream security needed, but not yet addressed
Digital rights management
X3D’s XML and Compressed Binary encodings
allow use of W3C’s Security recommendations
XML Encryption
XML Digital Signature (for authentication)
Public key infrastructure
DRM appears to be feasible, but not used yet
More uses exist than Hollywood-commercial
See Sun’s DReaM project
http://www.openmediacommons.org
Streamability
Numerous historic examples of streamed video in 3D
environments
Streamed video in X3D browsers is feasible but
infrequent
• Likely due to performance and format permissions
Real Time Streaming Protocol (RTSP) might be mapped
to distribute X3D event model among multiple players
• Exemplar is Distributed Interactive Simulation (DIS) protocol
Clear shared standards strategy might motivate Web3D
member support for necessary next-step efforts
Planet 9 RayGun scene 3
Browsers have ability to stream video within scenes
Synthesis
Merging recorded or live video with 3D
geometry
• Motion tracking for human avatars
• View from a window
Likely future capability of broad interest
Transparency
Goal: superposition of selected video pixels of
interest without box artifacts
• Example: video talking head or human avatar
without confounding background
Ability to encode some pixels as transparent
• Red Green Blue Alpha (RGBA) pixel encoding is
not practical for video streams
• Designating certain color as transparent is
feasible, e.g. “green screen”
Export
Recording of animated activity in immersive
environment, as seen from one or more
animated viewpoints
• Practice becoming more commonplace, (e.g.
video-game players save victorious sessions)
• Different from playing back events within 3D
• Hollywood cliché “What I really want is to direct”
now becoming feasible
• Similar requirements likely (e.g. video metadata)
Recommendations
Recommendations
X3D graphics requirements for Web video are
technically and politically feasible
• Interactive 3D performance is biggest challenge
• Plethora of oddball non-RF codecs unsustainable
• Royalty free model for video content thus essential
Consider X3D technical requirements for video
when building the foundation for emerging
multimedia Web
Contact
Don Brutzman
[email protected]
http://web.nps.navy.mil/~brutzman
Code USW/Br, Naval Postgraduate School
Monterey California 93943-5000 USA
1.831.656.2149 voice
1.831.656.7599 fax