Synchronized Multimedia Interface Language (SMIL) and W3C Multimedia E-Commerce Course November 14, 2002 Mike Christel.

Transcript Synchronized Multimedia Interface Language (SMIL) and W3C Multimedia E-Commerce Course November 14, 2002 Mike Christel.

Synchronized Multimedia Interface
Language (SMIL) and W3C
Multimedia E-Commerce Course
November 14, 2002
Mike Christel
World Wide Web Consortium
• W3C, founded in October 1994: http://www.w3.org/
• Purpose: develop common protocols that promote
WWW’s evolution and ensure its interoperability, in part
by providing:
• A repository of information about the WWW for developers
and users, especially specifications about the web
• A sample code implementation to embody and promote
standards
• Various prototype and sample applications to demonstrate
use of new technology
© Copyright 2002 Michael G. Christel and Alexander G. Hauptmann
2
Carnegie Mellon
More on W3C
• Directed by Tim Berners-Lee, inventor of the WWW
• 452 members as of 11/11/2002 (514 members as of
11/2001, 481 members as of 11/2000; 370 Members as
of 11/1999)
• vendor-neutral forum
• http://www.w3.org/Consortium/Member/List
• Specifications must be formally approved by the
membership via W3C Recommendation track
© Copyright 2002 Michael G. Christel and Alexander G. Hauptmann
3
Carnegie Mellon
W3C Recommendation Track
• Working Draft
Work in progress, commitment to pursue work in area
• Candidate Recommendation
Proposed for implementation experience and feedback
• Proposed Recommendation
Sent to the Advisory Committee for review
• Recommendation
Consensus within W3C, appropriate for widespread
deployment and promote W3C's mission
(source: http://www.w3.org/Consortium/Process/)
© Copyright 2002 Michael G. Christel and Alexander G. Hauptmann
4
Carnegie Mellon
Some W3C Specifications
• User Interface Domain
Hypertext Markup Language (HTML), Cascading Style
Sheets (CSS), Document Object Model (DOM), Scalable
Vector Graphics (SVG), SMIL
• Technology and Society Domain
Platform for Privacy Preferences (P3P), Platform for
Internet Content Selection (PICS), Resource Description
Framework (RDF)
• Architecture Domain
Hypertext Transfer Protocol (HTTP), Extensible Markup
Language (XML)
...Follow links at http://www.w3.org/ for more details…
© Copyright 2002 Michael G. Christel and Alexander G. Hauptmann
5
Carnegie Mellon
XML: A Quick Introduction
• XML is a method for putting structured data in a text file
extensible, support for internationalization/localization,
platform-independent
• XML makes use of tags in ‘<‘ ‘>’ but is not HTML
meaning of tags like “” and attributes left to the
application that reads it
• XML is text, not binary, but is not meant to be read
XML has strict parsing rules; more verbose that binary but
zip/gzip, cheap storage, etc. minimize that disadvantage
• XML is license-free, platform-independent and wellsupported, a W3C standard since February 1998
© Copyright 2002 Michael G. Christel and Alexander G. Hauptmann
6
Carnegie Mellon
SMIL 1.0: Synchronized Multimedia
Integration Language
• SMIL 1.0 specification, June 15, 1998, available at:
http://www.w3.org/TR/REC-smil/
• W3C SMIL references available at:
http://www.w3.org/AudioVideo/
• Purpose: a new language for scheduling multimedia
presentations where audio, video, text and graphics are
combined in real-time
• Result: an XML application enabling authors to specify
what should be presented when
XML implies: extensible, support for internationalization and
localization, platform-independent, editable by hand, etc.
© Copyright 2002 Michael G. Christel and Alexander G. Hauptmann
7
Carnegie Mellon
SMIL 2.0
SMIL 2.0 a W3C Recommendation as of August 7, 2001,
available at: http://www.w3.org/TR/smil20/, with two
design goals:
• Define an XML-based language that allows authors to
write interactive multimedia presentations
• describe the temporal behavior of a presentation
• associate hyperlinks with media objects
• describe the layout of the presentation
• Allow reusing of SMIL syntax and semantics in other
XML-based languages, in particular those that need to
represent timing and synchronization
• SMIL 2.0 components are used for integrating timing
into XHTML and into SVG
© Copyright 2002 Michael G. Christel and Alexander G. Hauptmann
8
Carnegie Mellon
SMIL Timeline
Oct 1996: Workshop
Mar 1997: W3C working group on synchronized multimedia
Jun 1998: W3C Recommendation (SMIL 1.0)
Sep 1998: Timed Interactive Multimedia Extensions for
HTML (HTML+TIME), W3C Note
http://www.w3.org/TR/NOTE-HTMLplusTIME
Feb 1999: 2nd Working Group on Synchronized Multimedia
Modules (SYMM) formed
http://www.w3.org/TR/NOTE-SYMM-modules
Aug 1999: First working draft (SMIL “Boston”) released
© Copyright 2002 Michael G. Christel and Alexander G. Hauptmann
9
Carnegie Mellon
SMIL Timeline, Continued
Dec 1999: Chinese Translation of SMIL 1.0
Jan 2000: RealSlideshow 2.0 from RealNetworks
Jan 2000: Internet Explorer 5.5 Preview by Microsoft
(supports selected modules of SMIL “Boston” draft)
Jan 2000: Apple QuickTime 4.1 now a SMIL 1.0 player
http://www.w3.org/TR/NOTE-HTMLplusTIME
Jun 2000: Fourth Working Draft of SMIL “Boston”
released
Oct 20, 2000: Last call for comments on Public Working
Draft of SMIL 2.0
© Copyright 2002 Michael G. Christel and Alexander G. Hauptmann
10
Carnegie Mellon
SMIL Timeline, Continued
March 2001: New SMIL 2.0 Public Working Draft
June 2001: SMIL 2.0 Proposed Recommendation
August 2001: XHTML + SMIL Profile Working Draft
August 2001: Microsoft Internet Explorer 6.0 includes
implementation of XHTML + SMIL Profile Working Draft
August 2001: W3C Recommendation (SMIL 2.0)
Sept. 2001: SMIL Animation a W3C Recommendation
Sept. 2001: RealNetworks releases RealOne Platform
with full support for SMIL 2.0
Jan. 2002: W3C Note for XHTML + SMIL Profile,
http://www.w3.org/TR/XHTMLplusSMIL/
© Copyright 2002 Michael G. Christel and Alexander G. Hauptmann
11
Carnegie Mellon
SYMM-WG Contributors
Synchronized Multimedia Working Group
contributing organizations:
Compaq, CSELT, CWI, France Telecom,
Gateway, Glocomm, IBM, INRIA, Intel,
Macromedia, Microsoft, Netscape/AOL,
NIST, Nokia, Oratrix, Panasonic, Philips,
RealNetworks, WGBH, W3C
See http://www.w3.org/AudioVideo/Activity
© Copyright 2002 Michael G. Christel and Alexander G. Hauptmann
12
Carnegie Mellon
Example of Contributing Org. Agendas
• WGBH a contributor to Synchronized Multimedia Working
Group
• WGBH partner in CPB/WGBH National Center for
Accessible Media
http://ncam.wgbh.org/aboutncam.html
• Mission: “expand access to present and future media for
people with disabilities…”
• Future directions include audio descriptions of content for
the blind, via new functionality provided in SMIL 2.0
http://ncam.wgbh.org/webaccess/pivot/accesslist.html
© Copyright 2002 Michael G. Christel and Alexander G. Hauptmann
13
Carnegie Mellon
SMIL 1.0 Modules
• Structure
• Meta (“who”, “what”)
• Layout (“where”)
• Timing and Synchronization (“when”)
• Media Elements (“what”)
• Linking
• Content Control
© Copyright 2002 Michael G. Christel and Alexander G. Hauptmann
14
Carnegie Mellon
SMIL High Level Document Structure
<smil>
<head>
<meta>

</meta>
<layout>

</layout>
</head>
<body>

</body>
</smil>
© Copyright 2002 Michael G. Christel and Alexander G. Hauptmann
15
Carnegie Mellon
SMIL 1.0 Structure
<smil>
<head>
</head>
<body>
</body>
</smil>
These elements used in documents based upon
traditional SMIL profiles.
© Copyright 2002 Michael G. Christel and Alexander G. Hauptmann
16
Carnegie Mellon
SMIL 1.0 Meta
<smil>
<head>
<meta … />
</head>
</smil>
The meta elements contain information describing the
document, either to inform the human user or to assist
some automation, e.g.,
<meta name="title" content=”My Italy Trip"/>
<meta name="copyright" content="©1998 WGBH" />
<meta name="base” content="http://test.edu/Italy/"/>
© Copyright 2002 Michael G. Christel and Alexander G. Hauptmann
17
Carnegie Mellon
SMIL 1.0 Layout
<smil>
<head>
<layout>
<root-layout … />
<region id=“R1” … />
<region id=“R2” … />
</layout>
</head>
</smil>
Includes the <layout>, <root-layout> and <region>
elements, and related attributes.
© Copyright 2002 Michael G. Christel and Alexander G. Hauptmann
18
Carnegie Mellon
SMIL 1.0 Layout Examples
<root-layout height="300" width="450" backgroundcolor="#FFFFFF" title="Venezia!"/>
<root-layout height="450" width="625" backgroundcolor="black"/>
<region id="title" left="5" top="150" width="400"
height="200" z-index="1"/>
<region id="videoregion" top="0" left="0" height="240"
width="352"/>
© Copyright 2002 Michael G. Christel and Alexander G. Hauptmann
19
Carnegie Mellon
SMIL 1.0 Timing and Synchronization
<smil>
<body>

</body>
</smil>
• Sequence and parallel timelines, via <seq> and <par>
• Timing control properties, via “begin”, “end”, “repeat”,
“dur”
• “The accuracy of synchronization between the children
in a parallel group is implementation-dependent”
• “soft synchronization” vs. “hard synchronization”
• how to realize hard synchronization
© Copyright 2002 Michael G. Christel and Alexander G. Hauptmann
20
Carnegie Mellon
Hard vs. Soft Synchronization
• Hard synchronization: player synchronizes the children
in the "par" (parallel play) element to a common clock
• Soft synchronization: each child of the "par" element
has its own independent clock
audio
video
audio
video
Hard sync. case 1: stop audio, e.g.,
wait to regain lip synchronization
Hard sync. case 2: drop video frames,
e.g., keep running narrative
audio
video
© Copyright 2002 Michael G. Christel and Alexander G. Hauptmann
Soft sync: audio unaffected
21
Carnegie Mellon
SMIL 2.0 New Focus on Sync. Behavior
<par>
<animation src="..." />
...
<par id="speech" syncBehavior="canSlip" >
<video src="speech.mpg" syncBehavior="locked" />
<audio src="speech.au" syncBehavior="locked" />
</par>
...
</par>
The syncTolerance attribute is referenced when
syncBehavior is “locked”, to ignore a given amount of
slew without forcing resynchronization
The syncMaster attribute defines the element that acts as
the playback "clock" for the time container
© Copyright 2002 Michael G. Christel and Alexander G. Hauptmann
22
Carnegie Mellon
SMIL 1.0 Media Elements
<smil>
<body>

</body>
</smil>
• Includes the media declaration elements <text>, <img>,
<audio>, <video>, <textstream>, <animation>, and
<ref>
• all absolutely positioned elements (animation, img, ref,
text, textstream and video) are contained within a single
containing block defined by the root-layout element
© Copyright 2002 Michael G. Christel and Alexander G. Hauptmann
23
Carnegie Mellon
SMIL 1.0 Linking
<smil>
<body>

</body>
</smil>
• Includes the <a> and <anchor> elements, e.g.,
<a href="http://www.w3c.org" >
<text src="media/w3c.txt" region="w3c”
begin="14.05s" dur="15.95s" />
</a>
• Timing applied to HTML <a> and <area> tags could
provide much or all of SMIL functionality; hence, linking
modules under review
© Copyright 2002 Michael G. Christel and Alexander G. Hauptmann
24
Carnegie Mellon
SMIL 1.0 Content Control
<smil>
<head>

</head>
<body>

</body>
</smil>
Content control based on “<switch>” and test attributes:
system-bitrate, system-captions, system-language,
system-overdub-or-caption, system-required,
system-screen-size, system-screen-depth
© Copyright 2002 Michael G. Christel and Alexander G. Hauptmann
25
Carnegie Mellon
Advantages of Content Control
• Different connections may have different speeds
system-bitrate
• Different workstations may have different capabilities
system-language, system-overdub-or-caption, systemrequired, system-screen-size, system-screen-depth
• Different users may have different needs or media
preferences
system-captions, system-language, system-overdubor-caption
© Copyright 2002 Michael G. Christel and Alexander G. Hauptmann
26
Carnegie Mellon
SMIL 1.0 Example with <switch>
<smil>
<head>
<switch>
<meta name="title" content="CNN Skim, CC via RealText"
system-captions="on"/>
<meta name="title" content="CNN Skim, CC off by player
request"/>
</switch>
<meta name="author" content="Mike Christel (using CNN footage)"/>
<meta name="copyright" content="©Cable News Network" />
<switch>
<layout system-captions="on">
<root-layout background-color="black" height="340" width="352" title="CNN Skim, CC via RealText"/>
<region id="videoregion" background-color="black" top="0" left="0" height="240" width="352"/>
<region id="textregion" top="245" left="10" height="90" width="330"/>
</layout>
<layout>
<root-layout background-color="black" height="240" width="352" title="CNN Skim, CC off by player request"/>
<region id="videoregion1" background-color="black" top="0" left="0" height="240" width="352"/>
</layout>
</switch>
</head>
© Copyright 2002 Michael G. Christel and Alexander G. Hauptmann
27
Carnegie Mellon
SMIL 1.0 <switch> Example, Continued
<body>
<par>

<switch>
<video src="cwv0t4_skm.mpg" region="videoregion" system-captions="on"/>
<video src="cwv0t4_skm.mpg" region="videoregion1"/>
</switch>

<textstream src="CNNskim.rt" region="textregion" system-captions="on"
dur="13s" />
</par>
</body>
</smil>
© Copyright 2002 Michael G. Christel and Alexander G. Hauptmann
28
Carnegie Mellon
SMIL 1.0 Modules, Revisited
• Structure
• Meta (“who”, “what”)
• Layout (“where”)
• Timing and Synchronization (“when”)
• Media Elements (“what”)
• Linking
• Content Control
SMIL 1.0 ~40 pages, while SMIL 2.0 ~400 pages…
© Copyright 2002 Michael G. Christel and Alexander G. Hauptmann
29
Carnegie Mellon
SMIL 2.0 Modules
• Animation Module
• Content Control Modules
• Layout Modules
• Linking Modules
• Media Object Modules
• Metainformation Modules
• Structure Module
• Timing and Synchronization Module
• Transition Effects Module
(see http://www.w3.org/TR/smil20/ for SMIL 2.0 details)
© Copyright 2002 Michael G. Christel and Alexander G. Hauptmann
30
Carnegie Mellon
SMIL Enhancements
• Layout: “media layout can be described using the
either SMIL basic layout syntax or CSS2 syntax (note
that these are not functionally identical)”
• Timing and Sync.: soft/hard synchronization
performance
• Transitions: wipes, fades, dissolves, etc.
• Animation: motion paths, rendering filters like position,
color, size, rotation
• Accessibility (closed captioning, audio descriptions,
localization)
http://ncam.wgbh.org
© Copyright 2002 Michael G. Christel and Alexander G. Hauptmann
31
Carnegie Mellon
SMIL Status Today
• SMIL 1.0 as W3C Recommendation, with players:
RealNetworks G2 (http://www.realnetworks.com/)
Helio SOJA written in Java (http://www.helio.org/)
GRiNS (http://www.oratrix.com/GRiNS/index.html)
Apple QuickTime 4.1
(http://www.apple.com/quicktime/authoring/qtsmil.html)
• SMIL 2.0 as W3C Recommendation, with players:
RealNetworks RealOne Player, Oratrix GRiNS
HTML+TIME in Microsoft IE
SMIL conference (Nov. 2002) and book
(http://www.smilbook.com/tools/index.html)
• SYMM Interest Group work continues for maintaining
SMIL specification…
Follow latest status via http://www.w3.org/AudioVideo/
© Copyright 2002 Michael G. Christel and Alexander G. Hauptmann
32
Carnegie Mellon
SMIL Resources
• Validator for SMIL 1.0 DTD, SMIL 2.0 DTD
http://www.cwi.nl/~media/symm/validator/
• Vendor notes/articles on SMIL
RealNetworks,
http://service.real.com/help/library/guides/production8/htmfiles/
smil.htm
Microsoft, http://msdn.microsoft.com/library/default.asp?
url=/library/en-us/dntime/html/htmltime.asp
Apple, http://www.apple.com/quicktime/authoring/qtsmil.html
• Other articles/examples
http://www.webreview.com/2000/05_05/designers/05_05_00_1.shtml
http://www.webtechniques.com/archives/1998/09/bouthillier/
• PRIMARY REFERENCE: http://www.w3.org/AudioVideo/
© Copyright 2002 Michael G. Christel and Alexander G. Hauptmann
33
Carnegie Mellon
For More Details...
• JustSMIL at http://smw.internet.com/smil/tutor/
• Check URLs referenced here
• View other online SMIL tutorials
• Use products that are newly released or soon to be
released, e.g.,
• RealNetworks SMIL 2.0 support via RealOne Platform; SMIL 1.0
support via RealSlideshow, RealText, etc.
• GRiNS support for SMIL 2.0 at http://www.oratrix.com/GRiNS
• Apple QuickTime SMIL support
• Microsoft IE 5.5 and 6 browser support for SMIL via HTML+TIME
http://msdn.microsoft.com/workshop/Author/behaviors/htmltime.asp
© Copyright 2002 Michael G. Christel and Alexander G. Hauptmann
34
Carnegie Mellon
From W3C SMIL Recommendation to
other W3C Recommendations
• XSLT – Extensible Stylesheet Language
Transformations
• XPath – XML Path Language
• XML Schema
• SVG – Scalable Vector Graphics
• Context will be Informedia Digital Video Library
• Supported by NSF DLI-2 and NSDL Programs
• Images in later slides ©1999-2001, Cable News Network
© Copyright 2002 Michael G. Christel and Alexander G. Hauptmann
35
Carnegie Mellon
Context: Informedia Digital Video Library
© Copyright 2002 Michael G. Christel and Alexander G. Hauptmann
36
Carnegie Mellon
Sample XML for Informedia Results Set
<?xml version="1.0" ?>
<IDVLResults xmlns:im="x-schema:idvResSchema.xml">
<im:doc>
<im:id>55195</im:id><im:pos>0</im:pos><im:shot>1667499</im:shot>
<im:d_yr>1999</im:d_yr><im:d_mo>12</im:d_mo><im:d_day>2</im:d_day>
<im:score>100</im:score><im:dur>29930</im:dur><im:mmss>0:29</im:mmss>
<im:title>The Maker of Cherry Garcia and Rainforest Crunch Ice Cream Could
Soon be Scooped Up; Ice Cream Maker Didn't Name Potential</im:title>
</im:doc><im:doc>
<im:id>54770</im:id><im:pos>1</im:pos><im:shot>2111578</im:shot>
<im:d_yr>1999</im:d_yr><im:d_mo>11</im:d_mo><im:d_day>19</im:d_day>
<im:score>70</im:score><im:dur>196480</im:dur><im:mmss>3:16</im:mmss>
<im:title>So, While Ice Cream Parlor Closed, in Suite Next Door to where Tim and
Heidi Manipulate Bones, Tim and Heidi now also Fill Jars </im:title>
</im:doc></IDVLResults>
© Copyright 2002 Michael G. Christel and Alexander G. Hauptmann
37
Carnegie Mellon
Adding Style to XML
• XML only text-based, but Informedia presentation is
visually rich; also ideally tailorable
• Extensible Stylesheet Language (XSL) and XSL
Transformations (XSLT) available (XSL as working
draft, XSLT as W3C Recommendation Nov. 16, 1999,
http://www.w3.org/TR/xslt)
• XSLT: a language for transforming XML documents into
other XML documents
• XPATH: a language for addressing parts of an XML
document, designed to be used by both XSLT and
XPointer
© Copyright 2002 Michael G. Christel and Alexander G. Hauptmann
38
Carnegie Mellon
XSLT for Informedia Results Display
<xsl:template match="IDVLResults"><xsl:for-each select="im:doc">
<xsl:sort select="im:score" order="descending" data-type="number" />

<xsl:attribute name="tip">
<xsl:value-of select="im:title" />, <xsl:value-of select="im:mmss" />, <xsl:value-of select="im:d_mo"
/>-<xsl:value-of select="im:d_day" />-<xsl:value-of select="im:d_yr" /></xsl:attribute>
<img id="Stamp_{im:pos}" src="graphics/Gstamp.gif" alt="" orgsrc="graphics/Gstamp.gif"
oversrc="graphics/Gltstamp.gif" width="112" height="91" />
<xsl:variable name="nScore" select="0.8*im:score" />
<xsl:variable name="nScoreTop" select="85 - $nScore" />
<xsl:variable name="sScoreHt" select="formatNumber($nScore, '#.')" />
<img id="Th_{im:pos}" src="graphics/red.gif" alt="" onmouseover="stampChangeOver();"
onmouseout="stampChangeOut();">
<xsl:attribute name="style">position:absolute; left:9; width:4; top:<xsl:value-of
select="formatNumber($nScoreTop, '#.')"/>; height:<xsl:value-of select="$sScoreHt"/>;
</xsl:attribute></img>
<img id="I_{im:pos}" src="http://test.edu/idvQuery/GetShot.asp?{im:shot}" style="position:absolute;
left:23; top:9" alt="" width="80" height="55" onmouseover="stampChangeOver();"
onmouseout="stampChangeOut();" />
</xsl:for-each></xsl:template>
© Copyright 2002 Michael G. Christel and Alexander G. Hauptmann
39
Carnegie Mellon
XSLT for Informedia, larger font, 1 of 2
<xsl:template match="IDVLResults">
<xsl:for-each select="im:doc">
<xsl:sort select="im:score" order="descending" data-type="number" />

<xsl:attribute name="tip">
<xsl:value-of select="im:title" />, <xsl:value-of select="im:mmss" />,
<xsl:value-of select="im:d_mo" />-<xsl:value-of select="im:d_day" /><xsl:value-of select="im:d_yr" /></xsl:attribute>
<img id="Stamp_{im:pos}" src="graphics/Gstamp.gif" alt=""
orgsrc="graphics/Gstamp.gif" oversrc="graphics/Gltstamp.gif"
width="112" height="91" />
<xsl:variable name="nScore" select="0.8*im:score" />
<xsl:variable name="nScoreTop" select="85 - $nScore" />
<xsl:variable name="sScoreHt" select="formatNumber($nScore, '#.')" />
© Copyright 2002 Michael G. Christel and Alexander G. Hauptmann
40
Carnegie Mellon
XSLT for Informedia, larger font, 2 of 2
<img id="Th_{im:pos}" src="graphics/red.gif" alt=""
onmouseover="stampChangeOver();"
onmouseout="stampChangeOut();">
<xsl:attribute name="style">position:absolute; left:9; width:4;
top:<xsl:value-of select="formatNumber($nScoreTop, '#.')"/>;
height:<xsl:value-of select="$sScoreHt"/>;
</xsl:attribute>
</img>
<img id="I_{im:pos}"
src="http://test.edu/idvQuery/GetShot.asp?{im:shot}"
style="position:absolute; left:23; top:9" alt="" width="80"
height="55"
onmouseover="stampChangeOver();"
onmouseout="stampChangeOut();" />
</xsl:for-each></xsl:template>
© Copyright 2002 Michael G. Christel and Alexander G. Hauptmann
41
Carnegie Mellon
XML Namespaces and Schemas
• Used for validation and data typing
• Informedia example:
• XML documents begin with:
<IDVLResults xmlns:im="xschema:idvResSchema.xml">
• XSL documents can then do:
<xsl:sort select="im:score" order="descending"
data-type="number" />
• So, Informedia web presentation consists of:
• XML data, validated and typed via XML schema,
converted via XSLT to XHTML for browser presentation
© Copyright 2002 Michael G. Christel and Alexander G. Hauptmann
42
Carnegie Mellon
Scalable Vector Graphics (SVG)
• World Wide Web Consortium (W3C) Recommendation
as of Sept. 4, 2001 (http://www.w3.org/Graphics/SVG)
• A language for describing two-dimensional graphics in
XML
• Vector graphic shapes
• Images
• Text
• Drawings can be dynamic and interactive
• SVG provides an ideal framework for presenting
interactive summarizations into a multimedia
information repository
© Copyright 2002 Michael G. Christel and Alexander G. Hauptmann
45
Carnegie Mellon
One Style of Web Presentation
Architecture
Informedia
Database
DB
Request
Results
Web Server
XML Conversion
and Validation
Query/Browse
Processor
Request
XML
XSL Processor
HTML
Thin XML
(e.g., Dublin
Core)
Web Browser
© Copyright 2002 Michael G. Christel and Alexander G. Hauptmann
XSL Style
Sheet
46
XHTML
Basic
Carnegie Mellon
Flexibility, Greater Interactivity through
Client-Side Processing
Query/Browse
Processor
DB
Request
XML
Conversion and
Validation
Results
Informedia
Database
Web Server
Request
XML
XSL Processor
XSL Style
Sheet
Web Browser
© Copyright 2002 Michael G. Christel and Alexander G. Hauptmann
HTML
View 1
47
…
HTML
View n
Script to
add to
SVG
DOM
SVG
View s
Carnegie Mellon
SVG Map for Entering Geographic
Queries
• SVG map can be panned, zoomed within client
• Can select specific countries and states/provinces, or
perform a region-based search, as shown here:
© Copyright 2002 Michael G. Christel and Alexander G. Hauptmann
48
Carnegie Mellon
XML Video Results Set as Thumbnail Grid
© Copyright 2002 Michael G. Christel and Alexander G. Hauptmann
49
Carnegie Mellon
Same Results Set Drawn as SVG VIBE Plot
Same Results Set Drawn as SVG Map
• Date filter illustrates one way to make SVG interactive
© Copyright 2002 Michael G. Christel and Alexander G. Hauptmann
51
Carnegie Mellon
Overview-Zoom-Details Support via SVG
• Overviews like world maps or Visualization by Example
(VIBE) plot can naturally be zoomed to smaller areas
which can then be rendered larger and in greater detail,
since SVG supports vector-based drawing:
© Copyright 2002 Michael G. Christel and Alexander G. Hauptmann
52
Carnegie Mellon
The Need for Visualization Strategies
• As digital video library grows, so do result sets
• As automated processing techniques improve, e.g.,
speech and image processing, more metadata with
which to build interfaces into video
• Need overview capability to deal with greater volume
• Prior work offered many solutions:
• Visualization By Example (VIBE) for matching entity
relationships
• Scatter plots for low dimensionality relationships, e.g.,
timelines
• Dynamic query sliders for direct manipulation of plots
• Colored maps for geographic relationships
• SVG can deliver such interactive visualizations
© Copyright 2002 Michael G. Christel and Alexander G. Hauptmann
53
Carnegie Mellon
Another Example, using Documentary Data
© Copyright 2002 Michael G.
Christel and Alex
Hauptmann
Filtering through VIBE SVG Interaction
Move anchor words, zoom into 8
documents for “(water AND
management) AND (irrigation OR
dam)” through mouse interaction.
© Copyright 2002 Michael G. Christel and Alexander G. Hauptmann
Manipulating Filtered Data with SVG Map
Select “North
Dakota” and see
remaining 2 video
clips, so that grid
of 100 thumbnails
for result set is
now reduced to an
easily navigable
grid of 2 clips:
© Copyright 2002 Michael G. Christel and Alexander G. Hauptmann
56
Carnegie Mellon
Benefits of SVG
• Quick generation
• Typically much smaller than raster pixel-based image
• Clean rendering at different scaling factors (vector-based)
• Efficient manipulation (e.g., XPath navigation of SVG tree allows
colors and sizes to be changed for specific elements)
• Standard ways for zooming into and out of regions of interest
• W3C Recommendation, well integrated with other W3C
Recommendations (XML, XPath, XSLT, SMIL Animation)
© Copyright 2002 Michael G. Christel and Alexander G. Hauptmann
57
Carnegie Mellon
What’s Next… Better Integrating Imagery
© Copyright 2002 Michael G. Christel and Alexander G. Hauptmann
58
Carnegie Mellon
Future SVG VIBE with More Detail
As document count drops when zooming into an area, add
representative thumbnails for those remaining documents
© Copyright 2002 Michael G. Christel and Alexander G. Hauptmann
59
Carnegie Mellon

Synchronized Multimedia Interface Language (SMIL) and W3C Multimedia E-Commerce Course November 14, 2002 Mike Christel.

Transcript Synchronized Multimedia Interface Language (SMIL) and W3C Multimedia E-Commerce Course November 14, 2002 Mike Christel.

Directory