Transcript Enabling integrated cross-media entertainment services

Enabling integrated cross-media
entertainment services over
heterogeneous networks
Introduction:
Reasoning, business aspects, involved processes
Dr. Kostas Karpouzis
Image, Video and Multimedia Systems Lab, National Technical University of Athens
Institute of Communication and Computer Systems
The current scene…

Digital Content
 A valuable asset nowadays
 Associated with a variety of emerging and novel distributed
multimedia services


Big revenue bringers: entertainment services
Convergence of media and technologies
 From single, distinct ones to multi-disciplinary ones

Allowing advanced service provisioning and consumption
The current scene…



The plethora of terminal devices available for networking access
lead to immense market opportunities for content and service
providers
The emergence of advanced network infrastructures enables fast,
efficient and reliable end-to-end transmission of huge multimedia
content
Media providers faced with two choices:
 select the most appropriate device for their content
and tailor the consumer experience and business models to
take advantage of this environment
 invest in multiple devices, access networks and content
formats


the current trend…
enables transparent access to services, thus more subscribers
Why entertainment..?

Entertainment and leisure market is an
attractive commercial target for many business
groups


It further allows increasing revenue streams


Data providers, telecommunication companies,
broadcasters, online publishers, device producers
A never-ending consumer interest
Online entertainment and leisure market is a
small percentage of the overall

it is expected to become the major boost of this
business sector
The underlying concepts…

Cross-media world: what does it mean?
 Publish/consume content to a variety of terminals
 fixed or mobile, accessing heterogeneous networks
 Actually refers to:
 Create once, Publish everywhere (COPE paradigm)


Universal access to content (UMA paradigm)


At content/service providers, mediators, network operators
From content/service providers to end-users
Heterogeneous networks
Diverse core and access networks
 Satellite, wireless, mobile, cable, ISDN, …
 Different QoS management
 User-oriented
 Services driven by users’ real needs and preferences

So…

How can the same content be transmitted over
different channels and consumed at various
terminals without costly re-processing and reproduction under specific QoS requirements
dictated both by content/service providers and
end-users?
The business implications

Content/service providers, network operators,
content/service mediators


increase market competitiveness and advantage
establish new user-centered business opportunities





definition of new business models
increase subscribers
open up new revenue streams
be technologically advanced and thus competitive
Consumers


Access to any type of entertainment service wherever they
are, using any type of terminal
Services tailored to their needs and preferences
The underlying value chain
Content
/ Service
Creation
Content
/ Service
Mediation &
Management
Content
Distribution
& Delivery
Digital Rights Management
Personalized
Consumption
The technological requirements

Enable




Create once, publish everywhere
Universal multimedia access
Adapting technology to people & people being the
centre
i.e., primarily

Context awareness, content adaptation



scalable coding, automatic transcoding
use of standardized metadata descriptions & profiling
(MPEG-7, MPEG-21)
Personalization
The technological requirements

Other requirements involved

At the service provider side




At the mediation/network layer



Monitoring
End-to-end QoS provisioning
At the end user layer




Service management, discovery, integration
Smart, self-adaptive content authoring & aggregation
Context modeling
interactivity
mobility
transparent access to services
In all, digital rights management and protection
Abstract end-to-end system design
Digital Rights Management
Content and
Service
Creation
Content and
Service
Mediation
Scalable
coding
Content description /
representation - metadata
Content
Authoring
Service
Providers
Content
Providers
Re-production
Workflow
Cross Media
Provisioning
Service
Management
Content
Aggregation
Content
Adaptation
Content
Transcoding
Context
Awareness
Identification
Context Modeling
Discovery
Registration
Service/
Content
Mediators
Integration
Content
Distribution
& Delivery
Personalized
Consumption
Transmission
Personalization
PC
PDA
QoS
Management
User/Terminal Profile
Mobile
Digital TV
Users
Enabling integrated cross-media
entertainment services over
heterogeneous networks
The concepts…:
Context Awareness, Content Adaptation
Context Awareness

Context

includes any information that characterizes an entity’s situation




is a function of time and environment
environment is a function of users, services, resources and other entities in
environment
Context-aware system

when using contexts to provide relevant information or services to the user


Relevance depends on user’s task
Context-aware applications


Entity: person, place, object relevant to an interaction between a user and an
application
must detect, interpret and respond to contexts
Contextualization



enables efficient content adaptable services and systems
important component of the ubiquitous/pervasive paradigm
aids in understanding the environment sufficiently
Context Awareness

Context-aware computing involves



collection of context information
dynamic program behavior dictated by knowledge environment
This is enabled by:

Use of profiles



Environment (network, terminal, service)
User (preferences, location, …)
represented in standardised format (MPEG-7, MPEG-21)





since a common representation format is required
to allow for interoperability in a distributed world, over heterogeneous
networks and a variety of terminals
Monitoring mechanisms gathering context information
Decision engines that use context information as input to adapt
content based on environment status

Usually processed at service mediation and adaptation layers, but
gathered stored and interpreted at different parts of the system such as
end-user terminal devices or access networks

a necessary step before content adaptation
So…
Context Awareness

A context profile should be

Structured


Aids in effectively filtering relevant information
Interchangeable


Among different system components
Mechanisms for transferring sub-trees (and not entire profiles) desirable


Composable/decomposable



Eases interpretation
Extensible


To allow maintenance in a distributed way
To grasp changes in environment
Uniform


To compensate for big network delays
To allow for future requirements/advances
Standardized

To allow interoperable exchange among different entities in a distributed
environment
Context Awareness

Context-aware infrastructure

Physical and logical sensors (software entity, network
components, software agents) used to collect context
information related to presence, location, identity and profile of
users and services/content

Typical context use






Passive


Locate services and users
Call up services according to user behavior
Provide information for service composition
Facilitate ad hoc communication mechanism between users
Adaptation of QoS to changes in environment as a result of user
and service mobility
context gathering and representation
Active

smart context information delivery
Content Adaptation

Why adapt contents?






Most contents for viewing are for the larger screens
Creating multiple versions a burden
Even if you don’t mind, there are just too many
possible devices
Different users want different things
Having one, original version is easier to manage
Content adaptation is about generating any
content version from one single original
version
 Create once, Publish everywhere (COPE)
 Universal Multimedia Access (UMA)
Content Adaptation

Pre-adaptation


Keeping just the original version (any other version is runtimegenerated) could be slow
Pre-adaptation



to create all possible versions (at creation/production phase),
and do static “selection” at runtime, or
to create just a few essential versions, and do dynamic
adaptation – hence the “balance”
Dynamic adaptation


On-the-fly adaptation
Where it happens


Either at end systems (end-to-end service)
Or at intermediate network nodes (active service)

At specially designed mediators
Content Adaptation

Major processes involved
Access to and management of context
information and content/service metadata
 Decision engine to negotiate further actions
 Adaptation by

Layered coding (scalable bitstreams)
 Transcoding

Content Adaptation
A Content
Adaptation
System
Mediation Layer
Content Adaptation
Use of context profile
Information and content
Instance metadata
to decide upon transcoding
strategies
Enabling integrated cross-media
entertainment services over
heterogeneous networks
The concepts…:
Scalable Coding/Transcoding
Transcoding introduced

Video transcoding: converting a
previously compressed video signal into
another one


Possibly different bit rate, frame rate, frame
size or compression standard
Initially proposed for compressed video
bit rate scaling

Sun et al., 1996
Transcoding introduced (2)

Also used for:

Video frame size conversion


spatial resolution
Video frame rate conversion

temporal resolution
Bit rate adaptation
 Multipoint video combining
 Error resilience

Transcoding for heterogeneous nets
Diversity of channel capacities
 Channel capacity of the outgoing
channel may be less than that of the
incoming channel



Or may change over time
Distribution to users with different
connections

target transmission channel conditions are
generally unknown during encoding
Major transcoder architectures

Cascaded Pixel Domain Transcoder
Keesman et al., 1996
 Close-loop transcoder connects a standard
decoder and a standard encoder together
 Most straightforward method

Major transcoder architectures (2)

Open-Loop Transcoder
Sun et al., 1996
 Input video bitstream is first partially
decoded to the DCT coefficient level
 Bit rate is scaled down by cutting higher
frequency coefficients or by requantizing all
coefficients with a larger quantization step
size

Major transcoder architectures (3)

DCT domain transcoder (DDT)
Zhu et al., 1999
 requantization of the DCT coefficients
 error is stored in a buffer and is fed back to
the requantizer to correct the requantization
error introduced in previous frames
 Simplifies architecture of OLT by reusing
motion vectors
 Merges two motion compensation loops in
the CPDT into one

Video rate control

Accurate and dynamic control of the
output bit rate according to the channel
bandwidth
Video rate control (2)

Two different steps:
Frame-Layer Rate Control: allocate target
bits for each frame according to image
complexities, buffer occupancy, or a given
channel bit rate
 Macroblock-Layer Bit Allocation: derive the
actual quantization parameter for each
picture Macroblock and make the number of
produced bits meet the bit target

Frame-Layer Rate Control
Bit budget for every frame has to be
determined considering the channel bit
rate and buffer occupancy
 Assumption for constant bit rate (CBR)

Internet transmission cannot provide a
guaranteed constant bit rate channel
 wireless channels: high bit error rate and
variable effective channel bit rate

Frame-Layer Rate Control (2)

Theoretically, compression and error
protection can be performed separately
and sequentially


Shannon’s separation theorem
In practical video communication
systems source coding and channel
coding are combined together
Wireless video communication

Wire line channels: signal strength is
relatively constant


reception errors mainly due to additive noise
Wireless channels: reception errors
mostly due to time-varying signal
strength
multi-path propagation from local scatters
 error bursts

Wireless video transmission
Delay constraints
 Channel errors hurt perceptual quality of
video at decoder
 Evident in standard coders

variable length coding
 predictive coding compression
 e.g., MPEG or H.263

Challenge transmission error

At the encoder side:
layered coding
 multiple description coding
 error resilience entropy coding


Increase robustness of video stream
against channel errors
Challenge transmission error (2)

At the decoder side:


Error concealment techniques recover lost
information without relying on information from the
encoder
Channel coding techniques


FEC codes may cause unnecessary overhead and
bandwidth waste when channel is in good state
ARQ error control requires retransmission only
during periods of poor channel conditions
Abstract architecture
Delay = Processing Delay + Transmission Delay + Buffer
Delay
Adaptive transcoding

Frame layer rate control based on video
content
frame types
 video source coding rate
 scene changes

Frame types and source video
MPEG-1, -2 and H.26x use motion
compensation
 Motion compensation reduces temporal
redundancy between successive frames

MPEG-1 and MPEG-2

In MPEG, a GOP (Group of Pictures) contains
one I-frame (intra) and several P- or B-frames
(inter) in a certain pattern
 I-frame: no motion compensation


motion compensation of following frames depends
on I-frame quality
P-frames use the previous I B-frames use both the previous and
successive I- or P-frames as references for
motion compensation
H.263

Focus on low bit rate


More popular for wireless applications
Fixed quantization parameter
Near constant visual objective quality
 Varying bit rate

scene content
 different frame types


Constant rate  quality degradation
H.263 (2)

Source bit rate can be recorded during
encoding and saved as side information


unified quantization parameter in same type
of frames  generated frame size indicates
scene variations and motion activities
Thus, video content can be measured by
the source bit rate

Used to calculate bit budget for every frame
Scene changes

Scene changes represent distinctive
differences between adjacent video
frames
Rapid motion of moving objects
 Changes to different visual content


Information obtained from previously
coded frames is no longer useful
Scene changes (2)
When scene changes happen, the first
frame after scene change should be
transcoded in high quality to prevent
quality degradation after scene change
 Scene change detection

Most MBs in the anchor frame will be
encoded as INTRA blocks
 INTRA mode MBs percentage can be used
to detect scene changes

Adaptive frame layer rate control
Relaxed requirement for end-to-end
delay
 Allowed initial startup delay



decoder buffer smoothes delay jitters
H.263 determines frame budget by buffer
occupancy and channel bandwidth

nearly constant bit budget  similar end-toend delay for each frame
Adaptive frame layer rate control (2)

Adaptive methods consider scene
changes, frame types and source coding
rate


e.g., I-frames are transcoded as I-frames,
with unified quantization parameter used for
every MB, while the bit budget for B-frames
will be half of that for P-frames
Scene change detected  anchor frame
is transcoded as an I-frame
A practical scenario

Offline encoding
Initial presentation is prepared according to
pre-defined scenarios
 Uses pre- created static content


Online transcoding
In charge of transmission of initial
presentation
 Generation, encoding and transmission of
real-time interactions

Enabling integrated cross-media
entertainment services over
heterogeneous networks
The technologies…:
Metadata & Profiling (environment, user, terminal)
The need for standardised metadata
(MPEG-7)


Today people cannot easily create, find, edit, share, and reuse
media
Computers don’t understand media content




Media is opaque and data rich
We lack structured representations
Without content representation (metadata), manipulating digital
media will remain like word-processing with bitmaps
Need standardized metadata framework




Designed for video and rich media data
Human and machine readable and writable
Standardized and scalable
Integrated into media capture, archiving, editing, distribution, and
reuse
MPEG-7

MPEG-7


Describing the multimedia content data that supports some
degree of interpretation of the information’s meaning, which
can be passed onto, or accessed by, a device or a computer
code
Motivation



Create standardized multimedia description framework
Enable content-based access to and processing of multimedia
information on the basis of descriptions of multimedia content
and structure (metadata)
Support range of abstraction levels for metadata from lowlevel signal characteristics to high-level semantic information
MPEG-7 Query Examples






Play a few notes on a keyboard and retrieve a list of musical
pieces similar to the required tune, or images matching the notes
in a certain way, e.g., in terms of emotions
Draw a few lines on a screen and find a set of images containing
similar graphics, logos, ideograms,...
Define objects, including color patches or textures and retrieve
examples among which you select the interesting objects to
compose your design
On a given set of multimedia objects, describe movements and
relations between objects and so search for animations fulfilling
the described temporal and spatial relations
Describe actions and get a list of scenarios containing such
actions
Using an excerpt of Pavarotti’s voice, obtaining a list of Pavarotti’s
records, video clips where Pavarotti is singing and photographic
material portraying Pavarotti
MPEG-7 Sample Application Areas

Architecture, real estate, and interior design



(e.g., personalized advertising, on-line
catalogues, directories of e-shops)
Education


(e.g., image catalogue, musical dictionary,
bio-medical imaging catalogues, film,
video and radio archives)
E-Commerce




(e.g., repositories of multimedia courses,
multimedia search for support material)
Home Entertainment

(e.g., systems for the management of
personal multimedia collections, including
manipulation of content, e.g. home video
editing, searching a game, karaoke)

(e.g., searching for clothes that you like)
Social


(e.g., cartography, ecology, natural
resources management)
Shopping


(e.g., personalized electronic news
service, media authoring)
Remote sensing


(e.g. yellow pages, Tourist information,
Geographical information systems)
Multimedia editing


(e.g. searching speeches of a certain
politician using his name, his voice or his
face)
Multimedia directory services


(e.g., human characteristics recognition,
forensics)
Journalism
(history museums, art galleries, etc.)
Digital libraries


(e.g., radio channel, TV channel)
Investigation services

Cultural services


(e.g., searching for ideas)
Broadcast media selection


(e.g. dating services)
Surveillance

(e.g., traffic control, surface transportation,
non-destructive testing in hostile
environments)
MPEG-7 Scope

To define the description framework
Description
Description
generation
generation
Description
Description
Description
Description
consumption
consumption
Scope of MPEG-7
MPEG-7 Metadata Framework

Data


Feature


“A representation of a Feature. A Descriptor defines the syntax and the
semantics of the Feature representation.”
Description Scheme


“a distinctive characteristic of the data [that] signifies something to
somebody.”
Descriptor


“multimedia information that will be described using MPEG-7,
regardless of storage, coding, display, transmission, medium, or
technology.”
“The structure and semantics of the relationships between its
components, which may be both Descriptors and Description Schemes.”
Description Definition Language (XML Schema)

“A language that allows the creation of new Description Schemes, and,
possibly, new Descriptors. It also allows the extension and modification
of existing Description Schemes.”
MPEG-7 Framework
Description Definition
 extension
Language
Definition
Tags
DS1
D2
D7
DS2
Instantiation
D1
D3
D6
D10
D1
D4
D9
D8
D5
D2
DS4
Structuring
D6
D3
<scene id=1>
<time> ....
<camera>..
<annotation
</scene>
DS3
D4
D5
Descriptors:
Description Schemes
(Syntax & semantic
of feature representation)
Encoding
&
Delivery
1 10101 0
MPEG-7 Standard Parts
MPEG-7 Systems
1.

The binary format for encoding MPEG-7 descriptions and the terminal architecture
MPEG-7 Description Definition Language
2.

The language for defining the syntax of the MPEG-7 Description Tools and for
defining new Description Schemes
MPEG-7 Visual
3.

The Description Tools dealing with (only) Visual descriptions
MPEG-7 Audio
4.

The Description Tools dealing with (only) Audio descriptions
MPEG-7 Multimedia Description Schemes
5.

The Description Tools dealing with generic features and multimedia descriptions
MPEG-7 Reference Software
6.

A software implementation of relevant parts of the MPEG-7 Standard with normative
status
MPEG-7 Conformance Testing
7.

Guidelines and procedures for testing conformance of MPEG-7 implementations
MPEG-7 Extraction and Use of Descriptions
8.

Informative material (in the form of a Technical Report) about the extraction and use
of some of the Description Tools (under development)
MPEG-7 Description Tools
MPEG-7 Top Level Hierarchy
MPEG-7 Still Image Description
Referencing Temporal Media
MPEG-7 Video Segments Example
MPEG-7 Segment Relationship Graph
MPEG-7 Conceptual Description
MPEG-7 Summaries
MPEG-7 Collections
MPEG-7 Application Framework
MPEG-7 Applications Today

IBM MPEG-7 Annotation Tool


Ricoh MPEG-7 MovieTool


Assists in annotating video sequences with MPEG7 metadata
A tool for creating video content descriptions
conforming to MPEG-7 syntax interactively
Canon MPEG-7 Speech Recognition engine

Web site allows you to create an MPEG-7 Audio
“SpokenContent” description file from an audio file
in “wav” format
IBM MPEG-7 Annotation Tool
IBM MPEG-7 Annotation Tool

The IBM MPEG-7 Annotation Tool assists in
annotating video sequences with MPEG-7 metadata




Each shot in the video sequence can be annotated with static
scene descriptions, key object descriptions, event
descriptions, and other lexicon sets
The annotated descriptions are associated with each video
shot and are stored as MPEG-7 descriptions in an XML file
Can also open MPEG-7 files in order to display the
annotations for the corresponding video sequence
Customized lexicons can be created, saved, downloaded, and
updated
Ricoh MovieTool








Creates an MPEG-7 description by
loading video data
Provides visual clues to aid the
user in creating the structure of the
video
Automatically reflects the structure
in the MPEG-7 descriptions
Visually shows the relationship
between the structure and MPEG7 descriptions
Presents candidate tags to help
choose appropriate MPEG-7 tags
Checks the validation of the
MPEG-7 descriptions in
accordance with MPEG-7 schema
Can describe all metadata defined
in MPEG-7
Is able to reflect any future
changes and extensions made to
MPEG-7 schema
Canon MPEG-7 ASR Tool
MPEG-7 Resources

http://www.chiariglione.org/mpeg/index.htm

http://www.mpegif.org/mpegif/index.php

http://www.josseybass.com/WileyCDA/WileyT
itle/productCd-0471486787.html
The need for standardized profiles
(MPEG-21)

Dictated by the emerging demands on






Universal multimedia access
Ubiquitous and pervasive computing
Heterogeneity of networks and end-user terminals
Use of content and services anywhere, at any time
To allow for uniform, exchangeable and interoperable
definition of context profiles
To allow for efficient negotiation mechanisms for
context retrieval and content adaptation anywhere in
the content production, transmission and consumption
chain
MPEG-21


A normative open framework for multimedia delivery
and consumption for use by all the players in the
delivery and consumption chain
Based on two essential concepts:

the definition of a fundamental unit of distribution and
transaction (the Digital Item)

Digital Items: the “what” of the Multimedia Framework


the concept of Users interacting with Digital Items


a video collection, a music album
Users: the “who” of the Multimedia Framework
Goal

define the technology needed to support Users to exchange,
access, consume, trade and otherwise manipulate Digital
Items in an efficient, transparent and interoperable way
MPEG-21 scope
Examples:
•“Container”
•“Item”
•“Resource”
Digital Item
Declaration
Multimedia
Content
Representation
Examples:
•Natural and Synthetic
•Scalability
Examples:
•Unique Identifiers
•Content Descriptors
Digital Item
Identification
and Description
User A
Transaction/Use/Relationship
Digital Item
Authorization/Value Exchange
User B
Content
Management
and usage
Examples:
•Encryption
•Authentication
•Watermarking
Examples:
•Resource Abstraction
•Resource Mgt. (QoS)
Intellectual
Property
management
and
Protection
Terminals &
Networks
Event Reporting
Examples:
•Storage Mgt.
•Content Personalization
MPEG-21 standard parts

MPEG-21 Digital Item Declaration


MPEG-21 Digital Item Identification


To uniquely identify Digital Items and link with related information such as
descriptive metadata
MPEG-21 Intellectual Property Management and Protection


To describe a set of abstract terms and concepts to form a useful model for
defining Digital Items
To define an interoperable framework for Intellectual Property Management
and Protection
MPEG-21Rights Expression Language

To define machine-readable language that can declare rights and permissions
using the terms of the Rights Data Dictionary

MPEG-21 Rights Data Dictionary
 MPEG-21 Digital Item Adaptation


To achieve interoperable transparent access to (distributed) advanced
multimedia content by shielding Users from network and terminal installation,
management and implementation issues
Definition of environmental profiles
MPEG-21 DIA

Conceptual architecture

a Digital Item is subject to a resource adaptation engine and a description
adaptation engine producing the adapted Digital Item
Digital Item
Adaptation Engine
Digital Item
Resource
Adaptation Engine
Adapted
Digital Item
Description
Adaptation Engine
Scope of
standardization

DIA Tools
Within scope


Descriptions and format-independent mechanisms in terms of resource adaptation,
descriptor adaptation, and/or Quality of Service management
Not adaptation engines
MPEG-21 DIA Tools

Three major categories
Usage Environment Description Tools
• User Characteristics
• Terminal Capabilities
• Network Characteristics
• Natural Environment Characteristics
Digital Item Resource Adaptation Tools
• Bitstream Syntax Description
• Terminal and Network QoS
• Bitstream Syntax Description Link
• Metadata Adaptability
Digital Item Declaration Adaptation Tools
• Session Mobility
• DIA Configuration
MPEG-21 DIA Tools & Other MPEG-21
Parts
DI-1
DI-3
MPEG-21 DID
MPEG-21 DII
MPEG-21 DID’
Digital Item
Adaptation Engine
MPEG-21 IPMP/REL
Descriptor
MPEG-21 DII’
MPEG-21 IPMP/REL’
Resource
Adaptation Engine
MPEG-21 DIA Tools
Descriptor’
MPEG-21 DIA Tools
Description
Adaptation Engine
Resource
Resource’
DI-2
MPEG-21 DID
MPEG-21 DII
MPEG-21 IPMP/REL
Descriptor
MPEG-21 DIA Tools
Usage Environment Description Tools
Digital Item Resource Adaptation Tools
Digital Item Declaration Adaptation Tools
Enabling integrated cross-media
entertainment services over
heterogeneous networks
The concepts…:
Personalization
Why personalize?

Multitude of concurrent programs
During the upcoming Olympic Games, more
than 5 events may be broadcast during
prime-time hours
 Athletics offers many events in parallel
 During weekends, many sports events and
movies are broadcast in parallel

Why personalize? (2)

Diversity in presentation options
Peripheral information, e.g. sports statistics,
critics’ reviews for movies, other results, etc.
 Visual enhancements, e.g. arrows pointing
distance to goal line, lines showing WR
time, etc.
 Different viewing options, e.g. multiple
viewing angles, virtual replays, etc.

So, personalization is…

Choosing what to view and when
Choice of a particular event or specific
portions, e.g. the final 20 mins
 Choice of a particular movie or movies
belonging to a specific genre
 Choice of live or recorded viewing

and…

Choosing how to view the program
Choice of specific enhancements
 Choice of available interaction, e.g. betting
opportunities during a football match
 Alerts when other programs of interest are
available

Is that all?

Adapting TV to individual viewers is a
topic in itself

O’Sullivan et al., 2004
Watching TV, especially sports and
movies, tends to be a social activity
 Strategies for combining individual user
models to adapt to groups


Social Choice Theory?
Group theory
In contrast to the use of PCs, television
viewing is largely a family or social
activity (Barwise and Ehrenberg, 1988)
 Unfortunately, statistics do not include
data on the average number of people
watching TV together and who watches
with whom
 Culturally dependent?

Group theory (2)

Groups can be heterogeneous
age, gender, and personality may influence
program choices
 watching TV together is a major activity
shared between parents and children
 young people like to watch TV with friends


Challenge: to adapt to a group of
viewers, in such a way that each
individual enjoys the broadcast
Strategies for Combining User
Models

Modeling of user preferences
Directly, via explicit profile definition
 Indirectly, via observation and inference


Group modeling
MUSICFX, McCarty and Anagnost, 1998
 POLYLENS, O’Conner et al., 2001
 INTRIGUE, Ardissono et al., 2002

Group modeling

MUSICFX is used in a fitness center to select
background music to suit a group of people
working out at any given time
 POLYLENS recommends movies based on
group tastes inferred from ratings and social
filtering
 INTRIGUE recommends places to visit for
tourist groups taking into account
characteristics of subgroups within that group,

e.g. children and disabled
However…

Limited actual evaluation


no indication of how effective these group
modeling strategies really are
Application domains differ from TV
viewing
people usually see one movie per evening
 music stations can play forever
 no need to select a balanced group of
items

Social choice

Also called group decision making
Deciding what is best for a group, given the
opinions of individuals
 Studied extensively in economics, politics,
sociology, and even math


Construction of a social welfare function
Pattanaik, 1971
 Taylor, 1995

Social choice (2)

Meta-Search


Database middleware


Results from multiple search engines are combined
into one list - rank aggregation (Dwork et al., 2001)
Objects with numerical values for multiple fields
have to be ordered (Fagin et al., 2003)
Collaborative Filtering

Preferences of a group of individuals have to be
aggregated to predict a third party preference
(Wallace et al., 2003)
A hands-on scenario

Choose a set of items available in an EPG


Group of viewers



e.g. video clips, TV series, movies, sports events,
etc.
Receiver knows who the viewers are
Has preference ratings for each of them
Problem: which items to show, given time for a
certain number but not all of them?
Assumptions are fine, but…

How will the receiver know who is watching?



IR card or login procedure registers viewers
Probabilistic mechanism: known probability of a
viewer watching at a particular time of day
How will user preferences be determined?



Social, and content-based filtering can be used
Inferences from group viewing  actions when
user watches alone?
An individual’s ratings may depend on the group
they are in
Assumptions are fine, but… (2)

How to deal with differences in rating
tendencies?



Some people either “really hate” or “really love” a
program
Others may never be very positive and never really
negative
How to deal with uncertainty?
 How to deal with changing groups?
Assumptions are fine, but… (3)

How to deal with users feeling strongly about
their preferences?



All users are equal?
Or some are “more equal than others”?
How to present recommendations to the
group?



Receiver decides which items to show?
Receiver gives viewers a choice?
List of recommendations?
Example strategies

Plurality Voting
Each voter votes for their most preferred
alternative
 Sequence of alternatives  repetitive
method

Example strategies (2)

Utilitarian Strategy
Utility values express expected happiness
 Additive/Average strategy (Masthoff, 2002)



Weighted form in multi-agent systems
Multiplicative
Individual viewers might always lose out,
because their opinion is a minority view
 In a small group, individual opinions will have a
large impact on the average/product

Example strategies (3)

Borda Count
Borda, 1781 (!)
 Points awarded according to position
preference list
 bottom alternative gets zero points, the next
one up one point, etc.
 Multiple alternatives with the same rating?

Example strategies (4)

Copeland Rule

Form of majority voting - orders alternatives
according to Copeland index


Number of times an alternative beats other
alternatives minus the number of times it loses
to other alternatives
Approval Voting
Voters allowed to vote for as many
alternatives as they wish
 Promotes election of moderate alternatives

Example strategies (5)

Least Misery Strategy
New list of ratings with minimum of
individual ratings
 Higher ranked items get selected sooner
 A group is as happy as its least happy
member
 Minority opinions dictate the group


If everybody wants to see something, but one
person does not like it, then it will never be seen.
Example strategies (6)

Most Pleasure Strategy
New list of ratings with maximum of
individual ratings
 Higher ranked items get selected sooner


Average Without Misery Strategy
New list of ratings with average of individual
ratings
 Skip items that score below a certain
threshold for individuals

Example strategies (7)

Fairness Strategy
Select top items from all individuals
 Equal rating  take into account others’
opinions
 It’s not so bad to watch something you hate,
as long as you get to watch the things you
really love as well

Example strategies (8)

Most Respected Person Strategy
Also called “Dictatorship”
 Use ratings of most respected person
 Similar ratings  use others’ ratings
 TV remote control is often operated by the
oldest male present

Adults may dominate children, visitors may have
more influence than inhabitants
 Birthdays or illness can influence who is ‘the
most respected’ person

The issue of satisfaction

A major difference between strategies is
the emphasis placed on individual
satisfaction
avoidance of misery
 satisfaction of group majority


Another major difference: use only
relative item positions in preference lists,
or also strengths of these preferences?
A basic satisfaction function
Input: a set of ratings for any sequence
of clips
 Output: a real number

e.g. summation of an individual’s ratings of
concerned clips
 how do these ratings compare to those of
unselected clips?  normalization


Linearity, misery, order, group solidarity?
The holy grail
Is there a strategy that keeps everyone
happy?
 Multiplicative Utilitarian seems the most
promising strategy

Length of clips may be important
 Discussion on the basis of the clips
 Specify and definitely include favorite clips

Presenting a Sequence

The receiver will also need to decide in
which order to show items
Group List Ranking
 Changing Order
 Changing Ratings

Open questions

Do people adjust ratings?
Having watched the first item influences
ratings for the other items?
 Mood? Thematic relatedness?
 Viewer’s satisfaction
 Interaction between these factors?
