Transcript PPT

Study-Element Based Adaptation of
Lecture Videos to Mobile Devices
Ganesh Narayana Murthy (M.Tech IIT Bombay)
Sridhar Iyer (Associate Professor, IIT Bombay)
Ganesh Narayana Murthy
NCC 2010, Chennai, 30/01/2010
1
Problem Definition
• Adapt CDEEP videos to be viewable on mobile devices:
– Viewable at low network bandwidths (like GPRS)
– Viewable at low cost
• Video bit-rate
–
–
–
–
Size of video stream over time
Total size = bit-rate * total time
CDEEP video bit-rate: 1150kbps
GPRS bit-rate: 40kbps
• The problem:
– Video playing incurs delays if available network bandwidth is less than
video-bit rate
Ganesh Narayana Murthy
NCC 2010, Chennai, 30/01/2010
2
Video Transcoding
• Converting from one video format to another
– Changing video bit rate
– Changing other parameters like frame rate, screen
resolution
Format Name
Typical Bit Rate
Application
MPEG-1
1.5Mbps or less
CD-ROM
MPEG-2
5-8Mbps
DVD, HDTV
H.263
Typically low bit rates
Low bit-rate video
conferencing
MPEG-4 / H.264
40Kbps to 10Mbps and
above
Internet Streaming,
Video Telephony
Flash Video (FLV)
Typically low bit rates
Embedded video in
websites
Ganesh Narayana Murthy
NCC 2010, Chennai, 30/01/2010
3
Video Quality at low-bit rates
(a) MPEG-1
(b) MPEG-2
Images from transcoded videos
(Target bit rate : 40kbps, No audio)
Ganesh Narayana Murthy
NCC 2010, Chennai, 30/01/2010
4
Video Quality at low bit-rates (contd.)
(c) H.264 (mp4)
(d) H.263 (3gpp)
Images from transcoded videos
(Target bit rate : 40kbps, No audio)
Ganesh Narayana Murthy
NCC 2010, Chennai, 30/01/2010
5
(e) Flash Video (flv)
Images from transcoded videos
(Target bit rate : 40kbps, No audio)
Ganesh Narayana Murthy
NCC 2010, Chennai, 30/01/2010
6
Comparison of Video Codecs
Format
Name
Original
Video
Size
Converted Video
Size
Video Quality at
low-bit rates
Remarks
MPEG-1
432MB
26MB
Poor
Cannot be used at low
bit-rates
MPEG-2
432MB
29.12MB
Poor
Cannot be used at low
bit rates
H.263
432MB
38.3MB
Poor
Cannot be used at low
bit rates
H.264
432MB
16.9MB
Good
Processing power /
Decoding complexity is
high.[1]
Flash
432MB
20.5MB
Good
Can be used, but cost is
still high.
(Note: Video bit rate = 1150kbps, No audio,
Target bit rate = 40kbps, No audio)
Ganesh Narayana Murthy
NCC 2010, Chennai, 30/01/2010
Video Sizes are still high for
viewing over GPRS
7
Study-Element Based Adaptation
Ganesh Narayana Murthy
NCC 2010, Chennai, 30/01/2010
8
Motivation
• CDEEP video usually consists of
– Presentation slides
– Instructor explaining on white paper
– Video of instructor talking
• Presentation slide is usually not changing
– Video of slide is not required. One image is sufficient
• Idea
– Extract one image every ‘n’ seconds and send to client.
– This would reduce amount of data sent for showing one
slide.
Ganesh Narayana Murthy
NCC 2010, Chennai, 30/01/2010
9
Method-1
• Send one image every ‘n’ seconds
– Server sends one image every ‘n’ seconds to client
– Audio is simultaneously streamed
• Network bandwidth and Size
– Network Overhead (NO) = Image Size / n
– Size Overhead (SO) = Total size of images
• What is the user experience?
Ganesh Narayana Murthy
NCC 2010, Chennai, 30/01/2010
10
User Experience Basis
• Presentation Study Element
– Portion of video showing one slide
• White Paper Study Element
– Portion of video showing instructor writing on
white paper
• Instructor Study Element
– Portion of video showing instructor talking
Presentation Slide
0
3 5
10
…………..
15
White Paper
25
30
………..
35
Video Time
(secs)
Delay in start of slide
Ganesh Narayana Murthy
NCC 2010, Chennai, 30/01/2010
11
User Experience
• Presentation Element
– Delay Experienced (D2) =
• Delay in start of slide as compared to audio
• White Paper Element
– Delay Experienced (D1) =
• Delay between any two consecutive images = Sending Rate
• Instructor Element
– Only audio important. No image need be sent.
• User Experience is assumed to be one
• User Experience (Ui) = 1 sec / Di
Ganesh Narayana Murthy
NCC 2010, Chennai, 30/01/2010
12
Method-2
• Trade-off for user experience
• Cost incurred in terms of number of images
sent
• Same sending interval for all elements,
cannot balance user experience and cost.
• Choose different sending interval for
each study element
• Probably:
– Higher user experience for white paper
element
– Lower user experience for presentation
element
Sending
Interval
User
Experience
Cost
Trade-Off Relation
Ganesh Narayana Murthy
NCC 2010, Chennai, 30/01/2010
13
System Overview
Ganesh Narayana Murthy
NCC 2010, Chennai, 30/01/2010
14
Building the index
• Corpus of 10 videos
– Representative of various departments
• Consider different sending intervals ‘r’
– For each ‘r’ find NO,SO and U for every study element in a
video.
– Repeat for all videos and take average.
• This relation can be used backwards:
– For calculating sending interval, given network bandwidth
and user experience.
Ganesh Narayana Murthy
NCC 2010, Chennai, 30/01/2010
15
Graphs of User Experience
Ganesh Narayana Murthy
NCC 2010, Chennai, 30/01/2010
16
Graphs of overheads
Ganesh Narayana Murthy
NCC 2010, Chennai, 30/01/2010
17
Results
Achieved Size Reduction
Original
Video
Size(MB)
Images
Size (MB)
Reduction
(%)
U1
U2
Supported
Network Bandwidth
432
2.85
97%
0.2
0.38
20kbps and above
Fig: Video stream size reduction
(note: Original video bit-rate = 1150kbps, No audio)
Ganesh Narayana Murthy
NCC 2010, Chennai, 30/01/2010
18
Results (contd.)
Balance User Experience and Cost
Sending Interval
U1
U2
NO1
(kbps
)
S01
(MB)
NO2
(kbps
)
SO2
(MB)
Total
Size
(SO)
White
Paper
Element
Presentation
Element
5
5
0.2
0.337
15.12
1.25
23.43
1.6
5.46
5
15
0.2
0.115
15.12
1.25
7.81
0.53
1.78
15
15
0.067
0.115
5.254
0.781
7.63
1.03
1.81
Reduction in size user
experience for white paper
element remaining same
Ganesh Narayana Murthy
Required Network Bandwidth
=max(NO1,NO2) is reduced
NCC 2010, Chennai, 30/01/2010
19
Conclusion
• Large size reduction can be achieved by using the
concept of slideshows
• Identifiying study-elements within the video helps
define user-experience of the slideshow.
• CDEEP Lecture videos can be adapted to low network
bandwidths and in a cost-controlled manner.
Ganesh Narayana Murthy
NCC 2010, Chennai, 30/01/2010
20
Future Work
Automated tagging
– Identifying study element boundaries
– Shot detection techniques
User Experience Correlation
– Identifying relation between obtained user
experience and actual user values
Ganesh Narayana Murthy
NCC 2010, Chennai, 30/01/2010
21
References
1. H.264 white paper.
http://ati.amd.com/products/pdf/h264_whitepaper.pdf.
2. Real-time Content-Based Adaptive Streaming of Sports Videos.
Shih-Fu Chang, Di Zhong, and Raj Kumar.
In CBAIVL '01: Proceedings of the IEEE Workshop on Content-based Access of
Image and Video Libraries (CBAIVL'01), page 139, Washington, DC, USA, 2001. IEEE
Computer Society.
3. Content-aware video adaptation under low-bitrate constraint.
Ming-Ho Hsiao, Yi-Wen Chen, Hua-Tsung Chen, Kuan-Hung Chou, and Suh-Yin Lee.
EURASIP J. Adv. Signal Process,2007(2):27-27, 2007.
4. A Characteristics-Based Bandwidth Reduction Technique for Pre-recorded Videos.
Wallapak Tavanapong and Srikanth Krishnamohan.
In IEEE International Conference on Multimedia and Expo (III), pages 1751-1754,
2000.
Ganesh Narayana Murthy
NCC 2010, Chennai, 30/01/2010
22
Questions?
Ganesh Narayana Murthy
NCC 2010, Chennai, 30/01/2010
23
Content-Aware Adaptation
Method Name
Adaption Mechanism Video Quality
Remarks
Hsiao et.al.[2]
Identify visual
attention regions in a
frame. Encode them
at high quality.
Poor
Quality of important
objects still depends
on network bandwidth
Chang.et.al [3]
Identify events in
sports videos at high
quality. Other regions
as slideshows.
Good
Slideshow of images
reduces network
bandwidth and size
Tavanapong. et. Identify non-changing
Al. [4]
portions of lecture
video and extract one
image from them
Good
Exploits redundancy in
lecture videos
Ganesh Narayana Murthy
NCC 2010, Chennai, 30/01/2010
24