4/26/2020

[Class 6: Multimedia Management]

IN350: Document Handling and Information Management

Judith Molka-Danielsen, Oct. 02, 2000

4/26/2020

Multimedia Services: Processes of Interest

1. Capture

 Synchronize audio, video, and other media streams  Compression in real-time

2. Presentation

 Decompression in real-time  Low end-to-end delay (< 150ms )

3. Storage

 Quick access for forward and backward retrieval  Random access (< 1/2 s)

4/26/2020

Multimedia Services need Compression

•

Typical Example of a Video Sequence

– 25 images/sec – 3 byte/pixel – Image resolution 640*480 pixel – Data rate= 640*480*3*25=23040000byte/s = ~23Mbytes/s –

Therefore, Compression is required

4/26/2020

General Requirements for Real Time Multimedia Services

• • • •

Low Delay Low Complexity of compression algorithm Efficient Implementation of compression algorithm High quality output after decompression

4/26/2020

Compression Coding Categories

Entropy Coding - Run Length Coding, Huffman Coding, Adaptive Algorithms, Arithmetic Coding (some examples discussed in previous lectures.) Source Coding Prediction, DCT (using FFT-fourier frequency transform), subband coding, quantization Hybrid Coding - JPEG, MPEG, H.261, DVI RTV, DVI PLV

4/26/2020

Source Coding

This type of coding tries to take into account the specific data characteristics (in a photo for example) and the human sensitiveness (vision) to that type of data. Luminance:

is the overall response of the total light energy dependant on the wavelength of the light. The human eye is more sensative to certain wavelengths than to others. (more to:yellow, green; less to: red, violet).

Lightness:

is the “whiteness” of a specific object

4/26/2020

Discrete Cosine Transformation (DCT)

Uses source coding: going from the spacial domain to the frequency domain

 typical pictures have minimal changes in colour or luminance between two adjacent pixels  the frequency representation describes the amount of variation. There are small coefficients for high frequencies. The coefficients of the low frequencies contain most of the information.

4/26/2020

Discrete Cosine Transformation (DCT)

Discrete Cosine Transformation (DCT) Assumption:

changes of colour and luminance between adjacent pixels is low

DCT can be used for photos DCT can not be used for vector graphics

or two coloured pictures (black text, white background)

DCT is a lossless technique:

But information can be lost at the

quantization

step. The divisor selected for the matrix can reduce precision.

4/26/2020

Hybrid Coding for images, video and audio

Basic Encoding Steps: Image Preparation - resolution, frame rate.

Video:lossy, Audio:lossless

Image Processing - DCT.

Lossy or lossless.

Quantization - analog to digital conversion of data flow.

Entropy Encoding - Run Length, Huffman.

Lossless.

4/26/2020

JPEG (Joint Photographics Experts Group)

Coding/Decoding of continuous-tone still images JPEG allows Different resolution of individual components Interleaved / non-interleaved coding of components Lossless / lossy modes

 loss of information by DCT is minimal  loss of information by quantisation may be significant, but amount of lost data is adjustable

Quantisation:

 the JPEG standard defines two default quantization tables, one for luminance and one for chrominance

4/26/2020

MPEG (Moving Pictures Experts Group)

Algorithms for Compression of Audio and Video: A movie is presented as a sequence of frames, the frames typically have temporal correlations

If parts of frame #1 do not change then they do not need to be encoded in Frame #2 and Frame #3 again.  Moving vectors describe the re-use of a 16x16 pixel area in other frames.  New areas are described as differences 

MP3

is just the 3rd Layer of the MPEG standard used for audio. Layer 1 and 2 are for video.

4/26/2020

MPEG frame types

MPEG frame types with different temporal correlations:



(I-Frames)

Intracoded full frames are transmitted periodically. 

(P-Frames)

predictive frames and

(B-Frames)

bidirectional frames are used for temporal correlations. They can look to the “past” using (P-Frames) and to the “future” using (B-Frames).

4/26/2020

Other Video Compression standards

H.261 (Px64):

•Video compression for video conferences. • Compression in real time for use with ISDN.

• Compressed data stream= px64kbps where p=1 to 30. There are 2 resolutions: •

CIF

(Common Intermediate Format) •

QCIF

(Quarter Intermediate Format)

Digital Video Interactive (DVI):

Intel/IBM technology for digital video. •

DVI PLV

(Production Level Video, for VCR), •

DVI RTV

(Real Time Video, lower quality).

4/26/2020

Requirements for real time systems

Required characteristics for Real time systems:

•

The processing time for all system components must be predictable

•

Reservation of all types of resources must be possible

•

Short response time for time critical processes

•

The system must be stable at high load

4/26/2020

Problems with continuous media

Computers usually handle data as discrete, so continuous media, like video and audio must be viewed as periodical and discrete by the computer.

There are few file system services for continuous media applications.

Kernel, and hardware availability varies over time. It is difficult to shedule processes. Many interupts are to be handled by the system.

Reservation of resources, like CPU time, is not possible on a standard desktop PC. Delivery delays in real time applications make scheduling periodical processing of the media difficult.

4/26/2020

Summary

• • •

Raw digitized data is much too large. Compression can be used to reduce the size of data.

Entropy coding:

loss less compression techniques take statistical occurrence of symbols within a stream

Source coding:

often lossy compression, that take into account specific data characteristics and the human perceptive sensitivities to data

Hybrid coding:

combination of the two types.

4/26/2020

Summary continued

•

The hardware architecture and the system software in desktop computers are not adapted for handling continuous media.

Hardware support

for handling continuous media would increase quality, such as replacement of the single asynchronous bus.

• • Standard operating

system software

has not changed to better schedule multimedia or reserve resources. But application level languages such as

SMIL

can help

. SMIL

(Synchronized Multimedia Integration Language) is a XML-based language to mix media presentations over low speed connections (>=28.8 kbps). See www.w3c.org