Multimedia

–

Digital Audio & Video

CSC1720 – Introduction to Internet Essential Materials

Outline

     Audio / Video on the Web Basic Digital Audio Concepts – Streaming Audio – Web Audio Formats Basic Digital Video Concepts – Video compression/decompression methods.

– Video File Formats Other Multi-media Formats Summary CSC1720 – Introduction to Internet 2 All copyrights reserved by C.C. Cheung 2003.

AV on the Web?

 It is part of the Internet.

– Radio Stations – Music, sound clips – Streaming Audio / Video – Video conferencing – Digital Cameras – Animation on the Web  Shockwave, Flash CSC1720 – Introduction to Internet 3 All copyrights reserved by C.C. Cheung 2003.

Basic Terms

Term Bit Kilobit (Kb) Byte Binary value 0 or 1 1,000 bits (approx.) 8 bits Definition Kilobyte (KB) 1,000 bytes Megabyte (MB) 1,000,000 bytes Gigabyte (GB) 1,000,000,000 bytes Kbps KB/sec Kilobits per second (1,000 bits in a second) Kilobytes per second (1,000 bytes in a second) CSC1720 – Introduction to Internet 4 All copyrights reserved by C.C. Cheung 2003.

Analog vs. Digital

  Analog: A nature rather than a pulsed or discrete nature. – Note: signal Electrical or physical analogies, such as continuously varying voltages, frequencies, or phases, may be used as analog signals.

that has a continuous Digital: A signal in which discrete steps are used to represent information. CSC1720 – Introduction to Internet 5 All copyrights reserved by C.C. Cheung 2003.

Digital Signal

 Use P ulse C ode M odulation (PCM) to represent an audio signal by digital data.

CSC1720 – Introduction to Internet 6 All copyrights reserved by C.C. Cheung 2003.

ADC & DAC

Figure 4.3 Conversion from Analog to Digital requires an Analog-to-Digital Converter Figure 4.4 Conversion from Digital to Analog requires a Digital-to-Analog Converter CSC1720 – Introduction to Internet 7 All copyrights reserved by C.C. Cheung 2003.

Basic Digital Audio Concepts

    Sampling rate – Number of sample taken of a signal in a given time (usually one second) Bit depth – Describes the accuracy of the audio data Channels – E.g. Stereo has two channels Bit rate – Measured in kilobits per second (Kbps) is a function of the bit depth and sampling rate CSC1720 – Introduction to Internet 8 All copyrights reserved by C.C. Cheung 2003.

Sampling rate

      The more sample taken per second, the higher the accuracy.

Typically measured in kilohertz (KHz).

CD audio has 44,100 samples per second (44.1KHz).

8 KHz produces lower quality radio sound.

Standard sampling rates include “ 8KHz ” , “ 11.025KHz

” , … The high-end 96K is used in DVD, but is not applicable to the Web.

CSC1720 – Introduction to Internet 9 All copyrights reserved by C.C. Cheung 2003.

Sampling Rate

demo CSC1720 – Introduction to Internet 10 All copyrights reserved by C.C. Cheung 2003.

Bit depth

    Also called “ sampling resolution ” length ” .

or “ word The more bits, the better is the quality of the audio (and a larger file of course).

Common bit depths are 8-bit ( telephone like ), 16-bit ( CD quality ), and 20, 24, 32, 48-bit depths.

How many signal can a 8-bit and a 16-bit data represent?

– 0000 0000  1111 1111 – 0000 0000 0000 0000  1111 1111 1111 1111 CSC1720 – Introduction to Internet 11 All copyrights reserved by C.C. Cheung 2003.

Quantization

demo CSC1720 – Introduction to Internet 12 All copyrights reserved by C.C. Cheung 2003.

Channels

      Audio file can support one to six channels of audio formats.

Mono – one channel Stereo – two channels Some others – three, four channels.

Six channels – 5.1-channel surround sound.

More multi-channel formats announce in the coming years.

CSC1720 – Introduction to Internet 13 All copyrights reserved by C.C. Cheung 2003.

Channel Examples

CSC1720 – Introduction to Internet 14 All copyrights reserved by C.C. Cheung 2003.

Bit rate

   Audio files are measured in terms of bit rate which is measured in kilobits per second (Kbps).

It can be calculated by dividing the file size by the time (in second) to play the audio clip.

– E.g. 3Mb file play in 30 seconds – 3000k / 30 = 100kbps.

Quality at different compression rates CSC1720 – Introduction to Internet 15 All copyrights reserved by C.C. Cheung 2003.

Optimizing for the Web

  Length of the audio clip – Keep the audio clip as short as possible.

– E.g. just keep the most sweetest part of your greeting.

Number of channels – A mono audio file is halved the space of a stereo file.

– Depends on your actual needs.

CSC1720 – Introduction to Internet 16 All copyrights reserved by C.C. Cheung 2003.

Optimizing for the Web

   Bit depth – Audio file on the Web are usually 8-bits.

– Half the size of a 16-bit file.

Sampling rate – Half the sampling rate will also halve the space needed.

– Voice only audio file can be reduced to 8KHz.

– 22 KHz music clips are acceptable.

Putting all things together: Mono, 8-bit, 22KHz, MP3 compression.

CSC1720 – Introduction to Internet 17 All copyrights reserved by C.C. Cheung 2003.

Calculate audio size

     8-bit mono: seconds x KHz 16-bit mono: (seconds x KHz) x 2 8-bit stereo: (seconds x KHz) x 2 16-bit stereo: ([seconds x KHz] x 2)x2 E.g. the file size of 30 seconds of 16 bit, 11KHz mono audio: – (30 x 11) x 2 = 660K.

CSC1720 – Introduction to Internet 18 All copyrights reserved by C.C. Cheung 2003.

Streaming Audio

    What is it?

– Play almost immediately after the request, continues playing the transferring data.

Advantages: – Address the problem of long download time.

– Control distribution and protect copyright, because the user cannot get a copy of the file.

Disadvantages: – Sound quality may be be affected by low speed or unstable Internet connection.

Reference CSC1720 – Introduction to Internet 19 All copyrights reserved by C.C. Cheung 2003.

What is Streaming?

CSC1720 – Introduction to Internet 20 All copyrights reserved by C.C. Cheung 2003.

How does it work?

Streaming audio

Packet are sent to a buffer on the receiving Computer, the RealPlayer will play the sound File when buffer full Web browser Web browser request a RealAudio from the web server CSC1720 – Introduction to Internet Web server 21 RealAudio Server All copyrights reserved by C.C. Cheung 2003.

Quality Comparison

http://www.cit.cornell.edu/atc/itsupport/streamcompare.shtml

 CSC1720 – Introduction to Internet 22 All copyrights reserved by C.C. Cheung 2003.

Web Audio Formats

 WAV/AIFF (.wav/.aif/.aiff) – The Wav eform Audio File format (.wav) was developed by Microsoft, supports arbitrary sampling rates and bit depths.

– The A udio platform.

I nterchange F ile format (.aif, .aiff) was developed for Macintosh – They are less likely used on the Web, because people use “ mp3 ” or “ streaming ” .

CSC1720 – Introduction to Internet 23 All copyrights reserved by C.C. Cheung 2003.

http://www.nuance.com/prodserv/demo_vocalizer.html

WAV/AIFF

  Try the Bell-lab synthesis link.

http://www.bell-labs.com/project/tts/voices.html

CSC1720 – Introduction to Internet Select the Audio format 24 All copyrights reserved by C.C. Cheung 2003.

MP3 (.mp3)

     Able to maintain excellent sound quality at very small file sizes.

The compression reduces an audio file to one-tenth of its original size.

– E.g. 40MB file  3.5MB

MP3 is actually MPEG-1 Layer-III Good for distribution of HQ audio.

Demo: www.mp3.com

CSC1720 – Introduction to Internet 25 All copyrights reserved by C.C. Cheung 2003.

What is MP3 digital encoding?

 MP3 is actually the file extension for MPEG, audio layer 3. Layer 3 is one of three coding schemes (layer 1, layer 2, and layer 3) for the compression of audio signals. Layer 3 uses perceptual audio coding and psychoacoustic compression to remove all superfluous information. (More specifically, it removes the redundant and irrelevant parts of a sound signal--the stuff the human ear doesn't hear anyway).

It also adds a MDCT (Modified Discrete Cosine Transform) that implements a filter bank, increasing the frequency resolution 18 times higher than that of layer 2.

CSC1720 – Introduction to Internet 26 All copyrights reserved by C.C. Cheung 2003.

MP3 Players

CSC1720 – Introduction to Internet 27 All copyrights reserved by C.C. Cheung 2003.

Apple QuickTime Audio (.mov)

    QuickTime is a well-known video format, but it can create audio-only movies.

QuickTime is a container format, which is able to store still images, movie formats, … – Excellent compression, true streaming Netscape and IE have Plug-in now.

Quicktime : demo CSC1720 – Introduction to Internet 28 All copyrights reserved by C.C. Cheung 2003.

MIDI (.mid/.midi)

     MIDI stands for “ M usical I nstrument D igital I nterface ” which is developed for electronic musical instruments.

MIDI files are very compact and very good for low-bandwidth delivery.

Instruments are “ piano, drums, bass, orchestral strings, …” It is very attractive for adding MIDI file to your website with very little download time.

Demo: www.findmidis.com

CSC1720 – Introduction to Internet 29 All copyrights reserved by C.C. Cheung 2003.

MIDI

http://www.findmidis.com/ CSC1720 – Introduction to Internet 30 All copyrights reserved by C.C. Cheung 2003.

RealMedia/RealAudio (.rm/.ra)

     RealAudio is a server-based streaming audio format.

The RealServer responses to the requests and delivers the streaming packets, including the bandwidth negotiation.

A robust RealServer can support thousands of simultaneous listeners.

Good for continuous-playing audio and live broadcasts to a large group of people.

Example: RTHK Radio CSC1720 – Introduction to Internet 31 All copyrights reserved by C.C. Cheung 2003.

RealMedia/RealAudio

CSC1720 – Introduction to Internet 32 All copyrights reserved by C.C. Cheung 2003.

Windows Media (.wma/.asf)

     Windows Media is a streaming system.

It wraps all media elements into a Active Streaming File (.asf).

Audio may be saved as non-streaming Windows Media Audio format (.wma).

Good for continuous-play audio.

The encoder and player is Free, shipped with Windows OS.

CSC1720 – Introduction to Internet 33 All copyrights reserved by C.C. Cheung 2003.

Windows Media

CSC1720 – Introduction to Internet 34 All copyrights reserved by C.C. Cheung 2003.

Choosing an audio format

Audio Needs Short voice greetings News broadcasts Background music Music samples for some audience Radio-style or Live broadcasting Musical E-greeting card Suggested formats WAV, AIFF, MP3 Streaming solutions (RealAudio, Windows Media) MIDI, WAV MP3 or QuickTime RealMedia System MIDI, WAV CSC1720 – Introduction to Internet 35 All copyrights reserved by C.C. Cheung 2003.

Add Audio to your Web Page

   A simple link –

” >Play the song –

” >

” > Background Sound –

” LOOP=3> Link to RealMedia –

” >Link to the song – pnm://domainname.com/song.rm

CSC1720 – Introduction to Internet 36 All copyrights reserved by C.C. Cheung 2003.

Break Time

–

15 minutes

CSC1720 – Introduction to Internet 37 All copyrights reserved by C.C. Cheung 2003.

Basic Digital Video Concepts

      Movie length Frame size Frame rate Quality Color bit depth Data rate (bit rate) CSC1720 – Introduction to Internet 38 All copyrights reserved by C.C. Cheung 2003.

Movie length

   File size is proportional to the movie length.

Videos longer than 1 or 2 minutes cause long download times.

If it is a long video, consider to use streaming video.

CSC1720 – Introduction to Internet 39 All copyrights reserved by C.C. Cheung 2003.

Frame size

    “ Full-screen ” video is 640x480 pixels.

The most common frame size for web video is 160x120 pixels.

Not recommend to use a frame size larger than 320x240.

The size depends on the CPU power and the Internet connection bandwidth.

CSC1720 – Introduction to Internet 40 All copyrights reserved by C.C. Cheung 2003.

Image and Video?

CSC1720 – Introduction to Internet 41 All copyrights reserved by C.C. Cheung 2003.

Frame rate

    Frame rate is measured in number of frames per second (fps).

Standard TV-quality video uses 30 fps.

For the web, 15 or even 10 fps is more appropriate and produces fair smooth quality for the user.

Commercial Internet Broadcasts are using 0.5, 0.25 frames per second.

CSC1720 – Introduction to Internet 42 All copyrights reserved by C.C. Cheung 2003.

Quality

    Many video-editors allow you to set the overall quality of the video.

The degree of compression controls the target quality.

The low or medium setting results a fairly high compression which is appropriate for web delivery.

Frame rate and quality are usually tradeoff in different applications.

CSC1720 – Introduction to Internet 43 All copyrights reserved by C.C. Cheung 2003.

Color bit depth

   The number of pixel colors in each frame affects the size of the video.

The file size of the video will be greatly reduced by changing the number of colors from 24-bit to 8-bit.

It sacrifices the image quality of the video.

CSC1720 – Introduction to Internet 44 All copyrights reserved by C.C. Cheung 2003.

Data rate (bit rate)

    This is the rate that the data must be transferred in order to ensure the video can play smoothly without interruption.

It is measured in kilobytes per second (K/sec or Kbps).

It can be calculated by dividing the size of the file (in K) by the movie length (in seconds).

– E.g. the video file size is 1.9MB  1900K – Play 40 seconds long, Data rate = 47.5K/sec Consider the Internet bandwidth!

CSC1720 – Introduction to Internet 45 All copyrights reserved by C.C. Cheung 2003.

Calculate space requirements of Video

  NTSC video (640 x 480 and 29.97 fps) – Frame size = ([Pixel width x pixel height x bit depth]/8)/1024 – E.g. 200KB/Frame : 6.0 MB/sec – 200KB x 30 fps = 6000KB/s, 6 MB/sec PAL video (768 x 576 and 25 fps) – E.g. 200KB/Frame : 5.0 MB/sec – 200KB x 25 fps = 5000KB/s, 5 MB/sec CSC1720 – Introduction to Internet 46 All copyrights reserved by C.C. Cheung 2003.

Video CODECs

   CODEC is “ Co mpression/ Dec ompression ” algorithms.

The sound and frame images of a digital video must be compressed.

– The vast amount of data Compressed in a number of ways – Lossless and Lossy compression – Spatial and Temporal compression CSC1720 – Introduction to Internet 47 All copyrights reserved by C.C. Cheung 2003.

Video Clip Demo

 reference CSC1720 – Introduction to Internet 48 All copyrights reserved by C.C. Cheung 2003.

Lossless and Lossy compression

  Lossless compression means no information is lost and the final file is the same as the original.

Most compression methods are human eye or ear.

lossy to achieve higher compression rates.

– Use complicated algorithm to toss out some data that is not discernible to the .

– Sacrifices some data from the file in order CSC1720 – Introduction to Internet 49 All copyrights reserved by C.C. Cheung 2003.

Spatial and Temporal compression

  Spatial ( Intraframe ) compression takes place on each individual frame of the video.

Temporal ( Interframe ) compression applies on a series of frames, it takes advantages of areas of the image remain unchanged from frame to frame.

– Relies on the key frames and delta frames .

– A key frame is placed once every second.

– E.g. 15 fps, a key frame once 15 frames.

CSC1720 – Introduction to Internet 50 All copyrights reserved by C.C. Cheung 2003.

Delta frame

CSC1720 – Introduction to Internet 51 All copyrights reserved by C.C. Cheung 2003.

Video File Formats

   QuickTime Movie (.mov) – Introduced by Apple Computer in 1991.

– First developed for Macintosh, now also supports the PCs.

– Also supports streaming.

How to create?

– Most video editor, QuickTime Pro.

How to play?

– QuickTime plug-in or QuickTime player.

CSC1720 – Introduction to Internet 52 All copyrights reserved by C.C. Cheung 2003.

http://www.apple.com/quicktime/gallery/cubicvr/times_square.html

Quicktime

CSC1720 – Introduction to Internet 53 All copyrights reserved by C.C. Cheung 2003.

RealMedia (.rm)

       Industry standard streaming format.

RealPlayer for playback.

RealServer for serving streams.

RealProducer for creating .rm files.

Good for – Long-playing video or broadcast to many people.

How to create?

– RealSystem Producer How to play?

– RealPlayer (Free), RealPlayer Plus (Commerical) CSC1720 – Introduction to Internet 54 All copyrights reserved by C.C. Cheung 2003.

Windows Media (.wmv/ .asf)

     Created by Microsoft, closely integrated with Windows OS.

Support Windows Media Video (.wmv) and Advanced Streaming Format (.asf) and other formats (.avi, .mpeg, … ) Also support streaming.

How to create?

– Windows Media Encoder, Windows Media Author How to play?

– Media Player in Windows OS CSC1720 – Introduction to Internet 55 All copyrights reserved by C.C. Cheung 2003.

AVI (.avi)

      Stands for Audio/Video Interleaved.

Introduced by Microsoft in 1992.

In a AVI file, the audio and video information are interleaved every frame.

Good for – Short web clips, high-quality video How to create?

– Most video editing tools.

How to play?

– Windows Media, QuickTime, etc.

CSC1720 – Introduction to Internet 56 All copyrights reserved by C.C. Cheung 2003.

Video Clip Demo

 reference CSC1720 – Introduction to Internet 57 All copyrights reserved by C.C. Cheung 2003.

MPEG (.mpg/ .mpeg)

       Created by Moving Picture Experts Group (MPEG).

Supports 1) Video, 2) Audio, 3) Streaming.

Extremely high compression rates with small quality degradation (lossy).

MPEG-1 : VHS quality MPEG-2 MPEG-4 : HQ standard for TV broadcast : Very HQ for AV compression MPEG can be compressed by using three schemes: Layer-I, Layer-II, Layer-III.

CSC1720 – Introduction to Internet 58 All copyrights reserved by C.C. Cheung 2003.

Official MPEG page

 reference CSC1720 – Introduction to Internet 59 All copyrights reserved by C.C. Cheung 2003.

MPEG Comparison

Standard since Default Video resolution (NTSC) Max. Audio Frequency range Max. audio Channel Regular data rate Frames per sec (NTSC) Video Quality Hardware requirement for encoding/decoding MPEG-1 1992 352 x 288 48 KHz 2 1380 kbit/s 30 Satisfactory Low MPEG-2 1995 640 x 480 96 KHz 8 6500 kbit/s 30 Very good Medium MPEG-4 1999 640 x 480 96 KHz 8 880 kbit/s 30 Very Good High CSC1720 – Introduction to Internet 60 All copyrights reserved by C.C. Cheung 2003.

Flash & Shockwave

  Advantages: – File sizes are small – Image quality is high – It uses streaming technology – It uses high-quality streaming audio – It is scriptable Disadvantages: – A plug-in player is required – Expensive authoring software – Problems on printing their content CSC1720 – Introduction to Internet 61 All copyrights reserved by C.C. Cheung 2003.

Summary

    Discuss the basic digital audio/video terminology.

Introduction to different formats: WAV, MP3, QuickTime, RealMedia, Windows Media, AVI, MPEG.

To deliver long-playing audio/video or live broadcasts, you should choose one of the streaming media.

Flash and shockwave are popular and appropriate format for the Web.

CSC1720 – Introduction to Internet 62 All copyrights reserved by C.C. Cheung 2003.

References

       Design Web Audio (J. Beggs, D. Thede), Oreilly.

E-Video (H. Peter Alesso), Addison-Wesley.

Audio Video Knowledge Center Bible in MP3 format MPEG Musiq Luke Video   The End.

Thank you for your patience!

CSC1720 – Introduction to Internet 63 All copyrights reserved by C.C. Cheung 2003.