Transcript Technical University of Crete Department of Electronic and
Technical University of Crete
Department of Electronic and Computer Engineering E.G.M. Petrakis Multimedia Data Management Euripides G.M. Petrakis http://www.intelligence.tuc.gr/~petrakis http://courses.ced.tuc.gr
Chania 2010 Introduction 1
Definition
Multimedia : composite entities combining text, audio, images, video (bit-stream objects), graphics Multimedia Information Systems : database systems that support all multimedia data types and handle very large volumes of information E.G.M. Petrakis Introduction 2
TEXT
The most common type of information The least space intensive data type The form in which text is stored varies (plain ascii, word files, spreadsheets, annotations, database fields etc.) Text fonts are becoming complex allowing special effects (color, shade, fill etc.) E.G.M. Petrakis Introduction 3
AUDIO
Mbytes), presented as analog, digital or MIDI Analog waveform : electrical signal, amplitude specifies the loudness of the sound speakers Digital waveform audio : digital, less sensitivity to noise and distortion involves larger processing and storage capacities Digital Audio Tape (DAT), Compact discs (CD) WAV (Microsoft’s wave file format) E.G.M. Petrakis Introduction 4
MIDI (Musical Instrument Digital Interface) Furht et.al.96
Commands that describe how the music should be played are stored (instead of sound) A music synthesizer generates sound Provides high data compression, Widely accepted E.G.M. Petrakis Introduction 5
IMAGES
Digital images: sequences of pixels Pixels: numbers interpreted to display intensity, color, contrast etc Binary (0-1 values), gray-scale (8 bits/pixel), colour (3x8 values for RGB) Space overhead depends on image type, resolution, compression scheme Image formats : tiff, bmb, jpeg etc. E.G.M. Petrakis Introduction 6
Image Concepts and Structures
Binary images : 1 bit/pixel black & white photos, facsimile images Computer Graphics : 4 bits/pixel Grayscale images : 8 bits/sample Color images : 16, 24 bits/pixel E.G.M. Petrakis Introduction 7
RGB Representation
A color is produced by adding red, green and blue The straight line R=G=B specifies gray values ranging from black to white E.G.M. Petrakis Introduction 8
YUV Representation
YUV describes the luminance and chrominance components of an image 1 luminance : gray scale version of an image Y = 0.299R + 0.587G + 0.114B
2 chrominance components: U = 0.564(B - Y) V = 0.713(R - Y) E.G.M. Petrakis Introduction 9
Conversions
Conversion between RGB and YUV requires multiplication operations an approximation: Y = R/4 + G/2 +B/2, U=(B-Y)/2, V=(R-Y)/2 R = Y + 2V, G = Y – (U + V), B = Y + 2U YCbCr is another color format for compression Cb = U/2 + 0.5, Cr = V/1.6 + 0.5
E.G.M. Petrakis Introduction 10
VIDEO
The most space intensive data type A sequence of frames Realistic video playback, transmission, compression/decompression require transfer rates about 30frames/sec Microsoft’s AVI and Apple’s Quicktime file formats integrate video and audio in the same presentation E.G.M. Petrakis Introduction 11
Sequence of Frames
Khoshafian Baker 96 E.G.M. Petrakis Introduction 12
Audio-video Modes of Operation
Can be either stored
or
used / transmitted live in real-time Can be used interactively or non interactively E.G.M. Petrakis Introduction Furht et.al.96
13
GRAPHICS
Objects described through their basic elements (e.g., 2D, 3D shapes) these elements can have different sizes, position, orientation, surface, fill etc. compact representations generated and can be manipulated by design tools (e.g., CAD tools) Their descriptions are stored in files E.G.M. Petrakis Introduction 14
2D and 3D graphics objects
Khoshafian Baker 96 E.G.M. Petrakis Introduction 15
MULTIMEDIA objects
Text, audio, images, video, graphics are elements of complex multimedia objects Various tools or applications integrate, process and combine multimedia Applications : multimedia authoring applications that output documents and databases and end-user applications (e.g., video on demand) Tools : for viewing, updating, querying (presentation viewers, browsers etc.) E.G.M. Petrakis Introduction 16
Multimedia Databases (MDB)
Means stored information or database management systems (dbms) Multimedia dbms (mdbms) integrate conventional database capabilities together with different technologies such as Hierarchical storage management (HSM) and Information retrieval (IR) E.G.M. Petrakis Introduction 17
Multimedia Technologies
Technologies integrated within a mdbms HSM IR support support (exact and approximate) Spatial data types and queries Interactive querying, relevance feedback, refining Automatic feature extraction Automatic content retrieval and indexing Query optimization E.G.M. Petrakis Introduction 18
Database capabilities
Persistence : object persist through invocations Transactions : content is inserted, deleted, updated Concurrency control: transactions run concurrently Recovery : failed transactions are not propagated to the db Querying : content can be retrieved Versioning : access previous states of objects Integrity : transactions guarantee consistency of content Security : constraints for accessing/updating objects Performance : optimal data structures and programs E.G.M. Petrakis Introduction 19
Hierarchical Storage Management (HSM) Support storage of multimedia objects On-line : on RAM, magnetic disk Near-on-line : on optical storage Off-line : on tapes, shelves Each level has different Performance : decreases from top to bottom Capacity : increases from top to bottom Cost : decreases from top to bottom E.G.M. Petrakis Introduction 20
Information Retrieval (IR) Capabilities Retrieval is the most common operation Deletions and updates are less common Exact match : search based on exact information Inexact: search based on inexact information e.g., probabilistic The results are ranked by order of relevance Query refinement Iterate over query results Adjust weights of query terms or features And finally resubmit queries E.G.M. Petrakis Introduction 21
MDBMS architecture
Khoshafian Baker 96 E.G.M. Petrakis Introduction 22
MDBMS Implementation
Relies on 3 rd component separate party vendors for each Relational dbms Text retrieval for typical records optical storage module Audio/image/video retrieval Feature extraction system Multimedia object interface for text/audio/graphics/images/video system (e.g. Lucene ) system system E.G.M. Petrakis Introduction 23
Object-Oriented Multimedia Databases Better design, better suited for multimedia applications Uniform handling of data and operations Data types are objects with internal structures and operations that capture the behavior of objects (e.g., audio playback, video browsing) OO dbms does not satisfy all MM requirements Provides primitives for object handling Multimedia components need to be implemented or integrated E.G.M. Petrakis Introduction 24
Object orientation in MMDBMS’s
Khoshafian Baker 96 E.G.M. Petrakis Introduction 25
Client-Server Architectures
Multimedia databases operate in client server architectures A number of interconnected server nodes provide I/O capabilities for multimedia objects Basic DBMS module IR module HSM module More services may include scanner, fax services etc.
E.G.M. Petrakis Introduction 26
Client-Server in mdbms
Khoshafian Baker 96 E.G.M. Petrakis Introduction 27
Various modules and servers in an MMDBMs architecture
Khoshafian Baker 96 E.G.M. Petrakis Introduction 28
Multimedia Applications
Multimedia Systems suggest a variety of applications Multimedia conferencing Multimedia on demand news on demand) (interactive TV, See next page for more … E.G.M. Petrakis Introduction 29
Multimedia Applications
Furht et.al.96
E.G.M. Petrakis Introduction 30
Multimedia Conferencing
Multimedia conferencing enable a number of participants to exchange multimedia information Skype, PoWWoWNow, Webex Each participant has a workstation linked to other workstations over high-speed networks Each participant can send or receive mm data and perform certain collaborative activities E.G.M. Petrakis Introduction 31
A video conference system
Furht et.al.96
The biggest performance challenge occurs when the participants transmit voice and video These are mixed together to form a composite stream consisting of video and voice streams E.G.M. Petrakis Introduction 32
Software Architecture
Furht et.al.96
E.G.M. Petrakis Introduction 33
System Functions
Multipoint connection set-up : the system negotiates for network resources Dynamic session control : add/delete participants Conference directory service : supports registration, announcement, query etc.
Automatic scheduling and recording : planning of network resources.
Conference teardown : release reserved resources E.G.M. Petrakis Introduction 34
Architectures
Fully distributed : direct connections between the participants Processing and mixing of media at every location Shortest delay The connections increase rapidly Centralized (star) network : a central is connected to every participant Processing and mixing at central node The central node waits until all media is received before mixing and broadcasting E.G.M. Petrakis Introduction 35
Architectures (cont.)
Double star network : a central node from one star network is connected to another central node of another star network Hierarchical network hierarchically : intermediate nodes, root and leaves (participants) connected intermediate nodes perform mixing and processing the completely mixed data is sent to root who broadcasts directly to the leaves reduces network traffic significantly E.G.M. Petrakis Introduction 36
Multimedia conferencing network architectures
Furht et.al.96
E.G.M. Petrakis Introduction 37
Video on Demand
Fast networks coupled with powerful computers and compression techniques will be capable of delivering stream data in real-time On-demand multimedia services interactive entertainment video news distribution video rental services digital multimedia libraries E.G.M. Petrakis Introduction 38
Interactive Television (ITV)
An ITV system must be capable of providing basic TV subscription TV pay per view video on demand shopping education electronic newspaper financial transactions single-user and multi-user games E.G.M. Petrakis Introduction 39
ITV network Architectures
Distributed architectures, components information (content) servers connected to stb : entertainment companies, TV stations ’s offering different types of services network : wide area network (ATM) connects the head-ends which are connected with subscribers set top boxes (stb) interactive services : terminal devices that allow users interact with a network providing personalized, on-demand, 40
A General ITV Architecture
Furht et.al.96
E.G.M. Petrakis Introduction 41
Hierarchical ITV Architecture
Furht et.al.96
E.G.M. Petrakis Introduction 42
News on-Demand
Delivers news on demand to various corporate and financial services The Nynex and Dow Jones systems: The Dow Jones production center produces news which are send through a wideband network to the Nynex media service center Customer sites can request specific news from the Media Service Center which are delivered over the network and stored at customer sites E.G.M. Petrakis Introduction 43
Architecture of Nynex and Dow Jones system
Furht et.al.96
E.G.M. Petrakis Introduction 44