34
E.G.M. Petrakis Introduction 1 Technical University of Crete Department of Electronic and Computer Engineering Multimedia Data Management Euripides G.M. Petrakis http://www.intelligence.tuc.gr/~petrakis http://courses.ced.tuc.gr Chania 2010

Technical University of Crete Department of Electronic and Computer Engineering

  • Upload
    sian

  • View
    47

  • Download
    0

Embed Size (px)

DESCRIPTION

Technical University of Crete Department of Electronic and Computer Engineering. Multimedia Data Management Euripides G.M. Petrakis http://www.intelligence.tuc.gr/~petrakis http://courses.ced.tuc.gr Chania 2010. Definition. - PowerPoint PPT Presentation

Citation preview

Page 1: Technical University of Crete Department of Electronic and Computer Engineering

1E.G.M. Petrakis Introduction

Technical University of CreteDepartment of Electronic and Computer

Engineering

Multimedia Data ManagementEuripides G.M. Petrakis

http://www.intelligence.tuc.gr/~petrakishttp://courses.ced.tuc.gr

Chania 2010

Page 2: Technical University of Crete Department of Electronic and Computer Engineering

2E.G.M. Petrakis Introduction

DefinitionMultimedia: composite entities

combining text, audio, images, video (bit-stream objects), graphics

Multimedia Information Systems: database systems that support all multimedia data types and handle very large volumes of information

Page 3: Technical University of Crete Department of Electronic and Computer Engineering

3E.G.M. Petrakis Introduction

TEXTThe most common type of

informationThe least space intensive data typeThe form in which text is stored

varies (plain ascii, word files, spreadsheets, annotations, database fields etc.)

Text fonts are becoming complex allowing special effects (color, shade, fill etc.)

Page 4: Technical University of Crete Department of Electronic and Computer Engineering

4E.G.M. Petrakis Introduction

AUDIO Space intensive (one minute can take up

Mbytes), presented as analog, digital or MIDI

Analog waveform: electrical signal, amplitude specifies the loudness of the sound in microphones, tapes, records, amplifiers,

speakersDigital waveform audio: digital,

less sensitivity to noise and distortion involves larger processing and storage

capacitiesDigital Audio Tape (DAT), Compact discs (CD)WAV (Microsoft’s wave file format)

Page 5: Technical University of Crete Department of Electronic and Computer Engineering

5E.G.M. Petrakis Introduction

MIDI (Musical Instrument Digital Interface)

Commands that describe how the music should be played are stored (instead of sound)

A music synthesizer generates sound Provides high data compression, Widely accepted

Furht et.al.96

Page 6: Technical University of Crete Department of Electronic and Computer Engineering

6E.G.M. Petrakis Introduction

IMAGESDigital images: sequences of pixelsPixels: numbers interpreted to

display intensity, color, contrast etc Binary (0-1 values), gray-scale (8

bits/pixel), colour (3x8 values for RGB)Space overhead depends on image

type, resolution, compression scheme

Image formats: tiff, bmb, jpeg etc.

Page 7: Technical University of Crete Department of Electronic and Computer Engineering

7E.G.M. Petrakis Introduction

Image Concepts and StructuresBinary images: 1 bit/pixel

black & white photos, facsimile imagesComputer Graphics: 4 bits/pixelGrayscale images: 8 bits/sampleColor images: 16, 24 bits/pixel

Page 8: Technical University of Crete Department of Electronic and Computer Engineering

8E.G.M. Petrakis Introduction

RGB RepresentationA color is produced

by addingred, green and blue

The straight line R=G=B specifies gray values ranging from black to white

Page 9: Technical University of Crete Department of Electronic and Computer Engineering

9E.G.M. Petrakis Introduction

YUV RepresentationYUV describes the luminance and

chrominance components of an image1 luminance: gray scale version of an

imageY = 0.299R + 0.587G + 0.114B

2 chrominance components:U = 0.564(B - Y)V = 0.713(R - Y)

Page 10: Technical University of Crete Department of Electronic and Computer Engineering

10E.G.M. Petrakis Introduction

ConversionsConversion between RGB and YUV

requires multiplication operationsan approximation: Y = R/4 + G/2 +B/2, U=(B-Y)/2, V=(R-

Y)/2R = Y + 2V, G = Y – (U + V), B = Y + 2U

YCbCr is another color format for compressionCb = U/2 + 0.5, Cr = V/1.6 + 0.5

Page 11: Technical University of Crete Department of Electronic and Computer Engineering

11E.G.M. Petrakis Introduction

VIDEOThe most space intensive data typeA sequence of framesRealistic video playback, transmission,

compression/decompression require transfer rates about 30frames/sec

Microsoft’s AVI and Apple’s Quicktime file formats integrate video and audio in the same presentation

Page 12: Technical University of Crete Department of Electronic and Computer Engineering

12E.G.M. Petrakis Introduction

Audio-video Modes of Operation

Can be either stored or used / transmitted live in real-time

Can be used interactively or non-interactively

Furht et.al.96

Page 13: Technical University of Crete Department of Electronic and Computer Engineering

13E.G.M. Petrakis Introduction

GRAPHICSObjects described through their basic

elements (e.g., 2D, 3D shapes)these elements can have different sizes,

position, orientation, surface, fill etc. compact representationsgenerated and can be manipulated by

design tools (e.g., CAD tools)Their descriptions are stored in files

Page 14: Technical University of Crete Department of Electronic and Computer Engineering

14E.G.M. Petrakis Introduction

Khoshafian Baker 96

2D and 3D graphics objects

Page 15: Technical University of Crete Department of Electronic and Computer Engineering

15E.G.M. Petrakis Introduction

MULTIMEDIA objectsText, audio, images, video, graphics are

elements of complex multimedia objectsVarious tools or applications integrate,

process and combine multimediaApplications: multimedia authoring

applications that output documents and databases and end-user applications (e.g., video on demand)

Tools: for viewing, updating, querying (presentation viewers, browsers etc.)

Page 16: Technical University of Crete Department of Electronic and Computer Engineering

16E.G.M. Petrakis Introduction

Multimedia Databases (MDB)

Means stored information or database management systems (dbms)

Multimedia dbms (mdbms) integrate conventional database capabilities together with different technologies such as Hierarchical storage management (HSM) and Information retrieval (IR)

Page 17: Technical University of Crete Department of Electronic and Computer Engineering

17E.G.M. Petrakis Introduction

Multimedia TechnologiesTechnologies integrated within a mdbms HSM support IR support (exact and approximate)Spatial data types and queriesInteractive querying, relevance feedback,

refiningAutomatic feature extractionAutomatic content retrieval and indexing Query optimization

Page 18: Technical University of Crete Department of Electronic and Computer Engineering

18E.G.M. Petrakis Introduction

Database capabilities Persistence: object persist through invocations Transactions: content is inserted, deleted,

updated Concurrency control: transactions run

concurrently Recovery: failed transactions are not propagated

to the db Querying: content can be retrieved Versioning: access previous states of objects Integrity: transactions guarantee consistency of

content Security: constraints for accessing/updating

objects Performance: optimal data structures and

programs

Page 19: Technical University of Crete Department of Electronic and Computer Engineering

19E.G.M. Petrakis Introduction

Hierarchical Storage Management (HSM)

Support storage of multimedia objects On-line: on RAM, magnetic diskNear-on-line: on optical storageOff-line: on tapes, shelves

Each level has differentPerformance: decreases from top to

bottomCapacity: increases from top to bottomCost: decreases from top to bottom

Page 20: Technical University of Crete Department of Electronic and Computer Engineering

20E.G.M. Petrakis Introduction

Information Retrieval (IR) CapabilitiesRetrieval is the most common operation

Deletions and updates are less commonExact match: search based on exact

informationInexact: search based on inexact information

e.g., partial, neighborhood search, can be fuzzy or probabilistic

The results are ranked by order of relevance to the query

Query refinementIterate over query results Adjust weights of query terms or featuresAnd finally resubmit queries

Page 21: Technical University of Crete Department of Electronic and Computer Engineering

21E.G.M. Petrakis Introduction

Khoshafian Baker 96

MDBMS architecture

Page 22: Technical University of Crete Department of Electronic and Computer Engineering

22E.G.M. Petrakis Introduction

MDBMS ImplementationRelies on 3rd party vendors for each

componentRelational dbms for typical records separate optical storage module for

text/audio/graphics/images/videoText retrieval system (e.g. Lucene)Audio/image/video retrieval systemFeature extraction systemMultimedia object interface system

Page 23: Technical University of Crete Department of Electronic and Computer Engineering

23E.G.M. Petrakis Introduction

Object-Oriented Multimedia Databases

Better design, better suited for multimedia applications Uniform handling of data and operationsData types are objects with internal structures

and operations that capture the behavior of objects (e.g., audio playback, video browsing)

OO dbms does not satisfy all MM requirementsProvides primitives for object handling Multimedia components need to be

implemented or integrated

Page 24: Technical University of Crete Department of Electronic and Computer Engineering

24E.G.M. Petrakis Introduction

Multimedia ApplicationsMultimedia Systems suggest a

variety of applicationsMultimedia conferencing Multimedia on demand (interactive TV,

news on demand)See next page for more …

Page 25: Technical University of Crete Department of Electronic and Computer Engineering

25E.G.M. Petrakis Introduction

Furht et.al.96

Multimedia Applications

Page 26: Technical University of Crete Department of Electronic and Computer Engineering

26E.G.M. Petrakis Introduction

Multimedia Conferencing (MC)

Multimedia conferencing enable a number of participants to exchange multimedia informationEach participant has a workstation

linked to other workstations over high-speed networks

Each participant can send or receive mm data and perform certain collaborative activities

Page 27: Technical University of Crete Department of Electronic and Computer Engineering

27E.G.M. Petrakis Introduction

Furht et.al.96

The biggest performance challenge occurs when the participants transmit voice and videoThese are mixed together to form a composite

stream consisting of video and voice streams

A video conference system

Page 28: Technical University of Crete Department of Electronic and Computer Engineering

28E.G.M. Petrakis Introduction

Furht et.al.96

Software Architecture

Page 29: Technical University of Crete Department of Electronic and Computer Engineering

29E.G.M. Petrakis Introduction

ArchitecturesFully distributed: direct connections

between the participants Processing and mixing of media at every

locationShortest delayThe connections increase rapidly

Centralized (star) network: a central is connected to every participantProcessing and mixing at central nodeThe central node waits until all media is

received before mixing and broadcasting

Page 30: Technical University of Crete Department of Electronic and Computer Engineering

30E.G.M. Petrakis Introduction

Architectures (cont.)Double star network: a central node from

one star network is connected to another central node of another star network

Hierarchical network: intermediate nodes, root and leaves (participants) connected hierarchicallyintermediate nodes perform mixing and

processingthe completely mixed data is sent to root who

broadcasts directly to the leavesreduces network traffic significantly

Page 31: Technical University of Crete Department of Electronic and Computer Engineering

31E.G.M. Petrakis Introduction

Furht et.al.96

Multimedia conferencing network architectures

Page 32: Technical University of Crete Department of Electronic and Computer Engineering

32E.G.M. Petrakis Introduction

Video on Demand (VoD)Fast networks coupled with powerful

computers and compression techniques will be capable of delivering stream data in real-time

On-demand multimedia servicesinteractive entertainment video news distributionvideo rental services digital multimedia libraries

Page 33: Technical University of Crete Department of Electronic and Computer Engineering

33E.G.M. Petrakis Introduction

Interactive Television (ITV)An ITV system must be capable of

providingbasic TVsubscription TVpay per viewvideo on demandshoppingeducationelectronic newspaperfinancial transactionssingle-user and multi-user games

Page 34: Technical University of Crete Department of Electronic and Computer Engineering

34

This Course Emphasis on

Text, images, video Information retrieval & systems Data organization Web information systems Semantic Web Video & MPEG standards

No emphasis on Architectures Specific applications (VoD, ITV,MC) Services

E.G.M. Petrakis Introduction