View
212
Download
0
Category
Tags:
Preview:
Citation preview
1
05MSE E13- Multi Media Computing
By
U.RAHAMATHUNNISA
ASSISTANT PROFESSOR
SITE
2
Introduction
Multimedia Description Why multimedia systems? Classification of Media Multimedia Systems Data Stream Characteristics
CIS 104 3
Multimedia Description Multimedia: is an integration of continuous media
(e.g. audio, video) and discrete media (e.g. text, graphics, images) through which digital information can be conveyed to the user in an appropriate way.
Multimedia technology is the computer-based integration of text, sound, still images, animation and digitized motion video.
Merges capabilities of computers with televisions, VCRs, CD players, DVD players, video and audio recording equipment, music and gaming technologies.
CIS 104 4
Why Multimedia Computing? Application driven
e.g. medicine, sports, entertainment, education Information can often be better represented using
audio/video/animation rather than using text, images and graphics alone.
Information is distributed using computer and telecommunication networks.
Integration of multiple media places demands on computation power
storage requirements networking requirements
CIS 104 5
Multimedia Information Systems
Technical challenges Sheer volume of data
Need to manage huge volumes of data Timing requirements
among components of data computation and communication. Must work internally with given timing constraints - real-time
performance is required. Integration requirements
need to process traditional media (text, images) as well as continuous media (audio/video).
Media are not always independent of each other - synchronization among the media may be required.
CIS 104 6
High Data Volume of Multimedia Information
Speech 8000 samples/s 8Kbytes/s
CD Audio 44,100 samples/s, 2 bytes/sample
176Kbytes/s
Satellite Imagery
180X180 km 2̂ 30m 2̂ resolution
600MB/image (60MB compressed)
NTSC Video 30fps, 640X480 pixels, 3bytes/pixel
30Mbytes/s (2-8 Mbits/s compressed)
CIS 104 7
Technology Incentive
Growth in computational capacity MM workstations with audio/video processing capability Dramatic increase in CPU processing power Dedicated compression engines for audio, video etc.
Rise in storage capacity Large capacity disks (several gigabytes) Increase in storage bandwidth,e.g. disk array technology
Surge in available network bandwidth high speed fiber optic networks - gigabit networks fast packet switching technology
CIS 104 8
Application Areas
Residential Services video-on-demand video phone/conferencing systems multimedia home shopping (MM catalogs,
product demos and presentation) self-paced education
Business Services Corporate training Desktop MM conferencing, MM e-mail
CIS 104 9
Application Areas
Education Distance education - MM repository of class videos Access to digital MM libraries over high speed
networks Science and Technology
computational visualization and prototyping astronomy, environmental science
Medicine Diagnosis and treatment - e.g. MM databases that
provide support for queries on scanned images, X-rays, assessments, response etc.
CIS 104 10
Classification of Media
Perception Medium How do humans perceive information in a computer?
Through seeing - text, images, video Through hearing - music, noise, speech
Representation Medium How is the computer information encoded?
Using formats for representing and information ASCII(text), JPEG(image), MPEG(video)
Presentation Medium Through which medium is information delivered by the
computer or introduced into the computer? Via I/O tools and devices paper, screen, speakers (output media) keyboard, mouse, camera, microphone (input media)
CIS 104 11
Classification of Media (cont.)
Storage Medium Where will the information be stored? Storage media - floppy disk, hard disk, tape, CD-ROM etc.
Transmission Medium Over what medium will the information be transmitted? Using information carriers that enable continuous data
transmission - networks wire, coaxial cable, fiber optics
Information Exchange Medium Which information carrier will be used for information
exchange between different places? Direct transmission using computer networks Combined use of storage and transmission media (e.g.
electronic mail).
CIS 104 12
Media Concepts
Each medium defines Representation values - determine the
information representation of different media Continuous representation values (e.g. electro-
magnetic waves) Discrete representation values(e.g. text characters
in digital form)
Representation space determines the surrounding where the media are presented. Visual representation space (e.g. paper, screen) Acoustic representation space (e.g. stereo)
CIS 104 13
Media Concepts (cont.)
Representation dimensions of a representation space are: Spatial dimensions:
two dimensional (2D graphics) three dimensional (holography)
Temporal dimensions: Time independent (document) - Discrete media
Information consists of a sequence of individual elements without a time component.
Time dependent (movie) - Continuous media Information is expressed not only by its individual value but
also by its time of occurrence.
CIS 104 14
Multimedia Systems
Qualitative and quantitative evaluation of multimedia systems Combination of media
continuous and discrete. Levels of media-independence
some media types (audio/video) may be tightly coupled, others may not.
Computer supported integration timing, spatial and semantic synchronization
Communication capability
CIS 104 15
Data Streams
Distributed multimedia communication systems data of discrete and continuous media are broken
into individual units (packets) and transmitted. Data Stream
sequence of individual packets that are transmitted in a time-dependant fashion.
Transmission of information carrying different media leads to data streams with varying features Asynchronous Synchronous Isochronous
CIS 104 16
Data Stream Characteristics
Asynchronous transmission mode provides for communication with no time restriction Packets reach receiver as quickly as possible, e.g. protocols for
email transmission Synchronous transmission mode
defines a maximum end-to-end delay for each packet of a data stream.
May require intermediate storage E.g. audio connection established over a network.
Isochronous transmission mode defines a maximum and a minimum end-to-end delay for each
packet of a data stream. Delay jitter of individual packets is bounded.
E.g. transmission of video over a network. Intermediate storage requirements reduced.
CIS 104 17
Data Stream Characteristics
Data Stream characteristics for continuous media can be based on Time intervals between complete transmission of
consecutive packets Strongly periodic data streams - constant time interval Weakly periodic data streams - periodic function with finite period. Aperiodic data streams
Data size - amount of consecutive packets Strongly regular data streams - constant amount of data Weakly regular data streams - varies periodically with time Irregular data streams
Continuity Continuous data streams Discrete data streams
CIS 104 18
Classification based on time intervals
Strongly periodic data stream
Weakly periodic data stream
Aperiodic data stream
T
T
T1
T3
T2
T1
T2
T
CIS 104 19
Classification based on packet size
TD1
D1
TD1D2D3D1D2D3
D1D2D3
Dn
Strongly regular data stream
Weakly regular data stream
Irregular data stream
t
t
t
CIS 104 20
Classification based on continuity
Continuous data stream
Discrete data stream
D
D1 D2 D3 D4
D
D1 D2 D3 D4
CIS 104 21
Logical Data Units
Continuous media consist of a time-dependent sequence of individual information units called Logical Data Units (LDU).
a symphony consists of independent sentences a sentence consists of notes notes are sequences of samples
Granularity of LDUs symphony, sentence, individual notes, grouped samples, individual
samples film, clip, frame, raster, pixel
Duration of LDU: open LDU - duration not known in advance closed LDU - predefined duration
CIS 104 22
Granularity of Logical Data Units
Film
Clip
Frame
Blocks
Pixels
Multimedia from a user‘s point of view
Presentation of Information also using Audio and Animation
Information is easier conveyed Integration of Media with the Computer allows for
Interactive Presentations Transmission across computer networks
23CIS 104
Multimedia and the Industry
Sectors affected by the developments in Multimedia
Telecommunications Consumer Electronics TV- and Radiostations Publishing Houses
=> Convergence of Sectors
24CIS 104
Properties of multimedia processing
High data troughput Realtime Synchronisation
25CIS 104
Global Structure/Areas in Multimedia-Systems
Databases Operating Systems Communication System
Computer Technology
Memory Networks
Compression
Audio Video26CIS 104
Areas in Multimedia-Systems
Abstractions
Documents
Applications
User Interface
27CIS 104
Terminology
Multi [lat. much], as a Prefix Medium, mediating element, means to reproduce
Information
28CIS 104
The term „medium“
Means for the distribution and presentation of information Examples: Text, Figure, Image, Voice,
Music Further differentiation of the term in the MHEG-
Standard (1993) On the following two slides
29CIS 104
Different media terms
Perception-Medium To See, to Hear (how?)
Representation-Medium JPEG, MPEG (Structure, Coding)
Presentation-Medium Screen, Speakers, Keyboard
(Input/Output)
30CIS 104
Different media terms
Storage-Medium Paper, Harddisk, CD-ROM (where from?)
Transmission-Medium Network (on what?)
Information-Exchange-Medium Transmitter / Storage of Information: CD,
Network (higher abstraction)
31CIS 104
Presentation values and domains
Presentation domains Screen, Paper (visual presentation
domains) Stereophonetics
Presentation values Character, shock wave
32CIS 104
Presentation dimension
Every presentation domain has presentation dimensions Screen: 2 spatial dimensions
In Addition, Time as a dimension possible Time as additional dimension
fundamental for multimedia systems
33CIS 104
Classes of media
Classification using the time dimension Discrete Media
Independent from time Examples: Text, Figures
Continuous Media Dependent on time Examples: Sound, Motion Picture
34CIS 104
The Term „Multimedia“
Qualitative rather than quantitative understanding What kind of media instead of multiple media
A Combination of text and picture is not sufficient for the term „Multimedia“
„Controversial term“
35CIS 104
Main properties of a multimedia system
Combination of Media Processing of discrete and continuous
media Independency
in the layout of the media in space and time
Computerized Integration Computer as a tool
(Systems able to communicate) Distributed Multimedia-Systems
36CIS 104
Definition of „Multimedia“
A multimedia system is defined through the computerized, integrated creation, manipulation, presentation, storage and communication of independent information coded in at least one continuous and one discrete medium.
37CIS 104
Usage of the term „Multimedia“
In a narrow sense Continuous Media Borders in Time
In a broader sense Shared processing of multiple media
38CIS 104
Transmission of media data
Transmission of data of discrete and continuous media
For information exchange Based on data streams
39CIS 104
Data stream
Units (Packets) Send from source to sink Data stream is a sequence of single packets For discrete and continuous media
40CIS 104
Asynchronous Transmission
No restrictions in time for the packets of the data stream Example: IP
Normally not suitable for multimedia
41CIS 104
Synchronous Transmission
Defined Maximum End-To-End Latency For multimedia this means a guarantee on when
packets will be arriving
42CIS 104
Isochronous Transmission
Defined maximum and minimum end-to-end latency Jitter of single packets is restricted Buffers can be calculated more exact Suitable for „Streaming“, transmission of very large
amounts of data
43CIS 104
Data streams of continuous media
Different Properties Time interval between packets Amount of data / size of packets Correlation of packets
Possibly because of compression Example: Audio- and video transmission
44CIS 104
Strongly periodic data stream
Time interval between neighbouring packets is constant
Example: Voice in a telephone operation systemT
45CIS 104
Weakly periodic data stream
Time interval between packets follows a function of finite period duration
Example: Animation T
T1 T2 T3 T1 T2
46CIS 104
Aperiodic data stream
Neither strongly nor weakly periodic
Example: Transmission of events (e.g. Mouse interactions)
T1 T2 Tn
• • •
47CIS 104
Strongly regular
Constant amount of data of all packets
D1
Dn
t48CIS 104
Weakly regular
Amount of data of packets varies periodicly
D1
t
D2
D3
D1
D2
D3
T
49CIS 104
Not regular
Amount of data is neither constant nor periodic
D1
t
D2
D3
Dn
• • •
50CIS 104
Correlation of successive packets
Coherent data stream Packets are transmitted one after another
without gaps
Incoherent data stream– Gaps between the data packets
t
t51CIS 104
Units of information
Medium consists of a sequence of single units of information
PDU: protocol data unit LDU: logical data unit Example: Logical units of a movie sequence
Movie, Clip, Frame, Raster, Pixel
52CIS 104
Characteristics of multimedia applications
Large quantities of continuous data Timely and smooth delivery is critical
deadlines throughput and response time guarantees
Interactive MM applications require low round-trip delays Need to co-exist with other applications
must not hog resources Reconfiguration is a common occurrence
varying resource requirements Resources required:
Processor cycles in workstations and servers Network bandwidth (+ latency) Dedicated memory Disk bandwidth (for stored media)
At the right timeand in the right quantities
53CIS 104
Application requirements
Network phone and audio conferencing relatively low bandwidth (~ 64 Kbits/sec), but delay times must be
short ( < 250 ms round-trip)
Video on demand services High bandwidth (~ 10 Mbits/s), critical deadlines, latency not critical
Simple video conference Many high-bandwidth streams to each node (~1.5 Mbits/s each),
high bandwidth, low latency ( < 100 ms round-trip), synchronised states.
Music rehearsal and performance facility high bandwidth (~1.4 Mbits/s), very low latency (< 100 ms round
trip), highly synchronised media (sound and video < 50 ms).
54CIS 104
System support issues and requirements
Scheduling and resource allocation in most current OS’s divides the resources equally amongst all comers (processes) no limit on load can’t guarantee throughput or response time
MM and other time-critical applications require resource allocation and scheduling to meet deadlines Quality of Service (QoS) management
Admission control: controls demand QoS negotiation: enables applications to negotiate
admission andreconfigurations
Resource management: guarantees availability of resources for admitted applications
real-time processor and other resource scheduling
55CIS 104
Characteristics of typical multimedia streams
Data rate(approximate)
Sample or frame frequency size
Telephone speech 64 kbps 8 bits 8000/secCD-quality sound 1.4 Mbps 16 bits 44,000/secStandard TV video(uncompressed)
120 Mbps up to 640 x 480pixels x 16 bits
24/sec
Standard TV video (MPEG-1 compressed)
1.5 Mbps variable 24/sec
HDTV video(uncompressed)
1000–3000 Mbps up to 1920 x 1080pixels x 24 bits
24–60/sec
HDTV videoMPEG-2 compressed)
10–30 Mbps variable 24–60/sec
Figure 15.3
56CIS 104
QoS Parameters
Bandwidth rate of flow of multimedia data
Latency time required for the end-to-end transmission
of a single data element
Jitter variation in latency :– dL/dt
Loss rate the proportion of data elements that can be
dropped or delivered late57CIS 104
What is a Multimedia System? A system that involves:
Generation: production/authoring tools Representation: compression and formats Storage: file system design Transmission: networking issues, QoS Search and retrieval: database
management Delivery: service design, QoS of
multimedia information
58
Recommended