Introduction to Audiovisual Communications

Audiovisual Communications

Introducción a las Comunicaciones Audiovisuales

Profesores: Francesc Tarrés, Gabriel Montoro

1


Introduction•Audiovisual Communications Technologies•Set of technologies for transmitting or storing signals

representing speech, audio, image or video information in different formats.

•Audiovisual Technologies Impact:•Communications among persons through sound and image is

(still today) the media of major social impact.•Classical Media: Films, TV, Radio, Telephone•Networked contents: Production webs, Podcast, Video On-Demand•Contents merchandising: iTunes, Spotify•Personal Communications: Facetime, WeChat•Social Networks: Facebook, Instagram, Flickr, etc.

2


Introduction

3


Historical Evolution•Constantly improving the audiovisual experience

•Technologies are ‘determined’ by the characteristics of human auditory and visual systems.

•Today almost all AV systems are digital. This course is centered on digital systems.

•However, many AV paradigms are understood when looking to the historical evolution of the system. Some KEY historical aspects are introduced to facilitate the understanding of current systems.

•Audiovisual Systems are evolutionary. The introduction of new technologies preserves some degree of compatibility with previous systems.

4


Introduction•Evolution example (Comercial TV)•Black & White Television•Analog Color•Digital Color (Standard Definition)•High Definition•3D•Ultra High Definition

•Non-evolutionary attempts have usually failed•Analog High Definition TV in Europe (late 80s, beg. 90s)

•Economic efforts for acquiring new technologies is almost constant along years

5


Future Systems•It is difficult to predict the evolution of AV systems. Past experience

show that many promising AV systems have not succeed: HD-MAC, DCC, Laser-disc, Minidisc, etc.

•A possible future line prediction could be

•Ultra-High Definition (4K and 8K commercially available at low cost)• Cameras, editing software, displays

•High Quality Audio Portable systems: Pono• Introduction of Multisense information into the experience

• Location, motion, body sensors, weather, light, etc.•Advances in 3D technologies

• Displays without glasses (multiuser)• Introduction of holographic technologies• Multicamera - Multiview

•Capturing and transmission of otner sensorial signals• Olphative• Tactil

6


Future Systems•No matter how real systems evolve the main characteristics

from the point of view of signal processing will remain the same•The following steps will be necessary

1. Signals must be captured and coded efficiently2. The coding has to be adapted to the transmission channel or to the storage support.3. Reproduction technologies must guarantee a quality experience to the audience

•The design and technologies of these stages has to be adapted to the psycho-physiology human sensorial system (auditory, visual, olphactive, etc)

7


Communication Systems Block Diagram

8

Capture SourceCoding Multiplexer

Encryption &ChannelCoding

ChannelDecoding

& DecryptionDemuxSource

DecodingPresentation

Storage o Transmission

Modulators & Demodulators(QAM, OFDM, PSK)IP data transmissionFile DownloadingDVD-Video, Blu-ray

Capture Presentation

Signal transducer

Source Coding

Compression

Multiplexing

Collectseveral channelsinto one stream

Encryption and ChannelCoding

EncriptionAdapt Data to

MediaError protection

MicrophonesVideo Cameras

DisplaysSpeakers

A/D and D/A PCM coding

FLAC, APEMP3, AAC

TIFF, PNGJPEG, JP2

MPEG-2, H.264

MPEG-TSMPEG- PS

AVI, MP4, MKV

Block CodingHamming

Reed-Solomon

Conv. CodingTrellisLDPC

Condtional AccessEncription Scrambling


Digital AV Systems•Advantages•Immunity to noise•Easy reprogramming•Low dependence on external factors (e.g. temperature)•Easy encryption, data protection, Digital Rights Management

•Disadvantages•Introduction of Quantization Noise•Bandwidth Increase•Difficulty for managing huge databases (video browsing and retrieval)

9


Quantization Noise•Example in audio•The number of levels in the A/D determines the quality of the

resulting digital signal.•Error in quantizing is equivalent to noise.•2 different criteria for digitalization

•Quality: Suppose analog signal is noise free. Determine the number of bits necessary for a human being not being able to distinguish between original analog and the digital version

• N bits audio = 16 bits (20 bits, 24 also possible in some systems).

•SNR. Suppose the analog signal has a given SNR. Choose the number of bits (number of levels) to have a quantization noise below the analog noise.

10

S NR = 6N + 1.76


Bandwidth Increase•Example audio (CD audio - 1 channel)

•Example video (Standard Definition)

11


Example in Digital Cinema

12

Bandwidth calculation in Digital Cinema Applications

Digital Cinema requires a huge bandwidth to support a full film if no compression is applied


Browsing and Retrieval•Insert Metadata in Audio and Video Files•Manually•Index data by humans: ex. soccer, TV News, others•Automatic•Find automatic descriptors (low-level) for audio and video data.•Use these low level descriptors to infer semantic contents•Standard MPEG-7•Other proprietary solutions:

•Google Images, Shazam, Midomi, SoundHound, etc.

13


Digital AV Systems

14

•Advantages•Immunity to noise•Easy reprogramming•Low dependence on external factors (e.g. temperature)•Easy encryption, data protection, Digital Rights Management

•Disadvantages•Introduction of Quantization Noise•Bandwidth Increase•Compression: MP3, AAC, MPEG-1, MPEG-2, MPEG4, H.264, H.265

•Difficulty for managing huge databases (video browsing and retrieval)• Metadata description• MPEG-7


List of Contents•Introduction•Image Processing I: Point Transforms•Image Processing II: Spatial Transforms•Cameras. Principles of geometric optics•Audiovisual Systems: Historical Perspective•Principles of Compression•Still Image Compression: JPEG, JPEG 2000•Video Compression: MPEG-2, MPEG4, H.264, H.265•Fundamentals of audiovisual description: MPEG-7

15


Organization•Theory•2 sessions of 1.5 h every week (Prof. Francesc Tarrés)•Controls and Exams•Arrange meetings: [email protected]•Updated slides and textbooks: francesctarres.wordpress.com

•Laboratory•1 session of 2 h every week (Prof. Gabriel Montoro)•Lab work and Lab reports•Arrange meetings: [email protected]

16

mailto:[email protected]


Assessment

17

Control (week 4-5) 10%

Mid-Term Exam 20%

Final Exam 30%

Lab work 15%

Lab reports 15%

Participation 10%

Education

Introduction to Audiovisual Communications