Seminar handout, June 18, 20001

1

Digital Media Technology : Streaming Video

Kim, TaeYong

2001. 6.

2

ContentsContents

Streaming videoMaking moviesMPEGMPEG transcoding System and Protocols

3

How to stream videoHow to stream video

Four Basic Steps: – Step 1: Creating Content– Step 2: Digitizing the Video– Step 3: Compressing/encoding the Video– Step 4: Serving the Video

Download the entire video filePseudo-streaming (Progressive): HTTP”True" streaming (Real-time) : RTSP

4

How to stream videoHow to stream video

MultimediaServer

MultimediaServer Multimedia

Client

MultimediaClient

DataBase DataBase

HeterogeneousNetwork

HeterogeneousNetwork

ATM, TCP/IP, Wireless, PSTN

5

Streaming BasicsStreaming Basics

What is Streaming?– Now generally refers to media, such as video and

audio, that is delivered over a network.

Architecture and Codecs– Synchronizing, managing and playing media – Architectures provide the overall structure and

synchronization for media delivery. – Codecs are the smaller encoding components that

fit within an architecture.

6

Video QualityVideo Quality

Frame Rate (Frames Per Second, fps)– To adequately lip sync audio to a "talking head," a

range of 8 to 12 fps is generally recommended. – Furthermore, for full motion video, the difference

between 15 fps and 30 fps is minimal.

Compression (codecs)– With 176x144 pixels, 25,344 pixels per a frame– 16 bits of color gives 405,504 bits per a frame– With motion of 15 fps 6,082,560 bits per second

7

Video QualityVideo Quality

Lossless : run-length encoding or Huffman coding

Lossy : Removes picture information that viewers aren't likely to notice

Spatial Compression : Compresses the data in each frame

Temporal Compression : Stores only the data that changes from one frame to the next

8

Making MoviesMaking Movies

Making multimedia movies– 1. Start with your source material– 2. Capture the video– 3. Edit your video with an editing program– 4. Compress your movie– 5. Distribute your final compressed movie

9

Making Movies (Source)Making Movies (Source)

Use high-quality originals Light for compressionUse a tripod Keep detail to a minimumKeep movement to a minimum

– The overall goal of creating good multimedia video is to produce a video signal with the least noise, camera movement, and fine detail possible so that the final movie will compress well and look good at a small screen size.

10

Making Movies (Capture)Making Movies (Capture)

How they workHardware Compression Pricing

– The capture step is where you change the original analog video into a digital signal in your computer with a capture card, and then store the video on a hard drive. You need a capture card, a fast hard drive, and a finely tuned system to do this well. The captured video is probably only viewable by people who have the same hardware as you, so you will need to make your final video in a format that is not dependent on your capture card.

11

Making Movies (Edit)Making Movies (Edit)

Basic edits Edit and Add EffectsComplicated edits Save the file in the right format

– Editing is the fun step where you take your raw video and turn it into a movie. You can also add effects at this point. There are many different programs to edit and add effects - choose your tools based on your needs and budget. I recommend you "cover your butt" by archiving your edited (but uncompressed)

movie for future use.

12

Making Movies (Compress)Making Movies (Compress)

Compression for CD-ROM Compression for WWW Frame-rate, data rateImage size, codec choice

– Compression reduces your huge video file to a size that will play properly from CD-ROM or the Web. Since you are going to lose quality, it is important that you use a dedicated compression tool to get the best results.

13

Making Movies (Distribute)Making Movies (Distribute)

Distribute– Making the CD-ROM

• Interactivity for CD-ROMs• CD-ROM Recordable "Burners"

– Making the WWW site• Putting movies on your site

– You can distribute your movies via CD-ROM or the Web. You'll need some hardware for making the actual CD-ROM, mainly a "burner" and some media. If you're making a web site, there are several different programs available.

14

MPEGMPEG

Intra-frame Compression

15

MPEGMPEG

Inter-frame Compression – Temporal Redundancy

16

MPEGMPEG

Inter-frame Compression – Motion Compensation

17

MPEGMPEG

MPEG compression flow

18

MPEGMPEG

기능별 처리 시간F unc t ion T ime(%)P ars ing B i ts t ream 17.4%I DC T 14.2%Rec ons t ruc t ion 31.5%Di ther ing 24.5%Mis c . A r i th. 9 .9%Other 2 .7%

MPEG Decoder processing time

MPEG multiplexing

19

MPEG TranscodingMPEG Transcoding

MPEG-1, MPEG-2, MJPEG, H.261ATM, PSTN, Wireless, TCP/IPQuality (PSNR), Time constraintFormat conversion, bit-rate conversionTranscoder Location – node to node (gateway, router)– end to end (client - server)

20

MPEG TranscodingMPEG Transcoding

Video applications– Video conferencing– Interactive video– Video editing/publishing

TranscoderTranscoder

EncoderEncoder DecoderDecoder DecoderDecoderEncoderEncoder

Video InVideo In Video OutVideo Out

NetworkInformation

NetworkInformation

21

MPEG transcodingMPEG transcoding

Transcoding of different levels of complexity– T1, T2, T3 level

22

MPEG transcodingMPEG transcoding

Transcoding of different levels of complexity– T1 : DCT level -> drift error (error propagation)– T2: Pixel level – T3: Pixel level with motion estimation

23

MPEG transcodingMPEG transcoding Frame dropping (temporal)

– 6~15 frames per GOP– I : 20~ 60 %– P: 20~40 %– B: 30~40 %

24

MPEG transcodingMPEG transcoding

Re-quantization (spatial)– Require Step-sizes model– Linear or non-linear model

Low-pass filtering (spatial)– 16~64 Good, 8~16 Visible – 4~8 possible, 1~4 poor

– 1/2 size at 16 AC

c

cii

Ciif

CiifACAC

0

25

System and ProtocolsSystem and Protocols Flow management model– Flow specification

• QoS Specification profile

– Flow Establishment• Resource management (RSVP)• Filter allocator• Packet scheduler

– Flow Maintenance• Monitor, Session manager

– Flow Release

26

System and Protocols System and Protocols (Example)(Example)

Service Request

Accept, Error report

Broker

USER

Session Info. Reporting

Network Information

Main Server Agent

Filtering Server

MPEG StreamAcceptance,Error report

Service RequestSession ConnectionNetwork Information

Information flow

27

System and Protocols System and Protocols (Example)(Example)

A. Storage B. Transcoder C. Comm. Server D. User

……

Connect(RTP)

Connect(RTCP)

Stream Read

Request Stream(RTP)

Stream Send(RTP)

Time Stamp ti-1(RTCP)

Stream ReadStream Send(RTP)

Time Stamp ti(RTCP)

E. Transcoding

E. Transcoding

2-way protocol

28

System and Protocols System and Protocols (Example)(Example)

A. Storage B. Transcoder C. Comm. Server D. User

……

Connect(HTTP)

Stream Read

Request Stream(HTTP)

Stream Send(TCP)

Stream ReadStream Send(TCP)

E. Transcoding

E. Transcoding F. Timer (ti-1)

F. Timer (ti)

1-way protocol