Video Production for Streaming

1

Producing Video

for the Web

Jan Ozer

Doceo Publishing

[email protected]

276-238-9135

Copyright © 2007 Jan Ozer, All Rights Reserved

Agenda

The streaming environment

Production

Setting the scene (background and clothing)

Lighting

Camera positioning, framing and motion

Optimizing camera controls

Editing/Preprocessing

Choosing your targets

Producing to your targets

Encoding

2


The Streaming Environment

Concepts

Bandwidth

Delivery modes

Resolution

Data rate

How streaming is different from analog


Why Do I Care About Bandwidth?

What is bandwidth?

Viewer’s connection speed

Why is it important?

Controls your viewer’s ability to retrieve and playvideo smoothly

Higher delivery bandwidths mean higher datarates, which means better quality

3


What are My Delivery Options?

StreamingViewer clicks, video quickly starts to play

Plays without interruption until finished

Progressive downloadViewer clicks, video quickly starts to play

Video may stop during playback, but stored locally

Once completely downloaded, plays smoothly

Download and playFile downloaded completely before playing


Why is Delivery Mode Relevant?

To stream effectively:

Must compress to bandwidth available to viewer

Since compression degrades quality, streaming is the mostchallenging delivery mechanism

Must distribute efficiently from the host (which often meansa streaming server)

Progressive download

Don’t have to compress to effective bandwidth, which maymean higher quality

But, user may have to wait (e.g. QuickTime movie trailers)

4


Why is Resolution Important?

Most video starts life at 720x480 or higher

Most video is scaled down for streaming

320x240 is the general minimum

Ranges up to 640x480 and higher

Resolution is key quality factor

At a set bit rate, increasing resolution degradesquality because you must compress more pixels

Can’t say that 300 kbps is “adequate” withoutknowing resolution


Why is Data Rate Important?

Largely determines video quality

At static resolution and frame rate, increasing bitrate increases quality

Because we have to pay for it

5


How is Streaming Different?

When truly streaming (not progressivedownload or download and play)

Typically have to compress video significantlyto:

Meet end user bandwidth requirements

Meet bandwidth costs

Streaming is “lossy” - the more you compress, themore you lose quality-wise


When to Customize Production for

Streaming

No difference

Must produceExclusively for

streaming

1 mbps 500 kbps 300 kbps 100 kbps

Similarity to Analog and DVD

DeliveryBandwidth

1.5 mbps

640x480

320x240

6


What’s this Mean?

At 1-1.5 mbps (640x480), most video looks finewithout special care

As data rates decrease, have to:

Limit motion in video (camera and subject)

Choose compressible backgrounds and clothing

Generally use compression friendly techniques

The lower the bit rate, the more you care


Streaming Environment -

Conclusion

Questions?

7


Producing for Max Streaming

Quality

Production

Setting the scene

Choosing a background

Clothing

Lighting

Camera positioning, framing and motion

Optimizing camera controls


Which Backgrounds Work Best?

Backgrounds

What’s different between streaming and analogproduction?

High level goals for choosing a background

Best practices

Choosing a background:

For my in-house studio

While on location

Conclusion and checklist

8


Background - What's Different?

Contrast issues:

Very similar between analog and streaming -need to separate subject from background

Other issues:

Backgrounds with motion can severely degradecompressed video quality

Poorly chosen backgrounds (well lit, reflectiveblank walls) can "create" noise


What do I Care About When

Choosing a Background?

Provide contrast with talentObviously relates to clothing worn by talent

Avoid color and contrast extremes

Choose one that compresses wellNo motion

Low detail

No wide open spaces (embrace clutter)

No highly saturated colors

9


Contrast

Good contrast

Deloitte - dark grey suit,light blue back

Cranky Geeks - brownsuit, black background

HP - grey shirt, darkbackground

HP - blue shirt, brownbackground

Amex - light blue shirt,grey background

Bad contrast

P&G - blue/blue

Price - black/black

NIST - black/black

Real - blue/blue


Contrast

Good contrast

Deloitte - dark grey suit,light blue back

Cranky Geeks - brownsuit,black background

HP - grey shirt, darkbackground

HP - blue shirt, brownbackground

Amex - light blue shirt,grey background

Bad contrast

P&G - blue/blue

Price - black/black

NIST - black/black

Real - blue/blue

10


Avoid Color/Contrast Extremes

Brightness extremes hard forcamera/codecs to preserve

Color extremes make it tough tomaintain contrast


Avoid Color/Contrast Extremes

11


Avoid Motion

Akamai

NAB


Avoid Wide Open Spaces

DON'T - Use flat, well-lit, lightcolor backgrounds withoutsome "clutter.

DO

Add "clutter" to the backgroundto contain artifacts

Darken background to

Reduce detail

Reduce reflection

12


Avoid Highly Saturated Colors

DON'T use highlysaturated colors – Whichtend to create motion inthe background

DO - Use more mutedcolors


Avoid Fine Patterns

DON'T - use backgroundswith fine patterns and/ordecorative lighting.

DO - use a simplebackground with little visibledetail

13


Creating the Perfect In-HouseBackground

Simple is better

Flat black works well

MS got fancy andlost contrast

Be Flexible

Curtain system withmultiple simplebackgrounds


Building the PerfectBackground - Office

1 stand (~ $100 US)

2-3 backgrounds(~$130 US)

Flat black

Dark grey

Light blue

Under $250, you’re set

14


Building the Perfect

Background - On Location

Need “on location” feel

Curtains and portable backdrops out

How to create compressible set?


Building the Perfect

Background - On Location

Avoid contrastextremes

Avoid complextextures and shapes

Limit detail withlighting (darken thebackground)

Or by blurring thebackground withcamera settings

Avoid backlighting

15


Background Checklist

Does the background provide contrast with subject's face,clothing and hair?

Are there extremes in contrast and/or color

Is there extraneous detail in lighting or pattern?

Is it moving?

Are the colors highly saturated (rich reds and blues)?

Are there well lit, wide open spaces?

Are there any light sources like lights or windows?

Have you tried compressing the footage and viewing theresults?


What Should Talent Wear?

What’s different between streaming andanalog production?

Feng Shui in clothing and backgrounds

Avoid contrast extremes

Other issues

Clothing checklist

16


Clothing - What's Different?

Must be more sensitive to detail that canproduce compression artifacts (jewelry,glasses, hair)

Similar in other aspects, though compressionexacerbates issues

Contrast (no black and white)

Details (no stripes, herringbones)


Clothing Checklist

ClothingSolid colors - dark blues, grays or browns.

Avoid white and light blue (which looks like white on camera)

Advise in advance to match background, and perhaps bringalternatives (or lighter shirt/darker coat)

Hair

Pulled back (Loose ends disappear once compressed)

Jewelry

Some OK, large bling is additional detail and reflective

Glasses may require special lighting (generally fromthe side) to avoid glare

17


Clothing Checklist

Clothing

Solid colors - dark blues, grays or browns

Avoid white and light blue (which looks like white on camera)

Advise in advance to match background, and to bringalternatives (or lighter shirt/darker coat)

Hair

Pulled back (Loose ends get messy after compression)

Jewelry

Some OK, large bling is additional detail and reflective

Glasses may require special lighting (generally fromthe side) to avoid glare


How do I Light for Max Quality?

What’s different between streaming andanalog production

Overview

Decision time - flat lighting or shadows

Producing three-point lighting (bonus)

Producing flat lighting (bonus)

18


Lighting - What's Different?

Overall, analog and streaming are similar,except that:

Compression decreases overall contrast ratio, solighting extremes are not well tolerated

Shadows present additional "detail" that must bepreserved, which steals bandwidth from importantdetail -- like subject’s faces -- and can degradequality


Overview

Lighting is the single most importantdeterminant of video quality

Low lighting causes gain (noise) in the video,complicating compression

Hierarchy of considerationsEnsure lighting is adequate to produce clear,easily compressible image

Then worry about style and mood

19


Know Your Lighting Styles

Three-point

Useful for setting mood

Hard to set up

Heavy shadows tough tocompress

Flat

Visually neutral (e.g. boring)

Easy to set up

Easy to compress


54Corporate case study

54Corporate informational

59Internet-only broadcast

014Traditional broadcast

3-PointFlatCategory

Which Lighting Style Should I Use?

Clearly can go either way

Origins of lighting

Set (or not set) mood

Model the face

20


3-Point Lighting Continuum

From subtle modeling

To “classy” look

To “artistic”

To “dramatic”

To going to far


3-Point Rules

Match lighting tomood

Subtle modeling todramatic

Match lighting toenvironment

Otherwise, lightingmay look contrived

21


The Net/Net on Lighting

No one ever got fired for using flat lighting

Easier to setup and compress, but can be boring

3-Point Variations

Subtle - to create modeling

Dramatic - to create mood

Good for case studies and interviews

Questionable for fact oriented business video

Best when lighting matches environment

Watch contrast ratio issues


Camera Positioning, Framing and

Settings

What’s different between streaming andanalog production

Framing for streamingSafe zone

Framing on the ‘Net

Rule of thirds

Managing camera motion

Producing a soft background (bonus topic)

Progressive vs. interlaced

22


Camera Skills - What's

Different?

No safe zone

Positioning for a smaller window

Minimize camera motion

Shooting in progressive mode


How do I Frame for Streaming?

Analog video has safezone

Outer 10-15% eliminatedas overscan

All pixels show instreaming

So:

Can zoom in a bit tighterb/c more room for othercontent

Title safe zone

OK for streaming, wouldn't be visibleon a TV set

23


Framing for the Internet

Most streaming video produced at 320x240 -480x360

Smaller viewing window, and no safe zone, soyou should frame more tightly

But, viewed from much closer to screen

But - don’t deviate from classic positioning*

Classic framing is what people expect

Don’t make what should be a medium shota close up just because it’s streaming

* How Low Can You Go, The Effect of low Resolutions on Shot Types in Mobile TV, Knoche, et al.


Framing on the Net

Extreme close-up, Edgy - “listen to me!”

Medium close-up, arm-pitsUp - Good all around shot

Medium Shot - Waist up -Good for high res videos

24


What’s the Rule of Thirds?

When facing the interviewerTalking to someone else

Position in back third of theframe, eyes at or near the top1/3

With "Look room" in directionthey are facing

When facing the camera

Talking to the viewer

Position in the middle

Eyes at or near top 1/3


Managing Camera Motion

Camera - shooting techniques

Use a tripod

Avoid panning and zooming whenever possible. Cutsare better

If possible, soften background (bonus topic)

Limit on-screen motion

Talent – consider sitting, and advise against excessivemotion

25


Should I Shoot Progressive?

What’s progressive (and why do I care?)

The short answer

Detailed findings (bonus)


Progressive vs. Interlaced

Most SD cameras capture interlaced video

60 fields per second, not 30 frames

Fields are captured 1/60th of a second apart

All streaming class codecs are frame based

Converting from fields to frames can cause a stair stepor Venetian blind artifacts which deinterlacing canresolve

Progressive cameras shoot 30 fps and divide into fields

Fields match up perfectly for streaming production

26


Can’t Editors Deinterlace?

Yes, but artifacts linger


Should I Shoot Progressive?

Does progressive source produce betterstreaming video than interlaced?

Ran serious of tests described in bonus materials

Short answer is yes - in all but the most controlledenvironments

27


Conclusion - Real World

Better retention of fine detail

Eliminate/minimize the jaggies


Conclusions

Progressive delivers noticeably better qualityin all but the most controlled shoots

So:

If you're buying a camera for streaming, buyprogressive

If you have progressive, use it

If your camcorder doesn't have progressive, andyou shoot lots for streaming, consider new unit

Article here: http://www.nxtbook.com/nxtbooks/streamingmedia/0607/index.php

28


Production - Conclusion

Any Questions?


Editing/Pre-processing

Choosing your targets

Producing the correct output resolution andaspect ratio

Deinterlacing

Noise reduction filtering

Workflow

29


What Output Parameters

Should I Use?

Codec

Resolution

Frame rate

Audio bit rate


About the Survey

Targets

Informal survey of 50+ web sites

Two categories, broadcast and corporate

Files chosen

Broadcast - most popular shows on the channel

Corporate - searched for “Flash Video,” “WindowsMedia” and QuickTime to find video files

Chose newest, highest resolution streaming file

30


About the Survey

Broadcast (partial)

ABC, CBS, Fox, CNN, ESPN, The Golf Channel,CNET, BusinessWeek, Forbes, Fortune, WSJ, NYTimes, SI, DL.TV

Corporate (partial)

Accenture, AMEX, Visa, IBM, E&Y, Chevron,Cisco, EDS, Microsoft, HP, GE


Codec Selection, 4/’07 & 11/’07

41034Corporate - aggressive

54065Corporate – conservative

Corporate

12004Broadcast – aggressive

550310Broadcast – midrange

21002Broadcast – conservative

Broadcast

4/07WMV

4/07Flash

11/07Other

11/07WMV

11/07Flash

What question does this slide answer?

31


Analysis

Broadcast channels almost all Flash

Notable WMV defectors (Weather Channel, Fox,MSNBC (????) and CNN)

Windows Media still popular in corporate

For streaming:

Saw Real once (EDS, newer videos were Flash)

Saw QuickTime for streaming once (Accenture,later videos all Windows Media)

iPod videos (of course) all QuickTime


What Resolution/Data Rate

Should I Use? (11/07 data)

2.9255424155,765477x319Corporate - aggressive (7)

6.78/3.99 *63502/313*76,800322x238Corporate – conservative (11)

Corporate

2.46104519211,641604x349Broadcast – aggressive (4)

3.0862432141,191446x322Broadcast – midrange (13)

6.059639464,536292x219Broadcast – conservative (2)

Broadcast

Bits/pixel/

sec

AudioDataRate

VideoData Rate

(Kbps)TotalPixels

AverageResolution

* Corrected for two sites with off the charts data rates

32


What Resolution/Data Rate

Should I Use? (4/07 data)

2.3632331141,120432x324Corporate - aggressive (5)

3.965230476,800320x240Corporate – conservative (9)

Corporate

1.8756455244,736597x408Broadcast – aggressive (3)

2.9452396136,552443x307Broadcast – midrange (10)

3.756628876,800320x240Broadcast – conservative (3)

Broadcast

Bits/pixel/

sec

AudioDataRate

VideoData Rate

(Kbps)TotalPixels

AverageResolution


Changes from April to

November

+4%*+15%*+8%Corporate

+12%+15%+3%Broadcast

Bits/ pixel/sec

Video DataRate (Kbps)

TotalPixels

* adjusted for AMEX and Visa

33


What Frame Rate?

23%465379Corporate - conservative (3/4)

21%457376Broadcast - midrange (4/9)

Difference30 fps15 fpsAverage Data Rate (Kbps)

Some (perceived) efficiency gained by encoding at 15 fps

Cost is smoothness, most evident in fast moving clips

Try 15 fps to save bandwidth, but assess quality and smoothness


How Much Audio?

Key points:39 kbps for midrange broadcast - is your WMA audio datarate too high?

Allocate roughly twice as much to audio with Flash b/c MP3is comparatively poor encoder

-45%37 Kbps68 KbpsCase study - aggressive (4/3)

-44%47 Kbps83 KbpsCase study - conservative (5/6)

-43%39 Kbps68 KbpsBroadcast - midrange (10/3)

DifferenceOtherFlashAudio Bit Rate

34


Switching to Flash

67%80 kbps48 kbpsGE

100%96 kbps32 kbpsCNN

100%96 kbps48 kbpsPrice Waterhouse

100%96 kbps48 KbpsChevron

100%96 kbps32 kbpsMSNBC

100%96 kbps48 kbpsWeather Channel

Additional

Bitrate

MP3

Bitrate

WMA

bitrateAudio Bit Rate

What question does this slide answer?


What Audio Configuration?

Research (based on 4/7 data)

Windows Media - only format that provides details

Used Windows Media Player and Sliq MediaTechnologies WMSnoop

Two groups

Broadcast

High volume, both cost and quality conscious

Corporate

Lower volume, more quality conscious than cost

35


What Audio Configuration? Bit Rate

(kbps)

Samples

kHz

Mono/

Stereo

CBR/VBR

Entertainment

BusinessWeek 20 44 kHz Mono CBR

CBS 48 44 kHz Stereo CBR

CNN 32 44 kHz Mono CBR

Sports Illustrated 32 44 kHz Mono CBR

Today Show 32 44 kHz Mono CBR

Weather Channel 48 44 kHz Stereo CBR

35 kbps 4M/2S All CBR

Corporate

Deloitte case study 64 44 kHz Stereo VBR

Accenture 96 48 kHz Stereo CBR

Microsoft 48 44 kHz Stereo VBR

HP 64 44 kHz Stereo CBR

68 kbps 4S 2/2


Audio Configuration - Analysis

Entertainment sites balance bandwidth costsand quality

Consistently use mono/relatively low bitrates

Lower stream count corporate sites worryprimarily about quality

Consistently use stereo and higher bitrates

Stereo adds no value to speech or mostmusic, so mono is preferred

Some audio codecs (WMA) default to stereo

36


Conclusions?

Use mono for content that’s primarily speech

Use 16-bit in all cases

Consider 22 kHz for speech

CBR is OK for audio


How Do I Produce the Right

Aspect Ratio?

What's wrong with this picture?

Producing aspect ratio correct video (theshort answer)

Explanation (bonus)

37





Anderson CooperDigital Photo

Frame grabfrom streaming file

(cnn.com)

38


Why Is This So Hard?

Different video formats have different “aspectratios” so are “stretched” for television display

4:3 DV - .9

16:9 DV - 1.2

HDV - 1.33

Computer display is all square pixel (1:1)


How Do I Get it Right?

Always output Square Pixel (1:1)

Irrespective of input PAR

When square not available, “distort”

You want to change aspect ratio from analog to square

Never choose “maintain aspect ratio” unless encodingfrom square pixel source

Easiest (but not essential) when outputresolution matches display aspect ratio

4:3 - 640x480, 480x360, 400x300, 320x240

16:9 - 640x360, 480x270, 320x180

39


Getting it Right (4:3 DV Input)

Compressor

Adobe Media Encoder Episode

Flix Pro

Squeeze


Why “Distort?”

To change aspect ratio from this To this

40


How Do I Optimize

Deinterlacing Quality?

About deinterlacing

Why it's important

Rules of the game


What is Deinterlacing (Take 2)?

Most SD cameras capture interlaced video

60 fields per second, not 30 frames

Fields are captured 1/60th of a second apart

All streaming codecs are frame based

Converting from fields to frames can cause astair step or Venetian blind artifacts whichdeinterlacing can resolve

41


Deinterlacing – Why it's Important


Deinterlacing - Rules of the Game

Always deinterlacewhen producing withinterlaced input

Editors have differentcontrols, but you mayneed to :

Select deinterlacing(and technique), AND

Output a progressivefile

Apple Compressor

42


Deinterlacing - Rules of the Game

Deinterlacing artifacts are easy tospot; once seen and recognized,you can resolve

Not all programs deinterlace equallywell


Maximizing Deinterlacing Quality

Windows Media Encoder, QuickTime Pro,Flash 8 Encoder

Scale beforeencoding

Squeeze, Flix ProOn the bubble

Premiere Pro/After Effects/Flash VideoEncoder 9

Compressor (Better setting)

Very good

Episode

Compressor (Best setting)

Very, verygood

AlgoSuite (www.algolith.com), plug-in forAfter Effects, Digital Fusion and Shake

The best

43


Rating the Players


Conclusion

Plan workflow to maximize deinterlacingquality

If encoding in tool that de-interlaces poorly, scaleand deinterlace in your editor

With trophy footage, pre-process after editing andbefore compression

44


Does Noise Reduction Work?

Many encoders offernoise reductionfeatures

Generally low qualitytools that blur entireframe, potentiallylosing detail

Good for witnessprotection videos


Does Noise Reduction Work?

Best tools are usually third party (andexpensive)

Use adaptive techniques that distinguish betweennoise and true motion

This reduces noise without blurring or otherartifacts

Generally worth a try when image is noisyfrom chroma gain or poor quality source

45


AlgoSuite from Algolith

Concert shot with poor lightingSome noise obvious from back camera

Trophy video, so worth the effort

Again - may have to modify workflow to apply best noisereduction filter

Flash1:45


Editing/Preprocessing -

Conclusion

Any questions?

46


Encoding

Terms and techniques

Working with MPEG-4/H.264/AVC

Landscape

iPod delivery

Working with Flash

Landscape

Encoding tools

Working with Windows Media Video

Landscape

Encoding tools


Terms and Techniques

Constant and variable bit rate encoding

Streaming vs. progressive download

Working with key frames

47


Constant vs Variable Bit Rate

Low Motion Moderate Motion LowMotion

ModerateMotion

HighMotion

300 kbps

800 kbps

500 kbps CBR

VBR


Constant vs Variable Bit Rate

Constant Bit Rate (CBR)

One bit rate applied to entire video,irrespective of content

Pros: Easy and fast

Cons: Doesn’t optimize quality

Variable Bit Rate (VBR)

Dynamic bit rate matches motion in video

Pros: Best quality

Cons: Slow, can produce erratic stream

48


When Should I Use VBR/CBR?

Consider VBR when:

Clips are longer than 60 seconds

Varied motion in clip (some action, some talking head)

Producing for progressive download

Not time constrained

Consider CBR when:

In a hurry (or live encoding)

Producing for streaming

Consistent motion (especially talking head)


How do I Produce the Best

Quality CBR?

Use 2-pass CBR

Scans file (like VBR), but packs data into aconsistent stream

Best of both worlds when available

1-pass of live or draft work

49


How Do I Produce the OptimalVBR File?

2 passes or more

Use “Constrained”Constrains to data rate

Set Target and Max/MinOverall target

Max/Peak bit rate-

Rule of thumb is 1.5 - 2X

Minimum bit rate (not shown)

Useful when low motion videosuffers in quality

Rule of thumb is .5X unless lotsof very low motion


VBR, CBR and Delivery Mode - Streaming vs.

Progressive Download (Review)

StreamingDistributed by "streaming server"

Stream monitored and dished out as needed

Progressive downloadStarts playing before fully downloaded

Stream pushed out as quickly as possible

50


VBR, CBR and Delivery Mode - Streaming vs.

Progressive Download (Review)

When producing for streaming

Determine if any unique requirements for server

Determine if any unique streaming relatedfeatures (multi-bit rate files)

Typically, use CBR for steady stream

When producing for progressive download

Determine if any unique features (fast-start)

Encode using VBR for best quality


Key Frames

Defined

Relevant controls

A note of caution

Key frame controls vary by codec and encoder

Controls in your program probably won't look likethose shown here

General principles should work similarly

51


What are I, B and P Frames?

P - looks backward to I and P frames (predicted)

B - looks forward and backward to previous I and Pframes (Bi-directional interpolated)

No frames refer to B-Frame

I-Frame - encodedwithout reference toother frames (alsocalled Key Frames)


What do I Need to Know About

Key Frames?

Key frames are needed forinteractivity

All playback starts on a key frame

When seeking to a random frame,must start playback at key frame

Key frames help "reset" quality:Useful at scene changes ("Natural"key frames)

Recommended:Make sure "natural" or equivalent isenabled

Go with default duration (5-10seconds)

52


What do I Need to Know About

B Frames?

Typically the most “efficient” frame

Hardest to decode with greatestbuffer requirements

Usually the key difference between“simple” and “advanced” profiles inMPEG and Windows Media codecs

In general:Use B-Frames when producing forcomputer playback

Don’t use when producing for deviceplayback (iPod, iPhone, etc.)

Interval is number of B framesbetween I and P-Frames.

1-2 is recommended setting


Codec Specific Controls

H.264

Flash

Windows Media

53


What Do I Need to Know About

MPEG-4/H.264?

How is MPEG-4 different from H.264?

When should I use each?

Anything special about producing MPEG-4/H.264?

How do I encode MPEG-4?

How do I encode H.264?

Which is the best H.264 Encoder?


How is MPEG-4 different from

AVC/H.264?

Streamcrest Associates

http://www.streamcrest.com/SDF%20Final1.pdf

AVC is the MPEG-4standard’s mostadvanced codec(MPEG-4 part 10)

AVC/H.264 are thesame

Highest currentevolution of MPEG-4encoding

First standard adaptedby ISO and ITU

Supposedly about 50%more efficient thanMPEG-4

54


Should I use MPEG or H.264?

H.264 produces betterquality

Virtually all computers candecode H.264 at this point

Use H.264 for computersand devices that supportH.264 (note Apple presets)

Use MPEG-4 for otherdevices or very oldcomputers


Anything Special About

Producing MPEG-4/H.264?

Quality varies by implementation

Apple is the best I’ve seen

Quality variance is less with other codecs

All Windows Media encoding performed via MScode modules (DLLs)

All Flash VP6 encoding performed via On2 DLLs

So: Choose encoding tool wisely

55


What Else Do I Need to Know

about Producing H.264?

Customize for deliverymode

Hinted streaming forstreaming server

Fast Start - CompressedHeader

Otherwise, must fullydownload before playing

Not all encoding toolssupport Fast Start

Adobe Media Encodermost notable


What is an MP4 file (and how is

it different from MOV)?

H.264 file extensionsMPEG “wrapper” is QuickTimebased, so formats generallyinterchangeable

QuickTime Player will play.MP4 files as well as .MOV

Common extensions are .m4vand .m4a, and .3GP

Extension DefaultsMost streaming files use MOV

Most files produced for devicesuse .MP4 or derivative

Follow the preset and don’tchange unless you’re havingproblems

56


How Do I Produce MPEG-4?

Know Your Profile

Simple Profile for low bit rate and low resolution applications

Advanced Simple Profile for best quality

Conclusion - use Advanced Simple for streaming (calledMPEG-4 Improved in QT Pro)

Unfortunately, presentation varies by application


How Do I Produce H.264/AVC?

Know Your ProfileBaseline Profile - For videoconferencing and mobile app(No B-Frames);

Main Profile/Extended: Extended is preferred, use Mainwhen not available (both have B-frames)

High Profile: For broadcast, HD DVD and Blu-ray

All have multiple levels

57


H264 Comparisons


Which H.264 Encoder is Best?

•Image got soft and blocky with motionGoodProCoder

• Soft appearance in low motion, somefading, very crisp in high motion

GoodSqueeze

• Good in low motion

• Falls apart in high motion

• Some fading

Fair toPoor

Episode Pro

• No Fast Start option; No data rate controlfor audio on streaming file; No CBRstreaming file or 2-pass CBR

• Very good low motion quality, blocky withhigh motion

Lacksfeatures,

quality fair

Adobe MediaEncoder

• The standard by which all others are judgedVery GoodCompressor

CommentsRatingEncoder

58



Flash?

Which codec should I use?

What do I need to know about producingVP6?

What’s the best Flash Encoder?

What’s happening with H.264?


Which Codec Should I Use?

Codecs - CurrentSorenson Spark - only used now for compatibility with olderplayers

VP6 - introduced with Flash 8; two quality levels

One pass - good, but not best quality, especially forchallenging (high motion/low data rate) clips

Two pass - best for challenging footage

Audio - MP3

Flash Player 9H.264/AAC playback support

MOV format support

59


Choosing a Flash Encoder

For maximum quality and flexibility you need:

Encoder with VP6

Encoder with 2-pass VBR

Encoder with quality scaling and deinterlacing

Other desirable features

Batch

Multiple format support


Survey of Flash Encoders

$4,995YesYesYesYesRhozet Carbon Coder

$199YesYesYesYesFlix Exporter (apps below)

$995YesYesYesYesEpisode Pro

$249YesYesYesYes (F only)On2 Flix Pro

$249YesYesYesYesSorenson Squeeze

freeNoYesYesNoPremiere Pro/After Effects

freeNoYesYesYes (F only)CS3 Video Encoder

freeNoYesNo/Yes

Yes (F only)Flash 8/9 Video Encoder

Price2 PassVP6De-intBatchTool

Discreet Media Cleaner, Canopus Pro Coder (VP6 for Flash video only), QuickTime Pro, Adobe After Effects,Apple Final Cut, and Apple Compressor

60


• Needs On2 Flix Exporter, Quality very goodVery GoodProCoder

• The standard by which all others are judgedThe BestFlix Pro

• Noisy backgroundsGoodSqueeze

• Almost identical to Flix ProVery GoodEpisode Pro

• 1-Pass CBR only, OK for draft and lowmotion files

FairAdobe MediaEncoder

• Need Flix Exporter,

• Quality identical to Flix Pro output

Very goodCompressor


Which Flash Encoder is Best?


Adobe Flash Announcement

Why H.264? Infrastructure/EcosystemNative graphics card support

Hardware based H.264 encode

Standard is widely adapted

AAC audio better quality than MP3

Will not support Alpha channel

When will it matter?Flash has traditionally fast adoption cycle

Start considering now

61



Windows Media?

How do I get the latest codecs?

What’s VC-1 and how does it relate to WMV9?

What are the relevant Windows Media profiles (anddo I care)?

Which is the best WMV encoder?

How can I “tweak” Windows Media video files formaximum quality?

What are Multiple Bitrate Files (and do I care)?

What’s Silverlight (and do I care)?


How Do I Get the Latest Codecs

Microsoft (quietly) released updated codecsinstalled with either:

Media Player 11

The Windows Media Format SDK 11

(http://msdn2.microsoft.com/en-us/library/aa387410.aspx)

Does it matter?

Yes - codec quality has noticeably (if notsignificantly) improved

62


What’s VC1 and How Does it Relate to

WMV9?

From the mouth of Ben Waggoner, MSFT

“VC-1 is SMPTE’s version of WMV 9. Windows MediaVideo 9 is Microsoft's brand for our VC-1 implementation.”

“Basically, just think of “WMV 9” and “VC-1” in a WMV fileas synonymous.”

After installing WMF SDK 11

Windows Media Encoder still calls the codecs WMV 9

Silverlight Expression Encoder calls the same codecs VC-1

Both create WMV files


What are the Windows Media Profiles?

Now two profilesMain - backwards compatible, no codec update required

Advanced Profile

Supports interlaced video (for VC1)

Not backwards compatible, codec update (not playerupdate) required

Problem for locked down corporate environments

Problem for off-line viewing

When should I consider Advanced Profile? (hold thatthought)

63


Windows Media Comparisons


ProCoder

• Slightly better than Windows Media EncoderVery GoodSqueeze

• Fair quality low motion, drops lots of framesin high motion and poor quality

PoorEpisode Pro

• Very close to Windows Media EncoderVery GoodAdobe MediaEncoder

• Need Telestream Plug-in, quality very goodVery goodCompressor


Which WMV Encoder is Best?

64


How Can I Tweak WMV Files

for Maximum Quality?

Both profiles have advanced encodingparameters that you can access:

Via command line

Manual registry changes

Registry changes via the “PowerToy” tool

Does tweaking improve quality?Yes, in some well defined cases, but tough to finda “one-size fits all video” tweak

Most effective when repeatedly producing similarvideos


What’s the PowerToy?

65


More on Tweaking

If you don’t tweak, the Main and AdvancedProfiles produce equivalent quality

Some critical tweaks are only supported inthe Advanced Profile, so if you’re going totweak, use the Advanced Profile


Tweaking Resources

ArticlesMy review of PowerToy and SDK 11http://www.streamingmedia.com/r/printerfriendly.asp?id=9659

Ben Waggoner’s article on tweakinghttp://www.streamingmedia.com/r/printerfriendly.asp?id=9456

DownloadsPowerToy - http://www.citizeninsomniac.com/WMV/

Windows Media Format SDK 11 http://msdn2.microsoft.com/en-us/library/aa387410.aspx

Waggoner SME Presentation (page 24-42)www.streamingmedia.com/east/SMEast2007-Waggoner-Webcasting.pdf

66


Should I Produce Multiple Bitrate

Files?

What are they? Single video file with multiplestreams optimized for different connection speeds

Streaming server requiredServer sends optimal stream for connection speed

Server monitors connection and adjusts stream ifpackets are lost or frames dropped


Should I use MBR?

Mixed use in Broadcast sites (limited sample)

Yes

CBS

Weather Channel

No

BusinessWeek

CNN

Sports Illustrated

Today Show

67


Usage Details from CBS

16 kbps20 kbps32 kbpsBitrate

Audio

127 kbps281 kbps570 kbpsData rate

121530Frame rate

480x360480x360480x360Resolution

Video

Stream 3Stream 2Stream 1


Multiple Bitrate Analysis

Value of feature seems to have waned withprevalence of broadband

No one cares about 56 kbps anymore

The success of Flash, which doesn’t offer MBR,seems to indicate that it’s not a USP for WMV

Still, if you have the infrastructure, it’s a niceconvenience feature

68


What is Silverlight (and Do I

Care?)

What is Silverlight?

What are Silverlight’s limitations?

Should I consider using Silverlight?


What is Silverlight?

It’s a playerCool, customizable player

Plays on Windows and Mac with relatively smalldownload

Same WMV 9 codecs at similar quality

It’s a platformRicher design environment with vector-basedgraphics, media, text, animation, and overlays

Flexible programming model that supports AJAX,Microsoft Visual Basic .NET, C#, Python, and Rubyand integrates with existing Web applications

• http://www.microsoft.com/silverlight/faq.aspx

69


Silverlight Player

Chapter marks

Captions

CustomizableSkins

Player Controls(hidden)


What are Silverlight’s Key

Limitations?

No Digital Rights Management

No multi-cast

No server side playlists

Limited support for multiple bitrate files

70


Should I consider Silverlight?

As a player? - Yes

Especially for casual producers of WindowsMedia files, if you can live with the limitations

As a platform? Beyond the scope

As an encoding tool for Windows MediaVideo files?

Expression Encoder can’t produce MBR files, butvery nice visual tool with excellent live features


How do I Try Silverlight?

Download Expression Encoder

Download Expression Blend 2 to customizethe player

Both at :http://www.microsoft.com/expression/products/download.aspx?key=encoder

71


Questions?


Bonus Topics (time permitting)

Producing a soft background

Setting up 3-point and Flat lighting

Assessing exposure on the set

Interlaced vs. progressive details

72


Producing a Soft Background

Why?

Classy filmic look

Heightens attentionon subject

Makes video easierto compress



Preliminaries

The larger the CCD, the easierthis is to do

Not all camcorders can do thisin all situations

Need largest aperture settingpossible (lowest f-stop)

Control exposure manually

Control lighting with ND filters,higher shutter speed or bymoving light further from thesubject

73



Procedure

1. Subject max distance frombackground

2. Camera max distance fromsubject

3. Use zoom for framing

4. Open aperture to maxsetting (lowest f-stop)


Setting up 3-Point Lighting

Key-major light sourceStrongest light

45oangle from camera, pointingdown at 45o

Fill-moderates shadows50% power of key light

45o angle from camera, pointingdown at 45o

Back light (or rim) createscontrast with background

"hard" light

Shining down from back onhead and shoulders

74


Perfecting the Key Light

PositioningLateral - nose caret shouldn’t extendbeyond nose/mouth crease

Vertically - shouldn’t extend into lips

Watch shadows beneath chinLower lights to minimize or usereflector

Watch for hot spots on face

Watch for shadow behindsubject

If not facing camera, place keylight opposite camera


Perfecting the Fill Light

KarmaShouldn’t appear as separate light, but shouldminimize key light

Laterally - start 90 degrees opposite key, then adjust totaste

Toggle on/off to ensure no hotspots

75


Perfecting the Back Light

All about contrast

Make sure stand/cords not in frame

Turn off key/fill to make sure back light isn’t creatinghot spots on the face

Check whether it shines on the background (OK ifthat’s what you want)


Producing Flat Lighting

Two approaches

Dual Key

Single Key

Both use back light

76


Flat Lighting - Dual Key

Two keys and a back lightTypically, 2 soft lights

45o angle from camera,pointing down at 45o

Back light-create contrast


Setup issues:No shadows on face

Minimize chin/nose shadows

Watch location of back shadows

Typical back light issues


Flat Lighting - Single Key

Single key above the camera

Typically, a soft light

As low as talent can stand it (tominimize chin/nose shadows)

Back light-create contrast


Key setup issues:No hotspots on face

Watch chin and noseshadows

Watch back shadows

Typical back light issues

77


Assessing Exposure at the Shoot

Rule #1: Some sets (with too much contrast) arechallenging to light for DV - get the set right first

Rule #2: It's better to be under exposed than overexposed

Rule #3: Zebra stripes are your best friend

Rule #4: Scopes are your other best friend

Rule #5: When in doubt, check auto


Better to be Under than Over

If overexposed, detail is lost forever

If underexposed, can boost brightness in post andimprove the result

Key point: make sure "gain" is disabled in thecamera

Gain creates "noise" that becomes exaggerated whenvideo is brightened

If underexposed and gain is enabled, quality suffersnoticeably

78


Waveform and IRE Levels

Brightness measured onIRE scale

0 or 7.5 - black

100 - white

This is broadcast "legal"

Waveforms shows levels ofsections of frame

White shirt - 100

Her face - 75-80

100 IRE

7.5 IRE


Zebras are Your Friend

Zebra stripes show areas that exceed 100 IRE

Above that, detail is lost

Adjust iris (f-stop) until zebra stripes are almost all gone

What about the face?

F1.7 F2.4 F2.8

79


More on Zebras

Generally configurable

Either 70/100 (VX2000)

Totally configurable

Common settings

100 - pure white

95 - white with a safe zone

70-80 - Caucasian face


3 Ways for On-Site YC Waveform

External Waveform Monitor ($1,000 - $4,000)

Adobe On Location on a notebook or desktop (shown)

Your video editor on a notebook or desktop

Capture video and apply monitor

Image from previous page from Premiere Pro

80


When in Doubt, Check Auto

Shoot using manual controls over gain,shutter speed and iris for reasons discussed

Once you've reached optimal settings

Switch into automatic mode

Investigate significant differences

Caution

Auto lights the entire frame, you care primarilyabout the face


Interlaced vs. Progressive

Testing

81


Progressive vs. Interlaced

Overview

The big questions

Test description

Scenarios

Conclusion


Overview - Theory

Traditional camcorders areinterlaced (shoot in fields, 1/60of a second apart)

Streaming media is FRAMEbased

To produce streaming videofrom interlaced source requiresthat you combine two fields

82


Overview - Theory

Shooting in progressivemode eliminates thatproblem and is vastlysuperior to interlaced

But!

Most editors/compressionprograms havedeinterlacing filters thattake you frame here:

To here.


The Big Questions

Does a progressive camcorder deliver better quality thaninterlaced?

If so, how much?

If so, when does it matter most?

Implications:

If you're buying new, should you buy a progressive unit?

If your camcorder is interlaced, should you buy a new one forstreaming?

If your camcorder has progressive and interlaced, which modeshould you use?

83


Test Description

Shoot same scenes with two camerassimultaneously

One progressive - Canon XH A1

One interlaced - Sony HDR-FX1

ScenesHigh speed sports

Low contrast talking head

Real world (uncontrolled) shoot


Conclusions - Sports

Even atrelatively slowshutter speeds(like 60)

If you havesharp edges,you have aproblem

84


Conclusion - Talking Head

In those instances where:Minimum sharp detail

Relatively slow shutter (like 60)

Not much distinguishable difference


Conclusion - Talking Head

Even when there's moderation motion

85


Conclusion - Real World

If you can't control the detail in the scene, goprogressive


Conclusions

Progressive delivers noticeably better quality in allbut the most controlled shoots

Especially with

Sharp edges, high motion and fast shutter speed

So:

If you're buying for streaming, buy progressive

If you have progressive, use it

If your camcorder doesn't have progressive, and you shootlots of streaming, consider new unit