Jeff SchertzMicrosoft Solutions Architect, PolycomLync MVP

Video - What in the World are You Doing to My Network?

NETW305

AgendaFoundational Conceptsfurther demystifying video conferencing in Lync 2013

Dissecting the Video Experiencelayouts, resolutions, and real estate

Doing the Mathbit rates and frame rates

Video Actualsuser behavior drives bandwidth usage

Foundational Concepts

Video Codecs in Lync 2013Real-Time Video (RTV)Peer-to-Peer Calls• High definition (720p) supported when both parties have four

physical CPU cores• No support for hardware acceleration

Multiparty Conference Calls• Standard definition (VGA 640x480) maximum with AVMCU

Panorama Video• Supported on either peer or conference calls with single resolution

New Resolutions• Additional widescreen resolutions provided for CIF (424x240) and

VGA (640x360)

Video Codecs in Lync 2013H.264 Scalable Video Coding (SVC)Hardware Acceleration• DirectX 9.0 compatible graphics chipset

decoding• Compatible Intel HD graphics chipset

encoding• Windows 8 w/ USB Video Class (UVC) 1.5

compatible camera encoding

Additional Resolutions• More 4:3 and 16:9 resolutions across entire

range including 1080p• Multiple Panorama resolutions for next

generation RoundTable devices

Video Codecs in Lync 2013H.264 Scalable Video Coding (SVC)Temporal Scaling (UCConfig Mode 1)• Provides multiple frame rates in single encoded stream

Receiving Streams• From 1 to 5 streams of proportional resolutions in Gallery view• Additional stream for panorama video from RoundTable/CX5x00

Sending Streams• Single stream per requested resolution for multiple rates

(7.5/15/30fps)• Maximum of 5 outbound streams per video source• Can vary based on requested resolutions, codecs, and encoder

capabilities

Client 4

Client 2

Client 3

7.5 fps

15 fps

Client 1

30 fps

15 fps

A single SVC encoded video stream provides multiple frame rate layersEncoder sends only the highest requested rate

SVC Multistream CapabilitiesMultiple Frame Rates

30 fps

Client 4

Client 2

Client 3

180p

360p

Client 1

720p

360p

Multiple encoded video streams required for each provided resolutionEach stream is independent and includes entire video payload

180p

720p

SVC Multistream CapabilitiesMultiple Resolutions

Multistream CapabilitiesMultiple Codecs and Video SourcesBy DesignMaximum of 5 possible concurrent outbound streams per video source• Main Video Source = 5 SVC or 4 SVC + 1 RTV• Panorama Video Source = 5 SVC or 4 SVC + 1 RTV

In RealityThat many outbound streams is highly unlikely

• Limited Panorama resolutions available• Largely disparate bit rates can also trigger additional streams

Calculated via a combination of factors determined by the sender• Hardware encoding capabilities (CPU, GPU, camera) and available uplink

bandwidth• Number of receiving clients requesting a specific video stream

Client 2

180p 15fps

360p 30fps

Client 4

Client 5

Client 1

720p 30fps

360p 30fps

180p 15fps

720p 30fps

SVC Multistream CapabilitiesPutting it all togetherEncoding endpoint has the final decision on how many streams to send

Client 3360p 15fps

Multiparty Video Conference

Lync 2013 Web App

Lync 2013

Lync 2013Lync 2013 Mobile App

Lync 2013 Windows App

Lync 2010

ActiveSpeaker

SVC

RTV

Main Camera

Panorama

MaxReceive Streams

Details

Lync 2013Lync Web App 6 Gallery (5)

Panorama (1)

Lync Windows App 4 Gallery (4)

Lync Mobile 1 Active Speaker

Lync 2010 2 Speaker (1)Panorama (1)

RoundTable

Dissecting the Video Experience

Lync 2013 Video ExperienceConferencing Views

Gallery View

Speaker View

Video Spotlight

Compact View

Lync 2013 Video ExperienceClient ComparisonLync 2013 and Lync Web App 2013• Up to 5 inbound video streams• Any can be in either square or widescreen• Support for RoundTable/CX5000 Panorama

Lync 2013 Windows App• Up to 4 inbound video streams• Gallery view is always cropped; ignores overrides• Native resolution available in speaker view

Lync 2013 Mobile App• No gallery - limited to single stream active-speaker view

Lync 2013 Video ExperienceSmart FramingFacial detection and coordinates sent in RTP streamNative aspect ratio displayed in peer-to-peer callsVideo cropped by default on multi-party conference calls

• Full resolution used only in Spotlight View, not Speaker View• Some scenarios automatically override cropping (e.g. LRS, CX5100)

Cropped Smart Framing Manual Override

Lync 2013 Video ExperienceCroppingVideo is encoded and sent in full capture resolution (e.g. 16:9)

There is no square resolution (1:1)

Behavior can be validated by comparing bandwidth of cropped versus uncropped

Video ResolutionIt’s all about the real estateActual resolution sent may be lower or higher than the ideal matchVideo competes with shared content, attendee list, chat window, etc.

360p 360p 360p

360p 360p

Video ResolutionMeetings are more than just videoVideo competes with content, attendee list, conversation window, etc.Large Meetings of 75+ attendees force Speaker View

Video Resolution

Participants

MaximumResolution

*Width Height Pixels Streams Pixels Screen

Usage

2 1080p 1060 1060 1,123,600 x1 1,123,600 54%

3 720p 700 700 490,000 x2 980,000 47%

4 540p 500 500 250,000 x3 750,000 36%

5 360p 400 400 160,000 x4 640,000 31%

6 360p 320 320 102,400 x5 512,000 25%

More is less

1080p

720p 720p540p 540p 540p

360p 360p 360p 360p

Video Resolution1080p is 1080pVideo resolutions are the same pixel depth regardless of the display’s physical size - thus bandwidth is the same

84” Perceptive Pixel Monitor• 1920 x 1080 = 2,073,600

pixels

5” Nokia Lumia 929 Phone• 1920 x 1080 = 2,073,600

pixels

Unique ExperiencesDual MonitorsStandard Lync 2013 Client• Maximized video Gallery on single

screen• Can manually be dragged across both

monitors but this is clunky and rarely used

Lync Room System• Default Gallery view is also single

screen• Includes “Video Only view” option to

span video Gallery view across both monitors

Bandwidth Impact• Dedicating displays to video can

trigger requests for higher resolutions

Unique ExperiencesPanorama VideoRoundTable/CX5000• Limited to low resolution and

frame rate

New CX5100/CX5500• Support for multiple, higher

resolutions and frame rates

Different Behavior• Local preview is not mirrored like the main video source• Panorama is active-speaker experience

Doing the Math

Video Bandwidth in Lync 2013Don’t Forget about Audio

• Bandwidth calculations should include entire RTP payload (audio + video) in addition to the RTCP bit rate

• Add 5Kbps per codec for RTCP payload

Payload Bitrates

Audio codec Scenarios Audio +IP Header+UDP RTP

SRTP+FEC

RTAudio Wideband

Peer-to-peer 29.0 Kbps 45.0 Kbps 57.0 Kbps 86.0 Kbps

RTAudio Narrowband

Peer-to-peerPSTN

11.8 Kbps 27.8 Kbps 39.8 Kbps 51.6 Kbps

G.722 Conferencing 64.0 Kbps 80.0 Kbps 95.6 Kbps 159.6 Kbps

G.722 StereoPeer-to-peerConferencing

128.0 Kbps 144.0 Kbps 159.6 Kbps 223.6 Kbps

G.711 PSTN 64.0 Kbps 80.0 Kbps 92.0 Kbps 156.0 Kbps

Siren Conferencing 46.0 Kbps 32.0 Kbps 47.6 Kbps 63.6 Kbps

MediaRTCP MaxBit Rate

Audio 5 Kbps

Video (Single Codec)

10 Kbps

Video (H.264 + RTV) 15 Kbps

Video Bandwidth in Lync 2013Video Resolutions & Bitrates

• Data from the Lync Bandwidth Calculator

• Averages are calculated on 80% of Maximum*

• Forward Error Correction (FEC) overhead is included in the RTP payload

• Add 5Kbps per codec for RTCP payload

Video codec Resolution

(Aspect Ratio)Average Bit Rate*

MaximumBit Rate

Minimum Bit Rate

MaximumFrame Rate

Low

H.264320x180 (16:9)212x160 (4:3)

200 Kbps 250 Kbps 15 Kbps 15 fps

H.264 / RTV424x240 (16:9)320x240 (4:3)

280 Kbps 350 Kbps 100 Kbps 15 fps

H.264480x270 (16:9)424x320 (4:3)

360 Kbps 450 Kbps 200 Kbps 15 fps

Standard

H.264 / RTV640x360 (16:9)640x480 (4:3)

640 Kbps 800 Kbps 300 Kbps 30 fps

H.264 848x480 (16:9) 1,200 Kbps 1,500 Kbps 400 Kbps 30 fps

H.264 960x540 (16:9) 1,600 Kbps 2,000 Kbps 500 Kbps 30 fps

HDH.264 / RTV 1280x720 (16:9) 2,000 Kbps 2,500 Kbps 700 Kbps 30 fps

H.2641920x1080

(16:9)3,200 Kbps 4,000 Kbps 500 Kbps 30 fps

Panorama

H.264 / RTV 960x144 (20:3) 400 Kbps 500 Kbps 15 Kbps 30 fps

H.264 1280x192 (20:3) 800 Kbps 1,000 Kbps 250 Kbps 30 fps

H.264 1920x288 (20:3) 1,600 Kbps 2,000 Kbps 500 Kbps 30 fps

Video Bandwidth in Lync 2013Capacity Planning

DirectionTypical

Stream Bit Rate

MaximumStream Bit Rate

Main Video

Send 270 Kbps 8015 Kbps

Receive 260 Kbps 8015 Kbps

Panorama

Send 190 Kbps 2010 Kbps

Receive 190 Kbps 2515 Kbps

CalleeTypical

Stream Bit Rate

MaximumStream Bit Rate

Main Video

H.264 460 Kbps 4010 Kbps

RTV 460 Kbps 2510 Kbps

Panorama

H.264 190 Kbps 2010 Kbps

RTV 190 Kbps 510 Kbps

Peer-to-Peer Sessions Conference Sessions

Bandwidth BehaviorPeer to Peer Calling Scenarios

• Default video window size is low resolution• Manual resizing of the window is required to trigger higher

resolutionsReceiver View Full Screen Maximized Resized Resized Default

Layout

Resolution 1920 x 1080 1280 x 720 960 x 540 640 x 360 320 x 240

Frame Rate 27.1 fps 29.6 fps 28.9 fps 29.7 fps 15.0 fps

Average Bit Rate 2888 Kbps 1727 Kbps 504 Kbps 596 Kbps 115 Kbps

Maximum Bit Rate 4415 Kbps 2768 Kbps 1275 Kbps 814 Kbps 499 Kbps

Bandwidth BehaviorBuilding out a Multiparty Conference

• Bandwidth utilization initially increases as participants are added

• Then decreases as resolutions of individual streams drop significantlyParticipants 2 3 4 5 6

Layout

Max ResolutionsReceived 1920 x 1080 1280 x 720 640x 360

640 x 360320 x 240

640 x 360320 x 240

Total AverageBit Rate 2128 Kbps 4050Kbps 1304Kbps 1224 Kbps 1565Kbps

Total MaximumBitRate 4063 Kbps 5890 Kbps 2860 Kbps 2699Kbps 3017 Kbps

Bandwidth BehaviorMultiparty Conference Individual Results

5 Participants

Video Stream 1 2 3 4 Totals

Resolution: 320x240 640x360 640x360 640x360

Inbound frame rate 12.3636 17.9455 28.0020 18.6618

Outbound frame rate 12.3973 17.9293 29.1303 19.1375

Avg. allocated bandwidth 325 Kbps 788 Kbps 788 Kbps 785 Kbps 2686 Kbps

Avg. bit rate: 63 Kbps 504 Kbps 403 Kbps 254 Kbps 1224 Kbps

Max. bit rate: 224 Kbps 989 Kbps 790 Kbps 696 Kbps 2699 Kbps

6 Participants

Video Stream 1 2 3 4 5 Totals

Resolution: 320x240 640x360 640x360 640x360 320x240

Inbound frame rate 14.9283 27.9690 14.9921 13.5170 14.6592

Outbound frame rate 14.8420 29.1817 14.9636 13.7433 14.6406

Avg. allocated bandwidth 342 Kbps 768 Kbps 787 Kbps 787 Kbps 346 Kbps 3030 Kbps

Avg. bit rate: 56 Kbps 455 Kbps 747 Kbps 152 Kbps 155 Kbps 1565 Kbps

Max. bit rate: 185 Kbps 729 Kbps 1003 Kbps 534 Kbps 566 Kbps 3017 Kbps

2 Participants

Video Stream 1

Resolution: 1920x1080

Inbound frame rate 27.1408

Outbound frame rate 29.8927

Avg. allocated bandwidth 3875 Kbps

Avg. bit rate: 2128 Kbps

Max. bit rate: 4063 Kbps

3 Participants

Video Stream 1 2 Totals

Resolution: 1280x720 1280x720

Inbound frame rate 28.3997 28.4843

Outbound frame rate 29.8783 28.4528

Avg. allocated bandwidth 2481 Kbps 2483 Kbps 4964 Kbps

Avg. bit rate: 2001 Kbps 2049 Kbps 4050 Kbps

Max. bit rate: 2894 Kbps 2996 Kbps 5890 Kbps

4 Participants

Video Stream 1 2 3 Totals

Resolution: 640x360 640x360 640x360

Inbound frame rate 29.0182 29.7247 13.0722

Outbound frame rate 29.7953 29.7133 13.2762

Avg. allocated bandwidth 795 Kbps 796 Kbps 794 Kbps 2385 Kbps

Avg. bit rate: 631 Kbps 449 Kbps 224 Kbps 1304 Kbps

Max. bit rate: 942 Kbps 1008 Kbps 910 Kbps 2860 Kbps

• Detailed Media Quality Report statistics from the same conference call

Controlling BandwidthConferencing PolicyGet-CsConferencingPolicy | fl *video*• Controls both peer and conference video sessions• Default policy compared to a customized policy

Controlling BandwidthDisabling Video or Limiting Legacy ResolutionAllowIPVideo

• Controls allowance of video in assigned user’s own conferences

EnableP2PVideo• Controls availability of video for assigned user on any peer-to-peer

session

MaxVideoConferenceResolution• Legacy clients only (RTV)

MaxVideoRateAllowed• Legacy clients only (RTV)• Get-CsMediaConfiguration

Controlling BandwidthDisabling Gallery ViewAllowMultiView• Controls Gallery view capability on the assigned user’s own

conferences

EnableMultiViewJoin• Used to disable Gallery view for assigned users when joining other’s

conferences

Controlling BandwidthLimiting Send and Receive Bit RatesVideoBitRateKb• Limit of video sent by a single client• Default value of 50Mbps

TotalReceiveVideoBitRateKb• Limit of the combined total of all received video streams• Also default value of 50Mbps• Must be at least 420Kbps to support full Gallery view experience

Both parameter values are measured individually per video source

Video Actuals

Video Usage at MicrosoftPeer and Conferencing Video Usage in 2013

1/1/

2013

1/19

/201

3

2/6/

2013

2/24

/201

3

3/14

/201

3

4/1/

2013

4/19

/201

3

5/7/

2013

5/25

/201

3

6/12

/201

3

6/30

/201

3

7/18

/201

3

8/5/

2013

8/23

/201

3

9/10

/201

3

9/28

/201

3

10/1

6/20

13

11/3

/201

3

11/2

1/20

13

12/9

/201

3

12/2

7/20

13

1/14

/201

4

0

5000

10000

15000

20000

Peer to Peer Calls

1/1/

2013

1/19

/201

3

2/6/

2013

2/24

/201

3

3/14

/201

3

4/1/

2013

4/19

/201

3

5/7/

2013

5/25

/201

3

6/12

/201

3

6/30

/201

3

7/18

/201

3

8/5/

2013

8/23

/201

3

9/10

/201

3

9/28

/201

3

10/1

6/20

13

11/3

/201

3

11/2

1/20

13

12/9

/201

3

12/2

7/20

13

1/14

/201

4

0

100000

200000

300000

400000

500000

600000

700000

800000

Conference Calls

Min

ut

es

Min

ut

es

Daily Averages• 6,000 minutes of peer to

peer• 226,000 minutes of

conferencing

• 11 million minutes of video in November 2013

Video Usage at MicrosoftUsers tend to keep default resolutions

424x240

960x144

320x180

320x240

640x360

212x160

1280x720

352x288

640x480

960x540

0

1000

0

2000

0

3000

0

4000

0

5000

0

6000

0

7000

0

Stream Resolution Distribution – Top 10

LowResolution

MediumResolution

High Resolution

Peer 71.4 % 15.0 % 13.6%

Conference 89.6 % 8.4 % 2.1 %

*Pano144p is 18.4%

Video Usage at MicrosoftBandwidth usage similar to Lync 2010Default Configuration Used• No bandwidth or CAC policy constraints in place on video• Conferencing video bandwidth is similar to Lync 2010

Average Bandwidth Average Total Streams

Send + Receive Receive Send Receive Send

Peer to Peer 794 Kbps 390 Kbps 404 Kbps 1 1

Conference 529 Kbps 211 Kbps 318 Kbps 1.41 1.33

SummaryFactors for growth in video usage

Age Ubiquity ExperienceCulture

SummaryImportance of the video modality

VideoAudio Content

Questions?

ReferencesNetwork Bandwidth Requirements for Media Traffic• http://technet.microsoft.com/en-us/library/jj688118.aspx

Understanding Lync Video Quality Reports• http://

blog.schertz.name/2013/06/understanding-lync-video-quality-reports

HD Video in Lync 2013• http://blog.schertz.name/2013/05/hd-video-lync-2013

2.2.6 Cropping Info SEI Message• http://msdn.microsoft.com/en-us/library/hh623747.aspx

Lync 2010 and 2013 Bandwidth Calculator• http://www.microsoft.com/en-us/download/details.aspx?id=19011

© 2014 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.