Pre-Workshop Tutorial HD Transport, Timing and …itea.org/images/pdf/conferences/2016 TIW/Proceedings...May 3, 2016 ITEA 20th Test Instrumentation Workshop Sheet 1 Pre-Workshop Tutorial

May 3, 2016 ITEA 20th Test Instrumentation Workshop Sheet 1

Pre-Workshop TutorialHD Transport, Timing and Compression

Paul HightowerCEO

Instrumentation Technology Systemswww.ITSamerica.com


HD Video

Benefits Order of magnitude improvement in

imagery Wider color gamut Recording and reproduction can be

identical to raw video Single Frames approach snap shot

quality Metadata space built in to the SMPTE

frames

New Concerns Resolution is a more complex subject More formats High transport data rates Large volumes of data for archive Alias artifacts in images


Translating between NTSC (analog) & HD-SDI (digital) Video

NTSC video -color bar test pattern



At the SDI source At the end of a 100 meter cable


HD-SDI Imagery Chain

ITU-R-BT-xxxImage sampling

standards709 includes 4:2:2

Sampling SMPTE 291HD Metadata

Standard SMPTE 292/424HD-SDI Serialization Standards (720p & 1080i/1080p-3G

SMPTE 296/274HD Format Standards

(720/1080)



6

Attribute Analog Video SDI Digital Video

Visible Scan Lines NTSC 480/framePAL 576/frame

480 SD576 (PAL) SD720 HD (750 total/frame)1080 HD (1125 total/frame)

Resolution/LineThis depends on the source and signal quality but ranges to the equivalent of 300 to 720 pixels

SD 720 pixelsHD 1280 (1650 total/line) pixels and 1920 pixels ( 2200 total/line) pixels

Color SamplingContinuous time domain signal,intensity swings are limited by the available 1.5 MHz bandwidth

YUV encoding samples intensity every pixel and color differently depending on the encoding chosen. 4:2:2 is most frequently usedSample rate = 74MHz (720p/1080i)

Color BW ≈ 10 MHzSample rate = 148MHz (1080p)

Color BW ≈ 20 MHz



7

Attribute Analog Video SDI Digital Video

Raw Video Complex AM, FM and phase modulated signal requiring 6 MHz bandwidth

Serial encoded bit stream at bit rates from 270 Mbits/s to 3000 Mbits/sec (12 GB for 4K)

Sync Pedestal and color burst sync areas scaled generally below the black level

A reserved bit patterns (SAV & EAV) defined by SMPTE in the image frame

Blanking A predetermined voltage level in the video signal

ANC space between EAV & SAV is Horizontal“blanking” data space, ANC space between SAV and EAV from first line to line 40 (1080) or 25 (720) is Vertical “blanking” data space.

Active Video An AM signal with overlaid phase modulatedcolor information

A stream of video samples between SAV & EAV the number of which and format varies with resolution, sampling scheme and color depth

Frame/FieldRate

RS 170 60Hz /30 Hz Field/Frame RS170A (NTSC) 59.94 (60/1.001)CCIR 50Hz /25Hz field/frameProgressive and Interlaced

Many from 24.975 to 60 Hz and beyondProgressive, segmented and Interlaced


Quick Idea About Subsampling

4:2:2 subsampling causes two luma samples to share one pair (Cr and Cb) of color samples

Graphic from “Chrominance Subsampling in Digital Images”, by Douglas Kerr


Video Data

4:2:2 10 Bit

Each Y sample represents a pixel > 1920 pixels on a line=1920 Y samples Each Y sample is coupled to ½ of a color sample as shown Each Pixel then is comprised of 1 Y and ½ Chroma @ 10-Bits each Each pixel is 20 bits as a way of thinking about it A 1920x1080 frame = 20x1920x1080 bits =41,472,000 bits or 5,184,000 bytes (image only) Add in the EAV, SAV, Line Count, CRC, HANC and VANC space; Frame = 2200 S x 1125 L

• 2200 samples x 1125 lines = 20x2200x1125/8=6,187,500 bytes/frame• At 60 FPS; Data rate = 371,250,000 bytes/second• Archive 1 minute of video = 22,275 Mbytes

Frame Rate only affects Data Rate

Cr0-1 Y0 Cb0-1 Y1 Cr2-3 Y2 Cb2-3 Y3

Sample Pair Sample Pair Sample Pair Sample Pair

Pixel 0 Pixel 1 Pixel 2 Pixel 3

Color Sample Color Sample

Active Video Data Stream

Cb = blue color valueCr= red color valueY = luma valueLN= line countANC is ancillary data spaceCA = color channel Y = Luma channelAncillary data is placed in Luma channel first until full


Image Frame Organization

The Samples build a frame 720 Frame Parameters Frame size = [750 lines x 1650 samples x 20 bits/pixel ]/(8 bits/byte) = 3,093,750 bytes/frame

VANC Space 25 Lines * 1280 samples = 32KCinema & broadcast generally only use lines 14-15 for scene switch & closed captioning

HANC Space370 Samples x

750

>2000 data itemsExample Packets• 16 channel

audio snippets

• Film Codes• Payload data• Workflow

data• Copyright

data• V-chip data• Billing data• Logging Info

Buffer Space Lines 746‐750

EAV

SAV

Line

Cou

nt

CRC

HANC

EAV

1280 luma samples

1650 luma samples


SDI = Frame Data Serialization

When serialized 720p/60 =3,093,750 bytes/frame * 60 fps =185,625,000 bytes/second 185,625,000 bytes/second *8 = 1,485,000,000 bps.

4:2:2 Multiplexed Encoded SDI Video StreamByte level stack up

SDI Serial Stream IAW SMPTE 292 or 454


Imagery

Movies Frame Rate (pictures per second)

• Set to give the illusion of smooth motion; beyond persistence of vision frequency.

• Rates above 16 images/second yield smooth motion• 24 fps is used in movies• 25/30 fps used in TV (PAL/NTSC)

Illumination Rate (flashes per second)• Flicker fusion is the frequency that pulsing light looks steady• Illumination rate is pushed high enough to achieve flicker fusion• Film generally uses 48 Hz flicker rate• TV targets 60 Hz flicker rate

Interlace TV scanning is 2x the frame rate; Odd lines then even lines Progressive 30 FPS TV displays the same frame twice @ 60 Hz Progressive 60 FPS displays each frame @ 1/60th of a second


NTSC

National Television System Committee (NTSC) Did set the standards for analog TV format Color upgrade for TV in the 50s required B&W TVs to receive Color Signal 60Hz scan rate slowed to make room for color burst signal = 59.94 flicker 29.97 frame rate

Digital Video on modern displays don’t need this Legacy rate is predominant and remains


Imagery


Imagery

Resolution is 3 dimensional Line Count Pixel Count/Line Bit Depth/Pixel

Line

s

Pixels


Imagery

More Bits = More Shades = More detail2

4

8


Imagery

2 bits = 4 colors

4 bits = 16 colors

8 bits = 256 colors

24 bits = 16 million colors

Pictures from http://en.wikipedia.org/wiki/Color_depth

According to http://en.wikipedia.org/wiki/Color, humans can distinguish up to 10 million colors

17

All pictures have the same number of pixels


Imagery

Rendering Effects of Pixels and Bit Depth Same Image

18

Few pixelsMany shades

Many pixelsFew shades

Many pixelsMany shades


Imagery

Sampling RGB; the primary colors Intensity; luminescence, (luma), the black & white information How many samples per image? (pixel count) How many shades per sample of each color? (bit depth)

Many Bits/Color

One Bit/Color


Imagery

4:2:2 Subsampling is not new Analog Color TV has a color content bandwidth of 1.5 MHz, BW info is 3.0 Mhz. Human physiology

• More sensitive to changes in intensity than changes in color

4:4:4 Sampling Equivalent to RGB In motion imagery, 4:2:2 sampling is not discernable from 4:4:4

Bit/Pixel Benefit 4:4:4 @ 8 bits/color =24 bits/ pixel 4:2:2 @ 10 bits + ½ 20 bit color sample = 20 bits

• 16.6% savings in data with no visual motion imagery degradation


Imagery

Subsampling Created to reduce bandwidth to transport video Takes advantage of human physiology where

• The eye far more sensitive to intensity changes vs. color changes In Analog

• Intensity (Luma) plus TWO of the color (chroma) components is used• Phase modulation used to encode color is about 1.5 MHz (vs. 3.25 for Intensity)

In Digital• One color sample per luma = 4:4:4 equivalent to RGB

Bits/frame = 1080*1920*24 (16 million colors) = 49,766,400• One color sample per two luma = 4:2:2, equivalent to broadcast analog TV

Bits/Frame = 1080*1920*(10 -bit luma + 10-bit Red or Blue sample) = 41,472,000 (16% reduction)

• Side effect: Saves storage capacity requirements as well• No effect on human perception of the imagery • EXCEPT there is a color alias at edges when the edge is between luma samples with shared color

sample.


Imagery

Subsampling Effects


Imagery

Color Alias when intensity changes on a constant color• Y-b-r = g • Y-Δ –b-r = g –Δ

Changes the ratio of RGB which changes the color, not just the brightness

Good Side effect for overlay• Natural “surround”


Pixels & Bit Count & Data Rate

October 26, 2011

Key Points to Remember Each pixel = a Y (luma) sample Vertical blanking space adds lines; e.g. 45 in 1080

• 1080 lines, plus blanking = 1125 lines/frame Horizontal blanking space adds samples; e.g. 280 Y samples per line in 1080/60

• 1920 visible pixels + 280 Y H blanking samples = 2200 pixels/line Each pixel in 4:2:2 sampling is 20 bits deep

• 10 bits of luma (Y) and 1 of the color components (Cr or Cb) @ 10 bits = 20 bits Interlace Video two types

• Segmented Frame delivers ½ the image in one field and the other half in a second field

• True Interlace One image odd lines, a second image (16 ms later) using even lines.

Progressive Video delivers a complete frame per scan• Frame rate = field rate (e.g. 1080p/60)• Frames may repeated at field rate (e.g. 1080p/30)


Pixels & Bit Count, Data Rates, Resolutions & Specs

SDTV HDTV

Inte

rlace

d (I)

Pro

gres

sive (

P)

Active lines per

frame

Total lines per

frame

Active Luma

samples perLine

LumaSamples in

Blanking Area

Total Luma

Samples (Pixels)

Sam

plin

g Fre

quen

cy (M

Hz)

Aspect Ratio

Frame Rate (Hz)

SMPTESDI

Bit Format

DigitizingSpecification

Bit Rate (MBit/Sec)

SDTV I 480 525 720 138 858 13.5 4:3 29.97 259M ITU-R BT.601 270.0

HDTV P 720 750 1280 370 1650 74.25 16:960 or

60/1.001292M ITU-R BT.709 1485.0

HDTV (PAL)

P 720 750 1280 700 1980 74.25 16:9 50 292M ITU-R BT.709 1485.0

HDTV I 1080 1125 1920 280 2200 74.25 16:930 or

30/1.001292M ITU-R BT.709 1485.0

HDTV (PAL)

I 1080 1125 1920 720 2640 74.25 16:9 25 292M ITU-R BT.709 1485.0

HDTV P 1080 1125 1920 280 2200 148.5 16:9 60 424M ITU-R BT.709 2970.0

25


Transport & Storage


Metadata

Storage Metrics SMPTE 296; 720 Frame Parameters

• 3.1 MB/frame• 1 minute of 720p/60 video = 11,160 MB

SMPTE 274; 1080 Frame parameters• 6.2 MB/frame• 1 minute of 1080p/30 video =11,160 MB• 1 minute of 1080i/60 video =11,160 MB• 1 minute of 1080p/60 video =22,360 MB

Data Rate Metrics 720p/60 = 1.485 Gbits/sec 1080i/60 =1.485 Gbits/sec 1080p/30=1.485 Gbits/sec 1080p/60=2.970 Gbits/sec


Transport Issue

How do you get raw video (SDI) from source to destination? Analog NTSC

• any channel with a 6 MHz bandwidth will work SD-SDI

• Requires a channel capable of passing 270 M Bit data rate (1.5 GHz for 7th harmonic) HD-SDI

• Requires 1.5 Gbit channel for 720p/1080i and 3 GHz for 1080p(10 GHz channel)

Choices Direct Connect (copper)

• SMPTE Specs HD-SDI be capable of operating to 100 meters of 75 Ω Coax (e.g Belden 1694A) ; LOW LOSS, tight tolerance

• These do require line equalizers and drivers; Reclocking is generally needed to properly decode • Short runs can use Cat 6A/Cat 7 copper cable for short runs (10 m); CAT 5 and standard CAT 6 will not

work


Transport Issue

Choices (cont.) Ethernet

• At 1G Ethernet, ED-SDI can work• At 1 G Ethernet, 720p/1080i will not work• 10G Fiber Only full duplex only• 100G Fiber Only, full duplex, still evolving

Fiber • 10GBASE-ER single-mode fiber supports transport @ 10.3 Gbit/sec up to 30-40 Km• Next level down “-LR” can support this rate up to 220 meters

Radio• Where would the band and bandwidth exist?

Transfer of Video as Data over GigE • No less than 0.07 seconds/1080 frame; 1 minute of 60 FPS video = 5 minutes to download• Requires use of full capacity of GigE channel• Assumes no lost/corrupted packets


Storage Issue

1 Hour of 1080p/60 = 1.3416 terabytes of storage

Cinema Industry captures 100s of petabytes per day

While cost/MB is going down, the real estate, power and maintenance is huge


Compression Tool


Compression Tool

Compression would never exist if infinite bandwidth and zero cost storage existedCompression is a tool to reduce data rate & storage needs

Transport• Alternative to whole new infrastructures• Typical Compression Ratios that maintain excellent image quality

H.263 and MPEG-2 ; 30:1 MJPG 2000; 20:1 to 40:1 H.264/MEG-4 part 10; 50:1, 100:1

• Compression Issues Interframe prediction (MPEG) vs. frame-to-frame image compression (M-JPG) MPEG is motion sensitive M-JPG can generate “rings” at the harsh image edges Trade off between image quality and frame rate/Frame dropping Latency

Storage• Gigabytes/clip become megabytes/clip


Compression Tool

Compression destroys pixels Lossless preserves all of the essence Lossy loses varying degrees of the essence MPEG, H.264, H.265 all divide each image into a mosaic

Average Intensity

Chan

ge in

Col

or >

Change in Color >

32 number values in a matrix of values

> 256 values replaced by 32 = 8:1

Macro-pixelStarts @ 16x16

pixels


Compression Tool

Differential Frames

Can achieve 1000:1 average compression or more

Reference FrameAll values in the matrix

Differential FramesDuplicate Matrix elements replaced with a reference to its duplicate in the reference frame

Reference FrameAll values in the matrix

Group of Frames


Options: Set the number of frames in a group, image quality/frame rate preference

• More frames, higher compression ratio• More frames, the more buffering needed to reconstruct• Motion prediction reduces aliasing

Set data rate• Closes loop on data rate; auto adjusts GOP, bit depth mosaic dimensions, motion comp, frame rate as

needed Set Frame Rate, Image Quality – all have different effects

Compression Tool

Reference Frame

Interframe 1

Interframe n

Group of Frames


Compression Tool

Compressor is Closed Loop A FIFO equivalent is used to monitor transfer rate into the transport channel

• FIFO content grows = more compression needed• FIFO content decreases = less compression needed• FIFO can’t empty (not data to send)• FIFO can’t overflow (lost frames)• Initial buffering required to pre-load the FIFO

Consequence of FIFO underflow• The video chain must resync, delays could be many frames

Consequence of FIFO overflow• Dropped frames

Compressor Options• Increase the GOP• Increase the macro-pixel dimension• Reduce the sampling• Reduce bit depth


Compression Tool

Consequences of Increasing the GOP A corrupted reference frame can lose more differential frames

• Dropped frames, sync loss More pre-buffer is needed, more decoder buffering is needed

Consequences of Increasing the macro-pixel dimension Fewer macro-pixels per image; loss of resolution

Consequences of Reducing the sampling Compressors can alter sampling to 4:2:0 Loss of detail and increased color aliasing

Consequences of Reducing the bit depth Loss of detail


Compression Tool







TransportCompression Tool

MJPEG Every frame independently compressed Good

• Less buffering to reconstruct (less end-to-end latency than MPEG)• Each image is independently reconstructable

Bad• Wavelet exhibits ringing at sharp edges• Compression ratio limited

Complex but static scenes don’t compress very well no differential frames used to take advantage of this


Compression Tool







Compression Tool

Compressor is Closed Loop FIFO Driven just as MPEG Compressor Options

• Increase the macro-pixel dimension• Reduce the sampling• Reduce bit depth

Each frame may only be compressed 8:1 to 16:1 without loss Wavelet is lossy but can only achieve 30:1 to 40:1 compassion with good image quality Frame to frame compression varies depending on the complexity of the scene Without the use of differential frames, no further average compression is available

Benefits of no use of differential frames No lost frames with corruption of a reference frame Each frame may be rendered independently Less pre-buffering is required at the destination to being rendering video


Compression Tool

Latency∑ decode (sdi-image stream) + compression + xmit latency + buffer time + decompress + decode for display

Compression• Many factors including image content, motion between frames, hardware speed

Buffer Time• Decompression requires a complete data set and enough buffered data to ensure every frame is

reconstructed at the full expected frame rate 1300 ms to complete a 60 frame buffer @ 100 MB Ethernet @1000:1 MPEG compression,

1080p/60 (2 reference frames) 1300 ms to complete 60 frame buffer @ 100 MB Ethernet @ 500:1 compression; 720p/60 500 ms to complete a 2 frame buffer using MJPEG2000 @ @ 40:1 compression

SDI Source Decode Compress xmit

ReceiveBufferDecompressDisplay


TransportCompression Tool

SDI & Ethernet SMPTE 2022-6:2012 STD for SDI over Internet Protocol As bandwidth continues to increase, SDI over IP will become practical

• Need to compress, at least lossy compression will dissipate. Specs are being developed for clean switch points and other work flow issues Good survey of the subject, SMPTE Motion Imaging Journal, March 2014

SMPTEStandard Video Type Example

FormatsBit Rates(Mbits/s)

Bit RatesMJP2000(Mbits/s)

@10:1

Bit RatesMPEG-2

(Mbits/s)@ 30:1

Bit RatesMJP2000(Mbits/s)

@40:1

Bit RatesH.264

(Mbits/s)@50:1

1 Stream Video over Ethernet

259M SD-SDI 480i, 576i

270 360 143177

27361418

9124.85.9

6.89

3.64.4

5.47.22.93.5

10 Base T

344M ED-SDI 480p, 576p 540 54 18 13.5 10.8 100Base T

292M HD-SDI 720p, 1080i

14851470

148147 49.5 37 29.7 100Base T

424M 3G-SDI 1080p 2970 297 99 74 59.4 1000 Base T


Timing


Timing

Analog Video 6 MHz bandwidth Can be transported in native form over long distances Timing compensation is predictable and could be calibrated to microsecond resolutions Timestamps can be anywhere in the transport stream and be accurate to a few microseconds.

HD Video Needs a bandwidth > 6 GHz for reliable transport Raw video is data Raw video may only be transported over coax 300 feet; packetized over 10G Ethernet Compression is needed to transport over general networks and radio links

• Introduces Latency• Latency is scene, network, codec hardware and frame rate dependent• Latency is a variable with average values as high as 10 seconds• Latency introduces the need to timestamp at the video source


Timing

Timestamping at the source Overlay

• Compression can blur fractional seconds Metadata

• Great solution• MISB adopted this; an example = Microsecond Timestamp• Many encoders strip VANC space


Metadata


Metadata

Metadata is not new Analog video has limited metadata in vertical blanking

• A few lines are used each holding perhaps 10-20 bytes Edge encoding is another technique used mostly by the Ranges

• The standard puts 1 bit per line for about 250 lines/field (30 bytes/field)

Edge Encoding (non-broadcast)Steals a bit of each video scan line

Supports 1 bits/line60 bytes per frame

Vertical Blanking SpaceA few lines above the visible image

Contains• VITC• Closed Captioning • Teletext• Billing data • Copy protection & V-chip data• ≈240 bytes per frame


Metadata

Analog Data Spaces

Edge Encoding (non-broadcast)Steals a bit of each video scan line

Supports 1 bits/line60 bytes per frame

Vertical Blanking SpaceA few lines above the visible image

Contains• VITC• Closed Captioning • Teletext• Billing data • Copy protection & V-chip data• ≈240 bytes per frame


Metadata

HD-SDI

Horizontal Ancillary (HANC) Space280-370 samples per line (“left edge” but out of view) 740 -1120 linesTotals up to 273K -313K bytes per frame

Vertical Ancillary (VANC) Space1280-1920 samples per line (“vertical interval”)25-40 linesTotals 30K – 76K bytes per frame


Metadata

HD Data Spaces

HDTV

Inte

rlace

d (I)

Pro

gres

sive (

P)

Activ

e lin

es p

er fr

ame

Tota

l lin

es p

er fr

ame

V Bla

nkin

g A

re

Activ

e Lum

asa

mpl

es p

erLi

ne

Lum

a Sa

mpl

es in

H B

lank

ing

Area

Tota

l Lum

aSa

mpl

es (P

ixels)

Aspe

ct R

atio

Fram

e Rat

e (Hz

)

SMPT

ESD

I Bi

t Fo

rmat

Digi

tizin

gSp

ecifi

catio

n

Bit R

ate (

MBi

t/Se

c)

HDTV P 720 750 30 1280 370 1650 16:960 or

60/1.001292M

ITU-R BT.709

1485.0

HDTV (PAL)

P 720 750 30 1280 700 1980 16:9 50 292MITU-R

BT.7091485.0

HDTV I 1080 1125 45 1920 280 2200 16:930 or

30/1.001292M

ITU-R BT.709

1485.0

HDTV (PAL)

I 1080 1125 45 1920 720 2640 16:9 25 292MITU-R

BT.709 1485.0

HDTV P 1080 1125 45 1920 280 2200 16:9 60 424MITU-R

BT.709 2970.0

HANC data spaceVANC data space


Metadata

Metadata Spaces are Designed into HD 720 Frame Parameters

VANC Space 25 Lines * 1280 samples = 32KCinema & broadcast generally only use lines 14-15 for scene switch & closed captioning

HANC Space370 Samples x

750


audio snippets

• Film Codes• Payload data• Workflow

data• Copyright

data• V-chip data• Billing data• Logging Info


EAV

SAV

Line

Cou

nt

CRC

HANC

EAV

1280 luma samples

1650 luma samples


Metadata

1080 Frame Parameters

VANC Space 40 Lines * 1920 samples = 76.8KCinema & broadcast generally only use lines 14-15 for scene switch & closed captioning

HANC)Space 280 samples *

1125


audio snippets• Film Codes• Payload data• Workflow data• Copyright data• V-chip data• Billing data• Logging Info


EAV

SAV

Line

Cou

nt

CRC

HANC

EAV

1920 luma samples

2200 luma samples


Metadata

Uses of Horizontal Ancillary (HANC) space HANC space used by broadcast & Cinema

• As many as 3000 data types could be present Audio (AES); 16 Channels Payload (format, frame rate, etc.) Advertisers, FCC logging, etc.

Uses of Vertical Ancillary (VANC) Space VANC space is not used much by broadcast

• Line 14 is scene switch point• Closed Captioning (mostly line 15)

GOV uses it• MISB has >900 data types defined• All are KLV type 02

Anyone can use the KLV Structure• Form valid SMPTE 291M packet• Insert desired data• Detect Packet• Extract Data


Metadata

Many Metadata Types Defined by SMPTE Dictionary

• Example Keys 16 Channels of sound bytes, source data, airing time, editing workflow SMPTE time code SAP, film codes… on and on

• Data content• Format

Metadata Elements Dictionary RP210 Version 13 of this registry contains more than 3000 data types.


Metadata

Why is VANC Metadata Important Sparsely used Can collect data permanently synchronized to the imagery Can collect data without burning it in to the imagery Can display some or all of the data later Can extract data from video in synchronism with the imagery Data never interferes with imagery Overlay never compromised by compression artifacts Overlay color can be changed during display time Overlay position can be changed during display time MISB has many VANC space keys reserved


KLV VANC Pack is a SMPTE Structure SMPTE Standard 291M type 02 Wrappers (ADF signature bytes) Type identifiers (DID and SSID) Length (DC)

MISB uses the Type 02 KLV SMPTE Structure Key, Length Value

• Key = data type identifier (group of variables)• Length = number of bytes of the value• Value = data itself

Size of KLV = Data Count (1-255 Bytes)• Message ID (1 byte)• Program Segment Counter (2 bytes)• Key (16 bytes)• Length (1 byte)• Value ( L number of bytes); any data

Metadata

UDW


Metadata

Value Element is Memory Space Bytes may be assigned to ASCII Binary blocks Values: integers, floating point

The Key Number identifies the organization of the memory

Memory LocationMemory LocationMemory LocationMemory Location

…Memory Location

Value

1-23

5 By

tes


How many KLV packs are needed? Data have a resolution of 24 bits = 3 bytes [59 parts/billion or 59*E-9]

• More than 74 data items of 24-bit resolution can fit in a single KLV pack Example: ITS sample “Instrumentation Pack”

• 41 data items Pointing Angles up to 24 bit resolution Ranges as SP values of ±9.999999E±16 64 bit time stamps Five 10 ASCII character fields

• 170 Bytes TOTAL!

Metadata

Test IDDAS TimeGeodetic Datum Run NumberClassificationTempPressurehumiditywind speeddirectionMount IDCamera IDCamera placementH resolutionV ResoluitonImage bit depth EncodingImage mode

Test IDDAS TimeGeodetic Datum Run NumberClassificationTempPressurehumiditywind speeddirectionMount IDCamera IDCamera placementH resolutionV ResoluitonImage bit depth EncodingImage mode

EncodingImage mode Integration time Trigger timeTime offsetTimestamp modeFrame RateLens ID Zoom factor UnitsFocus settingazimuthelevation Mount angles timeRange Range timestamp Object(target) IDTsens AZ

EncodingImage mode Integration time Trigger timeTime offsetTimestamp modeFrame RateLens ID Zoom factor UnitsFocus settingazimuthelevation Mount angles timeRange Range timestamp Object(target) IDTsens AZ

Object(target) IDTsens AZTsens ELTsens RangeTxTyTzTspeedT‐HeadingTtemp

Object(target) IDTsens AZTsens ELTsens RangeTxTyTzTspeedT‐HeadingTtemp


Metadata

How do you use KLV packs MISB

• Fixed keys• No general purpose software available• Most keys have specialized purposes

Private Keys• SMPTE 291 compliant (MISB foundation)• Up to 2 packs can be inserted per frame

After the MISB Microsecond Timestamp (line 9 of each frame)• KLV Tool Kit

Free with KLV upgrade to any inserter XL template to design the memory space (235 bytes of V) GUI to Program ITS Inserters GUI to simulate & test key designs


Metadata

Non-Registered Keys –Custom Uses Special purpose designs

• Fuse your data sampled at the frame rate to each image Must conform to SMPTE 291M Type 02 structure Can pass through all commercial equipment without harm

Using equipment capable of recognizing customs keys can: Pass unobstructed, unaltered imagery while carrying image relevant data Extract previously inserted custom key data

• Overlay• Output customer data to files


Metadata

MISB used the type 02 KLV SMPTE structure > 900 register keys

• MISB Standard 0807 lists the registered private keys http://www.gwg.nga.mil/misb/stdpubs.html

• All MISB keys are 6.0E.2B.34.xx.xx.xx.xx.0E.0y.xx.xx.xx.xx.xx.xxY=01 or 02 or 03

• Most Derived for UAV uses• Structured to result in continuous pulse streams for PCM systems

Microsecond Time Stamp Found in SMPTE RP210, and MISB 0807 Key = 06.0E.2B.34.02.05.01.01.0E.01.01.03.11.00.00.00 Must be start on 1st sample after SAV of Line 9

Status Value = 1 Byte; Locked/unlocked source, valid, etc. Time Value = 8 Bytes; UNIX Epoch > µsec since Jan 1, 1970


Custom Keys and UDWs

Design a Private Key Create a key number (16 byte;32 HEX digits) Name Fields Select a data type

• Signed/unsigned integers• Scaled singed/unsigned integers• Single and Double Precision Floating Point Numbers• ASCII strings• Binary blocks of data

Locate data items and allocate space• Offset byte and number of bytes

Define Scale & Decimal Point location• 3 bytes unsigned = 16,777,216 full-scale, scaled to 360 yields a resolution of 21*10-6

• Specify 1,2 or 3 decimal places!

K ey (HE X) 0 OKL ength (B ytes ) 200

57414A19360A00017F44A0904F013AABE s t D is play P roces s T ime



F ield R ef NameF ield

NumberS tart B yte

Pad FormatInput L E N

L en R ange

Tes t ID 1 1 0 AS C II 10 1‐234DAS T ime 2 11 0 B inary 8 1‐225Geodetic Datum 3 19 0 B inary 1 1‐217R un Number 4 20 0 B inary 1 1‐216C las s ification 5 21 0 AS C II 10 1‐215Temp 6 31 0 UI‐MAX 2 1‐4P res s ure 7 33 0 UI‐MAX 2 1‐4h idit 8 35 0 UI MAX 1 1 4


NumberS tart B yte


L en R ange


Field Ref NameField

NumberStart Byte

Pad Before This Byte

Format Input LENLen

RangeFull Scale Value

Decimal Places

Display Model

Focus setting 27 87 0 UI‐MAX 2 1‐4 1024 0 1024.azimuth 28 89 0 UI‐MAX 3 1‐4 360 8 359.99997854elevation 29 93 1 SI‐MAX 3 1‐4 180 8 +/‐179.99997854

Field Ref NameField

NumberStart Byte


Format Input LENLen


Decimal Places

Display Model



Custom Keys and UDWs

Assign a Key number Design the Value Key template calculates the length

UDW


Line 9

KLV = Game Change

ITS Insertion Engine w/KLV Option Adds up to two full KLV packs after MISB Timestamp Up to 470 bytes of YOUR data per frame; Data rate to 28.2KB/sec

SAV

Pack 0Microsecond Timestamp

Can turn offFixed MISB FormatAlways first pack

Option Pack 1Data block

Can turn on or offCan be any data from 1-235 bytesOptional basic decoder ;

ASCII, Integer, decimal

Option Pack 2Data block

Can turn on or offCan be any data from 1-235 bytesOptional basic decoder ;

ASCII, Integer, decimal


ITS KLV Tool Kit

Use a MISB key or Create with the ITS KeyTemplate© Excel worksheet Export design to ‘CSV’ Distribute design to all stakeholders





Build a 16 byte key


NumberS tart B yte


L en R ange



NumberS tart B yte


L en R ange


Create Fields & Assign Data Types

ResolutionTime Budget (uS)

Qualified Length

Plan to Show

100 10 Y20 8 Y20 1 Y20 1 Y100 10 Y

.002 500 2 N

.00 500 2 N

.4 500 1 N

.8 500 1 N

ResolutionTime Budget (uS)

Qualified Length

Plan to Show

100 10 Y20 8 Y20 1 Y20 1 Y100 10 Y

.002 500 2 N

.00 500 2 N

.4 500 1 N

.8 500 1 N

Field Ref NameField

NumberStart Byte


Format Input LENLen


Decimal Places

Display Model


Field Ref NameField

NumberStart Byte


Format Input LENLen


Decimal Places

Display Model


Choose how fields would be displayed

Plan what fields would be displayed


ITS KLV Tool Kit

Data Types Built Into KeyTemplate©• Types used to parse incoming data in monitor mode• Types used to parse data when decoded from KLV packs• Types used by Key Test to form data blocks• Types used by Key Read to display extracted KLV data

Data TypeMax Bytes that

May be Assigned

Binary 235ASCII 235

UI-NZ 4

UI-WZ 4

SI-NZ 4

SI-WZ 4

UI-MAX 4

SI-MAX 4SP 4DP 8

Binary pass-through format, L=number of bytes ASCII (printable characters only), field length = number of bytes L.

unsigned integer maximum scaled, no leading zeros, field length = number of bits of resolution (e.g. 1 byte = 8 bits, 2 bytes = 16 bits, etc.; max = 4 bytes), right justified. ;N;MMM required. N= number of fractional digits to display. MMM=scale factor.signed integer maximum scaled, no leading zeros, field length = number of bits of resolution (e.g. 1 byte = 8 bits, 2 bytes = 16 bits, etc.; max = 4 bytes), right justified. ;N;MMM required. N= number of fractional digits to display. MMM=scale factor.Binary 32 single precision FP; fixed byte length (4) number range ±9.999999E±16Binary 64 double precision FP; fixed byte length (8); number range ±9.99999999999999E±16

signed integer, with leading zeros, field length = number of digits of highest value (e.g. 1 byte = ±000 to ±127, 2 bytes = ±00000 to ±32767, etc.). ;±N optional to shift decimal.

Pack Definable Data Types

Description

unsigned integer, no leading zeros, field length = number of digits of highest value (e.g. 1 byte = 0 to 255, 2 bytes = 0 to 65535, etc.; max = 4 bytes), right justified. ;±N optional to shift decimal.unsigned integer, with leading zeros, field length = number of digits of highest value (e.g. 1 byte = 000 to 255, 2 bytes = 00000 to 65535, etc.; max = 4 bytes). ;±N optional to shift decimal.signed integer, no leading zeros, field length = number of digits of highest value (e.g. 1 byte = ±0 to ±127, 2 bytes = ±0 to ±32767, etc. where sign immediately precedes the first number, max = 4 bytes), right justified. ;±N optional to shift decimal.

Data TypeMax Bytes that

May be Assigned

Binary 235ASCII 235

UI-NZ 4

UI-WZ 4

SI-NZ 4

SI-WZ 4

UI-MAX 4

SI-MAX 4SP 4DP 8

Binary pass-through format, L=number of bytes ASCII (printable characters only), field length = number of bytes L.

unsigned integer maximum scaled, no leading zeros, field length = number of bits of resolution (e.g. 1 byte = 8 bits, 2 bytes = 16 bits, etc.; max = 4 bytes), right justified. ;N;MMM required. N= number of fractional digits to display. MMM=scale factor.signed integer maximum scaled, no leading zeros, field length = number of bits of resolution (e.g. 1 byte = 8 bits, 2 bytes = 16 bits, etc.; max = 4 bytes), right justified. ;N;MMM required. N= number of fractional digits to display. MMM=scale factor.Binary 32 single precision FP; fixed byte length (4) number range ±9.999999E±16Binary 64 double precision FP; fixed byte length (8); number range ±9.99999999999999E±16

signed integer, with leading zeros, field length = number of digits of highest value (e.g. 1 byte = ±000 to ±127, 2 bytes = ±00000 to ±32767, etc.). ;±N optional to shift decimal.

Pack Definable Data Types

Description

unsigned integer, no leading zeros, field length = number of digits of highest value (e.g. 1 byte = 0 to 255, 2 bytes = 0 to 65535, etc.; max = 4 bytes), right justified. ;±N optional to shift decimal.unsigned integer, with leading zeros, field length = number of digits of highest value (e.g. 1 byte = 000 to 255, 2 bytes = 00000 to 65535, etc.; max = 4 bytes). ;±N optional to shift decimal.signed integer, no leading zeros, field length = number of digits of highest value (e.g. 1 byte = ±0 to ±127, 2 bytes = ±0 to ±32767, etc. where sign immediately precedes the first number, max = 4 bytes), right justified. ;±N optional to shift decimal.


ITS KLV Tool Kit

Inserter internal fields (IF) • Leave locations (offset/byte lengths) in key design for them• Link them to your key with the Configuration Utility • Inserters will place bytes where specified for each field• Key can be parsed and data will be displayed and extracted

6980G ‐HD 6055C ‐nGHD 6055C ‐nHDS I‐MAX 3 ±90.00000 90 0‐5 Y Y N 1S I‐MAX 3 ±180.00000 180 0‐5 Y Y N 2S I‐NZ/S I‐WZ 3 ±8,388,608 NA 0 Y Y N 3AS C II 11 NA NA Y Y N 4

S I‐NZ/S I‐WZ 2 ±12.00 NA 2 Y Y N 5

AS C II 1 NA NA Y Y Y 6


UI‐NZ/UI‐WZ 2 655.36 NA 2 Y Y Y 8

S I‐NZ/S I‐WZ 4 ±2147483648 NA Y Y Y 9


UI‐NZ/UI‐WZ 2 32000 NA N Y N 11

UI‐NZ/UI‐WZ 1 0,1,2 NA N Y N 12

S I‐NZ/S I‐WZ 2 ±16,000 NA N Y N 13

S I‐MAX 3 ±359.999957 360 0‐6 N Y N 14

S I‐MAX 3 ±359.999957 360 0‐6 N Y N 15

UI‐NZ/UI‐WZ 3 2097152 NA N Y N 16

AS C II 1 NA NA N Y N 17

ITS Inserter Internal Field Specifications

IF #B yte

L eng th

"I"=IR IG, "G"=GP S

e.g. "0720P ", "108###.##;23.97, 24,25, 29.960.00e.g. ±####### µseRequires incoming Mmetadata, else 0000IP addres s & P ort"###.###.###.###;

Video Format

K TM AZ

KTM E L

T ime S ync S ource

T ime Zone O ffs et

K TM R ange

KTM R ange Units

Video F rame R ate

Measured Latency

IP Addres s

C amera S ync Delay

T imes tamp E vent

T imes tamp offs et

MGR S

AvailabilityDec imal places

Max ValueNumber R ange

Data T ypeField Name

LatitudeLongitudeHeight/Altitude

degrees /10000degrees /10000MetersLLL########

±##.## (Hrs ). (1/4

R equires K T optioR equires K T optio2,4,6 channel units

##### (µs ec )Only w/C S optiony (1=TTL camera s y, 2=The delayed sOnly w/C S option±##### (µs ec)Only w/C S Option

R equires K T optio

R equires K T optio

6980G ‐HD 6055C ‐nGHD 6055C ‐nHDS I‐MAX 3 ±90.00000 90 0‐5 Y Y N 1S I‐MAX 3 ±180.00000 180 0‐5 Y Y N 2S I‐NZ/S I‐WZ 3 ±8,388,608 NA 0 Y Y N 3AS C II 11 NA NA Y Y N 4

S I‐NZ/S I‐WZ 2 ±12.00 NA 2 Y Y N 5



UI‐NZ/UI‐WZ 2 655.36 NA 2 Y Y Y 8

S I‐NZ/S I‐WZ 4 ±2147483648 NA Y Y Y 9


UI‐NZ/UI‐WZ 2 32000 NA N Y N 11

UI‐NZ/UI‐WZ 1 0,1,2 NA N Y N 12

S I‐NZ/S I‐WZ 2 ±16,000 NA N Y N 13

S I‐MAX 3 ±359.999957 360 0‐6 N Y N 14

S I‐MAX 3 ±359.999957 360 0‐6 N Y N 15

UI‐NZ/UI‐WZ 3 2097152 NA N Y N 16

AS C II 1 NA NA N Y N 17

ITS Inserter Internal Field Specifications

IF #B yte

L eng th

"I"=IR IG, "G"=GP S

e.g. "0720P ", "108###.##;23.97, 24,25, 29.960.00e.g. ±####### µseRequires incoming Mmetadata, else 0000IP addres s & P ort"###.###.###.###;

Video Format

K TM AZ

KTM E L

T ime S ync S ource

T ime Zone O ffs et

K TM R ange

KTM R ange Units

Video F rame R ate

Measured Latency

IP Addres s

C amera S ync Delay

T imes tamp E vent

T imes tamp offs et

MGR S

AvailabilityDec imal places

Max ValueNumber R ange

Data T ypeField Name

LatitudeLongitudeHeight/Altitude

degrees /10000degrees /10000MetersLLL########

±##.## (Hrs ). (1/4

R equires K T optioR equires K T optio2,4,6 channel units

##### (µs ec )Only w/C S optiony (1=TTL camera s y, 2=The delayed sOnly w/C S option±##### (µs ec)Only w/C S Option

R equires K T optio

R equires K T optio


Key Development, Debug & Troubleshooting Tools

Key Read Select the previously saved data file The header will show

• Key length• What video channel the data was extracted

from• What pack number of the channel• The Time Flag (1=locked, 0= not locked)• The key (16 byte value) in hex• The Time Status microseconds

Each field of the Key Design will be shown along with the data associated with the frame from which it came

• Indicated by the microsecond value

Visual Basic Source Code supplied at not cost

Global Const MAX_PACK_FIELD = 64

Type BROWSE_INFO

hWndOwner As LongpIDLRoot As LongpszDisplayName As LonglpszTitle As LongulFlags As LonglpfnCallback As LonglParam As LongiImage As Long

End Type

Declare Sub CopyMemory Lib "kernel32" Alias "RtlMoveMemory" (ByRef lpDest As Any, ByRef lpSrc As Any, ByVal ByteCount As Long)Declare Function SHBrowseForFolder Lib "shell32" (lpbi As BROWSE_INFO) As LongDeclare Function SHGetPathFromIDList Lib "shell32" (ByVal pidList As Long, ByVal lpBuffer As String) As LongDeclare Function lstrcat Lib "kernel32" Alias "lstrcatA" (ByVal lpString1 As String, ByVal lpString2 As String) As Long

Global g_iFileNum As IntegerGlobal g_iOutFileNum As Integer

Global g_sInput As StringGlobal g_sKLVToolkitPath As String

Public Function IEEE32toDouble(ByVal x1 As Integer, ByVal x2 As Integer) As Double

Dim nInt(1) As IntegerDim fTemp As SingleDim sTemp As String

'Convert 32 bit IEEE754 value to floating pointnInt(0) = x2

………………………..


Insert into the HD video stream at the source Timestamp accurately Sample data at the frame rate

See and Extract from HD video stream at the Destination Overlay some or all data Extract data to a file


6520 End-To-End Solutions

FiberArtel ArtelVideo Capture

LE4

GUI (from Matrox SDK)GUI captures, displays, imagery and KLV metadata

Ethernet

Data Concentration for Metadata Insertion

KLV Data Concentrator GUI

Your Data Sources

Your Video Source

Control, Insert, Extract

Webserver & GUI

Live Video MonitorSDI

SDI

Download Raw video data and metadata

Uncompressed ImageMetadata

KeyRead©


VANC KLV Metadata Check List

KLV packs in SDI are a game changer KLV Packs can

• Transport data• Move cipher blocks• Enable recording of clean video• Maintain alignment of imagery and data

Video Encoders/Decoders Must Preserve VANC end-to-endSDI Recorders must

• Preserve VANC at record time• Restore VANC at playback time

Video Archiving must preserve VANC• SMPTE 2022-6 can support this capability

ITS software toolkit • Create KLV• Insert your data• Monitor your data• Display your data• Test your KLV design

Documents

Pre-Workshop Tutorial HD Transport, Timing and …itea.org/images/pdf/conferences/2016 TIW/Proceedings...May 3, 2016 ITEA 20th Test Instrumentation Workshop Sheet 1 Pre-Workshop Tutorial