8/6/2019 Video Solution V1.1
1/25
VIDEO SOLUTIONSRev. 1.1
8/6/2019 Video Solution V1.1
2/25
Title & Date Lattice Semiconductor ConfidentialPage: 2
Founded 1983; IPO 1989 (NASDAQ: LSCC)
~750 employees
Develops programmable products which allow the end customer todetermine functionality
$297.8M revenue in 2010, $194.4M in 2009
Customers are primarily in the communications, computing, video,consumer, and industrial end markets
Product development, manufacturing, and sales offices worldwide
World class quality certifications: ISO 9000, ISO/TS 16949, AEC-Q100
In the last ten yearsLattice has sold over
1 Billionprogrammable devices
LATTICE AT A GLANCE
8/6/2019 Video Solution V1.1
3/25
Title & Date Lattice Semiconductor ConfidentialPage: 3
Price $399
Industry
Leading AutoExposure,High Dynamic
Range &Auto White
Balance
Full HD1080p60
Capable HDRImageSignal
Processing
(ISP) Pipeline
LATTICE HDR-60 VIDEO CAMERA DEVELOPMENT KITOverview
OtherApplications:
Traffic,Automotive,
VideoConferencing
MajorApplication:Security &
SurveillanceCameras
FullyProduction-Ready HD
VideoCamera
with HDR
8/6/2019 Video Solution V1.1
4/25
Title & Date Lattice Semiconductor ConfidentialPage: 4
LATTICE HDR-60 VIDEO CAMERA DEVELOPMENT KITKit Contents
Lattice HDR-60Video Camera Development Kit
Lattice HDR-60 Video Camera Main Board
Lattice Nanovesta Sensor Board 1 x HDMI Cable
1x HDMI DVI Adapter
2 x Standard USB Programming Cables
Universal Power Supply
Quick Start Guide
8/6/2019 Video Solution V1.1
5/25
Title & Date Lattice Semiconductor ConfidentialPage: 5
LATTICE VIDEO CAMERA DEVELOPMENT KIT DEMO SETUPPlug-and-Play Demo
HDMI / DVI cable
HDMI / DVI Monitor
8/6/2019 Video Solution V1.1
6/25
Title & Date Lattice Semiconductor ConfidentialPage: 6
LATTICE VIDEO CAMERA DEVELOPMENT KITColor Image Signal Processing (ISP) Pipeline from Helion
8/6/2019 Video Solution V1.1
7/25
Title & Date Lattice Semiconductor ConfidentialPage: 7
HELION AT A GLANCE
More than
90Image Processing IP cores available
Founded 2003 as a spin-off company from the
Fraunhofer Institute More than 15 years experience in HDR/WDR and
image pre-processing
Engineering Services in customized digital cameras and displaysystems
Complete image processing solution from sensor to display, calledIONOS
Consulting in image sensor configuration and FPGA programming
Solutions in the field of Automotive, Medical, Security, Industrial
Inventor of the WDR/HDR-BLENDFEST technology and the IONOSIP suite
8/6/2019 Video Solution V1.1
8/25
Title & Date Lattice Semiconductor ConfidentialPage: 8
HDR: High Dynamic Range
Dynamic Range = Ratio of expressed in dB
Higher the dynamic range the better we can see detail in both lightand dark areas. HDR combines multiple exposures of differentduration and blends to form a clear image for both light and dark areas
High Dynamic Range Processing With Tone Mapping
Builds internal image statistics (histogram, average andmedian values) to calculate for each frame twoadaptive global tone map transfer curves and uses localtone mapping for contrast enhancements.
HDR
Intensity of brightest area
Intensity of darkest area
8/6/2019 Video Solution V1.1
9/25
Title & Date Lattice Semiconductor ConfidentialPage: 10
CMOS SENSOR INTERFACING ROADMAP
Sensors resolutions and frame rates have increased vastly Previously dominant CMOS parallel interface is no longer able to handle the
bandwidth requirements. Sensor manufactures have resorted to high-speed serial interfaces e.g. Aptina Imaging has introduced HiSPi HiSPi can operate from 1 to 4 lanes of serial data and 1 clock lane Each lane can operate up t0 700Mbps
Need for serial to parallel sensor interface bridging Majority of ISP devices support traditional CMOS parallel sensor interfaces They usually lack interfaces that support serial Therefore a bridge device is required to convert the serial data to a parallel
format.
The Lattice XP2-5 non-volatile FPGA provides an efficient and cost-effectivesolution for HiSPi to parallel bridging.
8/6/2019 Video Solution V1.1
10/25Title & Date Lattice Semiconductor ConfidentialPage: 11
SUMMARYLattice HDR-60 Video Camera Development Kit
Fully production ready design
HDR 1080p60 capable FPGA streamprocessing
Industry-leading features:- Fast auto exposure
- 120dB+ High Dynamic Range
- Support for up to 16MP sensors
Comprehensive Image SignalProcessing library
Low cost BOM
CMOS SENSOR INTERFACING
8/6/2019 Video Solution V1.1
11/25Title & Date Lattice Semiconductor ConfidentialPage: 12
CMOS SENSOR INTERFACINGIdeal Fit for Lattice XP2
Small Form Factor and Non-Volatility Ideal for tightinside-camera real-estate requirements
Sub-LVDS Support Well-Documented (TN1210)
XP2-5 and XP2-8 devices support the requiredprice-points and code footprints
Hi-Spi Design already developed,XP2 interfaces between the Aptina HiSpisensor and a parallel bus to the TI DM385.Productization Q1 2011
Other Sensors in Roadmap
Easy to Leverage Orion Board for Sensor I/F Eval Replace Orions Nanovesta Sensor Board with
available XP2 sensor I/F Board
Orion Nanovesta Board
Orion Baseboard
Vid I f S l i
8/6/2019 Video Solution V1.1
12/25Title & Date Lattice Semiconductor ConfidentialPage: 13
Video Interface Solutions
HDMI/DVI Physical Interface
Single-Link DVI Standard = 1.65 Gbps / Channel
Single-Link HDMI Standard = 3.4 Gbps / Channel
Validated on Lattice Devices = 1.65 Gbps / Channel
7:1 LVDS Physical Interface LCD display panel interface 750 Mbps / LVDS pin pair in ECP3 FPGAs
SMPTE SDI Tri-rate Interface
LATTICE HDMI PHYSICAL INTERFACE (RD1097)
8/6/2019 Video Solution V1.1
13/25Title & Date Lattice Semiconductor ConfidentialPage: 14
LATTICE HDMI PHYSICAL INTERFACE (RD1097)
Requires SERDES Channels with PLLs for Reliable Data Recovery
To Eliminate Inter-channel Skew Both Transmitter and Receiver Solution Available for LatticeECP3 and LatticeECP2M
Physical Layer Interface for Reliable and Efficient Communication Reliability
TMDS Encoding of Video Data for EMI Reduction TERC-4 Encoding of Voice Data for EMI and Error Reduction PCS Logic for Data Alignment within a Channel Hsync and Vsync Control Characters for Channel Alignment
Efficiency Data Serialization and De-serialization Using Precision SERDES
Common Interface with HDMI Protocol / Content Processors To Interface with 3rd Party HDMI Digital IP Cores (e.g., from GDA)
Transmitter
HDMI Physical Interface Reference Design (RD1097)
Receiver
TMDSEncodingfor
Reliability
Interfacewith
HDMIDigital IP
Serializerfor
Efficiency
TMDSDe-coding
De-serializerWord &Channel
Alignment
Interfacewith
HDMIDigital IP
HDMI Data
On GDA
Roadmap
LATTICE 7 1 LVDS PHYSICAL INTERFACE (RD1030)
8/6/2019 Video Solution V1.1
14/25Title & Date Lattice Semiconductor ConfidentialPage: 15
LATTICE 7:1 LVDS PHYSICAL INTERFACE (RD1030)
Leverages Pre-engineered I/O Components in all Lattice FPGA Families Source Synchronous LVDS I/Os (up to 900 Mbps Performance) LVDS Buffers, DDR Registers, Configurable PLLs, and Gearbox Logic Available for LatticeECP3, LatticeECP2M, and LatticeXP2
Physical Layer Interface for Reliable and Efficient Transmission Reliability
Data Capture High-speed LVDS Buffers DDR Registers for Data Capture on both Rising and Falling Edges
Skew Minimization Logic for Skew Minimization between Data and Clock Channels
Efficiency Data Serialization and De-serialization using Precision PLLs
Transmitter
7:1 LVDS Physical Interface Reference Design (RD1030)
Receiver
4:7De-serializer
High-speedLVDS
Buffers
IO DDRRegisters
4-Bit LVDS Data
LVDS ClockPLL (*3.5)
1 bit7:4Serializer
IO DDRRegisters
PLL (*3.5)
(7*4) BitParallelLVTTLData
(7*4) BitParallelLVTTLData
SMPTE SDI
8/6/2019 Video Solution V1.1
15/25Title & Date Lattice Semiconductor ConfidentialPage: 16
SMPTE SDI
Tri-rate RX/TX SD-SDI, HD-SDI and 3G-SDI interfaces
Support for 3G source formats, including 3G Level-B format: SMPTE425M [10]
Word alignment and timing reference sequence (TRS) detection
Field, vertical blanking (vblank) and horizontal blanking (hblank) timingsignals generation
CRC computation, error checking and insertion for HD/3GLine number(LN) decoding and encoding for HD/3G
Video Payload Identifier (VPID) insertion and extraction for HD/3G
10-bit parallel input/output support for SD
Soft-logic based low data-rate (LDR) serializer for SD transmission
VIDEO INTERFACES DEVELOPMENT KIT
8/6/2019 Video Solution V1.1
16/25Title & Date Lattice Semiconductor ConfidentialPage: 17
4 Evaluation Boards
ECP3 Video Protocol Evaluation Board (Board #1) 7:1 LVDS to DVI Converter Board (Board #2) DVI to 7:1 LVDS Converter Board (Board #3) HDMI Mezzanine Card All Necessary Cables and Power Supplies
3 Pre-built Demo Systems DVI to 7:1 LVDS Video Conversion Demo HDMI / DVI Loopback Demo HDMI Transmission Demo
Documentation Users Guides for Reference Designs, IP Core, Eval
Boards, & Demos Instructions to Setup and Operate the Demos Lattice Sales Tools on eLearn
Customer Presentation, Lobby Pitch, Marketing
Bulletin, etc.
*These are Loaner Kits Only, for Lattice FAEs in Various Regions
They are not meant for Customer Purchase
Please Contact Your Local HDBU Marketing Rep for Obtaining the Kit
VIDEO INTERFACES DEVELOPMENT KITAvailable To Demonstrate* Capabilities Of 3 Video Interfaces
ECP3 VIDEO PROTOCOL BOARD
8/6/2019 Video Solution V1.1
17/25Title & Date Lattice Semiconductor ConfidentialPage: 18
EQ
LED
R
x
D
ec
Enc
Tx
LvlShft
HDMI
IN1
HDMI
IN2
HDMI
OUT
Rsvd
Ch_algn_err
Ch_algnedWord_algn_errWord_algned
Rx_format
Tx_plolRlolRlos
From left to right:Rawdata_loopback,Audio_Mute,
Red_Fill, Green_Fill, Blue_FillRsvd HDM
I
Mezza
nine
Rev.B
ECP3VP
B
Rev.C
Used To Demonstrate HDMI / DVI Loopback Demos Simple Pass Through Demos with both Video and Audio HDMI Mezzanine Card has 2 Rx Connectors and 1 Tx Connector
Only Single Link HDMI and DVI
ECP3 VIDEO PROTOCOL BOARD +HDMI MEZZANINE CARD
Summary of Lattice FPGA Value in
8/6/2019 Video Solution V1.1
18/25Title & Date Lattice Semiconductor ConfidentialPage: 19
Summary of Lattice FPGA Value inVideo Camera Front End
FPGA offers a flexible method to implement image processingalgorithms using
Existing IPs or Custom-built IP created by end-user ECP3 offers the proper mix of DSP blocks, memories, and IOs for cost
effective video design
Integration of most common types of video interfaces within the FPGA
HDMI/DVI LVDS SMPTE SDI
8/6/2019 Video Solution V1.1
19/25
THANK YOU
8/6/2019 Video Solution V1.1
20/25Title & Date Lattice Semiconductor ConfidentialPage: 21
Back Up Slides
LatticeECP3 Family Overview
8/6/2019 Video Solution V1.1
21/25Title & Date Lattice Semiconductor ConfidentialPage: 22
Device ECP3-17 ECP3-35 ECP3-70 ECP3-95 ECP3-150
LUTs (K) 17 33 67 92 149
EBR SRAM Blocks 38 72 240 240 372
EBR SRAM (Kbits) 700 1,327 4,420 4,420 6,850
Distributed RAM (Kbits) 36 68 145 188 303
18x18 DSP Multipliers 24 64 128 128 320
3.2Gbps SERDES Channels 4 4 12 12 16
Maximum User I/O Pins 222 310 490 490 586
PLLs + DLLs 2+2 4+2 10+2 10+2 10+2
LatticeECP3 Family Overview
Key Features:
Full Featured sysDSP Blocks running at > 400 MHzUp to 320 18x18 multipliers
Cascadable, slice-based architecture
Up to 6.85Mbit Block RAM
Benefits of FPGAs in Video Processing Applications
8/6/2019 Video Solution V1.1
22/25Title & Date Lattice Semiconductor ConfidentialPage: 23
Benefits of FPGAs in Video Processing Applications
Video processing uses DSP functions
FPGA DSP implementation is more deterministic than DSP processor.
DSP Processors need to operate ~ 100x the Sample Rate of a System. FPGAscan be used in High Sample Rate Applications where Parallelism is present.
FPGAs use smaller independent memory elements, removing data bottlenecks and
faster access.
Evolving maturity of design tools for FPGA based DSP designs.
FPGA programmability allows designers to trade-off device area vs. performanceby selecting the appropriate level of parallelism to implement their functions.
FPGAs can exceed the computing power of DSPs with their inherent parallelism,breaking the paradigm of sequential execution and accomplishing more per clockcycle.
Lattice ECP3 sysDSP vs XP2/ECP2/M sysDSP
8/6/2019 Video Solution V1.1
23/25Title & Date Lattice Semiconductor ConfidentialPage: 24
Lattice ECP3 sysDSP vs. XP2/ECP2/M sysDSP
SLICE 0 SLICE 1
XP2/ECP2/M:1. No Dual Slice or Cascade capabilities
2. Need external FPGA resources to create addertree
3. Accumulator excludes other multiplierneedadditional DSP or Fabric Logic
ECP3:
1. Has Dual Slice and Cascadecapabilities
2. Internal accumulate or cascade to next
slice for wider tree
3. New Modes (MMAC, Adder Tree, WideMux, Barrel Shifter)
4. 54-bit Cascadable ALU supports
rounding and truncation.5. Neighboring ALU output chainable as
third input for ternary adders
6. Total DSP capability of up to 128,000Million Multiply accumulates per second
(MMACs)
M
A54
M
A
MM
54
18
18 18
Single sysDSP block
M
A
M
A
MM
18
18
XP2/ECP2/M vs ECP3 FIR Filter Comparison
8/6/2019 Video Solution V1.1
24/25Title & Date Lattice Semiconductor ConfidentialPage: 25
XP2/ECP2/M vs. ECP3 FIR Filter Comparison
z-1z-1 z-1z-1 z-1z-1 z-1z-1
+
ECP3: 370 MHz for sysDSP
ECP3: ~260 MHz for Fabric
ECP2: ~260 MHz for FabricLower Latency
Data_A
+
slice 3
z-1z-1z-1
+
slice 2
z-1z-1z-1
+
slice 1
z-1z-1z-1
+
slice 0
z-1z-1z-1
z-1 z-1 z-1
ECP3:400 MHz, Higher Latency
Transposed FIR Filter using MULTADDSUB and ADDER Tree (XP2, ECP2 and ECP3)
Transposed FIR Filter using MULTADDSUBSUM and Cascading (ECP3)
Data_A
+ +
+
+ +
+
MULTADDSUB
ADDER TREE
MULTADDSUBSUM
slice 0slice 1slice 2slice 3
8/6/2019 Video Solution V1.1
25/25
Common Video Resolution