TMS320DM357 Video Processing Front End (VPFE › lit › ug › sprug39 › sprug39.pdfTMS320DM357 DMSoC Video Processing Front End (VPFE) User's Guide Literature Number: SPRUG39 November

TMS320DM357 DMSoCVideo Processing Front End (VPFE)

User's Guide

Literature Number: SPRUG39November 2008

2 SPRUG39–November 2008Submit Documentation Feedback

http://www.go-dsp.com/forms/techdoc/doc_feedback.htm?litnum=SPRUG39

Contents

Preface....................................................................................................................................... 131 Introduction ....................................................................................................................... 16

1.1 Purpose of the Video Processing Front End......................................................................... 161.2 Features .................................................................................................................. 171.3 Functional Block Diagram .............................................................................................. 211.4 Use Case Statement .................................................................................................... 21

2 Camera Subsystem Environment ......................................................................................... 222.1 Parallel Generic Configuration (Raw) ................................................................................. 232.2 ITU-R BT.656 Configuration Functional Interface ................................................................... 252.3 Generic YUV Interface .................................................................................................. 282.4 VPFE/Camera Subsystem I/O Multiplexing .......................................................................... 29

3 Integration ......................................................................................................................... 313.1 Clocking, Reset and Power Management Scheme ................................................................. 313.2 Hardware Requests ..................................................................................................... 323.3 VPFE Top-Level Register Mapping Summary ....................................................................... 33

4 Functional Description ........................................................................................................ 334.1 Block Diagram ........................................................................................................... 334.2 Interfacing with Image Sensors ........................................................................................ 344.3 VPFE Data/Image Processing ......................................................................................... 384.4 VPFE Arbitration and Data Transfer .................................................................................. 794.5 Error Reporting........................................................................................................... 83

5 Programming Model ........................................................................................................... 845.1 Setup for Typical Configuration ........................................................................................ 845.2 Resetting the Camera Subsystem..................................................................................... 845.3 Configuring the Clocks and the Control Signals..................................................................... 845.4 Programming the CCD Controller ..................................................................................... 855.5 Programming the Preview Engine..................................................................................... 925.6 Programming the Resizer .............................................................................................. 975.7 Programming the H3A................................................................................................. 1015.8 Programming the Histogram.......................................................................................... 1045.9 Programming the Shared Buffer Logic (VPSS Registers) ........................................................ 1085.10 Error Identification...................................................................................................... 1095.11 Supported Use Cases ................................................................................................. 109

6 Video Processing Front End (VPFE) Registers ..................................................................... 1236.1 CCD Controller (CCDC) Registers................................................................................... 1236.2 Preview Engine (PREV) Registers................................................................................... 1476.3 Resizer Registers ...................................................................................................... 1696.4 Histogram Registers ................................................................................................... 1796.5 Hardware 3A (H3A) Registers........................................................................................ 188

7 Video Processing Subsystem (VPSS) Registers ................................................................... 2037.1 VPSS Peripheral Revision and Class Information Register (PID) ............................................... 203

SPRUG39–November 2008 Table of Contents 3Submit Documentation Feedback


www.ti.com

7.2 VPSS Peripheral Control Register (PCR)........................................................................... 2047.3 SDRAM Non-Real-Time Read Request Expand Register (SDR_REQ_EXP) ................................. 205

Contents4 SPRUG39–November 2008Submit Documentation Feedback


www.ti.com

List of Figures1 Video Processing Subsystem (VPSS) Block Diagram ................................................................ 162 Video Processing Front End (VPFE) Block Diagram .................................................................. 213 Raw Mode Timing Diagram ............................................................................................... 244 DDR2 Output Format ...................................................................................................... 245 BT.656 Signal Interface ................................................................................................... 266 BT.656 Mode Data Format in SDRAM .................................................................................. 277 Video Processing Subsystem Block Diagram .......................................................................... 338 CCD Controller Frame and Control Signal Definitions ................................................................ 359 CCD Controller Processing Block Diagram – Raw Data Mode ...................................................... 3810 CCD Controller Color Patterns ........................................................................................... 3811 CCD Controller Input Sampling Block Diagram – Raw Data Mode.................................................. 3912 CCD Controller Initial Processing Block Diagram – Raw Data Mode ............................................... 4013 CCD Controller Optical Black Averaging and Application............................................................. 4114 CCD Controller Video Port Interface and Data Formatter Block Diagram – Raw Data Mode ................... 4315 CCD Controller Data Formatter Conversion Area Selection.......................................................... 4416 CCD Controller Video Port Framing ..................................................................................... 4417 CCD Controller Output Formatter Block Diagram – Raw Data Mode ............................................... 4518 Example for Decimation Pattern.......................................................................................... 4619 A-Law Table................................................................................................................. 4720 Frame Image Format Conversion (de-interlaced, 2-field input) ...................................................... 5021 Example Formats of Input and Output Image .......................................................................... 5122 DDR2 Output Format ...................................................................................................... 5223 CCD Controller Processing Block Diagram – YUV Modes ........................................................... 5224 CCD Controller Input Sampling Block Diagram – YUV Modes....................................................... 5325 CCD Controller Initial Processing Block Diagram – YUV Modes .................................................... 5426 CCD Controller Video Port Interface and Data Formatter Block Diagram – YUV Modes ........................ 5527 CCD Controller Output Formatter Block Diagram – YUV Modes .................................................... 5528 Preview Engine Processing Flow Block Diagram ...................................................................... 5729 Horizontal Distance for Bayer Pattern input ............................................................................ 5830 Black Adjustment Functional Model ..................................................................................... 6031 Gamma Table Example ................................................................................................... 6032 Resizer Processing Flow Block Diagram................................................................................ 6333 Typical Sample Rate Converter .......................................................................................... 6434 Resizer’s Functionality..................................................................................................... 6535 Model of Resizer’s Approximation Scheme............................................................................. 6536 Alignment of Input Pixels to Tap Coefficients .......................................................................... 6737 High-Pass Gain as a Function of Absolute High Passed Luma...................................................... 6838 Pseudo-Code Description of the Resizer Algorithm: 4-Tap/8-Phase Mode ........................................ 6939 Resampling Algorithm for 4 Taps and 8 Phases....................................................................... 6940 Pseudo-Code Description of the Resizer Algorithm: 7-Tap/4-Phase Mode ........................................ 7041 Resampling Algorithm for 7 Taps and 4 Phases....................................................................... 7042 Chroma Processing Option 1: Filter With Luma........................................................................ 7243 Chroma Processing Option 2: Bilinear Interpolation................................................................... 7344 Histogram Processing Flow............................................................................................... 7545 Color Pattern Indices ...................................................................................................... 7646 Region Priority .............................................................................................................. 7747 Color Pattern Indices ...................................................................................................... 7848 Number of DMA Transfer Requests per Line........................................................................... 8049 Alignment of Starting Address Pointer................................................................................... 8050 Video Port Interface Bandwidth Balancing.............................................................................. 8151 SDRAM/DDRAM Read Bandwidth Balancing .......................................................................... 8252 VPFE Data Flow Block Diagram ......................................................................................... 84

SPRUG39–November 2008 List of Figures 5Submit Documentation Feedback


www.ti.com

53 Pixel Selection Locations in Data Flow Diagram....................................................................... 8754 Dependencies Among Framing Settings in Data Flow ................................................................ 8855 VDINT0/VDINT1 Interrupt Behavior when VDPOL = 0................................................................ 8956 VDINT0/VDINT1 Interrupt Behavior when VDPOL = 1................................................................ 8957 VDINT2 Interrupt Behavior ................................................................................................ 9058 Firmware Interaction for SDRAM-Input Preview ....................................................................... 9559 Firmware Interaction for SDRAM-Input Resizing....................................................................... 9860 Firmware Interaction for SDRAM-Input Histogram ................................................................... 10661 Preview/Movie Capture Mode Data Path .............................................................................. 11062 Preview Mode Two-Pass Resize Processing Time .................................................................. 11263 Still (Raw) Image Capture Mode Data Path........................................................................... 11464 Still Image Processing Mode Data Path ............................................................................... 11565 Multi-Pass Processing Through Preview Engine ..................................................................... 11666 Horizontal Slicing Through Preview Engine ........................................................................... 11867 Video Capture Mode Data Path......................................................................................... 12068 Processed Image Resize Data Path ................................................................................... 12169 Peripheral Revision and Class Information Register (PID).......................................................... 12470 Peripheral Control Register (PCR) ..................................................................................... 12571 Sync and Mode Set Register (SYN_MODE) .......................................................................... 12672 HD and VD Signal Width Register (HD_VD_WID) ................................................................... 12873 Number of Pixels in a Horizontal Line and Number of Lines in a Frame Register (PIX_LINES)............... 12874 Horizontal Pixel Information (HORZ_INFO) ........................................................................... 12975 Vertical Line—Settings for the Starting Pixel (VERT_START) ..................................................... 13076 Number of Vertical Lines (VERT_LINES) ............................................................................. 13077 Culling Information in Horizontal and Vertical Directions (CULLING).............................................. 13178 Horizontal Size (HSIZE_OFF)........................................................................................... 13179 SDRAM/DDRAM Line Offset Register (SDOFST).................................................................... 13280 SDRAM Address Register (SDR_ADDR).............................................................................. 13381 Optical Black Clamping Settings Register (CLAMP) ................................................................. 13482 DC Clamp Register (DCSUB) ........................................................................................... 13583 CCD Color Pattern Register (COLPTN) ............................................................................... 13684 Black Compensation Register (BLKCMP) ............................................................................. 13785 Fault Pixel Correction Register (FPC).................................................................................. 13886 Fault Pixel Correction SDRAM Address Register (FPC_ADDR) ................................................... 13987 VD Interrupt Timing Register (VDINT) ................................................................................. 14088 A-Law Setting Register (ALAW) ........................................................................................ 14189 REC656 Interface Register (REC656IF)............................................................................... 14290 CCD Configuration Register (CCDCFG)............................................................................... 14291 Data Reformatter/Video Port Configuration Register (FMTCFG)................................................... 14492 Data Reformatter/Video Input Interface Horizontal Information Register (FMT_HORZ) ........................ 14593 Data Reformatter/Video Input Interface Vertical Information Register (FMT_VERT) ............................ 14594 Video Port Output Settings Register (VP_OUT) ...................................................................... 14695 Peripheral Revision and Class Information Register (PID).......................................................... 14896 Peripheral Control Register (PCR) ..................................................................................... 14997 Horizontal Information/Setup Register (HORZ_INFO) ............................................................... 15198 Vertical Information/Setup Register (VERT_INFO)................................................................... 15299 Read Address From SDRAM Register (RSDR_ADDR) ............................................................. 152100 Line Offset for the Read Data Register (RADR_OFFSET).......................................................... 153101 Dark Frame Address From SDRAM Register (DSDR_ADDR) ..................................................... 153102 Line Offset for the Dark Frame Data Register (DRKF_OFFSET) .................................................. 154103 Write Address to the SDRAM Register (WSDR_ADDR) ............................................................ 154104 Line Offset for the Write Data Register (WADD_OFFSET) ......................................................... 155105 Input Formatter/Averager Register (AVE) ............................................................................. 156

6 List of Figures SPRUG39–November 2008Submit Documentation Feedback


www.ti.com

106 Horizontal Median Filter Register (HMED) ............................................................................ 157107 Noise Filter Register (NF)................................................................................................ 157108 White Balance Digital Gain Register (WB_DGAIN) .................................................................. 158109 White Balance Coefficients Register (WBGAIN)...................................................................... 158110 White Balance Coefficients Selection Register (WBSEL) ........................................................... 159111 CFA Register (CFA) ...................................................................................................... 160112 Black Adjustment Offset Register (BLKADJOFF) .................................................................... 160113 RGB2RGB Blending Matrix Coefficients Register (RGB_MAT1)................................................... 161114 RGB2RGB Blending Matrix Coefficients Register (RGB_MAT2)................................................... 161115 RGB2RGB Blending Matrix Coefficients Register (RGB_MAT3)................................................... 162116 RGB2RGB Blending Matrix Coefficients Register (RGB_MAT4)................................................... 162117 RGB2RGB Blending Matrix Coefficients Register (RGB_MAT5)................................................... 163118 RGB2RGB Blending Matrix Offsets Register (RGB_OFF1)......................................................... 163119 RGB2RGB Blending Matrix Offsets Register (RGB_OFF2)......................................................... 164120 Color Space Conversion Coefficients Register (CSC0) ............................................................. 164121 Color Space Conversion Coefficients Register (CSC1) ............................................................. 165122 Color Space Conversion Coefficients Register (CSC2) ............................................................. 165123 Color Space Conversion Offsets Register (CSC_OFFSET) ........................................................ 166124 Contrast and Brightness Settings Register (CNT_BRT)............................................................. 166125 Chrominance Supression Settings Register (CSUP) ................................................................ 167126 Maximum/Minimum Y and C Settings Register (SETUP_YC) ...................................................... 167127 Setup Table Addresses Register (SET_TBL_ADDRESS) .......................................................... 168128 Setup Table Data Register (SET_TBL_DATA) ....................................................................... 168129 Peripheral Revision and Class Information Register (PID).......................................................... 170130 Peripheral Control Register (PCR) ..................................................................................... 171131 Resizer Control Bits Register (RSZ_CNT) ............................................................................ 172132 Output Width and Height After Resizing Register (OUT_SIZE) .................................................... 173133 Input Starting Information Register (IN_START) ..................................................................... 174134 Input Width and Height Before Resizing Register (IN_SIZE) ....................................................... 175135 Input SDRAM Address Register (SDR_INADD) ...................................................................... 175136 SDRAM Offset for the Input Line Register (SDR_INOFF)........................................................... 176137 Output SDRAM Address Register (SDR_OUTADD) ................................................................. 176138 SDRAM Offset for the Output Line Register (SDR_OUTOFF) ..................................................... 177139 Horizontal Filter Coefficients Register (HFILToe) .................................................................... 177140 Vertical Filter Coefficients Register (VFILToe)........................................................................ 178141 Luminance Enhancer Register (YENH)................................................................................ 178142 Peripheral Identification Register (PID) ................................................................................ 180143 Peripheral Control Register (PCR) ..................................................................................... 180144 Histogram Control Register (HIST_CNT).............................................................................. 181145 White/Channel Balance Settings Register (WB_GAIN).............................................................. 182146 Region n Horizontal Information Registers (R0_HORZ-R3_HORZ) ............................................... 184147 Region n Vertical Information Registers (R0_VERT-R3_VERT) ................................................... 184148 Histogram Address Register (HIST_ADDR)........................................................................... 185149 Histogram Data Register (HIST_DATA) ............................................................................... 185150 Read Address Register (RADD) ........................................................................................ 186151 Read Address Offset Register (RADD_OFF) ......................................................................... 186152 Horizontal/Vertical Information Register (H_V_INFO) ............................................................... 187153 Peripheral Revision and Class Information Register (PID).......................................................... 189154 Peripheral Control Register (PCR) ..................................................................................... 189155 Setup for the AF Engine Paxel Configuration Register (AFPAX1) ................................................. 191156 Setup for the AF Engine Paxel Configuration Register (AFPAX2) ................................................. 192157 Start Position for AF Engine Paxels Register (AFPAXSTART)..................................................... 193158 Start Position for IIRSH Register (AFIIRSH) .......................................................................... 193

SPRUG39–November 2008 List of Figures 7Submit Documentation Feedback


www.ti.com

159 SDRAM/DDRAM Start Address for AF Engine Register (AFBUFST) ............................................. 194160 IIR Filter Coefficient Data for Set 0 Register (AFCOEFF010) ...................................................... 194161 IIR Filter Coefficient Data for Set 0 Register (AFCOEFF032) ...................................................... 195162 IIR Filter Coefficient Data for Set 0 Register (AFCOEFF054) ...................................................... 195163 IIR Filter Coefficient Data for Set 0 Register (AFCOEFF076) ...................................................... 196164 IIR Filter Coefficient Data for Set 0 Register (AFCOEFF098) ...................................................... 196165 IIR Filter Coefficient Data for Set 0 Register (AFCOEFF0010)..................................................... 197166 IIR Filter Coefficient Data for Set 1 Register (AFCOEFF110) ...................................................... 197167 IIR Filter Coefficient Data for Set 1 Register (AFCOEFF132) ...................................................... 198168 IIR Filter Coefficient Data for Set 1 Register (AFCOEFF154) ...................................................... 198169 IIR Filter Coefficient Data for Set 1 Register (AFCOEFF176) ...................................................... 199170 IIR Filter Coefficient Data for Set 1 Register (AFCOEFF198) ...................................................... 199171 IIR Filter Coefficient Data for Set 1 Register (AFCOEFF1010)..................................................... 200172 Configuration for AE/AWB Windows Register (AEWWIN1) ......................................................... 200173 Start Position for AE/AWB Windows Register (AEWINSTART) .................................................... 201174 Start Position and Height for Black Line of AE/AWB Windows Register (AEWINBLK) ......................... 201175 Configuration for Subsample Data in AE/AWB Window Register (AEWSUBWIN) .............................. 202176 SDRAM/DDRAM Start Address for AE/AWB Engine Register (AEWBUFST).................................... 202177 VPSS Peripheral Revision and Class Information Register (PID).................................................. 203178 VPSS Peripheral Control Register (PCR) ............................................................................. 204179 SDRAM Non-Real-Time Read Request Expand Register (SDR_REQ_EXP) .................................... 205

List of Figures8 SPRUG39–November 2008Submit Documentation Feedback


www.ti.com

List of Tables1 Interface Signals for Video Processing Front End ..................................................................... 222 Interface Signals for Raw Mode .......................................................................................... 233 Interface Signals for ITU-R BT.656 Mode............................................................................... 254 Video Timing Reference Codes for SAV and EAV .................................................................... 275 F, V, H Signal Descriptions ............................................................................................... 276 F, H, V Protection (Error Correction) Bits ............................................................................... 277 Interface Signals for YUV Interface ...................................................................................... 288 DDR Storage Format for YCbCr Processing ........................................................................... 299 Signals for VPFE Imager Interface Modes.............................................................................. 2910 CCD/LCD Supplemental Control Multiplexing .......................................................................... 3011 ARM Interrupts.............................................................................................................. 3212 EDMA Events ............................................................................................................... 3213 VPFE Module Register Map .............................................................................................. 3314 CCD Interface Signals ..................................................................................................... 3415 ITU-R BT.656 Interface Signals .......................................................................................... 3616 CCD Interface Signals ..................................................................................................... 3717 Fault Pixel Table Format .................................................................................................. 4218 Fault Pixel Correction Method ............................................................................................ 4219 A-Law Table – Part 1 ...................................................................................................... 4820 A-Law Table – Part 2 ...................................................................................................... 4921 DDR Output Format for YUV422 Mode ................................................................................. 5622 Non-linear Luminance Enhancement Table Entry Format ............................................................ 6123 YCC422 Programmable Output Options ................................................................................ 6224 Image Cropping by Preview Functions .................................................................................. 6225 Arrangement of the Filter Coefficients ................................................................................... 6626 Input Size Calculations .................................................................................................... 6727 Processing Example for 1:2.56 Horizontal Resize ..................................................................... 7428 White Balance Field-to-Pattern Assignments........................................................................... 7629 Regions and Bins for Histogram ......................................................................................... 7630 Recommended SHIFT Value to Avoid Bin Clipping ................................................................... 7831 Region Offset Addresses.................................................................................................. 7832 Color Offset Addresses within Each Region............................................................................ 7933 Maximum Data Throughput Capabilities ................................................................................ 7934 Alignment Performance.................................................................................................... 8035 VPSS Error Indicators ..................................................................................................... 8336 CCD Controller Required Configuration Parameters .................................................................. 8537 CCD Controller Conditional Configuration Parameters................................................................ 8638 Preview Engine Required Configuration Parameters.................................................................. 9239 Preview Engine Conditional Configuration Parameters ............................................................... 9340 Preview Engine Memory Address Ranges.............................................................................. 9441 Resizer Required Configuration Parameters ........................................................................... 9742 Resizer Conditional Configuration Parameters......................................................................... 9743 AF Engine Required Configuration Parameters ...................................................................... 10144 AF Engine Conditional Configuration Parameters.................................................................... 10145 AEW Engine Required Configuration Parameters.................................................................... 10246 Histogram Required Configuration Parameters....................................................................... 10447 Histogram Conditional Configuration Parameters .................................................................... 10548 VPSS Error Indicators.................................................................................................... 10849 Preview/Movie Capture Mode Data Path Register Configuration .................................................. 111

SPRUG39–November 2008 List of Tables 9Submit Documentation Feedback


www.ti.com

50 Still Image Capture Mode Data Path Register Configuration ....................................................... 11451 Still Image Processing Mode Data Path Register Configuration ................................................... 11552 Image Cropping by Preview Functions ................................................................................ 11653 Video Capture Mode Data Path Register Configuration ............................................................. 12054 Processed Image Resize Data Path Register Configuration........................................................ 12155 Video Processing Front End Subsystem Module Register Map.................................................... 12356 CCD Controller (CCDC) Registers ..................................................................................... 12357 Peripheral Revision and Class Information Register (PID) Field Descriptions ................................... 12458 Peripheral Control Register (PCR) Field Descriptions ............................................................... 12559 Sync and Mode Set Register (SYN_MODE) Field Descriptions.................................................... 12660 HD and VD Signal Width Register (HD_VD_WID) Field Descriptions ............................................. 12861 Number of Pixels in a Horizontal Line and Number of Lines in a Frame Register (PIX_LINES) Field

Descriptions ............................................................................................................... 12862 Horizontal Pixel Information (HORZ_INFO) Field Descriptions..................................................... 12963 Vertical Line—Settings for the Starting Pixel (VERT_START) Field Descriptions ............................... 13064 Number of Vertical Lines (VERT_LINES) Field Descriptions ....................................................... 13065 Culling Information in Horizontal and Vertical Directions (CULLING) Field Descriptions ....................... 13166 Horizontal Size (HSIZE_OFF) Field Descriptions .................................................................... 13167 SDRAM/DDRAM Line Offset Register (SDOFST) Field Descriptions ............................................. 13268 SDRAM Address Register (SDR_ADDR) Field Descriptions ....................................................... 13369 Optical Black Clamping Settings Register (CLAMP) Field Descriptions........................................... 13470 DC Clamp Register (DCSUB) Field Descriptions..................................................................... 13571 CCD Color Pattern Register (COLPTN) Field Descriptions ......................................................... 13672 Black Compensation Register (BLKCMP) Field Descriptions....................................................... 13773 Fault Pixel Correction Register (FPC) Field Descriptions ........................................................... 13874 Fault Pixel Correction SDRAM Address Register (FPC_ADDR) Field Descriptions............................. 13975 VD Interrupt Timing Register (VDINT) Field Descriptions ........................................................... 14076 A-Law Setting Register (ALAW) Field Descriptions .................................................................. 14177 REC656 Interface Register (REC656IF) Field Descriptions ........................................................ 14278 CCD Configuration Register (CCDCFG) Field Descriptions ........................................................ 14379 Data Reformatter/Video Port Configuration Register (FMTCFG) Field Descriptions ............................ 14480 Data Reformatter/Video Input Interface Horizontal Information Register (FMT_HORZ) Field Descriptions .. 14581 Data Reformatter/Video Input Interface Vertical Information Register (FMT_VERT) Field Descriptions...... 14582 Video Port Output Settings Register (VP_OUT) Field Descriptions................................................ 14683 Preview Engine (PREV) Registers ..................................................................................... 14784 Peripheral Revision and Class Information Register (PID) Field Descriptions ................................... 14885 Peripheral Control Register (PCR) Field Descriptions ............................................................... 14986 Horizontal Information/Setup Register (HORZ_INFO) Field Descriptions......................................... 15187 Vertical Information/Setup Register (VERT_INFO) Field Descriptions ............................................ 15288 Read Address From SDRAM Register (RSDR_ADDR) Field Descriptions ....................................... 15289 Line Offset for the Read Data Register (RADR_OFFSET) Field Descriptions ................................... 15390 Dark Frame Address From SDRAM Register (DSDR_ADDR) Field Descriptions ............................... 15391 Line Offset for the Dark Frame Data Register (DRKF_OFFSET) Field Descriptions............................ 15492 Write Address to the SDRAM Register (WSDR_ADDR) Field Descriptions ...................................... 15493 Line Offset for the Write Data Register (WADD_OFFSET) Field Descriptions ................................... 15594 Input Formatter/Averager Register (AVE) Field Descriptions ....................................................... 15695 Horizontal Median Filter Register (HMED) Field Descriptions ...................................................... 15796 Noise Filter Register (NF) Field Descriptions ......................................................................... 15797 White Balance Digital Gain Register (WB_DGAIN) Field Descriptions ............................................ 15898 White Balance Coefficients Register (WBGAIN) Field Descriptions ............................................... 15899 White Balance Coefficients Selection Register (WBSEL) Field Descriptions..................................... 159

10 List of Tables SPRUG39–November 2008Submit Documentation Feedback


www.ti.com

100 CFA Register (CFA) Field Descriptions................................................................................ 160101 Black Adjustment Offset Register (BLKADJOFF) Field Descriptions .............................................. 160102 RGB2RGB Blending Matrix Coefficients Register (RGB_MAT1) Field Descriptions ............................ 161103 RGB2RGB Blending Matrix Coefficients Register (RGB_MAT2) Field Descriptions ............................ 161104 RGB2RGB Blending Matrix Coefficients Register (RGB_MAT3) Field Descriptions ............................ 162105 RGB2RGB Blending Matrix Coefficients Register (RGB_MAT4) Field Descriptions ............................ 162106 RGB2RGB Blending Matrix Coefficients Register (RGB_MAT5) Field Descriptions ............................ 163107 RGB2RGB Blending Matrix Offsets Register (RGB_OFF1) Field Descriptions .................................. 163108 RGB2RGB Blending Matrix Offsets Register (RGB_OFF2) Field Descriptions .................................. 164109 Color Space Conversion Coefficients Register (CSC0) Field Descriptions ....................................... 164110 Color Space Conversion Coefficients Register (CSC1) Field Descriptions ....................................... 165111 Color Space Conversion Coefficients Register (CSC2) Field Descriptions ....................................... 165112 Color Space Conversion Offsets Register (CSC_OFFSET) Field Descriptions .................................. 166113 Contrast and Brightness Settings Register (CNT_BRT) Field Descriptions ...................................... 166114 Chrominance Supression Settings Register (CSUP) Field Descriptions .......................................... 167115 Maximum/Minimum Y and C Settings Register (SETUP_YC) Field Descriptions................................ 167116 Setup Table Addresses Register (SET_TBL_ADDRESS) Field Descriptions .................................... 168117 Setup Table Data Register (SET_TBL_DATA) Field Descriptions ................................................. 168118 Resizer Registers ......................................................................................................... 169119 Peripheral Revision and Class Information Register (PID) Field Descriptions ................................... 170120 Peripheral Control Register (PCR) Field Descriptions ............................................................... 171121 Resizer Control Bits Register (RSZ_CNT) Field Descriptions ...................................................... 172122 Output Width and Height After Resizing Register (OUT_SIZE) Field Descriptions .............................. 173123 Input Starting Information Register (IN_START) Field Descriptions ............................................... 174124 Input Width and Height Before Resizing Register (IN_SIZE) Field Descriptions................................. 175125 Input SDRAM Address Register (SDR_INADD) Field Descriptions................................................ 175126 SDRAM Offset for the Input Line Register (SDR_INOFF) Field Descriptions .................................... 176127 Output SDRAM Address Register (SDR_OUTADD) Field Descriptions........................................... 176128 SDRAM Offset for the Output Line Register (SDR_OUTOFF) Field Descriptions ............................... 177129 Horizontal Filter Coefficients Register (HFILToe) Field Descriptions .............................................. 177130 Vertical Filter Coefficients Register (VFILToe) Field Descriptions ................................................. 178131 Luminance Enhancer Register (YENH) Field Descriptions ......................................................... 178132 Histogram Registers...................................................................................................... 179133 Peripheral Identification Register (PID) Field Descriptions.......................................................... 180134 Peripheral Control Register (PCR) Field Descriptions ............................................................... 180135 Histogram Control Register (HIST_CNT) Field Descriptions........................................................ 181136 White/Channel Balance Settings Register (WB_GAIN) Field Descriptions ....................................... 182137 White Balance Gain Values ............................................................................................. 183138 Region n Horizontal Information Registers (R0_HORZ-R3_HORZ) Field Descriptions ......................... 184139 Region n Vertical Information Registers (R0_VERT-R3_VERT) Field Descriptions ............................. 184140 Histogram Address Register (HIST_ADDR) Field Descriptions .................................................... 185141 Histogram Data Register (HIST_DATA) Field Descriptions ......................................................... 185142 Read Address Register (RADD) Field Descriptions.................................................................. 186143 Read Address Offset Register (RADD_OFF) Field Descriptions................................................... 186144 Horizontal/Vertical Information Register (H_V_INFO) Field Descriptions ......................................... 187145 Hardware 3A (H3A) Registers .......................................................................................... 188146 Peripheral Revision and Class Information Register (PID) Field Descriptions ................................... 189147 Peripheral Control Register (PCR) Field Descriptions ............................................................... 190148 Setup for the AF Engine Paxel Configuration Register (AFPAX1) Field Descriptions........................... 191149 Setup for the AF Engine Paxel Configuration Register (AFPAX2) Field Descriptions........................... 192150 Start Position for AF Engine Paxels Register (AFPAXSTART) Field Descriptions .............................. 193

SPRUG39–November 2008 List of Tables 11Submit Documentation Feedback


www.ti.com

151 Start Position for IIRSH Register (AFIIRSH) Field Descriptions.................................................... 193152 SDRAM/DDRAM Start Address for AF Engine Register (AFBUFST) Field Descriptions ....................... 194153 IIR Filter Coefficient Data for Set 0 Register (AFCOEFF010) Field Descriptions ................................ 194154 IIR Filter Coefficient Data for Set 0 Register (AFCOEFF032) Field Descriptions ................................ 195155 IIR Filter Coefficient Data for Set 0 Register (AFCOEFF054) Field Descriptions ................................ 195156 IIR Filter Coefficient Data for Set 0 Register (AFCOEFF076) Field Descriptions ................................ 196157 IIR Filter Coefficient Data for Set 0 Register (AFCOEFF098) Field Descriptions ................................ 196158 IIR Filter Coefficient Data for Set 0 Register (AFCOEFF0010) Field Descriptions .............................. 197159 IIR Filter Coefficient Data for Set 1 Register (AFCOEFF110) Field Descriptions ................................ 197160 IIR Filter Coefficient Data for Set 1 Register (AFCOEFF132) Field Descriptions ................................ 198161 IIR Filter Coefficient Data for Set 1 Register (AFCOEFF154) Field Descriptions ................................ 198162 IIR Filter Coefficient Data for Set 1 Register (AFCOEFF176) Field Descriptions ................................ 199163 IIR Filter Coefficient Data for Set 1 Register (AFCOEFF198) Field Descriptions ................................ 199164 IIR Filter Coefficient Data for Set 1 Register (AFCOEFF1010) Field Descriptions .............................. 200165 Configuration for AE/AWB Windows Register (AEWWIN1) Field Descriptions................................... 200166 Start Position for AE/AWB Windows Register (AEWINSTART) Field Descriptions.............................. 201167 Start Position and Height for Black Line of AE/AWB Windows Register (AEWINBLK) Field Descriptions ... 201168 Configuration for Subsample Data in AE/AWB Window Register (AEWSUBWIN) Field Descriptions ........ 202169 SDRAM/DDRAM Start Address for AE/AWB Engine Register (AEWBUFST) Field Descriptions ............. 202170 Video Processing Subsystem (VPSS) Registers ..................................................................... 203171 VPSS Peripheral Revision and Class Information Register (PID) Field Descriptions ........................... 203172 VPSS Peripheral Control Register (PCR) Field Descriptions ....................................................... 204173 SDRAM Non-Real-Time Read Request Expand Register (SDR_REQ_EXP) Field Descriptions.............. 205

List of Tables12 SPRUG39–November 2008Submit Documentation Feedback


PrefaceSPRUG39–November 2008

Read This First

About This ManualThis document describes the video processing front end (VPFE) in the TMS320DM357 Digital MediaSystem-on-Chip (DMSoC).

Notational ConventionsThis document uses the following conventions.• Hexadecimal numbers are shown with the suffix h. For example, the following number is 40

hexadecimal (decimal 64): 40h.• Registers in this document are shown in figures and described in tables.

– Each register figure shows a rectangle divided into fields that represent the fields of the register.Each field is labeled with its bit name, its beginning and ending bit numbers above, and itsread/write properties below. A legend explains the notation used for the properties.

– Reserved bits in a register figure designate a bit that is used for future device expansion.

Related Documentation From Texas InstrumentsThe following documents describe the TMS320DM357 Digital Media System-on-Chip (DMSoC). Copies ofthese documents are available on the Internet at www.ti.com. Tip: Enter the literature number in thesearch box provided at www.ti.com.

SPRUG06 — TMS320DM357 DMSoC Video Processing Back End (VPBE) User's Guide. Describesthe video processing back end (VPBE) in the TMS320DM357 Digital Media System-on-Chip(DMSoC) video processing subsystem. Included in the VPBE is the video encoder, on-screendisplay, and digital LCD controller.

SPRUG25 — TMS320DM357 DMSoC ARM Subsystem Reference Guide. Describes the ARMsubsystem in the TMS320DM357 Digital Media System-on-Chip (DMSoC). The ARM subsystem isdesigned to give the ARM926EJ-S (ARM9) master control of the device. In general, the ARM isresponsible for configuration and control of the device; including the video processing subsystem,and a majority of the peripherals and external memories.

SPRUG26 —TMS320DM357 DMSoC Universal Asynchronous Receiver/Transmitter (UART) User'sGuide. This document describes the universal asynchronous receiver/transmitter (UART) peripheralin the TMS320DM357 Digital Media System-on-Chip (DMSoC). The UART peripheral performsserial-to-parallel conversion on data received from a peripheral device, and parallel-to-serialconversion on data received from the CPU.

SPRUG27 — TMS320DM357 DMSoC Inter-Integrated Circuit (I2C) Peripheral User's Guide.Describes the inter-integrated circuit (I2C) peripheral in the TMS320DM357 Digital MediaSystem-on-Chip (DMSoC). The I2C peripheral provides an interface between the DMSoC and otherdevices compliant with the I2C-bus specification and connected by way of an I2C-bus. Externalcomponents attached to this 2-wire serial bus can transmit and receive up to 8-bit wide data to andfrom the DMSoC through the I2C peripheral. This document assumes the reader is familiar with theI2C-bus specification.

SPRUG39–November 2008 Preface 13Submit Documentation Feedback

www.ti.comhttp://www-s.ti.com/sc/techlit/sprug06http://www-s.ti.com/sc/techlit/sprug25http://www-s.ti.com/sc/techlit/sprug26http://www-s.ti.com/sc/techlit/sprug27http://www.go-dsp.com/forms/techdoc/doc_feedback.htm?litnum=SPRUG39

Notational Conventions www.ti.com

SPRUG28 — TMS320DM357 DMSoC 64-Bit Timer User's Guide. Describes the operation of thesoftware-programmable 64-bit timer in the TMS320DM357 Digital Media System-on-Chip (DMSoC).Timer 0 and Timer 1 are used as general-purpose (GP) timers and can be programmed in 64-bitmode, dual 32-bit unchained mode, or dual 32-bit chained mode; Timer 2 is used only as awatchdog timer. The GP timer modes can be used to generate periodic interrupts or enhanceddirect memory access (EDMA) synchronization events. The watchdog timer mode is used toprovide a recovery mechanism for the device in the event of a fault condition, such as a non-exitingcode loop.

SPRUG29 — TMS320DM357 DMSoC Serial Peripheral Interface (SPI) User's Guide. Describes theserial peripheral interface (SPI) in the TMS320DM357 Digital Media System-on-Chip (DMSoC). TheSPI is a high-speed synchronous serial input/output port that allows a serial bit stream ofprogrammed length (1 to 16 bits) to be shifted into and out of the device at a programmedbit-transfer rate. The SPI is normally used for communication between the DMSoC and externalperipherals. Typical applications include an interface to external I/O or peripheral expansion viadevices such as shift registers, display drivers, SPI EPROMs and analog-to-digital converters.

SPRUG30 — TMS320DM357 DMSoC Host Port Interface (HPI) Reference Guide. This documentdescribes the host port interface in the TMS320DM357 Digital Media System-on-Chip (DMSoC).The HPI provides a parallel port interface through which an external host processor can directlyaccess the TMS320DM357 DMSoC processor's resources (configuration and program/datamemories).

SPRUG31 — TMS320DM357 DMSoC General-Purpose Input/Output (GPIO) User's Guide. Describesthe general-purpose input/output (GPIO) peripheral in the TMS320DM357 Digital MediaSystem-on-Chip (DMSoC). The GPIO peripheral provides dedicated general-purpose pins that canbe configured as either inputs or outputs. When configured as an input, you can detect the state ofthe input by reading the state of an internal register. When configured as an output, you can writeto an internal register to control the state driven on the output pin.

SPRUG32 — TMS320DM357 DMSoC Multimedia Card (MMC)/Secure Digital (SD) Card ControllerUser's Guide. Describes the multimedia card (MMC)/secure digital (SD) card controller in theTMS320DM357 Digital Media System-on-Chip (DMSoC). The MMC/SD card is used in a number ofapplications to provide removable data storage. The MMC/SD controller provides an interface toexternal MMC and SD cards. The communication between the MMC/SD controller and MMC/SDcard(s) is performed by the MMC/SD protocol.

SPRUG33 — TMS320DM357 DMSoC Asynchronous External Memory Interface (EMIF) User'sGuide. Describes the asynchronous external memory interface (EMIF) in the TMS320DM357Digital Media System-on-Chip (DMSoC). The EMIF supports a glueless interface to a variety ofexternal devices.

SPRUG34 — TMS320DM357 DMSoC Enhanced Direct Memory Access (EDMA) Controller User'sGuide. Describes the operation of the enhanced direct memory access (EDMA3) controller in theTMS320DM357 Digital Media System-on-Chip (DMSoC). The EDMA3 controller’s primary purposeis to service user-programmed data transfers between two memory-mapped slave endpoints on theDMSoC.

SPRUG35 — TMS320DM357 DMSoC Audio Serial Port (ASP) User's Guide. Describes the operationof the audio serial port (ASP) audio interface in the TMS320DM357 Digital Media System-on-Chip(DMSoC). The primary audio modes that are supported by the ASP are the AC97 and IIS modes. Inaddition to the primary audio modes, the ASP supports general serial port receive and transmitoperation, but is not intended to be used as a high-speed interface.

SPRUG36 — TMS320DM357 DMSoC Ethernet Media Access Controller (EMAC)/Management DataInput/Output (MDIO) Module User's Guide. Discusses the ethernet media access controller(EMAC) and physical layer (PHY) device management data input/output (MDIO) module in theTMS320DM357 Digital Media System-on-Chip (DMSoC). The EMAC controls the flow of packetdata from the DMSoC to the PHY. The MDIO module controls PHY configuration and statusmonitoring.

14 Read This First SPRUG39–November 2008Submit Documentation Feedback

http://www-s.ti.com/sc/techlit/sprug28http://www-s.ti.com/sc/techlit/sprug29http://www-s.ti.com/sc/techlit/sprug30http://www-s.ti.com/sc/techlit/sprug31http://www-s.ti.com/sc/techlit/sprug32http://www-s.ti.com/sc/techlit/sprug33http://www-s.ti.com/sc/techlit/sprug34http://www-s.ti.com/sc/techlit/sprug35http://www-s.ti.com/sc/techlit/sprug36http://www.go-dsp.com/forms/techdoc/doc_feedback.htm?litnum=SPRUG39

www.ti.com Notational Conventions

SPRUG37 — TMS320DM357 DMSoC Pulse-Width Modulator (PWM) Peripheral User's Guide.Describes the pulse-width modulator (PWM) peripheral in the TMS320DM357 Digital MediaSystem-on-Chip (DMSoC).

SPRUG38 — TMS320DM357 DMSoC DDR2 Memory Controller User's Guide. Describes the DDR2memory controller in the TMS320DM357 Digital Media System-on-Chip (DMSoC). The DDR2memory controller is used to interface with JESD79D-2A standard compliant DDR2 SDRAMdevices.

SPRUG39 — TMS320DM357 DMSoC Video Processing Front End (VPFE) User's Guide. Describesthe video processing front end (VPFE) in the TMS320DM357 Digital Media System-on-Chip(DMSoC) video processing subsystem. Included in the VPFE is the preview engine, CCD controller,resizer, histogram, and hardware 3A (H3A) statistic generator.

SPRUGH2 —TMS320DM357 DMSoC Peripherals Overview Reference Guide. This document providesan overview of the peripherals in the TMS320DM357 Digital Media System-on-Chip (DMSoC).

SPRUGH3 —TMS320DM357 DMSoC Universal Serial Bus Controller User's Guide. This documentdescribes the universal serial bus (USB) controller in the TMS320DM357 Digital MediaSystem-on-Chip (DMSoC). The USB controller supports data throughput rates up to 480 Mbps. Itprovides a mechanism for data transfer between USB devices and also supports host negotiation.

SPRUG39–November 2008 Read This First 15Submit Documentation Feedback

http://www-s.ti.com/sc/techlit/sprug37http://www-s.ti.com/sc/techlit/sprug38http://www-s.ti.com/sc/techlit/sprug39http://www-s.ti.com/sc/techlit/sprugh2http://www-s.ti.com/sc/techlit/sprugh3http://www.go-dsp.com/forms/techdoc/doc_feedback.htm?litnum=SPRUG39

1 Introduction

1.1 Purpose of the Video Processing Front End

Resizer

Preview

Histogram

H3A

CCDC

Videoport

interface(VPI)

VPFE

Control bus I/F

CMOS/CCDor video

INT Clk

gen

OSD

VENCAnalog data(DACS)

Digital data(LCD)

EMIF −> SDRAM/DDRAM

VPBE

decoder

Sha

red

buffe

r lo

gic

(SB

L)

User's GuideSPRUG39–November 2008

Video Processing Front End (VPFE)

The video processing subsystem (VPSS), Figure 1, provides an input interface (VPFE) for externalimaging peripherals such as image sensors, video decoders, etc. and an output interface (videoprocessing back end, (VPBE)) for display devices, such as analog SDTV displays, digital LCD panels,HDTV video encoders, etc.

There is a set of common buffer memory and DMA controls to ensure efficient use of the DDR2 burstbandwidth in addition to these peripherals. The shared buffer logic/memory is a unique block that istailored to allow seamless integration of the VPSS into an image/video processing system. The sharedbuffer logic/memory acts as the primary source or sink to all of the VPFE and VPBE modules that areeither requesting or transferring data to/from DDR2. In order to use the external DDR2 bandwidthefficiently, the shared buffer logic/memory interfaces with the DMA system via a high bandwidth bus(64-bit wide). The shared buffer logic/memory also interfaces with all of the VPFE and VPBE modules viaa 128-bit wide bus. The shared buffer logic/memory (divided into the read and write buffers and arbitrationlogic) is capable of performing the following functions:1. It is imperative that the VPSS use DDR2 bandwidth efficiently due to both its large bandwidth

requirements and the real-time requirements of the VPSS modules.2. A set of user-accessible registers is provided to monitor overflows or failures in data transfers because

it is possible to configure the VPSS modules in a way that exceeds DDR2 bandwidth.

Figure 1. Video Processing Subsystem (VPSS) Block Diagram

Video Processing Front End (VPFE)16 SPRUG39–November 2008Submit Documentation Feedback


1.2 Features

1.2.1 CCD Controller (CCDC)

www.ti.com Introduction

The VPFE is comprised of the CCD controller (CCDC), preview engine image pipe (IPIPE), hardware 3Astatistic generator (H3A), resizer, and histogram blocks. Together, these modules provide a powerful andflexible front-end interface. These modules can be broken down into two distinct types:• The first type consists of modules that are in the direct data flow path and affect the input image data

stream:– The CCD controller provides an interface to image sensors and digital video sources.– The preview engine IPIPE is a parameterized hard-wired image processing block whose image

processing functions can be customized for each sensor type to realize good image quality andvideo frame rates for displays and video recording modes.

– The resizer module provides a means to size the input image data to the desired display or videoencoding resolution.

• The second type consists of modules that provide statistics on the incoming images to aid camerasystems designers:– The H3A module is designed to support the control loops for auto focus (AF), auto white balance

(AWB), and auto exposure (AE) by collecting metrics on the raw image data from the CCDcontroller.

– The histogram module bins input color pixels, depending on the amplitude, and provides statisticsrequired to implement various H3A (AE/AF/AWB) algorithms and tune the final image/video output.The histogram module can operate on raw image data from CCD controller or DDR2.

The CCD controller is responsible for accepting raw (unprocessed) image/video data from a sensor(CMOS or CCD). In addition, the CCD controller can accept YUV video data in numerous formats,typically from video decoder devices. In the case of raw inputs, the CCD controller output requiresadditional image processing to transform the raw input image to the final processed image. This additionalimage processing can be done either on-the-fly in the preview engine IPIPE, or in software. In parallel, rawdata input to the CCD controller can also be used to compute various statistics (H3A, Histogram) toeventually control the image/video tuning parameters. The CCD controller is programmed via control andparameter registers. The following features are supported by the CCD controller module:• Conventional Bayer pattern sensor formats• Generates HD/VD timing signals and field ID to an external timing generator or synchronizes to the

external timing generator• Support for progressive and interlaced sensors (hardware support for up to 2 fields)• Support for up to 90 MHZ sensor clocks• Support for REC656/CCIR-656 standard (YCbCr 422 format, either 8-bit or 10-bit)• Support for YCbCr 422 format, either 8- or 16-bit with discrete H and VSYNC signals• Support for up to 16-bit input.• Generates optical black clamping signals• Support for shutter signal control• Support for digital clamping and black level compensation• Support for 10-bit to 8-bit A-law compression• Support for a low-pass filter prior to writing to SDRAM. If this filter is enabled, 2 pixels each in the left

and right edges of each line are cropped from the output• Support for generating output to range from 16-bits to 8-bits wide (8-bits wide allows for 50% saving in

storage area)• Support for down-sampling via programmable culling patterns• Ability to control output to the DDR2 via an external write enable signal• Support for up to 32K pixels (image size) in both the horizontal and vertical directions

SPRUG39–November 2008 Video Processing Front End (VPFE) 17Submit Documentation Feedback


1.2.2 Preview Engine – Image Pipe (IPIPE)

1.2.3 Resizer

Introduction www.ti.com

The preview engine image pipe (IPIPE) is responsible for transforming raw (unprocessed) image/videodata from a sensor (CMOS or CCD) into YCbCr 422 data that is amenable for compression or display.Typically, the output of the preview engine is used for both video compression and displaying it on anexternal display device, such as a NTSC/PAL analog encoder or a digital LCD. The preview engine isprogrammed via control and parameter registers. The preview engine supports the following features:• Conventional Bayer pattern• Accepting the input image/video data from either the CCD/CMOS controller or the SDRAM/DDRAM• An output width of up to 1280 pixels wide• Automatic/mandatory cropping of pixels/lines when edge processing is performed. If all of the

corresponding modules are enabled, a total of 14 pixels per line (7 left-most and 7 right-most) and8 lines (4 top-most and 4 bottom-most) will not be output. For more information, see Section 2.

• Simple horizontal averaging (by factors of 2, 4, or 8) to handle input widths that are greater than 1280(plus the cropped number) pixels wide

• Ability to capture a dark frame (instead of applying the conventional image processing to the raw data)and store it in the SDRAM/DDRAM

• Ability to subtract a dark frame (fetched from the SDRAM/DDRAM memory) for every input raw dataframe pixel-by-pixel to improve video quality

• Ability to perform lens shading compensation instead of the dark frame subtract. Each input pixel ismultiplied with a corresponding 8-bit gain value and the result is right shifted by a programmableparameter (0-7 bits)

• Support for A-law decompression to transform non-linear 8-bit data to 10-bit linear data. This feature,which allows data in the SDRAM/DDRAM to be 8-bits only, saves 50% of the area if the input to thepreview engine is from the SDRAM/DDRAM

• A horizontal median filter for reducing temperature induced noise in pixels• A programmable noise filter that operates on a 3 × 3 grid of the same color (effectively, this is a five

line storage requirement)• Digital gain and white balance (color separate gain for white balance)• Programmable CFA interpolation that operates on a 5 × 5 grid• Programmable RGB-to-RGB blending matrix (9 coefficients for the 3 × 3 matrix)• Fully programmable gamma correction (1024 entries for each color held in an on-chip RAM)• Programmable color conversion (RGB to YUV) coefficients (9 coefficients for the 3 × 3 matrix)• Luminance enhancement (non-linear) and chrominance suppression and offset

The resizer module can accept input image/video data from either the preview engine or DDR2. Theoutput of the resizer module will be sent to the SDRAM/DDRAM. The resizer module is programmed viaits registers that are accessible by a host processor in the system. The resizer module supports thefollowing features:• Maximal output width of 1280 horizontal pixels• Input from either the preview engine (on-the-fly processing) or from external SDRAM/DDRAM• Support for up to 4× upsampling (digital zoom)

– Bi-cubic interpolation (4-tap horizontal, 4-tap vertical) can be implemented with the programmablefilter coefficients

– 8 phases of the filter coefficients are supported– Optionally select bi-linear interpolation for the chrominance components– If the input source is the preview engine, this can be performed on-the-fly

• Support for up to 1/4× down-sampling (reducing image size to store more pictures in the memory card)– 4-tap horizontal and 4-tap vertical filter coefficients (with 8-phases) for 1× to 1/2× down-sampling– For 1/2× to 1/4× down-sampling, use 7-tap mode with 4-phases– If the input source is the preview engine, this can be performed on-the-fly

18 Video Processing Front End (VPFE) SPRUG39–November 2008Submit Documentation Feedback


1.2.4 Hardware 3A (H3A)

1.2.4.1 Auto Focus (AF) Engine


• There are further constraints for real-time preview-output resizing due to the limited on-chip memoryand processing resources. Horizontal resizer stage output rate is limited to resizer_clock/2.

– SDRAM-input path has no such restrictions.– For example, at a pixel clock of 75 MHZ, no upsampling of full input width can exist. Taking 3/4 of the

width and upsampling by 4/3 to full width is possible. At a pixel clock of 37.5 MHZ, upsampling by 2× ofthe full input width is affordable. By taking 3/4 of the full width, upsampling by as much as 8/3 canoccur.

• Support for resizing either YUV 422 packed data (16-bits) or color separate data (assumed to be 8-bitdata) that is contiguous. The input source for the color separate data should be the DDR2.

• Separate/independent resizing factor for the horizontal and vertical directions• Available upsampling and down-sampling ratios are: 256/N, with N ranging from 64 to 1024• Programmable luminance sharpening after the horizontal resizing and before the vertical resizing step

The H3A module is designed to support the control loops for auto focus (AF), auto white balance (AWB),and auto exposure (AE) by collecting metrics about the imaging/video data. The metrics are used to adjustthe various parameters for processing the imaging/video data. There are two main blocks in the H3Amodule:• Auto focus (AF) engine• Auto exposure (AE) and auto white balance (AWB) engine

The AF engine extracts and filters the red, green, and blue data from the input image/video data andprovides either the accumulation or peaks of the data in a specified region. The specified region is atwo-dimensional block of data and is referred to as a paxel for the case of AF.

The AE/AWB engine accumulates the values and checks for saturated values in a sub-sampling of thevideo data. In the case of the AE/AWB, the two-dimensional block of data is referred to as a “window”.Thus, other than referring to them by different names, a paxel and a window are essentially the samething. However, the number, dimensions, and starting position of the AF paxels and the AE/AWB windowsare separately programmable.

The following features are supported by the AF engine:• Support for a Peak Mode in a paxel (a paxel is defined as a two-dimensional block of pixels)• Accumulate the maximum Focus Value of each line in a paxel• Support for an accumulation/sum mode (instead of peak mode)• Accumulate focus value in a paxel• Support for up to 36 paxels in the horizontal direction and up to 128 paxels in the vertical direction. The

number of horizontal paxels is limited by the memory size, while the vertical number of paxels is not.Therefore, the number of paxels in horizontal direction is smaller than the number of paxels in verticaldirection.

• Programmable width and height for the paxel. All paxels in the frame are the same size.• Programmable red, green, and blue position within a 2 × 2 matrix• Separate horizontal start for paxel and filtering• Programmable vertical line increments within a paxel• Parallel IIR filters configured in a dual-biquad configuration with individual coefficients (2 filters with

11 coefficients each). The filters compute the sharpness/peaks in the frame to focus on.



1.2.4.2 Auto Exposure (AE) and Auto White Balance (AWB) Engine

1.2.5 Histogram

Introduction www.ti.com

The following features are supported by the AE/AWB engine:• Accumulate clipped pixels along with all non-saturated pixels• Support for up to 36 horizontal windows• Support for up to 128 vertical windows• Programmable width and height for the windows. All windows in the frame are the same size.• Separate vertical start coordinate and height for a black row of paxels that is different than the

remaining color paxels• Programmable horizontal sampling points in a window• Programmable vertical sampling points in a window

The histogram module accepts raw image/video data (either 3 or 4 colors) and bins the pixels on a value(and color separate) basis. The value of the pixel itself is not stored; but, each bin contains the number ofpixels that are within the appropriate set range. The source of the raw data for the histogram is typically aCCD/CMOS sensor (via the CCD controller module) or optionally from SDRAM/DDRAM. The followingfeatures are supported by the histogram module:• Support for up to four regions/areas

– Each region has its own horizontal/vertical start and end position.– When regions overlap, pixels from the overlapped area are accumulated into the highest priority

region only (the priority is region0 > region1 > region2 > region3)• Support for conventional Bayer pattern sensors. Each region is capable of accumulating 4 colors

separately.• Support for 32, 64, 128, or 256 bins per color per region

– If the number of regions is 1, then 32, 64, 128, or 256 bins per color is allowed.– If the number of regions is 2, then 32, 64, or 128 bins per color is allowed.– If the number of regions is 3, then 32 or 64 bins per color is allowed.– If the number of regions is 4, then 32 or 64 bins per color is allowed.

• Support for automatic clear of the histogram RAM once the ARM reads that location (programmableregister)

• Support for saturation of the pixel count if the count exceeds the maximum value that the memorylocation can hold (each memory location is 20-bits wide)

• Support for a downshift ranging from 0 to 7 bits (this implies that the maximum range of each bin willbe 128)

• The last bin (highest range of values) will accumulate any value that is higher than the lower bound.For example, if 32 bins are set up so that each bin accumulates a range of 8 or a downshift of 3 (0 to7, 8 to 15, etc.), the last bin shall accumulate all values higher than 248 and not just the range ofvalues from 248 to 255.



1.3 Functional Block Diagram

1

0Resizer

RSZ_CNT.INPSRC0

1

PCR.SDRPORT

Previewengine

1

0

PCR.RSZPORT

0

1

PCR.SOURCE

FMTCFG.VPEN

SYN_MODE.WEN

0

1

1

0

CCDCCCDC input

interface

H3A

Histogram1

0

Video port interface (10−bit RAW)

SYN_MODE.SDR2RSZ

0 1

DDR EMIF

VPFE

HIST_CNT.SOURCE

1.4 Use Case Statement


Figure 2 shows a high-level block diagram of the VPFE functional blocks, along with the different data flowpaths. These data flow paths show how the various modules of the VPFE interact and the data source(s)for the statistics generation modules (H3A and histogram).

Figure 2. Video Processing Front End (VPFE) Block Diagram

The VPFE supports image data acquisition from sensor and digital video sources in variousmodes/formats. YUV sources have minimal image processing applied and can either be passed directly toexternal memory/DDR2 or passed to the resizer for scaling prior to writing to DDR2. Raw imager datamodes (non-YUV sources) are supported by the statistics collection modules (H3A and histogram) as wellas full preview engine image signal processing functions, plus resizing after preview.

The same processing options are supported when processing data sourced from DDR2. The onlyexception is that the H3A module cannot operate on data from DDR2.

Zooming at ratios greater than the 4× ratio in a single pass are not supported by the resizer. However, thiscan be done by passing the resized data from DDR2 through the resizer again as long as the real-timedeadlines can be met. This will be discussed in more detail in Section 5.6.5.1.



2 Camera Subsystem EnvironmentCamera Subsystem Environment www.ti.com

The VPFE interface signals are shown in Table 1.

Note: These signals can take on different meanings for the DMSoC, depending on the specificinterface chosen. Pin multiplexing is controlled from the System module. The followingsections describe each of the supported scenarios.

Table 1. Interface Signals for Video Processing Front EndPin Name I/O Description Default Tristate Pullup/PulldownPCLK In Pixel Clock In - -VD Bi V sync In tri -HD Bi H sync In tri -CI7/CCD15 Bi C IN signal/CCD in signal In tri PDCI6/CCD14 Bi C IN signal/CCD in signal In tri PDCI5/CCD13 Bi C IN signal/CCD in signal In tri PDCI4/CCD12 Bi C IN signal/CCD in signal In tri PDCI3/CCD11 In C IN signal/CCD in signal In - PDCI2/CCD10 In C IN signal/CCD in signal In - PDCI1/CCD9 In C IN signal/CCD in signal In - PDCI0/CCD8 In C IN signal/CCD in signal In - PDYI7/CCD7 In Y IN signal/CCD in signal In - PDYI6/CCD6 In Y IN signal/CCD in signal In - PDYI5/CCD5 In Y IN signal/CCD in signal In - PDYI4/CCD4 In Y IN signal/CCD in signal In - PDYI3/CCD3 In Y IN signal/CCD in signal In - PDYI2/CCD2 In Y IN signal/CCD in signal In - PDYI1/CCD1 In Y IN signal/CCD in signal In - PDYI0/CCD0 In Y IN signal/CCD in signal In - PDC_WE Bi CCD Write Enable signal In tri -C_FIELD/R0 Bi CCD Field signal/R0 (VPBE) In tri -



2.1 Parallel Generic Configuration (Raw)

2.1.1 Parallel Generic Configuration (Raw) Signal Interface

2.1.2 Parallel Generic Configuration (Raw) Signal Interface Description

www.ti.com Camera Subsystem Environment

The generic raw interface configuration is typically used for interfacing to image sensors. The VPFEsupports up to 16 bits of resolution for each sample, but sensors typically only output 8, 10, 12, or 14 bitsof useful resolution, depending on the imager and the associated AFE.

Table 2 shows the interface connections for the Raw Mode interface. The device can support up to 16 bitsof resolution for each sample but sensors typically only output 8, 10, 12, or 14 bits of useful resolutiondepending on the imager and associated AFE. When the number of data lines is less than 16, it isrecommended to connect the Raw data to the lower data lines of CCD[15-0]. Then theSYN_MODE.DATSIZ register can be used to indicate the bit size of the input so that the hardware ignoresthe upper bits that are not connected.

Table 2. Interface Signals for Raw ModePin Name DescriptionPCLK Pixel ClockVD V syncHD H syncCCD15 CCD in signalCCD14 CCD in signalCCD13 CCD in signalCCD12 CCD in signalCCD11 CCD in signalCCD10 CCD in signalCCD9 CCD in signalCCD8 CCD in signalCCD7 CCD in signalCCD6 CCD in signalCCD5 CCD in signalCCD4 CCD in signalCCD3 CCD in signalCCD2 CCD in signalCCD1 CCD in signalCCD0 CCD in signalC_WE CCD Write Enable signalC_FIELD/R0 CCD Field signal/R0 (VPBE)

The VPFE can either generate the H/V sync signals needed to interface with sensors or source them fromthe sensor or timing generator. The PCLK or the pixel clock must always be provided as an input.



2.1.3 Parallel Generic Configuration (Raw) Protocol and Data Formats

NPH

SPH

Valid dataarea

HLPFR

Global frame

PPLN

NLV

SLVn

HDW

VDW

HDVD

VDW − Vertical data widthVD − Vertical dataSLVn – Start line vertical field 0 or field 1

HDW − Horizontal data widthHD − Horizontal dataPPLN − Pixels per line

SPH − Start pixel horizontalNPH − Number of pixels horizontal

NLV − Number of lines vertical

HLPFR – Half lines per frame

Camera Subsystem Environment www.ti.com

The timing generator in the CCD controller either enables the use of external sync signals (HD/VD) orinternally-generated timing signals. Figure 3 shows various CCD controller register settings related to thetiming. The shaded area is the physical imager size and the gray area is the valid data area. The imagedata in this area is processed and stored to external SDRAM/DDRAM or sent out to the various VPFEmodules. The vertical start position for even and odd fields can be configured independently.

Figure 3. Raw Mode Timing Diagram

The bits of data from each pixel are stored in the lower bits of a 16-bit SDRAM word, and the unused bitpositions are filled with zeros. The DDR data format is shown in Figure 4. There is an optional 10-bit to8-bit A-Law compression so that 10-bit data can be reduced to 8-bit dynamic range and packed to saveDDR memory usage.

Figure 4. DDR2 Output Format

Upper Word Lower Word

MSB (31) LSB (16) MSB (15) LSB (0)

16 bit Pixel1 Pixel0

15 bit 0 Pixel1 0 Pixel0








8-bit pack Pixel3 Pixel2 Pixel1 Pixel0



2.2 ITU-R BT.656 Configuration Functional Interface

2.2.1 ITU-R BT.656 Configuration Signal Interface


ITU-R BT.656 (sometimes referred to as either CCIR-656 or REC656) is a specification that provides amethod to transfer YCbCr-4:2:2 formatted digital video data over an 8/10-bit wide interface.

Table 3 shows the interface connections for the ITU-R BT.656 interface.

Data and timing codes are transferred over the same 8/10-bit interface. When in BT.656 mode, only thedata lines and clock signal are connected between the external device and the CCD controller module ofthe VPFE. An NTSC/PAL decoder is an example of an external device that may be connected to theCCIR-656 interface.

Data lines CCD[7:0] are used for 8-bit YCbCr data and data lines CCD[9:0] are used for 10-bit YCbCrdata. The video timing signals, HD, VD, and FIELD are generated internally by the CCD controller moduleof the VPFE.

Table 3. Interface Signals for ITU-R BT.656 ModePin Name DescriptionPCLK Pixel ClockCCD9 CCD Data/BT.656 Data (optional, for 10-bit interface)CCD8 CCD Data/BT.656 Data (optional, for 10-bit interface)CCD7 CCD Data/BT.656 DataCCD6 CCD Data/BT.656 DataCCD5 CCD Data/BT.656 DataCCD4 CCD Data/BT.656 DataCCD3 CCD Data/BT.656 DataCCD2 CCD Data/BT.656 DataCCD1 CCD Data/BT.656 DataCCD0 CCD Data/BT.656 Data



2.2.2 ITU-R BT.656 Configuration Signal Interface Description

EA

V

SA

V Valid data

EA

V

SA

V

EA

V

SA

V Valid data

EA

V

SA

V

Internal delay

SPH

NPH

Data in

InternalHD

InternalVD

Internaldata

2.2.3 ITU-R BT.656 Configuration Protocol and Data Formats


Two timing reference codes synchronize HD, VD, and FIELD to the video data. At the start and end ofeach video data block, the device sends a unique timing reference code. The start code is called the startof active video signal (SAV), and the end code is called the end of active video signal (EAV). The SAVand EAV codes proceed and follow valid data, as shown in Figure 5. HD, VD, and FIELD are generatedinternally by the CCD controller, based on the SAV and EAV codes. Other CCD controller register settingsallow you to control when to read/save valid data to DDR.

Figure 5. BT.656 Signal Interface

Both timing reference signals, SAV and EAV, consist of a four word sequence in the following format:FF 00 00 XY, where FF 00 00 are a set preamble and the fourth word defines the field identification, thestate of vertical field blanking, the state of horizontal line blanking, and protection (error correction) codes.The bit format of the fourth word is shown in Table 4 and the definitions for bits, F, V, and H, are given inTable 5. F, V, and H are used in place of the usual horizontal sync, vertical sync, and blank timing controlsignals. Bits P3, P2, P1, and P0 are protection (error correction) bits for F, V, and H. The relationshipbetween F, V, and H and the protection (error correction) bits is given in Table 6. To enable errorcorrection, set the ECCFVH bit in the REC656IF register to 1. The CCD controller will automatically detectand apply error correction when the ECCFVH bit is enabled.

When operating in CCIR-656 mode, data is stored in SDRAM according to the format shown in Figure 6when the PACK8 bit in SYN_MODE is set to 1.

Note that the CCD controller outputs the XY code in the SAV and EAV into the SDRAM. In order toeliminate this, you should set the SPH field in HORZ_INFO to SPH + 1. In addition, the NPH field inHORZ_INFO should be set to accurately represent the number of active pixels.




Table 4. Video Timing Reference Codes for SAV and EAVData Bit Number First Word (FF) Second Word (00) Third Word (00) Fourth Word (XY)

9 (MSB) 1 0 0 18 1 0 0 F7 1 0 0 V6 1 0 0 H5 1 0 0 P34 1 0 0 P23 1 0 0 P12 1 0 0 P01 1 0 0 00 1 0 0 0

Table 5. F, V, H Signal DescriptionsSignal Value Command

F 0 Field 11 Field 2

V 0 01 Vertical blank

H 0 SAV1 EAV

Table 6. F, H, V Protection (Error Correction) BitsF V H P3 P2 P1 P00 0 0 0 0 0 00 0 1 1 1 0 10 1 0 1 0 1 10 1 1 0 1 1 01 0 0 0 1 1 11 0 1 1 0 1 01 1 0 1 1 0 01 1 1 0 0 0 1

Figure 6. BT.656 Mode Data Format in SDRAM

31 08 bit × 4 Pixel3 (Y1/Cr0) Pixel2 (Cr0/Y1) Pixel1 (Y0/Cb0) Pixel0 (Cb0/Y0)



2.3 Generic YUV Interface

2.3.1 Generic YUV Configuration Signal Interface


The CCD controller can accept generic YCbCr-4:2:2 formatted digital video data over an 8/16-bit wideinterface..

Table 7 shows the interface connections for the generic YUV interface.

Unlike the BT.656 mode, discrete HD and VD signals are required. An example of an external device thatmay be connected to the YUV interface is the NTSC/PAL decoder.

In 8-bit mode, data lines YI[7:0] or CI[7:0] can be used for input. When using an 8-bit interface, the YI[7:0]inputs are typically used; however, either set of data inputs can be used. Alternately, two separateimagers can be physically connected (but only one can be active at any given time). The YCINSWP bit inCCDCFG determines which set of 8-bit inputs are active.

In 16-bit mode, data lines YI[

Documents

TMS320DM357 Video Processing Front End (VPFE › lit › ug › sprug39 › sprug39.pdfTMS320DM357 DMSoC Video Processing Front End (VPFE) User's Guide Literature Number: SPRUG39 November