PROJECT – VIDEO MANIPULATOR (BASED ON ZED BOARD) FINAL PRESENTATION VIDEO MANIPULATOR (BASED ON...
38
PROJECT – VIDEO MANIPULATOR (BASED ON ZED BOARD) FINAL PRESENTATION Yakir Peretz Idan Homri Supervisor - Rolf Hilgendorf Semester - winter 2014 Duration - one semester
PROJECT – VIDEO MANIPULATOR (BASED ON ZED BOARD) FINAL PRESENTATION VIDEO MANIPULATOR (BASED ON ZED BOARD) Yakir Peretz Idan Homri Supervisor - Rolf Hilgendorf
PROJECT VIDEO MANIPULATOR (BASED ON ZED BOARD) FINAL
PRESENTATION VIDEO MANIPULATOR (BASED ON ZED BOARD) Yakir Peretz
Idan Homri Supervisor - Rolf Hilgendorf Semester - winter 2014
Duration - one semester
Slide 2
AGENDA 1. Project goals Project goals 2. Component description
Component description 3. Data flow Data flow 4. Required tests and
check points Required tests and check points 5. Clock definitions
Clock definitions 6. Software description Software description 7.
Critical issues and solutions Critical issues and solutions 8.
Complete Program (without Uart) Complete Program (without
Uart)
Slide 3
PROJECT GOALS Creating a system that enables reading images
from an external device, saving it in the memory and displaying it
by VGA. Creating a programmable logic design that will handle the
transportation of the data from the main memory to the VGA output
via video direct mapped accessed (VDMA) component. agenda
Slide 4
COMPONENT DESCRIPTION -ZYNQ In the design we use the following
components: ZYNQ processor the ZYNQ is actually the PS (processing
system) part of the design, which means all the software
programmable part. This part is very powerful and includes many
features, but we use the following: UART connection memory
controller the memory itself - DDR3 One ARM processor CORTEX A9 All
the needed interface connections to the other components in the PL
side are built in. All the clocks of the design are generated by
the zynq, and given to the relevant components. PS side overview PS
side overview
Slide 5
COMPONENT DESCRIPTION -VDMA VDMA The VDMA is the core of the PL
side of the design. It is responsible for the transportation of the
data from the memory to the stream part. It is connected to three
other components via three buses: 1. To the processor via AXI4-lite
to get data regarding the address and size of the data to get from
the memory. 2. To the memory controller via full AXI4 to get the
data from. 3. To the stream_to_video_out via AXI4-stream to send
the data to. the data transportation to the stream part is done
with respect to the VTC timing signals. VDMA VDMA
Slide 6
COMPONENT DESCRIPTION - VTC Video timing controller this
component is responsible for timing the data transfer from the VDMA
to the stream to video out component. It generated signals
regarding the vertical data transfer (line count) and the
horizontal data transfer (pixels per line) as well as the active
video signal. It works with a clock that is set in order to fit the
data size and rate of pictures per second - Clock definitionsClock
definitions Video timing controller Video timing controller
Slide 7
COMPONENT DESCRIPTION - STREAM Stream to video out AXI4-Stream
to Video Out core converts AXI4-Stream Video protocol from Xilinx
video processing cores such as VDMA, that use this protocol, to
video output with explicit sync and timing such as the unit we
built to communicate with the VGA port. In our project, the unit is
used to convert the output of the VDMA in AXI_stream protocol to an
actual video protocol that consists of: 1. Active data signal 2.
Vertical sync and horizontal sync 3. Blank periods Stream to video
out interface Stream to video out interface
Slide 8
COMPONENT DESCRIPTION RGB_OUT RGB_out this unit was built by
us, to convert the data from 8 bit per color (for red green and
blue) to 4 bit per color. The output of this unit is the input of
the VGA ports RGB 4 bits per color Vsync Hsync. agendaRGB_out
Slide 9
DATA FLOW Step 1 : Sending the data from an external device to
the uart (bitmap to pixels only) PS side overview Step 2 :
Extracting the data from the Uart and saving it to the memory Step
3: the ZYNQ processor triggers the VDMA by sending the start
address and the size of the data in the memory, on an AXI_LITE.
Block Diagram Step 4: The data is being transferred to the VDMA via
memory controller and saved in a frame buffer. VDMA Step 5: The
VDMA sends the data to the stream to video out unit, with respect
to the VTC timing. Stream to video out interface Video timing
controller Step 6: the data is transferred from the stream to a
RGB_output component to be transferred in the right form RGB_out
agenda
Slide 10
REQUIRED TESTS AND CHECK POINTS We have some strategic check
points for validating our design: Uart to memory we first check
that the data we delivered from an external device thru the Uart is
saved in the memory where we wanted it to be saved. Memory to VDMA
we check that the data is transferred correctly from the memory to
the frame bufers inside the VDMA. VDMA to stream_to_video_out we
check that the data is transferred correctly from the VDMA to the
stream to video out by reading the data runs on the AXI_stream bus.
Control signals we need to check that the video_timing_controller
is sending the timing signals as we assumed it will.
stream_to_video_out to VGA we check if the data from the
stream_to_video_out is sent as we wanted in a 24 bit (8 bit per
color and 3 colors R,G,B) format. VGA output we need to check that
the data in the output of the VGA component is the picture we
delivered. This should be displayed on the screen. Block Diagram
Block Diagram agenda
Slide 11
CLOCK DEFINITIONS There are 2 main clocks in the design (beside
the ARM clock & DDR clock) The faster clock is used for the
AXI4_lite bus that connects the ARM and the VDMA. On that bus the
ARM transfers the data regarding the address and the size of the
picture in the memory. the clock is set to 200MHZ The slower clock
is used for the full AXI4 bus and the AXI_stream bus. On that bus
we move the data from the memory to the VDMA and then from the VDMA
to the stream to video out unit. That clock is defined to be 148.5
mega pixels per second. That is calculated to fit the amount of
data being transferred in one second, calculated as: (number of
lines including blank)*(number of pixels per line including
blank)*(number of pictures per second) for us - 2200*1125*60 =
148.5[MHz] In order to fit to the screen in the lab we needed a
1080*1920, and there are 60 pictures per second. (the sizes
represent pixels). agenda Component description - VTC
Slide 12
BLOCK DIAGRAM Data flowRequired tests and check points ZYNQ sub
system VDMA sub system
Slide 13
PS SIDE OVERVIEW To VDMA via Axi lite To VDMA via AXI-4 Data
flow Component description -ZYNQ
Slide 14
Pin to Pin
Slide 15
STREAM TO VIDEO OUT INTERFACE From VDMA From video timing
controller To VGA out Data flow Component description - stream
Slide 16
VIDEO TIMING CONTROLLER For write channel not in use Optionally
can be controlled by the processor. Not in use Output timing
signals for the stream_to_video_out unit Data flow Component
description - VTC
Slide 17
RGB_OUT Output Sync signals Data 8 bit per color Input Sync
signals Data 4 bits per color Data flow Component description
RGB_out
Slide 18
VDMA Connected to the memory on a full AXI4 bus. Required for
data transfer Connected to the memory on an AXI4_strea m bus.
Required for data transfer This is the connection to the processor
to get the address and the size of the picture from Those are the 3
clocks of the desine Data flow Component description -VDMA
Slide 19
SOFTWARE DESCRIPTION
Slide 20
Vdma Configuration and Setup Vdma Start Transfer Software
Flowchart Board - C code SDK Rescaling of BMP Image, open Uart for
writing and sending the Image Matlab Rescaling of BMP Image, open
Uart for writing and sending the Image Matlab Host - MATLAB agenda
Load the incoming Image from Uart into DDR HAND SHAKE PROTOCOL Load
the incoming Image from Uart into DDR HAND SHAKE PROTOCOL
Slide 21
CRITICAL ISSUES AND SOLUTIONS ProblemSolution 1 Uart Buffer is
limited and Uninitializeduart issues 2 Stream To Video Out Always
Output 0. Hardware Connect reset to active low, and clocken to
const 1. Software Parking on frames with appropriated Xilinx
function. 3 SAMSUNG Screen requires format picture of -1080*1920
60[Hz] Hardware Configure Peripheral clock to 2200*1125*60 =
148.5[MHz] 4 Image with Noise - Inconsistent writing
problemInconsistent writing problemSoftware disable caches 5 Video
Timing Controller driver isnt loaded into SDK. SDK loads drivers
only for components that are connected directly to ZYNQ with
AXI-lite. The driver (C code) can modify the registers during
running via the AXI-lite interface (like the VDMA), but driver is
not always necessary. In some cases you need to define settings
only during invocation of the hardware in Vivado (like the VTC).
Since there is no need for connection with the CPU, because the VTC
behavior is fixed it was not connected to the AXI_LITE, and no
drive is needed. 6 The output of the "stream to video out" is 8 bit
per color, while VGA is 4 bit per color. The RGB_out unit is taking
the top 4 bits from each color. agenda
Slide 22
Uart ProblemUart Solution 1 Colors of the picture are
completely mixed Matlab the same data was sent in a loop until it
was read by the board, instead of sending it once and waiting for
ACK. 2 The shape was almost correct but the colors were different
than the original picture The UART buffer works as a FIFO,
therefore the data is saved in the memory in reverse way. 3 The
picture on the screen starts with an offset. Cleaning the FIFO of
the UART at the beginning of the read operation. The function is
set Options(UART, reset) 4 The transportation of the data was stack
every time after a different amount of transfers. A simple
handshake protocol was created in which the matlab sends the data
to the UART in a 64 byte blocks (UARTs buffer size) and waits for
the acknowledge from the board. 5 Every some amount of transfers -
one transfer is unsuccessful. When it happens, the picture on the
screen has a shift and the colors are changed. The problem is that
if the communication problem happens due to acknowledge lost, which
means that the data was receive but the matlab think it didnt
because no ack was detected, data is then resent even though it was
successfully received by the board ACKNOWLEDGE LOST (SHIFT RIGHT)
It seems like the timeout of the SDK to resend acknowledge was
equal to the timeout of the matlab before sending the image again.
That way each time acknowledge is lost, the data is being sent
twice. The timeout of the matlab was changed to be 10 times bigger
than SDK and problem was solved. 6 The same thing happens when the
communication is unsuccessful due to transmitted data lost. DATA
LOST (SHIFT LEFT) Create a protocol that knows to differentiate the
reason for the error. In that case it should resend the data.
Slide 23
FINAL RESULTS Image initializationfinal presentation
Slide 24
SUMMERY AND CONCLUTION The project goals were to load data from
an external device and save it to the internal memory, and create a
data path in which the data is transferred from the memory, and
being eventually streamed aout to the VGA port in the right format.
The first goal, even though seems to be less complicated required
more "mind work" in the sense of going into small details when
creating a communication protocol. A lot of thought should be taken
when setting the rate of the timing controller, the timeouts of the
two parts of the communication and more.
Slide 25
The second goal of this project required a great amount of
learning. Starting with the board being used, the vivado tool and
the SDK tool. After that all the IPs that were mentioned here had
to be completely understood in order to be configured correctly and
connected correctly to the other components. Connecting a system
that contains more than a few parts is a very gentle task, and
should be done with a great deal of attention to small details.
Creating such a system can be a good preparation before doing any
project even outside the academy.
Slide 26
There are a few conclusions for this project: When getting a
fixed component, one should fully understand its use before getting
to work with it. If one is "stuck" in some part of the project, it
is best to overcome the problem in some way possible and move on
with the work while thinking how to solve it. That way the work is
in progress most of the time. One should always use all the help he
can get. Most of the time when not being able to proceed because of
technical issues, the solution is found with some technical person
or the supervisor.
Slide 27
APPENDIX SOFTWARE
Slide 28
MATLAB The main goal of the program is to create Matlab GUI
interface between the PC and Zedboard in order to load the desired
image. Step One: Determine the desired uart port configuration. (8
data bit, 1 stop bit, 115200 baud rate) Step two: Load the bitmap
image into Matlab and make dimensions rescale: 1080 * 1920 Step
Three: Open the port and send information (Hand shake protocol)
software flowchart
Slide 29
C CODE SDK LOAD PICTURE INTO DDR Program should read the
incoming data and load it into the DDR. At this point of the
Project we encounter a technical problem Zedboard buffer size at
polling mode is 65 bytes only, so there is no option at this time
to load the whole picture. In order to continue with the
development, the pictures were written manually into the DDR.
software flowchart
Slide 30
Initialize DMA engine A VDMA instance is set to VDMA Physical
address Setup the Read channel- The VDMA module use only Read
Channel (mm2s). Setup of vertical and horizontal lengths, frames
store start address, and other unused Registers. C CODE SDK VDMA
CONFIGURATION AND SETUP software flowchart
Slide 31
Start the DMA engine to transfer the VDMA read channel is
activated. parking on a frame The vdma reads the same image, in
order to display image on screen Continuously. C CODE SDK VDMA
START TRANSFER software flowchart
Slide 32
HAND SHAKE PROTOCOL 20 20
Slide 33
INCONSISTENT WRITING PROBLEM
Slide 34
SOLUTION DISABLE CAHCHES Critical issues and solutions