View
217
Download
0
Tags:
Embed Size (px)
Citation preview
3/9/2007 EECS150 Lab Lecture #8 1
Wireless Video Conferencing
EECS150 Spring 2007 - Lab Lecture #10
Neil WarrenAllen Lee
3/9/2007 EECS150 Lab Lecture #8 2
Checkpoint #3 Information(1)
Check off Requirement: #1) Be able to set 2 sets different source
and destination addresses on 2 boards and communicate. (CP4 video streaming)
#2) Be able to set both sides’ destination address to 0xFF and a unique source address on each and have communications occur. (CP4 Handshake)
3/9/2007 EECS150 Lab Lecture #8 3
Checkpoint #3 Information
If you’re behind, you can forfeit checkpoint #3 and receive a black box. You will receive a 0 for CP3 but will be
able to move on to CP4 which is worth many more points.
Talk to Neil if you want this option. You lose the points before you get the BB.
3/9/2007 EECS150 Lab Lecture #8 4
Welcome to Checkpoint 4
Combine CP2 and CP3 to make two-way wireless video conferencing system
Note: Specification has changed to make your life much easier – based on questions from students so far. Make sure you work with the non-draft version of the Checkpoint4.doc
3/9/2007 EECS150 Lab Lecture #8 5
Overview
Revisit CP2 and CP3 Communications protocol Video compression Picture-in-picture display
3/9/2007 EECS150 Lab Lecture #8 6
Recall from Checkpoint 2
3/9/2007 EECS150 Lab Lecture #8 7
Checkpoint 4 Datapath
3/9/2007 EECS150 Lab Lecture #8 8
Our Advice (1)
Separate the work between you and your partner to make the best use of your time. Communications System Graphics System
Agree on an interface and make sure you conform to it.
3/9/2007 EECS150 Lab Lecture #8 9
Our Advice (2) Debug Independently!
Communications System: Assume that the graphics system works perfectly
(conforms to your interface). Hardcode in data and requests using timers and
debug the system by using the packet sniffer. Graphics System:
Assume that the communications system works perfectly.
Hardcode transmission inputs, and whatever interface method you use for send/receive acknowledgements.
Much easier to debug each piece, then just put the 2 working pieces together. If you conform to your interface it should work!
3/9/2007 EECS150 Lab Lecture #8 10
Communications Module (1)
Responsibilities Maintain communications link.
Lost packet robustness Provide cross-compatibility
Conform to communications protocol Encode and Decode packets and pass
data to graphics engine Handle the initial handshake between
master and slave.
3/9/2007 EECS150 Lab Lecture #8 11
Communications Module (2) Recommended Pieces:
State Machine Keeps track of the current state of the communications
module. Handles handshaking and properly re-synchronizing the
communications link after lost packet. Sends commands to Packet Encoder, receives updates
from Packet Decoder Packet Encoder
Generate payload data for each type of packet and handle handshake with transceiver.
Take commands from state machine. Packet Decoder
Passes data to the graphics engine and gives status updates to the state machine.
3/9/2007 EECS150 Lab Lecture #8 12
Communications Module (3) Initial Handshaking
Purpose: To allow any 2 boxes without knowledge of the
other’s address to communicate. To set up the timing of master and slave send and
receives.
Note: At the beginning both the master and slave begin with their destination addresses being 0xFF and their source addresses being unique
3/9/2007 EECS150 Lab Lecture #8 13
Communications Module (4) Handshake Method:
Master broadcasts “Master-Seek” packets once every 2.4ms. When a slave receives a “Master-Seek” packet it sends a “Slave-ReqAck”
When the master receives the “Slave-ReqAck” packet it changes its destination address to the source address of the slave and then transmits a “Master-Ready” packet with the new destination address.
When the slave receives a “Master-Ready” packet the slave sends a “Slave-Ready” packet and moves into a state where it can receive active video packets from the master. The “Slave-Ready” packet should be sent with its destination address as the source of the address of the master.
When the master receives the “Slave-Ready” ready packet, it sends its first packet of active video data.
Now the communication should alternate between the master and slave transmitting active video data.
3/9/2007 EECS150 Lab Lecture #8 14
Communications Module (5) Communications Protocol
Purpose: To allow cross-compatibility between different
implementations. Abstracts away the communications aspect of the
project for the graphics module. The 8-bit header designates the type of
packet. The 8-bit line number indicates which line the
respective active video data corresponds to. The 80-bit data contains the AV data for 32
pixels. (16 pixel pairs) 8 bit headers and line numbers make it
easy to view on the packet sniffer!
3/9/2007 EECS150 Lab Lecture #8 15
Communications Module (6)Type 8-Bit
Header8-Bit Line Number 80-Bit Data Payload
Active Video 1 8’d1 8’b{Xmit Line} SAMPLE # 0-31
Active Video 2 8’d2 8’b{Xmit Line} SAMPLE # 32-63
Active Video 3 8’d3 8’b{Xmit Line} SAMPLE # 64-95
Active Video 4 8’d4 8’b{Xmit Line} SAMPLE # 96-127
Active Video 5 8’d5 8’b{Xmit Line} SAMPLE # 128-159
Active Video 6 8’d6 8’b{Xmit Line} SAMPLE # 160-176*
Master-Seek 8’d23 XX 80’h2
Slave-ReqAck 8’d24 XX 80’h1
Master-Ready 8’d25 XX 80’h2
Slave-Ready 8’d26 XX 80’h1
3/9/2007 EECS150 Lab Lecture #8 16
Communications Module (7) Time Outs – let you know if a communication loss has
occurred. After the expiration of 10ms from the last AV packet
transmission, if the slave not does receive a packet it should return to a state where it can receive AV packets from the master.
If this occurs 25 consecutive times without ever receiving an AV packet, then a communication loss is declared and the system must be reset. A message should appear on screen indicating a communication loss has occurred.
After the expiration of 10ms from the last AV packet received, if the master does not receive a packet it should send its next AV packet.
If this occurs 25 consecutive times without ever receiving an AV packet, then a communication loss is declared and the system must be reset. A message should appear on screen indicating a communication loss has occurred.
3/9/2007 EECS150 Lab Lecture #8 17
Communications Module (8)
Tips: Make use of the packet sniffer to
determine what is *actually* flying through the air.
The communications protocol was re-designed specifically for this purpose.
3/9/2007 EECS150 Lab Lecture #8 18
Video Compression (1)
Wireless transfer is the bottleneck Uncompressed video:
507x720x16 = 5,840,640 bits per frame
That’s ~175 Mbits per second! Theoretical max bandwidth for
wireless is 256kbps, so ~22.8 seconds per frame.
3/9/2007 EECS150 Lab Lecture #8 19
Video Compression (2)
Simple lossy compression algorithm
Resolution: 176x127 5 bits grayscale per pixel pair
3/9/2007 EECS150 Lab Lecture #8 20
Video Compression (3)
Compressed Video: 176x127x5/2 = 55,880 bits per frame Approximately 105:1 compression At theoretical max wireless
bandwidth, that’s 4.5 frames per second
3/9/2007 EECS150 Lab Lecture #8 21
Video Compression (4) Vertical compression
Sample every other odd active line (254 active odd lines)/2 = 127 lines
Horizontal compression Sample every fourth pixel pair (except
for the last eight) (360-8)/4 = 88 pixel pairs = 176 pixels For each sampled pixel pair, keep only
the upper five bits of the average luminance
3/9/2007 EECS150 Lab Lecture #8 22
Video Compression (5)
Why 176x127 instead of 180x127? Good question. There’s a good
answer, figure it out!
3/9/2007 EECS150 Lab Lecture #8 23
Video Decompression
Reverse of compression, but lossy due to lack of chrominance
5-bit grayscale represents a pixel pair Decompresses to 32 bits, with
appropriate padding for chrominance values
3/9/2007 EECS150 Lab Lecture #8 24
Writing Compressed Frames
Set aside an area in SDRAM for sending and receiving compressed frames
Update the stored frame to send only after the entire frame has been sent
It is okay to lose a local frame to store a compressed one
3/9/2007 EECS150 Lab Lecture #8 25
Writing Compressed Frames Sequence for handling compressed
frames: Wait for SDRAM and ChipCon initialization Store a compressed frame Tell transmitter FSM that frame in SDRAM is
ready Wait as compressed frame is sent Receive signal from transmitter that last
packet for frame has been sent Once entire compressed frame is sent, store
a new compressed frame
3/9/2007 EECS150 Lab Lecture #8 26
Writing Compressed Frames Two options for implementing compressed
video controller: Merge with VD processor, using the same FIFO
Advantage: Arbiter sees one write port for local video Drawback: Need to cross clock boundaries
Separate module with own FIFO Advantage: May not need to cross clock boundaries Drawback: Arbiter sees two write ports for local video
Pick the one that makes more sense to you
3/9/2007 EECS150 Lab Lecture #8 27
Dual-Clock Communication Modules running on the camera clock and
the system clock cannot safely communicate with direct control signals.
If your modules need to communicate but are on different clocks, you must either implement a safe handshaking protocol or use the memory element we provide.
For your convenience, we’ve included a dual-clocked one-bit-wide FIFO to safely exchange control signals, but you don’t have to use it.
3/9/2007 EECS150 Lab Lecture #8 28
Sender and Receiver FSMs Sender/Transmitter
Waits for frame to be available in SDRAM
Reads from SDRAM and packages data into a packet
Requests a new frame once last packet for frame is sent
Receiver Decodes the compressed data Writes received data into SDRAM
3/9/2007 EECS150 Lab Lecture #8 29
Displaying Compressed Video
3/9/2007 EECS150 Lab Lecture #8 30
Displaying Compressed Video
Constantly display compressed frames to avoid SDRAM decay
This way, you don’t have to auto-refresh!
3/9/2007 EECS150 Lab Lecture #8 31
Multi-Port Arbiter
At least two more ports are needed for the sender and receiver FSMs.
A fifth port may be needed depending on how you handle writing compressed frames.
3/9/2007 EECS150 Lab Lecture #8 32
Tips, Hints, & Common Pitfalls
Start early! If you’re already done with CP3, start
now! We are not providing any Verilog
besides the optional one-bit-wide memory element.
Run the TA solution to see what the final result should look like.
3/9/2007 EECS150 Lab Lecture #8 33
Tips, Hints, & Common Pitfalls
Split up the project into graphics and communications. These parts are fairly independent so
you and your partner can work in parallel.
Do not combine until you’ve verified each part works independently.
It can take up to 25 minutes to push to board after combining!
3/9/2007 EECS150 Lab Lecture #8 34
Tips, Hints, & Common Pitfalls Work Incrementally
Display a sectioned screen Display only compressed video at full frame
rate Simultaneously display local compressed
and uncompressed video at full frame rate Display uncompressed video at full frame
rate and local compressed at 1 FPS Test sender and receiver FSMs by sending
local “packets” to yourself
3/9/2007 EECS150 Lab Lecture #8 35
Tips, Hints, & Common Pitfalls
Be careful how you handle the half-full packet (packet 6): Make sure the receiver knows which
half of the payload to look at for the video data
3/9/2007 EECS150 Lab Lecture #8 36
Tips, Hints, & Common Pitfalls
2 Clocks with phase offset: Camera Clock != Normal Clock Use the 1-bit wide dual-clocked fifo
we provide to cross clock boundaries.
3/9/2007 EECS150 Lab Lecture #8 37
Requirements Must be TA Solution Compatible.
EC can be non-compatible, but you must still develop a compatible solution first.
1 bit file. Both master and slave systems should contained on 1 bit file.
Onscreen prompts. Should display to the user the communications status (see TA solution for more info)
Some method of outputting the (max) time between new frames.
3/9/2007 EECS150 Lab Lecture #8 38
Extra Credit Maximum 20% Early check off 10% Some Ideas (EC% TBD):
Faster frame rate Larger packet sizes Faster transmission Non-full duplex communications
UI options Menus Video output modifications (change contrast with
buttons, etc.) Use N64 controller for extra buttons to add functionality
We will provide a black box to interface Open-ended! Come up with something special,
cool, clever, unique, and/or challenging.
3/9/2007 EECS150 Lab Lecture #8 39
Questions?