Autonomous Tracking Unit
John Berglund
Randy Cuaycong
Wesley Day
Andrew Fikes
Kamran Shah
Professor: Dr. Rabi Mahapatra
CPSC 483 - Spring 1999
Autonomous Tracking Unit
CPSC 483 - Spring 1999
Autonomous Tracking Unit
• Project Goals– Develop a 2-dimensional servo control system
– Develop a camera platform
– Implement an FPGA to camera interface
– Develop a memory and camera control system
– Develop a motion tracking algorithm
– Implement the algorithm in hardware
– Integrate components
– Graduate
Project Status• Objective
– Develop an autonomous camera that can identify and track an object in motion
CPSC 483 - Spring 1999
Autonomous Tracking Unit
CPSC 483 - Spring 1999
Autonomous Tracking Unit
System Diagram
SRAM
Algorithm
Camera Interface
Camera
Servo Control
Memory & Camera Control
Servos
CPSC 483 - Spring 1999
Autonomous Tracking Unit
CPSC 483 - Spring 1999
Autonomous Tracking Unit
Servos: Pulse Width Modulation
CPSC 483 - Spring 1999
Autonomous Tracking Unit
17-20ms
0º 160º
1.25ms 1.75ms
CPSC 483 - Spring 1999
Autonomous Tracking Unit
• 8-bit input angle with 9-bit internal counter
• Test the MSB for minimum pulse length
• Minimum pulse length => 1|0000|0000b - minimum number of pulses
Servos: Final Design
CPSC 483 - Spring 1999
Autonomous Tracking Unit
Bit 7 6 5 4 3 2 1
If (bit 7 == 0) {Minimum pulse not reached
} else {Minimum pulse reachedPulse = 1; Test for angle
}
If (bit[6 :0] == angle) {
Minimum+angle pulse length generated.
Pulse = 0;
}
0
CPSC 483 - Spring 1999
Autonomous Tracking Unit
Servo Control Components
CPSC 483 - Spring 1999
Autonomous Tracking Unit
CPSC 483 - Spring 1999
Autonomous Tracking Unit
Camera Interface II
Memory and Camera
Control
Camera Interface Module
QuickCam
Start
Hold
Send
Valid
Data (8)
Nibble (4)
PCAck
Reset
Command
CamRdy
Command (8) • Remove asynchronous data exchange
• Reduce complexity of controlling logic
• Provide a configurable initialization sequence
Goals:
CPSC 483 - Spring 1999
Autonomous Tracking Unit
CPSC 483 - Spring 1999
Autonomous Tracking Unit
Camera Interface II Components
CPSC 483 - Spring 1999
Autonomous Tracking Unit
CPSC 483 - Spring 1999
Autonomous Tracking Unit
Camera Interface II Success!
• Testing Issues– BSII cannot trigger on an edge of a signal
– Camera’s CCD decays at a rapid rate
• Performance Measurements– Store a 128x120 image in 32 ms (31 fps)
• Future Improvements– Implement the byte mode transfer
– Implement memory storage logic inside the interface
CPSC 483 - Spring 1999
Autonomous Tracking Unit
CPSC 483 - Spring 1999
Autonomous Tracking Unit
Memory & Camera Control (MCC)
“Memory System: This will involve implementing an SRAM interface that will store image representations and allow for them to be accessed by other system components.”
CPSC 483 - Spring 1999
Autonomous Tracking Unit
• Isolates the Algorithm from the details of memory access and camera control
• This module went through several design changes
Original MCC Design
CPSC 483 - Spring 1999
Autonomous Tracking Unit
• Asynchronous communications with the Algorithm and the Camera Control Module
• 256x243 pictures and 6-bit pixels
• Expected to use 256K of memory to store 3 frames
• Did not unpack the pixels
• Complex initialization sequence
• Tested in simulation
CPSC 483 - Spring 1999
Autonomous Tracking Unit
MCC Final Design
Frame (3 bit) Vertical Offset (7 bits) Horizontal Offset (6 bits)
Final Memory Addressing Scheme
CPSC 483 - Spring 1999
Autonomous Tracking Unit
• Fifth and final design
• One bit Command
• Streams data from the camera synchronously
• Camera Control module writes directly to memory
• 64K of addressable memory, 8K needed
• Reduced frame size to 128x120
CPSC 483 - Spring 1999
Autonomous Tracking Unit
•Tested with BS2 and Camera Control module
•Observed inconsistent results
•We made a separate project to unit test the MCC
•Worked almost perfectly in testing
•Needed to the camera’s cord!
MCC Final Design (Continued)
CPSC 483 - Spring 1999
Autonomous Tracking Unit
CPSC 483 - Spring 1999
Autonomous Tracking Unit
CPSC 483 - Spring 1999
Autonomous Tracking Unit
MCC Final Block Diagram
CPSC 483 - Spring 1999
Autonomous Tracking Unit
Our “Simple” Environment
• Single rigid object
• Object is darker than its background
• Object is smaller than its background
• Object is travelling at a reasonable pace
CPSC 483 - Spring 1999
Autonomous Tracking Unit
CPSC 483 - Spring 1999
Autonomous Tracking Unit
Algorithm Test & Development
• Goal was to develop and test several algorithms
• Use software to help us visually compare solutions
• Tests were used to compare performance in different scenarios
CPSC 483 - Spring 1999
Autonomous Tracking Unit
CPSC 483 - Spring 1999
Autonomous Tracking Unit
Algorithm
• Step 1: Create a histogram of the object
• Step 2: Locate edges by subtracting
• Step 3: Calculate the center of the object
• Step 4: Adjust camera
CPSC 483 - Spring 1999
Autonomous Tracking Unit
CPSC 483 - Spring 1999
Autonomous Tracking Unit
Algorithm Issues
• Problems:– No object in frame
– Partial object in frame
– Handshaking timing issues
• Solutions:– No object: Use buffers to filter out non-significant deltas
– Partial object: Make frame extremes default boundaries of the object
– Timing issues: Add more states to ensure correct propagation
CPSC 483 - Spring 1999
Autonomous Tracking Unit
CPSC 483 - Spring 1999
Autonomous Tracking Unit
System Integration
• Phase 1: Integrate Camera Interface Module and Memory & Camera Control Module– Simulated in Xilinx
– BSII used to simulate algorithm module
– Capture data with PC serial port
• Phase 2: Algorithm Module Revision– Image simulated with a Xilinx stub module
– Unit integrated with one dimensional movement
– Two dimensional movement implemented after testing horizontal and vertical tracking individually
CPSC 483 - Spring 1999
Autonomous Tracking Unit
CPSC 483 - Spring 1999
Autonomous Tracking Unit
System Integration
• Phase 3: Performance Improvement.– Initial unit integration with the Algorithm Module on a 500kHz
clock
– Increasing Algorithm Module clock resulted in choppy movement, due to Servo operation
– Frame clock used to synchronize Servo operation
– Decreasing the frame clock period from 20.2ms to 17.0ms increased the image processing rate
CPSC 483 - Spring 1999
Autonomous Tracking Unit
CPSC 483 - Spring 1999
Autonomous Tracking Unit
Performance(Frame Clock = 20.2 ms)
Operation Phase Time (ms) Description
Download & Store Image 33.20Time for the image to be retrieved from thecamera and stored in SRAM.
Process Vertical Position 8.5Time to retrieve image data from SRAM andcalculate the objects vertical position.
Wait to Process Horizontal Position 8.3 Wait for Frame Clock
Process Horizontal Position 7.5Time to retrieve image data from SRAM andcalculate the objects horizontal position.
Wait to Capture Next Image 2.6 Wait for Frame Clock
Total 60.1 Time to process one image
CPSC 483 - Spring 1999
Autonomous Tracking Unit
16.6 images per second
CPSC 483 - Spring 1999
Autonomous Tracking Unit
Performance Improvement(Frame Clock = 17 ms)
Operation Phase Time (ms) Description
Download & Store Image 33.20Time for the image to be retrieved from thecamera and stored in SRAM.
Process Vertical Position 8.5Time to retrieve image data from SRAM andcalculate the objects vertical position.
Wait to Process Horizontal Position 0.640 Wait for Frame Clock
Process Horizontal Position 7.5Time to retrieve image data from SRAM andcalculate the objects horizontal position.
Wait to Capture Next Image 0.840 Wait for Frame Clock
Total 50.68 Time to process one image
CPSC 483 - Spring 1999
Autonomous Tracking Unit
19.7 images per second
18.7% improvementCPSC 483 - Spring 1999
Autonomous Tracking Unit
Possible Improvements
• Use 2 SRAM chips simultaneously– Image is retrieved and processed from one chip while the next
image is being stored
– Pin limitations
• Implement entire unit in Xilinx– Use internal RAM to store histogram information
– 130 11-bit storage locations required
– 45 CLBs required to store histogram information
CPSC 483 - Spring 1999
Autonomous Tracking Unit
CPSC 483 - Spring 1999
Autonomous Tracking Unit