MTA-SZTAKI

Roboteyepair

an ultra-high speed stereo vision system

based on CNN technology

MTA-SZTAKI

Motivation

• The mechanical properties of robots enable faster operation than their current vision systems;

• A high-speed, medium resolution, low cost vision system has many potential application fields in the industry;

• Both stereo and mono versions have potential applications;

MTA-SZTAKI

Goals

Main characteristics of the Roboteyepair:– Standalone vision system;– Capable to capture and process image pairs at

frame rates up to 5000 per second; – Medium resolution;– High performance focal plane CNN processors;– Equipped with serial and Ethernet communication

ports; – Developed for industrial use.

MTA-SZTAKI

The CNN visual processor• 128x128 sensor-processor array;• Optimized for performing array operations

on analog and logic values;• Time to perform a typical operation:

– Logic: 300ns (morphology, binary

logic operations, skeletonizing, etc.)– Analog: 3-5ms (diffusion, edge detection, Sobel, etc..)

• 8 internal image memories;• 128Mpixel/sec I/O speed (128 internal AD/DA)• 0.35mm technology, 12x12mm silicon surface;• Operates as a coprocessor, a master processor is needed

MTA-SZTAKI

Block diagram of the system

MTA-SZTAKI

Realization of the system

Ethernet 10/100 (Rj-45)

Serial Port (RS-232)

Clock circuit

JTAG connector

Reset circuit

Power block

ETRAX 100 communication processor with Linux kernel

Texas TMS320C6202

DSP @250MHz

Clock

JTAG

RS

1.8V 3,3V

XBUS

EMIF CONTROL ADDRESS DATA

PLDs

Flash 16 Mbit

Bus Interface

SDRAM 4Mx16

Fast Platform Bus III

optical input

32

Universal interface board

CVM platform

power block

2 CVM chips

analog test interface

MTA-SZTAKI

The finished boards

MTA-SZTAKI

The complete system

MTA-SZTAKI

Programming the DSP

• Image processing– Native languages (AMC, Alpha, C)

• Handling two CNN chips– The target chip has to be specified for every

command– Synchronization

• Adapting CNN image processing library• Communication using the ETRAX chip

MTA-SZTAKI

Programming the communication processor

• ETRAX 100 2.4 LINUX kernel– Driver (ETRAX XBUS)

• TCP/IP communication module (INET Daemon)– Assigns the communication driver to the given TCP port– standard input standard output redirecting

• A special daemon has also been implemented

– Boot device

MTA-SZTAKI

Monitoring software on the PC side

• Why is it needed:– Supporting system setup– Application development and testing – Parameter adjustment– Testing camera parameters

• Capabilities:– Multi-window graphical interface– Editor, compiler and software downloading

devices

MTA-SZTAKI

The project will be finished in May, 2003

MTA-SZTAKI

• Stereo machine vision system for robots;

• Visual navigation for NASA Mars Mission 2007

• Non-invasive speed and displacement measurement, quality control of narrow textile bands and tapes (textile industry);

• Classification of large number of tiny objects (pills, grains, pebbles, etc.);

• Monitoring fast industrial processes (sparkplug);

Applications

MTA-SZTAKI

• Replace the current DSP with a TMSC3206115 DSP running @600 MHz

• Equip the system with an optional 1.3 MPixel CMOS sensor

• Support of communication via USB port• These developments are expected to be finished

by September, 2003

Future developments

MTA-SZTAKI

Thank you for your attention!