Design of Embedded Iris Recognition System Based on ARM

Abstract:

An efficient portable embedded iris recognition system based on ARM is built. The

hardware system is composed of ARM S3C2410 processor, OV7620 USB Camera, and

TFT LCD. The system design includes image acquisition module, ARM processing

module, Linux platform and the recognition algorithm based on this platform. The test

results shows that the S3C2410 can read image data from the USB camera, store the

image data into NAND flash memory and gives the recognition result and preview on

the LCD in an acceptable time. Also the system is smaller, lighter, faster and with low

power consumption, so it is more convenient than the PC-based iris recognition system.

The use of open source code makes the software development on Linux free. This

biometric system works in identification mode i.e, the system recognizes an individual

by searching the templates of all the users preset in the NAND flash for a match.

Hardware and Software Requirements

Hardware Units:

1. ARM Core Board with Samsung S3C2410 chip whose CPU core is ARM 920T RISC

processor.

2. USB camera which has OV7620 CMOS image sensor, fixed focus lens and maximum

resolution of 640 x 480 pixels.

3. 3 groups of IR LEDs of 700-900nm for Infra red illumination.

4. IR SHAR sensor GP2D12 to measure distance between user and camera.

5. MEGA8 MCU for illumination control & to control the distance measurement module.

Software Units:

1. Boot-up, initialization and Embedded Linux2.6.12 operating System.

2. OV511+ driver to control the image sensor and communicate with ARM core

through USB port.

3. V4L(Video4Linux) kernel driver to transform the raw image data into YUV format.

4. Image capturing program loaded into NAND Flash memory.

5. Iris image data transmission.

6. Image preprocessing.

7. Feature extraction and coding.

8. Feature Matching and Result display

Hardware and Software Architectures

Hardware architecture of main processing chip (S3C2410):

Diagram and features taken from datasheet of S3C2410

Architecture and features of S3C2410

Page 2

Integrated system for hand-held devices and general embedded applications

16/32-Bit RISC architecture and powerful instruction set with ARM920T CPU core

Enhanced ARM architecture MMU to support WinCE, EPOC 32 and Linux

Instruction cache, data cache, write buffer and Physical address TAG RAM to

reduce the effect of main memory bandwidth and latency on performance

ARM920T CPU core supports the ARM debug architecture.

Internal Advanced Microcontroller Bus Architecture (AMBA) (AMBA2.0, AHB/APB)

To reduce total system cost, the S3C2410 also provides the following features:

separate 16KB Instruction and 16KB Data Cache

LCD controller (STN & TFT)

NAND Flash Boot loader

System Manager (chip select logic, SDRAM controller)

3-channel UART with handshake

4-channel DMA

4-channel Timers with PWM

I/O Ports

RTC

8- channel 10-bit ADC and touch screen interface

IIC-BUS interface

IIS-BUS interface

USB Host

USB Device

SD Host & Multimedia Card Interface

Clocked up to 203MHz

Page 3

Software Architecture:

The embedded Linux2.6.12 is a kind of miniature operating system, which is designed

for the demand of the embedded OS .It has some advantages, such as small code

amount, fast running speed, strong stability, open source software, efficient and steady

kernel, friendly develop environment, being compatible to many hardware platforms,

etc.. And this OS cuts out the normal Linux and becomes much smaller in size. It can

even be solidified in a memory chip with a few KB or MB. The kernel of Linux2.6.12 can

be customized by development engineers in terms of the actual demand. So it is

regarded as the ideal software platform to develop embedded application programs

Embedded Linux Operating system architecture

The Linux kernel is a core piece of the system. It provides the following major features: Process management Memory management Inter-process communication, timers Device drivers for the hardware: input, sound, network, storage, graphics, data

acquisition, timers, GPIO, etc. File systems Networking Power management The Linux kernel has thousands of options to selectively enable or disable features

depending on the system needs.

The Linux kernel runs in privileged mode. It can access and control the hardware. Its role is to multiplex the available resources, and provide a coherent and consistent interfaces toUser space.User space is the set of applications and libraries that run on the system. They work in unprivileged mode. They must go through the kernel to access the system resources, including the hardware. The kernel provides isolation between applications.

The Linux kernel has support for many common communication busses I2C SPI CAN 1-wire SDIO USB

And also extensive networking support Ethernet, Wifi, Bluetooth, CAN, etc. IPv4, IPv6, TCP, UDP, SCTP, DCCP, etc. Firewall, advanced routing, multicast

Page 4

Extra functional Properties:

A database can be maintained to store the iris images of all the employees and also their details. The system can then be used as Attendance System that notes the date, time and name of the employee. Hence a keypad can be provided to register employee details.

It can be used as security system by allowing only authorized people to enter the building by adding extra features to control the opening of the door. It can also be used to prevent the voter fraud in elections.

Also iris images of criminals can also be stored and if any of the images matches with these records, a message can be sent to police station through GSM modem.GSM modem can be connected to COMM1/COMM2 port of S3C2410. System Related Family Of Design:

1. Efficient Embedded Fingerprint-based User Authentication.2. Embedded Video Surveillance Systems3. Design of ARM-Linux based Video Surveillance System4. Small Embedded Vision System Using Microcontroller and CMOS Image Sensor5. Design of embedded automatic face recognition system based on Linux6. Design of Security System Using Embedded Linux Based Video Streaming

Modular Design:A) System Hardware Design:

The whole system is composed of following parts:

1. Iris image acquisition camera with the assistant module including infrared (IR)

illumination and distance measurement module controlled by a MCU;

2. An ARM core board to run the Linux system and iris recognition programs on it;

3. A LCD to preview the captured images and give the user some indication.

The system’s diagram is as shown below:

Page 5

1. ARM Core Board:

This board is the central module of the whole embedded recognition system. Its main

parts include S3C2410 (main processing chip), memory, power, and many global

interfaces.

1.1 Main processing chip

This main processing chip connects devices like

a) USB camera through USB device interface

b) The MCU for illumination and distance measurement controlling through UART0

c) Memories (NAND Flash, SDRAM etc.) through specified controlling interfaces.

d) A LCD through LCD controller interface.

The CPU is adopted to run the embedded Linux operating system and iris

recognition programs that run on Linux

It also controls data transmission between different parts of the system through

interfaces. The iris image flow into CPU through USB port, distance value flow into CPU

through UART0 port, the iris recognition result is shown out through LCD control port.

1.1.1 MemoryTwo kinds of memories SDRAM and Flash are used. The SDRAM has a fast speed for reading and writing. It reaches 1064Mbps.But the data stored in SDRAM will be lost without power supply. The Flash memory can store data no matter with or without power supply, but its speed is not as fast as SDRAM. So we use a SDRAM as the cache. The NAND Flash is adopted to store programs and the result of recognition. Some iris images and features are also stored in NAND Flash.

Page 6

1.1.2 InterfacesPlenty of interfaces are contained on the ARM core board, including USB2.0, SPI, IIC, UART, Ethernet interface, etc.That makes it easy and convenient to connect and communicate with other devices. The main ports used are USB and UART, which can transfer the data and control signals between the image acquisition module and the main core board.

1.1.3 Power supplyPower supply is definitive to whether the whole recognition system can work steadily or not. This system is powered by a 5V voltage source. But not every parts of the system can work under this voltage, so one must design respective powersupply for each of them. For embedded iris recognition system, it must be made sure that the kernel and the I/O port are powered at the same time. Otherwise, the chips may be damaged. In this design, MIC5207 is adopted for power supply.

2. The image acquisition moduleProviding high quality images are essential for high accuracy rate of recognition algorithm. This module combines the

Infra Red illumination, USB camera and Distance measurement

as the whole module for clear image acquisition.The basic structure of camera module and its assistant components are as shown below

2.1 IlluminationWith Infrared illumination, the iris image camera can get the iris textures more clearly. IR has another advantage than visible light illumination in iris image acquisition: it can not be felt by people’s eye. That can avoid causing users uncomfortable feeling. 3 groups of IR LEDs of 700-900nm for illumination are used in this design. Its light and close states are controlled by the MCU. The LEDs are easily controlled according to system’s need through sending different signal.

2.2 USB camera

Page 7

A fixed focus lens is used in the camera. Users need move forward and backward to image the iris in focus. And some additional methods should be adopted to help people find the best position to get an iris image good enough. That may be not convenient, but such an optical lens will bring great advantages such as small size and low cost. Since the use of fixed focus, the depth of field is fixed.

The fixed focus is related with the Lens’ relative aperture F, Diameter of pupil D, Focal length f, Distance between the user and the image sensor and Diameter of permitted “blur spot” which is decided by the pixel size of the

image sensor.According to these relationships, several different lens to image the user’s iris at different distances can be designed.

The image sensor used is OV7620. Its maximum resolution is 640×480. The power consumption is less than 120mW at work state and less than 12uW at standby state. The most important factor of this sensor is that it has high sensitivity to IR. The figure below shows the monochrome response of the sensor.

It is evident from the above figure; the sensor still has response to visible light. So a special filter is added in front of lens in order to get a clearer imaging under IR illumination.

2.3 Distance measurementAs the depth field is fixed, the user should stand at a certain distance to get the iris image in focus. For giving the user some cues to move to the specific area rapidly, an IR SHARP sensor GP2D12 to measure this distance is used. It can give the user’s distance precisely in real time. If the distance is not appropriate for clearly imaging, it can also sendsignals to the processing algorithm so that the blurred image will be discarded for the recognition. A MCU is adopted to control the distance measurement module. The value of

distance between user and camera will be read by this MCU. Here we use a MEGA8 MCU to process the distance data. This MCU is also used in illumination control.

Page 8

3. Display A TFT LCD is employed in this design to give some cues to the user and display the recognition result. It also supports a preview for the camera images. As S3C2410 contains a LCD controller and has the special display port, it is convenient to connect the LCD to the ARM core and drive it directly.

B. Design of Software Platform

1. Embedded Linux operating system The embedded Linux 2.6.12 operating system is selected to run recognition algorithms. In this system, embedded Linux operating system provides process management,memory management, document management, V4L data form transform and devices control. It controls all the communication between the hardware systems, such as the image acquisition and data transfer. Besides this, the recognition algorithm and the software also run on this platform. The system’s software structure is as shown below.

2. Software ModulesThere are mainly 3 function modules and they are

Iris image data acquisition and transformation module, Image recognition module and Image result display module.

Iris image data acquisition and transformation module: In this module, the camera is initialized and the image data is read from camera only if the distance between the user and the camera is appropriate for clearly imaging, the system will translate image data into YUV form and sends it to Image recognition module.Image recognition module : The iris image is first preprocessed and features are extracted and matched with standard iris feature code by the recognition algorithm.Image result display module: The result is shown through LCD in this module.

The workflow of the system is as shown below:

Page 9

2.1 Image data acquisition and transformation moduleThe devices driver is a very important part in Linux system. It establishes standard abstract interfaces for devices. With camera driver we can open/close and read the camera like common files. OV511+ driver is used to control the image sensor. The OV511+ communicates with ARM core through USB port, so the system should recognize USB device automatically, and load the OV511+ driver into kernel at start. Information and data from camera can be obtained by calling corresponding functions directly.1. Now the raw data from camera is not suitable for iris recognition. The way to resolve

this is transforming the raw iris image data into the format which is appropriate for the recognition algorithm. V4L(Video4Linux) kernel driver is used to transform the raw image data into YUV format. Video4Linux provides a unified application programming interface for the USB camera.

2.3 Iris recognition application programThe iris recognition process contains three steps and they are

Image pre-processing, Feature extraction and Feature matching/decision making.

Page 10

Image data acquisition and transformation module

Image Recognition Module

Image Result Display module

The whole iris recognition work flow as shown below:

2.3.1 Image pre-processingBefore feature extraction, some processing must be done in order to gain a better recognition result. Pre-processing operates in two steps.

First, localize the iris and remove the “unuseful” parts (e.g. eyelid, pupil etc.) from the original image. In this design the iris’ outer edge cursorily is found using histogram analyzing first, then using a Hough transform to confirm it accurately.

Second, normalize the localized iris in order to achieve invariance to the varying size of the pupil and to distance and angle of the image acquiring framework.

2.3.2 Feature extraction and codingThe useful information from irises is contained in abundant texture. Unlike other texture-based biometric identification technologies, it is difficult to classify and localize semantically meaningful features in an iris image. A 2-D Gabor filter is used to extract feature from iris image. Gabor filter was defined by D.Gabor in 1946. J.Daugman expanded it into 2-D form in 1980 and further elaborated its character in 1985. J.Daugman pointed out that using 2-D Gabor filter could get frequency and direction selectivity simultaneously when achieved space localized. Such characters are very available for texture analysis. The iris image is segmented into several blocks and frequency and direction information is acquired for each sub-block using 2-D Gabor filter. Then code the feature information depending on such information.

2.3.4 Feature MatchingWe match object iris image and standard iris image by judging the Hamming distance between their feature codes in this design. Hamming distance is the number of different values on corresponding Ds of two codes. The larger the Hamming distance between two codes, the more difference of those two codes. We set a threshold value about the Hamming distance. If the value of Hamming distance is smaller than this threshold value, the two iris image should come from a same person, on the contrary, they come from the different person.

UML Diagrams:

Page 11

Use case Diagram for Administrator

User interface Design :The LCD acts a user interface and the sample images displayed on the LCD screen are as shown below

A GUI user interface can be designed as shown below for enrollment:

This is L.G’s trandmarked IrisAccess3000 user interface for enrollment.

Mapping:

Page 12

Iris Recognition SystemSoftware

Iris image Acquisition and Transformation

Display Iris Image Recognition

Context Level Diagram:

Level 1 Data Flow Diagram

Mapping Level-1 DFD

Page 13

Iris image Acquisition and Transformation

Read Image Read Distance

Validate

Transform Image Discard Image

Display Cues

Level-2 DFD for Image Acquisition and Transformation Process

Mapping Acquisition and Transformation Process

Level-2 DFD For Iris Recognition Process

Page 14

Get Frequency Get Direction

Get Features

Feature Selection

Segmentation

Feature Extraction and Coding

Feature Coding

Level-3 DFD for Pre-processing Process

Level-3 DFD for Feature Extraction and Coding Process

Mapping Feature Extraction and Coding Process

Page 15

Level-3 DFD for Feature Matching Process

Mapping Feature Matching Process

Refinements :The main frequency of S3C2410 is 266 MHz. Its processing speed is up to 290MIPS. Taken the embedded Linux system and I/O controlling into account, the processing ability for iris recognition will be about 200 MIPS. This is fast enough for real-time iris image acquisition and recognition. Through transplantation and optimization, we can shorten this processing time. So this system has the capacity to process the iris image data and give the recognition result in an acceptable time.

References:[1] WANG Yuanbo, HE Yuqing, HOU Yushi and LIU Ting, “Design method of ARM based embedded iris recognition system”, Proc. Of SPIE Vol. 6625 66251G-1

Page 16

Display Image Display Not Matched

Compare Hamming Distance

Feature Matching