Bachelor Thesis - · PDF file1.2 Integration and software ... 8 API and project structure ... Bachelor Thesis

Ring-shaped barcodes Bachelor Thesis

August 2015

Robin Brügger University of Applied Sciences and Arts Northwestern Switzerland FHNW

[email protected]

Supervisor: Prof. Dr. Christoph Stamm

University of Applied Sciences and Arts Northwestern Switzerland FHNW [email protected]

Customer:

Paul Glendenning NANO 4 U, 6060 Sarnen

[email protected]

Abstract: Previous work has shown that a specialized, ring-shaped barcode format is preferable for

certain applications. This thesis develops a new ring-shaped, two-dimensional barcode format. The

encoder generates an image of a barcode containing the payload data. The decoder recognizes the

barcode and reads the payload data. The decoder is capable of dealing with perspective distortion by

correcting it prior to reading the payload. Barcodes with small defective areas can still be read thanks

to ECC (Error-Correction-Codes).

mailto:[email protected]



Bachelor Thesis (IP6) Ring-shaped barcodes Robin Brügger

2 / 36

Table of contents

1 Introduction ..................................................................................................................................... 4

1.1 Project goals and requirements .............................................................................................. 4

1.2 Integration and software ......................................................................................................... 4

1.3 Existing circular barcodes ........................................................................................................ 4

1.4 Schedule .................................................................................................................................. 5

2 Ring-shaped barcode format ........................................................................................................... 8

2.1 Usable payload and limitations ............................................................................................... 9

3 Encoder (Barcode generator) ........................................................................................................ 10

3.1 Arguments ............................................................................................................................. 10

3.2 Implementation ..................................................................................................................... 10

4 Recognition and decoding ............................................................................................................. 11

4.1 Preparations .......................................................................................................................... 11

4.2 Decoder pipeline ................................................................................................................... 11

4.3 Barcode recognition .............................................................................................................. 12

4.3.1 Observations .................................................................................................................. 12

4.3.2 Blob filter ....................................................................................................................... 13

4.3.3 Improvements to the OpenCV SimpleBlobDetector ..................................................... 13

4.4 Find the perspective correction markers and the startmarker ............................................. 14

4.4.1 Observations .................................................................................................................. 14

4.4.2 Algorithm ....................................................................................................................... 14

4.4.3 Common problems and their solutions ......................................................................... 15

4.4.4 Increasing accuracy ....................................................................................................... 15

4.5 Rough perspective correction ............................................................................................... 17

4.5.1 Transformation .............................................................................................................. 17

4.5.2 Reason for the limited accuracy .................................................................................... 18

4.6 Read radial timing information ............................................................................................. 18

4.6.1 Algorithm ....................................................................................................................... 19

4.6.2 Verification .................................................................................................................... 19

4.6.3 Selection of a square ..................................................................................................... 20

4.7 Accurate perspective correction ........................................................................................... 20

4.8 Read vertical timing information ........................................................................................... 20

4.8.1 Algorithm ....................................................................................................................... 21

4.9 Evaluation of the payload modules ....................................................................................... 21

4.9.1 Grid ................................................................................................................................ 21


3 / 36

4.9.2 Reading a module .......................................................................................................... 22

5 Protocol ......................................................................................................................................... 23

5.1 Error correction codes ........................................................................................................... 23

6 Tests............................................................................................................................................... 24

6.1 Tests using a webcam ............................................................................................................ 24

6.2 Batch testing robustness ....................................................................................................... 24

6.2.1 Added defects ................................................................................................................ 24

6.2.2 Tests ............................................................................................................................... 25

6.2.3 Interpretation ................................................................................................................ 27

6.3 PSNR (peak signal-to-noise ratio) .......................................................................................... 28

6.4 Performance .......................................................................................................................... 28

7 Further work and improvements .................................................................................................. 30

7.1 Scanning of actual coded products ....................................................................................... 30

7.2 ECC license ............................................................................................................................. 30

7.3 Parametrizing the decoder .................................................................................................... 30

7.4 Avoid data looking like a startmarker ................................................................................... 30

7.5 Error detection ...................................................................................................................... 31

7.6 Error correction in length field .............................................................................................. 31

8 API and project structure .............................................................................................................. 32

8.1 Project structure .................................................................................................................... 32

8.2 API.......................................................................................................................................... 32

8.2.1 EncoderLib ..................................................................................................................... 32

8.2.2 DecoderLib ..................................................................................................................... 32

9 Personal reflection ........................................................................................................................ 33

9.1 Acknowledgments ................................................................................................................. 33

10 Sources .......................................................................................................................................... 34

11 Academic honesty statement ........................................................................................................ 35

12 Appendix ........................................................................................................................................ 36

12.1 Glossary ................................................................................................................................. 36

12.1.1 Module .......................................................................................................................... 36

12.2 Software used ........................................................................................................................ 36

12.2.1 Libraries ......................................................................................................................... 36

12.2.2 Others ............................................................................................................................ 36


4 / 36

1 Introduction

NANO 4 U develops counterfeit prevention solutions for products. For certain products and applica-

tions, a ring shaped 2D barcode is considered to be more efficient than a standard square 2D bar-

code.

1.1 Project goals and requirements

The goal of this project is to develop a ring-shaped barcode format. For the specific applications of

the barcode, NANO 4 U has the following requirements:

Payload: The usable payload of the barcode should be at least 25 characters, resulting in 25

bytes.

Size: Smallest module size of at least 0.2mm or 5 times the lateral resolution of the scanner.

The maximal outer diameter of the barcode is 8mm.

Form: Ring-shaped.

The project asks for the following software components:

Encoder: Generates a new barcode as an image. Takes the payload, desired module size, in-

ner and outer radius as arguments und creates a barcode accordingly. The encoder is de-

scribed in the chapter Encoder (Barcode generator).

Decoder: Detects a barcode in an image and is able to recover the original payload data. The

decoder is covered in detail in the chapter Recognition and decoding.

Tests: It is not possible within the scope of this project to evaluate the results of the ring-

shaped barcodes on real products. The robustness of the algorithm shall therefore be tested

using software to distort and add noise to the barcodes.

1.2 Integration and software

Because it allows for a platform-independent solution

and easy integration into other software systems, C++

was chosen as the programming language for this

project.

I selected OpenCV as computer vision library because

of its excellent documentation, vast functionality and

my good experiences with it.

1.3 Existing circular barcodes

ShotCode is currently the only existing circular 2D

barcode1. An example is shown in Illustration 2. It was

developed to share links and can be read with low

resolution mobile phone cameras or webcams. Today,

it has been replaced by QR-Codes.

1 Barcoding Connected (2009): SHOTCODE BARCODE: THE CIRCULAR 2D BARCODE FOR MOBILE TAGGING. http://blog.barcoding.com/2009/02/shotcode-barcode-the-circular-2d-barcode-for-mobile-tagging/ (Accessed August 5, 2015)

Illustration 1: Shotcode4


5 / 36

ShotCode is not suitable for this project for two reasons:

Limited payload

Depending on the source, a ShotCode is limited to 492 or 403 bits (5 to 6 bytes) of data. How-

ever, NANO 4 U requires about 25 bytes.

Centre is not empty

A ShotCode uses the area in the centre of the circle. This area is not available for use on the

end-user products and has to be left blank.

1.4 Schedule

The project started on 24 February 20154with the deadline on 15 August 2015. Due to the unfore-

seeable challenges of decoding a barcode, it was not possible to break the decoding process down

into individual steps and schedule them separately. Illustration 2 shows the planned and effective

project schedule.

2 Wikipedia (2013): ShotCode. https://de.wikipedia.org/wiki/ShotCode (Accessed June 7, 2015) 3 Wikipedia (2014): Shotcode. https://en.wikipedia.org/wiki/ShotCode (Accessed June 7, 2015) 4 Mudie, Stuart: Shotcode. Licensed under CC BY-SA 2.0 via Wikimedia Commons - https://commons.wikimedia.org/wiki/File:Shotcode.png#/media/File:Shotcode.png


6 / 36

Illustration 2: Planned and effective project schedule


7 / 36

The workload for the bachelor’s thesis should be approximately 360 hours, spread over the whole

project duration. Because of lectures, exams and special occasions such as the project week, the

scheduled work hours varied from week to week. Illustration 3 shows the planned and effective work

hours per week. During the course of the project, slightly fewer hours were invested than planned.

However, this was made up in the last two weeks of the project, which had been allocated as re-

serve.

Illustration 3: Planned and effective work hours


8 / 36

2 Ring-shaped barcode format

In this chapter the developed barcode

format is explained. Illustration 5 is an

original barcode as it is generated by

the encoder. Illustration 6 is the same

barcode, but areas for recognition, per-

spective correction and timing are high-

lighted. The coloured areas serve the

following purposes:

Yellow: Solid black ring for lo-

cating the barcode in an image.

Orange: Markers for a rough

perspective correction. They are

arranged in a perfect square.

Green: The startmarker defines

the beginning in the otherwise

continuous circle. It is also used

to determine the outer radius of

the barcode.

Red: The vertical zebra pattern

determines the vertical timing

of the rings (how far apart the

rings are).

Violet: The radial zebra pattern

determines the angles between

the modules. It is also used to

achieve an accurate perspective

correction, therefore the radial

module count is always divida-

ble by 8. Thus, it is possible to

select four modules of the zebra

pattern which lay in a perfect

square, facilitating the perspec-

tive correction calculation.

Blue: Area used for the payload.

The data area can be interpret-

ed as a curved table, addressing

modules via their x and y index.

Illustration 7 shows how the ad-

dressing of the payload area is

organised. To make use of the

bigger modules toward the outside of the barcode, these modules are filled with payload

first.

Illustration 4: Ring-shaped barcode generated by the encoder

Illustration 5: Ring-shaped barcode with special areas highlighted


9 / 36

Illustration 6: Addressing of the payload modules

2.1 Usable payload and limitations

The barcode used in Illustration 5 and Illustration 6 was generated following the size requirements

described in the chapter Project goals and requirements and is therefore considered realistic in the

application at hand. It has a maximum payload of 56 bytes. However, some of that is consumed by

the protocol described in the chapter Protocol.

The maximum payload the ring-shaped barcode supports is 234 bytes. This is due to constraints of

the payload protocol. The barcode design itself poses no upper bound for the payload size.


10 / 36

3 Encoder (Barcode generator)

The encoder generates barcodes as an image. It takes the payload and three sizing arguments as

input.

3.1 Arguments

The size and shape of the barcode is determined by

three basic parameters, measured in pixels. Illustra-

tion 8 visualizes the meaning of the parameters.

Minimal module size (red): The minimal

size of a single module. This is also the

thickness of the individual rings. Technical

name: moduleSizePixels.

Minimal radius (blue): Distance from the

centre of the barcode to the inner perspec-

tive markers. The perspective markers are

the smallest module-sized structure in the

barcode. Technical name: minRadiusPixels.

Maximum radius (orange): Distance from

the centre of the barcode to its outermost

point. Technical name: maxRadiusPixels.

The barcode used in Illustration 8 was generated

using moduleSizePixels=8, minRadiusPixels=80, maxRadiusPixels=160.

3.2 Implementation

Implementing the encoder was a straightforward process. It uses the OpenCV drawing functions,

particularly the circle() function which is able to draw circle sectors. However, OpenCV does not pro-

vide a function to draw ring segments. Therefore, the barcode is rendered from the outside to the

inside. Firstly, the modules in the outermost circle are drawn as circle sectors. Upon completion of

that ring a slightly smaller white circle is drawn, transforming the already drawn circle sectors to ring

sectors. This process is repeated until the barcode is completely rendered. Illustration 9 provides

pseudocode for this process.

Illustration 7: Basic barcode encoder parameters

val minRadiusPixels, maxRadiusPixels, moduleSizePixels

var currentRadius = maxRadius

while(currentRadius >= minRadius)

for(angle : allAngles)

if(black) drawCircleSegment(angle, currentRadius, black)

drawCircle(currentRadius - moduleSizePixels, white)

curentRadius -= moduleSizePixels

Illustration 8: Pseudocode of the barcode encoder


11 / 36

4 Recognition and decoding

The decoder recognizes a barcode in an image and is able to read its payload. The payload itself is

subject to a simple protocol described in the chapter Protocol.

4.1 Preparations

The input for the decoder can be any image. It could be a greyscale image or even an image with RGB

colour channels. Because the barcode only features black and white modules, the input image must

be binarized. Selecting fixed threshold values or an adaptive threshold has not yielded good results in

correctly binarizing black modules as black and white modules as white. It is particularly challenging

that the scanned image not only contains the barcode, but also its irrelevant surroundings. The shape

and colour of these surrounding areas can vary widely and cannot be predicted.

Satisfactory results were achieved with Otsu’s binarization method, which is explained on “The Lab

Book Pages”5 and is readily available in OpenCV. Using this method, the input image is binarized be-

fore entering the decoder pipeline.

In order to cope with noisy images, a Gaussian blur with a 5x5 kernel is applied prior to binarization.

4.2 Decoder pipeline

Intuitively, the process of reading a barcode would be structured as following:

Illustration 9: Basic process pipeline of a barcode reader

Recognition: Finding the barcode in the image provided

Perspective correction: Perspective errors are caused by imperfect alignment of the barcode

and the camera or scanner. They need to be corrected before the payload can be read.

Timing information: Getting the size of the modules

Evaluation of the modules: The actual data reading process

5 The Lab Book Pages: Otsu Threshholding. http://www.labbookpages.co.uk/software/imgProc/otsuThreshold.html (Accessed August 1, 2015)

Recognition Perspective correction

Timing information

Evaluation of the modules


12 / 36

However, the decoder for the ring-shaped barcode cannot fully follow this clear process because, as

described in the chapter Ring-shaped barcode format, the radial zebra pattern contains the timing

information of the angles and is also used to achieve an accurate perspective correction. Therefore,

the decoder processes “perspective correction” and “reading timing information” are not strictly

sequential. Furthermore, a twostep approach is used for the perspective correction, resulting in the

process pipeline displayed in Illustration 11.

The following chapters will discuss the decoder pipeline in detail. Each stage of the pipeline works on

a best effort basis. If for example the recognition stage fails because it does not find a barcode, the

input image is discarded and the rest of the processing pipeline is not activated.

4.3 Barcode recognition

The goal of this processing step is to locate a ring-shaped barcode in an image and to estimate its

centre.

Input: Binarized image which potentially contains a barcode.

Output: Approximate centre, major and minor axis length of the central ellipse.

Barcodes are recognized using blob analysis. In Illustration 12

the area which is used to recognize a barcode is highlighted in

yellow. OpenCV provides the class SimpleBlobDetector. It al-

lows filtering the blobs by area, circularity, inertia, convexity

and colour6.

4.3.1 Observations

The following observations can be made on the input images

and lead to the filter settings of the blob detector:

The centre of the barcode is always a perfect circle.

However, due to perspective errors it can appear as an

ellipse on the input image. Henceforth, the central cir-

cular blob will therefore be referred to as ellipse.

The centre of the barcode is white.

The barcode’s size lies between certain boundaries. If it is bigger than the field of view of the

camera, it cannot be decoded because some information is not visible and therefore missing.

On the other hand, it can be expected that the barcode is not too small either.

6 Mallick, Satya (2015). Blob Detection Using OpenCV ( Python, C++ ). http://www.learnopencv.com/blob-detection-using-opencv-python-c/ (Accessed July 13, 2015)

Read radial timing

information

Accurate perspective correction

Read vertical timing

information

Evaluation of the modules

Recognition Find the perspective correction

markers and the startmarker

Rough perspective correction

Illustration 10: Process pipeline of the decoder

Illustration 11: The yellow area is used for the barcode recognition


13 / 36

4.3.2 Blob filter

Based on these observations, filters for the blob detector can be determined to find the barcode in

an image with high accuracy:

Circularity between 0.8 and 1.0, filtering out all objects that are not elliptic.

Colour must be white

Assuming the blob is perfectly circular, its radius must lay between 1/16 and 1/4 of the aver-

age image dimensions:

𝐴𝑣𝑔𝐷𝑖𝑚𝑒𝑛𝑠𝑖𝑜𝑛 =𝐼𝑚𝑎𝑔𝑒𝑊𝑖𝑡ℎ + 𝐼𝑚𝑎𝑔𝑒𝐻𝑒𝑖𝑔ℎ𝑡

2

𝑟𝑚𝑖𝑛 =𝐴𝑣𝑔𝐷𝑖𝑚𝑒𝑛𝑠𝑖𝑜𝑛

16

𝑟𝑚𝑎𝑥 =𝐴𝑣𝑔𝐷𝑖𝑚𝑒𝑛𝑠𝑖𝑜𝑛

4

𝑟𝑚𝑎𝑥2𝜋 > 𝐴𝐵𝑙𝑜𝑏 ≥ 𝑟𝑚𝑖𝑛

2𝜋

4.3.3 Improvements to the OpenCV SimpleBlobDetector

The SimpleBlobDetector of OpenCV is able to reliably detect ring-shaped barcodes. However, it has

some shortcomings which are relevant for the decoder:

It is quite slow because SimpleBlobDetector binarizes the image before running the detec-

tion, which itself is based on an edge filter. Binarization is not done once, but around 20

times using different fixed values as threshold. For every threshold the complete blob detec-

tion algorithm is run. However, the binarization is superfluous because we already binarize

the image using Otsu’s method, as described in the chapter Preparations.

The algorithm returns a KeyPoint-Object which contains only basic information such as size

and location of the blob. However, it was desirable for the encoder to be able to access the

blob’s moments to determine the length of both the ellipse’s major and minor axis. This in-

formation is useful for the next step in the pipeline and makes it possible to mark the recog-

nized ellipse in the input image for debugging purposes.

I therefore adapted the SimpleBlobDetec-

tor class by adding the mentioned features,

basing my work on Joel Temply’s improve-

ments of the SimpleBlobDetector class7.

Illustration 12 shows debug output of the

detector. Thanks to access to the blob’s

moments, the detected ellipse can be

drawn quite accurately. The green dot

marks the estimated centre of the barcode.

Illustration 12: Visualized output of the recognition step

7 Teply, Joel (2014): http://stackoverflow.com/a/25152785/2037769 (Accessed June 15, 2015)


14 / 36

4.4 Find the perspective correction markers and the startmarker

The goal of this step is to find the four perspec-

tive markers and the startmarker of an image. In

Illustration 14 the perspective markers are high-

lighted in orange and the startmarker in green.

In an unmodified barcode the startmarker is

black and the perspective correction markers

are white. In a barcode without perspective

distortion the four perspective markers lay in a

perfect square.

Input:

Binarized image with barcode

Estimated barcode centre

Major and minor axis length of the cen-

tral ellipse

Output:

Start and end angle of the startmarker

relative to a horizontal line through the

barcode centre

Location of the perspective correction markers

4.4.1 Observations

This step is based on the following observations: Imagining a straight line from the barcode centre to

infinity in any direction and counting the black pixels in the first continuous stretch, the startmarker

is where there is the longest continuous stretch of black pixels. The perspective markers are where

there is the shortest one.

4.4.2 Algorithm

Based on these observations, the algo-

rithm uses a radial scanline revolving

around the estimated centre of the

barcode. The scanline starts at 70% of

the minor axis of the central ellipse. In

Illustration 15 this is shown as a green

circle. The end of the scanline is theo-

retically at infinity. However, because

the centre is enclosed by a circle of

black pixels, it ends quite soon in prac-

tical terms. Along the scanline the

length of the first continuous stretch of

black pixels is evaluated. Illustration 15

shows a possible plot of the first con-

tinuous black distance at different an-

gles. The four smallest values signify

the location of the perspective mark-

ers, the maximum is where the start-

Illustration 13: Perspective markers (orange) and startmarker (green)

Illustration 14: Radial scanline revolving around the estimated centre of the barcode


15 / 36

marker is located.

Illustration 15: Plot of the continuous amount of black pixels at various angles

4.4.3 Common problems and their solutions

Because of the small discreet angular steps

the scanline uses to revolve around the cen-

tre, it is possible that two of the four lowest

points belong to the same perspective mark-

er. Errors are avoided by enforcing a mini-

mum angular distance from one perspective

marker to the next.

Sometimes the longest continuous black dis-

tance is not the startmarker but an edge be-

tween modules. Illustration 17 shows such a

situation. Under the red line lies a continuous

stretch of black pixels, making its length equal

to that of the startmarker. In order to avoid

these errors, the found startmarker is verified

by checking that it is surrounded by continu-

ous white pixels on both sides.

4.4.4 Increasing accuracy

Only having found any direction in which the

startmarker lies is not accurate enough in

later stages of the decoding process. Neither is just knowing any positon within each perspective

marker. The positions need to be determined as accurately as possible. Increasing the accuracy is

done in the same way for both the startmarker and the perspective markers. The first continuous

black distance is measured at angles in close proximity to the recognized features. If the measured

Illustration 16: Possible maximum continuous black distance at a location other than the startmarker


16 / 36

black distance is less than 10% different from the original feature, it is considered as still being part of

that feature. To find the beginning and the end of the feature quickly, a binary search is used, yield-

ing a O(log n) complexity search for the feature edges. Illustration 17 shows the process of the search

for the startmarker edge. Starting from the originally detected feature (line 0), the continuous black

distance at the lines 1 to 4 is evaluated, resulting in the discovery of the accurate edge of the start-

marker. The binary search is halted when the step distance is smaller than an arbitrary defined ε.

Illustration 17: Binary search for the edge of the startmarker

Illustration 18 visualises the recognized startmarker and the perspective markers in the original im-

age the camera captured.

Illustration 18: Debug output showing the located startmarker and the perspective markers


17 / 36

4.5 Rough perspective correction

This step performs a rough perspective correction, making the subsequent steps of the decoding

pipeline easier.

Input:

Input image containing a barcode (In)

Location of the four perspective markers in the input image

Output:

Perspective corrected image (Out)

4.5.1 Transformation

This step makes use of the information that the four perspective markers lie in a perfect square in

any generated barcode. Due to the perspective distortion induced by the scanner, the detected per-

spective markers in the input image may not form a square, but any quad. It is possible to compute a

transformation matrix M between a square and the quad detected in the input image. Illustration 19

shows how this transformation matrix works. It can be computed using the OpenCV function getPer-spectiveTransform().

Illustration 19: Transformation matrix between the quad in the perspective distorted input image and a square8.

The inverse matrix of M is then applied to every pixel in the input image:

In: Input image

Out: Output image

(Inx, Iny): Pixel in the input matrix

M: Transformation matrix

⋀(𝐼𝑛𝑥, 𝐼𝑛𝑦) ∶ (𝑂𝑢𝑡𝑥 , 𝑂𝑢𝑡𝑦) = 𝑀−1 ∗ (𝐼𝑛𝑥, 𝐼𝑛𝑦)

The OpenCV function warpPerspective()9 provides this functionality. Illustration 20 shows an example

input and output image:

8 Stamm, Christoph (2014): Projektive Abbildung. Slides for the module magb, file 09_Bildmatrix.pdf 9 OpenCV (2015): Geometric Image Transformations. http://docs.opencv.org/modules/imgproc/doc/geometric_transformations.html#warpperspective (Accessed August 10, 2015)


18 / 36

Illustration 20: Input image on the left and output image on the right. The output image is perspective corrected.

4.5.2 Reason for the limited accuracy

The limited accuracy of this perspective correction is mainly due to the perspective markers being

located well within the barcode, on the inside of the section carrying the payload data. Any inaccura-

cy in the location of the markers is amplified in outer areas of the barcode. Therefore a more accu-

rate perspective correction is applied later in the decoding pipeline.

4.6 Read radial timing information

This step reads the radial timing information of

the barcode do determine the angle between the

data modules. It also sets the foundation for a

more accurate perspective correction. The radial

timing information is stored in the zebra pattern

around the outer edge of the barcode. In Illustra-

tion 22 this area is highlighted in violet.

Input:

Roughly perspective corrected input im-

age

Estimated centre of the barcode

Location of the startmarker

Output:

Radial module count

Location of four black modules belonging

to the zebra pattern. They form a square

in undistorted barcodes.

Illustration 21: The radial timing information is stored in the zebra pattern around the outside of the barcode. It is high-lighted in violet.


19 / 36

4.6.1 Algorithm

To read the zebra pattern, a scanline revolving around the estimated centre of the barcode is used.

The scanline starts just outside the barcode and scans towards the centre of the barcode trying to

find the black modules. The distance

to the occurrence of the first black

pixel is evaluated. If this distance is

within certain bounds around B, we

have found a black module. Illustra-

tion 23 visualises the path of the

scanline.

An initial value of B is determined by

the distance between the blue circle

in Illustration 23 and the outside of

the startmarker. Although the input

image has been roughly perspective

corrected prior to this processing

step, the correction might not be

perfect. Illustration 24 shows such a

scenario. B must therefore be con-

tinuously updated to reflect the local

situation. This is done according to

the following formula as the scanline

progresses:

S: Constant to regulate the

weight of newer measured distances against older ones

x: Measured distance between the blue ring and the black module in the zebra pattern

𝐵𝑛+1 =𝐵𝑛∗𝑆+𝑥

𝑆+1

4.6.2 Verification

Even with measures for dealing with an imperfect

perspective correction, reading the radial zebra

pattern occasionally fails. To prevent the decoder

from progressing with wrong settings, the recog-

nized black modules in the zebra pattern are sub-

ject to verification.

The angle between two black modules relative to

the centre of the barcode can easily be estimated:

𝑎𝑛𝑔𝑙𝑒𝐸𝑥𝑝𝑒𝑐𝑡𝑒𝑑 =360

𝑏𝑙𝑎𝑐𝑘𝑀𝑜𝑑𝑢𝑙𝑒𝑠𝐼𝑛𝑍𝑒𝑏𝑟𝑎

The zebra pattern has been successfully read if any

two neighbouring black modules fulfil the follow-

ing inequality:

|𝑎𝑛𝑔𝑙𝑒1 − 𝑎𝑛𝑔𝑙𝑒2| ≤=𝑎𝑛𝑔𝑙𝑒𝐸𝑥𝑝𝑒𝑐𝑡𝑒𝑑

2

Illustration 22: Scanline starting outside the barcode and revolving around its centre.

Illustration 23: Example of an imperfect perspective correc-tion


20 / 36

4.6.3 Selection of a square

Since the number of modules in the outer zebra pattern is dividable by 8, the number of black mod-

ules is dividable by four. Given an array with the position of every black module in the outer zebra

ring, selecting four of them lying in a square is trivial.

4.7 Accurate perspective correction

This step performs a second perspective correction to achieve the accuracy needed for reading the

modules.

Input:

Input image containing a barcode (In)

Location of four black modules of the outer zebra ring forming a square

Rotation matrix used in the rough perspective correction (M1)

Output:

Accurately perspective corrected image (Out)

The basic principle is the same as described in the chapter Rough perspective correction. A transfor-

mation matrix M2 is computed. This matrix M2 performs a mapping between the roughly perspective

corrected image and the accurately perspective corrected image. Since the aim is to avoid precision

loss by doing two transformations on an image (first a rough correction, than the accurate one), the

two perspective correction matrices M1 and M2 can be multiplied to allow transformation from the

input image to the accurately perspective corrected image in one step:

In: Input image

Out: Output image

(Inx, Iny): Pixel in the input matrix

𝑀𝐶𝑜𝑚𝑝𝑙𝑒𝑡𝑒 = 𝑀2 ∗ 𝑀1

⋀(𝐼𝑛𝑥, 𝐼𝑛𝑦) ∶ (𝑂𝑢𝑡𝑥 , 𝑂𝑢𝑡𝑦) = 𝑀𝐶𝑜𝑚𝑝𝑙𝑒𝑡𝑒 ∗ (𝐼𝑛𝑥 , 𝐼𝑛𝑦)

4.8 Read vertical timing information

The vertical zebra pattern defines how many rings a

barcode has and how far apart they are spaced. In

Illustration 25 the vertical zebra pattern is highlighted

in red.

Input

Accurately perspective corrected image

Radial module count

Angle of the startmarker

Output

Number of rings

Radius of the innermost ring

Radius of the outermost ring

Direction of reading the barcode: clockwise or

anti-clockwise Illustration 24: The area of the vertical zebra pattern is highlighted in red


21 / 36

4.8.1 Algorithm

The vertical zebra pattern is always situated next to

the startmarker. Because the barcode has possibly

been mirrored, it could lie either to the right or to

the left of the startmarker. Illustration 9 depicts how

two lines are shot from the barcode centre through

the potential locations of the vertical zebra pattern.

To evaluate if the line goes through a zebra pattern

the alternating continuous stretches of black and

white pixels are measured. Because all rings are

equal in thickness, a perfect zebra patterns would

have X continuous black pixels followed by X contin-

uous white pixels followed by X continuous black

pixels again. If no continuous stretch of equally col-

oured of pixels significantly deviates in length from

the average, the vertical zebra pattern has been

found.

The vertical zebra pattern also determines the read-

ing direction of the barcode payload. If it is to the right of the startmarker, the reading direction is

clockwise.

4.9 Evaluation of the payload modules

This is the last step of the decoder pipeline. It reads the actual payload of the barcode.

Input

Perspective corrected input image

Centre of the barcode

Angle of the startmarker

Number of radial modules

Number of rings carrying payload data

Radius of the innermost and outermost

ring

Reading direction (clockwise or anti-

clockwise)

Output

Payload data of the barcode

In Illustration 27, the area of the barcode’s pay-

load is highlighted.

4.9.1 Grid

With the input, it is possible to draw a grid over

the barcode in the input image. Illustration 28

shows debug output where a grid was drawn

using the input parameters of this processing

stage. To keep the grid visualisation readable a

line was drawn only through every second radial

Illustration 25: Two test lines are shot form the centre of the barcode through the potential locations of the vertical zebra pattern

Illustration 26: The payload area is highlighted in blue


22 / 36

module. At every intersection of the grid a module shall be evaluated as either black or white.

4.9.2 Reading a module

A total of five pixels are sampled for each module in a cross-shaped pattern as shown in Illustration

28.

Illustration 28 Cross-shaped sampling pattern

Each pixel votes for the module to be either black or white according to its colour. Because the sam-

pling is performed on a binarized image and an uneven number of pixels are sampled, a module’s

colour is never ambiguous, it can always be classified as either black or white.

Illustration 27: Grid matching the barcode modules


23 / 36

5 Protocol

The payload of the barcode is structured according to a simple protocol as shown in Illustration 29:

Illustration 29: Payload protocol

Field Size in bytes Description

Length 1 Contains the length of the body including the space used for the error correction codes. Because this field has a fixed length of one byte, the maximum size of the body of a barcode is restricted to 255 bytes.

Data variable The data encoded in the barcode

ECC 20 20 bytes are always reserved for error correction

5.1 Error correction codes

The open source library Rscode10 was used in this project. It implements Reed-Solomon error correc-

tion. Rscode performs error correction on 8-bit blocks. The amount of error correction codes has

been set to 20 bytes because Rscode needs this as compile-time argument. 20 bytes of error correc-

tion codes allows to fix up to 10 faulty blocks of 8 bit each.

10 Minsky, Henry: Welcome to http://rscode.sourceforge.net/. http://rscode.sourceforge.net/ (Accessed August 5, 2015)


24 / 36

6 Tests

6.1 Tests using a webcam

During the development phase, most tests were done using a laptop webcam as scanner to read a

barcode shown on a mobile phone display. The scanning process is fast and reliable enough to cause

no inconvenience to the user, even under not ideal conditions such as a considerable amount of per-

spective distortion.

Because the results are dependent on the actions of the person holding the barcode, it is impossible

to reliably quantify the success rate and overall quality with this test method.

6.2 Batch testing robustness

To measure the quality and capabilities of the decoder in a reproducible way, a batch testing tool has

been developed. It generates a barcode, adds some defects to it and then checks if it can be correctly

decoded. To obtain reproducible results, each test is run several times.

6.2.1 Added defects

The test tool is capable of adding three types of defects to a barcode:

Perspective distortion

The perspective distortion can be controlled in percent. 0% means no distortion, with 100% it is pos-

sible that all four corners of the barcode image are mapped to a single point in the distorted image.

For examples, refer to Illustration 30.

Illustration 30: Perspective distortion added by the test tool. From left to right: 0% distortion to 50% distortion in 10% incre-ments.

Salt and pepper noise

This defect adds random black and white pixels to the image and is controlled by a percentage value.

At 100%, the image is fully covered by noise. For examples, see Illustration 31.

Illustration 31: Salt and pepper noise added by the test tool. From left to right: 0% to 20% noise in 10% increments.


25 / 36

Area destruction

This defect adds random ellipses to the barcode, obscuring some areas. It is controlled by a percent-

age value. At 100%, the cumulative area of all ellipses is equal to the image size. For examples, refer

to Illustration 32.

Illustration 32: Random ellipses obscuring parts of the image added by the test tool. From left to right: 0% coverage to 2% coverage in 1% steps.

6.2.2 Tests

All tests were conducted with three different barcode sizes:

Name Encoder parameters

300x300 moduleSizePixels=6; minRadiusPixels=60; maxRadiusPixels=120



Illustration 33: Successfully read barcodes at different perspective distortion levels. Sample size: 500


26 / 36

As shown in Illustration 33, the two bigger barcode sizes perform quite well with up to 30% distor-

tion. The smaller 300x300 barcode is not well readable when perspective distortion is applied.

Illustration 34: Successfully read barcodes at different noise levels. Sample size: 500

The noise test shown in Illustration 34 shows a picture that is similar to the distortion test. The two

bigger formats perform well with up to 10% noise, while the smaller 300x300 barcode struggles as

soon as noise is added.

Illustration 35: Successfully read barcodes at different levels of obscured areas. Sample size: 500

As shown in Illustration 35, the decoder cannot cope well when some areas of the barcode are ob-

scured. The barcode size does not seem to matter at all.


27 / 36

Illustration 36: Successfully read barcodes at different perspective distortion and noise levels. Sample size: 500

Illustration 36 shows that applying per-

spective distortion and noise at the

same time does not significantly wors-

en the decoding rate over only adding

noise. Illustration 38 shows a barcode

that was correctly read after perspec-

tive distortion as well as noise was ap-

plied.

6.2.3 Interpretation

As described in the chapter Recognition

and decoding the decoder works on a

best effort basis. This means that if

there is a problem in any step of the

decoding pipeline, the input image is

discarded. This naturally leads to de-

coding rates lower than 100% as soon

as defects are added.

The decoding rate needed varies from

application to application. If there is a

video feed such as from a webcam, a

decoding rate of 50% or even lower is acceptable, since if one frame is discarded, the next one is

available almost immediately. However, in an application where only a single image of the barcode is

available, a decoding rate lower than 100% naturally is a problem.

The barcode scanner performs fine for perspective and noise defects, but does not fare well when

some areas of the barcode are missing or destroyed. This is because even if only a small percentage

of the overall barcode area is destroyed, it is quite likely that at least one detection and recognition

Illustration 37: Example of a barcode that was recognized and correctly read


28 / 36

feature is affected. Illustration 40 shows such a barcode. The vertical zebra pattern is partly de-

stroyed, rendering the barcode unreadable. However, thanks to ECC the barcode in Illustration 40 is

perfectly readable, despite some of the data area is destroyed.

6.3 PSNR (peak signal-to-noise ratio)

“Peak signal-to-noise ratio, often abbreviated PSNR, is an engineering term for the ratio between the

maximum possible power of a signal and the power of corrupting noise that affects the fidelity of its

representation” - Wikipedia11

If only a 100% decoding rate considered acceptable, the PSNR is infinity because as can be seen in

Illustration 34, the decoding rate drops under 100% as soon as any noise is added to the image.

However, if the aim is to achieve a 50% decoding rate, calculating the PSNR is possible. As can be

seen in Illustration 34, up to 15% noise can be added to a 400x400 barcode before the decoding rate

drops below 50%.

The PSNR is defined as11:

𝑃𝑆𝑁𝑅 = 10 ∗ 𝑙𝑜𝑔10 ∗ (𝑀𝐴𝑋𝐼

2

𝑀𝑆𝐸)

MSE: Mean squared error

MAXI: Maximum intensity

Because the decoder operates on a binary image the maximum intensity is 1 and the mean squared

error is half the noise (Because when adding a salt and pepper noise pixel to a binary image there is a

50% chance that the pixel already has the new color). The lower bound of the PSNR is therefore 11.2

decibel if a decoding rate of 50% is desired.

6.4 Performance

Performance tests were conducted on the following computer:

11 Wikipedia (2015): Peak signal-to-noise ratio. https://en.wikipedia.org/wiki/Peak_signal-to-noise_ratio (Accessed August 5, 2015)

Illustration 39: Unreadable barcode Illustration 39: Readable barcode despite defects


29 / 36

Component Specification

Processor i7-3520M dual core, 2.9 Ghz

RAM 8 Gigabyte

Operating System Windows 7 64bit

Illustration 40 shows the encoding and decoding times using multiple barcode sizes. The encoding

process is about 10 times faster than decoding process. Not much time was spent during this project

on optimisations as performance was deemed fast enough. It is expected that there are some quite

low hanging fruits when it comes to performance optimisations.

Illustration 40: Encoding and decoding performance of multiple barcode sizes


30 / 36

7 Further work and improvements

7.1 Scanning of actual coded products

Because of the long lead time on producing real products with the barcode on them, it was clear

from the start of the project that tests with scans of real products is out of scope for this project.

Furthermore, it is not clear what kind of defects a barcode on a product would possibly have and

which areas are most susceptible to damage. In agreement with the client it has been assumed that

the smaller modules on the inside of the barcode are more likely to be unreadable. Data is therefore

preferentially placed in the bigger modules towards the outside of the barcode. It is possible that real

world experiences with the barcode on products call for changes to the barcode format.

7.2 ECC license

The ECC library Rscode12 that was used in this project is licensed under GPL (General Public Li-

cense)13. The GPL has a strong copyleft, requiring the whole project code to be licensed under GPL as

well. Because this is possibly not desirable for NANO 4 U, two versions of the project are delivered:

Version with ECC, licensed under GPL

Version without ECC, not specifically licensed

The reason a GPL-licensed library was used is was not possible to source a C++ ECC library licensed

under a liberal license. For the future, there are multiple options:

Obtain a commercial ECC library

Port the ECC library used in ZXing from Java to C++. ZXing is licensed under the quite liberal

Apache 2.0 license14.

7.3 Parametrizing the decoder

The only argument the decoder takes is the input image. All information it requires to decode the

barcode is extracted from the barcode itself. This includes the number of rings and the number of

modules in each ring.

It is conceivable that supplying information about the basic shape of the barcode to the decoder

could improve robustness, especially if parts of the barcode are unreadable. Possible arguments are

primarily the already mentioned number of rings and number of modules in a ring. Attempts to make

use of such arguments have been made during the course of this project. However, it proved to be

more complex than expected because of the complex decoder pipeline and dual-use barcode fea-

tures such as the radial zebra pattern which is used for perspective correction and also to determine

the number of modules in each ring.

7.4 Avoid data looking like a startmarker

There is a chance that the payload data in a barcode forms a pattern identical to the startmarker. If

two starmarker patterns are present, it is possible that the decoder selects the wrong one and there-

fore the payload cannot be read. The chance of a pattern equal to the startmarker occurring in a

barcode is dependent on the number of rings and the number of modules per ring. Due to the zebra

12 Minsky, Henry: Welcome to http://rscode.sourceforge.net/. http://rscode.sourceforge.net/ (Accessed August 5, 2015) 13 Wikipedia (2015): GNU General Public License. https://de.wikipedia.org/wiki/GNU_General_Public_License (Accessed August 5, 2015) 14 ZXing (2014): License Questions. https://github.com/zxing/zxing/wiki/License-Questions (Accessed August 5, 2015)


31 / 36

pattern around the outside of the barcode, a startmarker-lookalike can only occur at every second

angular module. The chance of data looking identical to the startmarker for each angle is:

𝑝𝑎𝑛𝑔𝑙𝑒 =1

23∗𝑟𝑖𝑛𝑔𝐶𝑜𝑢𝑛𝑡

It is likelier that a startmarker- lookalike appears next to the real startmarker because the white

space to the side of the startmarker is already there:

𝑃𝑔𝑙𝑜𝑏𝑎𝑙 =1

22∗𝑟𝑖𝑛𝑔𝐶𝑜𝑢𝑛𝑡

For a barcode that has 8 rings, the chance of a startmarker-lookalike is quite small (~1:50000). How-

ever, if the number of rings is reduced this quickly becomes a big problem.

A possible solution is to check the generated barcode for startmarker-lookalikes. If such a startmark-

er-lookalike is detected, a random bitmask is selected. To get rid of the startmarker, an XOR-

Operation is performed on the payload body and the bitmask. To be able to decode the barcode

again the used bitmask has to be encoded into the barcode payload as meta information.

7.5 Error detection

Although ECC has been implemented, its possibilities are limited. If too many modules have been

incorrectly sampled, ECC can no longer correct all errors and the payload read by the decoder is in-

correct. Currently, this goes undetected. To fix this, a checksum over the barcode’s payload could be

used.

7.6 Error correction in length field

Currently, the length field of the payload protocol is not protected by any error correction mecha-

nism because the ECC decoder itself needs to know the length of the ECC-protected payload. Due to

the lack of error correction, any reading error in the length field will lead to an unreadable barcode. A

possible solution to this problem would be transmitting the length field three times redundantly to

compensate for reading errors.


32 / 36

8 API and project structure

8.1 Project structure

The project workspace is structured in six subprojects:

Project Description

DecoderLib Library for decoding a barcode

EncoderLib Library for generating a barcode

DecoderVideoDemo Sample application using the decoder library. The webcam captures con-tinuous frames which are handed to the DecoderLib.

Encoder Sample application using the encoder library.

BatchTestTool Command line tool used to test the decoder. Uses the EncoderLib to gener-ate barcodes which are deformed and damaged in multiple ways before trying to decode them using the DecoderLib.

Rscode Third party ECC library (http://rscode.sourceforge.net/)

8.2 API

The EncoderLib and DecoderLib each provide a simple API.

8.2.1 EncoderLib

The EncoderLib exposes the method encode(). It creates a new barcode as an image:

Mat BarcodeEncoder::encode(BarcodeEncoder::EncoderConfiguration, StringDataProvider)

If the data provided is longer than the capacity, an exception is thrown. The EncoderConfiguration

determines the size and shape of the barcode by specifying:

Inner radius

Outer radius

Module size

It also provides a method to determine how many bytes of payload can be placed in the barcode with

that specific barcode configuration:

int BarcodeEncoder::EncoderConfiguration::capacityBytes()

8.2.2 DecoderLib

The Deocoderlib exposes the method decode():

vector<char> Decoder::decode(Mat M)


33 / 36

9 Personal reflection

It has been a very interesting but also challenging project. A ring-shaped barcode is without prece-

dent, which makes this project quite innovative. Developing the first ring-shaped barcode format also

meant going new ways in detecting the barcode.

Although I attended a C++ module during my studies, this project was the first time I used C++ for a

productive project. Working with C++ provides tremendous opportunities, but frankly has often

been frustrating for a newbie like me. Compiler errors are hard to interpret and I had my fair share of

linker problems while transitioning to a multi-project workspace. However, this is made up by C++’s

versatility, execution speed and the possibility to integrate it with basically any software on any op-

erating system.

Overall, I am happy with the outcome of the project, although time was tight towards the end. There

are still some small things that could be done to improve the decoding process. The decoder works

well when decoding a frame captured with the webcam, but it is difficult to anticipate the specific

challenges when the barcode is scanned from a real product. I reckon it is likely that some adjust-

ments are needed, but I feel confident that the overall concept will prove effective.

A special challenge of this project was that the documentation as well as the communication with the

customer was in English. I consider my command of English quite good, but it has proved that it is

much harder to express complex technical relationships in a language other than one’s mother

tongue.

9.1 Acknowledgments

I would like to express my gratitude to my supervisor, Prof. Dr. Christoph Stamm, who has guided me

through this project. We had a lot of interesting discussions which often sparked new and creative

ideas.

Furthermore, I would like to thank NANO 4 U and Paul Glendenning for providing me with this inter-

esting and challenging project.

Finally, thanks to Dr. Lorenz Frey, with whom I had an interesting debate about error correction

codes and their placement within the barcode.


34 / 36

10 Sources

Barcoding Connected (2009): SHOTCODE BARCODE: THE CIRCULAR 2D BARCODE FOR MOBILE

TAGGING. http://blog.barcoding.com/2009/02/shotcode-barcode-the-circular-2d-barcode-for-

mobile-tagging/ (Accessed August 5, 2015)

Mallick, Satya (2015). Blob Detection Using OpenCV ( Python, C++ ).

http://www.learnopencv.com/blob-detection-using-opencv-python-c/ (Accessed July 13, 2015)

Minsky, Henry: Welcome to http://rscode.sourceforge.net/. http://rscode.sourceforge.net/ (Ac-

cessed August 5, 2015)

Mudie, Stuart: Shotcode. Licensed under CC BY-SA 2.0 via Wikimedia Commons -

https://commons.wikimedia.org/wiki/File:Shotcode.png#/media/File:Shotcode.png

NANO 4 U : http://www.nano4u.net/ (accessed August 15, 2015)

OpenCV (2015): Geometric Image Transformations.

http://docs.opencv.org/modules/imgproc/doc/geometric_transformations.html#warpperspective

(Accessed August 10, 2015)

Stamm, Christoph (2014): Projektive Abbildung. Slides for the module magb, file 09_Bildmatrix.pdf

Teply, Joel (2014): http://stackoverflow.com/a/25152785/2037769 (Accessed June 15, 2015)

The Lab Book Pages: Otsu Threshholding.

http://www.labbookpages.co.uk/software/imgProc/otsuThreshold.html (Accessed August 1, 2015)

Wikipedia (2015): GNU General Public License.

https://de.wikipedia.org/wiki/GNU_General_Public_License (Accessed August 5, 2015)

Wikipedia (2015): Peak signal-to-noise ratio. https://en.wikipedia.org/wiki/Peak_signal-to-

noise_ratio (Accessed August 5, 2015)

Wikipedia (2013): ShotCode. https://de.wikipedia.org/wiki/ShotCode (Accessed June 7, 2015)

Wikipedia (2014): Shotcode. https://en.wikipedia.org/wiki/ShotCode (Accessed June 7, 2015)

ZXing (2014): License Questions. https://github.com/zxing/zxing/wiki/License-Questions (Accessed

August 5, 2015)

http://blog.barcoding.com/2009/02/shotcode-barcode-the-circular-2d-barcode-for-mobile-tagging/

http://blog.barcoding.com/2009/02/shotcode-barcode-the-circular-2d-barcode-for-mobile-tagging/

http://www.learnopencv.com/blob-detection-using-opencv-python-c/

http://rscode.sourceforge.net/

http://rscode.sourceforge.net/

http://www.nano4u.net/

http://docs.opencv.org/modules/imgproc/doc/geometric_transformations.html#warpperspective

http://stackoverflow.com/a/25152785/2037769

http://www.labbookpages.co.uk/software/imgProc/otsuThreshold.html

https://en.wikipedia.org/wiki/Peak_signal-to-noise_ratio

https://en.wikipedia.org/wiki/Peak_signal-to-noise_ratio

https://de.wikipedia.org/wiki/ShotCode


35 / 36

11 Academic honesty statement

Hiermit bestätigt der Autor, diese Arbeit ohne fremde Hilfe und unter Einhaltung der gebotenen Re-

geln erstellt zu haben.

Robin Brügger

_______________________ _________________________

Ort, Datum Unterschrift


36 / 36

12 Appendix

12.1 Glossary

12.1.1 Module

A module of the barcode is the smallest chunk of information a bar-

code stores. Depending on the barcode resolution, it can consist of

multiple pixels, all of which are either white or black. Illustration 42

shows a barcode in which a single module has been coloured red in-

stead of black.

12.2 Software used

12.2.1 Libraries

Library Version

OpenCV 2.4.10

Rscode (http://rscode.sourceforge.net) 1.3

12.2.2 Others

Software Version

IDE Visual Studio 2012 Ultimate

Operating system Windows 7 64bit

Illustration 41: A single module is coloured red

Documents

Bachelor Thesis - · PDF file1.2 Integration and software ... 8 API and project structure ... Bachelor Thesis