Upload
others
View
9
Download
0
Embed Size (px)
Citation preview
LAPPEENRANTA UNIVERSITY OF TECHNOLOGY
DEPARTMENT OF INFORMATION TECHNOLOGY
AUTOMATED HELIOTEST INSPECTION USING
MACHINE VISION
The topic of the master’s thesis has been accepted in the departmental council of
Department of Information Technology, November 26, 2003.
Supervisors: Professor Heikki Kälviäinen, Dr. Joni-Kristian Kämäräinen
Lappeenranta, December 17th, 2003
Albert Sadovnikov
Karankokatu 4 A 18
53810 Lappeenranta
Tel. 0 468 109 468
ABSTRACT
Lappeenranta University of TechnologyDepartment of Information TechnologyAlbert Sadovnikov
Automated Heliotest inspection using machine vision
Thesis for the Degree of Master of Science in Technology200372 pages, 19 figures, 8 tables and 1 appendix.Examiners: Professor Heikki Kälviäinen, Dr. Joni-Kristian KämäräinenKeywords: Heliotest, visual quality inspection, paper printability, paper making,machine vision, image processing
The main subject of this master’s thesis was to develop methods for automatedHeliotest inspection using machine vision techniques. Heliotest is one of paperprintability tests and it measures ability of paper to reproduce halftones in ro-togravure printing. Test print result consists of three differently evaluated areasand the most interesting area in this thesis is a dotted strip printed in the middle ofa tested paper sample. Quality measurement is made by observing the number ofmissing dots from the dotted strip. The size of a printed dot is small and difficultto evaluate by human experts which is, however, the current practice. Thus, anautomatic method would have an important practical impact.
In this work two frequency domain filtering methods and one model based spa-tial feature extraction approach are presented. All the proposed methods exploitregularity properties of the dot pattern structure.
In conducted experiments on real data all methods performed well. The presentedautomated Heliotest evaluation methods seem to be on a solid theoretical founda-tion and based on the practical results it seems possible to perform the Heliotestevaluation automatically in industry.
ii
TIIVISTELMÄ
Lappeenrannan teknillinen yliopistoTietotekniikan osastoAlbert Sadovnikov
Automated Heliotest inspection using machine visionDiplomityö200372 sivua, 19 kuvaa, 8 taulukkoa ja 1 liite.Tarkastajat: Professori Heikki Kälviäinen, TkT Joni-Kristian KämäräinenHakusanat: Heliotest, visuaalinen laadunvalvonta, paperin painettavuus, paperin-valmistus, konenäkö, kuvankäsittelyKeywords: Heliotest, visual quality inspection, paper printability, paper making,machine vision, image processing
Tässä työssä käsitellään konenäkömenetelmiä automaattisen Heliotest-mittauksensuorittamiseksi. Heliotest-mittaus on yksi paperin painettavuutta mittaavaa testija se mittaa erityisesti paperin kykyä tuottaa syväpainettuja rasterikuvia. Testiävarten tehtävä testipainatus sisältää kolme eri tavalla mitattua aluetta ja tässä työssäkeskitytään rasteripisteisiin, jotka ovat keskimmäisellä alueella paperinäytteessä.Painatuksen laatua mitataan tutkimalla alueelta puuttuvien rasteripisteiden määrääpainokuviossa. Rasteripisteiden koko on hyvin pieni ja niiden manuaalinen tutkimi-nen on hankalaa, mutta kuitenkin yleisin teollisuudessa käytetty menetelmä. Tämäntyön tarkastelun kohteena olevalla menetelmällä tulisi toimiessaan olemaan huo-mattava käytännöllinen merkitys.
Tässä työssä esitellään kaksi taajuutasoon perustuvaa menetelmää ja yksi spati-aalisessa tasossa olevaan malliin perustuva menetelmä. Kaikki kolme menetelmäähyödyntävät rasteripisteiden muodostamaa toistuvaa rakennetta.
Suoritetuissa kokeissa todellisella kuva-aineistolla kaikki esitetyt menetelmät me-nestyivät hyvin. Käytetty teoreettinen perusta näyttää olevan oikea ongelmanratkaisemisen kannalta ja käytännön tuloksien perusteella Heliotestin automati-soiminen teollisuudessa näyttää mahdolliselta.
iii
Preface
This master’s thesis was done in Lappeenranta University of Technology in De-
partment of Information Technology during the summer and the autumn semester
of 2003. The thesis was part of a larger project called "PAPVISION - Paper and
board printability tests using machine vision" which has been done in Informa-
tion Processing Laboratory of Lappeenranta University of Technology. Espe-
cially I would like to thank European Union and National Technology Agency of
Finland (TEKES) for financial support in the Papvision project (TEKES Project
No. 70049/03). Moreover, industrial partners UPM-Kymmene, Stora Enso, Myl-
lykoski Paper, Metso Paper, Future Printing Center (FPC: Raisio Chemicals, Hansa-
print, and Omya), and Labvision Technologies are greatly acknowledged for their
scientific and financial support.
I would like to thank my supervisors professor Heikki Kälviäinen and Joni-Kristian
Kämäräinen for support and overall encouragement. My project colleagues Pasi
Saarinen, Lasse Lensu, Jarkko Vartiainen, Petja Salmela and Alexander Drobchenko
also receive my sincere gratitude for information and help in various project as-
pects.
Special thanks to all the people who made the IMPIT program possible, particu-
lary professor Jan Voracek and Nina Kontro-Vesivalo. Whithout the existence of
the IMPIT program nothing could have been done.
This work could not be possible without backing from family, friends and many
other people and I express my thankfulness to them.
Lappeenranta
iv
Contents
1 Introduction 4
1.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.2 Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.3 Structure of the thesis . . . . . . . . . . . . . . . . . . . . . . . . 5
2 Paper printabiliy 6
2.1 Printing papers . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
2.2 Printability tests . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
2.3 Heliotest . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
3 Machine vision methods 13
3.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
3.2 Image acquisition and enhancement . . . . . . . . . . . . . . . . 16
3.3 Feature extraction . . . . . . . . . . . . . . . . . . . . . . . . . . 18
3.3.1 General feature picture . . . . . . . . . . . . . . . . . . . 18
3.3.2 Fourier domain filtering . . . . . . . . . . . . . . . . . . 19
3.3.3 Filtering using discrete cosine transform . . . . . . . . . . 23
3.3.4 Gabor filters . . . . . . . . . . . . . . . . . . . . . . . . 28
3.3.5 Pattern modelling . . . . . . . . . . . . . . . . . . . . . 29
3.4 Classification . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
3.4.1 Image segmentation . . . . . . . . . . . . . . . . . . . . 32
3.4.2 Spatial features and Gabor features . . . . . . . . . . . . 32
4 Automated Heliotest inspection 34
4.1 Common to all methods . . . . . . . . . . . . . . . . . . . . . . . 34
4.2 FFT based method . . . . . . . . . . . . . . . . . . . . . . . . . 36
4.2.1 Preprocessing . . . . . . . . . . . . . . . . . . . . . . . . 36
4.2.2 Extraction of the irregular component . . . . . . . . . . . 37
4.2.3 Classification . . . . . . . . . . . . . . . . . . . . . . . . 38
4.3 DCT based method . . . . . . . . . . . . . . . . . . . . . . . . . 39
4.4 Pattern modelling with spatial features method . . . . . . . . . . . 40
4.4.1 Grid extraction . . . . . . . . . . . . . . . . . . . . . . . 40
1
4.4.2 Classification . . . . . . . . . . . . . . . . . . . . . . . . 42
5 Experiments and results 43
5.1 Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
5.1.1 Heliotest samples . . . . . . . . . . . . . . . . . . . . . . 43
5.1.2 Training and testing data . . . . . . . . . . . . . . . . . . 43
5.2 Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
5.2.1 FFT based method . . . . . . . . . . . . . . . . . . . . . 45
5.2.2 DCT based method . . . . . . . . . . . . . . . . . . . . . 46
5.2.3 Pattern modelling with spatial features method . . . . . . 48
5.3 Results and discussions . . . . . . . . . . . . . . . . . . . . . . . 49
6 Conclusion 50
2
List of Symbols and Abbreviations
F{·} Fourier transform
F−1{·} inverse Fourier transform
H image filtering operator
M(x, y) mask function
µ image mean value
τ threshold value
ψ(x, y) 2-d Gabor filter in spatial domain
γ sharpness of Gabor filter along major axis
η sharpness of Gabor filter perpendicular to major axis
θ orientation of Gabor filter
χ characteristic function
f frequency of Gabor filter
I(x, y) unit function
rξ(x, y; f, θ) response of 2-d Gabor filter
S minimal area value
T{·} threshold operator
ξ(x, y) 2-d image in spatial domain
Ξ(u, v) 2-d image in frequency domain
u first frequency coordinate
v second frequency coordinate
x first spatial coordinate
y second spatial coordinate
2-d two dimensional
DCT discrete cosine transform
DFT discrete Fourier transform
FFT fast Fourier transform
MV machine vision
PCA principal component analysis
ROI region of interest
3
1 Introduction
1.1 Background
The paper making technology has a very long history ongoing from the distant
past. During the last 150 years it has been developed greatly and has arrived
at significant progress nowadays. In the 19th century Keller invented the wood
pulp and a little bit later the cellulose treatment process was made-up which has
allowed technology to be developed up to today’s state. The paper is the ma-
jor polygraphic material and meets all technological, consumer and economic re-
quirements showed to such materials.
There is a research project, called "PAPVISION - Paper and board printability
tests using machine vision" [1] in Lappeenranta University of Technology. The
goal of this project is to provide new innovative solutions for paper and printing
industry and to establish top knowledge on image processing in this industry. The
project gathers information about problems which can be solved using machine vi-
sion and image processing techniques and implements automatic applications for
the use of industrial partners. The project is carried out in intensive co-operation
with the industrial partners. The main objective is to serve industrial partners as
a main research resource and organization for machine vision and other scientific
computation solutions. [1]
Heliotest is one of tests which are intended to measure paper printability in lab-
oratory conditions. Printablility is the quality of paper in terms of its behavior
with ink, water and other liquids, as well as drying in the printing process. It is
affected by factors such as fiber furnish, coatings and surface finishing [2]. He-
liotest shows how good printing quality is, i.e., how many dots should have been
printed to halftone image, but due to some reasons were not printed. This partic-
ular work is intended to summarize and propose new approaches and methods for
automated Heliotest inspection.
4
1.2 Objectives
The objective of this master’s thesis is to develop a method or several methods
for automated Heliotest inspection. The developed method is not intended to be
’working in all cases’, though the topic should be researched for possible lim-
itations/circumstances in which the developed method should work at least not
worse comparing to if the test was conducted by a human expert.
1.3 Structure of the thesis
The thesis focuses on the measurement problem of paper printability, particulary
Heliotest. It starts with the paper printablility measurement problem raising and
describing it together with existing Heliotest methodology, and proposes several
new approaches based on machine vision and image processing methods.
The structure of the thesis is following. Section 2 is an introduction to a gen-
eral problem state. It includes a survey on paper printability, paper printability
testing and thorough Heliotest description. Section 3 deals with machine vision
methods for Heliotest inspection. This part contains several approaches on image
pattern removal and further classification methods. Section 4 presents algorithm
design using methods from the Section 3. It focuses on the implementation de-
tails describing practical properties omitted in Section 3. Section 5 represents
real experiments. In this part methods are tested and statistical data is collected,
also comparison of the methods is conducted. Section 6 finishes the thesis with
a review of key points, conclusions on performed research and propositions on
possible work to be undertaken in the future.
5
2 Paper printabiliy
2.1 Printing papers
Printed paper in its different manifestations is familiar to and used by everybody.
Products truly represent the much sought-after every citizen interface to infor-
mation. The time spent each day browsing printer matter for work, education,
information and leisure is considerable. Needs fulfilled are diverse, as are also
the user situations. In other words, printed matter is an essential part of the fabric
of our daily lives. Print production technologies have changed immensly during
the last forty years, more so than during the preceding four hundred years. This
is mainly due to the digitalization of the basic raw materials of prints, namely in-
formation. Previous progress was made largely through mechanisation. However,
during this time the appearance of printed matter has not changed radically.[3]
The term "paper grade" denotes papers that share the most relevant criteria. The
main criteria used to classify printing papers include following matters [3]:
• Fiber furnish of the base of paper.
• Coating.
• Type of surface finish.
• End use.
2.2 Printability tests
Printability of paper is a paper property that tells how paper behaves during a
printing process. Printability depends on interactions between paper and printing
ink and printing process variables. Good printability of paper means that the paper
6
is not very sensitive to variations in different process variables and easily gives
good print quality. Print quality describes the final result of printing or the quality
of the printed image. Print quality definition does not have absolute terms. It
depends on the print density, resolution, and eveness of the printed image. Many
other properties are also of interest. [4]
Printability tests depend on the printing process involved. Certain types of test
methods aim at testing general print quality. These tests are useful for all types
of papers regardless of the printing methods used. Some test methods are only
suitable for a certain printing process.
Printability and print quality depend on many different factors. The paper and
its properties are important components as is the printing process with its many
variables. Due to the complex nature of the printing process and different interac-
tions influencing it, testing of printabiliy and print quality can consist of different
techniques.
Papermakers and printers have searched for a fast, simple, and effective test meth-
ods to predict paper behavior in printing without actually printing the paper. Every
printing process has its own features that often require a separate testing device
and a specific test. In many cases, measuring similar properties after different pro-
cesses is possible. Table 1 lists laboratory printing tests commonly used. Some
tests may be used with all the listed printing methods, and some are more specific
to a certain printing process [4]. As it can be seen from Table 1, missing dots
testing (Heliotest) is performed only for rotogravure printing.
Rotogravure printing is an intaglio process meaning engraved or carved. The im-
pressions are achieved by the transfer of ink from cells or depressions of varying
depths, etched into the print cylinder to a web of paper at high speeds. The pro-
cess of printing involves coating the etched cylinder into an enclosed fountain or
trough of ink and the etched cells are filled with ink. While the cells fill with
ink, the surface of the cylinder (non-image area) also becomes coated with ink.
This non-wanted ink is removed by a doctor blade or knife which wipes all of
7
the surface ink from the cylinder. The printing cylinder comes in contact with
the paper and the ink which remains in the cells is transferred to the paper. High
cylinder cost generally limits gravure to run lengths of over 1 millions impres-
sions and thus, gravure is a long run process. Gravure presses are also much
wider than other printing type presses. Unlike letterpress or offset, the ink used is
very fluid and is usually solvent based which in today’s environment is undesir-
able. Typical rotogravure printed products would include packaging, catalogs and
magazines. [5]
Table 1: Laboratory test methods for predicting the printability and print qualityin different printing methods. [4]
Offset Digital Rotogravure FlexoPrinting Printing Printing Printing
Coldset Heatset Sheet Ink-jet LaserPrint density x x x x x x xInk requirement x x x x xInk set-off x x x xRub-off x xPrint through x x xInk Gloss x x x x xDry pick x xWet pick x xBlack trap pick x xMottle x xFiber roughening x x xToner adhesion xMissing dots xResidual solvent x xDot gain x x x x x xDots geometry x x x x x
8
2.3 Heliotest
The greatest problems in rotogravure printing involve reproduction of light and
medium tones. Two different types of defects are missing dots and waving. Miss-
ing dot defect means that the ink was not transferred to the paper while printing
due to the bad quality of that paper area, which did not allow ink to glue to the
paper. Missing dots are inevitable at 5% half tone but disastrous when occuring
at 20% and 30% half tones [4]. Literally, half tone means the tone which is in
between the paper color and the ink color. Needed half tone reproduction can be
achieved by transferring a certain amount of ink. So in dark areas (close to ink
color) missing dots are more fatal. The number of missing dots in a certain area is
a traditional measure of rotogravure printablity of a paper. In laboratory printing,
Heliotest indicates the number of missing dots. The Heliotest fitting, developed
by Centre Technique du Papier [6] is used with an IGT AC2 laboratory printer [7]
(see Figure 1).
Printing disk Doctoring system
Paper sample Gravure ink
Figure 1: Heliotest accessory.
The Heliotest attachment consists of an engraved printing disk, a doctoring system
9
and a special, not quick drying gravure ink. Some drops of the ink are put on the
printing disk, the surplus of ink is wiped off and a print is made on the substrate
which has been attached to the sector. The disk contains three types of engraving
as follows:
• Halftone screen area (see Figure 2): Disk carves in this part are the same
in size, but their depth changes from the beginning till the end in order
to reproduce less intensive halftones. More deep engraves produce larger
dots and less deep ones produce smaller. In this area the distance from
the beginning of the print till the twentieth missing dot is measured. The
longer the distance, the smoother the paper. This part of the disk is the most
important part.
• A conventional screen area: See dark rectangles on the top and bottom sides
of the Heliotest print result in Figure 2. This part is used for visual assess-
ment and is of low importance.
• Two lines of dots on the both sides of the halftone screen area (see Figure
2): In these lines the total number of missing dots is counted. This is done in
case the distance measured in the variable halftone screen area is too small,
that is when the paper is very rough. The more missing dots, the rougher
the paper.
After printing the test strips shown in Figure 2, an expert should make an as-
sessment of it, which is made according to IGT recommendation leaflet in the
following way [8]:
1. Starting from the side with the largest dots in the variable screen count the
missing dots until the 20th one.
2. Measure the distance in millimeters from the 20th missing dot to the begin-
ning of the variable screen area.
10
3. In the case the distance in Step 2 is very small (only some millimeters) count
the total number of missing dots in the four dotted lines on both sides of the
variable half tone.
4. Repeat Steps 1 and 2 or 3 for each test strip.
5. Compute the average and if required the standard deviation. In some cases
it may be useful to mention the highest and the lowest value as well.
6. If needed assess the printing quality visually in the conventional screen area.
11
Halftone screen area
Missing dots
Lines of dots
Conventional Screen Area
Figure 2: Detailed description of Heliotest print result.
12
3 Machine vision methods
3.1 Background
Machine vision (MV) is the ability of a computer to "see". Combining high speed,
high definition cameras and powerful embedded processing engines, machine vi-
sion has become a crucial part of production processes in many industries. With
the growing of personal computer power, new applications for machine vision
techniques arise [9]. MV is used in various industrial and medical applications for
example as follows [10]:
• Electronic component analysis.
• Signature identification.
• Optical character recognition.
• Handwriting recognition.
• Materials inspection.
• Currency inspection.
• Medical image analysis.
Among such applications is computer-based reading of graphical test results. Au-
tomated visual inspection of industrial materials such as textile, paper, plastic,
and, in particular, automated test print evaluation require adaptive solutions that
can be executed in real time. Currently, the quality assurance of paper printablity
is mainly carried out by manual inspection. However, manual inspection is labor
intensive and insufficient. Advantages of using a machine vision system in paper
printability testing can be defined as follows [11]:
• Less deviations in measurements.
13
• Better measurement accuracy.
• New printability parameters.
• Shorter measurement times.
• Less manpower to monotonic measurements.
• Many quality parameters by one system.
• Automatic data transfer to mill level information systems.
Therefore, automation of visual inspection tasks can increase the efficiency of
production lines and improve quality as well.
Standard machine vision routine includes several major stages as illustrated in
Figure 3. The most interesting step in this routine is feature extraction. Therefore,
quite a large part of the thesis work is devoted to this particular topic. Printed
Heliotest strip shown in Figure 4 can be considered as one of a regular texture.
It is clearly seen in Figure 4 that in the given local part, sizes of dots, angles and
distances remain nearly constant. This structure regularity allows to consider the
given problem as a particular case of a larger problem class, called defect de-
tection in textured materials [12, 13, 14, 15]. This problem domain is relatively
novel, and has not been studied thoroughly enough. However, there are a number
of general approaches which have been applied in this field (see Table 2).
Image
Aquisition
Image
Enhancement
Feature
Extraction Classification
Results
Output
Figure 3: Standard machine vision routine.
Textures are frequently seen in ordinary life and it is a common practice to think
of images as of textures. When one speaks about a texture, a textile fabric is often
14
associated. Hence, everybody has an idea of the term texture, but at the mean
time it is hard to define it precisely, and the definition is unfortunately ambigu-
ous. Nevertheless, there have been several attempts to define the term texture for
image processing needs. For example, in [16] the definition of texture is given
where texture is defined as "the feature, which describes spatial ordering of pixel
intensity in a region." Another definition given in [17] emphasizes structural prop-
erties of the texture. According to this definition "the term texture generally refers
to repetition of basic texture elements, called texels. Their placement can be pe-
riodic, quasi-periodic, or random." Image texture can be defined by the number
and types of its primitives together with the spatial organization or layout of those
primitives. The spatial organization may be random, may contain a pairwise de-
pendence of one primitive on a neighbouring primitive. The dependence itself
may be structural, probabilistic or functional, e.g., a linear relationship. On the
other hand, the notation of texture should admit to not rigid description.
Figure 4: Part of halftone screen area.
According to the texture regularity of Heliotest halftone screen area, the most
promising methods are, probably, those that include spectral analysis idea. In this
particular work three frequency domain approaches are proposed: using discrete
Fourier transform, using discrete cosine transform, and using Gabor filters. The
fourth model-based method presented also uses spectral analysis techniques.
15
Table 2: Methods for the detection of defects in textured materials. [12]
Approach Method1. Fractal dimension2. Bi-level thresholding3. Gray level statistics4. Morphological operations
Statistical 5. Edge detection6. Normalized cross-correlation7. Co-occurrence matrix8. Eigenfilters9. Local linear transforms10. Rank-order functions1. Discrete Fourier transform2. Optical Fourier transform3. Windowed Fourier transform
Spectral 4. Gabor filter5. Real Gabor function6. Wavelet transform7. Wavelet packets8. Wigner distribution1. Gauss Markov random field
Model-based 2. Poissonian model3. Model-based clustering
3.2 Image acquisition and enhancement
In the common machine vision routine, image acquisition is not the major part,
but it plays crucial role in the whole process. To find an appropriate acquisition
device one must think about several parameters, to make the whole method fast,
robust, and easy to use, which are defined as follows:
1. Minimal resolution needed to operate. This one is the base value for the
whole algorithm.
2. Color information needed. This is the parameter which tells how many color
halftones needed for optimal performance.
16
3. Ease of use. This one is quite simple, since for example using flatbed scan-
ner is less complicated than using an electronic microscope.
4. Price. If there are several devices with approximately same characteristics
described above, then cheapest one is, of course, more attractive.
Generally, resolution is measured in the number of dots per inch. While inves-
tigating the problem, it was found that with resolution of 600dpi, the width of a
dot on the halftone screen area is approximately 4 pixels. Presented methods can
work with this resolution quite well. However, further decreasing of resolution
seems not to be possible due to already small size of the dot.
Color information for that type of image is not necessary, and can be easily
changed to the grey-level. By losing color information during acquisition it is
possible to make the process faster and less memory dependent.
In the most machine vision laboratories there are two main acquisition devices.
First one is a digital camera, and second one is a flatbed scanner. The camera is
a very fast device, but however, it lacks resolution to take all the image in one
time with the given resolution (600dpi). This problem can be solved by taking
the image part by part, and processing separately. The flatbed scanner is a slow
device, but by losing time it is possible to win resolution needed. In the project
case, the flatbed scanner has been chosen because of the ease of use. Of course,
the price comparison of a flatbed scanner and a digital camera gives a significant
advantage to a scanner.
Since the project has an industrial goal, it is worth to design a special device
for reading test prints. From the thesis part, two parameters for such device are
defined: minimum resolution (600dpi) and grayscale color information (8bits).
Image enhancement is a subject of further research, since it seems clear that the
input image should be somehow cleared and standartised to fit processing needs.
For example, median filtering [18] can be used to remove minor noise from the
image.
17
3.3 Feature extraction
3.3.1 General feature picture
As it was mentioned before, feature extraction is the most interesting part in image
processing, from a scientific point of view. Properly extracted features are the base
of correct classification.
Let us consider Heliotest halftone screen area image as a 2-d functionξ(x, y)
which value in the given point(x, y) is a pixel intensity.{(xd, yd)} is a set of
missing dots centroid coordintes. LetH is an ideally designed image filter, then
the response of this filter should be
H{ξ(x, y)} = χ({(xd, yd)})
whereχ is a characteristic function of the missing dots set. However, filter re-
sponse for a certain image pixel is a valueH{ξ(x0, y0)} ∈ [0, 1] which describes
the amount of pixel belonging to a missing dot. So, the filter response is definitely
containing noise, caused by filter non-ideality (see Figure 5). The areas of miss-
ing dots on the filter response get larger, according to the actual dot size, which is
approximately4 × 4 pixels for the 600dpi resolution and 8x8 pixels for 1200dpi.
The response of such filter is called general feature picture, meaning that extracted
features (intensity values in this case) are attached to the image coordinate grid.
18
Figure 5: Original image and the corresponding filter response.
Following chapters will describe actual image filter design.
3.3.2 Fourier domain filtering
Fourier theorem states that any signal can be represented by the sum of sine and
cosine waves with various amplitudes and frequencies. That is, the relation-
ship between a repetitive, regular, and uniform dotted area pattern in the image
space and its spectrum in the spatial frequency can be linked by operating a two-
dimensional Fourier transform. Let a two dimensional image beξ(x, y) which is
real function representing the grey-level inx, y spatial coordinates, and let the im-
age height and image width are N and M, correspondingly. Then two-dimensional
discrete Fourier transform is given by the equation [19]
Ξ(u, v) = F{ξ(x, y)} =1
MN
M−1∑x=0
N−1∑y=0
ξ(x, y)e−j2π(ux/M+vy/N) (1)
This expression must be computed for values ofu = 0, 1, 2, ...,M−1, and also for
v = 0, 1, 2, ..., N − 1. Similarly, givenΞ(u, v), we obtainξ(x, y) via the inverse
19
Fourier transform, given by the expression [19]
ξ(x, y) = F−1{Ξ(u, v)} =M−1∑u=0
N−1∑v=0
Ξ(u, v)ej2π(ux/M+vy/N) (2)
for x = 0, 1, 2, ...,M − 1 andy = 0, 1, 2, ..., N − 1. Equations 1 and 2 comprise
the two-dimensional, discrete Fourier transform (DFT) pair [17]. The variablesu
andv are the frequency variables, andx andy are the spatial or image variables.
Let us consider Fourier spectra of the given texture part image (Figure 4). Accord-
ing to the complex nature of the Fourier spectra, it is useful to visualize it using
only its magnitude|Ξ(u, v)|.
In Figure 6 it is possible to see distinctive frequency peaks, which in turn are
responsible for the texture structure of the initial image. It is clear that all the peaks
are based on harmonic frequencies. Thus, for the separation of major and minor
frequencies in the Fourier spectra it is needed to multiply the complex Fourier
image by the maskM, which can be extracted from the Fourier magnitude image.
Figure 6: Original image and its Fourier spectra magnitude.
20
Finally, initial signal can be separated into so-called regular and irregular parts
ξ(x, y) = F−1{Ξ(u, v)} =
= F−1{M(u, v)Ξ(u, v) + (I(u, v)−M(u, v))Ξ(u, v)} =
= F−1{M(u, v)Ξ(u, v)}+ F−1{(I(u, v)−M(u, v))Ξ(u, v)}(3)
whereξ(x, y) is the spatial image,F andF−1 are forward and inverse discrete
Fourier transforms,M(u, v) is a mask (real valued function of the same definition
domain asΞ(u, v)), I(x, y) is a unit function. The decomposition in Eq. (3) is
possible according to that addition operation in spatial and frequency domain is
identical.
Let us consider maskM(u, v). To remove non-peak frequencies information it
should have ones in the frequency peak points (and harmonics) and zeros in other
frequencies. At first, corresponding peaks should be found at the frequency mag-
nitude picture. In practice, major peaks are the closest (among harmonic set) to
the (0, 0) frequency. Due to that fact, harmonics grid can be estimated (see Fig-
ure 7). After harmonics grid is found, it is worth to make the descent form the
grid points to other points smooth. Proposed instrument for that is to estimate 2-d
Gaussian in the each grid point (see Figure 8). Now when the mask is proposed, it
is possible to separate image into regular part and irregular one, according to Eq.
(3) as follows:
ξreg(x, y) = F−1{M(u, v)Ξ(u, v)}ξirr(x, y) = F−1{(I(u, v)−M(u, v))Ξ(u, v)} (4)
It is seen in Figure 9 that irregular partxiirr(x, y) looks like previously mentioned
general feature picture.
21
u
v
50 100 150 200 250 300 350
50
100
150
200
250
300
350
Figure 7: Fourier spectra magnitude and corresponding major frequencies withharmonics grid.
u
v
50 100 150 200 250 300 350
50
100
150
200
250
300
350
Figure 8: Mask with the Gaussians estimated in the grid points.
22
However, it needs to be said, that both regular and irregular parts contain compen-
sation noise near the borders which makes missing dots areas estimation harder.
This happens because the initial signalξ(x, y) has finite length, and during FFT
calculation it assumed equal to zero outside its definition domain.
Figure 9: Regular and irregular image parts.
3.3.3 Filtering using discrete cosine transform
The discrete cosine transform (DCT) is closely related to the discrete Fourier
transform. It is a separable linear transformation; that is, the two-dimensional
transform is equivalent to a one-dimensional DCT performed along a single di-
mension followed by a one-dimensional DCT in the other dimension. The def-
inition of the two-dimensional DCT for an input image A and output image B
is [20]
23
Bpq = αpαq
M−1∑m=0
N−1∑n=0
Amn cosπ(2m+1)p2M
cosπ(2n+1)q2N
,0 ≤ p ≤ M − 1
0 ≤ q ≤ N − 1
αp =
{1/√
M, p = 0√2/M, 1 ≤ p ≤ M − 1
αq =
{1/√
N, q = 0√2/N, 1 ≤ q ≤ N − 1
(5)
where M and N are the row and column size of A, respectively. If you apply DCT
to real data, the result is also real. DCT tends to concentrate information, making
it useful for image compression applications. The inverse transform for DCT is
defined as follows [20]:
Amn =M−1∑p=0
N−1∑q=0
αpαqBpq cosπ(2m+1)p2M
cosπ(2n+1)q2N
,0 ≤ m ≤ M − 1
0 ≤ n ≤ N − 1
αp =
{1/√
M, p = 0√2/M, 1 ≤ p ≤ M − 1
αq =
{1/√
N, q = 0√2/N, 1 ≤ q ≤ N − 1
(6)
In Figure 10 it is shown how certain frequency responses are much higher than
other. This fact leads to an obvious conclusion that these frequencies are respon-
sible for the texture structure.
However, the process of separating regular and irregular parts is not so simple.
According to energy conservation in image compression techniques (JPEG [21])
regular pattern can be removed as follows: sort all the DCT coefficients in the
descending order (see Figure 11) then apply some step function (see Figure 12) to
separate valuable frequencies from the others.
24
Figure 10: The DCT coefficients magnitude of the image presented in Figure 4.
25
0 0.5 1 1.5 2 2.5 3
x 104
101
102
103
104
Number of DCT coefficients
Mag
nitu
de (
log)
Figure 11: First 20% of absolute values of DCT coefficients sorted in descendingorder (presented in the logarithmic scale).
0 0.5 1 1.5 2 2.5
x 104
−1
−0.5
0
0.5
1
1.5
2
Number of DCT coefficients
Ste
p fu
nctio
n va
lue
Figure 12: Step function example.
26
Finding a proper step function is a matter of a trial method. While testing this
approach one phenomenon was found. As it can be seen in Figure 13, while
moving a step function threshold towards smaller coefficients, irregular picture
becomes less sharp, but minor noise remains nearly the same. This fact was used
to improve a step function to the appearance
T (t) =
0, t < n
1, n ≤ t ≤ m
0, t > m
(7)
wheret is a coefficient number,n is a lower coefficient number limit, andm is an
upper coefficient number limit. Tht idea of such threshold is not just to remove
major coefficients, but to cut minor coefficients tail also. In other words, to leave
only certain number of coefficients from n% to m%.
Figure 13: Initial image after removing 0.5%, 2.0%, and 5.0% DCT coefficients.
Using the DCT filtering gives less periodical noise at the borders than the DFT
filtering (compare Figure 9 and Figure 14). Also, it has to be mentioned that DCT
is significantly faster in computation than FFT.
27
Figure 14: Regular and irregular parts after the DCT filtering.
3.3.4 Gabor filters
A 2-dimensional Gabor filter can be defined as [22]
ψ(x, y; f, θ) = e−( f2
γ2 x′2+ f2
η2 y′2)ej2πfx′
x′ = x cos θ + y sin θ
y′ = −x sin θ + y cos θ
(8)
wheref is frequency,θ orientation, andγ andη control the frequency and orienta-
tion bandwidths. The normalized response of the Gabor filter,ψ(x, y), for image
ξ(x, y)
rξ(x, y, ; f, θ) =f 2
πγηψ(x, y, ; f, θ) ∗ ξ(x, y) (9)
can be used to construct a Gabor feature at any location(x, y) = (x0, y0). If
several frequencies and orientations of Gabor filter are used, a feature matrix of
filter responses at a single point can be constructed as
rξ(x0, y0, ; f0, θ0) . . . rξ(x0, y0, ; f0, θN−1)...
.. ....
rξ(x0, y0, ; fM−1, θ0) . . . rξ(x0, y0, ; fM−1, θN−1)
(10)
28
In the case of Heliotest the following frequencies and orientations were used (se-
lected empirically)f0,...,2 = 1
4, 1
8, 1
16
θ0,...,3 = 0, π4, 2π
4, 3π
4
(11)
Thus, in each point of initial image a feature vector of 12 complex values was
extracted. It is worth to mention that the extraction procedure takes quite a long
time in comparison with previously described methods (FFT, DCT).
3.3.5 Pattern modelling
One of the possible solutions for the missing dots inspection is to model the ideal
texture picture. The difference between the ideal pattern and the actual one will
show missing dots locations. This method performs pattern removal directly in
the spatial domain, not alike previously described methods which remove pattern
indirectly in frequency domain.
Let P(x,y) is an ideally modelled pattern. Then output result of pattern modelling
filtering will be
H{ξ(x, y)} = P (x, y)− ξ(x, y) = χ({xd, yd}) (12)
whereξ(x, y) is the actual image,χ({xd, yd}) is feature picture (characteristic
function of missing dots set with unknown noise). The main problem of this
approach is that an actual pattern is not enough ideal as it might seem (see Figure
4). A suggested method is to estimate pattern dots grid. Since it has generally
parallel nature, then it is worth to assign two grid basis vectors, let us name them
dx anddy. Having defined the starting point(x0, y0) (can be any actually printed
dot), the grid can be defined as
(xg, yg) = (x0, y0) + kdx + ldy (13)
29
wherek andl are integers such that(xg, yg) remains in theξ(x, y) definition do-
main.
Having estimated the dots grid it is possible to reproduce printed dot in each grid
knot. A dot can be modelled by a 2-d Gaussian curve, 2-d sine curve, or some
other hat-like function. If the dot modelling function in the grid knot(xg, yg) is
defined asD(xg ,yg)(x, y), then pattern model will look like
P (x, y) = max(xj ,yj)∈{(xg ,yg)}
D(xj ,yj)(x, y) (14)
which means that the pattern model is a maximum combination of all dot mod-
elling functions.
However, another approach seems to be more promising. Since the dots grid is
already defined, then it is better to compute some filter response based feature in
each grid knot, i.e., the average intensity of pixel neighborhood, etc. This method
will reduce filter response computations significantly, for example in a case of the
Gabor filters. Also, with the estimated grid, it is possible to operate not on the
actual image pixel level, but on the grid knots. This will eliminate the problem of
segmentation.
It is seen that the cornerstone of pattern modelling task is the dots grid estimation.
Finding dots grid in spatial domain seems to be a rather complicated problem.
However, dot coordinates can be achieved as an accompanying result of frequency
domain filtering methods, presented before (see Sections 3.3.2 and 3.3.3). While
applying these filters, regular and irregular image components are extracted. Let
us consider regular component. It models the ideal pattern structure quite well.
On the one hand, it is possible to subtract initial image from the pattern model
and to get the feature picture, but this is the same approach that was used in Sec-
tions 3.3.2 and 3.3.3 (subtraction was made directly in the frequency domain),
on the other hand it is possible to extract actual dots grid. This operation can
be performed using segmentation techniques. The first operation to be made is
thresholding, the second is labelling white areas, and the third is computing areas
30
centroid coordinates. These coordinates will comprise a grid (see Figure 15).
a b
c d
Figure 15: Pattern modelling: a) Initial image; b) Regular part from FFT filtering;c) Regular part thresholded; d) Initial image with the estimated grid (circles onthe grid knots).
This pattern modelling technique with the use of the frequency domain works
well. However, further research of pattern modelling only in the spatial domain is
also of interest.
31
3.4 Classification
3.4.1 Image segmentation
Filtering methods using DFT and DCT produce features as images of the same
size as the input one, and thus there should be more image processing techniques
applied, instead of proceeding directly to classification. As it can be seen in ir-
regular part of Figure 9 and Figure 14, the missing dot areas are large in size and
have more intensity. So, it is worthwhile to recognize missing dot by its intensity
and area. The proposed method consists of four following stages:
1. Binarize the image to a black and white image using a certain threshold.
2. Remove white areas which are smaller than a certain number of pixels.
3. Label all the separate areas.
4. Compute areas centroid coordinates and output these as missing dots loca-
tions.
This method requires the following parameters that have to be defined: a threshold
value and a minimal area. These values depend on the method implementation and
can be hard ones (numerical value), or dependent on the certain characteristics of
the given feature picture, i.e., on the image mean/minimum/maximum values.
3.4.2 Spatial features and Gabor features
Term spatial features in this context defines a set of values calculated for each
pixel in the spatial domain, which comprise a real valued feature vector for every
image point. In the case of Gabor filters are used for the feature extraction, several
complex values are produced by the filter, instead of getting a single value, like in
previously described methods. Generally speaking, for each vector (which in turn
32
represents single pixel on the image) two possible classes are defined: missing dot
pixel class and a regular pixel class.
Let us consider the classifier which produces a probability of pixel belonging
to the missing dots class. These probabilities comprise, in turn, another image,
which appearance is like the appearance of the general feature picture. This causes
another turn of segmentation and classification, using the techniques described in
Section 3.4.1.
Using Gabor features or spatial features in combination with pattern modelling
approach will eliminate need of further processing. In this case grid points, repre-
senting dots, will be classified, instead of classifying each image pixel.
For each grid knot features classification a subspace classifier presented in [23]
was used. Though, the subspace basis vectors were found using principal compo-
nent analysis (PCA) [24]. For the classification procedure following parameters
have to be defined:
• Subspace dimension.
• Maximum projection length to consider feature vector as belonging to a
missing dot class.
33
4 Automated Heliotest inspection
In this section designed algorithms are described. The description is quite thor-
ough, though presented in a more practical way. Before proceeding to the actual
methods, it needs to be said that presented algorithms are aimed at the detection
of missing dots in the halftone screen area. It is assumed, that the mentioned area
is already located and resized to the constant dimensions.
4.1 Common to all methods
Presented methods use the spectral analysis technique in different ways. This
means that they depend on the image regularity. So, it is worth to make the image
as regular as possible before applying the methods. Observing the halftone screen
area it was found that missing dots sizes change from the top of the strip to the
bottom. Actually, sizes do not change because printing cylinder of Heliotest ac-
cessory contains constant cell sizes, though the depth of the engraves varies which
means that print result intensity changes (see Figure 16). This fact leads to a an
idea to filter image parts separately, trying to preserve image regularity.
However, to keep certain frequency responses high input image should contain
enough regular information, meaning that it should be large enough. Due to this
fact, it was decided to part image in squares with the side equal to the halftone
screen area width. Each square is called a window. Such partial processing re-
quires some details to be considered. At first, as it was described in Sections 3.3.2
and 3.3.3, frequency domain filtering output lacks sharpness on the borders. This
means that a certain area near the borders of the image should be considered as
a false response area and not used in the classification. Thus, the actual region
of interest (ROI) is a bit smaller than the window size. The difference between
ROI and window is called window padding. Further, by cutting image into win-
dows, it is seen that there are gaps between the ROIs of two neighbor windows and
consequently there is a probability of losing some missing dots while performing
34
the inspection. To remove such gaps two neighbor windows should overlap. The
graphical presentation of such partitioning can be seen in Figure 17.
a b
Figure 16: Pattern difference of the halftone screen area: (a) At the top; (b) At thebottom.
Window
Window Padding
ROI Overlap
Figure 17: Sample partitioning.
After the image has been processed and missing dots have been detected, collected
information should be combined since there is a probability that some missing
dots were detected twice (in the overlap area). After the data is summarized, it is
ready for further considerations, i.e., for determining the quality of paper.
35
4.2 FFT based method
This method is based on the Fourier domain filtering presented in Section 3.3.2.
Designed filter outputs irregular image component which, in turn, is processed
using segmentation techniques described in Section 3.4.1. The method has been
summarized in Algorithm 1.
Algorithm 1 FFT based method
1. Produce digital image.
2. Preprocess the image.
3. Perform image partitioning.
4. With each image part do:
4.1 Extract the irregular component using FFT method.
4.2 Classify the irregular component.
4.3 Compute missing dots coordinates.
5. Combine missing dots coordinates from each image part.
6. Compute sample quality.
4.2.1 Preprocessing
The preprocessing stage includes simple median filtering [18] with a small domain
size (in comparison with the dot size). This filtering removes minor noise from
the image. Then the image is being normalized to the intensity level[0, 1] and
afterwards the image mean intensity value is being subtracted. Normalization is
required to reduce differences in intensity between samples. The subtraction of
the mean value prevents getting a high response in the central frequency(0, 0).
36
4.2.2 Extraction of the irregular component
The extraction of irregular image components was described in Section 3.3.2.
Some practical details need to be mentioned such as mask extraction. This proce-
dure includes following stages:
1. FFT magnitude computation.
2. Search of the maximum response frequency.
3. Estimation of harmonic frequencies set.
4. Computation of 2-d Gaussians in the grid points.
The first two steps are clear enough. However, the other two need explanation. As
it was experimentally found, the maximum frequency response ((uM , vM) on the
frequency plane) is the closest to the central frequency(0, 0) among the harmonic
set. With the mirrored frequency at(−uM , vM) they comprise the basis of the
harmonic set. So, each point of the harmonic set can be defined as
(uh, vh) = k(uM , vM) + l(−uM , vM) (15)
wherek, l ∈ Z. However, since the frequency plane is discrete, this harmonics set
estimation is not accurate. So, harmonics grid points need to be adjusted to actual
local maxima. This adjustment is performed by looking for a local maximum
in a certain neighborhood. This neighborhood was defined as a rectangular area
with the first approximation point in the centre which (area) should contain only
one local maximum from the harmonics set, i.e., not exceed sizesuM/2× vM/2.
Then, the first approximation point is replaced with the corresponding local area
maximum. In practice, the first approximation is already so good that it does not
miss the actual harmonics set points more than 5 pixels. So, the adjustment areas
can be reduced significantly. Gaussian estimation in the found points is based on
37
the values ofuM andvM and can be defined as:
Gh(u, v) = exp (−uM(u− uh)2 + vM(v − vh)
2
2σ) (16)
whereσ was found empirically, according to filtering results. It was defined as
σ =
√u2
M+v2M
10.
Filter output (feature picture) is adjusted before proceeding to the next algorithm
step. At first, all the values which are lower than the image mean are made equal
to mean value, and then image is normalized to fit intensity interval[0, 1]. This is
made to unify filter response for different inputs.
4.2.3 Classification
As it was stated in Section 3.4.1, the classification procedure consists of four
steps. This process requires two parameters to be defined: a threshold valueτ to
be applied and a certain areaS for the opening procedure [9]. The thresholding
operatorT is defined as follows:
T{ξ(x, y)} =
{0, ξ(x, y) < τ
1, ξ(x, y) ≥ τ(17)
This is actually a hard threshold, i.e.,τ is a certain intensity level. In this problem
case, such threshold is inappropriate. For example, if feature picture contains
no irregularities, which happens when there no missing dots in the input image,
then hard thresholding gives false information and the following result will be
incorrect. This problem can be solved by using proportional threshold coefficient
τ , based on the image mean valueµξ
T{ξ(x, y)} =
{0, ξ(x, y) < τµξ
1, ξ(x, y) ≥ τµξ(18)
38
Black-white opening is like the thresholding, but it removes areas in the picture
which are less than S pixels in size. These two parametersτ andS were estimated
using cross-validation technique. At first, a definition domain was assigned for
both parameters empirically
τ ∈ 1, 1.5, 2, 2.5...20;
S ∈ 1, 2, 3...20.(19)
Then the method was executed using the training set, and the best matching pa-
rameters were found. It should be noticed that each image part contains its own
parameters.
4.3 DCT based method
This method is based on the DCT based filtering presented in Section 3.3.3. The
whole approach is similar to the one described in Section 4.2, except for the use
of the DCT filter described in Section 3.3.3. The method has been summarized in
Algorithm 2.
Algorithm 2 DCT based method
1. Produce digital image.
2. Preprocess the image.
3. Perform image partitioning.
4. With each image part do:
4.1 Extract the irregular component using DCT method.
4.2 Classify the irregular component.
4.3 Compute missing dots coordinates.
5. Combine missing dots coordinates from each image part.
39
6. Compute sample quality.
4.4 Pattern modelling with spatial features method
This method uses the pattern modelling technique described in Section 3.3.5. Pre-
processing stage is identical to the one presented in Section 4.2. This method is
considered to be the most promising one. An important part of the method is the
grid extraction part. The classifier is made according to the proposal in Section
3.4.2. The method has been summarized in Algorithm 3.
Algorithm 3 Pattern modelling with spatial features method
1. Produce digital image.
2. Preprocess the image.
3. Perform image partitioning.
4. With each image part do:
4.1 Extract the pattern grid.
4.2 Classify each pattern grid knot.
4.3 Compute missing dots coordinates.
5. Combine missing dots coordinates from each image part.
6. Compute sample quality.
4.4.1 Grid extraction
The grid extraction part of this method is based on the technique presented in
Section 3.3.2. Though, instead of the irregular part of the input image, it considers
40
the regular component. The regular part from the FFT filter is better than the one
achieved by DCT filtering. Somehow, it contains less noise and therefore it is
more suitable for the grid extraction.
The grid extraction consists of the following stages:
1. Forward FFT transform.
2. FFT magnitude computation.
3. Mask extraction.
4. Mask application.
5. Inverse FFT transform.
6. Threshold application.
7. Segmentation.
8. Areas centroids computation.
The mask extraction procedure is identical to the one described in Section 4.2.
The only difference is that not the original mask is applied, but the inverse one
M = I(u, v) − M(u, v). When the filtering is made the image is converted to
black and white using a certain threshold. In this case a proportional threshold is
used, based on the output image mean. Since the regular output of the FFT filter
contains nearly no noise a segmentation procedure does not include noise removal.
So, after thresholding has been done, each white area on the image is labelled and
its centroid coordinates are computed. The actual dots grid is determined in(x, y)
spatial coordinates of each dot center estimation.
41
4.4.2 Classification
After dots grid has been estimated it is possible to observe each dot for making
a decision, whether the dot is missing or not. In Figure 18 it is possible to see
several dot observations.
a
e
b
f
c
g
d
h
Figure 18: Dot samples: (a)-(c) regular dots; (d) regular dot expectation; (e)-(g)missing dots; (h) missing dot expectation.
It was decided to use each grid knotn×n neighborhood as a feature vector andn
was selected the way it covers all the dot area. Finally, feature vector consists of
n2 values, each is in[0, 255] interval (8bit greyscale).
On the other hand, it is possible to extract not just spatial features in each grid
knot, but to use another class of features, namely Gabor features, described in
3.3.4.
42
5 Experiments and results
Every method proposal should be finalized with practical experiments. In this
section real data experiments are described and actual algorithmic decisions are
covered.
5.1 Data
5.1.1 Heliotest samples
Data for this project consist of a set of Heliotest print results which were printed
by the author and Dr. J.-K. Kämäräinen in the quality control lab of a paper mill.
Total number of the test prints obtained is 150 which include 10 different types
of coated paper. Printing of each sample took from 10 to 40 seconds, depending
on the ink consumption. Test prints were made on one printing device, using one
IGT printing cylinder which is actually not good in the sense of generality.
Original samples were digitized using a flatbed scanner. For the purposes of fur-
ther research it was decided to digitize as much information from test prints as
possible. Therefore, the resolution of 1200dpi and color depth of 24bit were de-
fined. One image scan occupies approximately 55-70 megabytes of hard drive
space. Scanning process took about 8 minutes for each sample. In practice we
need less information so the scanning process can consume significantly less time.
5.1.2 Training and testing data
This work is concentrated on the missing dots detection in the halftone screen
area. Therefore, this part of digital image must be located, whether automatically
or manually. Also a certain amount of missing dots has to be picked for testing and
43
training data. For the aforementioned needs the author developed an application
(see Figure 19) which allows to locate halftone screen area on the scanned sample
and to mark missing dots. This application was developed in Visual C++, using
MFC [25], and works under MS Windows operation system. It saves area corners
coordinates and missing dots locations in a plain text file. The half of the samples
were fully examined and missing dots locations were marked. Another half was
partially marked, i.e., only the first 20 dots from the beginning.
Figure 19: Screen shot of an application for halftone screen area location andmissing dots picking.
5.2 Experiments
The training set contained 75 images, fully marked with missing dot locations.
Experiments were tested on another set of 70 images with only first 20 missing
dots marked. However, all the statistical results used combined information of the
performance of the method on both training and testing sets. Before proceeding
44
to the actual experiments, it needs to be said that all the data sets were unified,
i.e., acquired images were resized to the region of interest (halftone screen area)
using corresponding coordinates. The training and testing sets (manually marked
missing dots) were recalculated according to the new ROI position. Original ROI
dimensions were changed to 5000 pixels (height) by 400 pixels (width).
5.2.1 FFT based method
Using supervised learning the following method parameters were esimated (see
Table 3)
Table 3: Segmentation parameters of the FFT based method.
Window number Threshold coefficient (τ ) Area (S)1 8.5 122 9 143 9.5 144 9.5 115 11 96 9 117 11 88 12 69 10 810 11.5 411 12.5 412 13 413 12 414 10 415 10 4
45
The method performance was measured in several parameters:
1. Number of missing dots detected out of the first 20 manually marked dots.
2. Number of false alarms (falsely detected dots) in the area before 20th man-
ually marked dot.
3. Actual quality (distance in millimeters from the beginning of the halftone
screen area till the 20th missing dot detected).
4. Difference between the results achieved by the method and the manual eval-
uation.
All results for each sample can be seen in Appendix 1. The performance summary
has been presented in Table 4.
Table 4: Performance summary of the FFT based method.
Average number of missing dots detected out of the first 20 marked17,90344828Average number of false alarms before the first 20 marked 6,282758621Mean square error in quality measurement 4,300962202Time needed for one sample processing ≈90 seconds
5.2.2 DCT based method
Using supervised learning the method parameters were estimated as shown in
Table 5.
Full data for each sample observation can be seen in Appendix 1. In this part, the
method performance is presented in Table 6.
46
Table 5: Segmentation parameters of the DCT based method.
Window Number Threshold coefficient (τ ) Area (S)1 7.5 122 8 143 8.5 144 8.5 115 10 96 8 117 10 88 11 69 9 810 10.5 411 11.5 412 12 413 11 414 9 415 9 4
Table 6: Performance summary of the DCT based method.
Average number of missing dots detected out of the first 20 marked17,69655172Average number of false alarms before the first 20 marked 3,075862069Mean square error in quality measurement 2,539949014Time needed for one sample processing ≈40 seconds
47
5.2.3 Pattern modelling with spatial features method
During observation the method parameters were estimated as shown in Table 7).
Table 7: Classification parameters for the pattern modelling with spatial featuresmethod.
Subspace dimensionMaximum projection length25 50
All results for each sample observation can be obtained from Appendix 1. In thispart, method performance summary (see Table 8) is presented.
Table 8: Performance summary for the pattern modelling with spatial featuresmethod.
Average number of missing dots detected out of first 20 marked17,71034483Average number of false alarms before first 20 marked 0,95862069Mean square error in quality measurement 2,390654698Time needed for one sample processing ≈120 seconds
48
5.3 Results and discussions
The results are considered to be quite good. It is seen that the DCT based method
performs a little better than the FFT based one. Even though, they both use nearly
the same frequency based approach. Also the DCT based algorithm works faster
since it requires less processing stages (no mask estimation, etc.) and generally,
DCT implementations are faster than FFT.
Although the DCT and FFT based methods perform well, best results are achieved
by the pattern modelling method. It is seen that classification in which pattern el-
ements are considered instead of image units gives significantly less false alarms,
and, in turn, produces better measurement accuracy.
More detailed methods comparison is of some interest. For example, do the false
alarms of both methods intersect, and if it is true then in what image parts they
respond a missing dot. It is maybe also worthwhile to combine all methods results
to achieve better measurement accuracy.
It is also clear that for the better results more testing data is needed. Data samples
should cover different types of Heliotest print results of paper and board with
different quality.
49
6 Conclusion
In this work several machine vision techniques for automated Heliotest inspection
were presented. It was found that spectral analysis idea works well in the case of
missing dots detection. All of the presented methods use this approach in different
ways. The best results were achieved with the pattern modelling with spatial
features method. This is because it classifies missing dots in a natural way, like if
the test is conducted by a human expert.
Future research will include:
• The current methods improvement.
• Data set increase.
• New methods development.
• Comparison with human expert results.
• Machine vision system design.
This work can be considered as the first step in the Heliotest automation research.
On this stage it is possible to say that machine vision system implementation in
printability testing process has a potential chance of success.
50
References
[1] PAPVISION project official website. Lappeenranta Uni-
versity of Technology, Information Technology Department,
http://www.it.lut.fi/project/papvision/, 2003. Referred on 12.2003.
[2] H. Juslin and E. Hansen.Strategic Marketing in the Global Forest Industries.
Authors Academic Press, 2003. ISBN 0-9703333-7-4.
[3] P. Oittinen and H. Saarelma.Printing. Papermaking Science and Technol-
ogy. Fapet Oy, 1998. ISBN 952-5216-13-6.
[4] J.-E. Levlin and L. Söderbjelm.Pulp and Paper Testing. Papermaking Sci-
ence and Technology. Fapet Oy, 1999. ISBN 952-5216-17-9.
[5] International Paper Corporation.IP coated official website. International
Paper, http://ipcoated.com/, 2003. Referred on 12.2003.
[6] Centre Technique du Papier.The Pulp and Paper Research and Technical
Centre. WWW, http://www.webctp.com/, 2003. Referred on 12.2003.
[7] IGT Testing Systems.IGT official website. WWW, http://www.igt.nl/, 2003.
Referred on 12.2003.
[8] IGT Testing Systems.IGT Information leaflet W41. IGT, 2003.
[9] R. C. Gonzalez and R. E. Woods.Digital Image Processing. Prentice-Hall,
Inc., 2002. ISBN 0-201-18075-8.
[10] Techtarget Network. Technical Dictionary. Techtarget Inc.,
http://whatis.techtarget.com/, 2003. Referred on 12.2003.
[11] H. Kälviäinen, P. Saarinen, P. Salmela, A. Sadovnikov, and A. Drobchenko.
Inspection on paper by machine vision.Proceedings of SPIE Conference on
Intelligent Robots and Computer Vision XXI: Algorithms, Techniques and
Active Vision, 2003. Providence, Rhode Island, USA.
51
[12] A. Kumar and G.K.H. Pang. Defect detection in textured materials using
optimized filters. IEEE Transactions on Systems, Man and Cybernetics,
32:553–570, October 2002.
[13] D. Chetverikov. Structural defects: general approach and application to tex-
tile inspection.Proceedings of the 15th International Conference on Pattern
Recognition, 1:521–524, September 2000.
[14] L.H. Monteiro and A. Conci. Multifractal characterization of texture-based
segmentation.International Conference on Image Processing, 1:792–795,
September 2000.
[15] L. Macaire and J.G. Postaire. Automated visual inspection of galvanized and
painted metallic strips.CompEuro ’93. ’Computers in Design, Manufactur-
ing, and Production’, Proceedings., pages 8–15, May 1993.
[16] IEEE Standard 610.4-1990.IEEE Standard Glossary of Image Processing
and Pattern Recognition Terminology.
[17] A. K. Jain.Fundamentals of Digital Image Processing. Prentice Hall, 1989.
ISBN 0-13-336165-9.
[18] Y. Zou and W.T.M. Dunsmuir. Generalized max/median filtering.Proceed-
ings of International Conference on Image Processing, 1:428–431, October
1997.
[19] B. Jähne. Digital Image Processing:Concepts, Algorithms and Scientific
Applications. Springer-Verlag, 1991. ISBN 3-540-53782-1.
[20] The MathWorks Inc. The MathWorks official website. WWW,
http://www.mathworks.com, 2003. Referred on 12.2003.
[21] JPEG Group.Official JPEG website. http://www.jpeg.org, 2003. Referred
on 12.2003.
[22] J.-K. Kämäräinen, V. Kyrki, H. Kälviäinen, M. Hamouz, and J. Kittler. In-
variant gabor features for evidence extraction.Proceedings of the IAPR
Workshop on Machine Vision Applications, pages 228–231, 2002.
52
[23] E. Oja and T. Kohonen. The subspace learning algorithm as a formalism for
pattern recognition and neural networks.IEEE International Conference on
Neural Networks, 1:277–284, 1988.
[24] R. O. Duda, P. E. Hart, and D. G. Stork.Pattern Classification. Wiley-
Interscience, 2001. ISBN 0-471-05669-3.
[25] Microsoft. Microsoft Developers’ Network. Microsoft Corp.,
http://msdn.microsoft.com/, 2003. Referred on 12.2003.
53