Advisor: Dr. Sreela Sasi. Introduction Image Colorization WHAT: Adding color to monochrome images...
If you can't read please download the document
Advisor: Dr. Sreela Sasi. Introduction Image Colorization WHAT: Adding color to monochrome images WHEN: Performed since the early 20 th century WHY: Improve
Introduction Image Colorization WHAT: Adding color to
monochrome images WHEN: Performed since the early 20 th century
WHY: Improve visual appeal of illustrations HOW: A painstaking and
subjective manual task 2
Slide 3
Introduction (contd.) Digital Image Colorization Automation of
colorizationImprove visual appeal of imagesColor accuracy, finer
details Add relevant information to images Make images more
understandable 3
Slide 4
Introduction (contd.) Applications of Image Colorization
Applications Homeland Security Satellite Imaging Old photos and
films Medical Imaging Video compression 4
Slide 5
Colorization Techniques Scribble-based colorization User add
color scribbles to image to be colorized laborious, time-
consuming, subjective, and painstaking manual task. Example-based
colorization automation by extracting colors from sample image
results can vary depending on example image chosen +=+= Previous
Research Image Colorization 5
Slide 6
Current Research Process Workflow Texture-based Segmentation
Image Sample Image Feature Extraction Color Descriptors Texture
Descriptors New Grayscale Image New Grayscale Image Texture-based
Segmentation Feature Extraction Texture Descriptors Texture
Matching Colorization Process Database 6
Slide 7
Image Segmentation Image segmentation: Is the partitioning of
an image into homogeneous regions based on a set of
characteristics. Is a key element in image analysis and computer
vision. 7
Slide 8
Image Segmentation (contd.) Clustering: Is one of the methods
available for image segmentation. Is a process which can be used
for classifying pixels based on similarity according to the pixels
color or gray-level intensity. 8
Slide 9
Image Segmentation (contd.) Despite the substantial amount of
research performed to date, the design of a robust and efficient
clustering algorithm remains a very challenging problem 9
Slide 10
Color-based Image Segmentation Composite Image 10
Slide 11
Color-based Image Segmentation Composite Image with salt &
pepper noise added 11
Slide 12
Texture-based Image Segmentation 12
Slide 13
Workflow Process Texture-Based Image Segmentation Original
Image Filtered Image Feature Image Blobs Gabor Filters Energy
Computation Segmentation Add, mean smoothing, normalization 13
Previous Research (contd.) Texture-Based Segmentation 15
Slide 16
16 Image Segmentation Normalized Sum of Gabor Responses
Slide 17
Current Research Process Workflow Texture-based Segmentation
Image Sample Image Feature Extraction Color Descriptors Texture
Descriptors New Grayscale Image New Grayscale Image Texture-based
Segmentation Feature Extraction Texture Descriptors Texture
Matching Colorization Process Database 17
Slide 18
Previous Research (contd.) Clustering and Feature Extraction
18
Slide 19
Previous Research The K-means algorithm has been used for a
fast and crisp hard segmentation. The Fuzzy set theory has improved
this process by allowing the concept of partial membership, in
which an image pixel can belong to multiple clusters. This soft
clustering allows for a more precise computation of the cluster
membership, and has been used successfully for image clustering and
segmentation. 19
Slide 20
The Fuzzy C-means clustering (FCM) algorithm [1] is a widely
used method for soft image clustering. However, the FCM algorithm
is computationally intensive. It is also very sensitive to noise
because it only iteratively compares the properties of each
individual pixel to each cluster in the feature domain. Previous
Research (contd.) [1]James C. Bezdek, Pattern Recognition with
Fuzzy Objective Function Algorithms. New York: Plenum, 1981.
20
Previous Research (contd.) Fuzzy C-means clustering (FCM)
Algorithm 22
Slide 23
Previous Research (contd.) FCM Pseudo-code Step 1 Set the
number c of clusters, the fuzzy parameter m, and the stopping
condition Step 2Initialize the fuzzy membership values Step 3Set
the loop counter b = 0 Step 4Calculate the cluster centroid values
using (3) Step 5For each pixel, compute the membership values using
(4) for each cluster Step 6Compute the objective function A. If the
value of A between consecutive iterations < then stop, otherwise
set b=b+1 and go to step 4 23
Slide 24
[2]Stelios Krinidis and Vassilios Chatzis, "A Robust Fuzzy
Local Information C-means Clustering Algorithm," Image Processing,
IEEE Transactions on, pp. 1-1, 2010. Previous Research (contd.)
Modified Fuzzy C-means clustering with G ki factor In order to
improve the tolerance to noise of the Fuzzy C-means clustering
algorithm, Krinidis and Chatzis [2] have proposed a new Robust
Fuzzy Local Information C-means Clustering (FLICM) algorithm by
introducing the novel G ki factor. The purpose of this algorithm is
to adjust the fuzzy membership of each pixel by adding local
information from the membership of neighboring pixels. 24
Slide 25
Previous Research (contd.) Modified Fuzzy C-means clustering
with G ki factor Sliding window of size 1 around the i th pixel The
G ki factor is obtained by using a sliding window of predefined
dimensions: 25
Slide 26
Previous Research (contd.) Modified Fuzzy C-means clustering
with G ki factor The G ki factor is calculated by using the
following equation: 26
Slide 27
Current Algorithm Modified Fuzzy C-means clustering with novel
H ik factor This algorithm is further improved by including both
the local spatial information from neighboring pixels and the
spatial Euclidian distance of each pixel to the clusters center of
gravity. In this research, the algorithm is also extended for
clustering of color images in the Red-Green-Blue (RGB) color space.
27
Slide 28
Current Algorithm (contd.) Illustration of the new H ik factor
displaying the spatial Euclidian distance to the center of gravity
of each cluster 28
Slide 29
Current Algorithm (contd.) Process Workflow Customize
Parameters Calculate cluster membership values Compute G ki
Readjust membership values Compute H ki Compute objective function
Defuzzification and clustering - - Image Calculate cluster centroid
29
Slide 30
Current Algorithm (contd.) Modified Fuzzy C-means Clustering
30
Slide 31
Simulation and Results Synthetic Grayscale Test Image 31
Slide 32
Natural test image FCM segmentation with 5 clusters FCM
segmentation using the modified FCM algorithm with 5 clusters, G ki
window=1 and H ik Simulation and Results Natural Test Image 32
Slide 33
Simulation and Results Synthetic Grayscale Test Image Synthetic
4-color test image with added salt and pepper noise FCM clustering
with G ki window=1 and with H ik FCM clustering with G ki window=5
and with H ik 33
Slide 34
Synthetic 4-color test image with added salt and pepper noise
FCM clustering with G ki window=1 and with H ik FCM clustering with
G ki window=5 and with H ik Simulation and Results Synthetic Color
Test Image 34
Slide 35
Image Segmentation Clustering Demo 35
Slide 36
Modified Fuzzy C-means Clustering Summary In this research, the
FCM with the G ki factor is modified using the H ik factor, and the
algorithm is extended for the clustering of color images. The use
of the sliding window in the G ki factor improves the segmentation
results by incorporating local information about neighboring pixels
in the membership function of the clusters. However, this resulted
in a significant increase in the number of calculations required
for each iteration for each pixel, and can be given by 36
Slide 37
Modified Fuzzy C-means Clustering Summary (contd.) By combining
the G ki and the H ik factors, this modified FCM algorithm
considerably reduced the number of iterations needed to achieve
convergence. The tolerance to noise of the Fuzzy C-means algorithm
is also greatly increased, allowing for an improved capability to
obtain coherent and contiguous segments from the original image.
37
Slide 38
Modified Fuzzy C-means Clustering Summary (contd.) However,
because of the radial nature of the spatial Euclidean distance to
the clusters center of gravity, this new method may not be as
effective for images containing circular shapes, or for images
where the clusters center of gravity are close to each-other. In
this research, the FCM is extended for the clustering of color
images in the RGB color space. The effectiveness of this algorithm
may be tested for images in other color spaces also. 38
Slide 39
Current Research Process Workflow Texture-based Segmentation
Image Sample Image Feature Extraction Color Descriptors Texture
Descriptors New Grayscale Image New Grayscale Image Texture-based
Segmentation Feature Extraction Texture Descriptors Texture
Matching Colorization Process Database 39
Slide 40
40 Sample Color Images
Slide 41
41 Image Segmentation Normalized Sum of Gabor Responses
Slide 42
Image Segmentation Feature Extraction 42
Slide 43
Image Segmentation Feature Extraction (contd.) 43 Blob
Filtering for color and texture extraction.
Slide 44
44 Texture and Color database Image Segmentation Feature
Extraction (contd.)
Slide 45
45 Current Research Process Workflow Texture-based Segmentation
Image Sample Image Feature Extraction Color Descriptors Texture
Descriptors New Grayscale Image New Grayscale Image Texture-based
Segmentation Feature Extraction Texture Descriptors Texture
Matching Colorization Process Database
Slide 46
46 Grayscale Image Processing
Slide 47
47 Current Research Process Workflow Texture-based Segmentation
Image Sample Image Feature Extraction Color Descriptors Texture
Descriptors New Grayscale Image New Grayscale Image Texture-based
Segmentation Feature Extraction Texture Descriptors Texture
Matching Colorization Process Database
Slide 48
48 Previous Research Visual descriptors Visual descriptors are
descriptions of the visual features of the contents of images. They
describe elementary characteristics such as the shape, color, and
texture. MPEG-7 is a multimedia content description standard. It
was standardized in ISO/IEC 15938 (Multimedia content description
interface). This description is associated with the content itself,
to allow fast and efficient searching for material that is of
interest to the user. MPEG-7 is formally called Multimedia Content
Description Interface. Thus, it is not a standard which deals with
the actual encoding of moving pictures and audio, like MPEG-1,
MPEG-2 and MPEG-4. It uses XML to store metadata.
Slide 49
49 Previous Research Visual descriptors
http://chatzichristofis.info/?page_id=213 The Img(Rummager)
application was developed in the Automatic Control Systems &
Robotics Laboratory at the Democritus University of Thrace-Greece.
The application can execute an image search based on a query image,
either from XML-based index les, or directly from a folder
containing image les, extracting the comparison features in real
time.
Slide 50
Previous Research (contd.) Content-Based Image Retrieval
50
Image Descriptors used: MPEG-7 Homogeneous Texture Descriptor:
Edge Histogram Descriptor (EHD). CCD for Medical Radiology Images:
Brightness and Texture Directionality Histogram (BTDH) Fuzzy rule
based scalable composite descriptor (BTDH) is a compact composite
descriptor that can be used for the indexing and retrieval of
radiology medical images. This descriptor uses brightness and
texture characteristics as well as the spatial distribution of
these characteristics in one compact 1D vector. The most important
characteristic of the proposed descriptor is that its size adapts
according to the storage capabilities of the application that is
using it. This characteristic renders the descriptor appropriate
for use in large medical (or gray scale) image databases.
Simulation Results (contd.) Content-Based Image Retrieval (CBIR)
52
54 Current Research Process Workflow Texture-based Segmentation
Image Sample Image Feature Extraction Color Descriptors Texture
Descriptors New Grayscale Image New Grayscale Image Texture-based
Segmentation Feature Extraction Texture Descriptors Texture
Matching Colorization Process Database
Slide 55
The RGB color space is defined by the three chromaticities of
the red, green, and blue additive primaries, and can produce any
chromaticity that is the triangle defined by those primary colors.
The YCbCr color space is used in video and digital photography
systems. Y is the luma (luminance ) component and Cb and Cr are the
blue-difference and red-difference chroma components. Simulation
Results (contd.) Image Colorization 55
Slide 56
56Image from Wikipedia Simulation Results (contd.) Image
Colorization
Slide 57
Simulation Results (contd.) Colorization 57
Slide 58
Conclusion and Future Work New and innovative method Automating
example-based colorization Combines several state-of-the-art
techniques Reasonably accurate results were obtained Several of the
steps require custom parameters computationally very intensive
Texture retrieval needs improvement Complex textures containing
multiple colors Anisotropic diffusion for preserving strong edge
information Combining these techniques in order to automatically
colorize grayscale images is a viable option 58
Slide 59
Conclusion and Future Work (contd.) Images segmentation and
clustering methods computationally very intensive, Processing time
for each 600x450 sample color image took 20 minutes on a quad-core
Intel 2.6 GHz processor. Texture retrieval methods still need to be
improved for scale and rotation invariance Store more complete
color descriptors to accommodate more complex textures containing
multiple colors. Anisotropic diffusion could also be used to smooth
the Gabor response images while preserving strong edge information.
Testing conducted as part of this research proved that the ability
to combine these techniques in order to automatically colorize
grayscale images is a viable option. 59
Slide 60
References [1]Anat Levin, Dani Lischinski, and Yair Weiss,
"Colorization using optimization," ACM Transactions on Graphics,
vol. 23, no. 3, p. 689694, 2004. [2]R. Irony, D. Cohen-Or, and D.
Lischinski, "Colorization by example," in Eurographics Symposium on
Rendering, 2005, p. 277280. [3]Ashikhmin M., Mueller K. Welsh T.,
"Transferring Color to Greyscale Images,". [4]X., Wan L., Qu Y.,
Wong T., Lin S., Leung C., Heng P. Liu, "Intrinsic colorization,"
ACM Trans. Graph., vol. 27, no. 5, p. 152, 2008. [5]Malik J. Perona
P., "Preattentive texture discrimination with early vision
mechanisms," J. Opt. Soc. Am. A, vol. 7, no. 5, May 1990. [6]A. K.
Jain and F. Farrokhnia, "Unsupervised texture segmentation using
Gabor filters," Pattern Recognition, vol. 24, no. 12, pp.
1167-1186, 1991. [7]Seo Naotoshi, "Texture Segmentation using Gabor
Filters," University of Maryland, College Park, MD, Project
ENEE731, 2006. [8]Xiaoming Hu, Xinghui Dong, Jiahua Wu, Ping Zou
Junyu Dong, "Texture Segmentation Based on Probabilistic Index
Maps," in International Conference on Education Technology and
Computer, 2009, pp. 35-39. 60
Slide 61
References (contd.) [9]X Muoz, J Freixeneta, X Cufa, and J
Marta, "Strategies for image segmentation combining region and
boundary information," Pattern Recognition Letters, vol. 24, no.
1-3, pp. 375-392, January 2003. [10]James C. Bezdek, Pattern
Recognition with Fuzzy Objective Function Algorithms. New York:
Plenum, 1981. [11]Chuang Keh-Shih, Tzenga Hong-Long, Chen Sharon,
Wu Jay, and Chen Tzong-Jer, "Fuzzy c-means clustering with spatial
information for image segmentation," Computerized Medical Imaging
and Graphics, vol. 30, no. 1, pp. 9-15, January 2006. [12]Zhou
Huiyu, Schaefer Gerald, Sadka Abdul H., and Celebi M. Emre,
"Anisotropic Mean Shift Based Fuzzy C-Means Segmentation of
Dermoscopy Images," IEEE Journal of Selected Topics in Signal
Processing, vol. 3, no. 1, pp. 26-34, February 2009. [13]Stelios
Krinidis and Vassilios Chatzis, "A Robust Fuzzy Local Information
C-means Clustering Algorithm," Image Processing, IEEE Transactions
on, pp. 1-1, 2010. [14]Gauge Christophe and Sasi Sreela, "Automated
Colorization of Grayscale Images Using Texture Descriptors and a
Modified Fuzzy C-Means Clustering, Journal of Intelligent Learning
Systems and Applications (JILSA), Vol. 4 No. 2, 2012, pp. 135- 143,
DOI: 10.4236/jilsa. 61
