Upload
noelia-yeatts
View
215
Download
1
Tags:
Embed Size (px)
Citation preview
Introduction
• RICH project (Reading Images for the Cultural Heritage)
• Initiated by NWO-CATCH (grant 640.002.401)
• Institutions involved:– MICC-IKAT (Maastricht University)– ROB (Dutch State Service for Archaeology)
• People involved:– E.O. Postma, A.G. Lange, H.H. Paijmans,
L.J.P. van der Maaten, P.J. Boon
Introduction
• Automatic coin classification based on visual features– May allow sorting heterogeneous coin
collections, both modern and historical– For modern coins, applications for charity
organizations, financial institutions, and change offices (MUSCLE CIS benchmark)
– For historical coins, applications in the cultural heritage domain
Introduction
• Currently, some coin historical collections are being disclosed on the Internet, e.g., in NUMIS1
• NUMIS website shows information (and sometimes photographs) on collected coins
• However, the use of such websites for non-experts is limited
1 A project of the Dutch Money Museum
Introduction
• Non-experts who find a coin would like to know what sort of coin it is– i.e. coin classification based on visual
features
• Non-experts would benefit from a system for automatic coin classification
• Also beneficial to experts to speed up and objectify the classification process
Introduction
• This presentation– Presents a number of features that can be
used for the classification of modern coins– Shows promising results for these features– Investigates the performance of the same
features on a medieval coin dataset– Tries to provide some insight in why the
features fail on the medieval coin data
Features
• Contour features– Edge distance distributions– Edge angle distributions– Edge angle-distance distributions
• Texture features– Gabor histograms– Daubechies D4 wavelet features
Contour features
• Measure statistical distributions of edge pixels
• Edge pixels computed using Sobel filter convolution (with non-maxima suppression and dynamic thresholding)
• Coin borders are removed
Edge distance distributions
• Estimate the distribution of the distances of edge pixels to the center of the coin
• Rotation invariant feature
• Can be measured on coarse-to-fine-scales
Edge angle distributions
• Measure distribution of angles of edge pixels w.r.t. the baseline
• Not rotation invariant by definition (however, the magnitude of the Fourier transform is)
• Can be measured on number of fine scales
Edge angle-distance distr.
• Incorporate both angular and distance information in the coin stamp
• We measure EADD using 2, 4, 8, and 16 distance bins and 180 angular bins
Gabor histograms
• Convolution of coin image with Gabor filters of various scales and rotations
• Compute image histograms of the resulting convolution images
• Apply PCA for dimensionality reduction (200 dimensional)
Daubechies D4 wavelet
• Perform wavelet decomposition using Daubechies D4 wavelet
• Computed wavelet coefficients are used as features (2-, 3-, and 4-level; ahvd)
• Do this for 16 rotated versions of the coins in the training set (for rotation invariance)
• Apply PCA for dimensionality reduction (results in 200-dimensional feature vector)
Experiments
• Performed on the MUSCLE CIS benchmark coin dataset1
• The dataset contains 692 coin classes with 2,270 coin faces
• Training set of 20,000 coins
• Test set of 5,000 coins
• Incorporate area measurements
1 Newer experiments than the ones described in the paper
Experiments
• Classification performances (5-NN classifier)
Edge distance distributions 68%
Edge angle distributions 62%
Edge angle-distance distributions 78%
Gabor histograms 55%
Daubechies D4 wavelet features 46%
Experiments
• Subsequently, we performed experiments on the Merovingen dataset1
• Contains 4,569 early-medieval coins
• Class distribution skewed• Experiments using 10-
fold cross validation
1 Dataset property of Dutch Money Museum
Experiments
• Skewed class distributions
Class type No. of classes
Mean class size
St. dev. of class size
City 18 53 125
Mint master 19 69 121
Currency 4 859 1,469
Nation 12 199 438
Experiments
• Classification performances (naïve Bayes classifier)
Feature City Mint master Currency Nation
Area 16% 10% 61% 17%
Edge dist. distr. 12% 8% 50% 20%
Edge angle distr. 8% 6% 34% 14%
Gabor histogram 5% 6% 25% 8%
D4 wavelet feat. 8% 5% 25% 6%
Discussion
• Although results on modern coin data are promising
• Results on Merovingen coin dataset disappointing
Discussion
• Reasons for results on medieval coins:– Contour features highly rely on the correct
estimation of the center of the stamp– Texture features more suitable for coins with
detailed artwork in stamps– Errors and inconsistencies in these kind of
datasets
Discussion
• Reasons for results on medieval coins :– Medieval coins have larger within-class
variances due to quick deterioration of medieval coin stamps
– Medieval coins have smaller between-class variances (stamps often contain similar pictures, such as a cross or the head of an authority)
Discussion
• Reasons for results on medieval coins :– Experts indicate that classifications are based
on small details– I.e. expert classifications are based on a large
number of small (undocumented) rules– Experts (consciously or not) take extrinsic
information into account (such as finding location)
Discussion
• How should a system for automatic classification of medieval coins work?– Text is highly discriminating, however, cannot
be read by state-of-the-art in character recognition
– We foresee the development of a semi-automatic adaptive system in which the expert indicates distinguishing features of the coin
– Over time, the system should be able to learn the undocumented rules
Conclusions
• Contour and texture features perform well in the classification of modern coins
• The results of these features on early-medieval coins are disappointing
• There are various reasons why the features fail in the classification of medieval coins
• Future work: semi-automatic approach