Jewelry Excerption – Aiding Jewelry selection using facial and computational method

8/7/2019 Jewelry Excerption Aiding Jewelry selection using facial and computational method

1/7

Copyright 2011 Page 1 of 7

Jewelry Excerption Aiding Jewelry selection using facialand computational method

Arvind Rajan

Bank of America Relationship

Plot No.G1, SIPCOT Information Technology Park,

Navalur Post, Chennai - 603 103, Tamil Nadu, India91 9940194895

[email protected]

AbstractJewellery has long been for long an outburst of creativeexpression. Jewellery plays a major role in everyones life. Be itrich or the poor, everyone one wants to possess Jewellery. One of

the major challenges in todays jewellery market is to bring the

right designs to market that sell. Every year hundreds, if notthousands of different designs are made, and bought to market.

Some are converted into samples, crafted and put on shop fordisplay. Often a customer has to manually select a piece ofjewellery from showcase, asks the storekeeper to remove it, wearit and see if it looks ok or good. Most of the times, the customertries couple of jewellery, picks one (or does not) and leaves. The

entire activity takes an hour, yet there are uncertainties ifcustomer will buy or if his needs are satisfied. Jewellery Selectionis the future of Jewellery for Retail stores, it analyses a milliondesigns, and recommends the ones that look best on a customer.

Its just as simple as going to a computer, inputting your photoand couple of details and getting access to hundreds of designs, allshowing how one would look had he/she worn the Jewellery.

The other major challenge Jewellery Selection addresses is theneed to stock a store with thousands of designs. This cost a lot ofspace, maintenance cost. Jewellery Selection could assist acustomer decide the best jewellery in minutes using predefined

metrics, thereby reducing cost and time.

As a part of this whitepaper, automation of jewellery selection

process has been explored. The process calculates the bestJewellery based on Facial and Height features.

Figure 1. : Key Aspects covered in this whitepaper

Whilst numerous research have been done in the areas of Face[2],Ear[3] and Height detection[4], this paper does serve as a first cuttowards implementation of those ideas for Automated Selection of

Jewellery. The paper restricts itself at the moment towardsJewellery for face and neck alone. Other aspects are beyond the

scope of this paper.

KeywordsRetail Domain, Jewelry, Automated recognition,Face, Neck and Ear detection

1. IntroductionJewellery has been always a prized commodity and conventionalprocesses of selling Jewellery through retail are increasingly

becoming a costly affair. There is also a growing need forJewellery in European countries. Speed to market, yet ensuringthat they meet customer need is the critical aspects for the day.

Todays customers also like to wear different jewellery based onoccasion. A business party would demand different jewellery asopposed to a family outing. Stocking the right Jewellery at storesis a challenging ordeal.

Kiosks that allow manual selection of Jewelry are time consumingand demand a lot of effort from customer. The customer has to

pick each jewel, put it to their face and match if it suits them. Anormal customer can only try 10-15 jewels at max during a visit to

a store. This would mean he would be looking into a limited poolthereby hampering decision making.

As a part of this whitepaper, intelligent Jewelry selection through

manual and automated decision making is envisaged.

1.1 Market ComparisonAs a part of this paper, an analysis of various existing tools whichsolve this purpose was evaluated. Some of the nearest matchesthat could be found are listed below

Figure 2. : Dream Necklace Maker


2/7


Dream Necklace Maker [5]: This site requires a lot of humanintervention. This activity is currently very manual in approach.The author had to drag and drop jewelry on top of image.

Figure 3. : Virtual Fashion Pro

Then there is the virtual fashion professional which does a partialjob

As a part of this exercise, we were not able to find any site orwebsite which focuses on automating Jewelry selection andSimulation of the Jewelry on a real person.

1.2 A real life exampleThe current Jewellery process is very manual and labourintensive. A lot of effort can be reduced by introduction of an e-virtual portal. The approach can be summed up in the following

way

1. A Customer visits a Jewellery virtual portal (a websiteor a kiosk at a retail store)

2. The virtual portal asks him/her toprovide a photo (face view /Front Face + torso View)

3. The portal requests some manualinputs like type of occasion,budget and so on, to assess at ahigh level what is theexpectation of customer is

4. The portal does some pre-computation based on featuresextracted from persons photo

5. Using steps mentioned in b and c ,the portal determinesthe best jewellery for the person

6. It displays the list of jewellery to the person, with avirtual morph as how he/she would look if they werewearing the Jewellery

2. SolutionAt a high level, the following approach is being used:

2.1 Procure inputs

2.2 Apply computation

2.3 Simulate the results

Figure 4. : Key Aspects covered in this whitepaper

2.1 Inputs Sourcing InformationThe first and the foremost step is to source the relevant

information either through user input or through face and patternrecognition. User input can be obtained via a touch screen or via aportal defined in section 1.1. As a part of the Portal, the followinginputs can be obtained:

Figure 5. : Key Inputs to be provided

Whilst the above data represents a sample representation, furthermetrics can be added to make this list more comprehensive.

2.1.1 Manual Process: Procuring Inputs from userAs a part of Jewelry selection, several parameters are based onuser taste and vary from user to user. For instance, on one side,there are the cost based users and on the other side, there are theusers who look at durability and are willing to pay a higher cost.

These are areas which are currently not automatable and requiremanual input.

Sr.No.

Parameter Mode Attribute1

Attribute2

Attribute

3

1 Occasion

to wear

jewelry

Manual Business Party Outing

2 Cost of

Jewelry

Manual High Medium Low

3 Durability Manual Long

lasting (3

years and

Medium

lasting (1-3Short term(less than a


3/7


beyond) years) year)

4 Frequency

of wearing

Manual Regular

use

Moderate

useNegligibleuse

Table 1: Key Aspects covered in this whitepaper [6]

2.1.2 Automated Process: Facial FeaturesExtractionSr.

No.

Parameter Mode Attribute

1

Attribute

2Attribute

3

1 Face

Shape

Automated Small

Elliptical

Large

Elliptical

2 Height

based

selection

Automated Tall Medium Short

Table 2: Automated process - Parameters

The automated processes assist in extracting human facial features

and processing them. This saves user time in terms of inputtinghis details. The same information is also stored in the systemthereby helping the end user from inputting it again and again.Asa backup, manual way to feed input is also provided.

2.1.3 Face Detection and ExtractionIn recent past, there has been a tremendous explosion of softwareand hardware industry, giving birth to newer avenues such as face

recognition.

Face recognition is considered far more complex as compared to

recognizing other objects. The human face as opposed to otherobjects is dynamic, built out of different forms and colors. One ofthe key aspects about facial recognition is the need to isolate theface from background.

Some of the key algorithm revolves around:

Viola-Jones object detection framework [7]

Schneiderman & Kanade (2000) [8]

Rowley, Baluja & Kanade: Neural Network-based Face

Detection] (1998) [9]

Whilst there are multiple algorithms to perform face detection,each one possesses its own strength and weakness. Classical

techniques are built using some use contours, flesh tones andother are even more complex involving neural networks,templates, or filters. Computational expense of this algorithm is amajor challenge. Images consist of a collection of color and light

intensity values. Analyzing these pixels often cost a lot of timeand is also difficult due to variations in human face shape.

Figure6: Preferred Algorithm

Why we chose Viola and Joness algorithm?

95% accuratecompared to

otherstraditionalalgorithm

15% fasterthan other

algorithm

Integral image improvescomputation speed

Table 3: Voila Jones Benefits

As a part of this whitepaper, we prefer using Viola and Jonessalgorithm, called Haar Classifiers [10] , to rapidly detect any

object, including human faces, using AdaBoost classifier cascadesthat are based on Haar-like features and not pixels.

Face detection can be performed using the Paul-Viola andMicheal-Jones, to extract face region from experiment images.The Viola-Joness detector performs AdaBoot in each node ofcascades to get high detecting rates with low rejecting ratio. The

algorithm classes miss and hits from a scale of -1 to 1.

Figure 7 : Sample Candidate Face Extraction

2.1.4 Eccentricity of a FaceA robust algorithm to detect elliptical shapes in images based onthe Randomized Hough Transform (RHT) [11] is used in thispaper. The algorithm performs detection of regions whose exactshape is not known, but for which there is enough prior

knowledge on to build an eclipse. This method has many desirablefeatures, as it is able to recognize partial and slightly deformedellipses, and it may find several occurrences of ellipses during thesame processing step. Thus, the task of ellipses detection consists

in three major steps:

1) Find the ellipses present in the image, limited to 2.

2) For each ellipse found, determine the parameters that describeit, namely the coordinates of the center of the ellipse, the majorand minor axis and the orientation.

3) Classify each pixel in the image as belonging to one of theellipses found or as a pixel that does not belong to any ellipse(background pixel).

To improve the performance of this algorithm as precise aspossible and to facilitate the understanding of the framework builtto detect faces, it is necessary to define what an ellipse in this

context is. An ellipse ci is expressed by a parametric function f(a,d) with a = [1,2] containing two ellipse parameters and d=d(x,y) being the coordinates of a contour point of the ellipse. A-band of ci is defined as a subset of the space, around the ellipse

ci with width . So, the elliptical shape mentioned before isdefined as an elliptical band around the ellipse subject to f (a,d) =0. Eccentricity is a number that describe the degree of roundness

of the ellipse. For any ellipse, 0 e 1. The smaller theEccentricity, the rounder the ellipse. If e == 0, it is a circle and F1,F2 are coincident. If e == 1, then it's a line segment, with foci atthe two end points.

Using the RHT, the ellipse detection algorithm works in theparameter domain. For each triplet of pixels in the image, a set of


4/7


5 ellipse parameters is computed (centre of the ellipse, the majorand minor axis and the orientation) that vote in the parameterspace

2.1.5 Getting the height of CandidateUsing a single image it has been demonstrated that one can

estimate body height. The human face especially the facialvertical distribution possesses important information which

correlates with the height. The vertical proportions keep uprelative constancy during the human growth. Only a few facial

features such as the eyes, the lip and the chin are necessary toextract. The metric stature is estimated according to the statisticalmeasurement sets and the facial vertical golden proportion. Theestimated stature is tested with some individuals with only a

single facial image. The performance of the proposed method iscompared with some similar methods, which shows the proposalperforms better. The experimental results highlight that thedeveloped method estimates stature with high accuracy.

Figure 8: Test Candidate, Golden Rule applied for computation

The human face especially the facial vertical proportion owns

some important information which correlates with the stature .The facial vertical proportions include the golden proportion andthe facial thirds method. The facial golden proportion is ap-proximately the ratio of 1.618 to 1. It states that the human facemay be divided into a golden proportion distribution by drawinghorizontal lines through the forehead hairline, the nose, and the

chin, or through the eyes, the lip, and the chin.

Figure 9: Golden Ratio

The facial thirds method states that the face may be divided intoroughly equal a third by drawing horizontal lines through the

forehead hairline, the eyebrows, the base of the nose, and the edgeof the chin as shown in Figure above.

Figure 10: Golden Rule applied for computation of Height

The camera model used is a central projection. An image ofhuman body is equal to approximately to 8 times the size of a

human face. This helps us to calculate the height of human

3. Applying MetricsThe next task in the cycle involves applying metrics. This processinvolves procuring the jewelry designs from Jewelry database, earand neck position from candidate via recognition. Jewelry metricsare applied on the Jewelry design and a handful of relevantdesigns are picked

3.1 Face Shape MetricsThe face shape metrics is based on section 2.1.4. Based on the

shape, the system can calculate the values for type of Jewelrybased on following metrics.

Face Shape Necklaces Earrings

Eccentricity 0.10

to 0.20

Any shape necklace

will work A chokerlooks just as good

as opera-lengthpearls or anynecklace that comes

to a "V."

Round shapes,button or hoopearrings look well

on the oval face,but triangular

shapes areespeciallyflattering.

Dangling earringslook well, if theyare not too long.

Earrings that moveup the ear or havewings that sweep

up minimize a too-thin face or longnose.

Eccentricity 0.0 to

0.10

To give adimension of

length, look forlong necklaces, 28"to 32".

Squares, oblongs,rectangles work

well as dodangling andangular designs.Elongated styles

also go wellbecause they draw


5/7


Face Shape Necklaces Earrings

attention downinstead of around.

Eccentricity 0.30and above

A high chokerreduces the lengthof the face,

particularly if theneck is too long. Inaddition, a 16" or18" necklace that

ends in a "U"flatters this face.

Eccentricity 0.20

to 0.30

A choker necklace

is a favorite stylebecause it softensand diminishes the

sharp angle of thechin.

Look for earrings

that are wider atthe bottom thanthe top. Dangling

earrings that forma triangle areespeciallyflattering.

Table 4: Jewelry Metrics based on Face Shape [12]

3.2 Height MetricsThe second key aspect is the height of a person. This can beobtained from the golden ration mentioned in section 2.1.5. Basedon the same, the following metrics can be applied :

Face

ShapeNecklaces Bracelets Earrings

Petite

(Under

5'4")

Petite women bestwear collar-lengthor longer necklaces.Styles with "V"

shapes and onesthat fall below thebreast but above the

waist elongate thefigure.

Banglesflatter petitewomen.

Severalnarrow onesare moreflattering

than onewide onebecause they

are more inproportion

to thepetite'soverall size.

Round shapes,button or hoopearrings look

well on theoval face, buttriangularshapes are

especiallyflattering.Dangling

earrings lookwell, if they

are not toolong. Earringsthat move up

the ear or havewings thatsweep upminimize atoo-thin face

or long nose.Medium(5'4"

5.7)

To give adimension oflength, look for

long necklaces, 28"to 32".

Widebracelets aremore in

proportionto theaverage-

heightwoman thanvery narrow

Squares,oblongs,rectangles

work well asdo danglingand angular

designs.Elongatedstyles also gowell because

Face

ShapeNecklaces Bracelets Earrings

ones. they drawattentiondown instead

of around.

Tall

(>5.7)

A high choker

reduces the lengthof the face,particularly if theneck is too long. In

addition, a 16" or18" necklace thatends in a "U"flatters this face.

The tall

woman hasa wide rangeof braceletchoices as

long as shestays awayfrom too-delicatepieces. The

full-figuredtall womanshould wearseveral thin

braceletstogether togive a morebalanced

feeling, or acouple ofwidebracelets.

Concentrate

on styles thatsweepupward,pulling the

viewer's eyeup. Full-figured petiteslookespecially

well in sharpgeometricshapes.

Table 5: Jewelry Metrics based on Face Shape [13]

3.3 Applying MetricsEach Jewel in the database is stored in the following format.

Chocker

3000

High

Less

5.6

0.3

Figure 11: Sample Jewelry


6/7


Figure: System returns output after Applying metrics for Jewelryselection

The following metrics are applied on the jewelry design

Parameter Mode Attribute

1

Attribute

2

Attribute

3

Occasion

to wearjewelry

Manual Business Party Outing

Cost ofJewelry

Manual High Medium Low

Durability Manual Longlasting (3years andbeyond)

Mediumlasting (1-3 years)

Short term(less thana year)

Frequencyof wearing

Manual Regularuse

Moderateuse

Negligibleuse

Height

basedselection

Automated

Section

3.1

Tall Medium Short

Face Shape Automated

Section

3.2

SmallElliptical

LargeElliptical

Table 6: Metrics for calculation

4. SimulationsTo perform simulation, the following tasks needs to be performed:

Figure 12: Representation of Simulation

4.1 Ear and Neck Position DetectionTo the output of the skin filter, a threshold is applied. The skinfilter is not perfect and so any non-black pixel in the skin filter

output might not always represent a skin. So in order to obtain thenose point, the first non-black pixel is noted as a skin point only ifit is surrounded by non-black pixels as well. In this way the firstnon-black pixel in each column is noted. Once a vector of the

pixel locations is available, the minimum position is noted. Again,the surrounding pixels are examined in order to ensure that theidentified point is a skin pixel. Figure 6 shows the figure with thenose point identified.

Extrapolating on a rectangular area of 10 X 10 gives the positionof ear.

Figure 13: Calculating Nose Point Region (Red). Identifying earand Neck region (Green)

4.2 Implementation using OpenCV

A partial implementation was carried out using Open CV

[15],Open CV Python and Xawt package. The scope of thisproject was to detect facial features and identify neck and ears. Asa part of this implementation, the face was detected; extrapolationtowards ears and neck region has also been done. Once this is

done, the final image is saved to a folder. The following are theresults:

Sample Conditions Detected

Sample 1 Normal light , fair distance(1-2 feet)

True

Sample 2 Extreme light , fair distance False

Sample 3 Normal light, close distance(0.3-0.5 feet)

True

Sample 4 Normal light, Far distance (4-10 feet)

Partial couldnot detect whensample moved far

4.3 Rendering Final ImageBy applying the Noise Point Detection method [14], An

approximate Jewelry placement can be done across the bounding

box. The current implementation is yet to be done and at themoment stands as a separate task.The same has been done as a

part of the figure provided

Figure 14: Candidate wearing Jewelry Avatar

5. ConclusionIn this paper we outlined the procedure for Automated Jewelryselection and matching. The resource we have used constitutes areusable source of information for many Face/Feature recognition

fields. The solution allows for completely unsupervised matching.However, over a larger test data, the system would require somesupervision. Not last but least the paper acts as a foundationtowards Jewelry Selection.

6. AcknowledgmentsThe author wishes to express his sincere thanks and gratitude to

the following people for assistance in proof reading the content:


7/7


a) Smt. Lalitha Sundararajan, M/o Arvind Sundararajan7. References[1] Neural network-based face detection - HA Rowley, S Baluja-

Pattern Analysis 2002 - ieeexplore.ieee.org

[2] Face detection system based on feature-based chrominance colourinformation, YH Chan- Computer Graphics, 2004 -

ieeexplore.ieee.org

[3] Human ear detection from side face range images, Chen- PatternRecognition, 2004 - computer.org[4] Human face detection in cluttered color images using skin color and

edge information-K Sandeep - URL citeseer

[5] Dream Necklace Maker, www.girlsgogames.com[6] Website,http://ezinearticles.com/?How-to-Wear-the-Right-Piece-of-

Jewelry-For-the-Right-Occasion&id=3304291

[7] [PDF] Rapid object detection using a boosted cascade of simplefeatures, psu.edu, Viola - IEEE Computer Society Conference onComputer , 2001 - Citeseer

[8] A statistical method for 3D object detection applied to faces andcars,psu.edu, H Schneiderman - cvpr, 2000 - computer.org

[9] Neural network-based face detection. psu.edu. HA Rowley, SBaluja - Pattern Analysis ,2002 - ieeexplore.ieee.org

[10] Facial feature detection using Haar classifiers,from psu.eduPIWilson - Journal of Computing Sciences in Colleges, 2006 -

portal.acm.org

[11] Randomized hough transform (rht),P Kultanen, L Xu - PatternRecognition, 1990. , 2002 - ieeexplore.ieee.org

[12] Jewelryselection,http://www.jsbeads.com/Articles/Designing_Jewelry_BS_FS/faceshape.asp

[13] Jewelry selection,www.horlacherjewelers.com/jajewelry.html[14] [BOOK] Detection and tracking of point features[15] Open CV , http://opencv.willowgarage.com/wiki/
http://www.girlsgogames.com/http://www.girlsgogames.com/http://www.girlsgogames.com/http://www.jsbeads.com/Articles/Designing_Jewelry_BS_FS/faceshape.asphttp://www.jsbeads.com/Articles/Designing_Jewelry_BS_FS/faceshape.asphttp://www.horlacherjewelers.com/jajewelry.htmlhttp://www.horlacherjewelers.com/jajewelry.htmlhttp://www.horlacherjewelers.com/jajewelry.htmlhttp://www.horlacherjewelers.com/jajewelry.htmlhttp://www.jsbeads.com/Articles/Designing_Jewelry_BS_FS/faceshape.asphttp://www.jsbeads.com/Articles/Designing_Jewelry_BS_FS/faceshape.asphttp://www.girlsgogames.com/

Documents

Jewelry Excerption – Aiding Jewelry selection using facial and computational method