Shape measurement system of foot sole surface fromatbed scanner image

Sandy Martedi and Hideo SaitoDept. of Information and Computer Science

Keio University, Japan{sandy,saito}@hvrl.ics.keio.ac.jp

Myriam Servie`resComputer Science and Math Department

Ecole Centrale de Nantes, Francemyriam.servieres@ec-nantes.fr

Abstract

We present a system to measure 3D shape of solesurface of human foot using atbed scanner. Our goalis to build a low cost system for reconstructing solesurface of human foot using atbed scanner for mak-ing tailored shoes. There are two main phases in oursystem: a photometric parameter estimation and a re-construction phase. The photometric parameter esti-mation calculates the position of the light source in thescanner. The reconstruction phase uses iterative algo-rithm to calculate normal vector and the depth informa-tion. First, we implement photometric parameter esti-mation by scanning some white paper in several slantand rotation angles. We conducted some experimentsto obtain the best slant and rotation angle combinationfor calculating light source position of the scanner. Wealso propose a new method for estimating light sourceposition of the scanner using foot model. Next, we con-duct reconstruction phase by scanning users foot. Wethen apply median lter with 55 mask sizes to removethe noise in scanned image. By using the calculatedlight source position from the previous step and pixel in-tensity of scanned image, depth and normal vector arecalculated iteratively. We acquire more accurate lightsource position by comparing albedo ratio and nallywe acquire the 3D shape which average error comparedwith ground truth data is up to 0.97mm.

1 Introduction

Foot shape reconstruction is required for capturingcustomers foot shape at shoes shops. This should below-cost for making all shoes shops can have such sys-tem.

Laser scanning system [1] is a one practical system,but it is expensive, and even some customers are afraidof laser illumination. Kouchi et al.[1] and Lee at al.[2]have applied Free Form Deformation (FFD) to the ba-sic shape acquired from database. By using deforma-tion they produce shoes model as captured from cus-tomers foot. Such system is good for the average shapeof the foot. However, we still need a suitable shapefor each of users foot without depending on modeldatabase.

Amstutz et al.[3] and Lee et al.[2] have proposedsystem using multiple cameras and database to createaverage model and to produce foot model by calcu-lating major foot parameters. Particularly, sole shapeis very important for foot reconstruction, while thesesystems can not reconstruct it.

Flatbed scanner is normally used for various purpose

in recent years such as digitization of documents andphotography. Optical Character Recognition (OCR) asone of technology using atbed scanner made it is easierto nd digital content of many books and literatures.Now we will use atbed scanner for digitizing part ofhuman body such as reconstruction of sole shape. Par-ticularly we can come up with an application that canbe used in general purpose scanner. It can easily bedistributed into every shoes shop.

Takahashi et al.[4] have proposed a measurement ofhand 2D shape using scanner image. This system mea-sures shape of human hand using silhouette informa-tion and database to create personal information.

Reconstruction system have been proposed to re-construct unfolded book shape using image scanner [5].Unfolded book has specic shape which can be modeledas polynomial surface. Scanner itself has light sourceswhich can be categorized as proximal light (where lightis located close to the object surface). Using scan-ner properties and approximation based on polynomialmodel, their method can also recover the shape of booksurface. However, this system only covers shape of ob-ject which can be modeled as polynomial surface.

This paper proposes a new system to reconstruct 3Dshape of foot sole surface using atbed scanner basedon implementation of 3D shape reconstruction usingthree light sources in image scanner[6]. Human solesurface skin is assumed to be uniform in albedo andlambertian surface which ideally fulll this method re-quirement.

The dierences of our system with the other sys-tem are our proposed method only takes into accountscanner parameters and intensity information of solesurface of human foot. Moreover, atbed scanner useslamps as light source for scanning. Hence it assures theusers safety of its eect.

2 Proposed Method

We implement a reconstruction method usingatbed scanner [6] to reconstruct sole surface of thefoot. Foot sole skin has the same color over the surface(constant albedo) and it has no reection over the sur-face (lambertian surface). It has specic patterns (fric-tion ridges) which make reconstruction of sole shapeneeds more eort. We assumed the light from the en-vironment has no eect on scanning process and no in-ter reection from the sole surface nor the environmentare taken into account. Our method is divided into twophases: photometric parameter estimation and recon-struction (Figure 1).

To evaluate our system, we calculate the dierence

MVA2009 IAPR Conference on Machine Vision Applications, May 20-22, 2009, Yokohama, JAPAN10-1

338

Figure 1: Reconstruction Flow

between measured 3D points with ground truth modelle.

2.1 Photometric Parameter Estimation

Scanner has several xed properties for reconstruc-tion such as camera position, gain, bias, and light po-sition. Photometric estimation is done to calculate thelight position in the scanner. Using a method in [6],we conducted several experiments to estimate scannerslight source position. First we calculate many scannerparameters by scanning white paper in some slant an-gles () and rotation angles () position (Figure 2).

Figure 2: Position of white paper (slope)

We selected many intensity samples of the scannedwhite paper. Let pi(xi, yi) be a point in the slope ofthe scanned image, di is the distance between pi andline l, we can compute height zi and normal vector(nxi,nyi,nzi) using the relation in the Eq. 1.

zi = di tan ,nxi = sin sin ,nyi = cos sin ,nzi = cos (1)

We can model foot sole skin as lambertian surfaceusing Eq. 2. Here we model the intensity for colorelement red (Pr) as follows:

Pr(xi, yi) = ar r Isr(xi, yj)cos(r(xi, yj))+r (2)

Figure 3: Clicking points over white papers slope

where ar and r denote the gain and the bias of thephoto-electric transformation in the image scanner re-spectively. r is the albedo on the surface for the redlight source. Intensity of green element (Pg) and blueelement (Pb) are computed in the same way.

By using the light source model we model the illu-minant intensity of the light source of each componentcolor as:

Isr(xi, yi) =r

(d2yr + (z(xi, yi) dzr)2+ Ier (3)

where z(xi, yi) is the height from scanning plane tothe object surface, (dyr,dzr) is the position of the lightsource of scanner (lamp) relative to edge of scanningplane.

r(xi, yi) is the angle between normal vector(nxi,nyi,nzi) and the direction from the surface to thered light source declared as:

cos(r(xi, yj)) =dyr ny(xi, yj) nz(xi, yj) (z(xi, yj) dzr)

d2yr + (z(xi, yj) dzr)2(4)

The red intensity of selected points from the slopeimage is inputted to the Eq. 2, Eq. 3, and Eq. 4 tocalculate dzr, dyr ,r,ar, r, and r. The equationis solved by using linear least squares. This process isrepeated for the green and blue component to calculatedzg, dyg, g, ag, g, g, dzb, dyb, ab, b, b, andb. We then use optimization algorithm to get optimalparameters.

However, photometric parameter estimation usingwhite paper depends on slant and rotation angles.Here, we propose new method by replacing white pa-per with white foot model and its known depth andnormal vectors. By using this model we assumed wewill acquire more accurate approximation of scannerparameters.

First, we scan white foot model using the same scan-ner. We then adjust the coordinate system of 3D modeland the scanned image. Then we project every ver-tex in the sole part of associated model le into thescanned image. Here we acquired the intensity of ev-ery vertex (Figure 4). By inputting the pixel intensity,normal vectors and depth from foot model into Eq. 4we can calculate the scanner parameters using the samemethod as white paper slope.

2.2 Shape Reconstruction

To reconstruct sole part of the foot, rst we scannedusers foot. Then the scanned image is inputted to thesystem. Sole skin has a ridge pattern which will be anoise in the shape reconstruction system. This noisecan be removed by applying median lter (5x5 masks)to the input image.

Next step is determining the albedo of the sole sur-face. We can not compute the albedo of the sole

339

Figure 4: Photometric Parameter Estimation usingfoot model slope

directly, therefore we compute the albedo ratio withwhite surface object albedo in the photometric parame-ter estimation process (Eq. 5). To calculate the albedoratio (), we select a pixel (Pr0) in the scanned imagewhich seems touches the scanning plane. Then we cal-culate the albedo ratio using Eq. 6.

r =rwr

(5)

r =P r0 r

Isr0 cos(r0) (6)

Figure 5: Reconstruction phase

Pr(xi, yi) = r r r Isr(xi, yj) cos(r(xi, yj))+r(7)

We then input the intensity information of red, dzr,dyr, r, ar, r, r, and r into Eq. 7. Here we iter-atively calculate the ny(xi, yj), nz(xi, yj) and z(xi, yj)until the height converged (Figure 5). Since there arethree intensity components (red, green and blue), theproduced height is average of zr(xi, yj), zg(xi, yj), andzb(xi, yj). We iterate this process for every pixel in theimage.

Table 1: Albedo Ratio

slant and rotation angle r g

b

10-0 0.89 0.76 0.610-0,30-30 1.25 1.21 0.92

30-30 1.4 1.24 0.8745-30,30-20 -6569.93 1.48 2151.64

40-60 -12.74 1.44 0.7430-20,40-60,10-60 1.21 1.21 0.69

3 Experiments and Result

We use Epson GT-8700F atbed scanner for exper-iments. The image resolution used for the input imageis 99257 pixels. The scanning process is done withoutinterference of light from environment.

We selected one pixel (red = 175, green = 149, blue= 106) which seems to touch scanning plane. We an-alyzed between the calculated albedo ratio (Table 1)and the reconstruction result.

Through set of experiments we obtained photomet-ric parameters such as dz, dy, , and . We choosealbedo ratio () which lays close to range from 0 to 1which means the color intensity is around 0 to 255 (seeTable 1) for reconstruction phase.

We also did experiments using foot model for esti-mating photometric parameters. We choose some pixelintensities in the image and compare the calculatedalbedo ratio using white paper in 10-0 angles, footmodel, and the real albedo ratio as shown in Figure6(a). The benet of using foot model are this methoddoesnt depend on the slant and rotation angles and thealbedo ratio is more accurate. The accuracy comparedwith real albedo ratio is shown in Figure 6(b).

(a) Albedo Ratio Average

(b) Albedo Ratio Error

Figure 6: Albedo Ratio

In the reconstruction phase, we scanned users foot

340

Table 2: EvaluationSam- Median average averagepling Filter distance distance

using usingwhite foot

paper (mm) model(mm)1 1 1.911 0.9732 1 1.949 1.0143 1 1.995 1.0584 1 2.046 1.1285 1 2.135 1.1701 3 1.916 0.9682 3 1.948 1.007

(Figure 7(a)). We then applied sampling and medianlter with 55 mask size (Figure 7(b)) to remove noisedue to friction ridges. We calculate the depth and nor-mal vector to produce 3D shape. Sampling from onepixel to ve pixels is depicted in the Figure 8 and theresult of using median lter as shown in Figure 9.

(a) ScannedImage

(b) Medianlter isapplied

Figure 7: Input Images

4 Evaluation

We use a foot model and its ground truth le tomeasure reconstruction accuracy. This foot model wasscanned and we reconstructed 3D shape by using oursystem. We reconstruct 3D shape using two ways ofphotometric parameter estimation: using white papermethod and foot model method.

For comparing two meshes, we implemented regis-tration onto 3D shape from our system and groundtruth le. We created many sole shapes under dier-ent conditions. We set sampling from one pixel to vepixels. And the median lter was applied one to vetimes for preprocessing.

The average distance is calculated between everypoint in both 3D shapes using Attribute DeviationMetric method [7]. The minimum mean error isreached in conditions: no sampling, the median lteris applied three times (Table 2).

5 Conclusion

We have presented a system for reconstructing 3Dshape of sole surface of human foot using atbed scan-ner. Even using a atbed scanner we can estimate 3D

Figure 8: Reconstruction with sampling (1 to 5 pixels)

Figure 9: Reconstruction with median lter

shape of the sole surface of the human foot with a min-imal error up to 0.97mm. Hence, we can make atbedscanner as a simple and low cost 3D shape reconstruc-tion device for human body. There are some issues ofsole shape reconstruction using scanner. Scanning po-sition of user will lead to a problem since the intensityof the sole skin changes due to the pressure to the scan-ning plane. Moreover, some pattern of the foot due toinjury, friction, or damage on the sole skin will makereconstruction process more dicult. In the future, theergonomic aspect of the human should be consideredfor reconstructing sole shape to make application forshoes designers.

References

[1] M. Kouchi and M. Mochimaru. Development of a lowcost foot-scanner for a custom shoe making system. In5th ISB Footwear Biomechanics, pages 5859, 2001.

[2] Hyunglae Lee, Kunwoo Lee, and Taekyu Choi. Devel-opment of a low cost foot-scanner for a custom shoe tai-loring system. In 7th Symposium on Footwear Biome-chanics, 2005.

[3] Edmee Amstutz, Tomoaki Teshima, Makoto Kimura,Masaaki Mochimaru, and Hideo Saito. PCA-based 3Dshape reconstruction of human foot using multiple view-point cameras. International Journal of Automationand Computing, 5(3):217225, 2008.

[4] Takuya Takahashi, Yoshimitsu Aoki, Masaaki Mochi-maru, and Makiko Kouchi. Image measurement of handdimensions based on anatomical structure and creases.In IWAIT 2008, 2008.

[5] T. Wada, Ukida H., and T. Matsuyama. Shape fromshading with interreections under proximal lightsource-3D shape reconstruction of unfolded book surface froma scannerimage. In Computer Vision, 1995. Proceed-ings., Fifth International Conference on, pages 6671,1995.

[6] Ukida H and Konishi K. Machine vision applications.3D shape reconstruction using three light sources in im-age scanner. IEICE Trans Inf Syst (Inst Electron InfCommun Eng), E84-D(12):17131721, 2001.

[7] Michael Roy, Sebti Foufou, and Frederic Truchetet. Meshcomparison using attribute deviation metric. Interna-tional Journal of Image and Graphics (IJIG), 4(1):127140, 2004.

341