A Practical System for Modelling Body Shapes from Single View Measurements

Yu Chen1, Duncan Robertson2, Roberto Cipolla1

Department of Engineering, University of Cambridge1

Me_tail Inc.2

Motivations and Tasks

Our Goal: – A reliable and

practical 3D body modelling system.

Requirements:– Good accuracy; – Quick;– Easy to manipulate for

non-expert users (a bit of interactions)

Different Means of Input Tape measurements

– E.g. [Magnenat-Thalmann et al., 03, 07], etc.

– Sparse representation;– Relatively invariant to pose

changes;– Anthropometrically meaningful;– Many users don’t know their

body measurements like chest, waist, and hips;

– Obtaining accurate tape measurements is not very easy.

Different Means of Input Full Silhouettes

– E.g. [Balan et al. 08], [Guan et al. 09], etc.

– Dense representation; – Informative; better local

constraints;– Hard to decouple pose

variations from body shape variations;

– Hard to extract silhouettes from images with arbitrary background;

A Solution

Silhouettes(dense)

Tape Measurements(sparse) Our Solution:

Image Measurements

The Interface How it work?

1. Upload a photo of the user standing against a doorway;

2. Mark up the 4 corners of the doorway;

3. Rectify the photo automatically;

4. Measure the several image measurements on the rectified image.

5. Provide body dimensions: height + weight;

6. Obtain the 3D body shape.

The Interface How it work?

1. Upload a photo of the user standing against a doorway;

2. Mark up the 4 corners of the doorway;

3. Rectify the photo automatically;

4. Measure the several image measurements on the rectified image;

5. Provide body dimensions: height + weight;

6. Obtain the 3D body shape.

The Interface How it work?

1. Upload a photo of the user standing against a doorway;

2. Mark up the 4 corners of the doorway;

3. Rectify the photo automatically;

4. Measure the several image measurements on the rectified image;

5. Provide body dimensions height + weight;

6. Obtain the 3D body shape.

The Interface How it work?

1. Upload a photo of the user standing against a doorway;

2. Mark up the 4 corners of the doorway;

3. Rectify the photo automatically;

4. Measure the several image measurements on the rectified image;

5. Provide body dimensions: height + weight;

6. Obtain the 3D body shape.

The Interface How it work?

1. Upload a photo of the user standing against a doorway;

2. Mark up the 4 corners of the doorway;

3. Rectify the photo automatically;

4. Measure the several image measurements on the rectified image;

5. Provide body dimensions height + weight;

6. Obtain the 3D body shape.

The Interface How it work?

1. Upload a photo of the user standing against a doorway;

2. Mark up the 4 corners of the doorway;

3. Rectify the photo automatically;

4. Measure the several image measurements on the rectified image;

5. Provide body dimensions height + weight;

6. Obtain the 3D body shape.

Image Rectification Automatic photo rectification. Use vanishing point geometry. Compute the homography from

the quadrilateral to the rectangle.

Homography H is dependent on the focal length f. Regular case:

– Both vanishing points are at finite positions.– [Cipolla et al. 99]:

Near-degenerated case: – One of the vanishing points is at infinity;– Result in an ambiguity in the aspect ratio α.

Image Rectification

Aspect ratio distortion need to be corrected!

V2 at infinity V1 at infinity

or

Selection of Image Measurements Criteria:

– Well defined; unambiguous to users;

– Good correlation with the corresponding 3D measurements;

– Good representative power. Selected measurement set:

– Height;– Crotch height;– Under bust;– Waist;– Hips;

Probabilistic Body Shape Estimation

3D training data:– Female CAESAR Data;

Registration: – Victoria 4 morphable model;– 10 different morphs are defined.

Regressor: Gaussian Process (GP)– Input: body dimensions (height

& weight) + normalised measurements;

– Output: morph weights.

Prediction and Aspect Ratio Correction (ARC)

Given a learned GP regressor G. Input:

– Actual body dimensions zV

– Image measurements zI

Horizontal image measurements zI,h;Vertical image measurements zI,v;

– Complete set of testing measurements:z = {zV, α zI,h, zI,v}

Output: – Morph weights y; aspect ratio factor α.

Prediction and Aspect Ratio Correction (ARC)

The shape prior gives clues of the proper aspect ratio.

GP Joint posterior.

MAP estimate of the y and α

Optimisation:– y = µy;– α : fixed point equations.

Prediction and Aspect Ratio Correction (ARC)

ARC, How effective is it? A simulation of distortion and correction

– Stretch a frontal view image into different aspect ratio α from 0.9 to 1.1.

ARC can neutralise more than 60% of distortion.

Experiments and Testing Data

Render >1000 synthetic doorway images using CAESAR laser scans– Frontal-view set;– Un-rectified set;

Cross validation. Input combinations

– H + W: height and weight only;– H + W + T: height, weight, and tape

measurements;– H + W + I: height, weight, and image

measurements.

Average error over 15 body measurements defined in CAESAR dataset.

Image measurements are good substitutions when tape measurements are not available.

Test on the Frontal-view Image Set

Test on the un-rectified image set

The efficacy of aspect ratio correction

Test on Real PhotosUser study by inviting volunteers

– Take photos at home; uncontrolled environment;– Tape measurements are collected for evaluations.

Performance– Average time for annotation (doorway + 5 image

measurements): 1.5 to 3 min;– Computation time: <1s for 2.4GHz CPU;– Accuracy:

Body part Chest Waist Hips Inner leg length

Error(cm) 1.52 ± 1.36 1.88 ± 1.06 3.10 ± 1.86 0.79 ± 0.90

Qualitative Results of Real Users

Qualitative Results of Real Users

Qualitative Results of Real Users

Take-home Messages Image measurements can define body shapes

well;

Image rectification may lead to an aspect ratio distortion;

The 3D shape prior gives important clues to correct the aspect ratio distortion;

A novel working system for quick 3D body shape modelling and garment fitting.

The End

Questions?