Working with Fashion Models

Eddie Bell - @ejlbell

Lyst

3

Fashion

general purpose visual fashion representation

Aim

Build a general purpose visual fashion model

Deep Learning

Composition and representation

Composition

Representation

AI must fundamentally understand the

world around us and this can only be

achieved if it can learn to identify and

disentangle the underlying explanatory

factors hidden in the observed milieu of

low-level sensory data.

2014 - Representation Learning: A Review and New Perspectives. - Bengio et al.

Male

FemaleCat

Dog

Male

FemaleCat

Dog

Male

FemaleCat

Dog

The model

Male

Jacket

Beige / Brown

Iridescent Zip

Training

20 epochs

5 hours per epoch

16 million parameters

4 million images

task accuracyall 0.80colour 0.73gender 0.91type 0.97category 0.78subcategory 0.61

task prediction confcolour blue 0.28gender men 0.93type shoe 0.88category sneaker 0.38subcategory low-top 0.21

Internals

What can we use this for?

Multi-modal embeddings

This elegant long black coat is perfect for pydata

Positive Image Negative Image

Anchor word

Textual context

Visual context

w1 w2 lexical semantic

opening openings 1 0.57

flared flare 1 0.87

chic chino 2 0.18

loops belt loops 5 0.99

embroidered logo

button down

curved hem

front button fastening

button placket

point collar

breathable

classic

slim fit

Composition and Representation

Thanks

@ejlbell