Weblab@The University of Tokyo

Procedural Modeling Using Autoencoder NetworksYumer, M. E., Asente, P., Mech, R., and Kara, L. B.Proceedings of the ACM Symposium on User Interface Software and Technology (UIST), 2015.

Presenter: Shuhei Iitsuka

Weblab@The University of Tokyo

Introduction: Problem

Procedural modeling (PM) … Geometry modeling created by a set of parameters and algorithms.

PM provides rich representation, but it’s difficult to configure a large number of parameters.→ More intuitive exploration method is required.

Example of PM: Tree modeling with L-system.L-system, Wikipedia

Screenshot from a conventional PM interface.[Figure 7 in paper]

!?!?

2

Weblab@The University of Tokyo

Introduction: Contribution

Proposed a dimensionality reduction method for PM using autoencoders.

An augmentation for continuous latent space generation (without using variational inference!) is also proposed.

Contributions:● A method to create a lower-dimensional representations of PM using autoencoders.● A PM design system interface using an explore-and-select interaction.

3

Screenshot from the PM design system interface.[Figure 3 in paper]

Weblab@The University of Tokyo

Why this paper?

● Future extraction should be a key component for generating variations of web design.

● Autoencoders are attracting great attention for future extraction, so I was searching a good application of them in the field of visual design.

● When I queried “visual design” and “autoencoder” on Google Scholar, this paper was the top hit.

4

Weblab@The University of Tokyo

Agenda

1. Related Works

2. Proposed Method

3. Evaluation and Results

4. Discussion & Conclusion

5

Weblab@The University of Tokyo

Related Works: Procedural Modeling

From when L-system is proposed, procedural modeling (PM) has many applications in digital design.

In order to migrate the burden of parameter adjustment, two approaches are proposed.

● Targeted design● Exploratory system

6

Cities modeled by procedural modeling.[Yoav 2001]

Weblab@The University of Tokyo

Related Works: Workaround for Parameter Adjustment

Targeted Design

Streamlines parameter adjustment by providing high-level design grammar.

PROS: Users are free from adjusting PM parameters directly

CONS: Less user control over the generated models.

7

Exploratory System

By providing helpful features for exploration, ease the burden to adjust the parameters (e.g. show frequently used models, devised parameter dialogue)

PROS: Users can start modeling from their liking initial model.

CONS: Users need to back to the conventional PM approach to tweak the parameters.

Design grammar for plant modeling.[Lintermann 1999]

Visualization of goodness estimated by using crowdsourcing.[Koyama 2014]

Weblab@The University of Tokyo

Related Works: Deep Neural Networks and Autoencoders

Deep neural networks model abstract characteristics of data using nonlinear transformations.

An autoencoder is a special form of a deep neural network which tries to reproduce the input layer at the output layer.

Autoencoders is superior to other dimensionality reduction methods [Hinton 2006].→ Employ autoencoders for dimensionality reduction in this research.

8

Weblab@The University of Tokyo

Proposed Method: User Experience

9

Screenshot from the proposed exploration system interface.[Figure 2 in paper]

Weblab@The University of Tokyo

Proposed Model: Sampling the Training Data

● Authors employ Categorization tree (C-tree) in order to sample data uniformly with respect to shapes rather than parameters.

○ Create quartets of shaped based on topological structure.

○ Construct the hierarchical tree assembling the quartets.

● After the initial tree is constructed, conduct subsamplingAs denoted with red rectangles in the figure.

● Iterates (1) adding new sample, (2) computing a new C-tree, (3) performing subsampling until the criteria is satisfied.

10

Example of C-tree.[Figure 4 in paper]

Example of topological quartet.[Huang 2013]

Close

Far

Weblab@The University of Tokyo

Proposed Method: Neural Network Architecture

● 5-hidden-layer symmetric autoencoder.

● In addition to PM parameters, shape features are included in the input layer in order to ensure geometric continuity.

○ A silhouette-based histogram extracted from Light Field Descriptors is utilized.

● Training is done layer-by-layer followed by a fine tuning of the entire structure.

● Weighted reconstruction error on shape features is applied to the first hidden layer.

11

Autoencoder architecture.[Figure 5 in paper]

Adding shape feature (f) for geometric continuity.[Figure 5 in paper]

Shape features

PM parameters

Example of Light Field Descriptors.[Chen 2003]

Weblab@The University of Tokyo

Proposed Method: Denoising

● Denoising is applied only to shape features in the results of comparison.

● PM parameters have no hidden process which introduce noise because they fully control the shape.On the other hand, shape features are sensitive to image-level processing details.→ Denoising only works on shape features.

12

datasets

Weblab@The University of Tokyo

Evaluation and Results: Setup

● PM rulesets○ Containers: 72 parameters○ Trees: 100 parameters

13

Example from Containers.[Figure 12 in paper]

Example from Trees.[Figure 10 in paper]

Weblab@The University of Tokyo

Evaluation and Results: Interpolation

● The latent space dimensionality is fixed to 2 or 3 in order to provide intuitive and easy-to-understand interface.

14

Weblab@The University of Tokyo 15

Weblab@The University of Tokyo

Evaluation and Results: Shape Continuity in the Reduced Dimensional Space

A latent space extracted from Container ruleset. It seems shape features are contributing to the shape continuity.

16

PM parameters + shape features PM parameters only

Weblab@The University of Tokyo

Evaluation and Results: Shape Continuity in the Reduced Dimensional Space

A linear interpolation from Trees ruleset.

17

PM params only

PM params + shape params

Weblab@The University of Tokyo

Evaluation and Results: Shape Continuity in the Reduced Dimensional Space

● Authors evaluate the continuity by introducing a cumulative shape dissimilarity measure, which sums up the feature difference between neighbors along the axes of the latent space.

● Results show that the dissimilarity decreases in both rulesets compared to the baseline (PM parameters only).

18

S: samples generated in the latent space.

N_i: neighbors of sample i along the axes of the latent space.

f_i: the feature vector of sample i.

Weblab@The University of Tokyo

Evaluation and Results: User Study

90 design non-expert users are asked to design one favorite container using system #1, then asked to replicate it using system #2.

Group A: PM params only (#1) → PM params + shape params (#2)Group B: PM params + shape params (#1) → PM params only (#2)

Users replicated the original container better when using PM params + shape params.

19

PM params only

PM params + shape params PM params only

PM params + shape params

Weblab@The University of Tokyo

Evaluation and Results: User Study

● Users using the proposed system designed and replicated the container faster and also satisfied with the quality of the replica.

● Users using the proposed system also gave positive feedback on its usability.

20

Weblab@The University of Tokyo

Discussion & Conclusion

Limitation of the proposed method

● Dimensionality reduction can result in making some shapes inaccessible any more which can be achieved by the original parameters.

○ Trade-off between representational capacity and usability of the interface.

○ “Future work will require more studies in the design space siualization techniques.”

Conclusion

● Introduced autoencoder neural networks for design dimensionality reduction.

● Demonstrated the combination of shape features and PM parameters generates the latent space which is primarily organized by shape similarity.

● Evaluated that the proposed system improves user experience and productivity.

21

Weblab@The University of Tokyo

References

[Hinton 2006] Geoffrey E Hinton and Ruslan R Salakhutdinov. 2006. Reducing the dimensionality of data with neural networks. Science 313, 5786 (2006), 504–507.

[Yoav 2001] Yoav IH Parish and Pascal Muller. 2001. Procedural modeling of cities. In ACM SIGGRAPH. ACM, 301–308.

[Lintermann 1999] Bernd Lintermann and Oliver Deussen. 1999. Interactive modeling of plants. Computer Graphics and Applications 19, 1 (1999), 56–65.

[Koyama 2014] Yuki Koyama, Daisuke Sakamoto, and Takeo Igarashi. 2014. Crowd-powered parameter analysis for visual design exploration. In UIST. 65–74.

[Huang 2013] S. Huang, A. Shamir, C. Shen, H. Zhang, A. Sheffer, S. Hu, and D. Cohen-Or. 2013. Qualitative organization of collections of shapes via quartet analysis. ACM Trans. Graph. 32, 4 (2013), 71.

[Chen 2003] D. Chen, X. Tian, Y. Shen, and M. Ouhyoung. 2003. On visual similarity based 3D model retrieval. In Computer graphics forum, Vol. 22. 223–232.

22