Daniel Berio - Graffiti synthesis, a motion centric approach - Creative AI meetup

Autograff

Optimality Principles

in the Procedural Generation of Graffiti Style

Graffiti synthesis, a motion centric approach

Daniel Berio

http://doc.gold.ac.uk/autograf

@colormotor

Graffiti art

Graffiti art

Graffiti – canvas in motion

Graffiti art - tags

• Elementary “Atom” of graffiti art

• Highly stylized signature denoting an artist’s pseudonym

• Generated by rapidly executed and well learned movements– Different style are referred to as “hand styles”

– In graffiti jargon, a well made tag has ”flow”

Embodied perception

• Well known relations between kinematics and geometry of human movements (e.g. power laws) (Lacquaniti et al. 1983)

• Observation of a human made trace triggers the mental recovery of the movement underlying its production (Freedberg and Gallese 2007, Longcamp et al. 2006, Pignocchi 2010)

• Such recovery influences aesthetic appreciation (Leder et al. 2012)

Embodied perception

• Well known relations between kinematics and geometry of human movements (e.g. power laws) (Lacquaniti et al. 1983)

• Observation of a human made trace triggers the mental recovery of the movement underlying its production (Freedberg and Gallese 2007, Longcamp et al. 2006, Pignocchi 2010)

• Such recovery influences aesthetic appreciation (Leder et al. 2012)

• It follows that an appropriate simulation of movement may trigger a similar effect in the viewer for computer generated traces.

Graffiti Taxonomy

Evan Roth, Graffiti Taxonomy https://www.moma.org/collection/works/147174

Common letter structure

Bi-level representation of “style”

Movement centric curve generation

Trajectory formation model Stylized trajectoriesControl polygon / motor plan

Parameters ?

Data driven approach

Daniel Berio, Memo Akten, Frederic Fol Leymarie, Mick Grierson, Réjean Plamondon

Calligraphic Stylisation Learning with a Physiologically Plausible Model of Movement and Recurrent Neural Networks, 2017, MOCO

Trajectory formation model Stylized trajectoriesStructure (motor plan)

Graphonomics

The scientific field ”concerned with the systematic

relationships involved in the generation and analysis of the

handwriting and drawing movements, and the resulting

traces of writing and drawing instruments”(Kao, Hoosain, & Van Galen, 1986)



Graphonomics - principles

• Aiming movements assume a “bell shaped” speed profile (Morasso, 1981)

• Handwriting movements can be decomposed into a discrete number of aiming movement primitives (strokes) (Teulings and Schomaker 1993, Mussa Ivaldi and Solla 2004, Sosnik et al. 2004, Plamondon et al. 2014)

– Also characterized by the same bell shaped speed profile.

– Each stroke is aimed at a virtual (imaginary) target



Kinematic Theory - Sigma Lognormal Model (Plamondon et al. 2014)









Sigma Lognormal Model

STRUCTURE“Virtual Targets”

KINEMATICS/HANDSTYLE“Dynamic parameters”



User input



Reconstruction

(a) (b)

(c)

virtual targets



Randomly perturb parameters of sigma lognormal model (e.g. +- 10%)

Data Augmentation

Video 1

Optimisation approach

Daniel Berio, Sylvain Calinon, Frederic Fol Leymarie

Generating Calligraphic Trajectories with Model Predictive Control, 2017, Graphics Interface

Trajectory formation model Stylized trajectoriesStructure (motor plan)

Opt imizat ion / performance crit erion

Computational Motor Control

• Complex hand and arm motions tend to be smooth– Minimization of a cost or performance criterion.

– Minimum Square Derivative models (Flash & Hogan 1985, Flash 1983, Dingwell et al. 2004)

• Minimization of the squared magnitude of high derivatives of positionsuch as jerk (3rd, change in acceleration), snap (4th, change in jerk) etc…

• Optimal feedback control - Minimal intervention principle (Todorov & Jordan 2002)

– Deviations from an average trajectory are only corrected if they interfere with the required task precision.

– Higher variability → reduced effort → smoother trajectory



Cost function



Gaussian targets



Video 2



Procedural generation – random covariance


Dynamic Graffiti Stylisation with Stochastic Optimal Control, 2017, MOCO

Procedural generation – tied/semi-tied covariance



Video 3



Generative Glyphs



The end

Daniel [email protected] - http://www.enist.org

More info:http://doc.gold.ac.uk/autograff

Relevant references:Berio D., Akten M., Fol Leymarie F., Grierson M., Plamondon R. Calligraphic Stylisation Learning with a Physiologically Plausible Model of Movement and Recurrent Neural Networks Proc. of 4th Int’l Conf. on Movement Computing (MOCO). London, UK, 2017.

Berio D., Calinon S., Fol Leymarie F. Dynamic Graffiti Stylisation with Stochastic Optimal Control ACM Proceedings of the 4th International Conference on Movement and Computing. London, UK, June 2017.

Berio D., Calinon S., Fol Leymarie F. Generating Calligraphic Trajectories with Model Predictive Control Proceedings of Graphics Interface. Edmonton, Canada: Canadian Human-Computer Communications Society, May 2017.

mailto:[email protected]

http://www.enist.org/

http://doc.gold.ac.uk/autograff

Technology

Daniel Berio - Graffiti synthesis, a motion centric approach - Creative AI meetup