Embed Size (px)
Citation preview
Data-Driven Handwriting Synthesis in a Conjoined
MannerHsin-I Chen, Tse-Ju Lin, Xiao-Feng Jian, I-Chao Shen, Bing-Yu Chen
National Taiwan University, University of British Columbia
1 1 1 2 1
1 2
Handwriting Font is Popular
Personal Website
Invitation letter
Personal Decoration
Handwriting Synthesis
“The grown-ups are certainly very, very old”, he said to himself, as he continued on his journey.
“The grown-ups are certainly very, very old”, he said to himself, as he continued on his journey
Segoe print
Lucida Handwriting
Different shape of the same character
he little prince he tippler
Previous work - Handwriting Style
From forensic science viewpoint, Important factors of handwriting style:
Elements of styles Elements of execution Natural variance between each writing
Other effects Alcohol, emotion, etc.
Previous work - Handwriting Synthesis
Handwriting
Synthesized
Handwriting Synthesized
Handwriting
Synthesized
Wang et al., IJDAR 2005 Chang and Shin, IJDAR 2012
Lin and Wang, PR 2007
Contribution
A statistical learning approach to synthesize non-existent paragraph:
A data-analysis stage
A character grouping method
A data-driven optimization framework
Approach pipeline
Data Collection Parameterization Shape Model
Paragraph Synthesis Word Synthesis
Character Synthesis
Approach pipeline
Data Collection Parameterization Shape Model
Paragraph Synthesis Word Synthesis
Character Synthesis
Data Collection
We should collect at least two instances for each letter.
We should cover more commonly used letter pairs(“aa”, “ab”, “ac”,…).
The collection sheet should not over be constrained.
10
Data Collection
Approach pipeline – Parameterization
Data collection Parameterization Shape Model
Paragraph Synthesis Word Synthesis
Character Synthesis
Character Parameterization
We assume each letter has only one topology. For each letter , we use B-spline fitting.
Handwritten
Reconstructed
Control points
Approach pipeline – Shape Model
Data collection Parameterization Shape Model
Paragraph Synthesis Word Synthesis
Character Synthesis
Shape Model
We build shape model from displacement:
Shape model :
Shape coefficient
Displacement
Approach pipeline – Character Synthesis
Data collection Parameterization Shape Model
Paragraph Synthesis Word Synthesis
Character Synthesis
Character Synthesis
Collected data
Synthesized data
Shape model with varying
Approach pipeline – Word Synthesis
Data collection Parameterization Shape Model
Paragraph Synthesis Word Synthesis
Character Synthesis
Character Grouping
We group letters to obtain more conjoining information
Example: Criteria
Synthesis target:
Data set:
How to connect ‘r’ and ‘i’ ?
Character Grouping
The probability we connect two neighboring letters
Ending group Starting group
Word Synthesis
Structure similarity constraint : shape parameter : the slope between control points control point position
Smoothness term
Data term
Boundary constraint
Word Synthesis
Synthesized Result
Connected
Approach pipeline – Paragraph synthesis
Data collection Parameterization Shape Model
Paragraph Synthesis Word Synthesis
Character Synthesis
Paragraph Synthesis
Line angle, word height and word angle
User Study : Visual discrimination
User Study : Visual discrimination
User Study : Visual discrimination
65 Subjects
User Study : Similarity test
Handwritten sample
Our result
User Study : Similarity test
Handwritten paragraph
Synthesized paragraph without layout info
Synthesized paragraph with layout info
Visual Result
Handwritten paragraph
Our synthesized paragraph
Visual Result
Style 1
Style 2
Style 3
Comparison II
Handwritten samples
Synthesized result by Lin et al., PR2007
Handwritten samples
Our result
Comparison I
Original writing
Wang et al., IJDAR 2005
Ours
Conclusion
We present a data-driven optimization approach to synthesize non-existed paragraph :
We analyzing cursiveness property in the data. A novel trajectory optimization for synthesizing
conjoining character. The user study and comparison results show that
our approach successfully imitate one’s handwriting style.
Q & A
Thank you
http://graphics.csie.ntu.edu.tw/~fensi/
