Jiuge: A Human-Machine Collaborative Chinese Classical PoetryGeneration System

Zhipeng Guo1∗ , Xiaoyuan Yi1∗, Maosong Sun1† ,Wenhao Li1, Cheng Yang1, Jiannan Liang1, Huimin Chen1, Yuhui Zhang1, Ruoyu Li21Department of Computer Science and Technology, Tsinghua University, Beijing, China

Institute for Artificial Intelligence, Tsinghua University, Beijing, ChinaState Key Lab on Intelligent Technology and Systems, Tsinghua University, Beijing, China

26ESTATES PTE LTD, Singapore

Abstract

Research on the automatic generation of po-etry, the treasure of human culture, has lastedfor decades. Most existing systems, however,are merely model-oriented, which input someuser-specified keywords and directly completethe generation process in one pass, with lit-tle user participation. We believe that themachine, being a collaborator or an assistant,should not replace human beings in poetic cre-ation. Therefore, we proposed Jiuge, a human-machine collaborative Chinese classical po-etry generation system. Unlike previous sys-tems, Jiuge allows users to revise the unsatis-fied parts of a generated poem draft repeatedly.According to the revision, the poem will be dy-namically updated and regenerated. After therevision and modification procedure, the usercan write a satisfying poem together with Ji-uge system collaboratively. Besides, Jiuge canaccept multi-modal inputs, such as keywords,plain text or images. By exposing the optionsof poetry genres, styles and revision modes, Ji-uge, acting as a professional assistant, allowsconstant and active participation of users inpoetic creation.

1 Introduction

Language is one of the most important forms ofhuman intelligence, among different genres, po-etry is a beautiful, poetic and artistic genre whichexpresses one’s emotions and ideas with relativelyfewer words. Across various countries, nationali-ties, and cultures, poetry is always fascinating, im-pacting profoundly on the development of humancivilization.

Recently, researchers have worked on automaticpoetry generation. Meanwhile, neural networkshave proven to be powerful on this task (Zhangand Lapata, 2014; Wang et al., 2016; Yan, 2016;

∗ indicates equal contribution† Corresponding author: M.Sun(sms@tsinghua.edu.cn)

Zhang et al., 2017; Yi et al., 2017). Besides theresearch value of exploring human writing mech-anism and computer creativity, these models andsystems could also benefit electronic entertain-ment, advertisement, and poetry education.

However, the recently released Chinese poetrygeneration systems are mainly model-oriented,which take some user inputs and directly completethe generation in one pass, resulting in poor userparticipation. Moreover, these systems generatepoetry in fewer styles and genres, and provide lim-ited options for users. For example, the Daoxi-angju system1 requires the user to determine therhyme, which creates a barrier for beginners. TheOude system2 simplifies the user’s choices andonly allows the input of a few options and genres.The Microsoft Quatrain3 provides limited candi-dates of a theme and each line, but it only supportsthe generation of quatrains.

Due to the lack of user participation, the abovesystems are mainly designed for entertainment.We argue that the leading role in literary creationshould not be a machine, or at least not only a ma-chine, because it is difficult for machines to handlethe complex expressions of one’s emotion and theuse of images in poetic creation.

Rather than completely replace humans, a betterway is to utilize the system to assist human cre-ation. The human-machine collaboration mech-anism in Jiuge system can not only improve theemotions and semantics of generated poems butalso guide and teach beginners to understand thepoetic creation process.

In summary, the contributions of our Jiuge sys-tem are as follows:

• Multi-modal input. Jiuge can accept multi-

1http://www.poeming.com/web/2https://crl.ptopenlab.com:8800/poem/index3http://duilian.msra.cn/jueju/

Keywords:

Plain Text:

Picture:

plane, blue sky

There is a plane in the blue sky.

Extract

Expand

Transform

Genre: Quatrain, Acrostic, Iambic

Style: Standard, Sadness about seasons, …

Input Preprocessing Module

Working Memory Model

Genere Control

Unsupervised Style Control

Main Framework

Acrostic Poetry Generation

Generation Model

Pattern Checking

Re-Ranking

Postprocessing Model

Automatic Reference Recommendation

Collaborative Revision Module

Revision Modes

Static/Local Dynamic/Global Dynamic

������

����

������

�����

Final Poetry

The swan goose is flying outside

the clouds.

The heavy mist almost make the

ferry invisible.

The vast road has extended to

thousands of miles away.

It's so remote that it seems like it

reaches the sky.

Keywords: fly, blue sky, swan goose, vast

Figure 1: The architecture of Jiuge system.

modal input such as keywords, plain text, andeven images. For modern concepts in the in-put, Jiuge utilizes a knowledge graph to mapthem into relevant keywords in classical Chi-nese poetry.

• Various styles and genres. Unlike previoussystems, Jiuge provides more than twenty op-tions of genre and ten options of style, andcan generate more diverse poems.

• Human-machine collaboration. Jiuge sup-ports human-machine collaborative and inter-active generation. The user can revise the un-satisfied parts of a generated poem. In termsof the revision, Jiuge will dynamically updateand re-generate the poem. During this pro-cess, Jiuge also offers candidate words andhuman-authored poetry as references for be-ginners.

2 Architecture

2.1 Overview

We show the overall architecture of Jiuge systemin Fig. 1, which mainly consists of four modules:1) input preprocessing, 2) generation, 3) postpro-cessing and 4) collaborative revision. Given theuser-specified genre, style, and inputs (keywords,plain text or images), the preprocessing moduleextracts several keywords from the inputs and thenconducts keyword expansion to introduce richerinformation. Jiuge also transforms the words inmodern concepts, which are incompatible withclassical Chinese poetry (written in ancient Chi-nese language), such as refrigerator and airplane,to appropriate relevant ones, e.g., airplane → fly.With these preprocessed keywords, the generationmodule generates a poem draft. The postprocess-ing module re-ranks the candidates of each line

and removes the ones that do not conform to struc-tural and phonological requirements. At last, thecollaborative revision module interacts with theuser and dynamically updates the draft for severaltimes according to the user’s revision, to collabo-ratively create a satisfying poem.

We detail each module in the following parts.

2.2 Input Preprocessing ModuleKeyword Extraction. Jiuge allows multi-modalinput to meet the needs of generating poetry ac-cording to keywords, tweets or photos.

For plain text, we first use THULAC4 (Li andSun, 2009) to conduct Chinese word segmentationand compute the importance r(w) of each wordw:

r(w)=[α∗ti(w)+(1−α)∗tr(w)], (1)

where ti(w) and tr(w) are the TF-IDF (TermFrequency-Inverse Document Frequency) valueand TextRank (Mihalcea and Tarau, 2004) scorecalculated with the whole poetry corpus respec-tively. α is a hyper-parameter to balance theweights of ti(w) and tr(w). Afterwards, we se-lect top K words with the highest scores.

For each image, we use the Aliyun image recog-nition tool5, which gives the names of five rec-ognized objects with corresponding probabilitys(w). Then we select top K words with the high-est s(w) · r(w).

Keyword Mapping. The extracted or recog-nized keywords could be some modern concepts,such as airplane and refrigerator. Since thesewords never occur in the classical poetry corpus,the generation module will take them as a UNKsymbol and generate totally irrelevant poems.

To address this problem, we build a PoetryKnowledge Graph (PKG) from Wikipedia data,

4http://thulac.thunlp.org/5https://ai.aliyun.com/image

PKG

! = 0.33

! = 0.13 ! = 0.08

��

�(fly)�

(airplane)��

�(wind) (wing)

�

(a)

PWCG

!"#$% = 25.3

���!"#$% = 27.1 !"#$% = 20.7

(egret)��

(swan goose)��

(fly)�

(butterfly)��

(b)

Figure 2: (a) A sampled subgraph of PKG. (b) A sam-pled subgraph of PWCG.

which contains 616, 360 entities and 5, 102, 192relations. 40, 276 of these entities occur in our po-etry corpus. Before keywords extension and selec-tion, we first use PKG to map the modern conceptsto its most relevant entities in poetry, to guaranteeboth quality and relevance of generated poems.

For a modern concept word wi, we score itseach neighbor word wj by:

g(wj)= tfwiki(wj |wi) · log(N

1+df(wj)) · arctan(p(wj)

τ),

(2)

where tfwiki(wj |wi) is the term frequency of wj

in the Wikipedia article of wi, df(wj) is the num-ber of Wikipedia articles containing wj , N is thenumber of Wikipedia articles, and p(wj) is theword frequency counted in all articles. We give anexample of mapping the modern word “airplane”in Fig. 2(a).

Keyword Extension. The generation modulecan handle multi-keywords input. More keywordscould lead to richer contents and emotions in gen-erated poems. Therefore, if the number of ex-tracted keywords is less than K, we further con-duct keywords extension. To this end, we builda Poetry Word Co-occurrence Graph (PWCG) asshown in Fig. 2 (b). This graph indicates the co-occurrence of two words in the same poem. Theweight of the edge between two words is calcu-lated according to the Pointwise Mutual Informa-tion (PMI) as follows:

PMI(wi, wj) = logp(wi, wj)

p(wi) ∗ p(wj), (3)

where p(wi) and p(wi, wj) are the word frequencyand co-occurrence frequency in poetry corpus. Fora given word w, we get all its adjacent words wk

in PWCG and select those with higher values oflog p(wk) ∗ PMI(w,wk) + β ∗ r(wk) where β isa hyperparameter.

2.3 Generation ModuleAs shown in Fig. 3, the core component of thegeneration module is our proposed working mem-

Figure 3: The simplified structure of the working mem-ory model, which mainly comprise an encoder, a de-coder and there memory components. xi is the i-th lineand xi,j is the j-word in the i-th line. Please refer to (Yiet al., 2018b) for more details.

ory model (Yi et al., 2018b), which takes at mostK preprocessed keywords as input. The encodermaps each word or line into vector representa-tions, and the decoder generates each line word-by-word. The topic memory stores keywords ex-plicitly and independently, which can learn a flex-ible order and form of keywords expression. Thehistory memory and local memory are dynami-cally read and written to improve the context co-herence of generated poems.

Genere Control. Chinese classical poetry in-volves various genres, and each genre strictly de-fines the structural and phonological pattern of apoem, such as the length of each line, the toneof each word, and the number of lines. We useour designed genre embedding (Yi et al., 2018b)to disentangle the semantic content and the genrepattern. The genre embedding indicates the linelength, word tone, and rhyme, which is fed to thedecoder. By this way, we can train one model withall genres of poems and control the genre of gen-erated poems by specifying a pattern.

Training patterns are automatically extractedfrom the corpus. For generating, we make thegenre as a user option. However, the selection ofrhyme may be difficult for users without relevantliterature knowledge. Therefore, we train a classi-fier (implemented with a feedforward neural net-work) to predict an appropriate rhyme in terms ofthe keywords.

Unsupervised Style Control. Besides genres,there are also diverse styles in Chinese poetry suchas battlefield, romantic, pastoral, etc. For certaincontents or topics, creating different styles of po-etry is one main user requirement. Since the la-belled data is quite rare and expensive, we use

our proposed style disentanglement model (Yanget al., 2018) to achieve unsupervised style control.This method disentangles the style space into Mdifferent sub-spaces by maximizing the mutual in-formation between the style distribution and thegenerated poetry distribution.

It is noteworthy that this method is transparentto model structures which can be applied to anygeneration model. In this stage, we employ it forthe generation of Chinese quatrain poetry (Jueju),which will be extended to more genres in the fu-ture. We set the number of styles M = 10. Af-ter training, we manually annotate each style withsome descriptive phrases, such as sorrow duringdrinking and rural scenes, to indicate the theme ofthe corresponding style. The style selection is alsoset as a user option.

Acrostic Poetry Generation In Chinese po-etry, there is another special genre called acros-tic poetry. Given a sequence of words seq =(x0,0, x1,0, · · · , xn,0), which could be someone’sname or a blessing sentence, the author is re-quired to create a poem using each word xi,0 asthe first word of each line xi and the created poemshould also conform to the genre pattern and con-vey proper semantic meanings.

The input for this genre is the sequence seq.As our generation module takes keywords as in-put, we first use pre-trained word2vec embeddings(Mikolov et al., 2013) to get K keywords relatedto seq according to the cosine distance of eachkeyword and the words in seq. Then we directlyfeed each xi,0 into the decoder at the first step.

To alleviate the disfluency caused by thisconstraint, we generate the second word withthe conditional probability: pgen(xi,1|xi,0) =pdec(xi,1|xi,0)+ δ ∗plm(xi,1|xi,0), where pdec andplm are probability distributions of the decoderand a neural language model respectively.

If the length of the input sequence is less thann (the number of lines in a poem), we also use thelanguage model to extend it to n words.

2.4 Postprocessing Module

Jiuge takes a line-to-line generation schema andgenerates each line with beam search (beamsize=B). As a result, we can get B candidates foreach line. We design a postprocessing module toautomatically check and re-rank these candidates,and then select the best one, which is used for thegeneration of subsequent lines in a poem.

Pattern Checking. The genre embedding intro-duced in Sec. 2.3 cannot guarantee that generatedpoems perfectly adhere to required patterns. Thus,we further remove the invalid candidates accord-ing to the specified length, rhythm, and rhyme.

Re-Ranking. Our preliminary experimentsshow that the best candidate may not be ranked asthe top 1 because the training objective is Maxi-mum Likelihood Estimation (MLE), which tendsto give the generic and meaningless candidateslower costs (Yi et al., 2018a). To automaticallyselect the best candidate, we adopt the automaticrewarders we proposed in (Yi et al., 2018a), in-cluding a fluency rewarder, a context coherencerewarder, and a meaningfulness rewarder. Thenthe candidate with the highest weighted-averagerewards given by them will be selected.

2.5 Collaborative Revision ModuleWe call the poem generated by the generationmodule in one pass the draft, since the user mayrevise it for several times to collaboratively cre-ate a satisfying poem together with the machine.We implement such collaboration with a revisionmodule.

Revision Modes. Define a n-line poem draftas X = (x1, x2, · · · , xn), and each line contain-ing li words as xi = (xi,1, xi,2, ..., xi,li). At ev-ery turn, the user can revise one word in the draft.Then the revision module returns the revision in-formation to the generation module which updatesthe draft according to the revised word. We imple-ment three revision modes in terms of the updatingway: static, local dynamic, and global dynamic.

• Static updating mode. The revision is re-quired to meet the phonological pattern of thedraft and the draft will not be updated exceptthe revised word. The rhythm and rhyme in-formation is given to the user together withthe generated draft and the invalid revisionwill be alerted. During the beam search pro-cess, we also store top 10 candidate words ineach position as recommendations.

• Local dynamic updating mode. If theuser revises word xi,j , then Jiuge willre-generate the succeeding subsequence,xi,j+1, · · · , xi,li , in the i-th line by feedingthe revised word to the decoder for the re-vised position.

• Global dynamic updating mode.

If the user revises word xi,j , Jiugewill re-generate all succeeding words,xi,j+1, · · · , xi,li , · · · , xn,ln . In terms of therevision position, e.g., the rhymed positions,a new phonological pattern may be adopted.

Thanks to this collaborative revision-updating pro-cess, the user can choose a mode and gradually re-vise the draft until she/he feels satisfied.

Automatic Reference Recommendation. Forpoetry writing beginners or these lacking profes-sional knowledge, it is hard to revise the draft ap-propriately. To aid in the revision process, weimplement an automatic recommendation compo-nent. This component searches several human-authored poems, which are semantically similarto the generated draft, for the user as references.Then the user could decide how to make revisionwith these references.

In detail, define a n-line human-created poem asY = (y1, · · · , yn) and a relevance scoring func-tion as rel(xi, yi) to give the relevance of twolines. Then we return N poems with the highestrelevance score rs(X,Y ), calculated as:

rs(X,Y )=

n∑i=1

rel(xi, yi) + γ ∗ I(Y ∈ Dmaster), (4)

where Dmaster is a poetry set of masterpieces, Iis the indicator function, and the hyper-parameterγ is specified to balance the quality and relevanceof searched poems. For more details of the rele-vance scoring function, we refer the reader to ourprevious work (Liang et al., 2018).

3 Demonstration

We implement a webpage version of Jiuge6 whichallows users to create diverse poems and sharewith the others conveniently. The initial page pro-vides some basic options: multi-modal input andthe selection of genre and style, as shown in Fig. 4.

Jiuge is easy to use. After selecting the in-put mode and providing corresponding content,the user can further choose a favourite genre andstyle7. As shown in Fig. 5 (a), the user inputs twokeywords, desert and cavalry, and chooses to gen-erate a quatrain with the normal style. Then theuser clicks the “Generate Poetry” button. After afew seconds, the system returns the processed key-words and a poem draft.

6Our system is available at https://jiuge.thunlp.cn/ .

7Temporarily only the Jueju genre supports multiplestyles.

Figure 4: The initial page of Jiuge System.

In order to help the user collaboratively revisethe generated poem, Jiuge provides some high-quality human-authored poems which are seman-tically similar to the generated one, as the refer-ences. The user can click the button to changethe references. At this time, the user can selectan unsatisfying word to be revised in the draftand then Jiuge will give some recommended re-vision candidates. Besides these candidates, otherchoices are also allowed. After selecting the revi-sion mode introduced in Sec. 2.5, Jiuge will updateor re-generate the draft according to the revisedword. Through several turns of collaborative revi-sion, the user and Jiuge work together to create asatisfying poem, as in Fig. 5 (b)8.

In addition, Jiuge also supports picture sharing.By clicking the “Share Poetry” button, Jiuge willprint the created poem on a beautiful picture sothat the user can share it with others.

4 Conclusion and Future Work

We demonstrate Jiuge, a human-machine collab-orative Chinese classical poetry generation sys-tem. Our system accepts multi-modal input andallows deep user participation in the choice of var-ious styles and genres, as well as in collaborativerevision. With an easy-to-use web interface, theuser can collaboratively create a satisfying poemtogether with the system. Instead of being a simpleentertainment software, Jiuge takes a step towardsprofessional AI assistant for poetic education.

We collect a large number of human usagerecords from the interface, laying the foundationfor enhancing the collaborative creation methodin the future. We will also continue to integratemore styles and extend the collaborative creation

8The poems in Fig.5 are manually translated for betterdemonstration. In practice, we utilize a neural translationmodel to conduct automatic translation.

(a) (b)

Figure 5: The collaborative creation page: an example of revision. (a) The poem draft generated in one pass. (b)The poem after four turns of revision and updating.

to more genres.

5 Acknowledgements

We would like to thank Cunchao Tu, Yunqiu Shaoand anonymous reviewers for their insightful com-ments. It is supported by Natural Science Foun-dation of China (NSFC) Grant 61532001 and theNExT++ project, the National Research Founda-tion, Prime Ministers Office, Singapore under itsIRC@Singapore Funding Initiative.

References

Zhongguo Li and Maosong Sun. 2009. Punctuation asimplicit annotations for chinese word segmentation.computational linguistics. Computational Linguis-tics, 35(4):505–512,.

Jiannan Liang, Maosong Sun, Xiaoyuan Yi, ChengYang, Huimin Chen, and Zhenghao Liu. 2018. Neu-ral network-based jiju poetry generation. In Pro-ceedings of the Seventeenth China National Con-ference on Computational Linguistics, Changsha,China.

Rada Mihalcea and Paul Tarau. 2004. Textrank: Bring-ing order into text. In Proceedings of the 2004 Con-ference on Empirical Methods in Natural LanguageProcessing.

Tomas Mikolov, Kai Chen, Greg Corrado, and Jef-frey Dean. 2013. Efficient estimation of wordrepresentations in vector space. arXiv preprintarXiv:1301.3781.

Zhe Wang, Wei He, Hua Wu nad Haiyang Wu, WeiLi, Haifeng Wang, and Enhong Chen. 2016. Chi-nese poetry generation with planning based neu-ral network. In Proceedings of COLING 2016,the 26th International Conference on ComputationalLinguistics:Technical Papers, pages 1051–1060, Os-aka, Japan.

Rui Yan. 2016. i,poet:automatic poetry compositionthrough recurrent neural networks with iterative pol-ishing schema. In Proceedings of the Twenty-FifthInternational Joint Conference on Artificial Intelli-gence, pages 2238–2244, New York, USA.

Cheng Yang, Maosong Sun, Xiaoyuan Yi, and WenhaoLi. 2018. Stylistic chinese poetry generation via un-supervised style disentanglement. In Proceedings ofthe 2018 Conference on Empirical Methods in Nat-ural Language Processing, pages 3960–3969, Brus-sels, Belgium.

Xiaoyuan Yi, Ruoyu Li, and Maosong Sun. 2017. Gen-erating chinese plassical poems with rnn encoder-decoder. In Proceedings of the Sixteenth ChineseComputational Linguistics, pages 211–223, Nan-jing, China.

Xiaoyuan Yi, Maosong Sun, Ruoyu Li, and WenhaoLi. 2018a. Automatic poetry generation with mu-tual reinforcement learning. In Proceedings of the2018 Conference on Empirical Methods in NaturalLanguage Processing, pages 3143–3153, Brussels,Belgium.

Xiaoyuan Yi, Maosong Sun, Ruoyu Li, and Zong-han Yang. 2018b. Chinese poetry generation witha working memory mode. In Proceedings of theTwenty-Seventh International Joint Conference onArtificial Intelligence, pages 4553–4559, Stock-holm, Sweden.

Jiyuan Zhang, Yang Feng, Dong Wang, Yang Wang,Andrew Abel, Shiyue Zhang, and Andi Zhang.2017. Flexible and creative chinese poetry gener-ation using neural memory. In Proceedings of the55th Annual Meeting of the Association for Compu-tational Linguistics, pages 1364–1373. Associationfor Computational Linguistics.

Xingxing Zhang and Mirella Lapata. 2014. Chinesepoetry generation with recurrent neural networks.In Proceedings of the 2014 Conference on Empiri-cal Methods in Natural Language Processing, pages670–680, Doha, Qatar.