Diversity regularized autoencoders for text generation

Hyeseon Ko, Junhyuk Lee, Jinhong Kim, Jongwuk Lee, Hyunjung Shim

Research output: Chapter in Book/Report/Conference proceedingConference contribution

2 Citations (Scopus)


In this paper, we propose a simple yet powerful text generation model, called diversity regularized autoencoders (DRAE). The key novelty of the proposed model lies in its ability to handle various sentence modifications such as insertions, deletions, substitutions, and maskings, and to take them as input. Because the noise-injection strategy enables an encoder to make the latent distribution smooth and continuous, the proposed model can generate more diverse and coherent sentences. Also, we adopt the Wasserstein generative adversarial networks with a gradient penalty to achieve stable adversarial training of the prior distribution. We evaluate the proposed model using quantitative, qualitative, and human evaluations on two public datasets. Experimental results demonstrate that our model using a noise-injection strategy produces more natural and diverse sentences than several baseline models. Furthermore, it is found that our model shows the synergistic effect of grammar correction and paraphrase generation in an unsupervised way.

Original languageEnglish
Title of host publication35th Annual ACM Symposium on Applied Computing, SAC 2020
PublisherAssociation for Computing Machinery
Number of pages9
ISBN (Electronic)9781450368667
Publication statusPublished - 2020 Mar 30
Event35th Annual ACM Symposium on Applied Computing, SAC 2020 - Brno, Czech Republic
Duration: 2020 Mar 302020 Apr 3

Publication series

NameProceedings of the ACM Symposium on Applied Computing


Conference35th Annual ACM Symposium on Applied Computing, SAC 2020
Country/TerritoryCzech Republic

Bibliographical note

Funding Information:
This work was supported by the National Research Foundation of Korea (NRF) grant (No. NRF-2018R1A2B6009135) and by the Korean National Police Agency and the Ministry of Science and ICT for Police field customized research and development project (No. NRF-2018M3E2A1081572). Also, this work was supported by the Institute of Information & communications Technology Planning & Evaluation (IITP) grant funded by the Korea government (MSIT) (No. 2019-0-00421, AI Graduate School Support Program and No. 2019-0-01590, High-Potential Individuals Global Training Program).

Publisher Copyright:
© 2020 ACM.

All Science Journal Classification (ASJC) codes

  • Software


Dive into the research topics of 'Diversity regularized autoencoders for text generation'. Together they form a unique fingerprint.

Cite this