# (2018. 1) Mask Gan

* Submitted on 2018. 1
* William Fedus, Ian Goodfellow, Andrew M. Dai

## Simple Summary

> Propose to improve sample quality using Generative Adversarial Networks (GANs), which explicitly train the generator to produce high quality samples and have shown a lot of success in image generation. GANs were originally designed to output differentiable values, so discrete language generation is challenging for them. We claim that validation perplexity alone is not indicative of the quality of text generated by a model. We introduce an actor-critic conditional GAN that fills in missing text conditioned on the surrounding context.

* Using GAN to generate samples in adversarial setup in texts has been limited due to the mode collapse and training instability issues. As a remedy to these problems an in-filling-task conditioning on the surrounding text has been proposed
* The approach is to use an actor-critic to train the generator of the GAN and use the usual maximum likelihood with SGD to train the discriminator.
* Architecture
  * use Seq2Seq and generator consists of an encoding module and decoding module.
  * encoder reads in the masked sequence.
  * generator decomposes the distribution over the sequence into an ordered conditional sequence.

![images](/files/-LRazLaOfUvBlq0Sby5i)

* Failure Cases
  * Mode Dropping is less extreme than SeqGAN but still noticeable
  * Matching Syntax at Boundaries
  * Loss of Global Context


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://humanbrain.gitbook.io/notes/notes/natural-language-processing/mask_gan.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.