Natural Language Processing

• Distributed Representations of Words and Phrases and their Compositionality (2013. 10)

  • Word2Vec, CBOW, Skip-gram

  • arXiv

• GloVe: Global Vectors for Word Representation (2014)

  • Word2Vec, GloVe, Co-Occurrence

  • paper

• Convolutional Neural Networks for Sentence Classification (2014. 8)

  • CNN, Classfication

  • arXiv, code

• Neural Machine Translation by Jointly Learning to Align and Translate (2014. 9)

  • Seq2Seq, Attention(Align), Translation

  • arXiv, note, code

• Text Understanding from Scratch (2015. 2)

  • CNN, Character-level

  • arXiv

• Ask Me Anything: Dynamic Memory Networks for Natural Language Processing (2015. 6)

  • Memory, QA, bAbi

  • arXiv, code

• Pointer Networks (2015. 6)

  • Seq2Seq, Attention, Combinatorial

  • arXiv, note

• Skip-Thought Vectors (2015. 6)

  • Sentence2Vec, Unsupervised

  • arXiv, note

• A Neural Conversational Model (2015. 6)

  • Seq2Seq, Conversation

  • arXiv

• Teaching Machines to Read and Comprehend (2015. 6)

  • Deepmind, Attention, QA

  • arXiv, note

• Effective Approaches to Attention-based Neural Machine Translation (2015. 8)

  • Seq2Seq, Attention, Translation

  • arXiv, note, code

• Character-Aware Neural Language Models (2015. 8)

  • CNN, Character-level

  • arXiv

• Neural Machine Translation of Rare Words with Subword Units (2015. 8)

  • Out-Of-Vocabulary, Translation

  • arXiv, note

• A Diversity-Promoting Objective Function for Neural Conversation Models (2015. 10)

  • Conversation, Objective

  • arXiv, note

• Multi-task Sequence to Sequence Learning (2015. 11)

  • Multi-Task, Seq2Seq

  • arXiv, note

• Multilingual Language Processing From Bytes (2015. 12)

  • Byte-to-Span, Multilingual, Seq2Seq

  • arXiv, note

• Strategies for Training Large Vocabulary Neural Language Models (2015. 12)

  • Vocabulary, Softmax, NCE, Self Normalization

  • arXiv, note

• Incorporating Structural Alignment Biases into an Attentional Neural Translation Model (2016. 1)

  • Seq2Seq, Attention with Structural Biases, Translation

  • arXiv

• Long Short-Term Memory-Networks for Machine Reading (2016. 1)

  • LSTMN, Intra-Attention, RNN

  • arXiv

• Recurrent Memory Networks for Language Modeling (2016. 1)

  • RMN, Memory Bank

  • arXiv

• Exploring the Limits of Language Modeling (2016. 2)

  • Google Brain, Language Modeling

  • arXiv, note

• Swivel: Improving Embeddings by Noticing What's Missing (2016. 2)

  • Word2Vec, Swivel, Co-Occurrence

  • arXiv

• Incorporating Copying Mechanism in Sequence-to-Sequence Learning (2016. 3)

  • CopyNet, Seq2Seq

  • arXiv, note

• Achieving Open Vocabulary Neural Machine Translation with Hybrid Word-Character Models (2016. 4)

  • Translation, Hybrid NMT, Word-Char

  • arXiv, note

• Adversarial Training Methods for Semi-Supervised Text Classification (2016. 5)

  • Regulaizer, Adversarial, Virtual Adversarial Training (Semi-Supervised)

  • arXiv, note

• SQuAD: 100,000+ Questions for Machine Comprehension of Text (2016. 6)

  • DataSet, Reading Comprehension

  • arXiv, note, dataset

• Sequence-Level Knowledge Distillation (2016. 6)

  • Distil, Teacher-Student

  • arXiv, note

• Attention-over-Attention Neural Networks for Reading Comprehension (2016. 7)

  • Attention, Cloze-style, Reading Comprehension

  • arXiv, note

• Recurrent Neural Machine Translation (2016. 7)

  • Translation, Attention (RNN)

  • arXiv

• An Actor-Critic Algorithm for Sequence Prediction (2016. 7)

  • Seq2Seq, Actor-Critic, Objective

  • arXiv, note

• Pointer Sentinel Mixture Models (2016. 9)

  • Language Modeling, Rare Word, Salesforce

  • arXiv, note

• Multiplicative LSTM for sequence modelling (2016. 10)

  • mLSTM, Language Modeling, Character-Level

  • arXiv

• Diverse Beam Search: Decoding Diverse Solutions from Neural Sequence Models (2016. 10)

  • Diverse, DBS

  • arXiv, note

• Fully Character-Level Neural Machine Translation without Explicit Segmentation (2016. 10)

  • Translation, CNN, Character-Level

  • arXiv, note

• Neural Machine Translation in Linear Time (2016. 10)

  • ByteNet, WaveNet + PixelCNN, Translation, Character-Level

  • arXiv, note

• Bidirectional Attention Flow for Machine Comprehension (2016. 11)

  • QA, BIDAF, Machine Comprehension

  • arXiv, note, code

• Dynamic Coattention Networks For Question Answering (2016. 11)

  • QA, DCN, Coattention Encoder, Machine Comprehension

  • arXiv

• Dual Learning for Machine Translation (2016. 11)

  • Translation, RL, Dual Learning (Two-agent)

  • arXiv, note

• Neural Machine Translation with Reconstruction (2016. 11)

  • Translation, Auto-Encoder, Reconstruction

  • arXiv, note

• Quasi-Recurrent Neural Networks (2016. 11)

  • QRNN, Parallelism, Conv + Pool + RNN

  • arXiv, note

• A recurrent neural network without chaos (2016. 12)

  • RNN, CFN, Dynamic, Chaos

  • arXiv

• Comparative Study of CNN and RNN for Natural Language Processing (2017. 2)

  • Systematic Comparison, CNN vs RNN

  • arXiv

• A Structured Self-attentive Sentence Embedding (2017. 3)

  • Sentence Embedding, Self-Attention, 2-D Matrix

  • arXiv, note

• Dynamic Word Embeddings for Evolving Semantic Discovery (2017. 3)

  • Word Embedding, Temporal, Alignment

  • arXiv, the morning paper

• Learning to Generate Reviews and Discovering Sentiment (2017. 4)

  • Sentiment, Unsupervised, OpenAI

  • arXiv

• Ask the Right Questions: Active Question Reformulation with Reinforcement Learning (2017. 5)

  • QA, Active Question Answering, RL, Agent (Reformulate, Aggregate)

  • arXiv, open_review

• Reinforced Mnemonic Reader for Machine Reading Comprehension (2017. 5)

  • QA, Mnemonic (Syntatic, Lexical), RL, Machine Comprehension

  • arXiv

• Attention Is All You Need (2017. 6)

  • Self-Attention, Seq2Seq (without RNN, CNN)

  • arXiv, note, code

• Depthwise Separable Convolutions for Neural Machine Translation (2017. 6)

  • SliceNet, Super-Separable Conv, Depsewise + Conv 1x1

  • arXiv, open_review, note

• MEMEN: Multi-layer Embedding with Memory Networks for Machine Comprehension (2017. 7)

  • MEMEN, QA(MC), Embedding(skip-gram), Full-Orientation Matching

  • arXiv

• On the State of the Art of Evaluation in Neural Language Models (2017. 7)

  • Standard LSTM, Regularisation, Hyperparemeter

  • arXiv

• Text Summarization Techniques: A Brief Survey (2017. 7)

  • Summarization, Survey

  • arXiv, note

• Adversarial Examples for Evaluating Reading Comprehension Systems (2017. 7)

  • Concatenative Adversaries(AddSent, AddOneSent), SQuAD

  • arXiv

• Learned in Translation: Contextualized Word Vectors (2017. 8)

  • Word Embedding, CoVe, Context Vector

  • arXiv

• Simple and Effective Multi-Paragraph Reading Comprehension (2017. 10)

  • Document-QA, Select Paragraph-Level, Confidence Based, AllenAI

  • arXiv, note

• Unsupervised Neural Machine Translation (2017. 10)

  • Train with both direction (tandem), Shared Encoder, Denoising Auto-Encoder

  • arXiv, open_review

• Word Translation Without Parallel Data (2017. 10)

  • Unsupervised, Multilingual Embedding, Parallel Dictionary Induction

  • arXiv, open_review

• Unsupervised Machine Translation Using Monolingual Corpora Only (2017. 11)

  • Unsupervised, Adversarial, Monolingual Corpora

  • arXiv, open_review

• Neural Text Generation: A Practical Guide (2017. 11)

  • Seq2Seq, Decoder Guide

  • arXiv, note

• Breaking the Softmax Bottleneck: A High-Rank RNN Language Model (2017. 11)

  • MoS (Mixture of Softmaxes), Softmax Bottleneck

  • arXiv

• Neural Speed Reading via Skim-RNN (2017. 11)

  • Skim-RNN, Speed Reading, Big(Read)-Small(Skim), Dynamic

  • arXiv, open_review

• Generalization without systematicity: On the compositional skills of sequence-to-sequence recurrent networks (2017. 11)

  • SCAN, Compositional, Mix-and-Match

  • arXiv

• The NarrativeQA Reading Comprehension Challenge (2017. 12)

  • NarrativeQA, Dataset, DeepMind

  • arXiv, dataset

• Hierarchical Text Generation and Planning for Strategic Dialogue (2017. 12)

  • End2End Strategic Dialogue, Latent Sentence Representations, Planning + RL

  • arXiv

• Recent Advances in Recurrent Neural Networks (2018. 1)

  • RNN, Recent Advances, Review

  • arXiv

• Personalizing Dialogue Agents: I have a dog, do you have pets too? (2018. 1)

  • Chit-chat, Profile Memory, Persona-Chat Dataset, ParlAI

  • arXiv

• Generating Wikipedia by Summarizing Long Sequences (2018. 1)

  • Multi-Document Summarization, Extractive-Abstractive Stage, T-DMCA, WikiSum, Google Brain

  • arXiv, note, open_review

• MaskGAN: Better Text Generation via Filling in the__ (2018. 1)

  • MaskGAN, Neural Text Generation, RL Approach

  • arXiv, open_review, note

• Beyond Word Importance: Contextual Decomposition to Extract Interactions from LSTMs (2018. 1)

  • Contextual Decomposition (CD), Disambiguate interactions between Gates

  • arXiv, open_review

• DeepType: Multilingual Entity Linking by Neural Type System Evolution (2018. 2)

  • DeepType, Symbolic Information, Type System, Open AI

  • arXiv, openai blog

• Deep contextualized word representations (2018. 2)

  • biLM, ELMo, Word Embedding, Contextualized, AllenAI

  • arXiv, note

• Ranking Sentences for Extractive Summarization with Reinforcement Learning (2018. 2)

  • Document-Summarization, Cross-Entropy vs RL, Extractive

  • arXiv

• code2vec: Learning Distributed Representations of Code (2018. 3)

  • code2vec, Code Embedding, Predicting method name

  • arXiv

• Universal Sentence Encoder (2018. 3)

  • Transformer, Deep Averaging Network (DAN), Transfer

  • arXiv

• An efficient framework for learning sentence representations (2018. 3)

  • Sentence Representation, True Context, Unsupervised

  • arXiv, open_review

• An Analysis of Neural Language Modeling at Multiple Scales (2018. 3)

  • LSTM vs QRNN, Hyperparemeter, AWD-QRNN

  • arXiv

• Analyzing Uncertainty in Neural Machine Translation (2018. 3)

  • Uncertainty, Beam Search Degradation, Copy Mode

  • arXiv

• An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling (2018. 3)

  • Temporal Convolutional Network (TCN), CNN vs RNN

  • arXiv

• Training Tips for the Transformer Model (2018. 4)

  • Transformer, Hyperparameter, Multiple GPU

  • arXiv

• QANet: Combining Local Convolution with Global Self-Attention for Reading Comprehension (2018. 4)

  • QA, Conv - Self-Attention, Backtranslation (Data Augmentation)

  • arXiv, open_review, note

• SimpleQuestions Nearly Solved: A New Upperbound and Baseline Approach (2018. 4)

  • Top-K Subject Recognitio, Relation Classification

    • arXiv

• Delete, Retrieve, Generate: A Simple Approach to Sentiment and Style Transfer (2018. 4)

  • Sentiment Transfer, Disentangle Attribute, Unsupervised

    • arXiv

• Parsing Tweets into Universal Dependencies (2018. 4)

  • Universal Dependencies (UD), TWEEBANK v2

  • arXiv

• Subword Regularization: Improving Neural Network Translation Models with Multiple Subword Candidates (2018. 4)

  • SR, Subword Sampling + Hyperparameter, Segmentation (BPE, Unigram)

  • arXiv

• Phrase-Indexed Question Answering: A New Challenge for Scalable Document Comprehension (2018. 4)

  • PI-SQuAD, Challenge, Document Encoder, Scalability

  • arXiv

• On the Practical Computational Power of Finite Precision RNNs for Language Recognition (2018. 5)

  • Unbounded counting, IBFP-LSTM

  • arXiv

• Paper Abstract Writing through Editing Mechanism (2018. 5)

  • Writing-editing Network, Attentive Revision Gate

  • arXiv

• A robust self-learning method for fully unsupervised cross-lingual mappings of word embeddings (2018. 5)

  • Unsupervised initialization scheme, Robust self-leraning

  • arXiv

• Efficient and Robust Question Answering from Minimal Context over Documents (2018. 5)

  • Sentence Selector, Oracle Sentence, Minimal Set of Sentences (SpeedUp)

  • arXiv, note

• Global-Locally Self-Attentive Dialogue State Tracker (2018. 5)

  • GLAD, WoZ and DSTC2 Dataset

  • arXiv

• Learning to Ask Good Questions: Ranking Clarification Questions using Neural Expected Value of Perfect Information (2018, 5)

  • Dataset, EVPI, ACL 2018 Best Paper

  • arXiv

• Know What You Don't Know: Unanswerable Questions for SQuAD (2018, 6)

  • SQuAD 2.0, Negative Example, ACL 2018 Best Paper

  • arXiv, leaderboard

• The Natural Language Decathlon: Multitask Learning as Question Answering (2018, 6)

  • decaNLP, Multitask Question Answering Network (MQAN), Transfer Learning

  • arXiv

• GLoMo: Unsupervisedly Learned Relational Graphs as Transferable Representations (2018, 6)

  • Transfer Learning Framework, Structured Graphical Representations

  • arXiv

• Improving Language Understanding by Generative Pre-Training (2018, 6)

  • Transformer, Generative Pre-Training, Discriminative Fine-Tuning

  • paper, open_ai_blog

• Finding Syntax in Human Encephalography with Beam Search (2018, 6)

  • RNNG+beam search, ACL 2018 Best Paper

  • arXiv

• Let's do it "again": A First Computational Approach to Detecting Adverbial Presupposition Triggers (2018, 6)

  • Task, Dataset, Weighted-Pooling (WP) ACL 2018 Best Paper

  • arXiv

• QuAC : Question Answering in Context (2018. 8)

  • Information-Seeking dialog, Challenge, Without Evidence

  • arXiv, leaderboard

• CoQA: A Conversational Question Answering Challenge (2018. 8)

  • Abstractive with Extractive Rationale, Challenge, Coreference and Pragmatic Reasoning

  • arXiv, leaderboard

• Contextual Parameter Generation for Universal Neural Machine Translation (2018. 8)

  • Parameter Generation, Language Embedding, EMNLP 2018

  • arXiv

• Evaluating Theory of Mind in Question Answering (2018. 8)

  • Dataset, Higher-order Beliefs, EMNLP 2018

  • arXiv

• Open Domain Question Answering Using Early Fusion of Knowledge Bases and Text (2018. 9)

  • GRAFT-Net, KB+Text Fusion, EMNLP 2018

  • arXiv

• HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering (2018. 9)

  • Dataset, Multi-hop, Sentence-level Supporting Fact, EMNLP 2018

  • arXiv

• BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding (2018. 10)

  • BERT, Discriminative, Large Pretrained, Transfer Learning

  • arXiv, glue_benchmark