It generates synthetic examples using pretrained transformer language models and selects the most informative and diverse set for augmentation.

University of Michigan - Ann Arbor

Seq2seq and language models have also been used for DA. Techniques include applying selective gates, generating different samples, and augmenting word representations with a context-sensitive attention-based mixture. 

Model-Based DA Techniques in NLP

https://aclanthology.org/2020.findings-emnlp.90/

Generative Data Augmentation for Commonsense Reasoning

https://arxiv-download.xixiaoyao.cn/pdf/2105.03075.pdf

A Survey of Data Augmentation Approaches for NLP

The popular BACKTRANSLATION method translates a sequence into another language and then back into the original language.

BackTranslation

Improving Neural Machine Translation Models with Monolingual Data

Keep Calm and Switch On! Preserving Sentiment and Fluency in Semantic Text Exchange

It involves adjusting the overall semantics of a text to fit the context of a new word/phrase that is inserted called the replacement entity (RE).

SEMANTIC TEXT EXCHANGE (STE) 

It learns a label-conditioned generator by finetuning GPT-2 on the training data, using this to generate candidate examples per class. A classifier trained on the original training set is then used to select top k candidate examples that confidently belong to the respective class for augmentation.

LAMBADA

Not Enough Data? Deep Learning to the Rescue!

In natural language processing (NLP), model-based data augmentation techniques customized for downstream tasks can have strong positive effects on performance, but they are often difficult to develop and utilize.

Learn Before

Related