"IBM's Lambada AI Generates Training Data for Text Classifier"

2019-11-15

If data scientists lack sufficient data to train machine learning models, what should they do?


IBM Research's researchers advocate the use of synthetic data in a newly published paper. They use pre-trained machine learning models to artificially generate new labeled data for text classification tasks. They state that this method, called Language Model-based Data Augmentation (Lambada), can improve the performance of classifiers on various datasets and significantly enhance the level of the latest data augmentation techniques. Currently, finding a suitable classifier model may require a large amount of labeled data. However, in many cases, especially when developing AI systems for specific applications, labeled data is scarce and costly to obtain.


The researchers point out that generating synthetic training data in the text domain is more challenging than in the visual domain because the transformations used in simpler methods often distort the text, making it grammatically and semantically incorrect. Therefore, most text data augmentation techniques (including the one detailed in this paper) involve replacing individual words with synonyms, deleting words, or changing the word order.


Lambada utilizes a generative model (OpenAI's GPT) to pre-train on large-scale text, enabling it to capture language structure and generate coherent sentences. The researchers fine-tuned their model on existing small datasets and used the fine-tuned model to synthesize new labeled sentences. Correspondingly, they trained classifiers on the aforementioned datasets and filtered them, retaining only the data that appeared "qualitative enough" before retraining on both the existing and synthesized data.To validate their approach, the researchers conducted experiments with three different classifiers: BERT, Support Vector Machine, and a Long Short-Term Memory network, on three datasets. They varied the training samples in each group. The discussed corpora included queries about flight-related information, open-domain and fact-based questions of several categories, and data from a telecommunications customer support system.


They report that Lambada statically improved the performance of all three classifiers on small datasets, partly due to its control over the number of samples for each category. They state that these controls allowed them to invest more time in generating samples for categories that were underrepresented in the original dataset.


IBM's researchers state, "Our augmentation framework does not require additional unlabeled data and achieves higher accuracy for most classifiers compared to simple weak labeling methods. Clearly, the generated dataset contributes more to improving classifier accuracy than samples obtained from the original dataset."

RECENT AI NEWS

RECENT AI TOOLS