Google Brain's AI Achieves State-of-the-Art Text Summarization功能

2019-12-24

Text summarization is a machine learning algorithm that is being improved, according to a recent paper published by Microsoft. The automatic summarization system can help employees reduce the time spent on reading information. Recently, a team from Google Brain and Imperial College London also developed a system that uses extracted blank sentences for abstract summarization before training. This system utilizes Google's transformer architecture and combines customized pre-training objectives for generating abstract text. They claim that this AI has achieved state-of-the-art results in 12 summarization tasks, covering news, science, stories, instructions, emails, patents, and legislative bills. It has demonstrated "astonishing" performance in low-resource summarization, surpassing the top performers on six datasets with just 1000 examples. As researchers pointed out, text summarization aims to generate accurate and concise summaries from input documents, rather than performing technical execution. Summaries are not simply copied fragments from the input, but may introduce new words or cover key information to maintain linguistic fluency. Transformers, introduced in the paper by Google Brain researchers, are a type of neural structure. Like all deep neural networks, they consist of interconnected layers that transmit signals from input data and slowly adjust the synaptic strength (weights) of each connection. This is how all AI models extract features and learn to make predictions. What sets transformers apart is that each output element is connected to each input element and the weights between them are dynamically calculated. The team designed a training task where important sentences in the document were masked. The AI had to fill in the blanks using the context and news articles. The research group selected their best-performing model with 5.68 billion parameters and trained it on 750GB of text extracted from 350 million web pages. According to the researchers, the new AI model achieves high language quality in terms of fluency and coherence, without the need for measures to mitigate fluency issues. Furthermore, the quality of the summaries generated in a low-resource environment with only 100 example articles is comparable to models trained on complete datasets of 20,000 to 200,000 articles.

RECENT AI NEWS

RECENT AI TOOLS