Microsoft's AI can rewrite sentences based on context

2019-10-22

Contextual modeling defines the structure and maintenance of contextual data, playing a crucial role in open-domain conversations. Recently, Microsoft researchers have developed a novel approach that rewrites the last sentence in a series of utterances by considering the historical context data. In a pre-published paper, they provide a detailed account of their work on unsupervised context rewriting for open-domain dialogues. They claim that empirical results demonstrate that their new method has achieved state-of-the-art baselines in terms of rewriting quality and multi-turn response generation. As explained by the researchers, conversation context poses challenges that do not exist in sentence modeling, such as topic transitions and co-references. Most systems address these issues by adding keywords in the last paragraph or using AI models to learn numerical representations. However, both methods have their limitations, such as the inability to select the correct keywords or handle longer contexts effectively. The Microsoft team's approach involves rephrasing the last sentence in the conversation by considering contextual information and generating an independent utterance that is neither self-referential nor dependent on previous utterances. This fundamentally avoids problems like topic transitions, co-references, and inter-sentence dependencies. The researchers designed a machine learning system called CRN to achieve end-to-end automation. It consists of a sequence-to-sequence model that maps fixed-length utterances to fixed-length rewritten sentences. The system utilizes an independent attention mechanism to focus on different words in the last sentence, helping the system directly copy words from the context. To train the model, the team initially used pseudo data generated by inserting keywords extracted from the context into the original last utterance. Then, to influence the final response during the rewriting process, they employed reinforcement learning techniques with reward mechanisms to drive the system towards the desired outcome. In a series of experiments, the team evaluated their method based on rewriting quality, multi-turn response generation, multi-turn response selection, and end-to-end retrieval tasks. They observed that the model occasionally becomes unstable after reinforcement learning, tending to extract more words from the context, which significantly increases the diversity of utterances and consequently raises the difficulty of the task. The Microsoft team believes that their work represents a step towards more interpretable and controllable contextual modeling. Explicit context rewriting results are easier to debug and analyze. In the pre-published paper, the researchers state, "Our model can extract important keywords from complex contexts and insert them into the final utterance. This not only facilitates control and explanation but also helps convey information directly to the last utterance."

RECENT AI NEWS

RECENT AI TOOLS