Google Releases Latest SGD New Corpus to Help Smart Assistants Become Smarter

2019-10-29

Google researchers recently conducted a study aiming to enhance the support for new services provided by artificial intelligence assistants like Google Assistant, without the need for additional data or retraining. The study introduced a method that allows models to be used across services without domain-specific parameters. Additionally, the team released the SGD corpus, claiming it to be the world's largest publicly available assembly for task-oriented dialogue.

In a blog post, software engineer Abhinav Rastogi and Google research engineering lead Pranav Khaitan wrote, "Today's virtual assistants can help users with a variety of tasks, including finding flights, searching for nearby events or movies, making reservations, and retrieving information from the web. However, despite significant progress, adaptability challenges are often overlooked in state-of-the-art models. The specific reason is the lack of appropriate datasets that match the scale and complexity faced by virtual assistants."

To address this, the SGD corpus consists of over 18,000 annotated dialogues between individuals and virtual assistants, involving interactions with services across 17 domains. For most domains, the dataset includes multiple APIs, many of which have overlapping functionalities, but different interfaces reflecting typical real-world scenarios. The evaluation set includes services not present in the training set, primarily used to quantify the robustness of models to API changes or additions.

As for the aforementioned pattern-guided approach, it leverages the natural language descriptions of each service or API and their relevant attributes to learn distributed semantic representations. These representations serve as additional inputs to the dialogue system and can be implemented as a single model.

The team states that the unified model is the core of Google's open-source dialogue state tracking model, facilitating the representation of common knowledge between similar concepts across different services, making it possible to operate on new services not present in the training data. Rastogi and Khaitan conclude in their article, "We believe that this corpus dataset will serve as a good benchmark for building large-scale dialogue models, and we are excited and look forward to the research community utilizing it in innovative ways to advance dialogue technology."

The dataset is a crucial factor in training AI models, providing them with rich knowledge and experience. The larger the dataset, the more information it contains, and the more it aids the model. Google openly states that the release of the new model and dataset, the SGD corpus, will significantly advance natural language system modeling technology.

RECENT AI NEWS

RECENT AI TOOLS