Check out the Latest AI Algorithms Released by Major Companies at NeurIPS 2019

2019-12-18

Nvidia: From 2D to 3D


Nvidia has proposed a reverse rendering framework that predicts 3D information about objects based on 2D photos and videos.


Nvidia's AI team has built a reverse renderer, which is crucial for machine learning as it can produce different results. Rendering is a well-known process in computer graphics where 3D data generates 2D projections of objects and scenes. The goal here is to do the opposite: from 2D to 3D.Nvidia's algorithm, called DIB-R (Differentiable Interpolation-based Renderer), is the "most sophisticated algorithm to date." It can predict the geometry, color, and texture of objects and infer lighting in the scene.


Professor Sanja Fidler, the author of the EETimes paper and head of Nvidia's AI, said, "Since all the parameters are computed analytically, it can perform this task very efficiently, which is crucial for machine learning. Older techniques in this field relied on hand-designed gradients and approximate gradients, making them difficult to train."


Training Nvidia's model on a single V100 GPU takes two days, while inference takes less than 100 milliseconds. The model creates 3D objects by continuously modifying a polygonal sphere.



Intel: Teaching Robot Teams to Collaborate


Researchers from Intel, Oregon State University, and the University of California have published a paper on a novel reinforcement learning technique that can teach robot teams to collaborate.


In reinforcement learning, two AI agents (algorithms that perform certain actions) work together to improve their overall performance. One agent generates results, while the other evaluates those results, rewarding higher scores for results that closely match the expected outcome.The new technique, called MERL (Multi-Agent Evolutionary Reinforcement Learning), builds upon Intel's existing CERL (Cooperative Evolutionary Reinforcement Learning) research, enabling agents to learn challenging continuous control problems, such as training humanoid robots from scratch.


MERL can train robot teams to collectively solve tasks. This is a challenging problem because the individual agents' goals may differ from the overall team goal. Individual goals are often more dense, meaning they are achieved faster and easier, making them more attractive. Training agents to simultaneously strive for both sets of outcomes is particularly difficult.


The team is also researching similar problems involving multitask learning without explicit reward feedback and the role of communication in these tasks.



IBM: Training with 8-bit Data


One of IBM's submitted papers focuses on low-precision training techniques. While low-precision data is typically used for inference, using higher-precision numbers during training can maintain the accuracy of the resulting models' weights. However, switching to lower-precision numbers (a process called quantization) is attractive because it requires less computation, storage, and power.Last year, IBM showcased its initial 8-bit training techniques, including an 8-bit floating-point scheme, at NeurIPS. This year, it has proposed a hybrid approach (Hybrid 8-bit Floating-Point or HFP8) aimed at improving computational performance while maintaining accuracy for the most challenging state-of-the-art deep neural networks.


This work improves upon last year's developments as the new format, HFP8, overcomes the loss of training accuracy previously caused by quantization. HFP8 uses a novel FP8 format in the forward path for higher resolution and another FP8 format in the backward path for a larger range of gradients.



Google: Training Massive Networks


As the scale of deep neural networks (DNNs) continues to increase, they are pushing the limits of memory in processing chips, including Google's own Tensor Processing Units (TPUs). Multiple accelerators can be used, but the sequential nature of DNNs means that only one accelerator is active at any given time, rendering it inefficient. Data parallelism can be employed, where different subsets of data on each chip are used to train the same model, but this does not scale to larger models.Google's team has attempted to address this issue using pipeline parallelism. They have created GPipe, a new machine learning library that works on any neural network represented as a sequence of layers. The new library allows researchers to easily deploy more accelerators to train larger models and scale performance without adjusting hyperparameters.


GPipe divides the model to be trained among multiple accelerators, using a novel batch splitting pipeline algorithm to split batch data into smaller micro-batches. Researchers say this can lead to nearly linear speedup when model training is distributed across multiple accelerators.



Facebook: SuperGLUE


A research team from Facebook, New York University, the University of Washington, and DeepMind has been developing new benchmarks for algorithms that understand language. The benchmark, initially called GLUE, was based on a set of nine natural language processing (NLP) tasks, providing a single metric for model performance.Last year, algorithms for understanding language made significant progress, surpassing non-expert performance in some cases. As a result, GLUE needed to evolve. The team has updated their work to a new benchmark that includes more challenging NLP tasks, software tools, and a public leaderboard. They have renamed it SuperGLUE.


According to the latest benchmark scores, Google's T5 model currently leads with a score of 88.9 (close to the human baseline of 89.8). Facebook's RoBERTa model scores 84.6, and IBM's BERT-mtl model scores 73.5.

RECENT AI NEWS

RECENT AI TOOLS