MIT Develops More Effective Deep Video Recognition Model

2019-10-18

One branch of machine learning called deep learning has already helped computers surpass humans in visual tasks such as medical imaging scans. However, as technology advances, the number of videos that it can recognize is increasing, resulting in larger models and greater computational requirements.


It is estimated that training a video recognition model can require up to 50 times more data than training an image classification model, while the processing power only increases by 8 times. This poses a problem as the demand for processing power to train deep learning models continues to grow exponentially. Running large video recognition models on many AI devices remains a challenge.Assistant Professor Song Han from the Department of Electrical Engineering and Computer Science (EECS) at MIT has addressed this issue by designing more efficient deep learning models. At the International Conference on Computer Vision, Song Han and MIT graduate student Jilin, along with MIT-IBM Watson AI Lab researcher Gan Zhuang, announced their research on compact video recognition models to improve the performance of models on smartphones and other smart devices. Their approach reduces the number of parameters from 150 million in the latest models to 25 million, shrinking the model to one-sixth of its original size.


Song Han said, "Our goal is to make AI accessible to anyone using smart devices. To achieve this, we need to design efficient AI models that consume less energy and can run smoothly on smart devices."


The decreasing cost of cameras and video editing software, along with the rise of new video streaming platforms, has flooded the internet with new content. Approximately 30,000 hours of new videos are uploaded to YouTube every hour. Researchers say that creating more efficient content classification tools will help viewers and advertisers find desired videos more quickly. Such tools will also enable institutions like hospitals to run AI applications locally, maintaining the privacy and security of sensitive data.The underlying image and video recognition models are neural networks that loosely model how the brain processes information. Whether it's digital photos or sequences of video images, neural networks search for patterns in pixels and present what they see in increasingly abstract forms. With enough examples, neural networks can learn to recognize people, objects, and the relationships between them.


Currently, top video recognition models use 3D convolutions to encode the passage of time in a sequence of images, but this increases the computational requirements. To reduce the involved computations, Song Han and his colleagues designed an operation called "temporal shift" module, which moves the feature maps of selected video frames to their adjacent frames. By blending past, present, and future spatial representations, the model can perceive the passage of time without explicitly representing it.


The result is that the model performs better than peers in recognizing actions in the Something-Something video dataset, ranking first in versions 1 and 2 in recent public rankings. The online version of the shift module is also fast enough for real-time action recognition.Typically, training such a powerful model on a machine with only one graphics processing unit takes about two days. However, the researchers managed to borrow the power of the US Department of Energy's Summit supercomputer, currently the fastest on Earth. They demonstrated that with the immense capabilities of Summit, the model can be trained using 1,436 graphics processing units in just 14 minutes, approaching its theoretical limit. They claim that this is three times faster than state-of-the-art 3D models.


Dario Gil, Director of IBM Research, said, "The computational requirements for large-scale AI training work double every 3.5 months. Whether we can continue to push the boundaries of technology depends on our ability to match ultra-efficient algorithms with powerful machines."

RECENT AI NEWS

RECENT AI TOOLS