Facebook's Slow Video Classifier AI

2019-11-05

Retinal ganglion cells in primates receive visual information from photoreceptors and transmit it to the brain. However, not all eye cells possess this sophisticated ability. Scientists have discovered that 80% of cells can only work at low frequencies and recognize subtle details, while the remaining 20% can respond to rapid changes. This biological dichotomy has inspired scientists at Facebook AI Research to pursue what they call "Slow AI." They have developed a framework called SlowFast, which is an implementation of the PyTorch framework and can be used on GitHub. It also includes pre-trained models. SlowFast is a machine learning architecture for video recognition, claiming to have "powerful performance" in action classification and shot detection. In a preprint paper, the research team points out that slow motion is statistically more common than fast motion and can slowly refresh the recognition of semantics such as color, texture, and lighting without compromising accuracy. On the other hand, slow motion develops faster than subject identification and can analyze executed actions at high temporal resolution. There are two ways to implement the "slow" effect. One approach runs at a lower frame rate and slower refresh rate, optimizing the capture of information from a small number of images or sparse frames. The other approach captures fast-changing motion with a fast refresh rate and high temporal resolution. The researchers assert that by processing the original video at different time rates, the slower path is better at recognizing static regions that are invariant or change slowly within the framework, while the faster path can accurately infer operations in dynamic regions. The information from these two paths is fused, and the data from the fast path enters the slow path through lateral connections in the network, allowing the slow path to perceive the results from the fast path and send them to a classification layer. To evaluate the performance of SlowFast, the team tested the model on two popular datasets: DeepMind's Kinetics-400 and Google's AVA. The former consists of thousands of 10-second short videos from YouTube, with at least 400 videos containing different categories of human behavior. AVA includes 430 annotated 15-minute YouTube videos and 80 annotated visual actions. SlowFast achieved the best performance on both datasets, with a median average precision (mAP) of 28.3. The researchers conclude in the paper, "We hope that the concept of 'slow' can promote further research in video recognition... We have demonstrated that the fast path can become very lightweight by reducing channel capacity while still learning useful temporal information for video recognition. The timeline is a special dimension, and this paper mainly compares the speed along this axis, studying an architectural design."

RECENT AI NEWS

RECENT AI TOOLS