Differences Between Deep Neural Networks and Human Perception

2019-12-16

When your mother calls your name, no matter how loud or soft, you can recognize her voice. When you see her face, even if she is far away and the lighting is poor, you know it's her face. This robustness to variations is a hallmark of human perception. On the other hand, we are prone to illusions: we may not be able to distinguish certain sounds or images, even though they are actually different. Scientists explain this as a lack of comprehensive understanding of the invariance of our auditory and visual systems. Deep neural networks can perform tasks such as speech recognition and image classification, exhibiting a high level of robustness to changes in auditory or visual stimuli, similar to the human perceptual system. However, a group of researchers from MIT has discovered that they are actually different. They presented their findings at the 2019 Conference on Neural Information Processing Systems. In their study, the researchers randomly selected natural images and speech segments from a standard database, and then synthesized sounds and images. The deep neural networks were able to classify these physically distinct stimuli, rather than being classified by humans. This represents a new approach to meta-modeling, which generalizes the concept of transforming the role of computer models into human perceivers. Therefore, they referred to these synthesized stimuli as "meta-models of paired natural stimuli". The researchers then tested whether humans could recognize words and images. In the experiment, participants listened to a short speech and had to identify the words that appeared in the accompanying video from a set of words. This task was easy for natural audio, but difficult for many modeled audio. The reason is simple: humans do not categorize synthesized stimuli and the word "bird" or the image of a bird in the same category. In fact, the meta-models generated to match the deepest responses of the models are often unrecognizable as words or images to human subjects. The basic logic is that if we have a good human perception model, such as speech recognition, and we select two models that say the same sound and present these two sounds to a human audience, they would perceive the sounds as the same. If the testers perceive the stimuli as different, it clearly indicates a mismatch between our model's performance and human perception. The importance of this work lies in improving perception models beyond deep networks. Furthermore, the team found methods to modify the model network to generate meta-models of sounds and images that are more reasonable for humans. They stated, "This gives us hope that we can eventually develop models that better capture invariance through tasks like conditional colorization."

RECENT AI NEWS

RECENT AI TOOLS