"Computational Model Can Understand the World Like the Brain"

In order to live in this world, our brains must develop an intuitive understanding of the physical world around us and then use this understanding to interpret sensory information entering the brain.

How does the brain achieve this intuitive understanding? Many scientists believe it may be a process similar to "self-supervised learning". This type of machine learning was initially developed to create more efficient computer vision models, allowing the models to learn about visual scenes based solely on their similarities and differences, without the need for labels or other information.

Two studies conducted by researchers at the K. Lisa Yang Integrated Computing Neuroscience Center (ICoN) at MIT provide new evidence supporting this hypothesis. The researchers found that when they trained a model called a neural network using a specific type of self-supervised learning, the resulting model generated activity patterns that closely resembled the patterns observed in the brains of animals performing the same task.

The findings suggest that these models are able to learn representations of the physical world and use them for accurate predictions, similar to how the mammalian brain may be using the same strategy.

Aran Nayebi, a postdoctoral researcher at the ICoN center, is the lead author of one of the studies, with co-authors including Rishi Rajalingham (formerly a postdoctoral researcher at MIT, now working at Meta Reality Labs), as well as senior authors Mehrdad Jazayeri (Associate Professor of Brain and Cognitive Sciences, member of the McGovern Institute for Brain Research) and Robert Yang (Assistant Professor of Brain and Cognitive Sciences, associate member of the McGovern Institute for Brain Research).

The research was co-led by MIT graduate student Mikail Khona and former senior research assistant Rylan Schaeffer.

In these two new NeurIPS studies, the researchers aimed to explore whether computational models of other cognitive functions could also exhibit similarities to the mammalian brain. In Nayebi's study, the researchers trained a self-supervised model to predict the future states of natural videos depicting everyday scenes.

"For the past decade or so, the main approach in cognitive neuroscience to building neural network models has been to train these networks on individual cognitive tasks. But models trained in this way rarely generalize to other tasks," said Yang. "Here, we tested whether we could build models of certain cognitive aspects by first training them on natural data using self-supervised learning, and then evaluating them in a laboratory setting."

Once the models were trained, the researchers tested their ability to generalize to a task they called "mental ping pong". This task is similar to the game of ping pong, where players move a paddle to hit a ball moving on a screen. In the mental ping pong version, the ball disappears suddenly before hitting the paddle, so players must estimate the trajectory of the ball in order to hit it.

The researchers found that the model was able to accurately track the hidden trajectory of the ball, similar to the trajectory simulated by neurons in the mammalian brain. Additionally, the neural activation patterns in the model were similar to the activation patterns in the brain when animals play the game, particularly in a brain region called the posterior parietal cortex. The researchers noted that other categories of computational models were not able to come as close to the biological data.

"A lot of effort has been put into creating artificial intelligence," said Jazayeri. "The relevance of these models to neurobiology depends on their ability to capture additional workings of the brain. The fact that Aran's model predicts neural data is important because it suggests that we may be getting closer to building artificial systems that mimic natural intelligence."

The research led by Khona, Schaeffer, and Fiete focused on a specialized type of neuron called grid cells. These cells are located in the entorhinal cortex and help animals navigate, working in conjunction with place cells located in the hippocampus.

When an animal is in a specific location, place cells fire, and grid cells only fire when the animal is at the vertices of triangular grid patterns. Groups of grid cells create overlapping grids of different sizes, allowing them to encode a large number of locations using relatively few cells.

In a recent study, the researchers trained a supervised neural network to simulate the function of grid cells by predicting the animal's next position based on the starting point and velocity, a task known as path integration. However, these models relied on having privileged information about absolute space, which animals do not have.

Inspired by the intriguing encoding properties of grid cells in multi-cycle grid cell spatial coding, the MIT team trained a contrastive self-supervised model that could perform the same path integration task while efficiently representing space in the process. For training data, they used sequences of velocity inputs. The model learned to differentiate them based on the similarity of their positions, generating similar encodings for nearby positions but different encodings for more distant positions.

"This is similar to how models are trained to process images, where if both images are of a cat's head, their encodings should be similar, but if one image is of a cat's head and the other is of a truck, their encodings should be different," said Khona. "We took the same approach but applied it to spatial trajectories."

Once this model was trained, the researchers found that the activation patterns of internal nodes in the model formed several grid-like patterns with different periods, very similar to the patterns formed by grid cells in the brain.

"What excites me about this work is that it connects the theoretical properties of grid cell encoding with the computation of path integration. While the mathematical work is analytical, the method of optimizing encoding efficiency and achieving grid-like tuning through self-supervised learning is synthetic: it demonstrates the possible necessary and sufficient properties for explaining why the brain has grid cells," said Fiete.