Relationship between DeepMind's novel AI training technique and the brain

2020-01-29

DeepMind recently published a paper that elaborates on a newly developed reinforcement learning technique, which has the potential to explain how reward pathways in the human brain operate. According to New Scientist, this machine learning training method is called "distributional reinforcement learning," and its underlying mechanism appears to provide a reasonable explanation for how brain neurons release dopamine. Neuroscience and computer science have a long history. As early as 1951, Marvin Minsky created a computer program that could solve mazes using a reward system. Minsky drew inspiration from the research of physiologist Ivan Pavlov, who demonstrated that dogs can learn through a series of reward and punishment rules. DeepMind's new paper delves into the application of reinforcement learning to gain a deeper understanding of how dopamine neurons may function, thus adding new content to the intertwined history of neuroscience and computer science. Whenever a person or animal is about to perform an action, a group of neurons in the brain responsible for dopamine release predicts the reward for this behavior. Once the action is executed and the outcome (reward) becomes apparent, the brain releases dopamine. However, the release of dopamine is measured based on the magnitude of prediction errors. If the reward is greater or better than expected, it triggers a stronger dopamine surge. Conversely, poorer rewards lead to a decrease in dopamine release. This is very similar to how reinforcement learning algorithms operate. In 2017, DeepMind researchers released an enhanced version of a commonly used reinforcement learning algorithm, which improved the performance of many reinforcement learning tasks. The DeepMind team believes that the mechanism behind the new algorithm can be used to better explain how dopamine neurons function in the human brain. To study the workings of dopamine neurons, DeepMind collaborated with Harvard University to investigate the activity of dopamine neurons in mice. The researchers had the mice perform various tasks and rewarded them based on a dice-rolling method, while recording how their dopamine neurons were activated. Different neurons seemed to predict different potential outcomes and released varying amounts of dopamine. Some neurons predicted rewards lower than the actual rewards, while others predicted rewards higher than the actual rewards. After plotting the distribution of predicted rewards, the researchers found that the predicted distribution closely matched the actual reward distribution. This suggests that the brain does indeed utilize a distributional system when making predictions and adjusting them to better match reality. This study can provide insights for both neuroscience and computer science, supporting the use of distributional reinforcement learning as a method to create more advanced artificial intelligence models. Additionally, it may also have implications for our understanding of how the reward system in our brain operates.

RECENT AI NEWS

RECENT AI TOOLS