More Gemini Features Coming to Google TV
Google TV integrates Gemini AI for image/video generation, enhances Google Photos, and adds a dedicated YouTube Shorts feed to the homepage.
Read moreArtificial neural networks are ubiquitous machine learning models that can perform many tasks through training. They are called artificial neural networks because their architecture is inspired by the way biological neurons process information in the human brain.
About six years ago, scientists discovered a new and more powerful neural network model called "transformer". These models can achieve unprecedented performance, such as generating text from prompts with near-human accuracy. For example, transformer is the basis of artificial intelligence systems like ChatGPT and Bard. While transformers are highly effective, they are also mysterious: unlike other brain-inspired neural network models, it is still unclear how to build them using biological components.
Now, researchers from MIT, MIT-IBM Watson AI Lab, and Harvard Medical School have proposed a hypothesis that explains how transformers can be built using biological elements in the brain. They believe that a biological network composed of neurons and other brain cells called astrocytes can perform the same core computations as transformers.
Recent studies have shown that astrocytes, which are non-neuronal cells abundant in the brain, communicate with neurons and play a role in physiological processes such as regulating blood flow. However, scientists still have a limited understanding of the computational role of these cells.
In a new study published this week in the Proceedings of the National Academy of Sciences, the researchers explored the role of astrocytes in the brain from a computational perspective and developed a mathematical model that demonstrates how astrocytes, together with neurons, can build a biologically plausible transformer.
Their hypothesis provides inspiration for future neuroscience research on how the human brain works. At the same time, it can help machine learning researchers explain why transformers are so successful in various complex tasks.
"The brain is even more superior than the best artificial neural networks we have developed, but we don't really know how the brain works. Thinking about the connection between biological organs and large-scale artificial intelligence networks has scientific value," said Dmitry Krotov, a researcher at MIT-IBM Watson AI Lab and senior author of the study.
The main authors of this paper, co-written with Krotov, are Leo Kozachkov, a postdoctoral researcher in the Department of Brain and Cognitive Sciences at MIT, and Ksenia V. Kastanenka, an assistant professor of neurobiology at Harvard Medical School and assistant researcher at Massachusetts General Research Institute.
The operation of transformers is different from other neural network models. For example, recurrent neural networks trained in natural language processing compare each word in a sentence with an internal state determined by the previous word. However, transformers compare all words in a sentence simultaneously to generate predictions, a process known as self-attention.
Krotov explained that in order for self-attention to work, transformers must store all words in some form of memory, which seems biologically impossible due to the way neurons communicate.
However, a few years ago, scientists studying a slightly different type of machine learning model called dense associative memory realized that this self-attention mechanism might occur in the brain, provided that there is communication between at least three neurons.
"The number '3' really caught my attention because in neuroscience, these cells called astrocytes are not neurons, but they form tripartite synapses, or so-called three-way synapses, with neurons," said Kozachkov.
When two neurons communicate, a presynaptic neuron transmits a chemical called neurotransmitter to the postsynaptic neuron connected to it by the synapse. Sometimes, astrocytes also connect together - they wrap around a long and thin process around the synapse, forming a tripartite synapse. An astrocyte can form millions of tripartite synapses.
Astrocytes collect some of the neurotransmitters flowing through the synapse. To some extent, astrocytes can send signals to neurons. Because astrocytes operate on a much longer timescale than neurons - they generate signals by slowly increasing and then decreasing calcium response - these cells can store and integrate information transmitted by neurons. Krotov said that in this way, astrocytes can form a memory buffer.
"If you consider it from this perspective, astrocytes are very natural for the precise calculations we need to perform attention operations within transformers," he added.
With this insight, the researchers formed their hypothesis that astrocytes can play a role in the computations of transformers. They then proceeded to build a mathematical model of a neuron-astrocyte network that would operate like a transformer.
They adopted the core mathematics that make up transformers and developed simple biophysical models of astrocytes and neurons communicating in the brain based on in-depth literature research and guidance from collaborating neuroscientists.
They then combined these models in a way that led to equations describing the neuron-astrocyte network's self-attention, similar to transformers.
"Sometimes, we find that certain things we want to be real cannot be reasonably implemented. So, we have to think of some workarounds. There are things in the paper that are very close to the structure of transformers and can be matched with them in a biologically plausible way," said Kozachkov.
Through their analysis, the researchers demonstrated that their biophysical neuron-astrocyte network theoretically matches transformers. Additionally, they conducted numerical simulations by inputting images and text paragraphs into the transformer model and comparing their responses with those of the simulated neuron-astrocyte network. Both responded in similar ways to prompts, confirming their theoretical model.
The researchers' next step is to bridge the gap between theory and practice. They hope to compare the predictions of their model with observations from biological experiments and use this knowledge to refine or possibly overturn their hypothesis.
Furthermore, one significance of their research is that astrocytes may be involved in long-term memory, as the network needs to store information for future actions. Krotov said that further research can further explore this idea.
"For many reasons, astrocytes are extremely important for cognition and behavior, and their operation is fundamentally different from neurons. My biggest hope for this paper is that it catalyzes a series of studies on glial cells, especially astrocytes, in the field of computational neuroscience," added Kozachkov.
Google TV integrates Gemini AI for image/video generation, enhances Google Photos, and adds a dedicated YouTube Shorts feed to the homepage.
Read moreAlphabet Q1 paid subscribers reached 350M, fueled by YouTube & Google One. YouTube ad revenue slightly missed at $9.88B amid Premium growth. Total Q1 revenue hit $109.9B, beating forecasts.
Read moreAnthropic releases Claude connectors for Adobe, Blender, and Ableton, enabling AI workflow automation and app control. It also sponsors the Blender Development Foundation.
Read moreApple introduces an App Store model splitting 12-month plans into monthly installments. On iOS 26.4+, it lowers upfront costs without reducing total price, adding billing tracking.
Read moreGoogle launches Aadhaar digital IDs in Wallet for India, enabling secure QR/NFC verification. A phased partner rollout targets key sectors, while passport-based credentials expand globally.
Read moreYouTube tests "Ask YouTube," an AI tool blending text and video for complex queries. Initially for US Premium users, it supports interactive follow-ups, expanding Google's AI search.
Read moreGoogle grants US DoD classified AI access, joining OpenAI and xAI in defense deals amid Anthropic's dispute. Contractual safeguards against surveillance and lethal weapons remain unenforceable.
Read moreMistral AI launches Workflows, a Temporal-powered orchestration platform for secure, Python-driven enterprise AI automation featuring human-in-the-loop approvals.
Read moreCustom subliminal audio maker

AI Social Media Manager

AI Search Visibility

Workout and Calorie Tracker
Stock research platform

AI interview assistant

Scientific figure generator

AI courses with real code