The Future of Artificial Intelligence is Vast and Enormous

ChatGPT has attracted people's attention, and at the same time, people are beginning to explore how generative artificial intelligence can play a role in our daily lives. On the backend, scientists are constantly advancing the potential applications of artificial intelligence, which cover various aspects of our lives.

In a recent study published in the Journal of Machine Learning Research, Dr. Fenglei Fan, an assistant professor of mathematics at the Chinese University of Hong Kong, Dr. Rongjie Lai, a professor in the Department of Mathematics at Purdue University, and Dr. Ge Wang, the Clark & Crossan Endowed Chair Professor, discovered how analyzing the topological structure of artificial neural networks reveals the best way to harness the power of artificial intelligence in the future.

Like a topological map, there are three dimensions that power artificial intelligence. ChatGPT, a hot topic in the field of artificial intelligence, is a deep neural network with multiple layers, also known as a deep learning algorithm. Wang and his collaborators found that network width, referring to the number of neurons in a layer, also plays a crucial role.

Interestingly, they discovered that one type of network can be transformed into another type to accomplish a given task, such as regression or classification, which is a key element of machine learning. (Machine learning is a subset of artificial intelligence that allows computers to make predictions without explicit instructions.) In other words, deep neural networks can be transformed into wide neural networks, and vice versa.

"In the early days of this technology, scientists focused on very wide and shallow networks (one to two layers) for universal approximation," Wang said. "Later, deep neural networks (working with many layers in a feedforward manner) were proven to be very powerful. However, we don't fully believe that the emphasis should only be on deep learning. We think depth is just one dimension, and width is another dimension, and both need to be considered and balanced."

In their study, the research team considered the relationship between depth and width in neural networks. Through quantitative analysis, they found that deep and wide networks can be transformed back and forth on a continuous spectrum. Using both can provide a more comprehensive perspective and avoid biases.

Their research implies the future of machine learning, where networks have both depth and width, and possess good dynamics and optimized width-to-depth ratios. Networks will become increasingly complex, and when they reach the desired state dynamically, they will produce astonishing results.

"It's like playing with LEGO bricks," Wang said. "You can build a very tall skyscraper, or you can build a flat, large building with many rooms on the same level. For networks, the number of neurons and their interconnections are the most important. In 3D space, neurons can be arranged in various ways. It's similar to the structure of our brain. Neurons just need to be connected in various ways to accomplish different tasks."

Lai said, "Understanding the conversion between depth and width in neural networks is still a dynamic and evolving field." "Both wide and deep networks have their unique advantages and disadvantages. In general, shallow networks are easier to understand. Our exploration of the inherent symmetry in these two network types reveals a new perspective for understanding deep networks through wide networks."

Dr. Shekhar Garde, the Dean of the Rensselaer School of Engineering, said, "Dr. Wang's research on the relationship between width and depth in neural networks opens up new avenues for harnessing the potential of artificial intelligence." "Artificial intelligence is impacting almost every aspect of our society, from medicine to new materials to finance. This is an exciting moment for the field, and Dr. Wang is at the forefront of this question."