Reduce Energy Consumption of Artificial Intelligence

2020-01-06

The operation of the human brain follows a strategy of "growth and pruning". Initially, it starts with a large number of neural connections and then prunes the unused connections over time. Recently, a research team in the field of artificial intelligence applied this approach to AI systems and found that it significantly reduces the energy required for training AI. A group of researchers at Princeton University has developed a new method for training AI systems. This method achieves industry standards while consuming less computational power and energy, making it more environmentally friendly than traditional machine learning models. In two papers, the researchers demonstrated how to construct networks by increasing neurons and connections, and then pruning the unused connections over time, leaving only the most efficient parts of the model. The model developed by the researchers follows a "grow-prune" pattern. Niraj Jha, a professor of electrical engineering at Princeton University, explained that the human brain is most complex at around the age of three, after which it starts pruning unnecessary synaptic connections. As a result, a fully developed brain can perform complex tasks while using only about half of its peak synaptic connections. The researchers strengthened AI training by mimicking this process. Thanks to this "growth and pruning" technique, the same good predictions can be made on patterns in the data using only a small fraction of the computational power previously required. The researchers aim to find methods to reduce energy consumption and computational costs, as this is crucial for bringing machine learning to small devices such as smartphones and smartwatches. Additionally, reducing the energy consumption of machine learning algorithms can help the industry reduce its carbon footprint. The first author of the paper added, "Models need to be trained locally because transferring them to the cloud requires a significant amount of energy." During the development process, the researchers attempted to create a neural network creation tool, called Neural Synthesis Tool (NeST), to design and create high-performing networks from scratch. This tool quickly increases complexity by adding more neurons to the network when provided with only a few neurons and connections. Once the network reaches the desired benchmark, it starts self-pruning over time. Traditional network models also use some pruning techniques, but what sets Princeton University's method apart is that it simulates the developmental stages from "infant brain" to "child brain" to "adult brain".

RECENT AI NEWS

RECENT AI TOOLS