Development of Artificial Neural Networks with Innate Abilities Similar to Animals

2020-01-10

Many animals are born with certain abilities. Horses can walk within hours of birth, ducks can swim shortly after hatching, and human babies cry as soon as they are born. Animal brains have evolved to possess certain skills without the need for learning, and many researchers hope to replicate this natural ability in artificial intelligence.


New research has found that artificial neural networks can evolve to perform tasks without the need for training, which could make AI better at handling various tasks such as tagging photos or driving cars.


An artificial neural network is a medium for information transmission between small computing elements (neurons). Networks typically learn to perform tasks such as playing games or recognizing images by adjusting the weights or strengths of connections between neurons. A technique called neural architecture search attempts to find networks with better learning capabilities for specific purposes by trying out various shapes and sizes. The new method uses the same search technique to find networks with lower weights. For such networks, their overall shape determines their intelligence, making them particularly suitable for certain tasks.This process starts with a set of very simple networks that link inputs (e.g., data from robot sensors) to behavioral outputs. It evaluates the network's performance on a given task by adding neurons, adding connections, or changing the sensitivity of neurons to the sum of their inputs, while keeping the network's performance optimal and subjecting it to mutations. During the evaluation phase, shared random numbers are assigned to all weights of the network (actually, a few random numbers are used and the results are averaged).


The result is called a Weight Agnostic Neural Network (WANN). These networks achieve excellent performance in tasks with simple requirements. While typical task networks in this study may have thousands of neurons and weights, WANNs have only a few neurons and one weight.


WANNs still perform remarkably well. The research team compared them to standard network architectures that refine weights based on experience and can accomplish three simulated tasks: driving a race car, making a robot walk, and controlling a wheeled cart to balance a pole.


WANNs scored about one-sixth to half the score of trained networks. When researchers assigned the best-performing weights instead of random weights, these numbers ranged from two-thirds to four-fifths of the score of trained networks. Furthermore, if WANNs are trained using the same methods as larger standard networks after evolution, their performance can be comparable.In tasks involving the recognition of handwritten digits, WANNs achieved an accuracy of over 90% (compared to 99% accuracy for larger networks trained for the task). The research was presented last month at the Neural Information Processing Systems (NeurIPS) conference in Vancouver, Canada.


The researchers say that this breakthrough was initially a loophole that assigned the same number to all weights, ultimately simplifying the architecture search.


While the performance of WANNs cannot replace that of well-trained large networks, this approach opens up a new avenue for finding network architectures that are specifically suited for various tasks, just as different parts of the brain are wired differently for specific purposes. For example, convolutional neural networks have architectures suited for image recognition, reflecting the structure of the brain's visual cortex. They believe that there are more building blocks that can make AI intelligent from the start.

RECENT AI NEWS

RECENT AI TOOLS