MIT Researchers Found an Automated Way to Understand How Neural Networks Work

Researchers at MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) have found a way to understand – in a fully automated way – how neural networks work.

Neural networks learn how to do computational tasks after being trained to large sets of data. They are credited for today’s best artificial intelligence systems, including speech recognition systems, automatic translators and self-driving cars. Once these neural networks have been trained, even their developers themselves have no idea how these neural networks do their tasks – what data components they are processing and in what way.

Two years ago, a team of computer-vision researchers at MIT was able to describe a method on how these neural networks work after they are trained to identify visual scenes. The method found by the team, however, has a loophole: it requires data to be sent to human reviewers. Some neural networks are simply too large and cost-prohibitive for human reviewers.

A new study by MIT researchers has found a method of understanding, in a fully automated way, how networks work after they are trained. In both studies, manual and automated version, MIT researchers trained the neural networks to perform computer vision tasks. The researchers found out which individual “nodes” fired in response to different input images.

Neural networks are loosely patterned after the human nervous system – a system composed of a large number but connected information-processing “nodes”. Similar to human’s neurons, a node of an artificial neural network receives information signals from its neighbors and either fire – release its own signal – or does not fire at all.

The results of this new MIT study may have significant implications for understanding the inner-workings of the human brain. Another research result has shown that individual neurons in the brain fire in response to specific visual stimuli, such as celebrity images, with neural networks firing a combination of nodes upon a specific stimulus.