MIT team shows neural networks can build their own learning machinery

Self-organization as a measure of biological plausibility

The paper comes from scientists at MIT's McGovern Institute for Brain Research and the K. Lisa Yang Integrative Computational Neuroscience (ICoN) Center. They argue that a crucial metric for the biological plausibility of an artificial neural network's approach to learning is its ability to organize itself, without the guiding hand of a developer.

Qianli Liao, a K. Lisa Yang ICoN Center Postdoctoral Fellow and co-first author of the paper, says current AI works very well but does not necessarily use a learning algorithm that the brain uses. Engineers design the rules for how connections change as a network processes training data and receives feedback about its mistakes. That approach is powerful for training AI, but Liao and his colleagues caution that such networks may not offer much insight into biological learning.

"The brain doesn't have an external designer or architect," Liao says. "The brain has to organize itself." According to the team, a network's value to neuroscience should be judged by how far it can organize itself, including not only its connections but also the machinery that adjusts them.

How the network works

Liao, co-first authors Liu Ziyin and Yulu Gan, and MIT professors Tomaso Poggio and Mark Harnett developed a self-organizing network that can learn. Its structure begins with multiple layers of randomly interconnected neuron-like units. Within each layer, every unit follows the same rules about how connections should be adjusted.

The rules are of the kind thought to rewire brain circuits as animals learn. One example is that "neurons that fire together wire together": when two connected neurons are active at the same time, the connection between them strengthens.

The team showed that a network governed by such rules can learn to recognize patterns and classify images. In their model, learning is enabled by self-assembling motifs that emerge from simple rules and then serve as building blocks for functional circuits. As the network learned, these motifs reorganized themselves using an optimization strategy similar to gradient descent, in which connections are gradually adjusted in response to errors.

Comparison with back-propagation

Most modern AI implements gradient descent through some form of the back-propagation algorithm, which neuroscientists say is unlikely to be used by the brain. The researchers compared their self-organizing network with networks designed to learn that way and found that it performed at a similar level.

Poggio says that finding a biological parallel to back-propagation could trigger significant progress in both AI and neuroscience. "If the synaptic motifs we propose are, in fact, used by the brain to do what backpropagation does for artificial neural networks, then we will have a strong connection at the level of fundamental mechanisms between artificial and natural networks," he says.

What it could mean

Liao stresses that a self-organizing network does not necessarily learn the way the brain does, but he calls it a critical starting point. "Self-organization is the first high-level framework," he says.

Understanding the rules that reshape neural connections matters for neuroscience, since they may differ across brain regions or change with maturity, and may be disrupted by aging or disease. The article describes knowing these rules as a vital step toward new therapies and prevention strategies. Liao points to brain-computer interfaces, which aim to restore communication when language abilities are lost or let paralyzed patients control neuroprosthetic limbs, as one potential application.

The group's main goal is a better understanding of the brain, but Liao says that knowledge could ultimately help engineers build machine learning networks that perform better or in more human-like ways.