New method for comparing neural networks reveals how artificial intelligence works

A team at Los Alamos National Laboratory has developed a new approach to comparing neural networks that looks inside the “black box” of artificial intelligence to help researchers understand neural network behavior. Neural networks recognize patterns in data sets; they are used throughout society, in applications such as virtual assistants, facial recognition systems and self-driving cars.

“The AI ​​research community doesn’t necessarily have a complete understanding of what neural networks do; they give us good results, but we don’t know how or why,” said Haydn Jones, a researcher in the Advanced Research in Cyber ​​Systems group at Los Alamos. “Our new method does a better job of comparing neural networks, which is a crucial step toward a better understanding of the mathematics behind AI.”

Jones is the lead author of the paper “If you’ve trained one, you’ve trained them all: similarity between architectures increases with robustness,” presented recently at the Uncertainty in Artificial Intelligence conference. In addition to studying network similarity, the paper is a crucial step towards characterizing the behavior of robust neural networks.

Neural networks are high performing but fragile. For example, self-driving cars use neural networks to detect signs. When conditions are ideal, they do this quite well. However, the smallest aberration – such as a sticker on a stop sign – can cause the neural network to misidentify the sign and never stop.

To improve neural networks, researchers are looking at ways to improve the robustness of the network. A state-of-the-art approach involves “attacking” networks during their training process. Scientists intentionally introduce aberrations and train the AI ​​to ignore them. This process is called adversarial training and essentially makes it harder to fool the networks.

Jones, Los Alamos collaborators Jacob Springer and Garrett Kenyon, and Jones’ mentor Juston Moore applied their new network similarity metric to resiliently trained neural networks and surprisingly found that resilient training causes neural networks in the computer vision domain to converge to very similar data representations, regardless of network architecture , as the scope of the attack increases.

“We found that when we train neural networks to be robust against adversarial attacks, they start doing the same things,” Jones said.

There has been extensive effort in industry and in the academic community to search for the “right architecture” for neural networks, but the Los Alamos team’s findings indicate that the introduction of adversarial training narrows this search space significantly. As a result, the AI ​​research community may not need to spend as much time exploring new architectures, knowing that resilient training causes different architectures to converge to similar solutions.

“By finding that robust neural networks are similar, we make it easier to understand how robust AI can really work. We may even reveal hints about how perception arises in humans and other animals,” Jones said.

More information:
Haydn T. Jones et al., If You’ve Trained One, You’ve Trained Them All: Architecture Similarity Increases with Robustness, (2022)