Is Deep Learning Necessary for Artificial Intelligence?

By MIKE RANKER

Recent research demonstrates that brain-inspired shallow feedforward networks are capable of efficiently learning non-trivial classification tasks with less computational complexity, compared to deep learning architectures. This is in addition to previous experimental findings on sub-dendritic adaptation and anisotropic properties of neurons.

Deep learning appears to be a key magical element for many artificial intelligence tasks. However, these tasks may be easily accomplished via simpler shallow architectures.

According to new scientific findings, shallow feedforward networks are capable of efficiently learning non-trivial classification tasks with less computational complexity than deep learning architectures.

Perceptron, the earliest artificial neural network, was established about 65 years ago and consisted of only one layer. However, more sophisticated neural network architectures requiring many feedforward (consecutive) layers were added later to address more complex classification tasks. This is the key component of deep learning algorithms’ current implementation.

Prof. Ido Kanter, Bar-Ilan University, discusses the difference between deep machine learning and shallow brain learning that consists of a few layers with a wide width (right).

The focus of a new paper published today (April 20) in the journal Scientific Reports is whether or not efficient non-trivial classification can be achieved using brain-inspired shallow feedforward networks, while potentially requiring less computational complexity. “A positive answer puts the need for deep learning architectures at the forefront,” said Prof. Ido Kanter of Bar-Ilan’s Department of Physics and the Gonda (Goldschmied) Multidisciplinary Brain Research Center.

Yarden Tzach, a PhD student and contributor to this research, believes that efficient learning on an artificial shallow architecture can achieve the same classification success rates as deep learning architectures that were previously constructed using many layers and filters with less computational complexity. “However, the efficient realization of shallow architectures requires a shift in advanced GPU technology and future dedicated hardware developments,” he concluded.

Prof. Kanter’s previous experimental work on sub-dendritic adaptation using neuronal cultures combined with other anisotropic properties of neurons, like different spike waveforms, refractory periods, and maximal transmission rates, have aided in efficient brain-inspired shallow architecture learning.

Brain dynamics and machine learning were initially studied separately, but recently brain dynamics was identified as a potential source for new forms of efficient artificial intelligence.