Exploring the Brain-like Properties of Deep Neural Networks: A Neural Encoding Perspective

doi:10.1007/s11633-022-1348-x

CORC > 自动化研究所 > 中国科学院自动化研究所 > 学术期刊 > International Journal of Automation and Computing

	Exploring the Brain-like Properties of Deep Neural Networks: A Neural Encoding Perspective
	Qiongyi Zhou 2,3
刊名	Machine Intelligence Research
	2022
卷号	19 期号:5 页码:439-455
关键词	Convolutional neural network (CNN) vision transformer (ViT) multi-modal networks spatial-temporal networks visual neural encoding brain-like neural networks
ISSN号	2731-538X
DOI	10.1007/s11633-022-1348-x
英文摘要	Nowadays, deep neural networks (DNNs) have been equipped with powerful representation capabilities. The deep convolutional neural networks (CNNs) that draw inspiration from the visual processing mechanism of the primate early visual cortex have outperformed humans on object categorization and have been found to possess many brain-like properties. Recently, vision transformers (ViTs) have been striking paradigms of DNNs and have achieved remarkable improvements on many vision tasks compared to CNNs. It is natural to ask how the brain-like properties of ViTs are. Beyond the model paradigm, we are also interested in the effects of factors, such as model size, multimodality, and temporality, on the ability of networks to model the human visual pathway, especially when considering that existing research has been limited to CNNs. In this paper, we systematically evaluate the brain-like properties of 30 kinds of computer vision models varying from CNNs and ViTs to their hybrids from the perspective of explaining brain activities of the human visual cortex triggered by dynamic stimuli. Experiments on two neural datasets demonstrate that neither CNN nor transformer is the optimal model paradigm for modelling the human visual pathway. ViTs reveal hierarchical correspondences to the visual pathway as CNNs do. Moreover, we find that multi-modal and temporal networks can better explain the neural activities of large parts of the visual cortex, whereas a larger model size is not a sufficient condition for bridging the gap between human vision and artificial networks. Our study sheds light on the design principles for more brain-like networks. The code is available at https://github.com/QYiZhou/LWNeuralEncoding.
内容类型	期刊论文
源URL	[http://ir.ia.ac.cn/handle/173211/49918]
专题	自动化研究所_学术期刊_International Journal of Automation and Computing
作者单位	1.Center for Excellence in Brain Science and Intelligence Technology, Chinese Academy of Sciences, Beijing 100190, China 2.School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing 100190, China 3.Research Center for Brain-inspired Intelligence and National Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China
推荐引用方式 GB/T 7714	Qiongyi Zhou. Exploring the Brain-like Properties of Deep Neural Networks: A Neural Encoding Perspective[J]. Machine Intelligence Research,2022,19(5):439-455.
APA	Qiongyi Zhou.(2022).Exploring the Brain-like Properties of Deep Neural Networks: A Neural Encoding Perspective.Machine Intelligence Research,19(5),439-455.
MLA	Qiongyi Zhou."Exploring the Brain-like Properties of Deep Neural Networks: A Neural Encoding Perspective".Machine Intelligence Research 19.5(2022):439-455.