Addressing Sparsity in Deep Neural Networks

doi:10.1109/TCAD.2018.2864289

CORC > 计算技术研究所 > 中国科学院计算技术研究所 > 中国科学院计算技术研究所期刊论文 > 英文

	Addressing Sparsity in Deep Neural Networks
	Zhou, Xuda 1,6,7; Du, Zidong 6,7; Zhang, Shijin 6,7; Zhang, Lei 4,5,6,7; Lan, Huiying 4,5,6,7; Liu, Shaoli 6,7; Li, Ling 3; Guo, Qi 6,7; Chen, Tianshi 6,7; Chen, Yunji 2,7
刊名	IEEE TRANSACTIONS ON COMPUTER-AIDED DESIGN OF INTEGRATED CIRCUITS AND SYSTEMS
	2019-10-01
卷号	38 期号:10 页码:1858-1871
关键词	Accelerator architecture deep neural networks (DNNs) sparsity
ISSN号	0278-0070
DOI	10.1109/TCAD.2018.2864289
英文摘要	Neural networks (NNs) have been demonstrated to be useful in a broad range of applications, such as image recognition, automatic translation, and advertisement recommendation. State-of-the-art NNs are known to be both computationally and memory intensive, due to the ever-increasing deep structure, i.e., multiple layers with massive neurons and connections (i.e., synapses). Sparse NNs have emerged as an effective solution to reduce the amount of computation and memory required. Though existing NN accelerators are able to efficiently process dense and regular networks, they cannot benefit from the reduction of synaptic weights. In this paper, we propose a novel accelerator, Cambricon-X, to exploit the sparsity and irregularity of NN models for increased efficiency. The proposed accelerator features a processing element (PE)-based architecture consisting of multiple PEs. An indexing module efficiently selects and transfers needed neurons to connected PEs with reduced bandwidth requirement, while each PE stores irregular and compressed synapses for local computation in an asynchronous fashion. With 16 PEs, our accelerator is able to achieve at most 544 GOP/s in a small form factor (6.38 mm(2) and 954 mW at 65 nm). Experimental results over a number of representative sparse networks show that our accelerator achieves, on average, 7.23x speedup and 6.43x energy saving against the state-of-the-art NN accelerator. We further investigate possibilities of leveraging activation sparsity and multi-issue controller, which improve the efficiency of Cambricon-X. To ease the burden of programmers, we also propose a high efficient library-based programming environment for our accelerator.
资助项目	National Key Research and Development Program of China[2017YFA0700902] ; National Key Research and Development Program of China[2017YFA0700900] ; National Key Research and Development Program of China[2017YFB1003101] ; NSF of China[61472396] ; NSF of China[61432016] ; NSF of China[61473275] ; NSF of China[61522211] ; NSF of China[61532016] ; NSF of China[61521092] ; NSF of China[61502446] ; NSF of China[61672491] ; NSF of China[61602441] ; NSF of China[61602446] ; NSF of China[61732002] ; NSF of China[61702478] ; 973 Program of China[2015CB358800] ; National Science and Technology Major Project[2018ZX01031102] ; Strategic Priority Research Program of Chinese Academy of Sciences[XDBS01050200]
WOS研究方向	Computer Science ; Engineering
语种	英语
出版者	IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
WOS记录号	WOS:000487193400007
内容类型	期刊论文
源URL	[http://119.78.100.204/handle/2XEOYT63/4675]
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Chen, Yunji
作者单位	1.Univ Sci & Technol China, Sch Comp Sci & Technol, Hefei 230000, Anhui, Peoples R China 2.Chinese Acad Sci, CAS Ctr Excellence Brain Sci, Beijing 100190, Peoples R China 3.Chinese Acad Sci, Inst Software, Beijing 100190, Peoples R China 4.Univ Chinese Acad Sci, Intelligence Processor Res Ctr, Beijing 100190, Peoples R China 5.Univ Chinese Acad Sci, State Key Lab Comp Architecture, Beijing 100190, Peoples R China 6.Cambricon Tech Ltd, Beijing 100191, Peoples R China 7.Chinese Acad Sci, Intelligence Processor Res Ctr, Inst Comp Technol, Beijing 100190, Peoples R China
推荐引用方式 GB/T 7714	Zhou, Xuda,Du, Zidong,Zhang, Shijin,et al. Addressing Sparsity in Deep Neural Networks[J]. IEEE TRANSACTIONS ON COMPUTER-AIDED DESIGN OF INTEGRATED CIRCUITS AND SYSTEMS,2019,38(10):1858-1871.
APA	Zhou, Xuda.,Du, Zidong.,Zhang, Shijin.,Zhang, Lei.,Lan, Huiying.,...&Chen, Yunji.(2019).Addressing Sparsity in Deep Neural Networks.IEEE TRANSACTIONS ON COMPUTER-AIDED DESIGN OF INTEGRATED CIRCUITS AND SYSTEMS,38(10),1858-1871.
MLA	Zhou, Xuda,et al."Addressing Sparsity in Deep Neural Networks".IEEE TRANSACTIONS ON COMPUTER-AIDED DESIGN OF INTEGRATED CIRCUITS AND SYSTEMS 38.10(2019):1858-1871.