NFLB dropout: Improve generalization ability by dropping out the best -A biologically inspired adaptive dropout method for unsupervised learning

CORC > 自动化研究所 > 中国科学院自动化研究所 > 复杂系统管理与控制国家重点实验室 > 机器人应用与理论组

	NFLB dropout: Improve generalization ability by dropping out the best -A biologically inspired adaptive dropout method for unsupervised learning
	Peijie Yin; Lu Qi; Xuanyang Xi; Bo Zhang; Hong Qiao
	2016
会议名称	2016 International Joint Conference on Neural Networks (IJCNN)
会议日期	24-29 July 2016
会议地点	Vancouver, BC, Canada
关键词	none
通讯作者	Peijie Yin
英文摘要	Generalization ability is widely acknowledged as one of the most important criteria to evaluate thequality of unsupervised models. The objective of our research is to find a better dropout method toimprove the generalization ability of convolutional deep belief network (CDBN), an unsupervised learningmodel for vision tasks. In this paper, the phenomenon of low feature diversity during the training process is investigated. The attention mechanism of human visual system is more focused on rare events and depresses well-known facts. Inspired by this mechanism, No Feature Left Behind Dropout (NFLBDropout), an adaptive dropout method is firstly proposed to automatically adjust the dropout rate feature-wisely. In the proposed method, the algorithm drops well-trained features and keeps poorly-trained ones with a high probability during training iterations. In addition, we apply two approximations of the quality of features, which are inspired by theory of saliency and optimization. Compared with themodel trained by standard dropout, experiment results show that our NFLB Dropout method improves not only the accuracy but the convergence speed as well.
会议录	2016 International Joint Conference on Neural Networks (IJCNN)
内容类型	会议论文
源URL	[http://ir.ia.ac.cn/handle/173211/12829]
专题	自动化研究所_复杂系统管理与控制国家重点实验室_机器人应用与理论组
推荐引用方式 GB/T 7714	Peijie Yin,Lu Qi,Xuanyang Xi,et al. NFLB dropout: Improve generalization ability by dropping out the best -A biologically inspired adaptive dropout method for unsupervised learning[C]. 见:2016 International Joint Conference on Neural Networks (IJCNN). Vancouver, BC, Canada. 24-29 July 2016.