Integrating supervised subspace criteria with restricted Boltzmann Machine for feature extraction

CORC > 自动化研究所 > 中国科学院自动化研究所 > 模式识别国家重点实验室 > 模式分析与学习团队

	Integrating supervised subspace criteria with restricted Boltzmann Machine for feature extraction
	Xie, Guo-Sen; Zhang, Xu-Yao; Zhang, Yan-Ming; Liu, Cheng-Lin
	2014
会议日期	2014
会议地点	北京
关键词	Rbm
英文摘要	Restricted Boltzmann Machine (RBM) is a widely used building-block in deep neural networks. However, RBM is an unsupervised model which can not exploit the rich supervised information of data. Therefore, we consider combining the descriptive (generative) ability of RBM with the discriminative ability of supervised subspace models, i.e., Fisher linear discriminant analysis (FDA), marginal Fisher analysis (MFA), and heat kernel MFA (hkMFA). Specifically, the hidden layer of RBM is regularized by the supervised subspace criteria, and the joint learning model can then be efficiently optimized by gradient descent and graph construction (used to define the scatter matrix in the subspace models) on mini-batch data. Compared with the traditional subspace models (FDA, MFA, hkMFA), the proposed hybrid models are essentially nonlinear and can be optimized by gradient descent instead of eigenvalue decomposition. More importantly, traditional subspace models can only reduce the dimensionality (because of linear transformation), while the proposed models can also increase the dimensionality for better class discrimination. Experiments on three databases demonstrate that the proposed hybrid models outperform both RBM and their counterpart subspace models (FDA, MFA, hkMFA) consistently.
会议录	Procceding of IJCNN 2014
内容类型	会议论文
源URL	[http://ir.ia.ac.cn/handle/173211/11958]
专题	自动化研究所_模式识别国家重点实验室_模式分析与学习团队
通讯作者	Xie, Guo-Sen
推荐引用方式 GB/T 7714	Xie, Guo-Sen,Zhang, Xu-Yao,Zhang, Yan-Ming,et al. Integrating supervised subspace criteria with restricted Boltzmann Machine for feature extraction[C]. 见:. 北京. 2014.