Body Joint Guided 3-D Deep Convolutional Descriptors for Action Recognition

doi:10.1109/TCYB.2017.2756840

CORC > 自动化研究所 > 中国科学院自动化研究所 > 模式识别国家重点实验室 > 图像与视频分析团队

	Body Joint Guided 3-D Deep Convolutional Descriptors for Action Recognition
	Cao, Congqi1,2 ; Zhang, Yifan1,2 ; Zhang, Chunjie2,3 ; Lu, Hanqing1,2
刊名	IEEE TRANSACTIONS ON CYBERNETICS
	2018-03-01
卷号	48 期号:3 页码:1095-1108
关键词	Action Recognition Body Joints Convolutional Networks Feature Pooling Two-stream Bilinear Model
DOI	10.1109/TCYB.2017.2756840
文献子类	Article
英文摘要	3-D convolutional neural networks (3-D CNNs) have been established as a powerful tool to simultaneously learn features from both spatial and temporal dimensions, which is suitable to be applied to video-based action recognition. In this paper, we propose not to directly use the activations of fully connected layers of a 3-D CNN as the video feature, but to use selective convolutional layer activations to form a discriminative descriptor for video. It pools the feature on the convolutional layers under the guidance of body joint positions. Two schemes of mapping body joints into convolutional feature maps for pooling are discussed. The body joint positions can be obtained from any off-the-shelf skeleton estimation algorithm. The helpfulness of the body joint guided feature pooling with inaccurate skeleton estimation is systematically evaluated. To make it end-to-end and do not rely on any sophisticated body joint detection algorithm, we further propose a two-stream bilinear model which can learn the guidance from the body joints and capture the spatio-temporal features simultaneously. In this model, the body joint guided feature pooling is conveniently formulated as a bilinear product operation. Experimental results on three real-world datasets demonstrate the effectiveness of body joint guided pooling which achieves promising performance.
WOS关键词	CLASSIFICATION ; TRAJECTORIES ; HISTOGRAMS ; PARTS
WOS研究方向	Automation & Control Systems ; Computer Science
语种	英语
WOS记录号	WOS:000424826800022
资助机构	National Natural Science Foundation of China(61332016 ; Youth Innovation Promotion Association CAS ; 61572500)
内容类型	期刊论文
源URL	[http://ir.ia.ac.cn/handle/173211/20410]
专题	自动化研究所_模式识别国家重点实验室_图像与视频分析团队
作者单位	1.Chinese Acad Sci, Inst Automat, Natl Lab Pattern Recognit, Beijing 100190, Peoples R China 2.Univ Chinese Acad Sci, Beijing 100190, Peoples R China 3.Chinese Acad Sci, Inst Automat, Res Ctr Brain Inspired Intelligence, Beijing 100190, Peoples R China
推荐引用方式 GB/T 7714	Cao, Congqi,Zhang, Yifan,Zhang, Chunjie,et al. Body Joint Guided 3-D Deep Convolutional Descriptors for Action Recognition[J]. IEEE TRANSACTIONS ON CYBERNETICS,2018,48(3):1095-1108.
APA	Cao, Congqi,Zhang, Yifan,Zhang, Chunjie,&Lu, Hanqing.(2018).Body Joint Guided 3-D Deep Convolutional Descriptors for Action Recognition.IEEE TRANSACTIONS ON CYBERNETICS,48(3),1095-1108.
MLA	Cao, Congqi,et al."Body Joint Guided 3-D Deep Convolutional Descriptors for Action Recognition".IEEE TRANSACTIONS ON CYBERNETICS 48.3(2018):1095-1108.