SCR-Graph: Spatial-Causal Relationships Based Graph Reasoning Network for Human Action Prediction

CORC > 沈阳自动化研究所 > 中国科学院沈阳自动化研究所 > 机器人学研究室

	SCR-Graph: Spatial-Causal Relationships Based Graph Reasoning Network for Human Action Prediction
	Chen B(陈博)2,3,4; Sun XS(孙晓舒)2,3; Li DC(李德才)2,3; He YQ(何玉庆)2,3; Hua CS(华春生)1
	2021
会议日期	January 28-30, 2021
会议地点	Stanford, CA, United states
关键词	Graph neural network knowledge graph action prediction
页码	1-9
英文摘要	Technologies to predict human actions are extremely important for applications such as human robot cooperation and autonomous driving. However, a majority of the existing algorithms focus on exploiting visual features of the videos and do not consider the mining of relationships, which include spatial relationships between human and scene elements as well as causal relationships in temporal action sequences. In fact, human beings are good at using spatial and causal relational reasoning mechanism to predict the actions of others. Inspired by this idea, we proposed a Spatial and Causal Relationship based Graph Reasoning Network (SCR-Graph), which can be used to predict human actions by modeling the action-scene relationship, and causal relationship between actions, in spatial and temporal dimensions respectively. Here, in spatial dimension, a hierarchical graph attention module is designed by iteratively aggregating the features of different kinds of scene elements in different level. In temporal dimension, we designed a knowledge graph based causal reasoning module and map the past actions to temporal causal features through Diffusion RNN. Finally, we integrated the causality features into the heterogeneous graph in the form of shadow node, and introduced a self-attention module to determine the time when the knowledge graph information should be activated. Extensive experimental results on the VIRAT datasets demonstrate the favorable performance of the proposed framework.
产权排序	1
会议录	Proceedings of the 2nd International Conference on Computing and Data Science, CONF-CDS 2021
会议录出版者	ACM
会议录出版地	New York
语种	英语
ISBN号	978-1-4503-8957-0
内容类型	会议论文
源URL	[http://ir.sia.cn/handle/173321/28937]
专题	沈阳自动化研究所_机器人学研究室
通讯作者	He YQ(何玉庆)
作者单位	1.Liaoning University, Shenyang 110016, China 2.Institutes for Robotics and Intelligent Manufacturing, Chinese Academy of Sciences, Shenyang 110016, China 3.State Key Laboratory of Robotics, Shenyang Institute of Automation, Chinese Academy of Sciences, Shenyang 110016, China 4.University of Chinese Academy of Sciences, Beijing 100049, China
推荐引用方式 GB/T 7714	Chen B,Sun XS,Li DC,et al. SCR-Graph: Spatial-Causal Relationships Based Graph Reasoning Network for Human Action Prediction[C]. 见:. Stanford, CA, United states. January 28-30, 2021.