Temporal-adaptive sparse feature aggregation for video object detection

doi:10.1016/j.patcog.2022.108587

	Temporal-adaptive sparse feature aggregation for video object detection
	He, Fei 2,3; Li, Qiaozhe 2,3; Zhao, Xin 2,3; Huang, Kaiqi 1,2,3
刊名	PATTERN RECOGNITION
	2022-07-01
卷号	127 页码:10
关键词	Video object detection Temporal-adaptive sparse sampling Pixel-adaptive aggregation Object-relational aggregation
ISSN号	0031-3203
DOI	10.1016/j.patcog.2022.108587
通讯作者	Zhao, Xin(xzhao@nlpr.ia.ac.cn)
英文摘要	Video object detection is a challenging task due to the appearance deterioration in video frames. To enhance feature representation of the deteriorated frames, previous methods usually aggregate features from fixed-density and fixed-length adjacent frames. However, due to the redundancy of videos and irregular object movements over time, temporal information may not be efficiently exploited using the traditional inflexible strategy. Alternatively, we present a temporal-adaptive sparse feature aggregation framework, an accurate and efficient method for video object detection. Instead of adopting a fixed-density and fixed-length window fusion strategy, a temporal-adaptive sparse sampling strategy is proposed using a stride predictor to encode informative frames more efficiently. A collaborative feature aggregation framework, which consists of a pixel-adaptive aggregation module and an object-relational aggregation module, is proposed for feature enhancement. The pixel-adaptive aggregation module enhances pixel level features on the current frame using corresponding pixel-level features from other frames. Similarly, the object-relational aggregation module further enhances feature representation at proposal level. A graph is constructed to model the relations between different proposals so that the relation features and proposal features are adaptively fused for feature enhancement. Experiments demonstrate that our proposed framework significantly surpasses traditional dense aggregation methods, and comprehensive ablation studies verify the effectiveness of each proposed module in our framework.
资助项目	National Natural Science Foundation of China[61721004] ; National Natural Science Foundation of China[61876181] ; Projects of Chinese Academy of Science[QYZDB-SSW-JSC006] ; Strategic Priority Research Program of Chinese Academy of Sciences[XDA27000000] ; Youth Innovation Promotion Association CAS
WOS研究方向	Computer Science ; Engineering
语种	英语
出版者	ELSEVIER SCI LTD
WOS记录号	WOS:000776971700003
资助机构	National Natural Science Foundation of China ; Projects of Chinese Academy of Science ; Strategic Priority Research Program of Chinese Academy of Sciences ; Youth Innovation Promotion Association CAS
内容类型	期刊论文
源URL	[http://ir.ia.ac.cn/handle/173211/48301]
专题	智能系统与工程
通讯作者	Zhao, Xin
作者单位	1.CAS Ctr Excellence Brain Sci & Intelligence Techn, Shanghai, Peoples R China 2.Chinese Acad Sci, Inst Automat, CRISE, Beijing, Peoples R China 3.Univ Chinese Acad Sci, Sch Artificial Intelligence, Beijing, Peoples R China
推荐引用方式 GB/T 7714	He, Fei,Li, Qiaozhe,Zhao, Xin,et al. Temporal-adaptive sparse feature aggregation for video object detection[J]. PATTERN RECOGNITION,2022,127:10.
APA	He, Fei,Li, Qiaozhe,Zhao, Xin,&Huang, Kaiqi.(2022).Temporal-adaptive sparse feature aggregation for video object detection.PATTERN RECOGNITION,127,10.
MLA	He, Fei,et al."Temporal-adaptive sparse feature aggregation for video object detection".PATTERN RECOGNITION 127(2022):10.