Forged facial video detection framework based on multi-region temporal relationship feature

doi:10.1063/5.0125032

CORC > 自动化研究所 > 中国科学院自动化研究所 > 舆论大数据科学与技术应用联合实验室

	Forged facial video detection framework based on multi-region temporal relationship feature
	Fang, Xing 1; Hao, YanNi 3; Luo, Yin 2,3; Xu, Nan 2,3; Cao, Jia 2,3
刊名	AIP ADVANCES
	2023-08-01
卷号	13 期号:8 页码:8
DOI	10.1063/5.0125032
通讯作者	Fang, Xing(fangxing@tju.edu.cn)
英文摘要	Face generation and manipulation techniques based on deep learning have enabled the creation of sophisticated forged facial videos, which are indistinguishable by human eyes. However, the illegal use of deep fake technology will have a serious impact on social stability, personal reputation, and even national security. Therefore, the detection technology of fake facial videos is of great significance to protect national security and maintain social order. Although the existing video-based fake face video detection technology has achieved good detection performance on the public fake face video database, there are still the following problems: (1) the existing technology uses a 2D attention mechanism to obtain local region features from face images and lacks a 3D attention mechanism to obtain local area features from face videos; (2) after obtaining local area features, the existing technology is directly used to classify or only model the inter-regional relationship of images without modeling the temporal relationship between regions of the video. This paper proposes a fake facial video detection framework based on multi-region temporal relationship features, including designing a three-dimensional attention mechanism to extract local features of multiple regions of the face from the video. In order to model the time series relationship between different face areas, a time series graph convolution neural network is also introduced to extract the time series relationship features between multiple areas. In order to model the time sequence relationship between different face regions, the convolution neural network of a time sequence diagram is also introduced to extract the characteristics of the time sequence relationship between multiple regions. Through the change characteristics of the time sequence relationship between face regions, the timing inconsistency of the face video is detected so as to determine whether the face has been deeply forged. Through experiments on multiple datasets, the experimental results of the model accuracy test show that the method proposed in the present invention achieves the highest detection accuracy, and the accuracy in the FaceForensics++ (low definition) dataset is 18.19% higher than that of the benchmark method. The experimental results of the generalization ability test show that the method proposed in the present invention achieves the highest generalization performance, and the detection accuracy in the Celeb-DF dataset is 11.92% higher than that of the benchmark method.
资助项目	National Key Research and Development Plan Science and Technology Winter Olympics Key Project: Research and Application Demonstration of the Global Communication Service Platform for the Winter Olympics[2020YFF0305300]
WOS研究方向	Science & Technology - Other Topics ; Materials Science ; Physics
语种	英语
出版者	AIP Publishing
WOS记录号	WOS:001052966400001
资助机构	National Key Research and Development Plan Science and Technology Winter Olympics Key Project: Research and Application Demonstration of the Global Communication Service Platform for the Winter Olympics
内容类型	期刊论文
源URL	[http://ir.ia.ac.cn/handle/173211/54089]
专题	舆论大数据科学与技术应用联合实验室
通讯作者	Fang, Xing
作者单位	1.Tianjin Univ, Sch Management, Tianjin 300072, Peoples R China 2.Chinese Acad Sci, Inst Automat, Beijing 100190, Peoples R China 3.Beijing Wenge Technol Co Ltd, Beijing 100000, Peoples R China
推荐引用方式 GB/T 7714	Fang, Xing,Hao, YanNi,Luo, Yin,et al. Forged facial video detection framework based on multi-region temporal relationship feature[J]. AIP ADVANCES,2023,13(8):8.
APA	Fang, Xing,Hao, YanNi,Luo, Yin,Xu, Nan,&Cao, Jia.(2023).Forged facial video detection framework based on multi-region temporal relationship feature.AIP ADVANCES,13(8),8.
MLA	Fang, Xing,et al."Forged facial video detection framework based on multi-region temporal relationship feature".AIP ADVANCES 13.8(2023):8.