Discrete soft actor-critic with auto-encoder on vascular robotic system

doi:10.1017/S0263574722001527

CORC > 自动化研究所 > 中国科学院自动化研究所 > 复杂系统管理与控制国家重点实验室 > 先进机器人控制团队

	Discrete soft actor-critic with auto-encoder on vascular robotic system
	Li, Hao 3,4; Zhou, Xiao-Hu 3,4; Xie, Xiao-Liang 3,4; Liu, Shi-Qi 3,4; Gui, Mei-Jiang 3,4; Xiang, Tian-Yu 3,4; Wang, Jin-Li 2; Hou, Zeng-Guang 1,3,4
刊名	ROBOTICA
	2022-11-17
页码	12
关键词	surgical robots vascular robotic system automation reinforcement learning deep neural network
ISSN号	0263-5747
DOI	10.1017/S0263574722001527
通讯作者	Zhou, Xiao-Hu(xiaohu.zhou@ia.ac.cn) ; Xie, Xiao-Liang(xiaoliang.xie@ia.ac.cn)
英文摘要	Instrument delivery is critical part in vascular intervention surgery. Due to the soft-body structure of instruments, the relationship between manipulation commands and instrument motion is non-linear, making instrument delivery challenging and time-consuming. Reinforcement learning has the potential to learn manipulation skills and automate instrument delivery with enhanced success rates and reduced workload of physicians. However, due to the sample inefficiency when using high-dimensional images, existing reinforcement learning algorithms are limited on realistic vascular robotic systems. To alleviate this problem, this paper proposes discrete soft actor-critic with auto-encoder (DSAC-AE) that augments SAC-discrete with an auxiliary reconstruction task. The algorithm is applied with distributed sample collection and parameter update in a robot-assisted preclinical environment. Experimental results indicate that guidewire delivery can be automatically implemented after 50k sampling steps in less than 15 h, demonstrating the proposed algorithm has the great potential to learn manipulation skill for vascular robotic systems.
资助项目	National Natural Science Foundation of China[62003343] ; National Natural Science Foundation of China[62222316] ; National Natural Science Foundation of China[U1913601] ; National Natural Science Foundation of China[62073325] ; National Natural Science Foundation of China[61720106012] ; National Natural Science Foundation of China[U20A20224] ; National Natural Science Foundation of China[U1913210] ; Beijing Natural Science Foundation[M22008] ; Youth Innovation Promotion Association of ChineseAcademy of Sciences (CAS)[2020140] ; Strategic Priority Research Program of CAS[XDB32040000]
WOS关键词	INTERVENTIONAL CARDIOLOGISTS ; STAFF ; RISK
WOS研究方向	Robotics
语种	英语
出版者	CAMBRIDGE UNIV PRESS
WOS记录号	WOS:000889895900001
资助机构	National Natural Science Foundation of China ; Beijing Natural Science Foundation ; Youth Innovation Promotion Association of ChineseAcademy of Sciences (CAS) ; Strategic Priority Research Program of CAS
内容类型	期刊论文
源URL	[http://ir.ia.ac.cn/handle/173211/50797]
专题	自动化研究所_复杂系统管理与控制国家重点实验室_先进机器人控制团队
通讯作者	Zhou, Xiao-Hu; Xie, Xiao-Liang
作者单位	1.Chinese Acad Sci, Ctr Excellence Brain Sci & Intelligence Technol, Beijing, Peoples R China 2.China Univ Min & Technol, Sch Mech Elect & Informat Engn, Beijing, Peoples R China 3.Univ Chinese Acad Sci, Sch Artificial Intelligence, Beijing, Peoples R China 4.Chinese Acad Sci, Inst Automat, State Key Lab Management & Control Complex Syst, Beijing, Peoples R China
推荐引用方式 GB/T 7714	Li, Hao,Zhou, Xiao-Hu,Xie, Xiao-Liang,et al. Discrete soft actor-critic with auto-encoder on vascular robotic system[J]. ROBOTICA,2022:12.
APA	Li, Hao.,Zhou, Xiao-Hu.,Xie, Xiao-Liang.,Liu, Shi-Qi.,Gui, Mei-Jiang.,...&Hou, Zeng-Guang.(2022).Discrete soft actor-critic with auto-encoder on vascular robotic system.ROBOTICA,12.
MLA	Li, Hao,et al."Discrete soft actor-critic with auto-encoder on vascular robotic system".ROBOTICA (2022):12.