Stein variational gradient descent with local approximations

doi:10.1016/j.cma.2021.114087

	Stein variational gradient descent with local approximations
	Yan, Liang 1; Zhou, Tao 2
刊名	COMPUTER METHODS IN APPLIED MECHANICS AND ENGINEERING
	2021-12-01
卷号	386 页码:20
关键词	Stein variational gradient decent Bayesian inference Deep learning Local approximation
ISSN号	0045-7825
DOI	10.1016/j.cma.2021.114087
英文摘要	Bayesian computation plays an important role in modern machine learning and statistics to reason about uncertainty. A key computational challenge in Bayesian inference is to develop efficient techniques to approximate, or draw samples from posterior distributions. Stein variational gradient decent (SVGD) has been shown to be a powerful approximate inference algorithm for this issue. However, the vanilla SVGD requires calculating the gradient of the target density and cannot be applied when the gradient is unavailable or too expensive to evaluate. In this paper we explore one way to address this challenge by the construction of a local surrogate for the target distribution in which the gradient can be obtained in a much more computationally feasible manner. More specifically, we approximate the forward model using a deep neural network (DNN) which is trained on a carefully chosen training set, which also determines the quality of the surrogate. To this end, we propose a general adaptation procedure to refine the local approximation online without destroying the convergence of the resulting SVGD. This significantly reduces the computational cost of SVGD and leads to a suite of algorithms that are straightforward to implement. The new algorithm is illustrated on a set of challenging Bayesian inverse problems, and numerical experiments demonstrate a clear improvement in performance and applicability of standard SVGD. (C) 2021 Elsevier B.V. All rights reserved.
资助项目	NSF of China[11822111] ; NSF of China[11688101] ; NSF of China[11731006] ; NSF of China[11771081] ; science challenge project, China[TZ2018001] ; Zhishan Young Scholar Program of SEU, China ; National Key R&D Program of China[2020YFA0712000] ; science challenge project[TZ2018001] ; Strategic Priority Research Program of Chinese Academy of Sciences[XDA25000404] ; youth innovation promotion association (CAS), China
WOS研究方向	Engineering ; Mathematics ; Mechanics
语种	英语
出版者	ELSEVIER SCIENCE SA
WOS记录号	WOS:000703985200009
内容类型	期刊论文
源URL	[http://ir.amss.ac.cn/handle/2S8OKBNM/59338]
专题	中国科学院数学与系统科学研究院
通讯作者	Yan, Liang
作者单位	1.Southeast Univ, Sch Math, Nanjing 210096, Peoples R China 2.Chinese Acad Sci, LSEC, Inst Computat Math, Acad Math & Syst Sci, Beijing 100190, Peoples R China
推荐引用方式 GB/T 7714	Yan, Liang,Zhou, Tao. Stein variational gradient descent with local approximations[J]. COMPUTER METHODS IN APPLIED MECHANICS AND ENGINEERING,2021,386:20.
APA	Yan, Liang,&Zhou, Tao.(2021).Stein variational gradient descent with local approximations.COMPUTER METHODS IN APPLIED MECHANICS AND ENGINEERING,386,20.
MLA	Yan, Liang,et al."Stein variational gradient descent with local approximations".COMPUTER METHODS IN APPLIED MECHANICS AND ENGINEERING 386(2021):20.