Output Tracking Control Based on Adaptive Dynamic Programming With Multistep Policy Evaluation

doi:10.1109/TSMC.2017.2771516

	Output Tracking Control Based on Adaptive Dynamic Programming With Multistep Policy Evaluation
	Luo, Biao1,2 ; Liu, Derong 3; Huang, Tingwen 4; Liu, Jiangjiang 5
刊名	IEEE TRANSACTIONS ON SYSTEMS MAN CYBERNETICS-SYSTEMS
	2019-10-01
卷号	49 期号:10 页码:2155-2165
关键词	Adaptive dynamic programming (ADP) Bellman equation heuristic dynamic programming neural networks (NNs) output tracking control
ISSN号	2168-2216
DOI	10.1109/TSMC.2017.2771516
通讯作者	Huang, Tingwen(tingwen.huang@qatar.tamu.edu)
英文摘要	In this paper, the optimal output tracking control problem of discrete-time nonlinear systems is considered. First, the augmented system is derived and the tracking control problem is converted to the regulation problem with a discounted performance index, which relies on the solution of the Bellman equation. It is known that policy iteration and value iteration are two classical algorithms for solving the Bellman equation. Through analysis of the two algorithms, it is found that policy iteration converges fast while requires an initial admissible control policy, and value iteration avoids the requirement of an initial admissible control policy but converges slowly. To achieve the tradeoff between policy iteration and value iteration, the multistep heuristic dynamic programming (MsHDP) is proposed by using multistep policy evaluation scheme. The convergence of MsHDP algorithm is proved by demonstrating that it converges to the solution of the Bellman equation. Subsequently, neural network-based actor-critic structure is developed to implement the MsHDP algorithm. The effectiveness and advantages of the developed MsHDP method are validated through comparative simulation studies.
资助项目	National Natural Science Foundation of China[61503377] ; National Natural Science Foundation of China[61533017] ; National Natural Science Foundation of China[U1501251] ; Qatar National Research Fund under National Priority Research Project[NPRP 9 166-1-031]
WOS关键词	CONTINUOUS-TIME SYSTEMS ; H-INFINITY CONTROL ; LINEAR-SYSTEMS ; NONLINEAR-SYSTEMS ; UNKNOWN DYNAMICS ; ITERATION ; ALGORITHM
WOS研究方向	Automation & Control Systems ; Computer Science
语种	英语
出版者	IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
WOS记录号	WOS:000487059800021
资助机构	National Natural Science Foundation of China ; Qatar National Research Fund under National Priority Research Project
内容类型	期刊论文
源URL	[http://ir.ia.ac.cn/handle/173211/26992]
专题	中国科学院自动化研究所
通讯作者	Huang, Tingwen
作者单位	1.Chinese Acad Sci, State Key Lab Management & Control Complex Syst, Inst Automat, Beijing 100190, Peoples R China 2.Cent S Univ, Sch Informat Sci & Engn, Changsha 410083, Hunan, Peoples R China 3.Guangdong Univ Technol, Sch Automat, Guangzhou 510006, Guangdong, Peoples R China 4.Texas A&M Univ Qatar, Dept Sci, Doha, Qatar 5.Univ Sci & Technol Beijing, Sch Automat & Elect Engn, Beijing 100083, Peoples R China
推荐引用方式 GB/T 7714	Luo, Biao,Liu, Derong,Huang, Tingwen,et al. Output Tracking Control Based on Adaptive Dynamic Programming With Multistep Policy Evaluation[J]. IEEE TRANSACTIONS ON SYSTEMS MAN CYBERNETICS-SYSTEMS,2019,49(10):2155-2165.
APA	Luo, Biao,Liu, Derong,Huang, Tingwen,&Liu, Jiangjiang.(2019).Output Tracking Control Based on Adaptive Dynamic Programming With Multistep Policy Evaluation.IEEE TRANSACTIONS ON SYSTEMS MAN CYBERNETICS-SYSTEMS,49(10),2155-2165.
MLA	Luo, Biao,et al."Output Tracking Control Based on Adaptive Dynamic Programming With Multistep Policy Evaluation".IEEE TRANSACTIONS ON SYSTEMS MAN CYBERNETICS-SYSTEMS 49.10(2019):2155-2165.