TY - GEN
T1 - A novel generalized value iteration scheme for uncertain continuous-time linear systems
AU - Lee, Jae Young
AU - Park, Jin Bae
AU - Choi, Yoon Ho
PY - 2010
Y1 - 2010
N2 - In this paper, a novel generalized value iteration (VI) technique is presented which is a reinforcement learning (RL) scheme for solving online the continuous-time (CT) discounted linear quadratic regulation (LQR) problems without exactly knowing the system matrix A. In the proposed method, a discounted value function is considered, which is a general setting in RL frameworks, but not fully considered in RL for CT dynamical systems. Moreover, a stepwise-varying learning rate is introduced for the fast and safe convergence. In relation to this learning rate, we also discuss the locations of the poles of the closed-loop system and monotone convergence to the optimal solution. The results from these discussions give the conditions on the stability and monotone convergence of the existing VI methods.
AB - In this paper, a novel generalized value iteration (VI) technique is presented which is a reinforcement learning (RL) scheme for solving online the continuous-time (CT) discounted linear quadratic regulation (LQR) problems without exactly knowing the system matrix A. In the proposed method, a discounted value function is considered, which is a general setting in RL frameworks, but not fully considered in RL for CT dynamical systems. Moreover, a stepwise-varying learning rate is introduced for the fast and safe convergence. In relation to this learning rate, we also discuss the locations of the poles of the closed-loop system and monotone convergence to the optimal solution. The results from these discussions give the conditions on the stability and monotone convergence of the existing VI methods.
UR - http://www.scopus.com/inward/record.url?scp=79953145872&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=79953145872&partnerID=8YFLogxK
U2 - 10.1109/CDC.2010.5718015
DO - 10.1109/CDC.2010.5718015
M3 - Conference contribution
AN - SCOPUS:79953145872
SN - 9781424477456
T3 - Proceedings of the IEEE Conference on Decision and Control
SP - 4637
EP - 4642
BT - 2010 49th IEEE Conference on Decision and Control, CDC 2010
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 49th IEEE Conference on Decision and Control, CDC 2010
Y2 - 15 December 2010 through 17 December 2010
ER -