Fuzzy cerebellar model arithmetic controller with automatic state partition for value function approximation
一种状态自动划分的模糊小脑模型关节控制器值函数拟合方法

In continuous-state space or large discrete-state space, the reinforcement learning (RL) uses function approximation approaches to represent the value function in seeking the optimal policy. However the structures of function approximators which will greatly influence the learning performance are often designed in advance. To generate the structure of function approximator automatically, a novel function approximator called the automatic state-partition-based fuzzy cerebellar model arithmetic controller (ASP-FCMAC) is proposed. In ASP-FCMAC, the variation tendency of Bellman error is used to determine the best time to perform state partition and two mechanisms are also discussed for determining which state should be partitioned at each time step. Experimental results in solving mountain car problem and RoboCup Keepaway problem demonstrate that ASP-FCMAC can automatically generate the structure of FCMAC for effective reinforcement learning.