An Improved Reinforcement Learning Algorithm for Cooperative Behaviors of Mobile Robots

Reinforcement learning algorithm for multirobot will become very slow when the number of robots is increasing resulting in an exponential increase of state space. A sequential Q-learning based on knowledge sharing is presented. The rule repository of robots behaviors is firstly initialized in the process of reinforcement learning. Mobile robots obtain present environmental state by sensors. Then the state will be matched to determine if the relevant behavior rule has been stored in the database. If the rule is present, an action will be chosen in accordance with the knowledge and the rules, and the matching weight will be refined. Otherwise the new rule will be appended to the database. The robots learn according to a given sequence and share the behavior database. We examine the algorithm by multirobot following-surrounding behavior, and find that the improved algorithm can effectively accelerate the convergence speed. 1. Introduction In recent years, multirobot systems (MRSs) have received considerable attention because such systems possess some special capabilities such as more flexibility, adaptability, and efficiency in dealing with a complex task [1]. Multirobot learning is the process of acquiring new cooperative behaviors for a particular task by trial and error in the presence of other robots. The desired cooperative behaviors may emerge by local interactions among the robots which are with limited sensing capabilities. Multirobot system can perform more complex tasks via cooperation and coordination [2, 3]. Normally, multirobot learning method can be classified as collective swarm learning and intentionally cooperative learning based on the various levels of explicit communication. The collective swarm systems allow participating robots to learn swarm behaviors with only minimal explicit communication among robots [4, 5]. In these systems a large number of homogeneous mobile robots interact implicitly with each other based on the sharing environment. The robots are organized on the basis of local control laws, such as the stigmergy introduced by Garnier et al. [6]. Stigmergy is a mechanism of indirect interaction mediated by modifications of the sharing environment of agents [7]. The information coming from the local environment can guide the participating individual activity. The complex intelligent behavior emerges at the colony level from the local interactions that take place among individuals exhibiting simple behaviors. At present, the swarm behaviors are often modeled using methods inspired by biology. Along with the advent of