Learning Algorithms for Cooperative Multi-Agent Systems

Decision making in distributed systems requires coordination among a number of individually acting agents, in order to achieve a common goal. Therefore, the agents' behavior policies must be harmonized perfectly. The notion Multi Agent Reinforcement Learning subsumes learning techniques, which enable agents to acquire cooperative decision policies automatically - based on a specification of the system's desired overall behavior. Application fields for distributed agents include resource management, scheduling, network routing, or robotics.

The capabilities of learning systems - as compared to non-adaptive solutions - for real-world tasks have been exemplified by numerous researchers. Most of the learning approaches published are of empirical nature and, in many cases, originate from an adaptation of learning algorithms that were designed for the single-agent case originally. Although guarantees for the validity and optimality of the behavior strategies found with those approaches cannot be given in general, the solutions found empirically are often better than existing standard solutions. As a result of lacking theretical grounding, however, many of the concepts applied cannot be generalized.

In this DFG-funded project (2005-09) the focus has been on investigating the learning in distributed systems both empirically and theoretically. Main goals included the development of distributed learning algorithms with verifiable convergence properties for stochastic environments, the analysis of modelling variants as well as their behavior regarding convergence, robustness and generalization of the solutions found.

The following list of selection publications provides further details on the results of the project.