The StarShips Learning Framework (SSLF) is a framework for teaching, learning, and exploring agent-based programming and the use of adaptive or learning approaches to agent programming. The SSLF is open-source software (download) and can be used and distributed freely.
StarShips is an arcade game in which two players take control over two star ships fighting against one another. The game play and story behind the game is leant to Star Trek Enterpise (TNG). Of course, each ship may also be controlled not by a human player, but by an intelligent (software) agent or even a learning agent. To facilitate the latter, the StarShips Learning Framework (SSLF) provides the necessary infrastructure in order to allow an intelligent agent, which is entirely decoupled from the StarShips program, i.e. runs as a stand-alone software, to control one of the ships.
Neural batch reinforcement learning (RL) algorithms have shown to be a powerful tool for model-free reinforcement learning problems. In this case study, the SSLF has been selected a learning benchmark from the realm of computer games and a variant of a neural batch RL algorithm has been applied to it. Specifically, a number of sub-tasks controlling the agent were selected and optimized using reinforcement learning. These optimized sub-behaviors were finally integrated into the overall AI-based agent, showing a substantial improvement in playing performance.
The video below visualizes some key aspects of the learning approach taken, shows a summary of the learning process in time lapse as well as an overview of the results achieved.
- T. Gabel, C. Lutz and M. Riedmiller.
Improved Neural Fitted Q Iteration Applied to a Novel Computer Gaming and Learning Benchmark.
In Proceedings of the IEEE Symposium on Approximate Dynamic Programming and Reinforcement Learning (ADPRL 2011), pages 279-286, Paris, France. IEEE Press, 2011.